Kyoto-Tohoku-Crawler
Kyoto University web crawler
About this crawler
Kyoto-Tohoku-Crawler is a web crawler identified by the regular-expression pattern Kyoto-(?:Tohoku-)?Crawler\/ in the User-Agent request header. It is categorised as academic. Use the regex above to detect, log, allow, or block Kyoto-Tohoku-Crawler traffic in your web server, CDN edge rules, or robots.txt.
Block-rate · top 25k sites
No block-rate data for this crawler.
Technical details
- Name
- Kyoto-Tohoku-Crawler
- Pattern
Kyoto-(?:Tohoku-)?Crawler\/- Tags
- academic
- Reference
- http://nlp.ist.i.kyoto-u.ac.jp/
- Added
- 2026/05/09
- Instances
- 3 known sample(s)
Sample User-Agent strings
Kyoto-Crawler/2.0 (Mozilla-compatible; kyoto-crawler-contact(at)nlp(dot)kuee(dot)kyoto-u(dot)ac(dot)jp; http://nlp.ist.i.kyoto-u.ac.jp/)
Kyoto-Crawler/n1.0 (Mozilla-compatible; [email protected]; http://nlp.ist.i.kyoto-u.ac.jp/?crawling)
Kyoto-Tohoku-Crawler/v1 (Mozilla-compatible; [email protected]; http://nlp.ist.i.kyoto-u.ac.jp/?crawling-kt)
Block this crawler
robots.txt — disallow Kyoto-Tohoku-Crawler:
User-agent: Kyoto-Tohoku-Crawler
Disallow: /
Apache .htaccess — return 403:
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} Kyoto-(?:Tohoku-)?Crawler\/ [NC]
RewriteRule .* - [F,L]
Nginx — return 403 inside a server block:
if ($http_user_agent ~* "Kyoto-(?:Tohoku-)?Crawler\\/") {
return 403;
}
← back to all crawlers