IBM Crawler
IBM enterprise crawler indexing content for Watson and cloud services
About this crawler
IBM Crawler is a web crawler identified by the regular-expression pattern IBM Crawler in the User-Agent request header. It is categorised as seo. Use the regex above to detect, log, allow, or block IBM Crawler traffic in your web server, CDN edge rules, or robots.txt.
Block-rate · top 25k sites
0.065%
Technical details
- Name
- IBM Crawler
- Pattern
IBM Crawler- Tags
- seo
- Reference
- https://datadome.co/bots/ibm-crawler/
- Added
- 2026/04/17
- Instances
- 1 known sample(s)
Sample User-Agent strings
IBM Crawler
Block this crawler
robots.txt — disallow IBM Crawler:
User-agent: IBM Crawler
Disallow: /
Apache .htaccess — return 403:
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} IBM Crawler [NC]
RewriteRule .* - [F,L]
Nginx — return 403 inside a server block:
if ($http_user_agent ~* "IBM Crawler") {
return 403;
}
← back to all crawlers