cb crawl
CompanyBook networking crawler
About this crawler
cb crawl is a web crawler identified by the regular-expression pattern cb crawl in the User-Agent request header. It is categorised as seo. Use the regex above to detect, log, allow, or block cb crawl traffic in your web server, CDN edge rules, or robots.txt.
Block-rate · top 25k sites
No block-rate data for this crawler.
Technical details
- Name
- cb crawl
- Pattern
cb crawl- Tags
- seo
- Reference
- http://www.companybooknetworking.com
- Added
- 2026/05/09
- Instances
- 1 known sample(s)
Sample User-Agent strings
cb crawl (+http://www.companybooknetworking.com)
Block this crawler
robots.txt — disallow cb crawl:
User-agent: cb crawl
Disallow: /
Apache .htaccess — return 403:
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} cb crawl [NC]
RewriteRule .* - [F,L]
Nginx — return 403 inside a server block:
if ($http_user_agent ~* "cb crawl") {
return 403;
}
← back to all crawlers