cohere-training-data-crawler

ai-crawler cohere-training-data-crawler

LLM training data collection for AI models

About this crawler

cohere-training-data-crawler is a web crawler identified by the regular-expression pattern cohere-training-data-crawler in the User-Agent request header. It is categorised as ai-crawler. Use the regex above to detect, log, allow, or block cohere-training-data-crawler traffic in your web server, CDN edge rules, or robots.txt.

Block-rate · top 25k sites

1.9%
latest snapshot
2026-06-04
matched key: cohere-training-data-crawler
2026-05-012026-06-042.4%

Technical details

Name
cohere-training-data-crawler
Pattern
cohere-training-data-crawler
Tags
ai-crawler
Reference
https://datadome.co/bots/cohere-training-data-crawler/
Added
2026/04/17
rDNS suffixes
.cohere.ai, .cohere.com
Instances
1 known sample(s)

rDNS verification (FCrDNS)

Verify a request is genuinely cohere-training-data-crawler with forward-confirmed reverse DNS: the client IP's PTR record must end in one of the suffixes below and a forward A/AAAA lookup of that hostname must return the same IP. UA strings alone are spoofable; FCrDNS is not.

Sample User-Agent strings

cohere-training-data-crawler ([email protected])

Block this crawler

robots.txt — disallow cohere-training-data-crawler:

User-agent: cohere-training-data-crawler Disallow: /

Apache .htaccess — return 403:

RewriteEngine On RewriteCond %{HTTP_USER_AGENT} cohere-training-data-crawler [NC] RewriteRule .* - [F,L]

Nginx — return 403 inside a server block:

if ($http_user_agent ~* "cohere-training-data-crawler") { return 403; }
← back to all crawlers