DF Bot
Web content indexing and competitive analysis
About this crawler
DF Bot is a web crawler identified by the regular-expression pattern DF Bot in the User-Agent request header. It is categorised as seo. Use the regex above to detect, log, allow, or block DF Bot traffic in your web server, CDN edge rules, or robots.txt.
Block-rate · top 25k sites
0.065%
Technical details
- Name
- DF Bot
- Pattern
DF Bot- Tags
- seo
- Reference
- https://datadome.co/bots/df-bot/
- Added
- 2026/04/17
- Instances
- 1 known sample(s)
Sample User-Agent strings
DF Bot 1.0
Block this crawler
robots.txt — disallow DF Bot:
User-agent: DF Bot
Disallow: /
Apache .htaccess — return 403:
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} DF Bot [NC]
RewriteRule .* - [F,L]
Nginx — return 403 inside a server block:
if ($http_user_agent ~* "DF Bot") {
return 403;
}
← back to all crawlers