Digg Deeper
Digg web crawler for content discovery
About this crawler
Digg Deeper is a web crawler identified by the regular-expression pattern Digg Deeper in the User-Agent request header. It is categorised as seo. Use the regex above to detect, log, allow, or block Digg Deeper traffic in your web server, CDN edge rules, or robots.txt.
Block-rate · top 25k sites
0.065%
Technical details
- Name
- Digg Deeper
- Pattern
Digg Deeper- Tags
- seo
- Reference
- http://digg.com/about
- Added
- 2017/09/26
- Instances
- 1 known sample(s)
Sample User-Agent strings
Digg Deeper/v1 (http://digg.com/about)
Block this crawler
robots.txt — disallow Digg Deeper:
User-agent: Digg Deeper
Disallow: /
Apache .htaccess — return 403:
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} Digg Deeper [NC]
RewriteRule .* - [F,L]
Nginx — return 403 inside a server block:
if ($http_user_agent ~* "Digg Deeper") {
return 403;
}
← back to all crawlers