DomainsDB.net MetaCrawler
DomainsDB domain database crawler
About this crawler
DomainsDB.net MetaCrawler is a web crawler identified by the regular-expression pattern DomainsDB\.net MetaCrawler in the User-Agent request header. It is categorised as seo. Use the regex above to detect, log, allow, or block DomainsDB.net MetaCrawler traffic in your web server, CDN edge rules, or robots.txt.
Block-rate · top 25k sites
No block-rate data for this crawler.
Technical details
- Name
- DomainsDB.net MetaCrawler
- Pattern
DomainsDB\.net MetaCrawler- Tags
- seo
- Reference
- http://domainsdb.net/
- Added
- 2026/05/09
- Instances
- 1 known sample(s)
Sample User-Agent strings
DomainsDB.net MetaCrawler v.0.9.7c (http://domainsdb.net/)
Block this crawler
robots.txt — disallow DomainsDB.net MetaCrawler:
User-agent: DomainsDB.net MetaCrawler
Disallow: /
Apache .htaccess — return 403:
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} DomainsDB\.net MetaCrawler [NC]
RewriteRule .* - [F,L]
Nginx — return 403 inside a server block:
if ($http_user_agent ~* "DomainsDB\\.net MetaCrawler") {
return 403;
}
← back to all crawlers