Netcraft Web Server Survey
Netcraft web server survey crawler
About this crawler
Netcraft Web Server Survey is a web crawler identified by the regular-expression pattern Netcraft Web Server Survey in the User-Agent request header. It is categorised as advertising. Use the regex above to detect, log, allow, or block Netcraft Web Server Survey traffic in your web server, CDN edge rules, or robots.txt.
Block-rate · top 25k sites
0.33%
Technical details
- Name
- Netcraft Web Server Survey
- Pattern
Netcraft Web Server Survey- Tags
- advertising
- Reference
- https://www.netcraft.com/blog/june-2025-web-server-survey
- Added
- 2026/04/26
- Instances
- 1 known sample(s)
Sample User-Agent strings
Mozilla/4.0 (compatible; Netcraft Web Server Survey)
Block this crawler
robots.txt — disallow Netcraft Web Server Survey:
User-agent: Netcraft Web Server Survey
Disallow: /
Apache .htaccess — return 403:
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} Netcraft Web Server Survey [NC]
RewriteRule .* - [F,L]
Nginx — return 403 inside a server block:
if ($http_user_agent ~* "Netcraft Web Server Survey") {
return 403;
}
← back to all crawlers