Neticle Crawler
Neticle web crawler for content analysis
About this crawler
Neticle Crawler is a web crawler identified by the regular-expression pattern Neticle Crawler in the User-Agent request header. It is categorised as seo. Use the regex above to detect, log, allow, or block Neticle Crawler traffic in your web server, CDN edge rules, or robots.txt.
Block-rate · top 25k sites
0.20%
Technical details
- Name
- Neticle Crawler
- Pattern
Neticle Crawler- Tags
- seo
- Reference
- https://neticle.com/bot/en
- Added
- 2020/03/02
- Instances
- 1 known sample(s)
Sample User-Agent strings
Neticle Crawler v1.0 ( https://neticle.com/bot/en/ )
Block this crawler
robots.txt — disallow Neticle Crawler:
User-agent: Neticle Crawler
Disallow: /
Apache .htaccess — return 403:
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} Neticle Crawler [NC]
RewriteRule .* - [F,L]
Nginx — return 403 inside a server block:
if ($http_user_agent ~* "Neticle Crawler") {
return 403;
}
← back to all crawlers