Jamie's Spider
Jamie Brown's personal web crawler bot
About this crawler
Jamie's Spider is a web crawler identified by the regular-expression pattern Jamie's Spider in the User-Agent request header. It is categorised as seo. Use the regex above to detect, log, allow, or block Jamie's Spider traffic in your web server, CDN edge rules, or robots.txt.
Block-rate · top 25k sites
0.065%
Technical details
- Name
- Jamie's Spider
- Pattern
Jamie's Spider- Tags
- seo
- Reference
- https://uk.linkedin.com/in/jamiebrown
- Added
- 2017/11/02
- Instances
- 1 known sample(s)
Sample User-Agent strings
Jamie's Spider (http://jamiembrown.com/)
Block this crawler
robots.txt — disallow Jamie's Spider:
User-agent: Jamie's Spider
Disallow: /
Apache .htaccess — return 403:
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} Jamie's Spider [NC]
RewriteRule .* - [F,L]
Nginx — return 403 inside a server block:
if ($http_user_agent ~* "Jamie's Spider") {
return 403;
}
← back to all crawlers