CC Metadata Scaper
Creative Commons metadata web crawler
About this crawler
CC Metadata Scaper is a web crawler identified by the regular-expression pattern CC Metadata Scaper in the User-Agent request header. It is categorised as archiver. Use the regex above to detect, log, allow, or block CC Metadata Scaper traffic in your web server, CDN edge rules, or robots.txt.
Block-rate · top 25k sites
0.065%
Technical details
- Name
- CC Metadata Scaper
- Pattern
CC Metadata Scaper- Tags
- archiver
- Reference
- https://wiki.creativecommons.org/wiki/Metadata_Scraper
- Added
- 2014/04/01
- Instances
- 1 known sample(s)
Sample User-Agent strings
CC Metadata Scaper http://wiki.creativecommons.org/Metadata_Scraper
Block this crawler
robots.txt — disallow CC Metadata Scaper:
User-agent: CC Metadata Scaper
Disallow: /
Apache .htaccess — return 403:
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} CC Metadata Scaper [NC]
RewriteRule .* - [F,L]
Nginx — return 403 inside a server block:
if ($http_user_agent ~* "CC Metadata Scaper") {
return 403;
}
← back to all crawlers