New York Times Newsgathering
News media content aggregation and research crawler bot
About this crawler
New York Times Newsgathering is a web crawler identified by the regular-expression pattern New York Times Newsgathering in the User-Agent request header. It is categorised as monitoring. Use the regex above to detect, log, allow, or block New York Times Newsgathering traffic in your web server, CDN edge rules, or robots.txt.
Block-rate · top 25k sites
0.065%
Technical details
- Name
- New York Times Newsgathering
- Pattern
New York Times Newsgathering- Tags
- monitoring
- Reference
- https://www.nytimes.com/
- Added
- 2026/04/26
- Instances
- 1 known sample(s)
Sample User-Agent strings
New York Times Newsgathering
Block this crawler
robots.txt — disallow New York Times Newsgathering:
User-agent: New York Times Newsgathering
Disallow: /
Apache .htaccess — return 403:
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} New York Times Newsgathering [NC]
RewriteRule .* - [F,L]
Nginx — return 403 inside a server block:
if ($http_user_agent ~* "New York Times Newsgathering") {
return 403;
}
← back to all crawlers