Google-CloudVertexBot

search-engineai-crawler Google-CloudVertexBot

Google Vertex AI crawler indexing content for enterprise search solutions

About this crawler

Google-CloudVertexBot is a web crawler identified by the regular-expression pattern Google-CloudVertexBot in the User-Agent request header. It is categorised as search-engine, ai-crawler. Use the regex above to detect, log, allow, or block Google-CloudVertexBot traffic in your web server, CDN edge rules, or robots.txt.

Block-rate · top 25k sites

1.4%
latest snapshot
2026-06-04
matched key: Google-CloudVertexBot
2026-05-012026-06-041.7%

Technical details

Name
Google-CloudVertexBot
Pattern
Google-CloudVertexBot
Tags
search-engine, ai-crawler
Reference
https://developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers
Added
2026/04/17
rDNS suffixes
.1e100.net, .googlebot.com, .googleusercontent.com
Instances
2 known sample(s)

rDNS verification (FCrDNS)

Verify a request is genuinely Google-CloudVertexBot with forward-confirmed reverse DNS: the client IP's PTR record must end in one of the suffixes below and a forward A/AAAA lookup of that hostname must return the same IP. UA strings alone are spoofable; FCrDNS is not.

Sample User-Agent strings

Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; Google-CloudVertexBot; +https://cloud.google.com/enterprise-search)
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Google-CloudVertexBot; +https://cloud.google.com/enterprise-search) Chrome/W.X.Y.Z Safari/537.36

Block this crawler

robots.txt — disallow Google-CloudVertexBot:

User-agent: Google-CloudVertexBot Disallow: /

Apache .htaccess — return 403:

RewriteEngine On RewriteCond %{HTTP_USER_AGENT} Google-CloudVertexBot [NC] RewriteRule .* - [F,L]

Nginx — return 403 inside a server block:

if ($http_user_agent ~* "Google-CloudVertexBot") { return 403; }
← back to all crawlers