Google-Extended
Google · reviewed 2026-11-05
About Google-Extended
robots.txt opt-out token for Gemini app training; has no user agent of its own (fetches come as Googlebot).
Operates Google Search, Gemini and agent products; publishes per-purpose crawlers verified by IP ranges and reverse DNS.
Usually means: A model provider fetched this URL for future training data. It does not mean today's answers cite the page.
Identity
This token is declared in robots.txt only. It does not send an HTTP user agent, so it cannot be matched by user-agent strings.
robots.txt token
Use this token in a User-agent line to allow or block the crawler.
Block this token
User-agent: Google-Extended Disallow: /
A Disallow only stops crawlers that honor robots.txt. Classify the category before blocking; check the live file with the robots.txt checker.
Verify a hit
This entry exists only as a robots.txt token. It does not send an HTTP request of its own, so there is no user agent, reverse DNS or IP range to verify. Fetches that honor the token arrive under the vendor's HTTP crawler identity.
Google-Extended: common questions
What is Google-Extended?
Google-Extended is a robots.txt token from Google, filed under Training — robots.txt opt-out token for Gemini app training; has no user agent of its own (fetches come as Googlebot).
Does Google-Extended train AI models?
It is a control rather than a fetcher: the token lets a site opt out of Google's use of its content for training, and it produces no hits of its own.
How do I block Google-Extended in robots.txt?
Add User-agent: Google-Extended with Disallow: / to robots.txt. Google-Extended is an opt-out token, not a fetcher: the directive changes how Google may use your content, while the vendor's HTTP crawlers keep their own tokens.
How do I verify a Google-Extended hit?
There is no Google-Extended hit to verify: the token sends no request of its own. Fetches you see arrive under Google's HTTP crawler identity instead.
Google-Extended vs Googlebot: what is the difference?
Googlebot is filed under Search engines, while Google-Extended is filed under Training. They use separate robots.txt tokens: blocking one does not block the other, and a hit from one says nothing about the other.
Testing a block? Run the live file through the robots.txt tester or check the whole AI crawler access group.
See when Google-Extended hits your site
Page scripts never see AI crawlers because they do not run JavaScript. Install the server probe and each hit lands in the Crawlers block with its path and status.