Claude-User
Anthropic · reviewed 2026-11-05
About Claude-User
Fetches a page when a Claude user asks for up-to-date web information.
Operates Claude; separates user-triggered fetches, search indexing and training crawlers into distinct user agents.
Usually means: A person asked an AI assistant a question, and the assistant fetched this URL to answer with fresh context or citations.
Identity
User agent
Crawlers are matched by token, not the full browser string. Providers may add version or product text around it.
robots.txt token
Use this token in a User-agent line to allow or block the crawler.
Block this token
User-agent: Claude-User Disallow: /
A Disallow only stops crawlers that honor robots.txt. Classify the category before blocking; check the live file with the robots.txt checker.
Verify a hit
A matching user agent is not proof: any client can send the string. When the vendor publishes network data, compare the request against it before you trust the hit.
Match the user agent
Request UA contains Claude-User.
Check reverse DNS
The vendor publishes no reverse-DNS suffix.
Check the IP range
Compare the request IP with the ranges below.
Sample IP ranges
160.79.104.0/21Range endpoints
A verified hit proves the network path; it does not prove what the crawler will do with the page.
Claude-User: common questions
What is Claude-User?
Claude-User is a crawler from Anthropic, filed under AI answers. Fetches a page when a Claude user asks for up-to-date web information. Requests advertise it in the User-Agent header as Claude-User.
Does Claude-User train AI models?
No. It fetches a page only when a person asks Anthropic's assistant a question that needs live context.
How do I block Claude-User in robots.txt?
Add a group with User-agent: Claude-User and Disallow: / to robots.txt, then re-test the live file. robots.txt matches the token, not the full User-Agent string, and blocking this token does not block the vendor's other crawlers.
How do I verify a Claude-User hit?
Match the user agent, then compare the request IP with the ranges or endpoints listed on this page. A matching user agent alone is not proof — any client can send the string.
Testing a block? Run the live file through the robots.txt tester or check the whole AI crawler access group.
See when Claude-User hits your site
Page scripts never see AI crawlers because they do not run JavaScript. Install the server probe and each hit lands in the Crawlers block with its path and status.