Check explainer · v1

Sitemap URLs that can never be indexed

Automated

Last reviewed

A sitemap entry is a request to index that exact URL. When the same URL is noindex, blocked for Googlebot, or canonicalizes elsewhere, the request contradicts itself.

How this check works

Each listed URL is checked for a noindex directive (meta robots or X-Robots-Tag), Googlebot access in robots.txt, and — when the crawl fetched it — a canonical target that points elsewhere. One finding is emitted per listed URL, canonical first. A sitemap entry that cannot be indexed wastes crawl attention and contradicts the page’s own signals.

What does a failure look like?

A template adds noindex to filtered or paginated URLs while the sitemap generator keeps listing them; robots.txt disallows a directory the sitemap still submits; a migration leaves absolute canonical URLs on the old host, so every listed URL points somewhere other than itself.

How to fix it, step by step

  1. Generate the sitemap from the same URL set your canonical tags describe — one source, not two.
  2. Resolve each finding on its own terms: drop noindex pages from the sitemap, stop submitting paths robots.txt blocks, or fix the canonical target.
  3. Re-run the audit after template, redirect or migration changes; canonical and noindex findings depend on the crawl reaching the URL.

What does a pass or fail look like?

Good

214 listed URLs: none noindex, all allowed for Googlebot, every one self-canonical.

Fail

A category page is listed in the sitemap but its canonical points to /collections/all — the submitted URL tells Google to look elsewhere.

Frequently asked questions

Does a sitemap URL have to be self-canonical?

Yes. The sitemap submits that exact URL; if the page’s canonical names a different one, the submission is discarded. The audit reports the listed URL and the canonical target it found.

How is a robots.txt block different from noindex?

A robots.txt disallow stops Googlebot from fetching the URL, so the noindex is never even read. Either way the listed URL cannot be indexed; the audit reports them as separate causes so the fix targets the right file.

Should I fix the page or remove it from the sitemap?

Depends on intent: if the page should rank, remove the blocker — noindex, robots rule or wrong canonical. If it should not rank, remove it from the sitemap instead of submitting a URL you have chosen to hide.

This rule is one of the checks the audit runs. Run the full site audit

Free preview; no account needed.

Visdenda check explainers · one page per published ruleSitemap indexability · v1