Check explainer · v1

Noindex directives: why a crawlable page drops from results

Automated

Last reviewed

noindex is a promise, not a fault: it is exactly right for pages you keep out of search, and a silent traffic leak the moment it sits on a page that should rank. The check tells you which pages carry it.

How this check works

The meta robots tag on every public page is read for noindex and nofollow. A public page carrying noindex fails — it tells crawlers to drop the page from the index — and nofollow on a public page fails too, because links from it pass no equity. The free noindex check on the tool page reports the same directives from both channels, including the X-Robots-Tag response header.

What does a failure look like?

CMS “discourage search engines” settings left on after launch; an SEO plugin’s page-level switch ticking noindex on a template; staging titles and staging robots defaults shipped to production; a theme or edge worker injecting the directive on every page — all of these make eligible pages disappear while every other check passes.

How to fix it, step by step

  1. Find where the directive is set: a CMS page-level switch, an SEO plugin default, a theme head include, or an edge/CDN header rule.
  2. Remove noindex from the pages that should rank and redeploy; keep it deliberately on pages meant to stay out of search, such as internal utilities or thank-you pages.
  3. Re-run the check on a sample of formerly failing URLs and confirm the directive is gone before requesting indexing in Search Console.

What does a pass or fail look like?

Good

Public pages carry no noindex or nofollow in meta robots; deliberate exclusions exist only on pages meant to stay out of search.

Fail

A landing page serves <meta name="robots" content="noindex"> from a plugin default — the page is fetchable, the content is fine, and it is excluded anyway.

Frequently asked questions

Where can a noindex directive live?

Two channels reach crawlers with the same effect: the meta robots tag in the page head and the X-Robots-Tag HTTP response header. The header variant is invisible in page-source tools that only read HTML, which is why the free noindex check reports both. robots.txt is not a noindex channel at all.

Does robots.txt noindex a page?

No. robots.txt blocks crawling, not indexing, and a blocked page cannot show crawlers its noindex tag. Pages excluded this way often surface as bare results learned from links. To remove a page from the index, allow the crawl and serve noindex.

noindex vs nofollow: what is the difference?

noindex keeps the page itself out of the index; nofollow tells crawlers not to pass link equity through the links on it. Both can sit in the same meta robots tag or X-Robots-Tag header, and they are independent — a nofollow page can still rank, and a noindex page’s nofollow is moot once the page is out.

Can noindex remove a page that is already indexed?

It is a removal request, not a switch. Google drops the page when it refetches and reads the directive — search scheduling decides when. Search Console removals handle urgent cases; everything else waits for the recrawl.

Run this check now: Noindex checker

Free, no signup. Or run the full site audit on one URL.

Visdenda check explainers · one page per published ruleNoindex · v1