Check explainer · v1
Noindex directives: why a crawlable page drops from results
Last reviewed
noindex is a promise, not a fault: it is exactly right for pages you keep out of search, and a silent traffic leak the moment it sits on a page that should rank. The check tells you which pages carry it.
How this check works
The meta robots tag on every public page is read for noindex and nofollow. A public page carrying noindex fails — it tells crawlers to drop the page from the index — and nofollow on a public page fails too, because links from it pass no equity. The free noindex check on the tool page reports the same directives from both channels, including the X-Robots-Tag response header.
What does a failure look like?
CMS “discourage search engines” settings left on after launch; an SEO plugin’s page-level switch ticking noindex on a template; staging titles and staging robots defaults shipped to production; a theme or edge worker injecting the directive on every page — all of these make eligible pages disappear while every other check passes.
How to fix it, step by step
- Find where the directive is set: a CMS page-level switch, an SEO plugin default, a theme head include, or an edge/CDN header rule.
- Remove noindex from the pages that should rank and redeploy; keep it deliberately on pages meant to stay out of search, such as internal utilities or thank-you pages.
- Re-run the check on a sample of formerly failing URLs and confirm the directive is gone before requesting indexing in Search Console.
What does a pass or fail look like?
Good
Public pages carry no noindex or nofollow in meta robots; deliberate exclusions exist only on pages meant to stay out of search.
Fail
A landing page serves <meta name="robots" content="noindex"> from a plugin default — the page is fetchable, the content is fine, and it is excluded anyway.
Frequently asked questions
Where can a noindex directive live?
Two channels reach crawlers with the same effect: the meta robots tag in the page head and the X-Robots-Tag HTTP response header. The header variant is invisible in page-source tools that only read HTML, which is why the free noindex check reports both. robots.txt is not a noindex channel at all.
Does robots.txt noindex a page?
No. robots.txt blocks crawling, not indexing, and a blocked page cannot show crawlers its noindex tag. Pages excluded this way often surface as bare results learned from links. To remove a page from the index, allow the crawl and serve noindex.
noindex vs nofollow: what is the difference?
noindex keeps the page itself out of the index; nofollow tells crawlers not to pass link equity through the links on it. Both can sit in the same meta robots tag or X-Robots-Tag header, and they are independent — a nofollow page can still rank, and a noindex page’s nofollow is moot once the page is out.
Can noindex remove a page that is already indexed?
It is a removal request, not a switch. Google drops the page when it refetches and reads the directive — search scheduling decides when. Search Console removals handle urgent cases; everything else waits for the recrawl.
Run this check now: Noindex checker
Free, no signup. Or run the full site audit on one URL.