This translation has not been editorially reviewed yet. The German version is authoritative. Deutsche Fassung →
Checking noindex: the most common cause of invisible pages
A noindex directive removes the page from the search index — completely, and regardless of everything else. This is the check point with the greatest individual weight, because it makes every other one worthless: nobody finds a perfectly built page set to noindex.
- The classic case is the forgotten switch after a relaunch: during development, the page was deliberately set to noindex, and when it went live, the switch never got reset. Weeks later, somebody asks why the new website never shows up in any search.
- The directive can be in two places: as a meta tag in the page head, or as an HTTP header. The second is treacherous, because it's not visible in the source code — anyone who only looks at the HTML never finds it. The check sees both.
- noindex is a legitimate tool: thank-you pages, filter views, and internal search results don't belong in the index. The finding states that the directive is there — whether it belongs there is for the operator to decide.
FAQ
Where can a noindex directive be located?
In the "robots" meta tag in the page head, and in the "X-Robots-Tag" HTTP header. The second one is invisible in the source code, and so it's the case that stays undetected the longest.
Is noindex the same as an exclusion in robots.txt?
No, and mixing them up has real consequences. A robots.txt forbids crawling; noindex forbids indexing. A page blocked via robots.txt can even end up in the index anyway — the search engine then only sees the address, not the content, and never even gets to read the noindex.
When is noindex the right choice?
For pages with no standalone value for searchers: confirmation pages, filter and sort views, internal search results, print versions.