This translation has not been editorially reviewed yet. The German version is authoritative. Deutsche Fassung
SEO
What are the Robots instructions none and unavailable after?
Both instructions take a page out of the search. "none" acts like "noindex, nofollow": Google does not show the page and does not follow its links. "unavailable_after" takes the page from the specified time from the search results. Expert mode reports both for viewing, as well as a robot meta tag outside the sidehead.
The side head is the intended location
Google recommends putting the robots meta tag in the head section of the page, but also notes it in the body. Screaming Frog indicates that search engines can pass a day outside the page header.
A date in the template takes many pages from the search
unavailable after sets an exact time from which Google no longer shows the page. If the instruction is in a template, for example for actions or job advertisements, every page that uses this template disappears after the date. Screaming Frog therefore advises checking the date.
This is how the expert mode measures
The expert mode reads the robots meta tags and their location in the source text on each measured page. It reports the instructions none and unavailable after and any robot meta tag that is outside the head element. The file shows the tag in the text.
This means the instructions
noindex means not showing the page in the search results, nofollow, not following its links, and none sums up both. For unavailable after, call the date and time in a common format such as ISO 8601. Without a valid date, Google passes the instruction. If two instructions contradict each other, the stricter applies according to Google. A page with none retrieves the search engine further to see the statement and then takes it from the index.
Examples
Take page only from search
<meta name="robots" content="none"><meta name="robots" content="noindex">noindex takes the page from search; The links of the page are followed by the search engine.
- For each reported day, check whether the page really should be out of search.
- If it is only to be found from the search, but its links continue, replace none with noindex.
- Put the robots meta tag in the head section of the page and check the date at unavailable after.
- Look for "robots" in the source text of the homepage and a subpage. If the same tag is in both, it comes from the template and applies to many pages.




