In beta, with limited placesOpen to everyone in Q1 2027

↑↓ to move · Enter to open · Esc to close

Crawling and indexing

What is the noindex tag and what is it for?

noindex is a rule placed on a web page, as a robots meta tag in the HTML or as an X-Robots-Tag HTTP header, that asks search engines not to include it in their index or show it in their results. For it to work, the crawler has to be able to download the page and read the rule: if robots.txt blocks it, the crawler will not see it.

Written by
Reviewed by
Published 7 min read
Takeaway:

Key takeaways

  • noindex asks search engines not to keep a page in their index; the page still exists and your customers can see it.
  • It is written in a robots meta tag in the HTML or in the X-Robots-Tag HTTP header, and the header does not show in the source code.
  • It only works if the crawler can read the page: if robots.txt blocks it, the rule never arrives.
  • On Google, a page with noindex also cannot appear in AI Overviews or AI Mode, which require an indexed page.
  • For duplicates inside your own store, Google prefers the canonical tag: noindex takes the page out of Search entirely.

Putting noindex on a page does not delete it or hide it from your customers: it stays published, linked and on sale. All it does is ask search engines not to keep it in their index, that is, to leave it out of indexing and so not show it in their results. In a store it has clear uses: internal search results, the thank-you page after checkout, a campaign's test pages. And it has an equally clear risk, because it is a silent rule: a noindex left on by mistake in a product page template takes those pages out of Google without anyone noticing anything in the store.

How it works

Google defines noindex as a rule set with a <meta> tag or with an HTTP response header that stops search engines that support it, such as Google, from indexing that content. There are two ways to set it:⁠[1]Google Search Central. Block Search indexing with noindex

  • In the HTML, inside the <head>: <meta name="robots" content="noindex"> applies to every search engine that respects it, and <meta name="googlebot" content="noindex"> only to Google.
  • In the HTTP header of the response: X-Robots-Tag: noindex. It is the way to mark what is not HTML, such as a PDF or a video.

Any rule that works in the meta tag also works in the header, and neither the header name, nor the agent name, nor the values are case sensitive. If two rules contradict each other, the more restrictive one applies.⁠[2]Google Search Central. Robots meta tag, data-nosnippet, and X-Robots-Tag specifications

When Googlebot crawls the page and finds the rule, Google removes it from its results entirely, even if other sites link to it. It is not immediate: the page drops out when Google crawls it again, and that depends on its importance, although you can ask for it sooner from the URL Inspection tool in Search Console.⁠[1]Google Search Central. Block Search indexing with noindex

The condition most often broken is access. For noindex to work, the page must not be blocked in robots.txt and has to be accessible to the crawler: if it cannot download the page, it cannot read the rule.⁠[1]Google Search Central. Block Search indexing with noindex⁠[2]Google Search Central. Robots meta tag, data-nosnippet, and X-Robots-Tag specifications The robots.txt entry explains in detail why that combination leaves pages in the index.

JavaScript has its own trap. Google queues pages that answer with a 200 for rendering, unless a robots meta tag or a header tells it not to index them. And if it finds noindex in the original HTML, it may skip rendering and running the JavaScript, so removing the rule with a script does not work the way you expect. If you want the page indexed, Google asks you not to put noindex in the original code.⁠[3]Google Search Central. Understand the JavaScript SEO basics

Other search engines may interpret the rule differently, and Google warns that the page could then keep showing up in them.⁠[1]Google Search Central. Block Search indexing with noindex With AI assistants the situation is even less uniform: OpenAI's crawler page explains how to control each of its agents with robots.txt and does not mention noindex.⁠[4]OpenAI. Overview of OpenAI Crawlers On Google, by contrast, the effect reaches its AI features: to appear as a link in AI Overviews or in AI Mode, a page has to be indexed, and noindex is listed among the controls for limiting what is shown in them.⁠[5]Google Search Central. AI features and your website

In a store

This is how the rule looks in its two forms, on a results page from a store's internal search. It is a made-up example: the domain and the path are invented.

<!-- https://store-example.com/search?q=backpack -->
<head>
  <meta name="robots" content="noindex" />
</head>
HTTP/2 200
content-type: text/html; charset=utf-8
x-robots-tag: noindex

One of the two is enough. The first can be read in the source code; the second only in the response headers, and that is why it is the one that slips through an audit most often.

On Shopify there are three routes, and they do not do the same thing. You can add the tag in the theme's theme.liquid, with a condition on the page's handle: the page then disappears from search engines but still shows up in your store's own search. You can set the seo.hidden metafield to 1, which hides it from sitemaps, search engines and your store's search. Or, for products only, you can set the product to unlisted, which hides it from internet searches, from Shopify Catalog and from the sitemap, and also from the store's collections, results and recommendations.⁠[6]Shopify Help Center. Hiding a page from search engines The Shopify snippet for the first route is this:

{% if handle contains 'page-handle-you-want-to-exclude' %}
<meta name="robots" content="noindex">
{% endif %}
Warning:

What people often get wrong

Not seeing noindex in the source code does not mean it is not there. The rule can sit in the HTTP header, which does not appear when you view the page's code,⁠[2]Google Search Central. Robots meta tag, data-nosnippet, and X-Robots-Tag specifications and Google asks you to look in both places, the source code and the response headers.⁠[7]Search Console Help. Page indexing report And the other way around: having it in the code does not prove Google reads it, if robots.txt stops Google from downloading the page.⁠[1]Google Search Central. Block Search indexing with noindex

How to check it

  1. Search the page's source code for the word: a <meta name="robots"> or <meta name="googlebot"> tag with noindex.⁠[7]Search Console Help. Page indexing report
  2. Look at the response headers, with curl -I https://yourdomain.com/path or with the browser's developer tools, and look for x-robots-tag.⁠[7]Search Console Help. Page indexing report
  3. Inspect the URL in Search Console. In the indexed version, "Indexing allowed?" tells you whether a noindex is preventing it; with "Test live URL" you check whether the rule is still on today's page. If it is gone, "Request indexing".⁠[7]Search Console Help. Page indexing report
  4. Review the reason 'URL marked "noindex"' in the page indexing report and confirm that it holds only pages you want out, never product pages or categories.⁠[7]Search Console Help. Page indexing report
  5. Check that robots.txt does not block those URLs. If it does, Google will not see the rule.⁠[1]Google Search Central. Block Search indexing with noindex

noindex and the canonical tag are sometimes used for the same thing, a store's repeated URLs (variants, filters, sort orders), and they do not do the same thing. The canonical says which of several similar URLs is the main one and concentrates the signals on it; noindex takes the page out of Search without pointing to any other. That is why Google does not recommend using noindex to keep a page from being chosen as canonical within a single site: it blocks it from Search entirely, and the preferred solution is rel="canonical".⁠[8]Google Search Central. How to specify a canonical URL with rel="canonical" and other methods If what you want is to stay in Search but out of its AI features, that is not noindex either: that is what the Search Console generative AI control is for, and Google points to noindex only for leaving Search altogether.⁠[9]Search Console Help. Search generative AI control

To go further, the indexing entry explains how to read the Search Console statuses, and the crawling and indexing reading path orders the rest of the terms.

Synonyms

English
noindex tag, noindex directive
Español
etiqueta noindex, directiva noindex

Sources

  1. [1] Google Search Central. Block Search indexing with noindex. Accessed . ↑ ↑ ↑ ↑ ↑ ↑
  2. [2] Google Search Central. Robots meta tag, data-nosnippet, and X-Robots-Tag specifications. Accessed . ↑ ↑ ↑
  3. [3] Google Search Central. Understand the JavaScript SEO basics. Accessed . ↑
  4. [4] OpenAI. Overview of OpenAI Crawlers. Accessed . ↑
  5. [5] Google Search Central. AI features and your website. Accessed . ↑
  6. [6] Shopify Help Center. Hiding a page from search engines. Accessed . ↑
  7. [7] Search Console Help. Page indexing report. Accessed . ↑ ↑ ↑ ↑ ↑
  8. [8] Google Search Central. How to specify a canonical URL with rel="canonical" and other methods. Accessed . ↑
  9. [9] Search Console Help. Search generative AI control. Accessed . ↑

Change history

  • First English version.

This definition is published under CC BY 4.0. Reuse this definition freely, crediting Clione with a link to this page.

Make your catalog easy for bots to read

Clione fills every product page with what bots read best: the JSON-LD graph with its structured data, the meta description and the FAQs. You review and press Push; then we check your live store so it all stays published.