What is indexing of a web page in Google?

Here indexing is not the kind that ties a lease to inflation, or a database index: it is what a search engine does with your page after visiting it. For a store, a product page that is not in the index does not exist for Google: it does not show up in results, it cannot bring traffic and it cannot serve as a source in its AI answers either. And it is not unusual for that to happen. Google says plainly that it does not guarantee crawling, indexing or serving any page, even if it meets its essential requirements.⁠^{[1]Google Search Central. In-depth guide to how Google Search works}

How it works

Google describes Search in three stages, and not every page makes it through all three: crawling, in which a crawler downloads the text, images and videos of the pages it finds; indexing, in which it analyzes that content and stores it in the index, a large database; and serving results, when someone searches and Google returns what it considers relevant.⁠^{[1]Google Search Central. In-depth guide to how Google Search works}

What happens inside indexing is more involved than the word "store" suggests. Google processes the text and key content tags and attributes, such as the <title> element or alt attributes, as well as images and videos. It groups the pages it considers duplicates of each other and picks the most representative one in the group as the canonical; from that page it collects signals it later uses when serving results.⁠^{[1]Google Search Central. In-depth guide to how Google Search works} That is why in a store one URL per group gets indexed, not one per variant or per filter, and Google says so in its report: your goal should be to get the canonical version of every important page indexed, and duplicate or alternate pages should not be indexed.⁠^{[2]Search Console Help. Page indexing report}

The URL that ends up in the index is not entirely your call. A redirect and the canonical tag are strong signals, and appearing in a sitemap is a weak one, but all three are hints, not orders: Google may pick a different canonical if it thinks it is better.⁠^{[3]Google Search Central. How to specify a canonical URL with rel="canonical" and other methods} The same goes for the sitemap as a discovery route: it helps Google find your URLs, but it does not guarantee that all of them are crawled and indexed.⁠^{[4]Google Search Central. Learn about sitemaps}

JavaScript adds an intermediate stage. Google renders the page with a recent version of Chrome and runs the JavaScript it finds, as your browser would.⁠^{[1]Google Search Central. In-depth guide to how Google Search works} Pages wait in a render queue, and what gets indexed is the rendered HTML.⁠^{[5]Google Search Central. Understand JavaScript SEO basics} Two nuances matter in a store. If the page responds with a status code other than 200, for example a 404, rendering may be skipped. And if Google finds a noindex in the original HTML, it may skip rendering and running the JavaScript, so removing it later with a script may do nothing.⁠^{[5]Google Search Central. Understand JavaScript SEO basics} Google still recommends server-side rendering or prerendering, because it makes the site faster for users and crawlers.⁠^{[5]Google Search Central. Understand JavaScript SEO basics}

Two instructions decide what gets in and what does not, and they get mixed up often. The robots.txt file tells crawlers which URLs they can access; it is mostly for avoiding overloading your server and it is not a mechanism for keeping a page out of Google: a blocked URL can still be indexed if other sites link to it, and then it shows up without a description.⁠^{[6]Google Search Central. Introduction to robots.txt} To keep a page out of the index there is the noindex rule, as a <meta name="robots"> tag or as an X-Robots-Tag HTTP header. For it to work, the page cannot be blocked in robots.txt: the crawler has to be able to read it to see the instruction.⁠^{[7]Google Search Central. Block Search indexing with noindex}

Indexing is also the gate to Google's AI answers. To appear as a supporting link in AI Overviews or in Google AI Mode, a page has to be indexed and eligible to be shown in Search with a snippet, and Google states there are no additional requirements.⁠^{[8]Google Search Central. AI features and your website}

In a store

This is how the page indexing report in Search Console would read for a backpack store. It is a fictitious example: the URLs and statuses are made up to show how each one is interpreted; the status names and their meaning are those in Google's help.⁠^{[2]Search Console Help. Page indexing report}⁠^{[9]Search Console Help. URL Inspection tool}

The row that usually costs the most money is "Crawled - currently not indexed," which does not appear in this example: Google visited the page and decided not to store it, although it may index it later.⁠^{[2]Search Console Help. Page indexing report} There is no switch for that; the page has to earn its place.

How to check it

1. Inspect a specific product page. In Search Console, the URL Inspection tool tells you whether "URL is on Google" or "URL is not on Google," and shows you the user-declared canonical and the Google-selected one.⁠^{[9]Search Console Help. URL Inspection tool} If they do not match, that is your first job. Keep in mind that the "URL is on Google" status does not assure the page appears in results.⁠^{[9]Search Console Help. URL Inspection tool}
2. Look at what Google sees, not what you see. Click "Test Live URL" and then "View tested page": you get a screenshot of the rendered page, the HTML, the HTTP headers, the JavaScript console and the loaded resources.⁠^{[9]Search Console Help. URL Inspection tool} Look in that HTML for the product name, the price and the description. If they only appear when JavaScript runs, check that they also show up in the rendered HTML.⁠^{[5]Google Search Central. Understand JavaScript SEO basics}
3. Do not read the live test as a verdict. Compare it with the indexed version, which is the one from the last crawl. The live test does not check, among other things, whether the page is a duplicate of another or the quality guidelines.⁠^{[9]Search Console Help. URL Inspection tool}
4. Review the page indexing report by reason, not by total. Do not expect every URL on your site to be indexed: check that the canonicals of your important product and category pages are.⁠^{[2]Search Console Help. Page indexing report}
5. Ask for it to come back once you have fixed it. For a URL, "Request indexing," with a daily limit on requests and no guarantee the page gets into the index.⁠^{[9]Search Console Help. URL Inspection tool} For a whole reason, "Validate fix": it usually takes about two weeks and sometimes much longer.⁠^{[2]Search Console Help. Page indexing report}

Related concept: crawling

Crawling is downloading the page; indexing is understanding it and deciding whether to store it.⁠^{[1]Google Search Central. In-depth guide to how Google Search works} Search Console separates the two in its statuses: "Discovered - currently not indexed" means Google found the page but has not crawled it yet; "Crawled - currently not indexed," that it crawled it and did not index it.⁠^{[2]Search Console Help. Page indexing report} The first problem is one of access and priority; the second, of what the page offers compared with the others. The comparison crawling vs indexing puts the two stages side by side.

To go further, the crawling and indexing reading path orders the rest of the terms, and the guide to what an AI crawler sees in your store goes down to the agents that are not Google.