What is RAG (retrieval-augmented generation) in AI?

RAG stands for retrieval-augmented generation. It isn't a model or a product but a way of building a system: first it searches, then it writes. If you search for "rag" on its own you'll find cloths, ragtime music and little else; here we mean artificial intelligence. A store should care because every time an assistant answers with sources and cites your product page, your page was the retrieved document, and what the system took from it decides what it says about you.

How it works

The name comes from a research paper by Patrick Lewis and eleven other authors, published on arXiv on May 22, 2020 and accepted at NeurIPS 2020. It started from a problem: pretrained language models store knowledge in their parameters but struggle to access it precisely, and explaining where an answer comes from or updating what they know were still open problems.⁠^{[1]arXiv (Lewis, Perez, Piktus, Petroni, Karpukhin, Goyal, Küttler, Lewis, Yih, Rocktäschel, Riedel and Kiela; NeurIPS 2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks} Its proposal was to combine two memories: the model's own and an external one, a vector index of Wikipedia queried with a neural retriever. On generation tasks, those models produced language that was more specific, diverse and factual than an equivalent model without retrieval.⁠^{[1]arXiv (Lewis, Perez, Piktus, Petroni, Karpukhin, Goyal, Küttler, Lewis, Yih, Rocktäschel, Riedel and Kiela; NeurIPS 2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks}

A typical RAG system today does four things, and Anthropic describes them step by step:⁠^{[2]Anthropic (September 19, 2024). Contextual Retrieval in AI Systems}

1. It splits the documents into chunks of text, usually no more than a few hundred tokens.
2. It turns each chunk into a vector with an embeddings model and stores it in a vector database, so it can search by similarity of meaning.
3. It retrieves the chunks most relevant to the question, often combining that vector search with an exact-word search (BM25).
4. It adds those chunks to the request and the language model writes the answer with them in front of it.

The weak point is the first step. A loose chunk can lose what made it useful: Anthropic gives the example of a sentence from a financial report, "the company's revenue grew 3% over the previous quarter", which without the rest of the document doesn't say which company or which quarter it is about. By adding a line of context to each chunk before indexing it, Anthropic measured 49% fewer retrieval failures, and 67% fewer if it also reranked the results.⁠^{[2]Anthropic (September 19, 2024). Contextual Retrieval in AI Systems}

RAG in AI search engines

In AI search, the index is the web. Google defines RAG, which it also calls grounding, as a technique for improving the quality, accuracy and freshness of answers: its usual ranking systems retrieve pages from its index, the system reviews the specific information on those pages and generates the answer with links to the ones that support it.⁠^{[3]Google Search Central. Optimizing your website for generative AI features on Google Search} That is why, for Google, getting into what is retrieved still runs through crawling and indexing.

ChatGPT does something similar with outside search providers: it rewrites the question into one or more queries, reviews the first results and can run more specific ones before answering.⁠^{[4]OpenAI Help Center. Searching the web with ChatGPT} That step of multiplying queries has a name of its own, query fan-out. And Bing shows site owners a trace of the retrieval: its AI Performance report shows the grounding queries, the key phrases the AI used to retrieve the content it ended up citing.⁠^{[5]Microsoft, Webmaster Tools blog. Introducing AI Performance in Bing Webmaster Tools Public Preview}

In a store

A fictional example: the product page of a travel tripod. Microsoft explains that assistants split content into pieces and assemble the answer from pieces of several sources,⁠^{[6]Microsoft Advertising (Krishna Madhavan, October 8, 2025). Optimizing Your Content for Inclusion in AI Search Answers} so it pays to read the page the way a retriever would, paragraph by paragraph:

It is the same problem as the financial report: a sentence that only makes sense with the whole page around it gets retrieved worse and cited worse. Clione uses this scheme in its own enrichment: when the store turns it on, it attaches to each product's prompt four chunks from the documents the store has uploaded, about 1,200 tokens.

How to check it

1. Read your product page paragraph by paragraph, as loose chunks. If a paragraph doesn't name the product or depends on the previous one to make sense, rewrite it so it stands alone.⁠^{[2]Anthropic (September 19, 2024). Contextual Retrieval in AI Systems}
2. Pull the facts out of images, PDFs and closed tabs and put them in the page's HTML text.⁠^{[6]Microsoft Advertising (Krishna Madhavan, October 8, 2025). Optimizing Your Content for Inclusion in AI Search Answers}
3. Confirm in Search Console that the page is indexed. In Google, what isn't in the index isn't retrieved.⁠^{[3]Google Search Central. Optimizing your website for generative AI features on Google Search}
4. Ask a question with search on and open every cited source. Check that the sentence credited to it is really on the page.⁠^{[4]OpenAI Help Center. Searching the web with ChatGPT}
5. If your site is in Bing Webmaster Tools, look at the grounding queries in AI Performance: which phrases led to retrieving the pages that cited you.⁠^{[5]Microsoft, Webmaster Tools blog. Introducing AI Performance in Bing Webmaster Tools Public Preview}

Related concept: grounding

Google treats RAG and grounding as the same process: its guide defines RAG and adds that it is also known as grounding.⁠^{[3]Google Search Central. Optimizing your website for generative AI features on Google Search} The useful difference is one of focus. RAG names the mechanism, retrieve and then generate; grounding names the result you are after, an answer anchored in specific sources. That is why Bing talks about grounding queries for the searches the AI runs before citing.⁠^{[5]Microsoft, Webmaster Tools blog. Introducing AI Performance in Bing Webmaster Tools Public Preview} For a store, the practical point is the same under both names: your page only counts if it is retrieved, and it is only cited well if each chunk makes sense alone.

To keep going, the LLM (large language model) entry explains what the model does with what is retrieved, and the AI search reading path puts the other terms in order.