What is RAG architecture?

RAG architecture is a retrieve-then-generate design, and RAG stands for Retrieval-Augmented Generation. It is what powers how ChatGPT (with web browsing), Claude, Perplexity, and Google AI Overviews answer questions using real web content rather than just training data.

The architecture has four moving parts: an index of the web, a retriever that pulls candidate pages for a query, a ranker that orders them, and a generator that writes the answer from what it was handed. The shape is the same across the engines. The index underneath is not, which is why the same question produces different sources on ChatGPT, Claude, and Gemini. Same architecture, different evidence base, different answer.

The basic idea: instead of relying only on what the model learned during training, the AI retrieves fresh web pages relevant to the query, extracts the useful content, and generates a response grounded in that content. The pages it retrieved become the citations.

This is why AEO is fundamentally different from SEO. Google asks "which pages does the web trust?" RAG asks "which pages best answer this specific query right now?" Different question, different algorithm, different signals.

What are the four stages of the RAG pipeline?

Every RAG-based AI system runs some version of this pipeline. The specific implementation varies between ChatGPT, Perplexity, and Google AI Overviews, but the four-stage structure is consistent according to AgentGEO's 2026 pipeline failure analysis.

At what stage of the RAG pipeline do citation decisions actually occur?

Citation decisions occur at two stages of the RAG pipeline, and they are different decisions. Ranking, stage three, decides whether your page enters the small set the model actually reads. Generation, stage four, decides whether a sentence from that page is quoted and attributed in the answer. Retrieval and parsing decide nothing about citation. They only decide whether your page is still in the running when the two decisions that matter are taken.

The distinction is practical rather than academic. A page can survive ranking, get read, and still not be cited, because the model found nothing in it worth lifting. That is a generation-stage loss, and it looks identical from the outside to never having been retrieved. The two failures need opposite fixes: a ranking loss is a vocabulary problem, and a generation loss is a quotability problem.

One consequence is worth stating plainly. Because the ranking decision is made against the retrieved set for that specific query, and not against the web as a whole, a small page can be cited over a large one whenever its language matches the question more closely. Citation is decided per query, at the moment of the query, from the sources the engine happened to pull.

How does the structure of your content determine whether an answer engine cites it?

The structure of your content determines citation probability by deciding how far down the pipeline your page gets, rather than by making the page better. Each structural property acts on a specific stage, and failing at an early stage makes every later property irrelevant. A page with immaculate sourced statistics that renders as an empty shell to a crawler never reaches the stage where statistics count.

Structural properties, the pipeline stage each acts on, and what it changes

Structural property	Stage it acts on	What it changes about citation probability
Crawler access in robots.txt and at the CDN	Retrieval	Nothing downstream runs if the fetch fails. This is a gate, not a ranking factor
Server-rendered HTML rather than a client-rendered shell	Parsing	A shell has no content to extract, so the page is eliminated before quality is ever assessed
A heading hierarchy that maps to real content sections	Parsing	Gives the extractor the boundaries of each answer unit instead of one undifferentiated block
Article and FAQPage JSON-LD	Parsing	States the structure explicitly rather than leaving it to be inferred from the markup
H2s phrased as buyer questions	Ranking	Matches the query and its sub-queries in the buyer's own words, which is where most pages lose
An opening paragraph that answers the query directly	Ranking and generation	Supplies a passage that can be lifted whole, which is what the generator is looking for
Sentences that stand alone without back-references	Generation	A sentence that needs the paragraph above it cannot be quoted on its own, so it is skipped
Sourced statistics and named quotes	Generation	Gives the model something specific and attributable rather than a claim it has to hedge

Read the table as a sequence rather than a checklist. The properties near the top are cheap, binary, and worth nothing on their own, because passing retrieval and parsing only earns the right to compete. The properties near the bottom decide the outcome but cannot be reached from a page that fails higher up. This is why the fix order below is the whole method, and why auditing a page for quotable sentences before checking whether a crawler can read it wastes the effort.

How is your page split into chunks before retrieval, and why does that decide what gets cited?

Your page is split into chunks because retrieval does not operate on pages. Before anything can be searched, a document is divided into smaller passages, each one embedded separately, and the unit scored against a query is the chunk rather than the whole document. This is the mechanism underneath everything in the section above. When people say an answer engine "read your page", what it read was one or two passages of it, chosen without reference to the rest.

The practical consequence is that a page competes passage by passage. A section that answers one question completely, inside its own heading boundary, can be lifted whole and stands a chance on its own. An answer assembled across three paragraphs with an anecdote in the middle may be split so that no single passage carries the whole thing, and each fragment then looks weaker than the page actually is. The page is not being judged. Its pieces are.

How those pieces are cut is an active research question rather than a settled one. The published work finds that chunk size, the overlap between neighbouring chunks, and where the boundaries fall all change retrieval quality, and that splitting which follows the document's own structure generally beats splitting at a fixed length. See Mix-of-Granularity: Optimize the Chunking Granularity for Retrieval-Augmented Generation (COLING 2025) and the survey of chunking methods at arXiv 2606.00881, which compares the main strategies against their computational cost.

What nobody outside those companies knows is the parameters. No answer engine publishes its chunk size, its overlap, or whether it splits on headings, on sentences or on embedding similarity, and the values differ between engines and change without notice. So there is no chunk length to write to, and anyone selling one is guessing. Treat any specific number you are given about this as unverified.

What does survive the uncertainty is the direction. Every splitting strategy in the literature is trying to find boundaries that coincide with where the meaning changes, and a page whose headings already sit at those points gives the splitter the answer for free. In practice that means the whole answer to a question lives under the heading that asks it, sentences name their subject instead of pointing back at the section above, and a table stays with the caption that explains it. Those are the same instructions as the previous section, seen one level down. The structural advice and the chunking mechanism are the same fact described at two scales.

How much does structure actually change the probability of being cited?

Nobody can give you a citation probability for your page, and anybody who quotes one has made it up. No answer engine publishes the odds it assigns a source, and the retrieval set changes with the query, the engine and the day. What has been measured, on a benchmark rather than on your site, is the size of the change that specific content edits produce, and those numbers are the closest thing to an answer that exists.

Measured effects of content edits on visibility in generated answers, from the GEO benchmark

Edit	Measured effect	What it tells you
Adding relevant statistics	Among the strongest methods tested, up to a 41% improvement over baseline on position-adjusted word count	A specific figure gives the generator something it can repeat without hedging, which is what earns the sentence a place in the answer
Adding direct quotations	Among the strongest methods tested, with a 28% improvement on subjective impression	An attributable quote is easier to carry into an answer than a paraphrase the model has to take responsibility for
Citing authoritative sources	In the same group, contributing to visibility gains of up to 40%	Grounding is read as a property of the passage, not of the site, so it is available to a small domain
Keyword stuffing	No improvement	The optimisations that worked on a ranking system do not transfer to a generative one
Stylistic padding without new substance	No improvement	Length is not the variable. What the passage can be used for is

Effect sizes vary by domain, so treat these as the shape of the effect rather than as a forecast for your page.

The figures come from the study that named this problem. Aggarwal and colleagues built a benchmark of real queries, applied content methods one at a time, and measured what each did to a source's visibility in the generated answer, finding that adding statistics, quotations and cited sources raised visibility by up to 40%, with the best methods improving on baseline by 41% on position-adjusted word count and 28% on subjective impression, while keyword stuffing and stylistic padding did nothing (GEO: Generative Engine Optimization, Aggarwal et al., KDD 2024). The same study found that how well each method works depends on the domain the query sits in, which is the reason to read the table as a direction rather than as a rate card.

Set that against the pipeline and the numbers stop being a list of tactics. Every method that worked acts at the generation stage, on what a retrieved passage can be used for once it is already in the context window. Every method that did not work is an attempt to influence ranking with the vocabulary of an older system. That is the practical reading of the whole page: structure decides whether your passage is retrieved at all, and substance decides whether the generator can do anything with it once it is, and no amount of the second compensates for failing the first.

How much do the engines’ retrieval sets actually overlap?

The engines’ retrieval sets overlap barely at all, and that is the bound on any single answer to the probability question. Profound ran 100,000 identical prompts through ChatGPT and Perplexity and found that 11.0% of cited domains appeared in both, while 37.4% were cited only by ChatGPT and 51.6% only by Perplexity, according to its citation overlap analysis, checked 28 September 2026. Other 2026 datasets put the same pair in the same region, at or below roughly one domain in ten. Perplexity is the one worth treating separately, because it runs its own crawler and its own index rather than borrowing anyone else's, and how to get cited by Perplexity covers the two agents it fetches with and what controls each one.

Where cited domains sat across two engines answering the same 100,000 prompts (Profound)

Where the domain was cited	Share of cited domains	What it means for a publisher
Both ChatGPT and Perplexity	11.0%	The only group for which one engine’s result predicts the other
ChatGPT only	37.4%	Visible on one assistant and absent from the other, on the same question
Perplexity only	51.6%	Same, in the other direction. Neither absence is a verdict on the page

Widen it to four engines and the divergence holds, and the day of the week starts to matter as much as the engine. GetMentions asked the same 2,398 questions of ChatGPT, Google AI Mode, Perplexity and Gemini every day for a week, logged 530,875 citations, and found that 84% of the sources cited for a question were used by only one of the four engines, with 69% of the sources behind a typical answer changing from one day to the next, according to its citation volatility study, checked 28 September 2026. A citation is a sample of one engine on one day, not a position you hold.

That sets the limit on what structure can do, and it is the honest answer to the question this page keeps returning to. Structure decides how a page fares inside a corpus it is already in. It cannot place the page in a corpus that never crawled it, and with overlap at this level the two failures need separating before anything is rewritten. Absence from one engine and presence on another points at index coverage and crawler access, which our guide to checking whether AI can read your website covers. Absence across all of them, on a page the crawlers reach, points at parsing and chunking, which is the pipeline order set out above.

It also decides how the result is read. An average taken across engines is four series wearing one name, and at 11% overlap a gain on one of them predicts almost nothing about the others, so per-engine reporting is a requirement rather than a presentation choice. The same divergence is why a drop needs a threshold before it means anything, which our guide to measuring AI citation decay works through.

What information is passed to search engines by ChatGPT compared with Claude?

Both pass a short search query derived from your request, not the conversation itself, and each passes it to a different index. Neither company documents the internals of its consumer assistant, but both document the developer version of the same tool, and those two pages are the closest thing to a published specification a publisher can read. The table and the paragraphs below are taken from them rather than inferred.

Anthropic's documentation describes a search query that is visible rather than hidden: the query Claude runs is returned in its own block with a single query field, and the worked example in the documentation turns a request about Claude Shannon's birth into the four-word search "claude shannon birth date" (Anthropic, web search tool documentation, read 21 October 2026). OpenAI describes the same step in one line for the simpler case: "The non-reasoning model sends the user's query to the web search tool, which returns the response based on top results" (OpenAI, web search guide, read 21 October 2026). Either way a few terms go to the index, not the paragraph a person typed.

What each vendor documents about its web search tool, read 21 October 2026

What a publisher wants to know	OpenAI, web search guide	Anthropic, web search tool
What reaches the index	The model sends the user's query to the search tool; the queries that were run can be returned with the response	A short generated query, returned in the response as a query field
What comes back per result	Top results, plus the full list of URLs retrieved if the caller asks for it	URL, title, page_age, defined as "when the site was last updated", and the page content
Whether everything retrieved is read	No: the sources field returns every URL retrieved, and the documentation says "the number of sources is often greater than the number of citations"	With basic search every result enters the context window; from web_search_20260209 the model can write code that filters results before they get there
What a citation carries	A url_citation with the URL, the title and the location in the answer; the documentation says inline citations show "only the most relevant references"	URL, title and cited_text, documented as "up to 150 characters of the cited content". Citations "are always enabled for web search"

Both rows describe each vendor's developer tool, which is the documented version of the retrieval step. The consumer assistants are not specified in the same detail.

Where the query lands differs by assistant, and this page's index question below sets that out: Claude runs its web search on Brave, ChatGPT uses its own crawler and falls back to Bing, and Gemini and Google AI Overviews read Google's index. The Brave connection is known from Anthropic's subprocessor list, where Brave Search was added in March 2025 rather than announced as a partnership (TechCrunch, 21 March 2025). Same question, three different corpora, which is why a page cited by one assistant can be absent from another for reasons that have nothing to do with the page.

What do ChatGPT's and Claude's documented search fields change about how you write a page?

The documented search fields change three things about how a page should be written, and the first is the one most publishers get wrong. Being retrieved and being cited are separate events, and OpenAI documents the gap: every URL the model pulled is available in one field, the citations shown to the reader are a subset of it, and the subset is usually smaller. A page can be fetched, read and used to compose an answer without its name appearing anywhere, which is the mechanism behind the question of whether an AI citation means your page was actually used, now with a vendor's own documentation behind it.

The second is the length of what gets lifted. Anthropic caps the quoted span it returns with a citation at 150 characters of the cited content, which is roughly one sentence. Whatever the consumer assistants do, that is the unit a retrieval system in this family is built to carry: a sentence that survives on its own. A claim that needs the two sentences before it to make sense does not fit in the slot, which is the same instruction as the chunking section above arriving from a different direction.

The third is that a last-updated date travels with the result. Claude's search results carry page_age before the model reads a word of the page. Anthropic does not say how, or whether, that field is weighed, so nobody should claim it is a ranking factor. What it does establish is that the age of a page is in front of the model as metadata rather than something it has to infer from the text, and that an inaccurate visible date is therefore contradicted by a value the model already holds.

What the documentation adds to all three is not a new tactic but a field name. The instructions were already readable off the pipeline. They are now readable off a vendor's own schema, which is a harder thing for a buyer to argue with.

The consequence for retrieval is the one most publishers miss. Your page is compared against the rewritten query, not against the prompt a person typed. A long conversational request gets compressed into a handful of terms before any page is scored, so writing a page to match some exact long-form phrasing optimises for a string the retrieval stage never sees. What survives the compression is the intent and its vocabulary, which is why an H2 phrased as the question a person would ask out loud tends to match, while a heading built from a keyword string tends not to.

A second consequence is worth holding on to. Because the rewrite happens before retrieval and the user does not see it happen, two people asking the same thing in very different words can be handed the same sources, and one person asking twice can be handed different ones. The rewrite is a step in the pipeline like any other, and it varies.

What are the limitations of RAG in models like ChatGPT and Claude?

The limitations of RAG in ChatGPT and Claude all come from one place: each assistant is held to what its index already contains and to what a single retrieval pass happens to surface, which is a much narrower window than the web. Four limits follow from the four stages above, and each one explains a failure that looks like something else from the outside.

The first is index coverage. A page that is not in the corpus cannot be cited however well it answers the question, and because the assistants read different corpora, being citable on one says nothing about the others. The second is that retrieval runs before the model reasons about your question. The model cannot go looking for a page it did not retrieve, so a page that loses at ranking is not weighed and found wanting. It is never seen.

The third is the size of the read set. Retrieval returns more sources than the model reads closely, so surviving retrieval earns consideration rather than a citation. The fourth is the one that matters for anyone relying on these answers: the model composes the answer from what it read together with what it already holds, so a citation beside a sentence is not proof that the cited page states that sentence. Checking a claim means opening the source, not counting the footnotes.

These limits are also why a single answer is poor evidence about your own visibility. The same question, asked again, runs a different rewrite against a partly different retrieved set, so the roster moves. Anything you want to conclude about whether an engine cites you has to come from repeated runs rather than from one screenshot, which is the method set out in how many runs a visibility measurement needs.

Does ChatGPT use RAG when web search is off?

No, because with web search off nothing is retrieved: ChatGPT answers from its trained parameters alone, and nothing is fetched or cited. RAG is not a property of ChatGPT or Claude as products. It is what happens on the subset of turns where the assistant decides to go and look, and on every other turn the pipeline on this page does not run at all.

Anthropic publishes the criteria its own tool uses, which is the only vendor statement of this we have found in writing. Claude searches when the request depends on information that is "current, changing, or outside its training data", and the documentation lists recent events and announcements, current prices, rates, scores or statistics, information about specific organisations, people or products that might have changed, and explicit requests to look something up. It answers directly, without searching, on what the documentation calls stable knowledge: established facts, maths and science fundamentals, coding concepts, creative writing and brainstorming, analysis of content already supplied in the conversation, and conversational turns (Anthropic, web search tool documentation, read 21 October 2026). The same page notes that the behaviour is steerable by the system prompt and cappable with a max_uses limit, so the split is a configuration rather than a fixed law.

Two consequences follow for anyone trying to be cited, and they pull in opposite directions from the rest of this page.

The first is that your page only competes on prompts that trigger a search. A question the assistant treats as settled knowledge is answered from memory, and no amount of structure, grounding or crawler access earns a citation on it, because no retrieval ever happens. That is a different failure from losing at ranking, it is invisible in exactly the same way, and it is the mechanism behind the problem of whether AEO works when nobody is searching your category.

The second is a measurement instruction. When you run a prompt set against an assistant, record whether it searched, not only what it cited. A prompt answered without retrieval tells you nothing about your page and should not be counted as a miss against it, and a prompt set made mostly of such prompts measures the model's memory rather than your visibility. That is one of the reasons a single run is weak evidence, which how many prompts a visibility measurement needs works through.

How the SIGNALS framework maps to the pipeline

Each of the 7 SIGNALS dimensions targets a specific stage of the RAG pipeline. This is why fixing the right dimension matters — there's no point improving Generation-stage signals if your page is failing at Parsing.

In what order should you fix RAG pipeline failures?

Fix RAG pipeline failures in pipeline order: retrieval, then parsing, then ranking, then generation. The most common mistake in AEO is optimizing the wrong stage. Businesses spend weeks improving their content quality (Generation stage) while their pages are failing at Parsing because they're built on a JavaScript SPA that renders an empty shell to crawlers.

The SIGNALS framework diagnoses which stage is failing first, so optimization happens in priority order. There's no point adding sourced statistics (Generation fix) to a page that AI systems can't extract content from (Parsing failure).

The correct order: fix Retrieval (accessibility) → fix Parsing (structure, SSR) → fix Ranking (vocabulary, FAQ, headings) → fix Generation (statistics, quotable claims). Most pages need work at Ranking — that's where vocabulary alignment lives, and it's the highest-leverage fix available.

Each stage has a cheap test that tells you whether to move on. For retrieval, fetch the page the way a crawler would and confirm the bots are allowed in robots.txt and at the CDN. For parsing, view the page source rather than the rendered page and check that the body copy and headings are in the HTML that arrives. For ranking, read your own H2s and ask whether a buyer would type any of them. For generation, take any sentence out of the page and ask whether it still stands up alone, without the paragraph above it. A page that passes all four is not guaranteed a citation, but it is in the running for one, which the majority of pages are not.

Frequently asked questions