How LLMs retrieve information to answer questions

Two sources of an answer

When you ask a plain language model a question, it answers from parametric memory — patterns and facts compressed into its weights during training. That knowledge is broad but frozen at the model's training cutoff and impossible to influence after the fact. It can also be vague or out of date on specifics.

Answer engines like ChatGPT Search, Perplexity, and Google's AI Overviews add a second source: live retrieval. At query time they fetch relevant, current content from the web and feed it to the model as context. This both freshens the answer and gives the engine something concrete to cite.

How retrieval works, step by step

The retrieval path is roughly the same across engines, even though implementations differ.

• Interpret the query, sometimes rewriting it into one or more search queries.
• Search an index (its own crawl or a partner's) for candidate documents.
• Pull the most relevant passages — not whole pages — into the model's context.
• Synthesize an answer grounded in those passages.
• Attribute the parts it relied on to their source URLs as citations.

Why passages, not pages, get cited

Retrieval works at the passage level. The engine is looking for the specific chunk that answers the question, not your whole article. A page that buries its answer under a long preamble, or splits it across loosely related paragraphs, gives the retriever nothing clean to grab.

This is the practical reason answer-first writing wins. A self-contained paragraph that states the answer plainly, near a heading that matches the question, is exactly what the retriever is built to find and lift.

What this means for your content

You can't edit a model's training data, but you can shape what it retrieves. Make sure the engine's crawlers can reach and render your pages, write each key answer as a self-contained passage under a question-shaped heading, and back claims with verifiable facts so the engine is confident attributing them to you. Keep content fresh, since retrieval favors current sources for time-sensitive questions.