What is retrieval-augmented generation?

Give the model evidence it did not learn

An employee asks: “How many days do I have to submit an expense claim?” The company's current policy says 30 days. Last year's policy said 14. A model's fluent recollection cannot tell us which policy this employee may access or which version applies today.

Retrieval-augmented generation, or RAG, brings external evidence into the current model call. After reading this article, you should be able to follow that evidence from a document to a claim—and explain what changes when the document disappears, becomes obsolete, or is no longer authorized.

The original RAG paper combined a trained generator with a retrieved document index. Today's RAG applications use many variations of that pattern. At request time, retrieving a passage normally changes the model's input, while its learned parameters stay fixed. Updating a corpus is distinct from retraining a model.

An illustrative metaphor: the archive stays outside the model; selected fragments enter one reading tray. The image does not show a literal model architecture.

From a question to an evidence packet

Before a question arrives, the application parses documents, splits them into passages, and indexes them for search. Each passage needs provenance: its document, version, location, and access rules. Splitting the deadline from its exception could make an apparently relevant fragment misleading.

At request time, search finds candidates. The application filters for access and freshness, ranks relevance, and assembles a bounded context packet. The model then receives the question, instructions, and selected evidence. Generating an answer and checking its claims are further stages; finding a document does not complete them.

Try the local example below. Remove the current policy first. Then allow archived versions. Finally, deny policy access or switch to an answer that invents a deadline. Watch the selected packet and the support verdict separately.

A citation answers only one of the questions

With the current policy present, the authored answer can cite a passage stating 30 days. With only the archive available, 14 days is supported by that old passage but fails the requirement for a current answer. With no deadline evidence, the grounded template abstains. The invented seven-day answer remains unsupported even when it has a citation.

Those are different failures: missing evidence, obsolete evidence, and a claim that does not follow from evidence. A real model might also ignore a correct passage. Even a faithful answer can be false if the source itself is wrong. Keep provenance, freshness, and claim support visible rather than collapsing them into “the system found something.”

Retrieved documents are also data, not higher-authority instructions. An embedded command cannot grant access or authorize a tool effect. Access checks must happen before restricted material reaches model input; deleting it from the final answer cannot undo that exposure. These are application controls within the broader lifecycle approach described by NIST's AI Risk Management Framework.

Search is a design choice

The demo's exact keyword matching is deliberately small. Lexical retrieval is useful for identifiers and exact phrases. Dense retrieval compares learned vector representations and can find related wording. Hybrid retrieval combines signals; reranking revisits a smaller candidate set with a more expensive relevance method. BEIR evaluates retrieval across diverse tasks and shows why a method should be tested in its intended setting.

There is no requirement to use a vector database. A direct database lookup may be the strongest route to an exact deadline. Use RAG when selecting and interpreting external evidence is useful; choose a simpler authoritative lookup when it answers the product question better.

Inspect the first stage that lost the contract

Save the question, corpus version, authorized evidence, selected passages, generated claims, and citations. If the correct passage was never selected, improve retrieval or ingestion. If it was selected but omitted from context, inspect assembly. If the answer misstates it, inspect generation and claim checking.

RAGAs separates retrieval focus, faithful evidence use, and answer quality. Automated scores still need validation against the cases that matter to your product. Include revoked access, stale versions, missing documents, conflicting passages, and expected abstentions. The useful result explains where evidence was lost and whether the final claim remained justified.