Legal AI Glossary: RAG, Hallucination, Grounding, and 11 More Terms Defined

RAG — retrieval-augmented generation — is the technique behind most trustworthy legal AI. Instead of answering from memory, the tool first retrieves real documents (cases, statutes, your own files) and then writes an answer grounded in what it found. In legal AI, RAG is what lets an answer cite sources that actually exist, because the tool is quoting documents it just pulled rather than inventing citations.

That one idea sits under almost every term in the legal-AI vocabulary, and most of that vocabulary is used to sell you something. This glossary is the plain-English version: fourteen terms, each defined in a sentence or two with a real legal-practice example. Start with the summary table, then read the entries that matter to you.

The 30-second glossary

The building blocks

Large language model (LLM)

An LLM is the prediction engine underneath every AI chatbot. Trained on enormous amounts of text, it works by predicting the next word, which is why its output reads so fluently. The catch for lawyers: an LLM has no built-in, verifiable database of the law. Ask a raw model for a case and it will produce something that looks like a citation, because plausible text is exactly what it is built to generate.

Prompt

A prompt is everything you hand the model — your question plus any context, documents, or instructions you attach. The quality of a legal answer depends heavily on it: a vague "summarize this case" invites a generic response, while "list each element the court required for promissory estoppel, with the paragraph it appears in" constrains the model toward something checkable. The prompt is where a careful lawyer does much of the steering.

Context window

The context window is how much text the model can hold in view at once, measured in tokens (roughly, word-pieces). Paste a 90-page deposition into a tool with a small window and the model may only "see" part of it, then answer as if the rest does not exist. Modern tools handle long documents, but the limit is real — it is why a tool that silently truncates your brief can miss the argument on page 40.

Hallucination

A hallucination is a confident, fluent output that is factually wrong. In legal work the dangerous form is a fabricated authority: a case, statute, or quotation that does not exist or does not say what the tool claims. Stanford researchers found that general-purpose models hallucinate between 58% and 88% of the time on specific legal queries; the studies behind AI-research accuracy explain how those rates were measured and why retrieval tools do better. We cover the mechanics in depth in why AI invents legal citations.

How grounded legal AI works

Retrieval-augmented generation (RAG) — and its legal application

RAG bolts a search step onto the model. Before answering, the system retrieves relevant documents from a real corpus, then asks the model to write its answer using them. In legal AI that corpus is a case-law and statute database, and the payoff is traceability: the answer can point to the exact opinion it relied on. RAG is why the better AI research tools can show their work instead of asking you to trust a black box.

Grounding

Grounding is the property of an answer whose every claim is tied to a specific retrieved source. It is the whole point of RAG, and its failure mode is subtle. Stanford's study of leading commercial legal tools flagged answers that were "misgrounded" — the law was stated correctly, but the citation attached to it did not actually support the proposition. That kind of error survives a quick skim, which is why grounding has to be checked at the source, not assumed.

Embeddings and vector search

An embedding turns a passage of text into a list of numbers that captures its meaning, so two passages about the same idea land near each other in mathematical space. Vector search uses that to find relevant law even when the words differ — a search for "landlord's duty to mitigate" can surface a case that says "reasonable efforts to re-let" without sharing a keyword. It is the engine that lets RAG find the right documents rather than just matching strings.

Retrieval versus generation — the sanctions-relevant distinction

This is the most important line in the glossary. Retrieval means the tool looks a source up and returns it. Generation means the model predicts text that merely sounds correct. A generated citation can be flawless in form and still refer to a case that was never decided. That is the exact failure in Mata v. Avianca, where a lawyer filed ChatGPT-invented cases and drew a $5,000 sanction. The fix is procedural, not magical: verify every citation against a real source before it goes in a brief.

Dual-RAG

Dual-RAG is CaseRead's name for retrieval that searches two libraries in a single query: the public law — statutes, regulations, and case law across all 53 US jurisdictions — and the firm's own matter files. Ordinary RAG searches one corpus. Dual-RAG lets an answer draw on controlling authority and the specific documents in your matter at once, so research starts from your facts, not a blank search box, and stays grounded in both.

Making it smarter, and safer

Fine-tuning versus RAG

Fine-tuning retrains a model's internal weights on example data. It is good for teaching a consistent style or output format — say, your firm's memo structure — but it does not give the model live access to current law; a fine-tuned model still answers from frozen knowledge and can still fabricate citations. RAG instead feeds real documents in at query time. For research and drafting, RAG is usually the safer foundation because the answer traces to sources you can open.

Agentic AI

Agentic AI describes systems that plan and take multiple steps toward a goal rather than answering one prompt and stopping. A legal example: an agent that reads a complaint, identifies the causes of action, retrieves the elements of each, checks your documents for supporting facts, and drafts a response outline. Agents are also compounding — an ungrounded step early on poisons everything downstream, so verification matters more, not less, as autonomy increases.

Human-in-the-loop

Human-in-the-loop means a person reviews and approves the AI's work before it has real-world effect — nothing reaches a court or a client without a lawyer's sign-off. It is the ethical baseline, not a nicety. ABA Formal Opinion 512 tells lawyers they must understand a tool's limits and remain responsible for the output. The full duty is unpacked in our guide to whether lawyers can use AI for legal research.

The privacy vocabulary

Zero data retention and no-training guarantees

Zero data retention means the vendor does not store the text you send after answering the request; a no-training guarantee means your inputs are never used to train the model. For lawyers this is a confidentiality question, not a preference — pasting client facts into a consumer tool that trains on your prompts can implicate your duty under Rule 1.6, the issue we examine in can I upload client documents to ChatGPT. Get these terms in writing before privileged material goes anywhere.

Tenant isolation

Tenant isolation is how a multi-firm platform keeps one firm's data walled off from every other firm's. The strongest form gives each firm its own separate storage rather than mixing everyone's records in one pile and filtering by a permission tag — separation by architecture, not by access rule. It is the difference between "we try to keep your files private" and "another firm's query cannot reach your files because they are not in the same place."

The bottom line

Strip away the jargon and legal AI reduces to one test: does the tool retrieve real sources and ground its answer in them, or does it generate confident text and hope? Everything in this glossary is a different angle on that question. The vocabulary matters because it tells you which tools were built to be verifiable and which were built to be impressive.

Whatever tool drafts your research, run the citations before you file. Our Hallucination Shield checks each citation in any AI-drafted text (up to 25 per run) for existence and support — free, no signup. It is the two-minute habit that turns all of this theory into a filing you can stand behind.

Frequently asked questions

What is RAG in legal AI? RAG stands for retrieval-augmented generation. Instead of answering a legal question from the model's memory, a RAG system first retrieves real documents — cases, statutes, or the firm's own files — and then writes an answer grounded in what it found. In legal AI, RAG is what lets an answer cite sources that actually exist, because the tool is quoting documents it just pulled rather than inventing citations.

What is the difference between retrieval and generation in legal AI? Retrieval means the tool looks a source up in a real database and returns it. Generation means the model predicts text that sounds right based on patterns it learned in training. A generated citation may look perfect and still refer to a case that does not exist. This is the distinction behind every AI-citation sanction: a model generated authorities and no one confirmed they were retrievable.

What does grounding mean in legal AI? Grounding means every claim in an answer is tied to a specific source the tool actually retrieved. A well-grounded answer to a research question links each proposition to the case or statute that supports it. A poorly grounded answer states the law correctly but cites a source that does not support it — a failure Stanford researchers call "misgrounded," and one a lawyer can only catch by reading the cited source.

What is an AI hallucination in a legal context? A legal AI hallucination is a confident, fluent output that is factually wrong — most dangerously, a citation to a case, statute, or quotation that does not exist or does not say what the tool claims. Stanford studies found that general-purpose models hallucinate on the majority of specific legal queries, and even specialized retrieval tools still err often enough that every citation must be verified before filing.

What is Dual-RAG? Dual-RAG is CaseRead's name for retrieval that searches two libraries in one query: the public law (statutes, regulations, and case law) and the firm's own matter files. Ordinary RAG searches one corpus. Dual-RAG lets a research answer draw on controlling authority and the specific documents in your matter at the same time, so the answer is grounded in both the law and your facts.