Hallucinations and Context Problems: A Production Mitigation Playbook

September 25, 2026Michał Woliński21 min read

Every team that ships an LLM feature has the same meeting. Someone reports that the model "made something up". The room agrees this is bad. Someone proposes adding retrieval. The proposal is accepted, the ticket is closed, and three weeks later the model is making things up again — but now the fabricated answer comes with a citation, which is worse, because it looks verified.

The reason the loop repeats is that "hallucination" is treated as one bug with one fix. It is not. It is a family of failures that happen at different layers of the pipeline, and adding retrieval addresses exactly one of them. The same is true of "context problems", which is really two different problems that get merged in conversation: the information you needed was never in the window, or it was in the window and the model did not use it. Those have different fixes, and the second one gets worse when you try to fix the first one naively by adding more text.

This article is a mitigation playbook. Not a survey of the hallucination literature, not a philosophical discussion of whether models "know" things. The engineering questions: what are the distinct causes, which layer does each one live in, what actually moves the number, and — the part most articles skip — how do you design the system so that the hallucinations you cannot prevent are cheap when they happen.

It assumes you have a production LLM feature: a retrieval pipeline, a summarizer, an extraction job, a support assistant. Code samples are Python, since that is where LLM pipelines live; the architecture is language-agnostic. Where a fix is measured rather than argued, I point at the measurement instead of re-deriving it — evaluating the result is its own discipline, covered in the companion piece on measuring the quality of an AI system.

Hallucination is not one bug

Start by refusing the single label, because the remediation depends entirely on which failure you are looking at. In production I have seen five, and they are distinguishable by where they occur:

Knowledge gap. The fact is not in the model's weights and was never in the context. The model has nothing to retrieve and nothing to recall, so it produces the most plausible continuation. This is a coverage problem, and no prompt engineering fixes it.
Retrieval gap. The fact exists in your corpus, but the retriever did not surface it for this query. The model then does its best with the wrong documents. This is a retrieval-quality problem — the fix lives in search, not in the model.
Utilization gap. The right document was in the context, in the top results, and the model still produced something contradicted by it. The information was available and unused. This is the failure people misdiagnose as a model limitation when it is usually an assembly or ordering problem.
Synthesis error. The pieces were all present and the model combined them incorrectly: the right numbers, the wrong arithmetic; the right clauses, the wrong logic; a temporal ordering inverted. Each individual retrieval was fine. The composition is wrong.
Compliance failure. The model had the answer, knew the answer, and produced an unsupported one anyway because the format, the instruction, or the incentive pointed that way. It answered a question it should have declined because nothing in the setup told it that abstaining was allowed.

Read that list and notice that "add RAG" only addresses the second one, partially the first, and changes nothing about the fourth and fifth. A team that reaches for retrieval to fix a synthesis error will spend a quarter improving recall and watch the hallucination rate stay flat. Diagnosis first.

There is also a framing worth stating plainly: the machinery that fabricates is the same machinery that generalizes, summarizes, and paraphrases. A model that could only ever emit verbatim, verified text would be useless for the tasks you actually want. So "solving hallucination" does not mean eliminating fluent generation. It means aligning the system's willingness to assert with the cost of being wrong, in each specific context. A brainstorming tool and a medical summary have the same model and opposite requirements.

Coverage versus utilization

The two context problems need separate names, because conflating them produces the wrong fix.

Coverage is whether the information needed to answer is present in the window at all. If it is not, the model is being asked to answer from memory, and the failure is upstream: the corpus is missing the document, the chunking destroyed the relevant passage, the query never matched the right chunk, or the reranker pushed the right chunk below the cutoff.

Utilization is whether the model uses the information that is present. A model can have the exact sentence in front of it and still answer from parametric memory, especially when the retrieved passage is buried in the middle of a long context.

There is a third property that makes both harder: degradation. Adding context is not neutral. The assumption that a model processes the 10,000th token as reliably as the 100th does not hold — Chroma's 2025 report on what they call context rot evaluated eighteen models and found performance varying significantly with input length even on simple tasks, well before any window limit is reached. The Lost in the Middle result is the sharpest single illustration: when the relevant document sits at the beginning or the end of a long context, models find it; when it sits in the middle, accuracy drops, and it drops more as the context grows. Position is a variable, not a detail.

And the advertised window is not the usable window. NoLiMa tested models that claim 128K-plus contexts on questions deliberately written with minimal lexical overlap with their answers — the realistic case, where the user's phrasing and the document's phrasing do not match. Performance falls off well before the advertised limit, and much faster than on the synthetic needle tests where the question and the needle share keywords. Both findings point the same way: effective context is smaller than advertised context, and the gap widens as the task gets harder.

This is why the naive fix to a coverage problem can make utilization worse. You cannot retrieve your way out of a context problem if the cure is "put four times as much text in front of the model".

How context fails in practice

Drew Breunig's taxonomy of context failures is the most useful practical framing I have found, because each mode has a different symptom and a different fix. Four of them:

Context poisoning. A hallucination or an error enters the context and is then referenced repeatedly, so the error compounds. An agent invents a customer id in turn two, reads it back in turn three, and by turn six that id is load-bearing. This is the failure mode of long-running conversations and multi-step agents, and it is why an error early in a session is more expensive than the same error at the end.

Context distraction. The context grows past the point where the model pays attention to all of it, and it starts over-fitting to what it has seen most recently or most often. The information is present; the attention is not. This is the failure described by the long-context research above, and the fix is subtraction, not addition: fewer, better documents.

Context confusion. Superfluous content influences the output. A tool definition the task does not need, a format example from a different workflow, an old draft still in the history — none is needed, all are read. The model does not know which parts were meant for it.

Context clash. Two pieces of information conflict and the model resolves the conflict arbitrarily. Two retrieval chunks, one stale and one current. A system prompt that says one thing and a retrieved policy document that says another. The model picks one, with no signal about which is authoritative — so you get the wrong answer half the time and have no way to predict which half.

Each of these points at a specific discipline: keep early errors out of the long-lived context, keep the window tight, keep the window relevant, and make authority explicit when sources conflict. The next three sections are the concrete versions of those.

Fix the coverage first

Coverage is a search problem, and the standard retriever is weaker than most teams assume. A pure dense (embedding) retriever is good at semantic similarity and bad at exact tokens — and the tokens that matter most are exactly the ones embeddings fuzz: error codes, SKUs, product names, identifiers, quoted phrases. Ask a dense index for the reason for error ORA-01555 and it will happily return documents about database errors in general.

The baseline that holds up is hybrid retrieval: dense plus lexical (BM25 or equivalent), merged by rank rather than by score. Reciprocal Rank Fusion is the standard merge and it is a few lines, because it works on positions instead of scores — which means a BM25 score of 12 and a cosine similarity of 0.71 can be combined without normalising either, a step that is otherwise fiddly and fragile.

from dataclasses import dataclass


@dataclass
class Document:
    id: str
    text: str
    metadata: dict[str, str]


def reciprocal_rank_fusion(rankings: list[list[str]], k: int = 60) -> list[str]:
    """Merge ranked id lists (dense + lexical) into one ranking.

    RRF scores by position, not by raw score, so a BM25 score and a
    cosine similarity can be fused without normalising either.
    """
    scores: dict[str, float] = {}
    for ranking in rankings:
        for rank, doc_id in enumerate(ranking, start=1):
            scores[doc_id] = scores.get(doc_id, 0.0) + 1.0 / (k + rank)
    return sorted(scores, key=scores.get, reverse=True)


def retrieve(query, dense_index, lexical_index, reranker,
             top_k: int = 4, candidates: int = 50) -> list[str]:
    dense_ids = dense_index.search(query, limit=candidates)
    lexical_ids = lexical_index.search(query, limit=candidates)
    fused_ids = reciprocal_rank_fusion([dense_ids, lexical_ids])[:candidates]

    # Retrieve wide and vague with the bi-encoder, then rerank narrow
    # and precise. A cross-encoder scores each (query, document) pair
    # jointly and reads both in full, which a bi-encoder cannot do.
    scored = reranker.score(query, fused_ids)
    ranked = sorted(scored, key=lambda pair: pair[1], reverse=True)

    # Four good documents beat twenty mediocre ones. The cap is a
    # quality control, not a cost optimisation.
    return [doc_id for doc_id, _score in ranked[:top_k]]

The two-stage shape — retrieve wide with a fast model, rerank with a slow precise one — is worth more than any embedding-model upgrade I have seen applied. Recall comes from the candidate pool; precision comes from the reranker; and precision is what determines whether the four documents in the window are the four that matter.

Three more coverage levers, in rough order of payoff:

Chunking, which is where most coverage is lost. A fixed-size splitter that cuts mid-sentence and strips the heading away produces chunks that are individually unretrievable, because the sentence that answers the question no longer sits next to the words that identify the topic. Structure-aware chunking keeps headings with their sections; overlap preserves sentences that straddle a boundary; and a parent-document (small-to-big) retriever indexes small chunks for precision but hands the model the larger surrounding passage for context. The right question is not "what chunk size" but "what is the smallest unit that is still self-contained".

Query understanding. A multi-hop question — "did we exceed the data-retention limit on the EU tenants" — is two questions, and a single embedding of the compound query will match neither well. Decompose it and retrieve per sub-question, then merge. Where the user's phrasing is far from the corpus's phrasing, generating a hypothetical answer document and embedding that instead of the raw query — the HyDE trick — reliably retrieves better than the query itself, because a plausible answer looks more like the real document than the question does.

An honest empty result. If reranking produces nothing above a threshold, the correct pipeline output is "I do not have this in the sources", not a generation call. This is the single highest-leverage line of code in the whole pipeline, and it is the one teams skip, because an empty result feels like a broken feature. An abstention is a working feature. A confident fabrication in its place is not.

Grounding the generation

Coverage decides what the model could use. Grounding decides what it is allowed to say. The pattern that works is not "ask the model to be accurate" — that is a wish, not a mechanism. It is to make the answer's shape enforce the constraint.

Extract, then answer. When the output is a set of fields, do not ask for prose and parse it. Ask for the fields, with a schema, and validate the schema. A closed answer space (a category, an enum, a yes/no with a citation) is a much stronger constraint than any amount of "do not hallucinate" in the prompt, because the model cannot produce a plausible invention outside the space it is given.

Citation-first, claim by claim. Require each sentence to carry the id of the evidence it came from, and then check that every cited id actually exists in the evidence you passed in. This catches a specific and common failure — the fabricated citation — for almost nothing. It is the difference between a hallucination that looks verified and one that is caught by a set comparison.

import re

NOT_FOUND = "NOT_FOUND"

ANSWER_CONTRACT = """Answer using ONLY the evidence inside the context block.
Append the evidence id to every sentence, like [doc-3].
If the evidence does not contain the answer, reply exactly: NOT_FOUND.
Do not use prior knowledge. Do not guess."""


def answer_question(model, question: str, evidence: list[Document]) -> str:
    # Evidence first, instruction and question last. Attention is uneven
    # across a long context; the instruction sits at the strongest edge.
    context = "\n\n".join(f"[{d.id}] {d.text}" for d in evidence)
    prompt = (
        f"<context>\n{context}\n</context>\n\n"
        f"{ANSWER_CONTRACT}\n\nQ: {question}"
    )

    raw = model.complete(prompt)

    if raw.strip() == NOT_FOUND:
        return NOT_FOUND

    if not citations_resolve(raw, evidence):
        # An id that is not in the evidence is a fabricated citation -
        # treat the whole answer as unverified rather than partially so.
        return NOT_FOUND

    return raw


def citations_resolve(answer: str, evidence: list[Document]) -> bool:
    known = {d.id for d in evidence}
    cited = set(re.findall(r"\[(doc-\d+)\]", answer))
    return bool(cited) and cited <= known

Make abstention a first-class, non-penalised output. This is the part that is cultural as much as technical. Models are trained to be helpful and to answer; if the only two outcomes your product recognises are "correct" and "wrong", then declining is scored as wrong and the system is incentivised to guess. Give abstention its own output value, its own UI treatment, and its own place in evaluation, and users get a system that says "not in the sources" instead of inventing. The mechanism matters too: telling the model that NOT_FOUND is a correct answer is not decoration, it changes the distribution of what it emits.

Verify before it crosses a boundary. A generated string that only a human reads has a small blast radius. A generated string that writes to a system of record, sends an email, or updates a number has a large one. Between the model and that boundary, put a check — schema plus a deterministic reconciliation wherever it exists. The discipline is the same one that applies to agent tool calls: the model produces a candidate, and something that is not the model decides whether it is allowed to take effect.

Context assembly and ordering

Assembly is the layer where a coverage win turns into a utilization loss if you are careless, because the obvious implementation — concatenate everything that scored above the threshold — is precisely what degrades attention.

Three rules do most of the work. Cap the count. Four well-reranked documents beat twenty marginally relevant ones; the extra sixteen are distraction with a token bill. Deduplicate before you cap, not after. Near-identical chunks are routine when retrieval returns overlapping passages, and they are the most expensive kind of noise, because they consume attention without adding information. Slicing to the top N first and deduplicating that slice discards a distinct, relevant candidate at position five whenever two of the top four are duplicates — the cap has to apply to the pool you actually intend to draw from. Use both edges of the window. Attention is weakest in the middle of a long context, so the strongest evidence goes first and the second-strongest goes last, closest to where the model starts generating. Strictly descending order puts the weakest document in that second-strong position, which is the opposite of what the long-context research implies.

def order_for_attention(docs: list[Document]) -> list[Document]:
    """Put the strongest evidence at both edges of the window.

    Attention is weakest in the middle of a long context, so both ends
    are prime real estate. Best first, second-best last, the rest in
    between: 1, 3, 4, 2 - not 1, 2, 3, 4, which would park the weakest
    document in the second-strong position.
    """
    if len(docs) <= 2:
        return list(docs)
    head, second, *rest = docs
    return [head, *rest, second]


def assemble_context(
    evidence: list[Document],
    max_docs: int = 4,
    max_chars: int = 12_000,
) -> str:
    """Assemble the window deliberately instead of concatenating results.

    Three rules matter more than they look:
      1. Draw from the whole candidate pool and cap afterwards.
         Capping first throws away a good candidate whenever the top
         slice contains near-duplicates, which retrieval returns
         routinely when chunks overlap.
      2. Deduplicate - duplicates cost attention and add nothing.
      3. Order for attention, keeping both edges of the window strong.

    Both limits - document count and character budget - skip the
    candidate that does not fit and keep scanning. Neither surrenders
    the rest of the pool, because document lengths vary and a smaller
    relevant passage sitting behind an oversized one is still worth
    including. Giving up early is how a window that had coverage ends
    up empty.
    """
    seen: set[str] = set()
    chosen: list[Document] = []
    used = 0

    for doc in evidence:                       # the full pool, not a slice
        if len(chosen) >= max_docs:
            break
        fingerprint = doc.text[:200]
        if fingerprint in seen:                # skip near-duplicates
            continue
        if used + len(doc.text) > max_chars:
            # Skip this candidate, keep looking. Breaking here assumes
            # that one oversized document means every later one is too
            # large - false whenever lengths differ, and it returns an
            # empty window when only the top-ranked candidate is big.
            continue
        seen.add(fingerprint)
        chosen.append(doc)
        used += len(doc.text)

    return "\n\n".join(
        f"[{doc.id}] {doc.text}" for doc in order_for_attention(chosen)
    )

Two further levers when the evidence genuinely does not fit. Compress rather than truncate. Summarising the retrieved passages down to what bears on the question keeps the signal and removes the noise; truncating a chunk simply deletes text, which is how you lose the one clause that mattered. Retrieve for the question, not for the topic. The most common cause of a bloated context is a retriever that was tuned to return everything related to the subject when the task needed the two documents that answer the specific question.

And a note on the long-context temptation: with a million-token window available, stuffing the whole corpus looks like it removes the retrieval problem. It does not. It converts a retrieval problem into a utilization problem, at a higher price and with worse latency, which is exactly the trade-off the companion piece on choosing between RAG and simple search works through. Long context is a real capability; it is not a substitute for selection.

Detecting hallucinations

Prevention lowers the rate. Detection is how you stop the remainder from reaching a user. The key framing: detectors are probabilistic, so they do not prove correctness — they route. A detection signal should trigger one of three actions: retry with better retrieval, escalate to a human, or degrade gracefully to an abstention. Treating a detector as a correctness proof is how teams end up shipping a second model's hallucinations as a verdict.

Self-consistency. Sample the same question several times with a non-zero temperature and compare the answers. A model that is guessing produces varied output; a model that knows produces the same answer repeatedly. The comparison has to be semantic, and this is the detail that decides whether the technique works at all: at a non-zero temperature a correct answer is rephrased every time — "The refund was approved." and "Approval for the refund was granted." are the same answer and two different strings. An equality check on the raw text would flag a consistent model as inconsistent on nearly every prose answer, which is a false alarm that destroys the signal. Compare with an entailment model, an embedding-similarity threshold, or a small judge; anything that treats paraphrase as agreement. Even done right, it catches only the "filling a gap" case — an answer the model holds uniformly wrong passes — so it is a filter, not a gate. Two implementation choices shape the result: take the judging model from a different family than the system under test, or the check becomes the model grading its own output — the same trap the companion piece on measuring quality describes for judge-based scoring — and comparing each sample against the first makes the verdict sensitive to which sample landed first, which is a deliberate saving against the cost of comparing every pair.

Claim-level verification, run independently. The strongest published pattern here is Chain-of-Verification: draft an answer, generate verification questions for its factual claims, answer those questions, and only then produce the final response. The mechanism that makes it work is easy to get wrong in implementation: the verification questions must be answered without the draft in the prompt. If you show the model its own draft while asking whether the draft is true, it tends to agree with itself, and you have built an expensive rubber stamp.

from typing import Callable


def make_semantic_equivalence(model) -> Callable[[str, str], bool]:
    """Agreement judged by meaning, not by string equality.

    An NLI model, an embedding cosine threshold, or a small judge all
    work; the only requirement is that a paraphrase counts as the same
    answer. Swap this for embeddings if the judge cost matters.
    """
    def equivalent(a: str, b: str) -> bool:
        verdict = model.complete(
            "Do these two answers state the same thing? They may be "
            "worded differently. Reply EQUIVALENT or DIFFERENT.\n\n"
            f"A: {a}\nB: {b}"
        ).strip().upper()
        return verdict.startswith("EQUIVALENT")

    return equivalent


def is_self_consistent(
    model,
    prompt: str,
    runs: int = 5,
    equivalent: Callable[[str, str], bool] | None = None,
) -> tuple[bool, list[str]]:
    """Cheap filter for confabulation: a guess rarely survives sampling.

    Answers are compared semantically. At a non-zero temperature a
    correct answer is rephrased every run, so an == check would flag
    correct answers constantly - a false alarm on nearly every prose
    response.

    Three failure directions, not two. False positives when the
    equivalence check is too strict (hence the pluggable comparison).
    False negatives when the model is uniformly wrong. And a
    sensitivity to WHICH sample is the reference: every answer is
    compared against answers[0], so when the opening sample happens to
    be the outlier and the rest agree, this reports inconsistent. That
    is a deliberate cost trade - k-1 comparisons instead of k^2 - but
    it is worth knowing which sample you anchored on when a verdict
    looks wrong.

    The default equivalence wires the comparison back to the same
    client that produced the answers. That is convenient, and it is the
    same-model-as-judge pattern; for anything load-bearing, pass a
    judge from a different model family than the one under test.
    """
    if equivalent is None:
        equivalent = make_semantic_equivalence(model)

    answers = [model.complete(prompt, temperature=0.7) for _ in range(runs)]
    first = answers[0]
    for other in answers[1:]:
        if not equivalent(first, other):
            return False, answers
    return True, answers


def verify_claims(model, draft: str, evidence: list[Document]) -> list[dict]:
    """Decompose the draft into claims and check each against evidence.

    The load-bearing detail: the verification prompt contains the
    evidence and the question, and deliberately NOT the draft. Show a
    model its own draft while asking "is this true?" and it agrees.
    """
    planned = model.complete(
        "List the atomic factual claims in this draft as standalone "
        "questions, one per line, with no preamble.\n\n" + draft
    ).splitlines()

    evidence_block = "\n\n".join(f"[{d.id}] {d.text}" for d in evidence)
    checks: list[dict] = []

    for question in (q.strip() for q in planned):
        if not question:
            continue
        verdict = model.complete(
            f"<evidence>\n{evidence_block}\n</evidence>\n\n"
            f"Using only the evidence, answer this question: {question}\n"
            "Reply with exactly one of: SUPPORTED, CONTRADICTED, UNSUPPORTED."
        ).strip()
        checks.append({"claim": question, "verdict": verdict})

    return checks

Deterministic checks, wherever a fact is checkable. The most reliable detectors are not models at all. Does the cited span exist, and does it actually contain the claim. Do the numbers in the answer reconcile with the numbers in the source. Do the entity names in the output appear in the retrieved set. For extraction tasks, does the extracted value match a checksum or a second extraction path. Every check you can move from a model to code is a check that stops drifting when a provider ships a model update.

Source-level signals. Retrieval scores, the gap between the top two candidates, and whether the answer's claims spread evenly across the evidence or all lean on one weak chunk are all useful. A confident answer supported entirely by the lowest-ranked document is a suspicious shape, and it costs nothing to flag. Citation quality itself is measurable — the ALCE benchmark for citation generation is the reference point here — and it is reasonable to gate on citation correctness before gating on answer correctness, because a fabricated citation is unambiguous while a subtly wrong sentence is not.

Conversation context is a different problem

Multi-turn conversations fail differently from single-shot retrieval, and the difference is structural: the context is not assembled fresh each turn, it accumulates. That is where poisoning compounds, and where a fourth failure shows up that single-shot systems never hit — constraint amnesia. The user states a requirement in turn one ("amounts in EUR, no vendors outside the EU"), and twenty turns later the model proposes a USD figure from a US vendor, because the constraint has scrolled out of the verbatim window or been buried by everything since.

The fix is not a bigger window, and it is not summarisation alone — a rolling summary is a lossy compression of exactly the constraints you most need to keep. The fix is to move durable state out of the conversation and into the application, where it can be pinned:

conversation_state:
  pinned_constraints:          # re-injected verbatim on every turn
    - "Currency: EUR only"
    - "No vendors outside the EU"
    - "Budget ceiling: 5000 EUR"
  task:
    status: awaiting_customer_confirmation
    order_id: "ORD-4471"
  rolling_summary: >
    Customer asked for a refund on ORD-4471. The agent confirmed the
    order shipped late. Refund approved pending bank confirmation.
  verbatim_window: 6           # last N turns kept word-for-word

Three properties make this work. Constraints are pinned — they survive summarisation because they are not in the summary, they are re-injected every turn. Task state is structured — an enum and an identifier, not a sentence, so the app can act on it and the model cannot drift it. And the summary is bounded — it compresses narrative, which is where compression is safe, while facts that must not change live outside it.

Two related disciplines. First, a correction from the user should overwrite the stale value, not append to it; leaving both in context produces a clash the model resolves arbitrarily. Second, treat anything written to long-term memory as untrusted input on the way back in — memory written in one session is read as trusted context in the next, which makes it an injection sink as well as a drift risk, a point the agent permissions piece covers in detail. Structured, schema-validated memory records, never free-form natural language.

Containment: when it still happens

You will not drive the rate to zero, so the last layer is blast radius. This is the part that separates a system that has an incident from a system that has an outage.

Never let generated text write to state directly. Route it through validation first: schema, then a range or reconciliation check where one exists, then the write. The model proposes; deterministic code disposes.

Match the human in the loop to the cost of being wrong. Draft generation, summarisation for review, internal search — the user corrects, so automation is fine. Customer-facing claims, financial figures, medical or legal content, anything irreversible — a human confirms before it takes effect. The rule is not "high stakes gets a human"; it is "the cost of a wrong answer, times the probability, against the cost of the review step".

Show the evidence, and make it clickable. A user who can see which document a claim came from catches fabrications your detectors missed, at zero inference cost. Progressive disclosure — answer first, sources one click away — gets the verification benefit without making every interaction a reading exercise.

Make the consequences reversible. Where the action cannot be undone, require confirmation; where it can, make the undo obvious. Idempotency and soft deletes are hallucination mitigations when the failure mode is "the agent wrote the wrong row".

Put together, the layers spell a single sequence — prevent, detect, contain, observe. Retrieval and grounding reduce the rate. Verification and routing catch what gets through. Containment bounds the cost of what is not caught. And observation — the evaluation harness, the sampled human review, the drift alarm — is what tells you whether any of it is working. That last layer is its own discipline and its own article; here it is enough to say that a mitigation with no measurement is a belief.

When none of this fits

This framework assumes the task has a ground truth and the evidence exists somewhere. Where that is false, the playbook above is not just unhelpful, it is actively misleading, because it pushes effort into the wrong layer.

Genuinely generative tasks have no faithfulness to measure. Brainstorming, naming, drafting fiction, exploring options — there is no evidence to ground against and no claim to verify. Grounding constraints here produce bland, hedged output that fails the actual task. The right instrument is acceptance: did the human use it, how much did they change it. Different measurement, different mitigation, and no retrieval.

When coverage does not exist, retrieval makes things worse. If the answer is not in your corpus, adding retrieval does not create it; it surfaces the nearest irrelevant passage and hands the model a licence to compose an answer that looks sourced. The honest fix is coverage — ingest the source, or connect the system that has it — or an explicit "not in our data" path. A retrieval pipeline over a corpus that does not contain the answer is a fabrication machine with a citation feature.

Premise errors defeat retrieval entirely. "Why was my March invoice higher than my February one" presupposes a difference that may not exist. Retrieval finds nothing useful because there is nothing to find, and a compliant model confirms the premise and invents a reason — the failure the SelfAware work documents, where models answer questions that have no answer rather than say so. The mitigation is a verification step before generation: check that the entity exists and that the premise holds, and if it does not, say so. Answering the question as asked is the trap.

Adversarial context is a security problem, not an accuracy problem. When the retrieved content is attacker-controlled, the goal is not faithfulness — the model is being faithful to the wrong principal. That is prompt injection, it has its own architecture, and grounding techniques do not defend against it.

And when the verification budget does not exist, say so. Claim-level verification costs tokens and latency; if the product cannot afford either, then the mitigation is containment, not prevention. Running the cheap layers — hybrid retrieval, capped context, schema-constrained output, citation resolution — and skipping the expensive ones is a coherent position. Pretending the expensive layer is happening is not.

Which is the honest summary of the whole article: for open-ended generation you do not eliminate hallucination, you reduce its frequency and bound its cost. The verb "solve" belongs to the tasks with a ground truth and the evidence in hand. For everything else, the engineering goal is a system where being wrong is survivable — and where being wrong is visible.

Summary

Hallucination is not one bug and has no single fix. Diagnose which of the five causes you are looking at — knowledge gap, retrieval gap, utilization gap, synthesis error, compliance failure — because each lives in a different layer and responds to a different remedy. Adding retrieval addresses one of them.

Context problems are two problems, coverage and utilization, plus a third property that makes both harder: adding context degrades attention. Effective context is smaller than advertised, attention is weakest in the middle, and the gap widens as the task gets harder — so the fix for a coverage problem can easily worsen utilization.

The layers, in order: hybrid retrieval with reranking, honest empty results, and chunking that keeps chunks self-contained; grounding by construction — extract rather than compose, cite and resolve the citations, make abstention a first-class output; assembly that caps and orders rather than concatenates; detection that routes rather than proves, with the verification step kept independent of the draft; conversation state pinned and structured in the application instead of accumulating in the window; and containment, so that the failures you did not catch are cheap and visible.

Build the cheap layers first — they carry most of the benefit — and treat the expensive ones as a deliberate trade rather than a checkbox. Then measure, because the only difference between a mitigation and a belief is whether anything is counting.