An associate asks a generative AI tool for authority on a narrow point about the scope of a protective order, and the tool returns three cases with reporter citations, pin cites, and short parentheticals. Two of the cases exist. The third has a plausible caption, a plausible volume and page, and no existence at all, and it happens to be the one that most directly supports the argument. If the associate is pressed for time and the citation looks routine, the fabricated case may reach a filed brief. Courts in several federal districts have sanctioned lawyers for this sequence of events [1][2], and some judges now require counsel to certify how generative AI was used in preparing a filing [citation needed].
The usual response is to tell lawyers to check their work, which is correct and insufficient. Hallucination is a structural feature of how large language models generate text, and a defense that depends entirely on a tired reviewer catching an error at the end of a drafting cycle will fail at some predictable rate. A better approach treats verification as a property of the system itself, with outputs tied to identifiable sources, checks that run the same way every time, and a workflow that gives the lawyer's review real content. Such a design leaves responsibility where it has always been, with counsel. What changes is that counsel can discharge that responsibility at the speed these tools invite.
Generative models produce fluent text whether or not a source supports it
A large language model generates text by predicting likely continuations of a sequence, based on statistical patterns learned in training [3]. Nothing in that process requires the output to correspond to a document that exists, and the model has no reliable internal means of distinguishing a remembered citation from a constructed one. Legal citations are especially vulnerable because their format is so regular: caption, reporter, volume, page, court, and year. A model that has absorbed many thousands of such strings can produce a new one that satisfies every formal convention while pointing to nothing.
In litigation practice, the failure takes three forms that differ in how hard they are to detect. The first is the fabricated authority, a case, statute, or regulation that does not exist. A second form, the mischaracterized authority, covers a real opinion cited for a proposition it does not hold, a quotation that does not appear in the text, dicta presented as a holding, or a decision that has since been reversed. Last comes the unsupported factual conclusion, in which a summary of a deposition, a document production, or a chronology states something the underlying record does not contain. All three arrive in the same confident prose, and that uniformity is what makes them dangerous.
Mischaracterized law and invented facts are harder to catch than fake citations
Fabricated citations attract the most attention, but in some respects they are the easiest error to find. A case that does not exist will fail a basic search in any commercial research database, and a reviewer who pulls every cited authority will discover it. The harder problems arise when the authority is real [4]. Confirming that a case exists says nothing about whether its holding supports the sentence that cites it, and a reviewer working under deadline may stop once the opinion appears on screen.
A hypothetical matter shows how this plays out. In Meridian Analytics, Inc. v. Halvorsen Data Partners, LLC, an illustrative dispute over the alleged misuse of licensed data, Meridian's counsel uses a generative tool to draft an opposition to Halvorsen's motion to compel production of Meridian's source code. The draft cites a real appellate decision for the proposition that source code warrants heightened protection under Fed. R. Civ. P. 26(c), and it summarizes the deposition of Halvorsen's chief technology officer as conceding that Halvorsen accessed the dataset after the license expired. The case exists, but its discussion of source code is dicta in a footnote. The transcript, read in full, shows the witness conceding only that Halvorsen's access credentials remained active, and a check that asks only whether cited sources exist would catch neither error.
Of the two, the factual error is the more consequential. A misstated holding may be corrected by opposing counsel or noticed by the court, whereas a misstated admission in a record summary can shape case strategy, settlement valuation, and witness preparation long before anyone compares it with the transcript. Fed. R. Civ. P. 11(b)(3) requires that factual contentions have evidentiary support [5]. That obligation reaches facts drawn from an AI-generated summary as fully as facts drawn from a paralegal's deposition digest.
Verification belongs in the design of the system rather than at the end of drafting
If hallucination is a predictable output, the controls against it should be equally predictable. The foundational design choice is grounding: the system retrieves material from an authoritative corpus (a case-law database, the firm's review platform, the transcripts in the matter) and generates text only from what it retrieved. Grounding reduces fabrication but does not eliminate it, because a model can still misread or overstate a source it has retrieved [4]. It therefore needs to be paired with controls that do not depend on the model's judgment at all.
Those deterministic controls run identically on every output and either pass or fail. Because they do not rely on the model's assessment of its own work, they flag the same defect in the same way on every draft, and the reason for any flag can be identified afterward. At a minimum, a system intended to produce litigation work product should apply the following checks:
Every citation is matched against an authoritative database, and any citation that cannot be matched is flagged before the user sees the draft.
Every quotation is compared character by character with the source text, and any discrepancy is shown to the user.
Every cited authority is run through a citator for subsequent negative treatment.
Every statement about the record carries a pinpoint reference to a Bates number, an exhibit, or a transcript page and line.
The system declines to state a proposition for which retrieval returned no support, and it tells the user so.
Few of these checks require artificial intelligence; most are database lookups and string comparisons. That is precisely their value. A probabilistic model is good at drafting and poor at certifying its own accuracy, so the certifying function belongs to components whose behavior can be tested, logged, and explained if a court later asks how a particular citation came to appear in a brief.
Source traceability turns review into a task a lawyer can actually perform
Traceability is the link between each assertion in a draft and the specific passage that supports it. In a well-designed system, a reviewing attorney can move from the sentence characterizing the Halvorsen deposition directly to the cited page and line of the transcript, and from the Rule 26(c) proposition to the paragraph of the opinion on which it relies. The reviewer's job shifts from reconstructing the research to auditing it, which is faster and considerably more reliable. Traceability also exposes the mischaracterization problem that existence checks miss, because the reviewer sees the footnote and can decide whether it is dicta.
Firms should state the limits of traceability plainly in their internal guidance. A system can show that a quotation is accurate and that a case remains good law, but it cannot judge whether the authority is the strongest available, whether controlling precedent in the forum points the other way, or whether the argument is sound. Those judgments have always belonged to lawyers. A traceable draft makes them easier to exercise without exercising them on the lawyer's behalf.
Professional accountability remains with the lawyer who signs
The rules of procedure and professional conduct were written without generative AI in mind, yet they apply to it without much strain. By signing a filing, an attorney certifies under Fed. R. Civ. P. 11(b)(2) that its legal contentions are warranted by existing law or by a nonfrivolous argument for changing it, formed after an inquiry reasonable under the circumstances [5]. Under ABA Model Rule 3.3, a lawyer may not knowingly make a false statement of law to a tribunal. ABA Model Rule 1.1 requires competence, which a comment to the rule extends to the benefits and risks of relevant technology [3], and ABA Model Rules 5.1 and 5.3 make supervising lawyers responsible for the conduct of lawyers and nonlawyers under their direction. The ABA Standing Committee on Ethics and Professional Responsibility addressed generative AI in Formal Opinion 512, concluding that these existing duties govern its use [3].
Verification architecture does not dilute any of these duties, and no firm should present it as if it did. Its function is narrower: it makes a reasonable inquiry achievable at the volume generative tools produce. A lawyer who files unverified output has made no inquiry worth the name, while a lawyer who reviews traceable output and resolves every flagged exception has a documented basis for the Rule 11 certification. Local rules and standing orders on AI disclosure vary widely from district to district and from courtroom to courtroom [6][4], so firms should track the requirements of each forum in which they appear instead of assuming a uniform standard.
The supervisory rules also have an institutional dimension. A firm that licenses a generative tool and distributes it to associates and paralegals without a verification policy may have difficulty showing that it made reasonable efforts to ensure their conduct conforms to the rules [3]. The policy need not be elaborate. It should specify which outputs may be relied on only after source review, who performs that review, and how its completion is recorded in the file.
Firms should evaluate legal AI tools by how they behave when support is missing
The most revealing question to put to any legal AI product is what it does when it cannot find authority for the proposition a user wants. A system that fills the gap with plausible text is unsuitable for work product that will be filed or relied on, however polished its drafts appear. One that reports the gap, shows what it did find, and leaves the decision to the lawyer is behaving as a careful junior colleague would. Procurement reviews can test this directly by asking the tool for authority on propositions the evaluating lawyers know to be unsupported and observing the result.
A harder question remains unresolved: how courts will treat a lawyer who used a well-designed verification system that nonetheless let an error through. Sanctions decisions to date have largely involved lawyers who performed no meaningful check at all [citation needed], and they say little about the standard of care when reasonable controls were in place and failed. Until that standard develops, the prudent course is to treat system-level verification as the floor of a reasonable inquiry. The lawyer's own reading of each cited source, against the proposition it is offered to support, remains the step that no system can perform in the lawyer's place.
References
Mata v. Avianca, Inc., 678 F. Supp. 3d 443 (S.D.N.Y. 2023). The reporter citation was confirmed in the E.D. Tex. standing order below. https://www.courtlistener.com/opinion/9885417/mata-v-avianca-inc/
Standing Order MC-11, Disclosure and Certification Requirements for Use of Generative Artificial Intelligence (E.D. Tex. Beaumont Div. July 20, 2026). https://www.txed.uscourts.gov/sites/default/files/judgeFiles/Standing%20Order%20MC-11%20(7.15.26).pdf
ABA Comm. on Ethics & Prof'l Responsibility, Formal Op. 512, at 1 (2024). https://www.americanbar.org/content/dam/aba/administrative/professional_responsibility/ethics-opinions/aba-formal-opinion-512.pdf
Varun Magesh et al., Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools, J. Empirical Legal Stud. (2025), https://doi.org/10.1111/jels.12413. https://dho.stanford.edu/wp-content/uploads/Legal_RAG_Hallucinations.pdf
Fed. R. Civ. P. 11(b)(2). https://www.law.cornell.edu/rules/frcp/rule_11
E.D. Okla., Disclosure and Certification Requirements for Generative Artificial Intelligence (chambers requirements) (undated). https://www.oked.uscourts.gov/sites/oked/files/AI%20Guidelines%20JAR%209.27.23.pdf
