NewAnswers MCP: one question, one checked answer for your agent.See how it works

Product

Checking AI citations: five ways they fail

Updated

An AI citation checker answers a narrower question than people expect: does the cited source exist, and does it say what the sentence citing it says. That is not the same as an AI fact checker, which would judge whether a claim is true in the world. A citation can be checked; a source can still be wrong. Below are the five ways a citation in AI-written text fails, a documented case of each, how to catch each by hand in a minute or two, and which kind of tool catches which. A browser is all you need.

The scale is not small. Damien Charlotin’s public database of AI hallucinations in court filings listed 2,312 decisions on 18 November 2026. It tags 1,894 of them for fabricated material, 977 for misrepresented material and 613 for false quotes; one case can carry several tags. An AI hallucination checker that only asks whether a source exists sees the first group.

The source does not exist

In Mata v. Avianca, lawyers filed a brief citing Varghese v. China Southern Airlines and other decisions that no court had issued. The opinion records that one lawyer asked ChatGPT whether Varghese was real, and it replied that the case “does indeed exist” and was on Westlaw and LexisNexis. The court imposed a $5,000 penalty, jointly and severally, on the two lawyers and their firm. Outside law, a 2023 study in Scientific Reports found that 55% of GPT-3.5’s citations and 18% of GPT-4’s were fabricated across 636 references in 84 short literature reviews. And the May 2025 MAHA Commission report on children’s health cited seven sources that do not appear to exist, including a JAMA Pediatrics paper whose named first author said she never wrote it.

Catch it by hand. Never ask the model that wrote the citation; Mata shows why. For a paper, open https://doi.org/ followed by the DOI, or paste the reference list into Crossref’s Simple Text Query, which takes up to 1,000 references and returns the DOIs it can match. For a case, look the reporter citation up in a court database and check that the case name on that page matches; CourtListener also offers a citation lookup API built as a guardrail against hallucinated citations. A reference that resolves nowhere is fabricated.

The source exists but says something else

This is the failure a reference list hides. In Coomer v. Lindell, the judge found nearly thirty defective citations in one brief. One, to Perkins v. Federal Fruit & Produce, named a real District of Colorado case, but the order notes it “does not stand for the proposition asserted”, and the reporter citation given belonged to an unrelated False Claims Act case. The MAHA report cited a real review as showing therapy works as well as medication; a statistician on that review said it did not include psychotherapy at all. In medicine, a 2025 Nature Communications study found that 50% to 90% of LLM answers were not fully supported by the sources they cited, and about 30% of individual statements from GPT-4o with web search were unsupported. The Tow Center asked eight AI search tools to name the article behind 1,600 news excerpts and found more than 60% of answers were wrong.

Catch it by hand. Open the cited page itself, not its abstract or a search snippet. Search it for the sentence’s number first, then for its two most distinctive nouns. If no passage says what the sentence says, the citation is unsupported, however real the source.

The figure belongs to a different row, year or place

Numbers are where AI writing slips most quietly: the figure is on the page, attached to something else. In November 2026 we asked ChatGPT and Claude 40 everyday questions and checked every cited sentence against its page; the full data is public. Asked for the current US median home price, Claude wrote:

In the Northeast, the median price was $361,000, up 0.2% year-over-year.

It cited the National Association of Realtors’ September release. The check graded the sentence unsupported, and a separate AI grader that never saw that verdict agreed. The NAR page gives “Median price: $361,000, up 0.2% year-over-year” for the South. Its Northeast figure is $523,600, up 4.7%. Across the test, 23.5% of ChatGPT’s 485 checkable cited sentences and 12.0% of Claude’s 706 were not backed by the page they cited.

The MAHA report shows the same slip with years and people: it said antipsychotic prescriptions for children rose 800% from 1993 to 2009, where the cited paper found an eight-fold rise from 1995 to 2005, and it described a sleep study of college students as a study of children.

Catch it by hand. Once you find the number, read across its row and up its column. Compare five fields: place, period, unit, population, and whether the figure was measured or projected. Note the date you read the page, because monthly releases like NAR’s are replaced.

The quote was altered

A quotation is the easiest check of all and still fails. In the same Lindell brief, a parenthetical quote from Mata v. City of Farmington did not match the opinion. Asked about it, counsel told the court he “may have paraphrased and put quotes by mistake”. The Charlotin database’s 613 false-quote cases are this failure.

Catch it by hand. Copy four or five distinctive words from the middle of the quote and search the source for them. Then compare the full span word for word. A dropped hedge (“may”, “in this sample”) or a strengthened verb makes it a paraphrase in quotation marks, which counts as altered.

The paper was retracted

A retracted paper still resolves, keeps its metadata and stays downloadable, so an existence check passes it. A 2026 study in the Journal of Medical Internet Research gave nine free AI tools questions about 15 retracted articles from the Retraction Watch database. None handled retractions consistently; the best answered every question correctly for 8 of the 15, and the three research-focused tools never produced a fully correct set.

Catch it by hand. Open the DOI and look for a retraction banner or notice on the publisher’s page. To be sure, check the Retraction Watch data, which Crossref now publishes free through its API and as a CSV. Our guide to checking retraction status shows which field holds the answer; it is not the one Crossref’s documentation points to.

Which tool catches which failure

Kind of tool Example Catches Misses
Format checker Evernote’s free checker (APA, MLA, Chicago) A badly formatted reference All five failures above
Existence checker Copyleaks, GPTZero, Crossref Simple Text Query, CourtListener Made-up sources; wrong title, author or year A real source cited for something it does not say; shifted figures; altered quotes
Retraction lookup Crossref’s Retraction Watch data Retracted papers with a DOI Anything without a DOI
Claim-support checker A checker that reads each cited sentence against its source Unsupported claims, shifted figures, altered quotes Whether the source itself is right

Copyleaks compares title, authors and year field by field, a metadata match. GPTZero says it detects hallucinated sources and flags poorly supported claims. If your documents fail by inventing references, an existence checker is fast and enough. The step-by-step manual procedure covers the order to run these checks in.

Where Hyperresearch fits

Hyperresearch’s Verify check is a claim-support checker. It reads each cited sentence against the page it cites and grades it supported, partially supported, unsupported or wrong source, with the passage it relied on. It also matches quoted spans word for word, looks up cited DOIs for retractions and groups sources that copy one another. The result is a verification receipt with a row per citation, so you can read the passage and overrule a verdict. It reads public pages only; for a PDF or a page behind a login, you paste the source text. It skips author-year citations and figures published as images, and it is stricter than a person when a source rewords a claim. A supported verdict means the source says it, nothing more.

The same claim check runs on Hyperresearch’s own research reports. Light and Deep reports both have their cited claims checked against their sources and corrected where they do not hold; only Deep ships with the full receipt. How a report is researched before that check is on how it works and the deep research AI page, and the research scores 77.7 on DRACO.

Starting

The public paste box at /verify is not taking submissions right now. With an account you can check a document on the console’s Verify page, over the citation verification API, or from an agent through the verify_citations tool over MCP. The verification docs list the citation styles read, the limits and the price; pricing covers research reports, and your first Light report is free when you sign up, with no card. Sign up at the Hyperresearch console.