Product
ChatGPT and Claude citation check: the data
On November 7, 2026 we asked 40 everyday questions to ChatGPT and to Claude with web search, then checked every cited sentence against the page it cites, the claim-support check described in how an AI citation fails. When a sentence cited one page, both were about level: roughly 1 in 10 of those sentences were not backed by the page. Overall, about 1 in 4 to 1 in 5 of ChatGPT’s cited sentences were not backed, against about 1 in 8 for Claude, and most of ChatGPT’s gap came from sentences that cite several pages at once. This page holds every prompt, answer and verdict so the result can be checked or regraded.
Results
Share of checkable cited sentences that the cited page does not back:
| ChatGPT (logged out) | Claude (Opus 5.5) | |
|---|---|---|
| Verify, fixed rules | 23.5% of 485 | 12.0% of 706 |
| Independent AI judge, sample | about 21% | about 13% |
| Sentences citing a single page (Verify, fixed rules) | 8.6% of 370 | 10.9% of 652 |
| Sentences citing two or more pages (Verify, fixed rules) | 71.3% of 115 | 25.9% of 54 |
“Not backed” means the page does not say what the sentence says. The sentence can still be true. The checker is stricter than the judge on sentences that stitch several pages together, so the multi-page figures overstate the gap somewhat.
About 8% of cited pages could not be read (blocked automated readers, failed to load, or needed a login). Those sentences are marked unread and left out of the percentages.
How it was run
- Prompts. 40 everyday questions across shopping, software, statistics, companies, health, legal and money, and travel and recent events, written and frozen before any answer was collected.
- Assistants. ChatGPT logged out with default settings, which shows no model name. Claude in an incognito chat on a paid account, Opus 5.5 with web search on. This is not an even fight: logged-out ChatGPT runs a smaller model than a paid account. These are the setups most people meet.
- Capture. Each prompt was asked once per assistant on November 7, 2026, and each answer was saved with its real links.
- Grading. Hyperresearch Verify, in citations-only mode, graded each cited sentence against each page it cites. A sentence keeps its best verdict across its pages.
- Independent check. A separate AI model (GPT-5.5) re-read a random sample of 349 sentences against freshly fetched pages without seeing Verify’s verdicts, and graded 300 of them.
Checker errors we found and fixed
The first run said 40% for ChatGPT and 17% for Claude. The independent judge showed most of that gap was the checker:
- When a sentence cited two pages and one could not be read, the checker graded it against the readable page only and called it unsupported. These are now unread.
- Page titles that contained links of their own knocked some claims off their real source.
- It graded lines that are not claims: lead-ins ending in a colon, quiz options, ChatGPT’s “Worked for 1m 44s” status line, and chart captions.
verdicts-first-run.csv has the original verdicts. verdicts-fixed-rules.csv applies the fixed rules to the same page-by-page verdicts, which is a replay rather than a fresh run. Eight answers were also re-checked live with the fixed checker, and a further 12 fresh answers were graded as a holdout: on the 12 sampled claims a blind grader could read, it agreed with Verify on 10, and neither disagreement was a false “unsupported.”
Exclusions
- ChatGPT’s answer to prompt 12 (vector databases) is out of the totals. Most of its graded lines were chart labels and the status line, copied with the answer. Its rows stay in the files, marked excluded.
- Claude’s answer to prompt 38 (entering Japan as a US tourist) searched the web and gave no citations, so there was nothing to check.
- Lines that are not claims are excluded, each with its reason in the file.
Download the data
- All files as one zip
- prompts.csv: the 40 frozen prompts and their categories.
- answers.csv: one row per answer with capture time (Mountain Time), the model label shown, whether it searched, link count and capture notes. Each answer as captured, with its links, is in the zip under
answers/. - verdicts-fixed-rules.csv: every cited sentence with its cited URLs, whether it cites one page or several, the verdict under the fixed rules, and whether and why it is excluded. These are the reported numbers.
- verdicts-first-run.csv: the original verdicts with the evidence quote and source URL.
- judge-sample.csv: the 349 sampled sentences with Verify’s verdict, the judge’s verdict, its reason and the quote it relied on.
The tables are released under CC BY 4.0. Corrections to support@hyperresearch.ai.
