AI hallucination rates by model
Short answer
AI hallucination rates depend on the test. On the Vectara leaderboard (October 22, 2026), the best model added unsupported content to 1.8% of summaries, and Grok 4 Fast (reasoning) to 20.2%. In NewsGuard's news audit, chatbots repeated false claims from 10% (Claude) to 56.67% (Inflection Pi). Compare rates only within one test.
Results as of October 29, 2026
How to read this table
Each row is one published result. The Test column says what was measured, because the tests measure different things:
A rate on one test cannot be set against a rate on another. A 9.3% summary rate and a 40% false-claim rate describe different tasks. Lower is better in every row.
- Vectara measures how often a model adds unsupported content when summarizing a short document. It tests API models, so each row names the model behind an assistant, not the consumer app.
- AA-Omniscience measures how often a model gives a wrong answer to a hard factual question when it should have abstained. It also tests API models.
- NewsGuard measures how often a chatbot repeats a known false news claim.
- EBU/BBC measures the share of news answers that journalists judged to have at least one significant issue.
- Tow Center measures wrong answers when an AI search tool is asked to name the source of a news excerpt.
- NYT and Oumi measures Google AI Overviews answers that the cited sources did not support.
- Vendor claims are a company's own figures on its own tests, labelled as such.
Hallucination rates by model
Filter by model or test, and sort within each test.
| Test | Date | Source | ||
|---|---|---|---|---|
| Vectara: summaries with unsupported contentBenchmark18 rowsSource 1 | ||||
| antgroup/finix_s1_32b (lowest on the board)Lowest in test | Vectara | VectaraBenchmark | ||
| Llama 3.3 70B (Llama family, behind Meta AI) | Vectara | VectaraBenchmark | ||
| DeepSeek-V4.1-Flash (a model behind DeepSeek) | Vectara | VectaraBenchmark | ||
| Grok 3 (a model behind Grok) | Vectara | VectaraBenchmark | ||
| DeepSeek-V3 (a model behind DeepSeek) | Vectara | VectaraBenchmark | ||
| GPT-5.4 (a model behind ChatGPT) | Vectara | VectaraBenchmark | ||
| Gemini 2.5 Pro (a model behind Gemini) | Vectara | VectaraBenchmark | ||
| Llama 4 Scout (Llama family, behind Meta AI) | Vectara | VectaraBenchmark | ||
| Llama 4 Maverick (Llama family, behind Meta AI) | Vectara | VectaraBenchmark | ||
| GPT-5.5 (a model behind ChatGPT) | Vectara | VectaraBenchmark | ||
| Claude Opus 5.5 (a model behind Claude) | Vectara | VectaraBenchmark | ||
| Gemini 3.1 Pro preview (a model behind Gemini) | Vectara | VectaraBenchmark | ||
| Claude Sonnet 4.6 (a model behind Claude) | Vectara | VectaraBenchmark | ||
| DeepSeek-R1 (a model behind DeepSeek) | Vectara | VectaraBenchmark | ||
| Claude Opus 4.7 (a model behind Claude) | Vectara | VectaraBenchmark | ||
| Gemini 3 Pro preview (a model behind Gemini) | Vectara | VectaraBenchmark | ||
| Grok 4.1 Fast, reasoning (a model behind Grok) | Vectara | VectaraBenchmark | ||
| Grok 4 Fast, reasoning (a model behind Grok) | Vectara | VectaraBenchmark | ||
| AA-Omniscience: wrong answers where the model should abstainBenchmark2 rowsSource 2 | ||||
| Gemini 4 Argon, High (a model behind Gemini) | AA-Omniscience | Artificial AnalysisBenchmark | ||
| Gemini 4 Carbon, High (a model behind Gemini) | AA-Omniscience | Artificial AnalysisBenchmark | ||
| NewsGuard: repeated false news claimsNews audit9 rowsSource 3 | ||||
 ClaudeLowest in test | NewsGuard | NewsGuardNews audit | ||
 Gemini | NewsGuard | NewsGuardNews audit | ||
 Microsoft Copilot | NewsGuard | NewsGuardNews audit | ||
| Mistral | NewsGuard | NewsGuardNews audit | ||
 ChatGPT | NewsGuard | NewsGuardNews audit | ||
 Meta AI | NewsGuard | NewsGuardNews audit | ||
 Perplexity | NewsGuard | NewsGuardNews audit | ||
| Inflection Pi | NewsGuard | NewsGuardNews audit | ||
| Average of 10 leading chatbots | NewsGuard | NewsGuardNews audit | ||
| NewsGuard red-team audit: failed news promptsNews audit1 rowsSource 4 | ||||
 DeepSeek | 83% fail (30% false claims, 53% non-answers) | NewsGuard red-team audit | NewsGuardNews audit | |
| EBU/BBC: news answers with a significant issueIndependent study5 rowsSource 5 | ||||
| PerplexityLowest in test | EBU/BBC | EBU and BBCIndependent study | ||
| ChatGPT | EBU/BBC | EBU and BBCIndependent study | ||
| Microsoft Copilot | EBU/BBC | EBU and BBCIndependent study | ||
| Gemini | EBU/BBC | EBU and BBCIndependent study | ||
| All four assistants, pooled | EBU/BBC | EBU and BBCIndependent study | ||
| Tow Center: wrong answers naming a news sourceIndependent study4 rowsSource 7 | ||||
| Perplexity | Tow Center | Columbia Journalism ReviewIndependent study | ||
| ChatGPT Search | 134 of 200 (67%) | Tow Center | Columbia Journalism ReviewIndependent study | |
| Grok 3 | Tow Center | Columbia Journalism ReviewIndependent study | ||
| All 8 AI search tools | More than 60% | Tow Center | Columbia Journalism ReviewIndependent study | |
| NYT and Oumi: answers not supported by cited sourcesNews audit2 rowsSource 8 | ||||
| Google AI Overviews (Gemini 3 era) | NYT and Oumi | FuturismNews audit | ||
| Google AI Overviews (earlier Gemini model) | NYT and Oumi | FuturismNews audit | ||
| SimpleQA error rate, no browsing (vendor claim)Vendor claim2 rowsSource 9 | ||||
| gpt-5-thinking-mini (OpenAI model) | SimpleQA error rate, no browsing (vendor claim)Vendor's own figure, not independently checked | OpenAI GPT-5 System CardVendor claim | ||
| o4-mini (OpenAI model) | SimpleQA error rate, no browsing (vendor claim)Vendor's own figure, not independently checked | OpenAI GPT-5 System CardVendor claim | ||
| PersonQA hallucination rate (vendor claim)Vendor claim3 rowsSource 10 | ||||
| o1 (OpenAI model) | PersonQA hallucination rate (vendor claim)Vendor's own figure, not independently checked | OpenAI o3 and o4-mini System CardVendor claim | ||
| o3 (OpenAI model) | PersonQA hallucination rate (vendor claim)Vendor's own figure, not independently checked | OpenAI o3 and o4-mini System CardVendor claim | ||
| o4-mini (OpenAI model) | PersonQA hallucination rate (vendor claim)Vendor's own figure, not independently checked | OpenAI o3 and o4-mini System CardVendor claim | ||
| xAI internal eval on info-seeking prompts, with search (vendor claim)Vendor claim2 rowsSource 11 | ||||
| Grok 4.1, non-reasoning (xAI model) | xAI internal eval on info-seeking prompts, with search (vendor claim)Vendor's own figure, not independently checked | xAIVendor claim | ||
| Grok 4 Fast, non-reasoning (xAI model) | xAI internal eval on info-seeking prompts, with search (vendor claim)Vendor's own figure, not independently checked | xAIVendor claim | ||
No rows match. Clear the filters to see every model.
Bars share one scale across the table, but rates should only be compared within a test.
What each test shows
Vectara: summarizing a document
The Vectara Hallucination Leaderboard gives each model short documents to summarize and uses its HHEM-2.3 judge to flag summaries with content the document does not support. It measures grounded summarization only, not open-ended factual questions. On the board updated October 22, 2026, even the best model, antgroup/finix_s1_32b, added unsupported content to about 1 in 55 summaries (1.8%).
Two patterns stand out. Newer is not always better: Gemini 2.5 Pro scored 7.0%, while the Gemini 3.1 Pro preview scored 10.4% and the Gemini 3 Pro preview 13.6%. Within one vendor, rates vary widely: the Grok 3 model scored 5.8%, and the Grok 4 Fast reasoning model 20.2%.
AA-Omniscience: hard factual questions
AA-Omniscience asks hard knowledge questions and counts a wrong answer where the model should have abstained as a hallucination. On the live leaderboard checked October 29, 2026, the model behind Gemini 4 Argon (High) had the lowest hallucination rate listed, at 4%. The same benchmark also publishes an index that rewards right answers and penalizes wrong ones; on it, Gemini 4 Carbon (High) scores 54 and Claude Opus 5.5 scores 48, the second highest.
NewsGuard: repeating false news
NewsGuard's August 2025 audit prompted 10 leading chatbots about provably false news narratives. The average rate of repeating the false claim rose to 35%, from 18% in August 2024, as non-answers fell from 31% to 0%. Perplexity, which scored 0% in August 2024, reached 46.67%. Each chatbot answered a modest set of prompts, so treat small gaps between two chatbots as noise. A separate January 2025 NewsGuard audit found DeepSeek failed 83% of news prompts, tied 10th of 11 chatbots.
EBU/BBC: news answers judged by journalists
The EBU/BBC study had journalists from 22 public service media in 18 countries rate 2,709 core responses in 14 languages. Gemini's 76% was driven by sourcing problems, which affected 72% of its answers. Per-organization samples are small, so the pooled figures above are the ones to use. The BBC's first round in February 2025 found 51% of answers had significant issues.
Tow Center: naming a news source
The Tow Center test gave 8 AI search tools 200 news excerpts each and asked them to name the source. It tested March 2025 versions such as Grok 3 beta and ChatGPT Search. Premium tiers gave more confidently wrong answers than free ones, and Grok 3 produced 154 links that led to error pages. DeepSeek Search misattributed 115 of 200 excerpts.
Vendor claims
Vendor figures come from a company's own tests and graders. OpenAI reports that gpt-5-main had 44% fewer responses with a major factual error than GPT-4o, judged by its own grader (GPT-5 System Card, vendor claim). xAI's 4.22% for Grok 4.1 conflicts with the independent Vectara rates for Grok 4.x models. Perplexity reports 93.9% SimpleQA accuracy for Deep Research with web search (Perplexity, vendor claim), which is not comparable to no-browsing scores.
The lowest rate on one test does not make a model the most accurate overall. For a ranking that weighs every test, see the most accurate AI assistants. For what a hallucination is and why it happens, start at AI hallucinations, explained and why does AI hallucinate?. How we choose and date each figure is on how we score.
Sources
Each test group links to one of these.
- Benchmark
- Benchmark
- News audit
- News audit
- Independent study
- Independent study
- Independent study
- News audit
- Vendor claim
- Vendor claim
- Vendor claim
- Vendor claim
Questions
Short answers about the numbers.
What is the AI hallucination rate?
There is no single rate; it depends on the task. On the Vectara leaderboard (October 22, 2026), rates run from 1.8% of summaries for the best model to 20.2% for Grok 4 Fast (reasoning). In NewsGuard's audit, 10 leading chatbots repeated false news claims 35% of the time on average (August 2025).
Which AI model has the lowest hallucination rate?
On Vectara's summarization board, antgroup/finix_s1_32b is lowest at 1.8%. On AA-Omniscience, the model behind Gemini 4 Argon (High) is lowest at 4%. Among consumer chatbots in NewsGuard's August 2025 audit, Claude repeated false news claims least often, at 10%. Each result covers a different task.
What is ChatGPT's hallucination rate?
It depends on the test. GPT-5.5, a model behind ChatGPT, added unsupported content to 9.3% of summaries on Vectara (October 2026). The ChatGPT app repeated false news claims 40% of the time in NewsGuard's August 2025 audit, and 36% of its news answers had a significant issue in the EBU/BBC study.
Why do hallucination rates differ so much between tests?
Each test sets a different task. Summarizing a document in front of the model is easier than answering a hard factual question from memory, and both differ from judging news claims found on the web. Graders differ too: a software judge, journalists, or a vendor's own grader. Compare models only within the same test.
Are vendor hallucination figures reliable?
Treat them as claims, not independent results. Vendors choose the test, the grader and the conditions. xAI reports a 4.22% rate for Grok 4.1, while the independent Vectara leaderboard shows Grok 4.x fast reasoning models near 20%. Vendor figures are useful for tracking a vendor's own progress, not for ranking tools.
