— Research · updated every 6 hours
Do AI models say yes too often?
2,513 real yes/no questions, 24,363 answers from 20 AI models, every one graded against what happened (15 Oct – 15 Nov 2026).
Questions come from real people, in 27 languages. Answers are given before the event and graded after it.
Key findings
- AI models said yes to 39.7% of 2,513 real yes/no questions, but only 35.5% of those events actually happened.
- When every model said yes (569 questions), the event happened 51.5% of the time. When every model said no (978 questions), they were right 79% of the time.
- GPT-6 Luna said yes least often (19.1%) and was right 68.3% of the time; MiMo V2.5 Pro said yes most often (63.1%) and was right 60.4%.
- Most accurate on these questions: Claude Sonnet 5.5 (68.5% of 1,058 answers). The majority of the panel was right 64.5% of the time.
How often each model says yes
The line marks how often the events really happened (35.5%). Bars past it mean the model says yes more often than reality does.
| Model | Says yes | Right | Answers |
|---|---|---|---|
| Claude Sonnet 5.5 |
24%
|
68.5% | 1,058 |
| GPT-6 Luna |
19.1%
|
68.3% | 1,058 |
| Tencent Hy4 |
28.5%
|
66.3% | 608 |
| Meta Muse Spark 1.3 |
28.9%
|
66.2% | 1,663 |
| Gemini 4 Argon |
32.2%
|
65.6% | 456 |
| DeepSeek V4 Pro |
38.9%
|
64.9% | 2,359 |
| NVIDIA Nemotron 3 Ultra |
42.6%
|
64.6% | 613 |
| Grok 4.7 |
25%
|
63.9% | 1,513 |
| Kimi K3 |
42%
|
63.6% | 1,899 |
| MiMo V2.6 Pro |
35.7%
|
63.4% | 2,252 |
| Claude Opus 5.5 |
32.1%
|
63.1% | 455 |
| Qwen 3.8 Max |
43.2%
|
63% | 1,901 |
| GPT-6.1 Sol |
34.6%
|
62.7% | 456 |
| Gemini 3.8 Flash |
42.8%
|
62.5% | 1,907 |
| GLM 5.3 |
47.3%
|
61.9% | 2,510 |
| Grok 4.6 |
56.5%
|
60.5% | 848 |
| MiMo V2.5 Pro |
63.1%
|
60.4% | 260 |
| Claude Sonnet 5 |
55.9%
|
60.2% | 849 |
| GPT-5.6 Luna |
58.4%
|
59.1% | 849 |
| Mistral Medium 3.5 |
59.4%
|
58% | 849 |
How often it happened, by how many AIs said yes
Each bar is a group of questions, by the share of the panel that said yes (out of 10). Groups with fewer than 10 questions are left out.
1,053 q 1/10 yes
185 q 2/10 yes
117 q 3/10 yes
88 q 4/10 yes
61 q 5/10 yes
68 q 6/10 yes
57 q 7/10 yes
55 q 8/10 yes
108 q 9/10 yes
113 q 10/10 yes
608 q
How it was measured
- People asked yes/no questions about real events on TuringStats, mostly about sports. Ten AI models from different labs each answered yes or no, after reading the same short web brief.
- After the event, two independent web checks had to agree on the result and name a score or date. Otherwise the question was not graded.
- Questions that are not yes/no ("Who will win?", "How many goals?") are excluded. A question counts once, whatever the panel size.
- The questions are chosen by users, who often ask whether their own team will win, so "yes" is the hoped-for answer. That is part of what this study measures.
Related: AI prediction accuracy report (10 models graded on every sports pick) · AI vs humans: who predicts sports better?
Data and citation
Download the CSV (one row per model answer, no question text). Free to use under CC BY 4.0.
TuringStats, "Do AI models say yes too often? AI yes-bias study", November 2026, https://turingstats.com/research/ai-yes-bias
Questions
- Do AI models have a yes bias?
- AI models said yes to 39.7% of 2,513 real yes/no questions, but only 35.5% of those events actually happened.
- Can you trust it when every AI says yes?
- When every model said yes (569 questions), the event happened 51.5% of the time. When every model said no (978 questions), they were right 79% of the time.
- Which AI says yes the most?
- GPT-6 Luna said yes least often (19.1%) and was right 68.3% of the time; MiMo V2.5 Pro said yes most often (63.1%) and was right 60.4%.
- How was this measured?
- People asked TuringStats yes/no questions about real events, mostly sports. Ten AI models each answered yes or no before the event. Afterwards two independent web checks had to agree on the result, with a score or date, before a question was graded; questions that were not yes/no were left out.
- Can I use this data?
- Yes, under CC BY 4.0: credit "TuringStats" with a link to this page. The CSV has one row per model answer and leaves out the question text.