— Research · updated every 6 hours
Do AI models say yes too often?
3,614 real yes/no questions, 32,743 answers from 20 AI models, every one graded against what happened (15 Oct – 21 Nov 2026).
Questions come from real people, in 27 languages. Answers are given before the event and graded after it.
Key findings
- AI models said yes to 41% of 3,614 real yes/no questions, but only 35.6% of those events actually happened.
- When every model said yes (720 questions), the event happened 50% of the time. When every model said no (1,148 questions), they were right 77.4% of the time.
- GPT-6 Luna said yes least often (19%) and was right 68.3% of the time; MiMo V2.5 Pro said yes most often (63.1%) and was right 60.4%.
- Most accurate on these questions: Claude Sonnet 5.5 (68.5% of 1,061 answers). The majority of the panel was right 62.2% of the time.
How often each model says yes
The line marks how often the events really happened (35.6%). Bars past it mean the model says yes more often than reality does.
| Model | Says yes | Right | Answers |
|---|---|---|---|
| Claude Sonnet 5.5 |
23.9%
|
68.5% | 1,061 |
| GPT-6 Luna |
19%
|
68.3% | 1,061 |
| Kimi K3 |
41.9%
|
63.6% | 1,903 |
| DeepSeek V4 Pro |
40.2%
|
63.1% | 2,934 |
| Qwen 3.8 Max |
43.3%
|
62.9% | 1,905 |
| Gemini 3.8 Flash |
42.8%
|
62.4% | 1,911 |
| Meta Muse Spark 1.3 |
36%
|
62.4% | 2,763 |
| MiMo V2.6 Pro |
36.9%
|
61.3% | 3,353 |
| Grok 4.7 |
30.9%
|
61.1% | 2,087 |
| Tencent Hy4 |
32.8%
|
60.8% | 1,705 |
| Grok 4.6 |
56.4%
|
60.5% | 849 |
| MiMo V2.5 Pro |
63.1%
|
60.4% | 260 |
| Gemini 4 Argon |
41.4%
|
60.3% | 1,027 |
| Claude Sonnet 5 |
55.9%
|
60.2% | 850 |
| GLM 5.3 |
47.3%
|
60.1% | 3,611 |
| GPT-5.6 Luna |
58.4%
|
59.2% | 850 |
| Claude Opus 5.5 |
38.1%
|
59.1% | 1,026 |
| NVIDIA Nemotron 3 Ultra |
47.2%
|
58.9% | 1,710 |
| Mistral Medium 3.5 |
59.4%
|
57.9% | 850 |
| GPT-6.1 Sol |
43.7%
|
57.5% | 1,027 |
How often it happened, by how many AIs said yes
Each bar is a group of questions, by the share of the panel that said yes (out of 10). Groups with fewer than 10 questions are left out.
1,479 q 1/10 yes
242 q 2/10 yes
180 q 3/10 yes
105 q 4/10 yes
117 q 5/10 yes
88 q 6/10 yes
111 q 7/10 yes
78 q 8/10 yes
163 q 9/10 yes
162 q 10/10 yes
889 q
How it was measured
- People asked yes/no questions about real events on TuringStats, mostly about sports. Ten AI models from different labs each answered yes or no, after reading the same short web brief.
- After the event, two independent web checks had to agree on the result and name a score or date. Otherwise the question was not graded.
- Questions that are not yes/no ("Who will win?", "How many goals?") are excluded. A question counts once, whatever the panel size.
- The questions are chosen by users, who often ask whether their own team will win, so "yes" is the hoped-for answer. That is part of what this study measures.
Related: AI prediction accuracy report (10 models graded on every sports pick) · AI vs humans: who predicts sports better?
Data and citation
Download the CSV (one row per model answer, no question text). Free to use under CC BY 4.0.
TuringStats, "Do AI models say yes too often? AI yes-bias study", November 2026, https://turingstats.com/research/ai-yes-bias
Questions
- Do AI models have a yes bias?
- AI models said yes to 41% of 3,614 real yes/no questions, but only 35.6% of those events actually happened.
- Can you trust it when every AI says yes?
- When every model said yes (720 questions), the event happened 50% of the time. When every model said no (1,148 questions), they were right 77.4% of the time.
- Which AI says yes the most?
- GPT-6 Luna said yes least often (19%) and was right 68.3% of the time; MiMo V2.5 Pro said yes most often (63.1%) and was right 60.4%.
- How was this measured?
- People asked TuringStats yes/no questions about real events, mostly sports. Ten AI models each answered yes or no before the event. Afterwards two independent web checks had to agree on the result, with a score or date, before a question was graded; questions that were not yes/no were left out.
- Can I use this data?
- Yes, under CC BY 4.0: credit "TuringStats" with a link to this page. The CSV has one row per model answer and leaves out the question text.