How we grade the AI models

1. The models

Ten AI models answer every match: OpenAI GPT, Anthropic Claude, Google Gemini, xAI Grok, DeepSeek, Qwen, Kimi, GLM, Xiaomi MiMo and Mistral, called through the OpenRouter API. The current versions are listed on the leaderboard. When a provider releases a newer version of the same model family we switch to it and keep the family's record on one row; for example MiMo V2.6 Pro continues the record that earlier MiMo versions built. The exact model ID returned by the API is stored with every pick, and each model's match-by-match history is public, so the pick behind every number can be checked.

2. What every model sees

All ten models get the same prompt and the same data at the same time: the fixture (teams, competition, venue, kickoff), recent form from our data provider, the last head-to-head results with scores, and team news such as confirmed lineups and absentees when available (for the NBA: standings, averages, rest days and key players). The models never see bookmaker odds, so their picks are independent of the market. Each model answers on its own; no model sees another model's answer.

3. When picks are made and locked

Predictions are generated in the days before the event (at most 3 days ahead; picks made earlier are refreshed closer to kickoff). Only the last pick a model made before kickoff counts. Nothing is changed after the event starts.

4. The consensus

The headline pick for a match is the result most models chose. Since 22 October 2026, the consensus picks between the two sides only, because in our own data draw-majority picks were right far less often than the leading side. When at least 40% of the models pick a draw, the match is flagged "draw in play". Matches before that date keep the rule that applied at the time, so the published record is never rewritten.

"10/10 models agree" means ten independent models picked the same result. It is not a 100% probability: unanimous picks are right more often than split ones, but they still miss (see the current rate on the leaderboard).

5. How picks are graded

• Hit rate (1X2 / winner): the share of graded matches where the model picked the correct result (home, draw or away; winner only in sports without draws).
• Correct score and goals markets (over/under 2.5, both teams to score) come from each model's predicted score and are graded separately; they are never mixed into the winner hit rate.
• Confidence is the model's own stated probability for its pick. The confidence we show on a match is the plain average of the models' stated confidence. Confidence is not accuracy: a model saying 70% is not right 70% of the time.
• Brier score measures how well confidence matched reality (lower is better; spreading 1/3 on each football outcome scores 0.667). Models tend to sound more certain than they are, so for the Brier score only we calibrate each model's confidence using that model's own earlier results.

6. Samples and fair comparisons

Each model's row on the leaderboard counts every match that model answered, so totals differ slightly between models (a model may have failed on a match or joined later). For a like-for-like ranking, the leaderboard also shows a same-matches table: only matches where every current model made a graded pick. Baselines such as "always pick the home team" are computed on matches with at least one valid graded pick. Every published figure states its sample size and period.

7. Failed answers, postponed and cancelled matches

• If a model returns no valid answer, we retry several times before kickoff. If it still fails, that model has no pick for the match: placeholder answers are excluded from every statistic and hidden from public pages.
• Postponed matches are graded when they are actually played. Cancelled or abandoned matches without a result are not graded.
• Only fixtures accepted into our coverage are graded; we do not drop matches after the result is known.

8. Reports, data and corrections

The AI prediction accuracy report publishes a fixed monthly snapshot with a stable URL and a CSV download (CC BY 4.0), separate from the live leaderboard, so a cited number does not change later. If you find an error in a grade or a result, write to us via the contact page; confirmed corrections are applied to the live data and noted in the next report.

9. Limits

Sample sizes per competition can be small, especially early in a season; small differences between models (a few percentage points) are often within normal variation. Past accuracy does not guarantee future results. TuringStats is an analysis service: it does not take bets or sell tips.