We tested five player markets. One ships a probability. The other 4 are on this page too, with the numbers that stopped them — because a calibration page that shows only the model that worked is an advert, not a receipt.
The model is fit on 2009–2018 and scored on 2019–2022, four seasons the calibration map has never seen. 2023–2025 are sealed and were not touched to make this page look better.
The benchmark is not a bookmaker — it is the player's own game log. If a model cannot beat “he cleared this in 8 of his 11 games”, it has told you nothing you could not already count.
Four markets passed that bar as originally written. Then we noticed the comparison was tilted: our probabilities got a calibration step and the game log got none, and a raw game log over four-to-sixteen games says “0%” and “100%” constantly, which scores terribly. We gave the game log the same calibration and re-ran it. Three of the four passes stopped being distinguishable from it, so they ship nothing. That check was ours to skip and nobody would have known.
Failed on calibration, not on skill: when it said 75.7%, it happened 67.1% of the time across 213 games. Under the registration that is terminal for the public number, and it stays terminal — the bar is not negotiable after you have seen the curve.
| We said | Mean claim | Actually happened | n |
|---|---|---|---|
| 0–10% | 0.0% | 7.1%below gate | 14 |
| 20–30% | 21.8% | 21.4%below gate | 84 |
| 30–40% | 36.4% | 36.0% | 464 |
| 40–50% | 44.1% | 45.9% | 547 |
| 50–60% | 53.3% | 52.0% | 1,307 |
| 60–70% | 68.8% | 71.8%below gate | 195 |
| 70–80% | 75.7% | 67.1% | 213 |
That interval contains zero, so this number is not measurably better than simply counting how often he has cleared the line before. We ship the count instead.
| We said | Mean claim | Actually happened | n |
|---|---|---|---|
| 0–10% | 0.0% | 0.0%below gate | 2 |
| 10–20% | 18.7% | 25.7%below gate | 148 |
| 20–30% | 26.4% | 24.2%below gate | 120 |
| 30–40% | 35.2% | 36.2% | 1,237 |
| 40–50% | 46.2% | 45.7% | 1,296 |
| 50–60% | 53.8% | 54.9% | 971 |
| 60–70% | 65.1% | 63.3% | 972 |
| 70–80% | 74.8% | 75.3% | 396 |
| 80–90% | 86.6% | 85.7% | 1,318 |
| We said | Mean claim | Actually happened | n |
|---|---|---|---|
| 0–10% | 0.0% | 0.0%below gate | 1 |
| 10–20% | 10.5% | 0.0%below gate | 4 |
| 20–30% | 26.3% | 26.8% | 661 |
| 30–40% | 36.8% | 38.5% | 4,448 |
| 40–50% | 44.7% | 46.2% | 4,191 |
| 50–60% | 53.8% | 51.7% | 3,493 |
| 60–70% | 64.6% | 63.6% | 2,763 |
| 70–80% | 77.3% | 78.2% | 4,053 |
That interval contains zero, so this number is not measurably better than simply counting how often he has cleared the line before. We ship the count instead.
| We said | Mean claim | Actually happened | n |
|---|---|---|---|
| 0–10% | 2.5% | 13.3%below gate | 15 |
| 10–20% | 17.1% | 19.2% | 532 |
| 20–30% | 27.0% | 27.5% | 2,169 |
| 30–40% | 34.6% | 35.4% | 6,119 |
| 40–50% | 45.2% | 45.3% | 3,112 |
| 50–60% | 54.3% | 53.9% | 2,410 |
| 60–70% | 64.1% | 63.5% | 1,721 |
| 70–80% | 75.7% | 76.8% | 3,528 |
That interval contains zero, so this number is not measurably better than simply counting how often he has cleared the line before. We ship the count instead.
| We said | Mean claim | Actually happened | n |
|---|---|---|---|
| 0–10% | 6.9% | 5.6%below gate | 18 |
| 10–20% | 17.6% | 20.8% | 283 |
| 20–30% | 26.4% | 28.6% | 486 |
| 30–40% | 36.1% | 33.0% | 642 |
| 40–50% | 43.1% | 41.3% | 1,279 |
| 50–60% | 54.4% | 53.5% | 1,636 |
| 60–70% | 63.3% | 64.4% | 466 |
| 70–80% | 76.8% | 75.3% | 441 |
| 80–90% | 85.4% | 84.9% | 358 |
| 90–100% | 93.4% | 91.2% | 855 |
A row reads green when what happened sits within 5 points of what we claimed. Amber means the claim needs fixing, not defending. Rows under 200 observations are marked below gate and were not used to judge anything: a reliability curve off a handful of games is a horoscope.
Nothing here is a betting edge, and passing this bar does not create one. You can be perfectly calibrated and still lose to the vig — that is the normal state of a good forecast facing an efficient market. These numbers describe a player. They say nothing about a price.
Every model this project has killed, and the figure that killed it, is on /receipts.