Market Brier Score by Platform and Category, 2024–2026

Bigger Cells Score Closer to Zero

Kalshi's markets, weighted by how many resolved in each cell, average a Brier score of 0.0219 across the sample (0 is perfect, 0.25 is a coin flip's baseline). Manifold's weighted average is 0.0598 — an order of magnitude higher — which the real-money-vs-play-money gap above is the leading candidate explanation for, though this table alone can't rule out other differences between the two platforms' typical markets.

Across all 209 cells, cells with more resolved markets tend to sit closer to zero: a log-linear fit of Brier score against the number of markets in a cell has a Spearman rank correlation of 0.146 (r²=0.002 for the fitted line itself, which explains only part of the spread). The best-covered Kalshi cells in the busiest recent weeks — Exotics (30281 markets, Brier 0.0282), Commodities (3223 markets, Brier 0.0101), Commodities (788 markets, Brier 0.0048) — all land near zero, while thin cells with under 10 markets, like Crypto (1 markets, Brier 0.4624), Crypto (1 markets, Brier 0.1225), Economics (8 markets, Brier 0.1209), swing much higher on noise alone.

Brier Score vs. Markets per Cell, All Platforms

One dot per platform-category-week cell; dashed line is a log-linear OLS fit (slope +0.0028 per 10x markets, r²=0.00). Hover a dot for platform and category.

Full Brier Score Table

brier_score is the mean squared error between price and outcome for the cell; lower is better.

Download the full table (CSV) · 209 rows

How the Numbers Were Made

Kalshi rows use the platform's own /series category field (matched by ticker prefix), not a guess. Manifold's public search API does not return a populated category field at this volume, so every Manifold row uses category "All" — its 129 rows cannot be split further; do not compare a single Manifold week against a single Kalshi category directly.

Both platforms keep only resolved markets with a definite YES/NO result and nonzero trading volume, so a zero-volume market's last quote is never counted as a real forecast.

Kalshi is real money; Manifold is play money. Kalshi settles contracts in U.S. dollars under CFTC oversight; Manifold runs on a fake in-app currency ('mana') with no cash value. The two platforms are shown side by side because they are the only ones with a usable, non-circular price-to-outcome record (see below), not because their numbers are directly comparable — a Manifold bettor risking play money has different incentives than a Kalshi trader risking cash, and that difference is a known, plausible explanation for any gap between the two.

Kalshi's settled-market history is heavily concentrated in 2026 — 5 of its 7 covered weeks are isolated single weeks in 2024/2025 with only 3-60 markets each, and the bulk of both volume and category diversity comes from the four weeks spanning 2026-07 through 2026-09.

Polymarket is not in this table. Its public Gamma API caps the reachable sample at roughly 2,100 closed markets, almost all from 2020–2022, before Polymarket's central-limit order book existed. On those markets the only price field available is the final settlement value (effectively exactly 0 or 1), not a pre-resolution trading price, and the CLOB's price-history endpoint returns no data for them. Using that field as a "forecast" would be circular — it would compare the outcome to itself — so Polymarket is left out rather than faked.

Sources

1. Kalshi public API (/markets, /series)primaryKalshi · primary · retrieved 2026-09-13 · Public API, no key required. Terms at https://kalshi.com/terms-of-use
2. Kalshi public API (/series, category listing)primaryKalshi · primary · retrieved 2026-09-13 · Public API, no key required. Terms at https://kalshi.com/terms-of-use
3. Manifold public API (/v0/search-markets)primaryManifold Markets · primary · retrieved 2026-09-13 · Public API, no key required. Terms at https://manifold.markets/terms