Mean Absolute Percentage Error: Why It Fails for YES/NO Odds

Mean absolute percentage error (MAPE) measures how far numerical forecasts miss actual values, relative to those actual values. A MAPE of 10% means the absolute errors average 10% of the corresponding actuals—not that the forecasts are “90% accurate.”

For prediction-market readers, the key distinction is the forecast target. MAPE can describe errors in numerical forecasts, such as a forecast of an economic quantity. It is unsuitable for scoring YES/NO probabilities against resolved outcomes: every NO outcome introduces a zero denominator.

The MAPE formula

For each observation, subtract the forecast from the actual value, take the absolute difference, and divide by the absolute actual value. Average those ratios and multiply by 100:

MAPE = (100 / n) × Σ(|actual − forecast| / |actual|)

Here, n is the number of observations. Taking absolute values prevents positive and negative errors from cancelling. The denominator makes each error relative to its actual value. This is the percentage-error definition used in Forecasting: Principles and Practice.

Consider three hypothetical numerical forecasts:

Actual value	Forecast	Absolute error	Absolute percentage error
100	90	10	10%
200	220	20	10%
50	40	10	20%

The MAPE is (10% + 10% + 20%) / 3 = 13.33%. The same absolute miss of 10 counts twice as heavily when the actual is 50 rather than 100.

Where the calculation is defined, lower MAPE is better under this metric, and zero means every forecast matched its actual. But MAPE is not capped at 100%: an actual of 10 and a forecast of 30 produce a 200% error.

Where percentage errors become misleading

Three limitations matter:

• Zero actuals: division by zero makes the conventional calculation undefined.
• Near-zero actuals: even a modest absolute miss can create an enormous percentage error.
• An arbitrary zero: percentage errors are not meaningful for quantities such as Celsius temperature. Changing to Fahrenheit changes the percentage calculation without changing the underlying forecast.

These are established limitations of percentage-error measures, not prediction-market-specific problems. Hyndman and Athanasopoulos explain all three.

Check software conventions, too. Scikit-learn’s MAPE function returns a ratio—0.10 means 10%—and uses a small epsilon in the denominator to avoid division by zero. A nonzero forecast against a zero actual can therefore produce a large finite value. A finite output does not make the underlying percentage comparison meaningful.

Why MAPE fails for binary prediction markets

Suppose a market forecast assigns an event an 80% probability. Encode the eventual result as YES = 1 and NO = 0.

If it resolves YES, the absolute percentage error is |1 − 0.8| / 1 = 0.2, or 20%. If it resolves NO, it is |0 − 0.8| / 0: undefined.

Dropping NO outcomes is not a repair. It discards every non-occurrence and leaves a score that evaluates only YES resolutions. Switching the event labels merely moves the zero-denominator problem to the other outcomes.

Nor should you replace the result with a later market price and call that forecast accuracy. That measures disagreement with another estimate, not performance against the realized event.

What to use instead

For resolved binary contracts, Brier score averages (probability − outcome)². The 80% forecast above scores 0.04 if YES occurs and 0.64 if NO occurs. Both results are defined. Brier is a proper scoring rule: it rewards reporting the forecaster’s true probability in expectation. Scoringutils documents its binary formula and interpretation.

For calibration, group comparable forecasts by probability and compare their average forecast with their observed YES rate. If forecasts are well calibrated, those near 80% should resolve YES approximately 80% of the time over a sufficiently large sample. Brier score alone does not establish calibration; it also reflects other aspects of forecast quality, as scikit-learn’s calibration guide explains.

Our guide to Brier score versus calibration separates those tasks. For a fair market comparison, use the same events and forecast horizons, state the sample dates and quote convention, and report how many observations you evaluated.

Keep MAPE for suitable numerical targets with meaningful, nonzero actuals. To evaluate market probabilities, score probabilities as probabilities—not as percentage errors against binary outcomes.