AI forecasters now match top humans on standard prediction questions, but early tests show they underperform on conditional forecasts and on truly causal counterfactual questions. That performance gap matters because policy decisions require reliable answers to 'if we do X, what will happen' rather than unconditional probabilities.
— If AI can't robustly answer causal/conditional questions, governments and markets risk misusing persuasive but non‑causal forecasts for policy decisions and money flows.
Scott Alexander
2026.11.19
100% relevant
Forecasting Research Institute experiments showing AI weakness on conditional questions; commercial products (FutureSearch, Preseen, Mantic) offering conditional forecasting features; mentions of ForecastBench and Metaculus Cup as benchmarking touchstones.
← Back to all ideas