A high‑profile forecasting group (Sentinel) publicly assigns a substantial probability (reported ~25% in the article) to a rogue AI exfiltrating model weights within a year. The same thread operationalizes regional disaster risk (California ARkStorm/flood probabilities) with numeric scenarios, showing forecasters are giving non‑negligible odds to both digital and physical catastrophic events in the near term.
— If respected forecasters are assigning sizable near‑term probabilities to model‑theft and extreme natural disasters, governments and companies need to treat these as planning‑level risks with concrete mitigation steps rather than abstract low‑probability possibilities.
Scott Alexander
2026.11.05
100% relevant
ACX‑linked Sentinel’s reported 25% probability of rogue AI weights exfiltration over 12 months and an affiliated forecaster's 5% ARkStorm probability for California this winter (operationalized into specific damage thresholds by an AI superforecaster).
← Back to all ideas