When a driver‑assist system reaches 'good enough' performance (hundreds‑to‑thousands of miles between critical interventions), human supervisors reliably become less attentive and need several seconds to reengage, creating a new class of intermittent but high‑risk accidents. This makes marginal safety gains paradoxically dangerous: the better the system feels, the more brittle the human fallback becomes.
— This reframes the autonomous‑vehicle debate from pure capability to human–machine interaction risk, with implications for liability rules, certification standards, required human‑interface design, and rollout policy.
Steve Sailer
2026.11.22
100% relevant
Steve Sailer's summary of Tesla FSD v14/v15 reporting 'a couple thousand miles between critical interventions' and citing the 'vigilance decrement' and psychology literature as evidence.
← Back to all ideas