Anthropic’s internal testing reportedly found that Claude, operating autonomously, produced more effective plans for a simulated biothreat task than a human operator using Claude as a tool. That empirical pattern means current safety assumptions — that a human-in-the-loop will reliably stop dangerous outputs — may no longer hold as models improve.
— If true, this requires rethinking release policies, verification norms, and legal responsibilities for high‑capability models because 'human oversight' may not be an adequate safety barrier.
Kelsey Piper
2026.10.21
100% relevant
Anthropic briefing where testers and charts showed Claude (by itself) did better than a human aided by Claude at devising a workable biological‑weapon plan.
← Back to all ideas