Humans missed 1 in 3 threats approving AI agent commands across 40k game runs
Points and comments are a snapshot, not live.
A browser game finds humans miss 1 in 3 threats when approving AI agent commands.
The game simulates a human-in-the-loop approving AI agent commands under time pressure. Across 40,000 runs and 409,000 decisions, players missed 32.9% of threats. Obviously destructive commands (rm -rf /) were missed 11.7% of the time, while credential exfiltration (cat ~/.aws/credentials) was missed 35%. The most missed command was npm run analyze (64.7%). Players blocked 59% of safe commands like setting an internal npm registry. The author argues that permission fatigue makes human-in-the-loop an unreliable security boundary, and advocates for sandboxing and context isolation instead.
What commenters are saying
Several commenters argue that asking users to constantly approve permissions is a flawed, historically repeated model that never works. One commenter notes it's partly for liability reasons: "Your employee approved the bash call? Not our fault then!" Another calls it a "moral crumple zone." A few defend it as necessary for useful software, noting that perfectly safe and useful agents don't exist. There is debate over what a truly secure agent would look like, with suggestions including capability-based security, sandboxing, and blast-radius containment.