AI models can provably hide arbitrary capabilities
Points and comments are a snapshot, not live.
Researchers built backdoor attacks that hide arbitrary capabilities inside AI models, provably hard to detect.
The paper constructs backdoor attacks that embed encrypted neural networks within host model weights, which decrypt and execute only upon a secret trigger. Using cryptographic 'digital lockers,' hidden circuits remain provably hard to elicit under cryptographic assumptions, extending prior work from hiding fixed strings to arbitrary computations. The authors implemented their proof of concept in PyTorch and tested elicitation resistance via fine-tuning, weight noising, gradient-based trigger search, and latent adversarial training on a chemical reaction prediction task. They argue this reveals a fundamental limit of elicitation-based capability audits and release models to support defense improvement. Accepted at NeurIPS 2026.
What commenters are saying
Commenters broadly view this as a formal demonstration of a long-feared attack vector. A top comment frames the threat as an 'encrypted malicious payload' that only responds to specific inputs. Several commenters draw parallels to binary blobs and the xz crisis, noting we already install opaque models from companies like OpenAI and Anthropic. A technical comment grounds the result in Rice's theorem, arguing any sufficiently expressive computational formalism contains undetectable runtime properties. Some push back on the framing, claiming the technique is trivially detectable at the network structure level despite the cryptographic claim.