Neural networks are clean
Points and comments are a snapshot, not live.
Neural networks are directly interpretable without untangling, argues Goodfire researcher Nick Cammarata.
Cammarata argues networks use one learning principle (recursive generalization), not two (superposition).
Canonical polysemanticity examples were misinterpreted; architectural issues made later layers overflow. Modern models show ~90% of neurons have one main job. With better tools, networks are legible without sparse autoencoders.
Understanding compounds as the map fills in. Larger models learn narrower, more recognizable neurons.
What commenters are saying
Commenters clarify the article's core claim: neural networks are not hopelessly entangled; neurons mostly encode one main concept (or related ones) rather than arbitrary mixtures.
Some question the reliability of mechanistic interpretability techniques like sparse autoencoders. One commenter notes small models merge features into overly general semantic buckets (like 羊 for various sheep/goats), making them harder for humans to parse. Larger models have more room for distinct concepts.