A neuron is a single number.
First, what are we even talking about? Here's an image model, the kind that looks at a photo and names what's in it. Zoom in until you reach one neuron.
Same model. Closer each step.
IllustrationSimplified. Real models have dozens of layers and thousands of neurons in each, and nobody assigns their numbers. The model learns them. Layer labels are rough tendencies.
That last step is basically how researchers read a neuron. Find what lights it up the most, then ask what those things have in common. Wheels. Easy.
Most neurons aren't that easy. Figuring them out is called interpretability.
A list of things can hide one pattern.
This neuron is famous. It lives in InceptionV1, an image model from 2014, and it lights up for cats. And cars. For years it was the go-to example of a neuron doing two unrelated jobs, which researchers call polysemantic.
But “cats and cars” is a prediction. So test it.
Same ten pictures. Three explanations.
Illustration of a real neuronIn the essay: neuron 4e:55. The real pictures are photos; icons stand in here. Its inputs include V-shaped flames and fox snouts, and what pushes it negative is round.
Cammarata had looked at this neuron's top pictures hundreds of times. Then he spent a few seconds asking what one thing it might be detecting. V shapes. And its opposite? O shapes.
That's a habit worth stealing. We describe things as lists of nouns. Neurons are often tuned to something the nouns share. Try it: if you couldn't name a single object, how would you describe what lights this neuron up?
Two theories make opposite predictions.
So why would a neuron mix things up at all? There were two explanations, and Cammarata helped come up with both.
Generalization: a neuron groups similar things, because what they share is useful.
Superposition: when a model runs out of room, it crams unrelated things into one neuron, because they rarely show up at the same time.
Take room away and see what each one predicts.
Same 32 things. Fewer neurons.
IllustrationTap a neuron to find what's inside it on both sides. Tap it again to clear.
This matters a lot. If superposition is true, a neuron's grouping tells you nothing, and you can't read the model directly. You first train a second model, called a sparse autoencoder, to untangle it, then study that instead.
Much of the field bet on superposition. Cammarata did too. His essay says the bet rested on evidence we misread.
The tangle came from the tools.
One way to see what a neuron wants is to generate a picture that lights it up as much as possible. The generator works backward through the neuron's inputs and takes the easiest path it can find. In InceptionV1, that path usually ran through dogs. Lots of dogs.
Same neuron. Two paths to a picture.
Diagram of an example in the essayIn the essay: Cammarata found this circuit in 2019 and, with Chris Olah, presented it as the first example of superposition. Rendered along the paths its real top photos take, it shows cars, plus cars and dogs facing the camera the same way. InceptionV1 has more than a hundred of these shared-pose neurons.
The model's design mattered too. The last layer of InceptionV1 was the best evidence that models are a mess. Almost none of its neurons made sense. Fix one thing at a time.
Same size. Same data. Fewer mysteries.
In the essayNeurons in the last layer that are easy to read; each square is 1%. The 5% is the author's recollection (others said near 0%). “Modern design” is a ConvNeXt with residual connections and LayerNorm, the same size and trained on the same data.
Didn't the model just need more room?
Mixed-up neurons grow one step at a time.
Some neurons still look mixed up. Cammarata calls the cause recursive generalization. During training, a neuron branches into nearby territory, then branches again from there. Train one and watch.
Same neuron. More training.
Illustration of examples in the essayEvery step makes sense from the one before. The ends just look unrelated. That's good news. Under superposition, a weird grouping is noise. Here, it's a trail you can follow back.
Chatbot neurons look readable too.
Language models, like the ones behind Claude and ChatGPT, are built from the same parts. Text goes in a word at a time, and each neuron's number rises and falls as it reads. An automated tool labeled one neuron in Qwen3-32B a “comma neuron.” Watch it read.
Same commas. Different context.
Illustration of a neuron in the essayIt's a Colorado neuron. Language models tend to stash a running summary of what they've read on punctuation, so it lights up on commas, not place names. Crank it up and the model wants to go hiking:
Cammarata's team audited a sample of that model's neurons. Compare that with what we could explain three years ago.
How many neurons make sense?
In the essayThe split depends on judgment calls, like how weak a response still counts. Cammarata reports similar results in models up to trillions of parameters and credits better tools and newer models, not neurons that were secretly different.
Some of this is shown. Some of it is a bet.
Cammarata calls the essay a working hypothesis. A technical paper is still coming.
| Claim | Status | Evidence in the essay |
|---|---|---|
| The cat-and-car neuron detects V shapes | Shown | Its inputs, outputs and negative side all point to shape. |
| The original superposition circuit was misread | Shown | Re-examined by the person who found it; cars and dogs shared a pose. |
| InceptionV1's messy last layer came from tools and design | Shown | Readable share rises from ~5% to ~99% at the same size. |
| Most chatbot neurons have one main job | Preliminary | Qwen3-32B audits; details are coming in the paper. |
| Recursive generalization explains most remaining mixing | Preliminary | Seen wherever he watched neurons grow during training. |
| Understanding models may be within reach soon | Speculative | He labels this highly speculative himself. |
Keep two things in mind. Cammarata works at Goodfire, a company that does exactly this kind of work, so he has a stake in it. And researchers invested in the untangling approach will push back, especially on the chatbot numbers.
Either way, I'm keeping the lesson. When something looks hopelessly tangled, check your tools and your labels before you blame the thing.
Terms, in the order they show up
- Neural network
- Layers of simple math that turn an input into an output. It learns the rules from examples.
- Layer
- One step in that stack. Early layers see simple patterns, late layers whole objects.
- Neuron
- One number in a layer that goes up or down with the input.
- Interpretability
- Figuring out what neurons and their connections do.
- Polysemantic
- A neuron that seems to have several unrelated jobs.
- Generalization
- Grouping similar things because what they share is useful.
- Superposition
- The theory that crowded models cram unrelated things into one neuron.
- Sparse autoencoder
- A second model trained to untangle a layer into cleaner pieces.
- Recursive generalization
- A neuron branching, step by step, into nearby territory.
- Language model
- A neural network trained on text to predict the next word.