A Neuron Atlas for a Speech Model: What a Local 4B Can Label, What Replays Bit for Bit, and the Voice Levers We Had to Re-check

We run a steering explorer on LFM3-Audio-2B, Liquid AI's 2B model that listens and speaks. It is our Rust port of lyramakesmusic's neuron-explorer, running on our Rust engine on a Mac (Metal) and on our rig (CUDA). Pick any of the model's 184,320 MLP neurons, push it up or down, and hear the model answer differently. What the explorer does not do is remember what you learned: a neuron you named in one session is a number again in the next, and finding a neuron worth naming is mostly luck.

So we wanted an atlas: one card per neuron with a name, a description, the evidence, how it was made, and a test anyone can re-run. Local models would write most of the cards. If it works, other people's machines could write and check cards too.

Before building it we spent one day on a feasibility probe: six experiments, one Mac, local models only, no dataset downloads. This is what we measured, including what did not work.

The short version

• A local 4B model can label selective neurons, not the whole model. Its labels predict held-out activations (mean r 0.32 over 300 neurons; shuffled labels 0.00). For the most selective neurons, 59 % score above 0.3. For neurons drawn at random, only 12 % do.
• Steering backs the labels, weakly. Pushing a neuron up made a blind judge pick "the reply that shows more of the label" 62 % of the time, 66 % for well-simulated labels. The 4B judge is too biased by answer order to certify a single card.
• Every test replays exactly. 300 of 300 text tests and 51 of 51 voice tests gave identical token ids and identical WAV bytes in new processes. Metal and CUDA produced the same speech codes, frame for frame. This is the result an open atlas can be built on.
• Families written from descriptions are topical, not functional. Dosing a whole "family" beat its best member in 2 of 12 families.
• Labels transfer between models. LFM3 neurons have counterparts in Qwen3.5-4B, a text-only model with another tokenizer. For the 40 best pairs, labels written independently describe the same feature in 30 of 40, against 0 of 40 shuffled.
• Three pitch neurons survived; our first answer did not. With two seeds we reported five neurons that "move pitch". With 16 seeds and 16 random neurons beside them, three survived (26 to 37 Hz between pushing down and up) and three were no better than random neurons, half of which move pitch a little too.

---

I – Setup

LFM3-Audio-2B has a 24-block backbone with 7,680 MLP neurons per block: 184,320 neurons. A "neuron" here is one input of a block's down projection, the place where the explorer's pushes write. A dose of 1 moves the residual stream by about its median size, so dose 1 is a strong push and dose 0.5 a moderate one.

The local models ran in LM Studio on the same Mac, bound to localhost:

• Qwen3.5-4B wrote the labels, simulated them, judged steered replies and wrote part of the corpus.
• A Qwen3.8 27B served as a second labeller and grouped labels into families.

The corpus was text already on this machine: the 69 articles of this blog (1,465 excerpts), the explorer's own prompts, some Rust source, and 320 short passages the 4B wrote across 40 genres, from recipes and legal text to slang, six other languages and descriptions of voices. That makes 2,799 excerpts and 164,712 tokens. One forward pass over it reads all 184,320 neurons at once: 162 seconds on the Mac.

II – Can a 4B model label a neuron?

We used the method of Bills et al. (2023):

1. Explain. The explainer sees a neuron's ten strongest excerpts, with the activation marked on each token, and writes a short label: what it fires on and what it promotes.
2. Simulate. A simulator sees only the label and ten held-out excerpts it has not seen, and predicts the activation of every token.
3. Score. The score is the correlation between predicted and real activations.

The baseline is the same simulation with labels shuffled between neurons. We labelled 300 neurons: the 10 most selective per layer (240) and 60 drawn at random.

labels	neurons	mean r	above 0.3
own label, all	300	0.32	49 %
own label, selective	240	0.38	59 %
own label, random neurons	60	0.10	12 %
shuffled label	300	0.00	0 %

• Own against shuffled. The neuron's own label beats the shuffled one on 248 of 300 neurons.
• Selective neurons win. Selectivity comes free with the first corpus pass, and it predicts which labels will work.
• Random neurons get plausible labels that predict nothing: "ocean and water imagery", "terminal command syntax". They fire on a little of everything.
• Easy and hard layers. The first two layers are the easy ones (mean r 0.54 to 0.61). Most of their selective neurons detect a token: "kick", "several", French "de", "out of". The last four layers are the hardest (0.05 to 0.26).
• A wrong label the score catches. L10/N2983 fires on " even", mostly after "before". The 4B called it "temporal precedence marker" because of "before", and the simulator, given that label, predicted the wrong tokens.

A bigger explainer helps, at a price. On 60 of the neurons, the 27B's labels scored 0.42 against 0.34 for the 4B (better on 36 of 60, sign test p = 0.03). It took 27.8 seconds per label against 5.1. With four parallel slots on a shared Mac, the 4B wrote about 2,830 labels per hour and simulated about 1,030.

III – Is the label causal?

A label says what a neuron reads. An atlas for steering needs to know what it does. For each of the 300 neurons:

• Push. We dosed the neuron +1 and -1 on two neutral prompts ("Tell me about your day." and "Write a few sentences about anything you like.").
• Judge. We asked the 4B, shown only the label and the two replies as A and B, which reply shows more of the label. Each pair was asked in both orders.

The judge picks the up-dosed reply in 61.6 % of 1,200 judgments (95 % interval 58.8 to 64.3 %), and in 66 % for labels that simulated well. A clean case is L0/N7366, labelled "the act of kicking a ball or object":

• Up dose: asked about its day, the model says "It's 2:00! I'm going to kick a goal! I'm going to kick a goal! …".
• Down dose: it says "I don't have personal experiences or a day like humans do …".

The signal is real but weak, and the judge has a bias of its own:

• Position: it picks answer A 59 % of the time.
• Order: in a third of the pairs, swapping the order changed its answer.

So a local 4B judge can rank labels in bulk but cannot vouch for one card. The fix we plan is an output-side score with no judge in the loop: how much the push shifts the probability of the tokens the label promises. Gur-Arieh et al. (2025) make the same argument: descriptions built from inputs miss what a feature does to the output.

IV – Can you hear a neuron?

LFM3-Audio-2B speaks by generating audio frames, 12 codes each, between its text tokens. The explorer can push a neuron on the audio-frame positions only. The words stay the same, and only the voice can change.

To find candidate neurons, we contrasted recordings:

• Female against male voice. The model spoke four neutral sentences in each of its two built-in voices. We replayed those frames through the backbone and ranked neurons by how differently they fired on the two voices.
• Emotional against neutral text, read aloud the same way.
• Two input-side contrasts: clips sped up against slowed down, and louder against softer.

That gave 16 candidates. We measured each one's pitch (median f0), speaking rate, loudness and intelligibility (word error rate from the model's own speech recognition) on a fixed sentence.

Our first answer was wrong

The first pass used two seeds per dose and declared five neurons "pitch levers". Three problems sat behind that:

• The noise floor came from two unsteered clips, 3.9 Hz apart. Sixteen unsteered seeds of the same sentence spread with a standard deviation of 7.0 Hz (163 to 189 Hz). A single-seed before/after comparison therefore carries about 10 Hz of noise on its own.
• Random neurons did just as well. Neurons we had picked at random as controls swung pitch by 20 to 40 Hz at single doses.
• Two of the five were carried by one seed. One of them gave 177.8 Hz on one seed and 210.5 Hz on the other at the same dose.

We retracted the first pass and ran it again properly.

The rerun: 16 seeds against 16 random neurons

The rerun covered the five neurons plus one dropped earlier, and 16 random neurons as controls. We pushed each one by +1 and by −1 on 16 seeds: 912 clips, each transcribed by the model's own speech recognition.

• The measure: pitch at +1 minus pitch at −1, on the same seed.
• The test: a candidate counts if the sign holds across seeds and its effect is large compared with the random neurons. Both tests are corrected for testing six candidates.

• Three survive: L13/N2426 (+37 Hz between −1 and +1, 95 % interval 32 to 42), L19/N5684 (+32 Hz, 25 to 39) and L16/N3753 (+26 Hz, 21 to 31).
  • All 16 seeds agree in sign, and each is larger than every one of the 16 random neurons (the largest of those is 22.5 Hz).
  • The result holds on the 14 seeds the first pass never saw.
• Three do not: L14/N3240, L17/N65 and L1/N2912 are no larger than random neurons.
• The null is the lesson.
  • At dose 1 about half of all neurons move pitch detectably: 7 of the 16 random ones passed the same-sign test at p < 0.05.
  • One random neuron, L23/N1403, raises pitch by about 60 Hz whichever way it is pushed, and garbles the words. That is disruption, not a lever.
  • "This neuron changes pitch" means nothing without the random neurons beside it.

A lever also has to survive free speech. We pushed each survivor in eight spoken chat turns per direction. A turn failed if it ran on (more than 1.6 times slower per word than the unsteered turn) or if its words got garbled (more than 35 % of words wrong in the model's own transcription).

neuron	push up (+1)	push down (−1)
L16/N3753	+13 Hz, clean	−12 Hz, clean
L19/N5684	+25 Hz, clean	−8 Hz, words garbled in 4 of 8 turns
L13/N2426	+27 Hz, runs on in 6 of 8 turns	−10 Hz, clean

So L16/N3753 works both ways, L19/N5684 only for raising pitch and L13/N2426 only for lowering it. The strongest push is not the usable one.

Listen

The fixed sentence in the built-in female voice. We chose seeds 42, 43 and 44 (the first three) before listening to anything. The pitch given is the clip's median f0.

Seeds 43 and 44

Seed 43

Unsteered · 174 Hz

L16/N3753 down (−1) · 167 Hz

L16/N3753 up (+1) · 183 Hz

L19/N5684 up (+1) · 203 Hz

L13/N2426 down (−1) · 174 Hz

Seed 44

Unsteered · 173 Hz

L16/N3753 down (−1) · 170 Hz

L16/N3753 up (+1) · 189 Hz

L19/N5684 up (+1) · 209 Hz

L13/N2426 down (−1) · 177 Hz

L13/N2426 on seed 44 shows why one clip proves nothing: pushed down, it came out above the unsteered clip, 177 against 173 Hz. Its effect is an average over 16 seeds. The unsteered clip of seed 42 also came out byte for byte the same in both passes, hours apart, which is section V in miniature.

The dose sweep of L19/N5684 on seed 42 alone: an illustration, not evidence. The evidence is the 16-seed chart above.

Three more things came out of the rerun:

• Every push slows speech a little. On average, over all 22 neurons, it costs 0.09 words per second (95 % interval 0.05 to 0.13). The first pass had put this at 0.30, another two-seed artifact, and its claim that pushes flatten the pitch range did not replicate.
• The sign flips. For all three survivors, pushing toward the side that the female voice (or emotional text) activates lowers pitch. Three neurons make that an observation, not a finding.
• Rate, loudness and emotion: nothing. No candidate from the rate or loudness contrasts moved its own target measure. We measured no emotion, so we claim none.

The voice verdict is a narrow go: three pitch levers, not yet a voice atlas. From here on, every voice card needs at least 16 seeds and a set of random neurons measured the same way.

V – Does a card's test replay?

This is the part an open atlas depends on. If a stranger's machine claims "pushing L19/N5684 by +1 does X", anyone should be able to check it without trusting the stranger.

Each card pins its test:

• Text test: the model revision, the engine commit, the prompt, the push (layer, neuron, dose) and greedy decoding. It expects the SHA-256 of the generated token ids.
• Voice test: it also pins the speech binary, the voice and the seed. It expects the SHA-256 of the WAV bytes and of the 12 codes of every audio frame, plus the acoustic measures with tolerances.

replay	result
Mac, a new process	300/300 text tests: same token ids. 51/51 voice tests: same WAV bytes and frame codes
Mac, another new process	300/300 and 51/51 identical
Mac, CPU interpreter instead of Metal	300/300 text tests: same token ids
Mac Metal against rig CUDA (another engine commit, unsteered speech, two seeds)	74 of 74 and 72 of 72 frames identical, all 12 codes; WAV within 1 bit of 16 (SNR 94 to 95 dB)

What this means for a shared atlas:

• Within one build, a card verifies by hash. A mismatch means a different model, engine, push or prompt, not noise.
• Across backends, the discrete outputs hold. Token ids and audio codes agree; the floats and the WAV need tolerances. So a card hashes the codes and gives the audio as measures with tolerances.
• Checking is about ten times cheaper than labelling. A text test takes 1.4 seconds on the Mac and a voice test about 6. A network of contributors can check every card two or three times for less than it cost to make them.

Caveat: we have not yet replayed a steered voice test on CUDA. The rig's speech build predates steering, so the cross-backend row above is unsteered speech. The audio codes are sampled (temperature 1.0), so getting identical codes on both backends means the seeded draws matched, not only the greedy choices.

We found no interpretability platform that ships a test with each label that anyone can re-run and check. Neuronpedia holds a large set of community explanations and supports steering, and Transluce has described every neuron of Llama-3.1-8B; their descriptions are text.

VI – Do families help?

The 27B grouped the 300 labels into families ("Markdown formatting", "terminal commands", "ocean", "cooking", "ordinal numbers"). We dosed each family of three or more against its best single member.

• The family won in 2 of 12. Winning meant more effect on the family's target tokens at no more than 1.1x the perplexity.
• Full doses mostly bought a little more effect at a large cost in coherence. For Markdown, the perplexity ratio went from 1.5 to 7.0; for cooking, from 4.9 to 12.3.
• Same topic, different function. Two members of "Universal Quantifiers" sit in different layers and do not add up.

Families need functional evidence: neurons that fire together over the corpus, neurons that write in similar directions, or attribution graphs. Patel et al. (2026) group attribution-graph nodes, which are functional by construction. A list of descriptions is not.

VII – Does a label transfer to another model?

We ran 500 prose excerpts through LFM3 and through Qwen3.5-4B, a text-only model with a different tokenizer and 294,912 MLP neurons. We aligned the two on words: 20,499 word positions. For each labelled LFM3 neuron we took its best-correlated Qwen neuron. The null was the same search with the LFM3 words shifted by three positions.

• Correlation: the median best match is r = 0.35, against 0.14 for the null; 60 % of neurons beat the null's 95th percentile.
• Labels: the 4B labelled the matched Qwen neurons from their own top excerpts, without seeing the LFM3 label. For the 40 best pairs, a blind judge said both labels describe the same feature in 30 of 40, and in 0 of 40 when the pairs were shuffled.

LFM3 label	Qwen3.5-4B label
shadows stretching across space	elongation or extension
finality of a desired outcome	conclusion of a process or event
trip or trips	trip or journey tokens
reflexive verb "sich" or "se"	passive reflexive verbs

The judge was strict: "the word even" against "concessive adverb 'even'" counted as different.

This extends the "Rosetta neurons" line of work (Dravid et al. 2023; 2606.03990) to a speech model against a text model with another tokenizer. For an atlas it is a free second opinion: a label two models agree on is better evidence than either alone.

VIII – What it would take

At the measured rates, labelling and simulating every one of the 184,320 neurons would take about 10 days on this Mac. The causal check would bring it to 28 days. Spread over 50 contributors' Macs running the same pinned build, that is about 5 and 14 hours.

Compute is not the limit. Two other things are:

• Most neurons are not labelable this way. At 12 % on random neurons, about 160,000 neurons would get labels that predict nothing. Better to label the selective ones, about 9,000 at the top 5 % (about 4 days here), and describe the rest by what they do to the output, or not at all.
• A small judge cannot certify a card. The budget saved belongs in better causal checks.

So the order we will build in:

1. The card format and the replay command. They have the strongest evidence and are the part no one else has.
2. Labels for the selective neurons only, each with its simulation score and its shuffled baseline on the card.
3. An output-side causal score in place of the A/B judge.
4. Cross-model transfer between our two explorers, LFM3 and Qwen3.5-4B.
5. Voice cards only with 16 seeds and random neurons measured the same way, starting from the three pitch levers.

Credits and notes

• Model. LFM3-Audio-2B by Liquid AI, under the LFM Open License v1.0. See Liquid AI's post Introducing LFM3: Every Modality for Specialized Intelligence. Liquid AI did not take part in or endorse this work.
• Explorer. The steering explorer is our Rust port of lyramakesmusic/neuron-explorer.
• Audio. Every clip on this page is synthetic. It was generated locally with one of the model's two built-in voices, with the activations steered as stated. No voice was cloned.
• Local models. The labels, simulations and judgments came from Qwen3.5-4B and a Qwen3.8 27B, run locally in LM Studio.
• Prior art we read:
  • Transluce, Scaling automatic neuron description, and Language Model Circuits Are Sparse in the Neuron Basis;
  • Paulo et al., Automatically Interpreting Millions of Features in LLMs;
  • Marin-Llobet et al., InterpAgent;
  • Zhao, Schuller and Sisman, Emotion-Sensitive Neurons in Large Audio-Language Models, and Neuron-Level Emotion Control in Speech-Generative LALMs.