SAE and TensorLens: The Age of Feature Interpretability
Individual neurons are uninterpretable. Sparse Autoencoders extract monosemantic features from model internals, and TensorLens analyzes the entire Transformer as a single unified tensor.
SAE and TensorLens: The Age of Feature Interpretability
In the previous two posts, we:
- Logit/Tuned Lens: Read the model's intermediate predictions
- Activation Patching: Traced which activations are causally responsible for the answer
But here we hit a fundamental problem:
What do the activations we observe and manipulate actually mean?
Each dimension of an activation vector corresponds to an individual neuron. But these neurons are polysemantic -- a single neuron fires for academic citations, English dialogue, HTTP requests, and Korean text simultaneously. Clean interpretation at the neuron level is impossible.
This post covers two modern approaches that address this problem:
- Sparse Autoencoder (SAE): Decompose dense activations into sparse, monosemantic features
- TensorLens: Unify the entire Transformer computation into a single high-order tensor
This part is for subscribers
A subscription unlocks every premium series and its Jupyter notebooks.
You need a free account to subscribe. Cancel anytime.
Related Posts
Paper of the Week #3 — Half the FLOPs Is Not Half the Time
One integer halves a fine-grained MoE's expert compute (arXiv 2609.04575) and its Table 5 replicates on one A100 to within a point. The paper never reports time, so I measured it: nothing in HF transformers, nothing at batch 1 in the vLLM you run today, 1.35x at batch 8. Plus the OLMoE control and the iso-cost harness control promised in issue #2.
The paper stopped at half the experts. A quarter loses four points instead of thirty, and a model trained without renormalization tells you why.
Two cells arXiv:2609.04575 never ran: k₁=2 and k₁=3 on Qwen3.6-35B-A3B, and the whole k₂ trick on OLMoE, which was trained without renormalization. The second one is a clean test of the paper's mechanism.
One integer halves MoE expert compute. We measured the speed the paper didn't: free at batch 8, not at batch 1.
We reproduced Table 5 of arXiv:2609.04575 on one A100 and measured the throughput the paper leaves out: nothing in HF transformers, nothing at batch 1 in stock vLLM, 1.35× at batch 8.