LESSON 5.9 · Theory · 100 min
Look inside activations: an interpretability entry point
Look inside activations: an interpretability entry point. Learn probing, activation patching, circuits, and SAE intuition and complete: Run an activation-patching experiment with T
Learning objectives
- Explain what problem “Look inside activations: an interpretability entry point” solves without hiding behind terminology.
- Trace the variables and causal links across probing, activation patching, circuits.
- Complete “Run an activation-patching experiment with TransformerLens” and judge the result with evidence rather than intuition.
Core concepts
probing
probing is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment before moving to an optimized implementation.
activation patching
activation patching is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
circuits
circuits is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
and SAE intuition
and SAE intuition is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
Build and verify
Run an activation-patching experiment with TransformerLens
- Predict: write the expected output, trend, or failure before running code.
- Build: implement only the minimum components needed to answer the question.
- Verify: compare with a baseline or trusted implementation; save seeds, parameters, and raw outputs.
- Transfer: change one shape, dataset, scale, or workload condition and explain whether the conclusion still holds.