Unmixing: a sparse autoencoder
Why it matters If features are stored in superposition, looking at units will not find them. A sparse autoencoder (Bricken et al. 2023, Cunningham et al. 2023) tries to undo it: rebuild each vector from many more directions than it has numbers, but only a few at a time.
Take the GPT's 64-number vectors after block 1, for 19,200 characters of lesson text. A sparse autoencoder with 256 features computes f = relu(z We + be) and rebuilds z = f Wd. The loss is the rebuild error plus l1 times the total size of f, which pushes most features to exactly zero.
interp.train_sae(A, features=256, l1=...) trains one; sae.unexplained(A) is the share of the vectors' spread the rebuild misses (0 is perfect) and sae.features(A) gives f.
Predict first: with a strong push (l1 = 0.3), will the rebuild still explain most of the spread (unexplained below 0.2)?
Words and symbols in this lesson
- superposition
- storing more features than there are numbers, as directions that are rarely on together
- sparse autoencoder
- a wide layer trained to rebuild vectors from a few active features at a time
Set my_guess to "yes" or "no". Train at l1 0.03 and 0.3 (600 steps of 128 vectors each). Fill unexplained[l1] on the held-out vectors A_new, and active[l1]: the mean number of features above zero per held-out character.
Spotted a problem in this lesson? Report it · Ask about this lesson
This exercise opens soon
Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.
This browser is blocking cookies for learn.floati.life, so lessons you finished here cannot open in order. Allow cookies for this site and reload.