How you remove a part changes the answer
Why it matters "Removing this part raises the loss by X" sounds like a fact about the part. Part of it is a fact about how it was removed.
In Module 21 you removed parts of the GPT by setting their output to zero. But the rest of the network never saw a zero there in training: zero may be an input it has never met, so the damage measures the shock as well as the loss of information. A gentler removal replaces the output with its average over many inputs (mean ablation): the part stops carrying anything about this input, but the next layer still gets a value of the usual size.
Measure both for the four parts: 'attn_out0', 'mlp_out0', 'attn_out1', 'mlp_out1'. The average comes from cache[site].mean((0, 1)).
Predict first: for block 0's MLP, will mean ablation cost less than zeroing, or about the same?
Words and symbols in this lesson
- mean ablation
- replacing a part's output with its average instead of zero
Set my_guess to "less" or "same". Fill zero[site] and mean[site] with the rise in loss. Set most_overstated to the part where zeroing overstates the effect most.
Spotted a problem in this lesson? Report it · Ask about this lesson
This exercise opens soon
Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.
This browser is blocking cookies for learn.floati.life, so lessons you finished here cannot open in order. Allow cookies for this site and reload.