Gate: find where the information is (AI off)
Why it matters Patching is how most circuit claims are made. Doing it from scratch, on inputs you did not choose, is the test that you can make one.
Gate rules: AI tutor off. Write restoring_positions(clean, bad, answer, site) for the recorded GPT. clean and bad are two strings of the same length that differ somewhere; answer is the character clean predicts next.
Let p_clean and p_bad be the probability of answer after the clean and the corrupted string. A position restores the answer when patching the clean value there (at site) gives a probability of at least p_bad + keep x (p_clean - p_bad). Return the list of such positions, smallest first.
The check runs your function on three new pairs from held-out text, different on every check, at 'resid0', 'resid1' and 'final', and compares with its own patching.
Fill restoring_positions. Try it on ("the loss went dow", "the loss went bow", "n").
AI off: no tutor, hints or worked solution on this one. Your own thinking only.
Spotted a problem in this lesson? Report it · Ask about this lesson
This exercise opens soon
Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.
This browser is blocking cookies for learn.floati.life, so lessons you finished here cannot open in order. Allow cookies for this site and reload.