A probe needs a control
Why it matters A probe (a small linear classifier trained on a model's inside vectors) is the most common tool in interpretability, and the most over-read. High accuracy can mean the model computed the thing, or that the probe did.
You met linear probes in Module 18. Here, train one on the GPT's vectors at two places: 'embed' (the character's embedding plus its position, before any layer) and 'resid1' (after the first block). Three questions, each a 0/1 label for every character:
'vowel': is this character a vowel?'control': is it one of b, c, g, n, r, s, t? A grouping that means nothing, picked to be as common as the vowels (28.7% of characters, against 28.8%).'third letter': is it the third letter of its word, or later?
Predict first: if the vowel probe scores close to 100%, does that show the model has learned what a vowel is?
Words and symbols in this lesson
- interpretability
- finding out what a trained network computes inside, and checking it
- probe
- a small linear classifier trained to read something from a model's inside vectors
Set my_guess to "yes" or "no". Fill scores[kind][site] with probe(site, kind) for the three kinds and the two sites. Then set computed to the one kind whose probe shows something the layer worked out (it is far easier after the block than from the embedding alone).
Spotted a problem in this lesson? Report it · Ask about this lesson
This exercise opens soon
Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.
This browser is blocking cookies for learn.floati.life, so lessons you finished here cannot open in order. Allow cookies for this site and reload.