One finding inside your model, with a control
Why it matters Module 21's lesson, applied to your own model: a probe's score means nothing until you compare it with what the same probe reads from a network that never trained.
The probe set for your track asks one question about each input (meaning): on the language track, "is the next character a vowel?"; on the images track, "which object is in the photo?".
Train your baseline (seed 0). Read its hidden layer (studio.hidden_units(model, X)) and train a linear probe on the first half of the probe set, testing on the second (ssl.probe, 300 steps). The control: the same probe on an untrained network of the same shape and seed (studio.MLP(track.n_in, width, track.n_out, seed=0)). Random ReLU units already turn inputs into many features, and a probe can read a lot from those; only the part above the control is something training put there.
Words and symbols in this lesson
- baseline
- the plain run every change is measured against, over the same seeds
Fill trained, control and base_rate (the share of the most common label in the test half). Set finding to "above control" if trained beats control by more than 0.02, else "not above control".
Spotted a problem in this lesson? Report it · Ask about this lesson
This exercise opens soon
Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.
This browser is blocking cookies for learn.floati.life, so lessons you finished here cannot open in order. Allow cookies for this site and reload.