Testing features with a simple classifier (linear probes)
Why it matters Nearest neighbours gave a rough answer. The standard test of learned features is a linear probe: freeze the features, train only a linear classifier on top with a few labels, and measure it on new photos. If a straight line can separate the objects, the features did the hard work.
The probe is the softmax regression of Module 3, on features instead of pixels:
- Standardise each feature with the training set's mean and spread (and use the same numbers on the test set).
- Gradient descent on cross-entropy, all training rows at once: 1000 steps, step size 0.5, weight decay 0.001.
- Accuracy on the test photos.
It is deliberately weak: a linear model with 400 labels. That is the point. It cannot invent good features; it can only read what the features already say.
Words and symbols in this lesson
- linear probe
- a linear classifier trained on frozen features: a test of what the features already say
- SimCLR
- self-supervised training with two views of each picture and InfoNCE (Chen et al. 2020)
Write my_probe(F_train, y_train, F_test, y_test) as described (weights and biases start at zero). Then fill accuracy for a random encoder (ssl.Encoder(32, seed=0)) and the recorded SimCLR encoder with your probe. The raw-pixel baseline (768 numbers per photo) is computed for you with the course's probe, which does the same arithmetic in float32 on the fast matrix multiply: your float64 version would take half a minute on 768 inputs in the browser.
Spotted a problem in this lesson? Report it · Ask about this lesson
This exercise opens soon
Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.
This browser is blocking cookies for learn.floati.life, so lessons you finished here cannot open in order. Allow cookies for this site and reload.