Reusing a trained network (transfer learning)
Why it matters You rarely have lots of labelled data for your own task. A network trained on a big related dataset already knows useful features; reusing it lets a tiny dataset go a long way.
The plan: train the conv layers on a big, clean shapes dataset. Then face a new, harder task with only 40 labelled images: the same shapes, but much noisier. Keep the trained conv layers frozen and train only a new final layer on features averaged over the whole image (position-free features generalise from few examples). Compare with training the whole network from scratch on the 40 images.
Frozen: layers whose weights are not updated during training. Fine-tuning: continuing to train a pretrained network on new data.
Words and symbols in this lesson
- transfer learning
- reusing layers trained on one task for another
- RECORDED
- a real run the course trained once and ships for you to study
The pretraining was done once by the course and its weights are loaded for you (a recorded network). Write train_head(w1, w2, X, y, steps) that trains only a new 16-to-4 output layer on top of frozen conv layers (Adam at 1e-2 works) and returns a predict function, and train_scratch(X, y, steps) that trains all three layers from new random weights. The test runs both on 1,000 noisy test images. Your transfer model must beat the course's own scratch network (fixed settings: 8 and 16 channels, Adam 3e-3, 300 steps, about 58%) by at least 8 points; the course's transfer run gets about 72%. Your own scratch network just has to learn something (above 35%, where chance is 25%). If yours beats the course's scratch network, well done: tuning scratch training well is a real skill, and it is also why a fair comparison fixes the settings in advance.
Spotted a problem in this lesson? Report it · Ask about this lesson
This exercise opens soon
Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.
This browser is blocking cookies for learn.floati.life, so lessons you finished here cannot open in order. Allow cookies for this site and reload.