Syllabus Lesson 191 of 246 · Module 17: Noise in, pictures out (diffusion)
Module 17: Noise in, pictures out (diffusion)

Teaching a trained model something new (fine-tuning)

Why it matters Nobody trains a big image model from scratch to get a new style: they fine-tune one that already knows how pictures work. A few hundred steps on new data can be enough, because most of what the model knows still applies.

The new task: mirror writing. Every training digit is flipped left to right. The trained digit model has never seen one.

difflab.train_digits(steps=..., model=model, data=(pictures, labels)) keeps training an existing model on your data.

How do you check it learned mirror writing and not something else? Flip its samples back: if they read as the requested digits, it draws mirrored digits. And the un-flipped samples should now mostly fail to read correctly, except for digits that look the same in a mirror (0, 1 and 8 nearly do): if most still read correctly, it has not changed much.

Words and symbols in this lesson
eps
the bell-curve noise that was added; the network learns to predict it
DDIM
sampling by jumping between a few steps, with no fresh noise
fine-tuning
continuing to train a trained model on new data
Your turn

Write mirror(pictures) (rows of 64 pixels, flipped left to right), fine-tune the model on mirrored training digits for at most 400 steps (a few seconds; 300 is enough), and fill after with: the share of 500 samples (w = 2, 20 DDIM steps) that read as the requested digit when flipped back, the share that read correctly without flipping, and the closeness of the flipped-back samples.

Spotted a problem in this lesson? Report it · Ask about this lesson

This exercise opens soon

Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.