Syllabus Lesson 190 of 246 · Module 17: Noise in, pictures out (diffusion)
Module 17: Noise in, pictures out (diffusion)

Steering the sample (classifier-free guidance)

Why it matters "Draw a 7" works because of one line of arithmetic. Every text-to-image model uses it to make pictures follow the prompt, and the same line explains why turning it up too far makes pictures worse.

The digit model was trained with its label, and 15% of the time with the label hidden (a "no label" token, difflab.NULL). So one network can predict the noise both ways. Classifier-free guidance (Ho and Salimans, 2022) mixes them:

eps = (1 + w) eps_with_label - w eps_without_label

The difference eps_with_label - eps_without_label points towards "more like this label". w = 0 is plain conditional sampling; bigger w pushes harder. Watch the scene: the same noise drawn with w = 0, 1, 2 and 4.

Words and symbols in this lesson
eps
the bell-curve noise that was added; the network learns to predict it
DDIM
sampling by jumping between a few steps, with no fresh noise
classifier-free guidance
(1 + w) x noise predicted with the label - w x noise predicted without it
Your turn

Write guided(model, x, t, labels, w). Then fill table: for w in 0, 1, 2 and 4, the share of 500 samples (20 DDIM steps, which the last lesson showed are as good as 100 DDIM steps) that read as the requested digit, and their closeness. Set best_w to the w with the best (lowest) closeness among those reaching at least 0.99 requested.

Spotted a problem in this lesson? Report it · Ask about this lesson

This exercise opens soon

Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.