Arrows back to the data (See it)
Why it matters Removing noise sounds like magic until you see what the network actually learns: for any noisy point, an arrow pointing to where the clean data probably was. Generation is following those arrows home.
The network is trained to predict the noise eps that was added. Turn the shortcut around and its guess becomes a guess of the clean point:
x0_estimate = (x_t - sqrt(1 - abar_t) eps_predicted) / sqrt(abar_t)
Draw an arrow from each noisy point to its x0 estimate and you get a field of arrows. Watch the scene: the same grid of points, at a very noisy step and at a nearly clean step.
Predict first: at which step do the arrows agree more with each other, all pointing to roughly the same place: step 60 (very noisy) or step 10 (nearly clean)?
Words and symbols in this lesson
- abar
- the share of the original signal left after a number of steps (its square root multiplies x0)
- x0, x_t
- the clean data, and the same data after t steps of noise
- eps
- the bell-curve noise that was added; the network learns to predict it
Set my_guess to 60 or 10. Train a model (difflab.train('spiral'), a few seconds), then for the 11 x 11 grid compute the x0 estimates at step 60 and at step 10. Fill spread (the spread of the estimates, est.std(axis=0).mean()) and near (their mean distance to the spiral, difflab.nearest_distance(est).mean()) for both steps. Then set agree_more to the step (60 or 10) where your numbers show the estimates agree more. Your guess stays as you made it.
Spotted a problem in this lesson? Report it · Ask about this lesson
This exercise opens soon
Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.
This browser is blocking cookies for learn.floati.life, so lessons you finished here cannot open in order. Allow cookies for this site and reload.