Hook: ten lines that train a digit reader
Why it matters In Module 3 you wrote every gradient by hand. Real networks have dozens of layer types; nobody derives those by hand each time. An autograd engine does it for you, and by the end of this module you will have written one.
The first lines import the finished engine you are about to build (it is kept out of sight, so the coming lessons stay yours to solve). Look at the ten lines after the data: they build a network, and train it on digits, and nowhere do they mention a gradient formula. loss.backward() works them all out.
Call the Curve first: guess the mean training loss after epoch 1 and after epoch 10.
Autograd: automatic differentiation. You write only the forward calculation; the engine records it and computes every gradient by running the chain rule backward.
Words and symbols in this lesson
- autograd
- automatic gradients: you write the forward pass, the engine does the backward
- Tensor
- here: an array plus its gradient plus the Tensors it came from
- zero_grad
- resetting gradients before the next step, because backward adds to them
Fill in my_guess, then run. Read the ten training lines and notice what is missing.
Spotted a problem in this lesson? Report it · Ask about this lesson
This exercise opens soon
Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.
This browser is blocking cookies for learn.floati.life, so lessons you finished here cannot open in order. Allow cookies for this site and reload.