Syllabus Lesson 51 of 246 · Module 4: Your own autograd engine
Module 4: Your own autograd engine

Hook: ten lines that train a digit reader

Why it matters In Module 3 you wrote every gradient by hand. Real networks have dozens of layer types; nobody derives those by hand each time. An autograd engine does it for you, and by the end of this module you will have written one.

The first lines import the finished engine you are about to build (it is kept out of sight, so the coming lessons stay yours to solve). Look at the ten lines after the data: they build a network, and train it on digits, and nowhere do they mention a gradient formula. loss.backward() works them all out.

Call the Curve first: guess the mean training loss after epoch 1 and after epoch 10.

Autograd: automatic differentiation. You write only the forward calculation; the engine records it and computes every gradient by running the chain rule backward.

Words and symbols in this lesson
autograd
automatic gradients: you write the forward pass, the engine does the backward
Tensor
here: an array plus its gradient plus the Tensors it came from
zero_grad
resetting gradients before the next step, because backward adds to them
Your turn

Fill in my_guess, then run. Read the ten training lines and notice what is missing.

Spotted a problem in this lesson? Report it · Ask about this lesson

This exercise opens soon

Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.