Syllabus Lesson 63 of 246 · Module 4: Your own autograd engine
Module 4: Your own autograd engine

Gate: add a new operation (AI off)

Why it matters Adding an operation, with its backward rule, and checking it, is exactly what you will do for every new layer in Leg 3. Here you do it alone. Already know this module? You can take this gate first, before the other lessons: passing it marks this module done and opens the next one.

AI tutor off. Add tanh, exp and log to your engine, each with its backward rule. Work out each one's local slope yourself (Module 2's slopes, and check each with nudging before you trust it).

Words and symbols in this lesson
Tensor
here: an array plus its gradient plus the Tensors it came from
.grad
the gradient of the final loss with respect to this Tensor
unbroadcast
summing a gradient back down to the shape it was stretched from
zero_grad
resetting gradients before the next step, because backward adds to them
Your turn

Write the three methods on Tensor. The hidden test gradient-checks a formula using all three on new inputs.

AI off: no tutor, hints or worked solution on this one. Your own thinking only.

Spotted a problem in this lesson? Report it · Ask about this lesson

Code · runs in your browser
Output

        

Your progress is counted anonymously. Change this