Gate: add a new operation (AI off)
Why it matters Adding an operation, with its backward rule, and checking it, is exactly what you will do for every new layer in Leg 3. Here you do it alone. Already know this module? You can take this gate first, before the other lessons: passing it marks this module done and opens the next one.
AI tutor off. Add tanh, exp and log to your engine, each with its backward rule. Work out each one's local slope yourself (Module 2's slopes, and check each with nudging before you trust it).
Words and symbols in this lesson
- Tensor
- here: an array plus its gradient plus the Tensors it came from
- .grad
- the gradient of the final loss with respect to this Tensor
- unbroadcast
- summing a gradient back down to the shape it was stretched from
- zero_grad
- resetting gradients before the next step, because backward adds to them
Write the three methods on Tensor. The hidden test gradient-checks a formula using all three on new inputs.
AI off: no tutor, hints or worked solution on this one. Your own thinking only.
Spotted a problem in this lesson? Report it · Ask about this lesson
Your progress is counted anonymously. Change this
This exercise opens soon
Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.
This browser is blocking cookies for learn.floati.life, so lessons you finished here cannot open in order. Allow cookies for this site and reload.