Point the checker at your engine
Why it matters An engine with one wrong rule will train, slowly and badly, and you may never know. From now on, every new operation is checked before it is used.
Your Module 3 checker nudged NumPy arrays. Here, nudge a Tensor's data in place, rebuild the calculation, and compare with what backward left in .grad.
The recipe: (1) build the output and call backward() once to get the engine's gradients. (2) For each input and each entry, nudge data up and down, rebuild, and read the output's value. (3) Report the largest relative error.
Recap Never trust a new backward rule until it has passed a check.
Words and symbols in this lesson
- Tensor
- here: an array plus its gradient plus the Tensors it came from
- .grad
- the gradient of the final loss with respect to this Tensor
- unbroadcast
- summing a gradient back down to the shape it was stretched from
Write engine_check(build, inputs): build() returns a scalar Tensor computed from the Tensors in the list inputs. Return the largest relative error between the engine's gradients and nudged ones. Reset each input's grad to zeros before calling backward. The hidden test plants three bugs in the engine, one at a time; your checker must report above 1e-3 for each, and below 1e-6 for the correct engine.
Spotted a problem in this lesson? Report it · Ask about this lesson
This exercise opens soon
Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.
This browser is blocking cookies for learn.floati.life, so lessons you finished here cannot open in order. Allow cookies for this site and reload.