Passing a nudge along (the chain rule)
Why it matters A network is functions inside functions. The chain rule says how a nudge at the start travels to the end. Backpropagation is nothing more than this.
See it
Three steps: double x, then add 1, then square. Start at x = 2: double gives 4, add 1 gives 5, square gives 25.
Now nudge x by a tiny amount. Doubling makes the nudge 2 times bigger. Adding 1 leaves it the same size (times 1). Squaring at 5 multiplies it by 2 times 5 = 10. So the end moves 2 times 1 times 10 = 20 times the nudge.
Chain rule: the slope of a chain of steps is the product of each step's own slope, each measured where that step is.
Check by nudging: x = 2.001 gives 2 * 2.001 + 1 = 5.002, squared 25.020004. The end moved 0.020004 for a nudge of 0.001: 20 times. It works.
Recap Multiply the local slopes along the chain.
Words and symbols in this lesson
- chain rule
- the slope of steps in a row is the product of each step's own slope
The chain is g(x) = (3x - 2) ** 2, then h(u) = u ** 3, so the end is h(g(x)). Write chain_slope(x) that returns the slope of the end with respect to x using the chain rule (multiply the local slopes), not by nudging. Notice that g is itself two steps, first 3x - 2 and then squaring, so the whole chain has three steps: multiply all three local slopes, each measured where its step is.
Spotted a problem in this lesson? Report it · Ask about this lesson
This exercise opens soon
Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.
This browser is blocking cookies for learn.floati.life, so lessons you finished here cannot open in order. Allow cookies for this site and reload.