Walking downhill (gradient descent by hand)
Why it matters Picking weights by hand does not scale to a million weights. The gradient tells every weight which way is downhill at once.
Name it
For the loss L = mean((X @ w + b - y) squared), nudge w1 and L changes by 2 * mean(error * x1), where error is prediction minus truth. That is the derivative of L with respect to w1. All at once: grad_w = 2 * X.T @ error / n, and grad_b = 2 * mean(error).
Read it in plain words
If the predictions are too high (error positive) on houses with a large size, then lowering w1 helps, and the gradient for w1 is positive, so we subtract it. Each step: w = w - lr * grad_w, where lr, the learning rate, sets the step size.
Gradient descent: repeat "compute the gradient, take a small step against it". Learning rate: the size of each step.
Call the Curve
Before you run your training, guess the loss at steps 0, 10, 50 and 200 in my_guess. The code prints your guess beside the truth.
Recap Gradient, step against it, repeat.
Words and symbols in this lesson
- weights (w, W), bias (b)
- the numbers a neuron learns; the bias shifts its output
- learning rate (lr)
- how big each training step is
Write gradients(X, y, w, b) returning (grad_w, grad_b), and train(X, y, lr, steps) starting from zeros and returning (w, b, losses), where losses holds the loss before each step. Fill in my_guess first. The trained loss must end below 0.01 with lr 0.5 and the 201 steps the starter asks for.
Spotted a problem in this lesson? Report it · Ask about this lesson
This exercise opens soon
Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.
This browser is blocking cookies for learn.floati.life, so lessons you finished here cannot open in order. Allow cookies for this site and reload.