Syllabus Lesson 90 of 246 · Module 7: Working on new data
Module 7: Working on new data

Early stopping

Why it matters The cheapest regulariser of all: stop training when the model stops improving on validation data.

Validation loss is noisy: it can tick up for an epoch and then fall again. So do not stop at the first rise. Use patience: stop once the validation loss has failed to beat its best for patience epochs in a row, and keep the model from the best epoch.

Tiny numbers

Losses 1.0, 0.8, 0.7, 0.72, 0.69, 0.71, 0.73, 0.74 with patience 2: the best is 0.69 at epoch 5. Epochs 6 and 7 fail to beat it, so stop after epoch 7 and keep epoch 5.

Words and symbols in this lesson
patience
how many epochs without improvement before stopping
Your turn

Write early_stop(val_losses, patience) returning (stop_epoch, best_epoch), both counting from 1. If it never runs out of patience, stop_epoch is the last epoch.

Spotted a problem in this lesson? Report it · Ask about this lesson

This exercise opens soon

Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.