Syllabus Lesson 86 of 246 · Module 7: Working on new data
Module 7: Working on new data

Too simple, too complex, measured

Why it matters A model is only useful on data it has not seen. Measuring that, instead of the training score, is the first habit of anyone who builds models.

Fit polynomials of increasing degree to 30 noisy points from a smooth curve. Degree 1 is a straight line: too simple, it misses the bend (underfitting). Degree 15 bends to follow the noise in these particular 30 points: its training error is the lowest of all (0.014), but on new points it does worse than degree 3 (0.075 against 0.051). That gap, training error far below the error on new data, is overfitting. Somewhere between is a sweet spot.

The training error only ever falls as the degree rises, so it cannot find the sweet spot. The validation error, on points the fit never saw, can.

Validation set: data held back from training and used to choose settings such as model size.

Words and symbols in this lesson
overfitting, underfitting
fitting the noise in the training data; or too simple to fit the pattern
validation set
held-back data used to choose settings
Your turn

Write errors(deg) returning (train_mse, val_mse) for a degree-deg polynomial fitted with np.polyfit on the 30 points. Then set best_degree to the degree from 1 to 15 with the lowest validation error, computed from your own curve.

Spotted a problem in this lesson? Report it · Ask about this lesson

This exercise opens soon

Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.