Syllabus Lesson 33 of 246 · Module 2: Maths and probability, only as needed
Module 2: Maths and probability, only as needed

Which guess explains the data best (likelihood)

Why it matters Almost every loss in this course is secretly the same idea: choose the model under which the data you saw is most likely.

Tiny numbers

You flip a coin 10 times and see 7 heads. Is the coin fair? Ask: if the chance of heads were p, how likely is exactly this sequence? For p = 0.5 it is 0.5 to the 10th, about 0.00098. For p = 0.7 it is 0.7 to the 7th times 0.3 to the 3rd, about 0.0022, more than twice as likely. Scan every p from 0 to 1 and the best is exactly 0.7: the share of heads you saw.

Likelihood: how probable the observed data is under a particular setting of the model. Maximum likelihood: the setting that makes it most probable.

Products of many small probabilities become too small for a computer, so we add logarithms instead: the log-likelihood. The best setting does not change, because log only ever goes up when its input goes up.

Recap Pick the setting that makes what you saw least surprising.

Words and symbols in this lesson
likelihood
how probable the data you saw is under one setting of the model
Your turn

Write log_likelihood(p, heads, tails) and best_p(heads, tails), which scans p over np.linspace(0.001, 0.999, 999) and returns the p with the highest log-likelihood.

Two log rules you need here: the log of a product is the sum of the logs, log(a × b) = log a + log b, and so a power comes out in front, log(p7) = 7 log p.

Spotted a problem in this lesson? Report it · Ask about this lesson

This exercise opens soon

Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.