Beyond accuracy
Why it matters On a rare-disease test where 95% of people are healthy, a model that always says "healthy" scores 95% accuracy and is useless.
A confusion matrix counts, for every true class, how often each class was predicted: row = truth, column = prediction. From it, for one class you care about:
- Precision: of the times the model said "positive", how often was it right?
- Recall: of the real positives, how many did the model find?
Tiny numbers
100 people, 5 sick. The lazy model says healthy for all: accuracy 95%, recall 0 of 5 = 0. A real model flags 8 people, 4 of them sick: precision 4/8 = 0.5, recall 4/5 = 0.8, and accuracy (4 + 91) / 100 = 95%. Same accuracy, completely different usefulness.
Words and symbols in this lesson
- confusion matrix
- counts of true class (row) against predicted class (column)
- precision, recall
- of what it flagged, how much was right; of what was there, how much it found
Write confusion(y_true, y_pred, k) (a k by k array of counts) and precision_recall(y_true, y_pred, positive) returning a pair (use 0.0 when there is nothing to divide by).
Spotted a problem in this lesson? Report it · Ask about this lesson
This exercise opens soon
Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.
This browser is blocking cookies for learn.floati.life, so lessons you finished here cannot open in order. Allow cookies for this site and reload.