Syllabus Lesson 102 of 246 · Module 9: Is it really better?
Module 9: Is it really better?

Hook: which model is better?

Why it matters Teams ship the wrong model every week because one number on one test set looked higher. This module teaches you not to.

Two digit readers were tested on the same 100 test images. correct_A and correct_B hold 1 where each model was right and 0 where it was wrong. Model A scores 95%. Model B scores 91%.

Which would you ship? Decide now, honestly, before you learn anything else in this module. Lesson 8 reveals the truth, measured on 200,000 images.

(These two readers are simulated, so that the truth can be known exactly: every image has a difficulty, and each model's chance of reading it right depends on that difficulty. The results behave like real test results. Lesson 8 also says how these 100 images were picked.)

Your turn

Set my_pick to "A", "B" or "no clear difference".

Spotted a problem in this lesson? Report it · Ask about this lesson

This exercise opens soon

Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.