Gate: evaluate a new pair of models (AI off)
Why it matters Making a sound call, with an interval, on fresh results is what this module is for.
AI tutor off. Two new models, 1,000 shared test items.
Words and symbols in this lesson
- bootstrap
- resampling your test set with repeats to see how much a score wobbles
Write paired_interval(a, b), the 95% paired bootstrap interval of B minus A (2,000 resamples, seed 0, as in the last lesson), and make_call(a, b), returning "A", "B" or "no clear difference". Then set interval and call for this pair. The check also runs your two functions on three more pairs of models it makes up itself.
AI off: no tutor, hints or worked solution on this one. Your own thinking only.
Spotted a problem in this lesson? Report it · Ask about this lesson
This exercise opens soon
Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.
This browser is blocking cookies for learn.floati.life, so lessons you finished here cannot open in order. Allow cookies for this site and reload.