Seeds and spread
Why it matters Train the same model twice with different seeds and you get different scores. Until you know how big that wobble is, you cannot tell whether any change helped.
Here the seed decides three things: which images land in the training and test sets, the starting weights and the order of the data. None of them should matter, but all of them do, a bit. So always train several seeds, and report the mean and the spread (standard deviation), never a single run.
A rough rule for comparing two settings, each averaged over n seeds: the difference of the two averages wobbles by about the spread times sqrt(2 / n) (each average wobbles by spread / sqrt(n), and two independent wobbles add in squares). A difference smaller than about twice that is not one you can trust. With 5 seeds and a spread of 0.02, that is 2 times 0.02 times sqrt(0.4) = 0.025.
Write seed_report(n_seeds) returning (mean, std) of train_and_score over seeds 0 to n_seeds - 1. Use np.std as it is (it divides by n; dividing by n - 1, ddof=1, is also accepted). Then set wobble to the spread over 5 seeds.
Spotted a problem in this lesson? Report it · Ask about this lesson
This exercise opens soon
Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.
This browser is blocking cookies for learn.floati.life, so lessons you finished here cannot open in order. Allow cookies for this site and reload.