Syllabus Lesson 120 of 246 · Module 10: Networks that see (CNNs)
Module 10: Networks that see (CNNs)

Deeper networks, and the recorded runs

Why it matters Stacking more layers should help, until it suddenly does not. The course trained this experiment on its own machine so you can see all eighteen runs in seconds.

The recorded experiment cnn-depth-shortcuts trained CNNs with 2, 8 and 20 conv layers (16 channels each) on the shapes dataset, each with and without shortcuts (adding a block's input to its output, from Module 6), and each with three seeds. Everything else was identical: 2,000 training images, 4 epochs, Adam. Replaying is instant because nothing trains; you are reading what really happened.

Three seeds, because Module 9 is right: one run can mislead. Look at the spread of each setting, not just its average.

Call the Curve first: which setting ends with the highest average test accuracy, and does the plain 20-layer network learn on every seed?

Recorded run: a real training run the course made once and ships, so lessons can study runs too big or too many to train live on every laptop. Every printout says RECORDED.

Words and symbols in this lesson
RECORDED
a real run the course trained once and ships for you to study
Your turn

Set my_best to the (depth, shortcuts) pair you think ends highest on average, and my_plain20_reliable to True if you think the plain 20-layer network learns on all three seeds. Then use nnsim.recorded to fill final_acc: (depth, shortcuts) to the list of the three seeds' final test accuracies (seed 0 first), set mean_acc to the average of each list, and best to the pair with the highest average. exp.seeds(depth=..., shortcuts=...) gives the three runs of one setting. Chance is 0.25 (four shapes).

Finally, use Module 9's noise rule to say which settings really differ. With three seeds each, two averages differ for real only if the gap is more than about 2 × s × sqrt(2/3), where s is the larger of the two settings' seed spreads (np.std of the three values). Anything closer is a tie.

Set ties_with_best to the list of every pair that ties with best by that rule, best itself included.

Spotted a problem in this lesson? Report it · Ask about this lesson

This exercise opens soon

Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.