Syllabus Lesson 163 of 246 · Module 14: Run Clinic II
Module 14: Run Clinic II

Diagnosing from the record alone

Why it matters Often you cannot rerun anything: the run was someone else's, or it cost a week. All you have is what was logged. Two recorded experiments from this course hold real failures.

In gpt-width-lr, a setting is broken by its learning rate if its final held-out loss, averaged over the three recorded seeds, is more than 0.5 worse than the best setting of the same width. In cnn-depth-shortcuts, a single run is stuck if its test accuracy is exactly the same at every epoch: it is answering one class for everything (there are four shape classes, so about 0.25).

Predict before you look: which GPT runs are broken, and which CNN is stuck?

Your turn

Fill broken with (width, lr) pairs and stuck with (depth, shortcuts, seed) triples, computed from the records. exp.seeds(d=..., lr=...) gives the three runs of one setting.

Spotted a problem in this lesson? Report it · Ask about this lesson

This exercise opens soon

Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.