Syllabus Lesson 165 of 246 · Module 14: Run Clinic II
Module 14: Run Clinic II

Gate: three unseen broken runs (AI off)

Why it matters Diagnosing alone, with a budget, is the skill this course has been building since Module 8.

AI tutor off. Three runs, each with one fault from this module's training cases. They are new runs, not the practice cases: another seed, and faults of other sizes, so remembered numbers will not match. Your budget is 9 probes in all (training a sealed case counts as one), counted across every run since the page opened (repeating a probe on the same case is free; a wrong diagnosis or a fix that does not work costs one probe).

Each case below is a training run with one fault planted in it. For each: spend probes to find the fault, then write a fix as config changes that put the run back to healthy. The faults in this lesson: any training fault from this module. A good diagnostician rarely needs more than two probes per case.

ProbeWhat it tells youHealthy run
loss_curve(c)Training loss every 25 steps, and the loss on unseen text at the end. Always look first.4.55 down to about 2.9; unseen about 3.0
loss_at_init(c)Loss before training. A fresh model guessing evenly over 95 characters gives ln(95).4.55
leak_check(c)Can a later character change an earlier prediction?False
show_batch(c)One training input and its target, as text.target = input moved one character left
update_ratio(c)After the first Adam step, how far weights moved relative to their size. Not Module 8's rule: that 0.001 to 0.1 range is for plain SGD steps. Adam's first step moves every weight by about the learning rate whatever its gradient, and these weights start near 0.02, so 0.01 / 0.02 gives about 0.5. Compare a case with this healthy run, not with Module 8's range.about 0.5
block_change(c)How much the first block changes each vector at the start.about 0.3
Words and symbols in this lesson
ln(V)
the starting loss of a model that guesses evenly over V symbols: 4.55 for 95 characters
block change
how much one block changes a vector at the start; near 1 means it replaced it
Your turn

Fill in diagnosis (case to fault name) and fixes (case to a dictionary of config changes) for cases A, B and C. After your fix, every setting must match the healthy run.

AI off: no tutor, hints or worked solution on this one. Your own thinking only.

Spotted a problem in this lesson? Report it · Ask about this lesson

This exercise opens soon

Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.