Syllabus Lesson 162 of 246 · Module 14: Run Clinic II
Module 14: Run Clinic II

Clinic: a good model writing badly

Why it matters A trained model can still produce garbage if the code around it is wrong. These faults live in the sampling code, not the weights, and people retrain for days before looking there.

Every case below uses the same recorded width-64 model, which writes reasonable text at temperature 0.8. Each case has one fault in how text is sampled or decoded:

  • too_cold: temperature near 0. The model always takes its favourite next character and falls into a loop.
  • too_hot: temperature far above 1. The distribution is flattened until rare symbols come out as often as letters.
  • shifted_decode: the ids are turned back into characters with the alphabet off by one, so a fine sample reads like a code: every space becomes "!".

Probes: sample(c) shows the text; repeat_share(text) and space_share(text) measure it.

Your turn

Fill diagnosis and fixes for cases A, B and C. After your fix, every setting must match the healthy one, and the sample must read like text again.

Spotted a problem in this lesson? Report it · Ask about this lesson

This exercise opens soon

Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.