Clinic: a good model writing badly
Why it matters A trained model can still produce garbage if the code around it is wrong. These faults live in the sampling code, not the weights, and people retrain for days before looking there.
Every case below uses the same recorded width-64 model, which writes reasonable text at temperature 0.8. Each case has one fault in how text is sampled or decoded:
- too_cold: temperature near 0. The model always takes its favourite next character and falls into a loop.
- too_hot: temperature far above 1. The distribution is flattened until rare symbols come out as often as letters.
- shifted_decode: the ids are turned back into characters with the alphabet off by one, so a fine sample reads like a code: every space becomes "!".
Probes: sample(c) shows the text; repeat_share(text) and space_share(text) measure it.
Fill diagnosis and fixes for cases A, B and C. After your fix, every setting must match the healthy one, and the sample must read like text again.
Spotted a problem in this lesson? Report it · Ask about this lesson
This exercise opens soon
Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.
This browser is blocking cookies for learn.floati.life, so lessons you finished here cannot open in order. Allow cookies for this site and reload.