Gate: plan a training run (AI off)
Why it matters This is the decision scaling laws exist for: you have a budget and one shot. Plan it from evidence.
Gate rules: AI tutor off. You have 3 x 10^10 FLOPs for one run of the course's character model, on the same text as the grid. Choose a width (a power of 2) and the number of tokens the budget then allows. Use the recorded grid any way you like: fits, tables, both.
The check predicts the loss of your choice and of the best possible choice with the course's two-variable fit, and passes when yours is within 0.02 of the best and you spent the budget (within 1%).
Words and symbols in this lesson
- scaling law
- how loss falls as weights, data or compute grow, usually a power law towards a floor
- token
- here, one prediction: one character
- FLOPs (6 N D)
- training cost: about 6 operations per weight per token
Set width and tokens.
AI off: no tutor, hints or worked solution on this one. Your own thinking only.
Spotted a problem in this lesson? Report it · Ask about this lesson
This exercise opens soon
Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.
This browser is blocking cookies for learn.floati.life, so lessons you finished here cannot open in order. Allow cookies for this site and reload.