Syllabus Lesson 237 of 246 · Module 23: Capstone studio: a model card you can run
Module 23: Capstone studio: a model card you can run

Baseline: call its curve first

Why it matters A prediction made before a run is the cheapest way to find out how well you understand your setup. A baseline over 3 seeds is what every later change will be measured against.

Call the Curve: before running, write down the score you expect at a third of the steps, two thirds and the end. Then train the baseline with seeds 0, 1 and 2, logging every run (log=log, evals=3) and charging the ledger.

For calibration: on the language track, guessing every character equally often scores about -4.4 (ln of 83 characters); on the images track, guessing scores about 0.17.

Words and symbols in this lesson
baseline
the plain run every change is measured against, over the same seeds
Your turn

Set call to your three guesses (before you run). Then fill log with the three baseline runs (studio.train(track, **PLAN, seed=s, ledger=ledger, log=log, evals=3)) and print each curve next to your call.

Spotted a problem in this lesson? Report it · Ask about this lesson

This exercise opens soon

Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.