Gate: a new task on a budget (AI off)
Why it matters Real adaptation work has a target, a budget and something you must not break. All three, at once, alone.
Gate rules: AI tutor off. Adapt the recorded GPT to Python code with at most 60 training steps of 8 windows each. Targets on a fresh run (new seed):
- held-out code loss at most 2.43 (the base scores 3.25);
- held-out English loss at most 2.00 (the base scores 1.70);
Set plan: method "full" or "lora", the rank (for LoRA, 1 to 64), the steps and the learning rate. The check trains your plan once on a new seed.
Words and symbols in this lesson
- LoRA
- low-rank adaptation: a frozen weight matrix plus two thin trainable ones, x W + (alpha / r) x A B
- rank (r)
- how thin LoRA's two matrices are: (d_in + d_out) x r trainable weights per matrix
Fill plan.
AI off: no tutor, hints or worked solution on this one. Your own thinking only.
Spotted a problem in this lesson? Report it · Ask about this lesson
This exercise opens soon
Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.
This browser is blocking cookies for learn.floati.life, so lessons you finished here cannot open in order. Allow cookies for this site and reload.