Choosing a method under a budget
Why it matters Each method in this module is the right answer to a different question. Picking by fashion wastes money; picking from measurements is the job.
What you measured, in one place:
- Full fine-tuning (200 steps): code 2.19, English 2.26. Trains all 107,072 weights; a whole new copy of the model to store.
- LoRA rank 4: code 2.27, English 2.04. Trains 8,192; stores only those.
- Distillation to a 16-wide student (3000 steps): English 1.98 at about 1/8 of the weights, against 2.18 trained alone.
- Quantisation: 8 bits, a quarter of the size, no change; 4 bits, an eighth, +0.13.
For each situation, choose the method the measurements support.
Words and symbols in this lesson
- fine-tuning
- continuing to train a trained model on new data
- LoRA
- low-rank adaptation: a frozen weight matrix plus two thin trainable ones, x W + (alpha / r) x A B
- rank (r)
- how thin LoRA's two matrices are: (d_in + d_out) x r trainable weights per matrix
- distillation
- training a small student to copy a big teacher's full probabilities
- temperature (distillation)
- dividing scores before the softmax so second choices count more
- quantisation
- storing weights in fewer bits by rounding them to a few steps
Fill choice with one of "full fine-tune", "LoRA", "distillation" or "quantisation" for each situation.
Spotted a problem in this lesson? Report it · Ask about this lesson
This exercise opens soon
Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.
This browser is blocking cookies for learn.floati.life, so lessons you finished here cannot open in order. Allow cookies for this site and reload.