Telling the model what step it is
Why it matters The same noisy point needs a small correction at step 5 and a big one at step 90. A network that is not told the step has to guess it from the point alone. How you tell it is a design choice, and the famous choice is not always the best one.
Three ways (difflab.time_input):
- none: the network sees only the point.
- raw: one extra number, t / T.
- sine: 8 sines and 8 cosines of t / T at frequencies from 1 to 100, the embedding transformers use for positions. Big image models use it, because many frequencies let a deep network react sharply to small changes in t.
The recorded runs train each on the spiral, 3 seeds. Compare them with Module 9's rule: a setting beats another when its mean on-shape score is at least 0.1 higher AND it is higher on every seed (3 of 3).
Words and symbols in this lesson
- on shape, coverage
- the 2D judge: share of samples near the real shape, and share of the shape with a sample near it
Write sine_time(t, T) returning the 16 numbers per step (sines first, then cosines, frequencies np.exp(np.linspace(0, np.log(100), 8))). Then write beats(a, b) for two recorded setting names, and fill verdicts for raw vs none, sine vs none and sine vs raw.
Spotted a problem in this lesson? Report it · Ask about this lesson
This exercise opens soon
Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.
This browser is blocking cookies for learn.floati.life, so lessons you finished here cannot open in order. Allow cookies for this site and reload.