All courses Free · No account · In your browser

Neural Networks: Neuron to Transformer

Build neural networks yourself, from one neuron to a transformer, then generative models, diffusion, scaling laws and looking inside a model, in plain Python and NumPy, trained in your browser. Write your own autograd, diagnose broken runs, judge claims by evidence, and finish with a capstone you can replay. Free, no account.

25 modules 246 lessons 246 auto-graded exercises Self-paced · ~143h of material Trained in your browser, no GPU neededPredict the curve, then run it
Your progress 0 / 246 exercises complete
Daily goal
Time: 60 min

Module 0: Start line

You will be able to: say how the course works and what it measures; make a prediction and check it against a run; record your starting score on the four dials; measure your device's speed in FLOPs.

Leg 1 of 7 · network layer Start line Pass all 5 exercises in this leg to earn its layer.
Time: about 7 h

Module 2: Maths and probability, only as needed

You will be able to: compute dot products, matrix multiplies and broadcasts; find slopes by nudging and by the chain rule, for numbers and for whole matrices; explain how spread adds up and what an eigenvalue does; derive the softmax gradient; write log-sum-exp, softmax and cross-entropy that never overflow (the gate).

Leg 2 of 7 · network layer Foundations Pass all 3 exercises in this leg to earn its layer.
Leg 3 of 7 · network layer Making training work Pass all 40 exercises in this leg to earn its layer.
Leg 4 of 7 · network layer Architectures for structure Pass all 56 exercises in this leg to earn its layer.
Time: about 5 h

Module 15: Squeeze, rebuild, generate (autoencoders and VAEs)

You will be able to: squeeze digits into a few numbers and rebuild them; say, with measurements, why an autoencoder cannot make new digits; compute the KL between two bell curves and the ELBO; sample in a way you can train through; train a VAE that makes new digits; walk between digits; diagnose and fix a collapsed VAE alone (the gate).

Time: about 4 h

Module 16: Two networks in a game (GANs, optional)

Optional module, not needed for the certificate. You will be able to: write the GAN losses and say from their slopes when the saturating one stalls; train a GAN on eight blobs and judge its coverage; make mode collapse happen; test stabilising tricks over recorded seeds and report what helped; run a convolutional generator; rescue a broken GAN alone (the gate).

Time: about 8 h

Module 17: Noise in, pictures out (diffusion)

You will be able to: add noise by a schedule, step by step and in one jump; read the network's arrows back to the data; train a noise predictor; sample with DDPM and say why the clean estimate is clipped; compare noise schedules and time inputs by measurement; build a U-Net; sample in fewer steps with DDIM; steer samples with classifier-free guidance; fine-tune a trained model; fit a new shape alone (the gate).

Time: about 6 h

Module 18: Learning without labels

You will be able to: judge features by nearest-neighbour agreement and by a linear probe you write; pretrain with masked prediction; write InfoNCE and say why it is cross-entropy; pretrain with SimCLR and read what its augmentations decide; match captions to pictures (optional); test features on objects the model never saw, alone (the gate).

Leg 5 of 7 · network layer Networks that make things Pass all 26 exercises in this leg to earn its layer.
Time: about 5 h

Module 19: Bigger, and how much better (scaling laws)

You will be able to: cost a training run in FLOPs; fit a power law with a floor and error bars; read whether weights or data limit a run; plan the best model size for a fixed budget; predict a bigger run and check it; tell a real jump from a metric that makes one; plan a run alone (the gate).

Time: about 5 h

Module 20: Adapting a trained model cheaply

You will be able to: show catastrophic forgetting; count, build and gradient-check LoRA; choose its rank from measurements; fine-tune a model on your own text without wrecking it; say when distillation helps; quantise weights and predict the cost; tune preferences with DPO and stop in time; choose a method for a situation; meet an adaptation target on a budget alone (the gate).

Time: about 8 h

Module 21: Looking inside a model

You will be able to: say whether evidence about a model's inside is a pattern, an effect or a mechanism; run a probe with a control; patch activations to find where information is; remove parts and check for backup; test a textbook mechanism on the model in front of you; show superposition in a toy; train a sparse autoencoder and test a feature label on unseen text; read grokking from recorded runs and check its waves by removing them; find where information is on new inputs alone (the gate).

Time: about 5 h

Module 22: Judging claims

You will be able to: find a leaked test set by overlap and report the clean score; split an average loss along the part that matters; say how zero and mean ablation differ and report the method; tell a one-seed difference from a measured one with standard errors; replicate a result on new seeds and find the setting where it stops holding; review a claim alone (the gate).

Leg 6 of 7 · network layer Scale and understand Pass all 36 exercises in this leg to earn its layer.
Time: about 7 h

Module 23: Capstone studio: a model card you can run

You will be able to: plan a project with a FLOP budget; call and run a baseline over seeds; judge changes with error bars; find a bug that still trains by gradient checking; make a finding inside your model against an untrained control; evaluate once on a hidden set with error bars; write a model card whose every number matches your run log; export one offline file whose runs replay.

Time: 50 min

Module 24: Final exam and retest

You will be able to: show, with the AI off, what Part 2 taught on all four dials (form C); compare your dials from the start line to the end of Part 2; check a month later what stayed (form D, optional).

Leg 7 of 7 · network layer Capstone and proof Pass all 10 exercises in this leg to earn its layer.