Neural Networks: Neuron to Transformer
Build neural networks yourself, from one neuron to a transformer, then generative models, diffusion, scaling laws and looking inside a model, in plain Python and NumPy, trained in your browser. Write your own autograd, diagnose broken runs, judge claims by evidence, and finish with a capstone you can replay. Free, no account.
Real Python and NumPy are ready in this tab.
No install, no account, no setup. Every network in this course trains on your own device, and nothing you write leaves it. Module 1 brushes up the Python and NumPy you need. Already fluent? Its last lesson is a short gate you can take first: pass it and the course moves you straight on.
Daily review
Spaced repetition keeps what you learned from fading.
Module 0: Start line
You will be able to: say how the course works and what it measures; make a prediction and check it against a run; record your starting score on the four dials; measure your device's speed in FLOPs.
Module 1: Python for this course
You will be able to: write and test functions over lists and dictionaries; use NumPy arrays, shapes, slicing and broadcasting instead of loops; seed randomness reproducibly; write small classes and closures. Checked by the Python gate (already fluent? take it first: passing skips the module).
Module 2: Maths and probability, only as needed
You will be able to: compute dot products, matrix multiplies and broadcasts; find slopes by nudging and by the chain rule, for numbers and for whole matrices; explain how spread adds up and what an eigenvalue does; derive the softmax gradient; write log-sum-exp, softmax and cross-entropy that never overflow (the gate).
Module 3: Neuron to MLP
You will be able to: build a neuron, activations and a two-layer network in NumPy; derive backprop by hand and check it with a gradient checker; train a digit reader past 95%; build and train your own network for a new dataset (the gate).
Module 4: Your own autograd engine
You will be able to: build a reverse-mode autograd engine (graph, topological order, backward rules, broadcasting, reductions); check it against nudging; train a network on it with no hand-written gradients; add a new operation alone (the gate).
Module 5: Optimisation
You will be able to: explain why a stretched landscape is slow; build minibatch SGD, momentum, RMSProp, Adam and AdamW, schedules and clipping; pick and tune an optimiser for a problem you have not seen, under a budget (the gate).
Module 6: Starting weights and normalisation
You will be able to: measure signals through depth and choose Xavier, LeCun or He scales; find dead units; build BatchNorm forward and backward, LayerNorm and RMSNorm; avoid the train/eval trap; use shortcuts in deep networks.
Module 7: Working on new data
You will be able to: measure over- and underfitting, bias and variance; use weight decay, dropout, augmentation and early stopping; read double descent and learning curves honestly; tune a model on validation data and report it on held-out data (the gate).
Module 8: Run Clinic I
You will be able to: run the two first checks on any training run; pick probes that split likely faults; diagnose and fix broken runs with a small change, on a budget (the gate); judge an AI helper's diagnoses by evidence.
Module 9: Is it really better?
You will be able to: split data without leakage; use precision and recall; measure spread over seeds; put bootstrap error bars on a score; measure calibration; compare two models with a paired interval and make the call (the gate).
Module 10: Networks that see (CNNs)
You will be able to: build convolution, padding, pooling and their gradients, and im2col; train CNNs on the house engine; read recorded runs over seeds; test saliency against a baseline; use transfer; design a CNN under a weight budget (the gate).
Module 11: Networks with memory (RNNs)
You will be able to: build an RNN and a character model; derive backprop through time; measure vanishing gradients; build LSTM gates; read recorded memory and bottleneck experiments; sample with temperature; repair a broken recurrent model (the gate).
Module 12: Words as numbers (embeddings)
You will be able to: treat an embedding as a lookup; build a pair-merging tokenizer; make word vectors by counting and by skip-gram; measure and remove a skew they copied; do all of it alone on a world you have not seen (the gate).
Module 13: Attention and transformers
You will be able to: build attention, the causal mask, heads, positions, layer norm and a whole block that matches a real model; derive attention's backward pass; train a small GPT; read recorded runs; generate with a cache; map your engine onto PyTorch; repair a broken transformer (the gate).
Module 14: Run Clinic II
You will be able to: diagnose broken sequence models, including cheats that look great, stalls, step sizes and sampling faults; read failed recorded runs; judge an AI helper by evidence; diagnose alone on a budget (the gate); compare your exit exam with your baseline.
Module 15: Squeeze, rebuild, generate (autoencoders and VAEs)
You will be able to: squeeze digits into a few numbers and rebuild them; say, with measurements, why an autoencoder cannot make new digits; compute the KL between two bell curves and the ELBO; sample in a way you can train through; train a VAE that makes new digits; walk between digits; diagnose and fix a collapsed VAE alone (the gate).
Module 16: Two networks in a game (GANs, optional)
Optional module, not needed for the certificate. You will be able to: write the GAN losses and say from their slopes when the saturating one stalls; train a GAN on eight blobs and judge its coverage; make mode collapse happen; test stabilising tricks over recorded seeds and report what helped; run a convolutional generator; rescue a broken GAN alone (the gate).
Module 17: Noise in, pictures out (diffusion)
You will be able to: add noise by a schedule, step by step and in one jump; read the network's arrows back to the data; train a noise predictor; sample with DDPM and say why the clean estimate is clipped; compare noise schedules and time inputs by measurement; build a U-Net; sample in fewer steps with DDIM; steer samples with classifier-free guidance; fine-tune a trained model; fit a new shape alone (the gate).
Module 18: Learning without labels
You will be able to: judge features by nearest-neighbour agreement and by a linear probe you write; pretrain with masked prediction; write InfoNCE and say why it is cross-entropy; pretrain with SimCLR and read what its augmentations decide; match captions to pictures (optional); test features on objects the model never saw, alone (the gate).
Module 19: Bigger, and how much better (scaling laws)
You will be able to: cost a training run in FLOPs; fit a power law with a floor and error bars; read whether weights or data limit a run; plan the best model size for a fixed budget; predict a bigger run and check it; tell a real jump from a metric that makes one; plan a run alone (the gate).
Module 20: Adapting a trained model cheaply
You will be able to: show catastrophic forgetting; count, build and gradient-check LoRA; choose its rank from measurements; fine-tune a model on your own text without wrecking it; say when distillation helps; quantise weights and predict the cost; tune preferences with DPO and stop in time; choose a method for a situation; meet an adaptation target on a budget alone (the gate).
Module 21: Looking inside a model
You will be able to: say whether evidence about a model's inside is a pattern, an effect or a mechanism; run a probe with a control; patch activations to find where information is; remove parts and check for backup; test a textbook mechanism on the model in front of you; show superposition in a toy; train a sparse autoencoder and test a feature label on unseen text; read grokking from recorded runs and check its waves by removing them; find where information is on new inputs alone (the gate).
Module 22: Judging claims
You will be able to: find a leaked test set by overlap and report the clean score; split an average loss along the part that matters; say how zero and mean ablation differ and report the method; tell a one-seed difference from a measured one with standard errors; replicate a result on new seeds and find the setting where it stops holding; review a claim alone (the gate).
Module 23: Capstone studio: a model card you can run
You will be able to: plan a project with a FLOP budget; call and run a baseline over seeds; judge changes with error bars; find a bug that still trains by gradient checking; make a finding inside your model against an untrained control; evaluate once on a hidden set with error bars; write a model card whose every number matches your run log; export one offline file whose runs replay.
Module 24: Final exam and retest
You will be able to: show, with the AI off, what Part 2 taught on all four dials (form C); compare your dials from the start line to the end of Part 2; check a month later what stayed (form D, optional).
Only you can print or save these. The links are saved in this browser. To show someone, share the verify link from the certificate.