A digit reader in plain NumPy
Why it matters This is a real neural network reading real handwriting, with every line written by you. It is the classic first network, built without a library.
The data is 1,797 handwritten digits, each an 8 by 8 image flattened into 64 numbers (a small, low-resolution cousin of the famous 28 by 28 MNIST set, from a different collection). The network is the one from the last lesson: 64 inputs, a hidden layer of 64 ReLU neurons, and 10 outputs.
The training loop
- Start the weights small and random:
rng.normal(0, 0.1, shape), biases at zero. - Each epoch, shuffle the training data and walk through it in minibatches of 32.
- For each minibatch, run
forward_backwardand step every weight against its gradient.
Minibatch: a small random slice of the data used for one step. It is much faster than using all the data each step, and the noise even helps.
Call the Curve: guess the training loss after epoch 1 and after epoch 20 in my_guess.
Recap Shuffle, slice, forward, backward, step. Repeat.
Words and symbols in this lesson
- neuron
- w . x + b, then an activation
- weights (w, W), bias (b)
- the numbers a neuron learns; the bias shifts its output
- ReLU
- max(0, z): keeps positives, zeroes negatives
- learning rate (lr)
- how big each training step is
- hidden layer
- a layer between input and output that builds new features
Using your forward_backward, write train(X, y, hidden, lr, epochs, seed) returning a dictionary of weights, and predict(params, X) returning the predicted digit for each row. Fill in my_guess. The hidden test measures accuracy on 397 digits you never trained on: reach at least 95%. (lr 0.5, 20 epochs, 64 hidden works.)
Spotted a problem in this lesson? Report it · Ask about this lesson
This exercise opens soon
Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.
This browser is blocking cookies for learn.floati.life, so lessons you finished here cannot open in order. Allow cookies for this site and reload.