Syllabus Lesson 49 of 246 · Module 3: Neuron to MLP
Module 3: Neuron to MLP

A digit reader in plain NumPy

Why it matters This is a real neural network reading real handwriting, with every line written by you. It is the classic first network, built without a library.

The data is 1,797 handwritten digits, each an 8 by 8 image flattened into 64 numbers (a small, low-resolution cousin of the famous 28 by 28 MNIST set, from a different collection). The network is the one from the last lesson: 64 inputs, a hidden layer of 64 ReLU neurons, and 10 outputs.

The training loop

  1. Start the weights small and random: rng.normal(0, 0.1, shape), biases at zero.
  2. Each epoch, shuffle the training data and walk through it in minibatches of 32.
  3. For each minibatch, run forward_backward and step every weight against its gradient.

Minibatch: a small random slice of the data used for one step. It is much faster than using all the data each step, and the noise even helps.

Call the Curve: guess the training loss after epoch 1 and after epoch 20 in my_guess.

Recap Shuffle, slice, forward, backward, step. Repeat.

Words and symbols in this lesson
neuron
w . x + b, then an activation
weights (w, W), bias (b)
the numbers a neuron learns; the bias shifts its output
ReLU
max(0, z): keeps positives, zeroes negatives
learning rate (lr)
how big each training step is
hidden layer
a layer between input and output that builds new features
Your turn

Using your forward_backward, write train(X, y, hidden, lr, epochs, seed) returning a dictionary of weights, and predict(params, X) returning the predicted digit for each row. Fill in my_guess. The hidden test measures accuracy on 397 digits you never trained on: reach at least 95%. (lr 0.5, 20 epochs, 64 hidden works.)

Spotted a problem in this lesson? Report it · Ask about this lesson

This exercise opens soon

Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.