Matrix multiply: a whole layer at once
Why it matters A layer is many neurons, and a batch is many inputs. One matrix multiply computes every neuron on every input in one line.
Stack the inputs as rows of X (shape: inputs by features). Stack each neuron's weights as a column of W (shape: features by neurons). Then X @ W has one row per input and one column per neuron: entry (i, j) is the dot product of input i with neuron j's weights.
Tiny numbers
X = [[1, 2], W = [[1, 0, -1],
[3, 4]] [0, 1, 1]]
X @ W = [[1, 2, 1],
[3, 4, 1]]
Row 1, column 3: 1 times -1 plus 2 times 1 equals 1. The shapes chain: (2, 2) @ (2, 3) gives (2, 3). The inner numbers must match.
Matrix multiply: every row of the left times every column of the right, as dot products.
Recap (batch, in) @ (in, out) = (batch, out). That one shape rule runs the whole course.
Words and symbols in this lesson
- dot product (a . b)
- multiply matching items and add them up; big when the vectors agree
Write layer(X, W, b) returning X @ W + b, and layer_shape(batch, n_in, n_out) returning the output shape as a tuple without building any arrays.
Spotted a problem in this lesson? Report it · Ask about this lesson
This exercise opens soon
Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.
This browser is blocking cookies for learn.floati.life, so lessons you finished here cannot open in order. Allow cookies for this site and reload.