Syllabus Lesson 23 of 246 · Module 2: Maths and probability, only as needed
Module 2: Maths and probability, only as needed

The dot product, and why a neuron is one

Why it matters One neuron is a dot product plus a number. Understand the dot product and you understand a neuron's first half.

The dot product multiplies two vectors item by item and adds the results: [1, 2, 3] . [4, 5, 6] = 1*4 + 2*5 + 3*6 = 32. In NumPy: a @ b.

What it means

If the vectors point the same way the dot product is large and positive; at right angles it is 0; pointing opposite ways it is negative. So it measures how much two vectors agree.

A neuron

A neuron has one weight per input. It computes w . x + b: how much the input agrees with the pattern stored in its weights, plus a bias b that shifts the result up or down. Weights [0.5, 0.5, -1.0] on our houses say: bigger and more bedrooms push up, distance from the station pushes down.

Dot product: multiply matching items and add. Bias: a number added at the end so the neuron is not forced to output 0 for an all-zero input.

Recap A neuron scores how well the input matches its weights.

Words and symbols in this lesson
vector
a list of numbers, one per feature
dot product (a . b)
multiply matching items and add them up; big when the vectors agree
Your turn

Write dot(a, b) with a loop (to see the sum yourself), and neuron(w, x, b) that returns w . x + b using @.

Spotted a problem in this lesson? Report it · Ask about this lesson

This exercise opens soon

Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.