Syllabus Lesson 43 of 246 · Module 3: Neuron to MLP
Module 3: Neuron to MLP

A hidden layer solves XOR

Why it matters This is the moment depth earns its keep: two neurons in a middle layer bend the space so one line can finish the job.

See it

Play the animation. Two hidden neurons each draw their own line. Neuron 1 fires for "at least one input is 1" (an OR). Neuron 2 fires for "both are 1" (an AND). In terms of those two new features, XOR is just "OR but not AND": the output neuron computes h1 minus h2.

Tiny numbers

Use a step activation (1 if the total is above 0, else 0). h1 = step(x1 + x2 - 0.5) is OR. h2 = step(x1 + x2 - 1.5) is AND. Output = step(h1 - h2 - 0.5). Check (1,1): h1 = 1, h2 = 1, output step(-0.5) = 0. Correct.

Hidden layer: a layer between input and output whose neurons build new features the next layer can use.

Recap Each layer re-describes the data so the next layer's job is easier.

Words and symbols in this lesson
neuron
w . x + b, then an activation
weights (w, W), bias (b)
the numbers a neuron learns; the bias shifts its output
hidden layer
a layer between input and output that builds new features
Your turn

Set the weights W1 (2 by 2), b1 (2), w2 (2) and b2 of a two-layer network with step activations so that forward(X) gets all four XOR points right. Use your own numbers or the ones above.

Spotted a problem in this lesson? Report it · Ask about this lesson

This exercise opens soon

Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.