A hidden layer solves XOR
Why it matters This is the moment depth earns its keep: two neurons in a middle layer bend the space so one line can finish the job.
See it
Play the animation. Two hidden neurons each draw their own line. Neuron 1 fires for "at least one input is 1" (an OR). Neuron 2 fires for "both are 1" (an AND). In terms of those two new features, XOR is just "OR but not AND": the output neuron computes h1 minus h2.
Tiny numbers
Use a step activation (1 if the total is above 0, else 0). h1 = step(x1 + x2 - 0.5) is OR. h2 = step(x1 + x2 - 1.5) is AND. Output = step(h1 - h2 - 0.5). Check (1,1): h1 = 1, h2 = 1, output step(-0.5) = 0. Correct.
Hidden layer: a layer between input and output whose neurons build new features the next layer can use.
Recap Each layer re-describes the data so the next layer's job is easier.
Words and symbols in this lesson
- neuron
- w . x + b, then an activation
- weights (w, W), bias (b)
- the numbers a neuron learns; the bias shifts its output
- hidden layer
- a layer between input and output that builds new features
Set the weights W1 (2 by 2), b1 (2), w2 (2) and b2 of a two-layer network with step activations so that forward(X) gets all four XOR points right. Use your own numbers or the ones above.
Spotted a problem in this lesson? Report it · Ask about this lesson
This exercise opens soon
Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.
This browser is blocking cookies for learn.floati.life, so lessons you finished here cannot open in order. Allow cookies for this site and reload.