Productive failure: XOR with one neuron
Why it matters Failing on purpose here teaches you, better than any explanation, why networks need more than one layer.
XOR ("exclusive or") is 1 when exactly one of two inputs is 1. Four points: (0,0) is 0, (0,1) is 1, (1,0) is 1, (1,1) is 0. Your job: make your logistic neuron from the last lesson get all four right. Try any learning rate, any number of steps and any seed.
The neuron here starts from random weights. That matters: started from all zeros, on XOR its gradient is exactly 0 by symmetry (the four points pull equally in opposite directions), so it would never move at all. That is a different failure from the one this lesson is about, so we rule it out.
Spend a few minutes really trying before you read the hint. Struggling first is the point: research on learning finds that people who attempt a problem before being taught the idea understand it better afterwards.
Words and symbols in this lesson
- neuron
- w . x + b, then an activation
- weights (w, W), bias (b)
- the numbers a neuron learns; the bias shifts its output
- sigmoid
- 1 / (1 + exp(-z)): squashes any number into 0 to 1
- learning rate (lr)
- how big each training step is
Use train_logistic(X, y, lr, steps, seed) on XOR with at least three different settings, and put each result's accuracy in my_attempts. Then set reason to the letter of the true explanation:
A: it needs more steps. B: a single neuron can only split the plane with one straight line, and no straight line separates XOR. C: the learning rate is wrong. D: four points are too few to learn from.
Spotted a problem in this lesson? Report it · Ask about this lesson
This exercise opens soon
Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.
This browser is blocking cookies for learn.floati.life, so lessons you finished here cannot open in order. Allow cookies for this site and reload.