Syllabus Lesson 42 of 246 · Module 3: Neuron to MLP
Module 3: Neuron to MLP

Productive failure: XOR with one neuron

Why it matters Failing on purpose here teaches you, better than any explanation, why networks need more than one layer.

XOR ("exclusive or") is 1 when exactly one of two inputs is 1. Four points: (0,0) is 0, (0,1) is 1, (1,0) is 1, (1,1) is 0. Your job: make your logistic neuron from the last lesson get all four right. Try any learning rate, any number of steps and any seed.

The neuron here starts from random weights. That matters: started from all zeros, on XOR its gradient is exactly 0 by symmetry (the four points pull equally in opposite directions), so it would never move at all. That is a different failure from the one this lesson is about, so we rule it out.

Spend a few minutes really trying before you read the hint. Struggling first is the point: research on learning finds that people who attempt a problem before being taught the idea understand it better afterwards.

Words and symbols in this lesson
neuron
w . x + b, then an activation
weights (w, W), bias (b)
the numbers a neuron learns; the bias shifts its output
sigmoid
1 / (1 + exp(-z)): squashes any number into 0 to 1
learning rate (lr)
how big each training step is
Your turn

Use train_logistic(X, y, lr, steps, seed) on XOR with at least three different settings, and put each result's accuracy in my_attempts. Then set reason to the letter of the true explanation:

A: it needs more steps. B: a single neuron can only split the plane with one straight line, and no straight line separates XOR. C: the learning rate is wrong. D: four points are too few to learn from.

Spotted a problem in this lesson? Report it · Ask about this lesson

This exercise opens soon

Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.