Syllabus Lesson 41 of 246 · Module 3: Neuron to MLP
Module 3: Neuron to MLP

Yes or no answers (logistic regression)

Why it matters Most tasks ask for a category, not a number: spam or not, cat or dog. One neuron plus a squashing function answers yes or no with a probability.

See it

The sigmoid squashes any number into the range 0 to 1: sigmoid(z) = 1 / (1 + exp(-z)). sigmoid(0) = 0.5, sigmoid(3) is about 0.95, sigmoid(-3) about 0.05. Read the output as "the probability the answer is yes".

The loss is the cross-entropy from Module 2 with two classes: -mean(y log p + (1 - y) log(1 - p)). Its gradient comes out beautifully simple: grad_w = X.T @ (p - y) / n. Prediction minus truth again.

Logistic regression: one neuron with a sigmoid, trained with cross-entropy. It draws a straight-line boundary between two classes.

Recap Score, squash to a probability, train on (p - y).

Words and symbols in this lesson
neuron
w . x + b, then an activation
sigmoid
1 / (1 + exp(-z)): squashes any number into 0 to 1
learning rate (lr)
how big each training step is
Your turn

Write sigmoid(z) and train_logistic(X, y, lr, steps) returning (w, b), starting from zeros. Then accuracy(X, y, w, b): the share of points where p > 0.5 matches the label. Reach at least 90% on hidden points from the same two groups.

Spotted a problem in this lesson? Report it · Ask about this lesson

This exercise opens soon

Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.