Interactive guide
Learning from mistakes, one point at a time
The Perceptron is the oldest learning algorithm for a linear classifier, and it doesn't minimize a loss function at all; it just looks at one point, checks if it got it wrong, and if so, nudges the decision boundary exactly enough to fix that one mistake. Below, watch that rule play out on three datasets with three very different endings.
A rule, not a loss function
Logistic regression and SVM both define a smooth loss over the whole dataset and use calculus (gradient descent) to minimize it. The Perceptron does neither. It keeps a weight vector w and bias b, and for each point x with true label y ∈ {−1, +1}:
Predict: ŷ = sign(w·x + b). If correct, do nothing. If wrong, update: w ← w + η·y·x and b ← b + η·y, a single step that makes this exact point's score move in the right direction.
The Perceptron Convergence Theorem guarantees this reaches zero mistakes in finitely many steps, if the data is linearly separable. It guarantees absolutely nothing otherwise, as the next two datasets below make very concrete.
The update loop
-
1
Pick the next point
Cycle through the dataset in a fixed order, one point per step.
-
2
Predict its side
ŷ = sign(w·x + b): which side of the current line is it on?
-
3
Wrong? Nudge the line
w ← w + y·x, b ← b + y. Correct? Change nothing at all.
-
4
Repeat until a full pass has zero mistakes
That's convergence, guaranteed eventually, but only if the classes are truly separable.
Separable data
class +1 class −1 boundary
The neuron, computing this same point
Press Step to see the first point flow through.
—
- Epoch
- 1
- Mistakes this epoch
- 0
- Total mistakes so far
- 0
- Weights (w₁, w₂, b)
- 0.00, 0.00, 0.00
All three datasets, side by side
| Metric | Separable | Non-sep. | XOR |
|---|
Why XOR breaks every straight line
No line separates these 4; one hidden layer of 2 units does.
Single perceptron
+ 1 hidden layer (real, trained)
The Perceptron's guarantee is real but narrow: give it linearly separable data and it will find a separating line, provably, in finite steps. Give it anything messier (overlapping classes, or a pattern like XOR that no line can ever separate) and it doesn't fail gracefully. It doesn't converge to a "good enough" boundary the way logistic regression or a soft-margin SVM would; it just keeps correcting one mistake by creating another, forever. That gap between "provably correct sometimes" and "no fallback the rest of the time" is exactly why the field moved toward loss functions you can partially satisfy, and eventually toward stacking these units into layers.
The jargons
The perceptron’s convergence guarantee holds under exactly one condition — linear separability — and the terms below trace what happens on either side of that line: convergence when it holds, XOR-style failure when it doesn’t, and the fix that came after.
w ← w + η·y·x, b ← b + η·y, applied only when a point is misclassified. No loss function, no gradient.
A dataset where some straight line (or hyperplane) can put every point of one class on one side and every point of the other class on the other.
Guarantees the algorithm reaches zero mistakes in finitely many updates, but only when the data is linearly separable.
One full pass through every point in the dataset. Convergence means an entire epoch with zero mistakes.
The historic example (Minsky & Papert, 1969) proving a single perceptron cannot represent every simple boolean function: it isn't linearly separable.
Stacking perceptron-like units with a hidden layer lets the network learn nonlinear boundaries (including XOR) via backpropagation. See the Multi-Layer Perceptron page to train one live.