Interactive guide
One has corners. That's the entire reason.
The Regularization page showed Lasso zeroing out coefficients while Ridge only shrinks them, without saying why. The reason is entirely geometric: an L1 penalty's constraint region is a diamond with corners sitting exactly on the axes; an L2 penalty's is a smooth circle with no corners anywhere. Drag the point below and watch the same shrinkage land on a corner for one and never for the other.
Shrinking toward a shape, not just toward zero
Adding a penalty to a loss function is the same as minimizing that loss subject to staying inside a region: the L1 penalty λ(|w₁|+|w₂|) restricts w to a diamond, ‖w‖₁ ≤ t; the L2 penalty λ(w₁²+w₂²) restricts it to a circle, ‖w‖₂ ≤ t. The strength λ and the budget t move together, stronger penalty, smaller region.
The unconstrained best fit sits at some point (a, b). As the region shrinks, the best point inside it slides from (a, b) toward the origin, stopping wherever the loss's own contour lines first touch the region's edge. On a smooth circle, that touching point can land anywhere, almost never exactly on an axis. On a diamond, an enormous share of that edge is concentrated at just four corners, and each corner sits exactly on an axis: touch a corner, and one coordinate becomes exactly zero.
Because this loss is a simple, uncorrelated quadratic, the two coordinates don't interact, so each one has its own closed-form answer: soft-thresholding for L1, a plain shrink-toward-zero for L2. Both are computed live below, not simulated.
Reading the diagram
-
1
The gray point is the unconstrained optimum
Where the loss alone, with no penalty at all, would land. Drag it anywhere.
-
2
Each dashed region is sized to just touch its own solution
The diamond (purple) for L1, the circle (teal) for L2, both shrink together as λ grows.
-
3
The solid ellipse is the loss contour touching that region
Centered on the gray point; where it just grazes the region's edge is the actual constrained answer.
-
4
Watch the green (L1) dot land exactly on an axis
The orange (L2) dot shrinks toward the origin too, but essentially never lands exactly on one.
Drag the gray point. Both penalties recompute live from the same loss.
unconstrained (a, b) L1 solution L2 solution
- L1 solution (w₁, w₂)
- —
- L2 solution (w₁, w₂)
- —
Penalty strength
Loss curvature (how tightly each axis is pinned down)
—
w₁ as λ grows
L1 (hits zero, stays there) L2 (shrinks, never zero)
Slide λ up slowly and watch the L1 path's straight line hit zero at a specific, computable point, λ = |a| / s₁, and go perfectly flat forever after. The L2 path is a smooth curve that keeps approaching zero at every step without ever arriving: a/(1+2λs₁) is nonzero for every finite λ whenever a is. That one algebraic difference, a hard floor versus an asymptote, is the sparsity gap between Lasso and Ridge, and it's the same fact as the corner on the diamond, just seen without the picture.
The jargons
Same two penalties as the Regularization page; this page derives the mechanism rather than just showing the effect.
The set of points within a fixed "distance" t of the origin, under a chosen norm. Its shape depends entirely on which norm.
A diamond (a square rotated 45°). Its corners sit exactly on the axes, at (±t, 0) and (0, ±t).
An ordinary circle. Perfectly smooth, no corners anywhere on its boundary.
Lasso's exact per-coordinate update: shrink toward zero by a fixed amount, and clip anything that would cross zero to exactly zero.
L2's exact per-coordinate update: multiply by a factor just under 1. Approaches zero smoothly, never reaches it at any finite penalty.
A solution with some coefficients exactly zero, not just small. Only the corner-having penalty (L1) can produce this from a smooth loss.