Interactive guide
Every matrix is a rotation, a stretch, and another rotation
The Eigenvalues & Eigenvectors page only worked because every matrix there was symmetric. Most matrices, including every non-square one, aren't. The Singular Value Decomposition (SVD) is the fix that works on any matrix: A = UΣVT, a rotation, a scale along perpendicular axes, and another rotation, always, no exceptions. Drag the matrix below and watch the split happen live.
Where the pieces come from
Build ATA. It's always square and symmetric, even when A itself is neither, so it always has the perpendicular eigenvectors the PCA page relied on. Those eigenvectors are the columns of V, and the square roots of ATA's eigenvalues are the singular values, σ₁ ≥ σ₂ ≥ ... ≥ 0.
Push each vi through A itself and divide by its own σi: ui = Avi / σi. Those become the columns of U, automatically perpendicular to each other too, this is the one algebraic fact that makes the whole decomposition exist unconditionally.
Geometrically: A sends the unit circle to an ellipse. V's columns are where the circle's own perpendicular axes started; U's columns are the ellipse's actual axes; the singular values are the ellipse's semi-axis lengths. Three moves, always in this order: rotate by VT, scale by Σ, rotate by U.
Reading the diagram
-
1
The dashed circle is where every vector starts
V's two columns are marked on it, the only two directions that come out of A still perpendicular.
-
2
The filled shape is A applied to the whole circle
Always an ellipse (or a flattened line, if a singular value hits zero).
-
3
U's columns are the ellipse's own axes
Their lengths are the singular values σ₁, σ₂, exactly how far the circle got stretched in each direction.
-
4
The three mini-panels replay it as three separate moves
Rotate so V's axes align with the grid, stretch along the grid, then rotate the result into place.
Drag the sliders. The circle, ellipse, and both axis pairs update live.
v₁ v₂ σ₁u₁ σ₂u₂
The matrix
—
—
The three moves, separately
1. Rotate by VT
2. Scale by Σ
3. Rotate by U
- Singular values (σ₁, σ₂)
- —
- U
- —
- VT
- —
Compressing an image: keep only the biggest singular values
A 24×24 image is just a matrix of brightness values. Its SVD exists exactly the same way; keeping only the top k singular values reconstructs an approximation from far less data.
Reconstructed from the top k singular values.
- Reconstruction error
- —
- Storage used
- —
Rank kept (k)
—
Singular value spectrum
Original (full rank)
Watch how little rank it takes for the picture to become recognizable: the smooth background is almost entirely captured by the first one or two singular values, and only the sharp ring edge needs the higher ones. That split, smooth global structure lives in the largest singular values, sharp local detail lives in the smallest, is exactly why image and video compression, and low-rank neural network approximations, both lean on this decomposition.
The jargons
SVD generalizes the Eigenvalues page's machinery to matrices that aren't symmetric, or aren't even square.
Square roots of ATA's eigenvalues. Always real and non-negative, unlike eigenvalues of a general matrix.
Orthogonal matrices: U's columns are an orthonormal basis for the output space, V's for the input space.
The number of non-zero singular values: how many genuinely independent directions the matrix actually uses.
Keeping only the top k singular values (and their vectors) gives the best possible rank-k approximation of the original matrix, provably.
Every matrix has an SVD; only symmetric (or more generally diagonalizable) square matrices have an eigendecomposition. PCA's covariance matrix happens to qualify for both.
The matrix version of Euclidean length: the square root of the sum of every entry squared. Used here to measure reconstruction error.