Interactive guide
The one direction that captures the most spread
PCA asks a single question: if you could only describe these points using one number each instead of two, which direction would lose the least information? The answer is always an eigenvector of the data's own covariance matrix, the same eigenvectors from the Eigenvalues & Eigenvectors page, just applied to real data instead of a fixed matrix. Drag the points below and watch that direction chase the shape of the cloud, live.
Variance is the whole game
Center the data at its mean, then ask: along which direction is the spread (variance) of these points largest? That direction is the first principal component (PC1), and it is exactly the eigenvector of the covariance matrix with the largest eigenvalue, the eigenvalue itself is the variance you'd measure along it. The second component (PC2) is forced to be perpendicular to the first; in 2D that leaves no freedom at all, it's just whatever direction is left over.
This is why PCA is called a dimensionality-reduction technique: if PC1 captures 95% of the variance, keeping only each point's position along PC1, one number instead of two, throws away only 5% of the information in the entire dataset.
From points to principal components
-
1
Center the data
Subtract the mean from every point, so the cloud sits at the origin. PCA only ever cares about spread, not position.
-
2
Build the covariance matrix
A 2×2 summary of how x and y vary together: their individual variances, and whether they move with or against each other.
-
3
Find its eigenvectors
Exactly the same eigendecomposition as any other matrix, just this one is always symmetric, so its eigenvectors are always perpendicular.
-
4
Sort by eigenvalue
Largest eigenvalue's eigenvector is PC1, the direction of maximum variance. The rest follow in decreasing order.
Drag any point. Mean, covariance, and both components recompute live.
PC1 (max variance) PC2 your guess
- Mean
- —
- Variance explained (PC1 / PC2)
- —
Try to beat PCA: rotate a direction by hand
Slide the angle and try to maximize the variance yourself.
Compress to one dimension
- Variance retained
- —
- Reconstruction error (SSE)
- —
Covariance matrix
—
- Eigenvalues (λ₁, λ₂)
- —
- PC1 angle
- —
Try a different cloud
Drag the points into a perfect circle and watch both eigenvalues converge to the same number: with no preferred direction in the data, PCA has nothing to find, PC1's angle becomes arbitrary and compressing to one dimension would throw away almost half the information. Now stretch the cloud into a thin line: PC1 rotates to align with it and its variance share climbs toward 100%, that thin line was always secretly one-dimensional data wearing a two-dimensional coordinate system.
The jargons
Every term below is either a step in the same eigendecomposition already covered on the Eigenvalues page, or a name for what that decomposition buys you once it's run on real data.
A 2×2 (or larger) symmetric matrix summarizing how every pair of features varies together. Its diagonal holds ordinary variances.
An eigenvector of the covariance matrix. Ranked by eigenvalue, largest first, each one perpendicular to all the others.
A component's eigenvalue divided by the sum of all eigenvalues: the share of the data's total spread that direction alone accounts for.
Keeping only the top few principal components as a point's new coordinates, discarding the rest with minimal lost variance.
The total squared distance between original points and their projections onto the kept components; exactly equal to the sum of the discarded eigenvalues.
A side effect of rotating onto the principal components: the new coordinates are always uncorrelated with each other, whatever the original data's correlation was.