Interactive guide
When one class barely shows up at all
24 loan applicants repaid, 4 defaulted. Train a classifier on that as-is and it can score 88% accuracy while catching barely a fifth of actual defaults, because "always predict repaid" is a great strategy for the scoreboard and a useless one for the bank. Below, the exact same tiny dataset gets balanced three different ways.
The accuracy paradox
Class imbalance is when one outcome is far rarer than the other: fraud, disease, equipment failure, loan default. A classifier trained on imbalanced data can lean entirely on the majority class and still post a great accuracy number, because accuracy counts every correct guess equally, and most of the guesses are on the easy, common class.
The number that actually matters for a rare, costly class is recall: of the defaults that really happened, how many did the model catch? On the untouched 24-vs-4 dataset below, recall for "default" is only 21%, even though overall accuracy looks fine at 88%.
Three fixes all work by giving the training set a more even class balance before fitting anything: undersampling (remove majority points), oversampling (duplicate minority points), and SMOTE (synthesize new minority points instead of copying existing ones).
Reading the demo below
-
1
Pick a method
Watch how each one changes the actual training points, not just a count.
-
2
Same held-out test set, every time
A small k-NN classifier is retrained on each version and scored on one fixed, still-imbalanced 80-vs-14 test set.
-
3
Watch recall vs. precision trade off
All three methods lift minority recall a lot. They don't all do it equally cleanly.
Raw training set: imbalanced
Repaid (majority) Defaulted (minority)
Training set balance
Classifier performance (same test set)
- Accuracy
- —
- Minority (default) recall
- —
- Minority (default) precision
- —
- Majority (repaid) recall
- —
—
All four methods, side by side
| Metric | Raw | Undersample | Oversample | SMOTE |
|---|
Undersampling is the simplest fix and the easiest to explain, but it throws away real majority-class data; with only 24 points to begin with, that hurts. Oversampling keeps every original point but only ever repeats what's already there, so the model can memorize exact duplicates instead of learning the minority region's real shape. SMOTE keeps all the majority data and adds genuinely new minority points along the lines between real ones, which is why, above, it matches or beats the other two on every metric at once instead of trading one off against another.
The jargons
Imbalanced data breaks the metric before it breaks the model: accuracy quietly rewards ignoring the minority class. The last three terms are the fixes for that, each trading simplicity against how much information it adds or throws away.
One class vastly outnumbers another in the training data: fraud, disease, and default detection are almost always imbalanced this way.
A model can score high accuracy on imbalanced data purely by favoring the majority class, while being nearly useless at the task that actually mattered.
Of every real minority-class case, the fraction the model actually caught. The metric the accuracy paradox hides.
Randomly discard majority-class points until the classes are balanced. Simple, but throws away real data.
Randomly duplicate minority-class points (with replacement) until the classes are balanced. Keeps all data, but adds no new information.
Synthetic Minority Oversampling Technique: generates new minority points by interpolating between a real point and one of its nearest minority neighbors.