Class Imbalance and Resampling Techniques

The one idea: The accuracy-paradox lesson showed you how to measure a model on imbalanced data honestly — precision, recall, and AUC instead of a misleading accuracy number. This lesson is about fixing the model itself: resampling techniques change what the model actually trains on, so it stops treating the rare class as a rounding error it can safely ignore.

⚠️ The trap: Resampling fixes the training data, never the test data. If you oversample, undersample, or SMOTE before splitting into train and test — or apply it to the test set at all — you're evaluating the model against a fictional world where fraud is common, and every metric you report from that point on is meaningless outside that fiction. Resample training data only, after the split, exactly the same rule as fitting a scaler.

Why the model itself needs to change, not just the metric

The previous lesson's fraud example — 10,000 transactions, 100 genuinely fraudulent (1%) — showed a logistic regression catching only 30% of real fraud (recall 0.300) even while posting a headline-friendly 99.2% accuracy. Switching from accuracy to precision/recall/AUC told you the truth about that model. It didn't make the model better. The model still only caught 30% of fraud, because during training, 99 correctly-classified "not fraud" rows contribute vastly more to the loss function than one fraud row does — the optimizer's easiest path to a low loss is to mostly ignore the minority class, and it takes that path by default.

Resampling techniques address the cause directly: they change the class balance the model actually learns from, so getting the minority class right stops…