Feature Scaling and Train Test Split

The one idea: Distance- and gradient-based algorithms can't tell the difference between "a big number" and "an important number" — a feature measured in dollars (0–500,000) will silently steamroll a feature measured in years (0–40) unless you put them on the same scale first. And that scaling has to happen after you split off a test set, or you've let the model peek at data it's supposed to be evaluated on.

⚠️ The trap: Fitting a StandardScaler (or any scaler) on the full dataset and then splitting into train/test. It runs without error, the numbers look normal, and it quietly leaks information from the test set into training — because the mean and standard deviation used to scale every training row were computed with the test rows baked in. Your reported accuracy will be optimistic in a way you can't detect just by looking at it.

Why scaling matters: the unit-conversion analogy

Imagine comparing two people's "distance from average" using age in years (range roughly 20–60) and income in dollars (range roughly 30,000–200,000). If you just subtract and compare raw numbers, a $10,000 income difference will dwarf a 10-year age difference in any calculation that adds or measures distance between them — not because income is more important, but because it's measured in bigger units. It's the same mistake as comparing "3 kilometers" to "3 miles" without converting them to the same unit first — the numbers look comparable, but they aren't measuring the same thing.

This isn't cosmetic. Two whole families of algorithms depend on features being roughly comparable in scale:

• Distance-based algorithms (KNN, K-means, SVM with RBF kernel): they literally compute distances between points…