Outlier Detection and Robust Statistics

The one idea: An outlier isn't just "a big number" — it's a point far enough from the bulk of the data that it can distort the very summary statistics (mean, standard deviation) you'd normally use to describe that data. Flagging outliers needs a rule (z-score or IQR), and describing data that contains outliers needs different tools (median, MAD) that don't get dragged around by the points they're supposed to be measuring past.

⚠️ The trap: "Average order value is $340, so a $12,000 order is 35 standard deviations out — must be fraud." Maybe. But that $12,000 order was probably sitting inside the standard deviation you just used to judge it — one whale can inflate the very yardstick you're measuring it against, making genuinely wild points look tamer than they are and making borderline points look more extreme than they are. **The mean and SD are not neutral referees when outliers are present — they're already partly built out of the outliers, so use them to flag outliers with real caution, and don't use them at all to describe data you know contains some.**

The central tendency and variability lesson established that mean and standard deviation are sensitive to extreme values. This lesson makes that sensitivity precise: how to actually flag an outlier with a rule instead of a gut feeling, and what to use instead of mean/SD once you know outliers are in the mix.

Two rules for flagging an outlier

The z-score rule. Convert each point to a z-score — (x − mean) / SD, in standard-deviation units — and flag anything beyond a threshold, commonly |z| > 3. A dataset of daily…