Data Cleaning and Visualizing Distributions

The one idea: Before you trust any summary statistic, look at the shape of the data and hunt for values that don't belong — a single data-entry error can double a mean while leaving the median almost untouched, and shape (histogram) plus a five-number summary (box plot) catch problems a bare average never will.

⚠️ The trap: Trusting the mean as your data-quality check. A typo that turns $58 into $580 doesn't announce itself in an average — it just quietly drags the average up and looks like "the number went up this month." The mean has no immune system against bad data; you need z-scores, IQR fences, and a look at the shape to catch what the mean will happily absorb and hide.

Every real dataset has typos, sensor glitches, and copy-paste errors mixed in with genuine values. The tools in this lesson exist to find them before they distort a decision, and to give you an honest visual read on the data's shape once it's clean.

Z-scores: how many standard deviations away is this point?

A z-score converts any value into "number of standard deviations from the mean":

z = (x − mean) / s (sample) or z = (X − μ) / σ (population)

Positive z sits above the mean, negative below, z = 0 is exactly at the mean. The common rule of thumb: treat |z| > 3 as a candidate outlier (this follows straight from the empirical rule — under a roughly normal distribution, virtually all data sits within 3 SDs, so a point beyond that is rare enough to warrant a second look).

Worked example. Ten employees' one-way commute times, in…