Prediction vs Intervention

The one idea: There are two completely different questions you can ask of data — "given what I see about someone, what will happen to them?" and "if I change something, what happens because of it?" Dashboards and models answer the first. Every budget decision needs the second. Using one to answer the other is the most expensive mistake in analytics.

⚠️ The trap: "The regression is statistically significant and we controlled for everything, so the coefficient tells us what happens when we pull that lever." That feels right and it's wrong. A regression only describes how outcomes differ across groups you already observed — it says nothing about what changing the variable does; "we controlled for everything" didn't rescue the car-waxing regression, and it won't rescue yours.

There are two kinds of questions you can ask of data, and confusing them is the most expensive mistake in analytics. The first: "Given what I observe about a customer, what will happen to them?" The second: "If I do something to a customer, what will happen because of it?" They sound like cousins, but they are different species. The statistician Cosma Shalizi has spent a career pointing out that regression — the workhorse of business analytics — answers only the first while everyone quietly uses it for the second.

Here's the distinction in notation, in plain English first. Conditioning is looking; do-ing is acting. Written out, P(Y | X = x) is the distribution of outcomes among units you observed to have X = x — filtering down to a subpopulation that already exists, with everything that made them that way still attached. P(Y | do(X = x)) —…