Difference in Differences

The one idea: When you launch something in one market and not another, "before vs after" and "us vs the control market" each hand you a wrong number. The fix is to take the launched market's change and subtract the change the control market saw anyway — what's left is the effect.

⚠️ The trap: "We launched in one market and signups jumped afterward, so the campaign caused that jump." That feels right and it's wrong. The control market drifted upward on its own in the same window; the real effect is the treated market's change minus the change the untreated market got anyway — usually a small fraction of the raw before-and-after number.

Your company launched something in one market and not another — an ad campaign, a pricing change, a new checkout flow rolled out by geography. Nobody randomized. Now the readout meeting has two slides fighting for the truth: one shows the launch market before vs after ("it went up 38!"), the other shows launch market vs holdout market after launch ("we're 64 behind the control market!"). Both slides use real numbers. Both are wrong.

Difference-in-differences (DiD) is the design that reconciles them — arguably the most-used quasi-experimental method in business, and one of the oldest tricks in the causal toolkit. It's also the method most often faked: plenty of decks wear DiD's vocabulary while actually showing a decorated before/after. This lesson teaches you the real thing and how to spot the costume.

The trick: borrow the trend, keep the level

Before/after comparisons fail because things trend — seasonality, macro conditions, product-wide growth all move the metric whether or not you launched…