Causal Inference · Cheat Sheet
The Core Problem
Cheatsheet: The Core Problem
One page for Topic 1. If a comparison lands on your desk, work down this page top to bottom.
The bias equation (memorize this)
Observed difference = ATT + BIAS
(Notation: E[...] means "the average of"; T=1 is the treated group, T=0 the control group; Y1/Y0 are the outcomes if treated / if not.)
| Term | Formula | In English |
|---|---|---|
| Observed difference | E[Y | T=1] − E[Y | T=0] | The dashboard number: treated average minus control average |
| ATT | E[Y1 − Y0 | T=1] | True effect of treatment on the people who got it |
| BIAS | E[Y0 | T=1] − E[Y0 | T=0] | How the treated would have fared untouched, minus how the controls actually fared — how different the groups already were |
| ATE | E[Y1 − Y0] | Average effect if everyone were treated (differs from ATT under targeting) |
Bias = 0 exactly when groups are exchangeable — same untreated destiny. Randomization guarantees it; every other method in this track is a strategy for approximating it.
The four explanations for any correlation
| # | Explanation | Business example |
|---|---|---|
| 1 | X causes Y | The email actually reduces churn |
| 2 | Y causes X (reverse) | Committed users explore more, so they find Feature X — retention causes adoption |
| 3 | Z causes both (confounder) | Problems drive both support contact and churn |
| 4 | Coincidence at scale | Track 200 metrics, some pairs correlate by chance — guaranteed |
Default status of any observational correlation: "interesting hypothesis." Not "finding."
Prediction vs intervention
| Prediction | Intervention | |
|---|---|---|
| Question | "Which users will churn?" | "Will this email reduce churn?" |
| Formal object | P(Y | X = x) — filter a subpopulation | P(Y | do(X = x)) — set X, new world |
| Right tool | Any accurate ML model | Experiment or causal design |
| History answers it? | Yes | No — history recorded who was chosen for treatment |
| Accuracy certifies it? | Yes (holdout accuracy) | Never — a model can be accurate because of selection bias |
Shalizi's test: "waxing frequency significantly predicts top speed, controlled for make and age." If the causal reading is absurd there, it's unsupported for "targeted ads drive purchases" too — same structure, less intuition.
The onboarding sign-flip (optimal assignment)
10 accounts; manager assigns premium onboarding to the shakiest accounts.
| Truth (all-seeing view) | Naive comparison (analyst's view) | |
|---|---|---|
| Effect of premium onboarding | ATE = +0.8 months | Premium group 7.0 vs standard group 8.0 → −1.0 months |
| Why | Averages both potential outcomes over everyone | Premium accounts' Y0 was 2.4, not 8.0 — groups never comparable |
The lesson: optimal targeting alone flips the sign. Same math, churn edition: retention program targets at-risk accounts → program "correlates with" 34% churn vs 12%. Rational choice pushes against comparability — the better the targeting, the worse the naive comparison lies.
"Who chose the groups?" — the first audit question
| Who chose | Likely bias | Example | Naive comparison |
|---|---|---|---|
| A coin (randomized) | Zero | A/B test | Trust it (argue about power, not bias) |
| Targeting the struggling | Negative — buries good programs | Retention outreach, support, remedial training | Effect looks worse than truth, sign may flip |
| Targeting the promising | Positive — flatters the treatment | Tablets in rich schools, ads aimed at likely buyers | Effect looks better than truth |
| Users themselves | Usually positive | Feature adoption, webinar attendance | Adopters differ in motivation — hypothesis only |
| Market conditions | Often sign-flipping | Hotel discounts in slow seasons | Correlation points backwards |
Ten-second checklist before believing any comparison
- Who chose the groups? (Coin → proceed. Anyone optimizing → suspect bias, guess its sign from the row above.)
- What would the treated group have done untouched? (Is the control group a credible stand-in for that?)
- Is the "cause" a response to conditions? (Discounts, support, remedial onboarding → expect the sign to flip.)
- Is this a prediction question or an intervention question? (Only the first is answered by fit/accuracy/significance.)
- Causal verbs on descriptive slides? ("Drives," "leads to," "generates" — each owes evidence beyond a two-column table.)
- Big effect ≠ causal effect. Selection routinely produces 2x-5x gaps; significance at large n is guaranteed, not informative.
Further reading
The ideas in this track draw on the standard causal-inference literature. To go deeper: Cunningham, Causal Inference: The Mixtape (potential outcomes, quasi-experimental designs); Facure, Causal Inference for the Brave and True (selection bias, applied Python); Shalizi, Advanced Data Analysis from an Elementary Point of View (prediction vs intervention, the limits of regression). All examples and numbers in these lessons are our own.
Aligned to the Cunningham’s Mixtape, Facure’s Brave and True, Shalizi’s ADAfaEPoV, and Chernozhukov et al.’s CausalML · last reviewed 2026-07-08.
Make this cheat sheet yours
Personalize this sheet — focus it however you study, or build one from the exact questions you keep getting wrong.