Omitted Variable Bias and Sensitivity Analysis

The one idea: You can never prove you've controlled for every confounder — "I included the variables I had" is not the same claim as "I included the variables that matter." Sensitivity analysis doesn't fix that; it quantifies it, answering a narrower and answerable question: how strong would an unobserved confounder have to be to erase this result? A small answer is a warning. A large answer is (partial) reassurance.

⚠️ The trap: "The result is statistically significant, so it's solid." That feels right and it's wrong. Significance measures whether the estimate is distinguishable from zero given sampling noise. Robustness measures whether the estimate would survive a plausible confounder. They are different properties, computed from different quantities, and a result can score high on one and low on the other — a highly significant estimate can also be the one most easily explained away.

The last lesson showed omitted variable bias doing real damage: dropping one confounder more than doubled a coefficient, exactly, by a formula. That example had a clean resolution — the missing variable (ability) was identified and added back in. Most of the time you don't get that luxury. You control for what you measured; you can never be certain you measured everything that matters. Sensitivity analysis, formalized by Cinelli and Hazlett's robustness value framework (implemented in the sensemakr / PySensemakr package), is the tool for reasoning rigorously about the confounder you didn't measure — without needing to know what it is.

The logic: bound the enemy, don't identify it

You cannot run a regression on a variable you don't have. What you can do is ask:…