Colliders and Bad Controls

The one idea: Adding more control variables is not automatically safer. Some variables, once you control for them or filter your data on them, manufacture a relationship that was never there — so every control needs a reason, not just a spot in the spreadsheet.

⚠️ The trap: "When someone challenges a finding, adding more control variables makes the analysis more rigorous." That feels right and it's wrong. Controlling for a collider — a variable your treatment and outcome both cause — invents a relationship that was never there, and a filter hidden in your WHERE clause ("among users who…") counts as controlling for it just as hard.

There is a reflex in every analytics org: when someone challenges a finding, you add more control variables. Controlled for region? Add tenure. Someone mentions plan tier? Throw it in the regression. The instinct feels rigorous — more controls, fewer excuses. This lesson exists to break that reflex, because it's backwards often enough to cost real money: controlling for the wrong variable doesn't just fail to remove bias, it creates bias that wasn't there.

The mechanism is the collider from the last lesson — a variable that two others both point into. Written out, it is X → C ← Y: X and Y each cause C, but nothing directly connects X and Y. In the raw data, X and Y are independent — the collider blocks the path. Condition on C (that is, split the data and look only within groups that share the same value of C), or filter your dataset on C, and the path snaps open. And here's the killer detail:…