Randomization the Gold Standard

The one idea: A coin flip decides who gets treated without knowing anything about the people involved, so the treated and control groups would have turned out the same if you had done nothing. That is what makes a plain treated-minus-control comparison the honest answer — and it is the standard every other method is only trying to imitate.

⚠️ The trap: "To judge the new flow fairly, I'll compare the people who actually completed it against the control group." That feels right and it's wrong. The instant you filter to who completed, used, or opted into the treatment after assignment, you've swapped the coin flip for self-selection and the bias comes back — analyze everyone by the arm they were randomized into, compliant or not.

Every method in this track is trying to be one thing: a randomized experiment. Matching, difference-in-differences, instrumental variables — all workarounds for the situation where you couldn't flip the coin. Before learning the workarounds, you need to understand exactly what the coin flip buys you, because that's the standard every workaround is graded against.

The answer is more subtle than "it makes the groups the same." What randomization actually does is sever the connection between who gets treated and what would have happened to them either way — and once you see the claim stated that precisely, you'll understand why nothing else fully replaces it.

What the coin flip actually does

Recall the potential-outcomes setup: every user has two potential outcomes — Y1 if treated, Y0 if not — and you only observe one. In observational data, the treated-vs-untreated gap is the true effect *plus selection…