Maximum Likelihood Estimation Intuition

The one idea: Maximum likelihood estimation (MLE) asks one question: out of every possible value a parameter could take, which one makes the data you actually observed the most probable? You don't need calculus to feel the logic — you need to compare a handful of candidate parameter values, compute how likely each one makes your data, and pick the winner.

⚠️ The trap: "We flipped a coin 10 times and got 7 heads, so the MLE says the coin's true bias is exactly 70% heads forever." The MLE from a small sample is still just an estimate from a small sample — it's the best-supported single value given this data, not a certified fact about the coin. A maximum-likelihood estimate is only as trustworthy as the sample it was squeezed out of; report it with the sample size attached, or it reads as more certain than it is.

The question MLE actually answers

Every estimation problem has the same shape: there's some unknown parameter (a coin's true probability of heads, a population's true defect rate, a website's true conversion rate) and some observed data (flip outcomes, defect counts, converted visitors). MLE flips the usual direction of the question. Instead of "given this parameter, what data would I expect," it asks: "given this data, which parameter value makes it most probable that I'd see exactly this?"

The parameter value that wins that comparison — the one that makes the observed data the least surprising, the most likely to have happened — is the maximum likelihood estimate.

Worked example: estimating a coin's bias

Flip a coin 10 times and observe **7 heads,…