Skip to main content
Back to Experimentation

Experimentation · Reading the Results

The Bootstrap in Practice

The one idea: The bootstrap is a mechanical recipe you can run by hand on a tiny table: resample your data with replacement, recompute the statistic, write it down, repeat. Do that enough times and the spread of the results is your confidence interval — no formula, no distributional assumption, just arithmetic repeated many times.
⚠️ The trap: I don't have a formula for the standard error of a trimmed mean, so I can't put a confidence interval on it. That feels right and it's wrong. The bootstrap needs no formula for any statistic — mean, median, trimmed mean, a ratio, a correlation — because it estimates the sampling distribution directly from resampled data instead of deriving it algebraically.

Confidence Intervals and the Bootstrap made the case for reporting a range instead of a verdict, and introduced the bootstrap's core move: resample your own data with replacement, recompute the statistic, repeat. This lesson doesn't re-argue that case — it puts you at the table with a real, small dataset and walks the mechanics end to end, by hand, so the recipe stops being an abstraction.

The dataset

Eight users' time-on-page, in seconds, from a landing page test's control arm:

User12345678
Seconds121592214111810

The statistic of interest is the mean. On the original sample: (12+15+9+22+14+11+18+10) / 8 = 111 / 8 = 13.875 seconds. That single number is your point estimate. The question the bootstrap answers: if…

Keep reading: The Bootstrap in Practice

Unlock the full Experimentation track — every lesson, the AI tutor, and full mock assessments.

  • Full lesson content
  • AI tutor for this section
  • Practice questions