Upper Confidence Bound

The one idea: When you don't know which option is best, the fastest way to find out is to try the option you're most uncertain about among the ones that could plausibly be the best — not the one you merely haven't tried in a while. UCB turns "how sure am I?" into a number and adds it to your current estimate, so every choice is scored on best-guess-performance-plus-benefit-of-the-doubt, and the doubt shrinks automatically as you gather data.

⚠️ The trap: Running a fixed 50/50/50/50/50 A/B/C/D/E test until you hit "statistical significance," then switching to the winner, feels rigorous. It also means every impression shown to the four losing variants during the entire test was a wasted conversion. **A bandit algorithm updates which variant gets shown during the test, not just after it — the test and the optimization are the same process, so you stop paying full price to run the experiment.**

The problem: which slot machine pays out?

The classic setup, and the one that gives the field its name: you're in a casino facing five slot machines ("one-armed bandits" — so called because they steadily take your money). Each machine pays out at its own fixed but unknown rate. You have a fixed number of pulls. Which machine do you pull each time, to walk out with as much money as possible?

Swap the casino for a product team and nothing about the math changes. You're running five ad creatives (call them A through E) and want to maximize clicks. Each time a visitor loads the page, you choose one variant to show — that's a "pull." A click is…