Multi-Armed Bandit (MAB)

MAB · Multi-Armed Bandit · Autopilot · Bandit Testing

Definition

Multi-armed bandit solves the exploration/exploitation trade-off: it balances trying out new variations against exploiting the best one found so far. The clearer the lead of one variation, the more traffic it receives automatically.

Exploration vs exploitation

A classic A/B test is pure exploration: we spend the same amount of traffic on both variations right
up to the final decision. MAB balances the two:

Exploration — it keeps sending a small share of traffic to the weaker variation, in case the
data so far is misleading
Exploitation — it gives most of the traffic to the leading variation while the test is still running

Day 1:  A — 50%,  B — 50%  (not much data yet)
Day 5:  A — 35%,  B — 65%  (B starts to pull ahead)
Day 10: A — 10%,  B — 90%  (B is clearly better)
Day 14: A — 3%,   B — 97%  (B has all but monopolised the traffic)

Thompson sampling — how the algorithm works

Each variation carries a beta distribution, Beta(α, β), where α = conversions and β = non-conversions
For every new visitor the algorithm samples a value from each variation’s distribution
The visitor is shown the variation with the highest sampled value
As data accumulates the distributions narrow → the leader receives more and more traffic

When MAB beats a classic A/B test

Scenario	Classic A/B	MAB
Long test, high traffic	✓ Preferred	Works fine
Short seasonal test	May never reach its sample	✓ Optimal
Three or more variations	Needs a Bonferroni correction	✓ Native
Strict p-value significance required	✓ Preferred	Not guaranteed
Recommendation algorithms	Works fine	✓ Optimal

The limits of MAB

MAB is not perfect. Its limitations:

It does not guarantee strict statistical significance in the classic sense
It can pick a winner too quickly on thin traffic, off early fluctuations
The final result is harder to interpret when the decision has to be documented

Frequently asked questions

How is MAB different from a classic A/B test?

A classic A/B test keeps a fixed 50/50 split until the end. MAB reallocates traffic dynamically toward the better variation, which minimises the conversions lost on the weaker one. The trade-off is statistical rigour: MAB does not always guarantee a given significance level.

What is Thompson sampling?

Thompson sampling is a MAB algorithm. Each variation carries a beta distribution over its conversion rate. For every new visitor the algorithm samples a value from each variation's distribution and shows the one that drew the highest value. Over time the leading variation is served more and more often.

When is MAB a better choice than a classic A/B test?

MAB fits when (1) the test is short or seasonal and there is no time to accumulate a classic sample; (2) the cost of a mistake is high and every day spent on the weaker variation is expensive; (3) there are more than two variations, where MAB scales better; (4) there is no hard requirement for statistical significance.

Can MAB be used for recommendation algorithms?

Yes, and it is one of the best cases for it. MAB automatically shifts traffic between algorithms toward whichever is converting better right now. When the season or the assortment changes, MAB adapts faster than a classic A/B test.

Gravity Field