Uplift

What uplift is and how to compute it

Uplift is the relative gain of a metric in the treatment compared with the control:

Uplift = (metric_variation − metric_control) / metric_control × 100%

An example:

CR(control)   = 2.0%
CR(variation) = 2.4%
Uplift(CR)    = (2.4 − 2.0) / 2.0 × 100% = +20%

RPV(control)   = 4.00
RPV(variation) = 4.40
Uplift(RPV)    = +10%

Tip: in e-commerce, compute uplift on revenue per visitor as the primary metric — it accounts
for conversion and order value at once. Conversion can rise while average order value falls, and
the reverse.

Relative versus absolute

Both numbers are needed for the full picture:

Relative uplift is convenient for comparing tests with each other. Absolute gain is what turns into
money. On a base of 100 million in annual revenue, a 1% uplift in revenue per visitor is a million
a year.

Incremental uplift in personalization

In personalization it matters to separate organic shopper behaviour from the effect of the
algorithm. That is what a holdout group is for — a cohort deliberately excluded from
personalization over a long period.

Incremental uplift is the metric difference between the personalized audience and the holdout. It is
the honest valuation of a personalization platform, cleared of seasonality and other factors.

Common misreadings

• Peeking — stopping at the first attractive uplift before the sample is there. A false positive
  is all but guaranteed.
• Ignoring SRM — if the actual group ratio differs from the expected one, the uplift is distorted.
• The wrong period — a test during a sale or a holiday draws an unrepresentative audience and an
  unrealistic uplift.
• A single metric — uplift on conversion without checking order value can hide a drop in basket
  size. Always look at revenue per visitor.