The problem with simple attribution
The most common way to measure recommendation performance is to count the revenue from shoppers who
clicked the widget. It is convenient and wrong: among those shoppers were people who would have
bought anyway. The real added value is only the incremental part.
Important: attributed revenue with no control group is an upper bound on the effect, not the
effect. The gap can be substantial — particularly in popular categories, where the shopper would
have found the product regardless.
Attribution methods in e-commerce personalization
Click-based attribution
The simplest: credit all revenue from shoppers who clicked a recommendation within the attribution
window. It overstates the effect. Useful for operational monitoring, not for strategic decisions.
A/B attribution
The correct method: compare two groups, with and without the recommendation, and treat the revenue
difference as the incremental effect. It requires a configured test, and it gives a precise answer.
Group A (no recommendation): RPV = 4.80
Group B (with recommendation): RPV = 5.28
Incremental effect: +0.48 per visitor (+10%)
Holdout attribution
The long-run variant: a group of users is excluded from personalization for three to six months. The
difference in total revenue shows the cumulative effect of the whole programme rather than of one
widget.
Attribution window: how many days?
| Initiative type | Recommended window |
|---|---|
| Onsite recommendations | 7–14 days |
| Popup or banner | 1–7 days |
| Email with recommendations | 7–14 days |
| Push notification | 1–3 days |
The window choice moves the final number: 30 days inflates attributed revenue relative to 7. What
matters most is using the same window when comparing strategies.
Cannibalisation: the hidden threat
Cannibalisation is when a recommendation redistributes purchases inside a session without raising
total revenue. The shopper intended to buy product A, saw a recommendation for B and bought B
instead. The click is credited, and total revenue did not move.
An A/B test at the level of group revenue — revenue per visitor rather than revenue from clicks —
neutralises this automatically, because what was bought does not matter, only the total per group.