🤖 AI / ML

Mean Average Precision

mean average precision · map · map metric · average precision · ranking metric · map@k

Definition

Mean average precision (MAP) is a ranking quality metric computed as the mean of average precision values across all users or queries; it accounts for both precision and the position of relevant items in the list.

How MAP is computed

MAP is built on two levels of aggregation.

Step 1 — average precision for a single user:

Picture a list of 5 recommendations in which positions 1, 3 and 5 are relevant:

Position: 1    2    3    4    5
Hit:      yes  no   yes  no   yes

P@1 = 1/1 = 1.0  (the 1st relevant item was found at position 1)
P@3 = 2/3 = 0.67 (the 2nd relevant item was found at position 3)
P@5 = 3/5 = 0.60 (the 3rd relevant item was found at position 5)

AP = (1.0 + 0.67 + 0.60) / 3 = 0.76

Step 2 — MAP is the mean of the AP values across all users or queries.

Why order matters

The difference between two algorithms is often not how many relevant items they find, but where
they place them. A shopper typically scans the first three to five recommendations. An algorithm
that puts relevant products at the top scores higher on MAP — and has a real effect on clicks and
conversion.

MAP versus NDCG: which to choose

Criterion	MAP	NDCG
Relevance type	Binary (yes/no)	Graded (1, 2, 3…)
Sensitivity to order	High	High
Uneven number of relevant items	Handled correctly	Handled correctly
Typical use	Search, basic recommendations	Personalized recommendations

Tip: for offline evaluation of a recommendation engine, compute several metrics at once — MAP,
NDCG@10 and coverage. A single metric can mask problems along the other dimensions.

Practical limitations

MAP assumes that the full list of relevant products is known for every user. In practice that is
hard: interaction history is incomplete, because the shopper simply never saw part of the catalogue.
Offline metrics therefore have to be complemented by online experiments — A/B tests — where the
winner is decided by real conversion and revenue.

Frequently asked questions

How is MAP different from precision@K?

Precision@K looks only at the top K positions and ignores where inside them a relevant item sits. MAP accounts for the position of each relevant item, so an algorithm that pushes all relevant items higher scores better.

When should you use MAP and when NDCG?

MAP suits binary relevance, where an item either is or is not relevant. NDCG works better with graded relevance — when some products are a very good match and others are only a partial one. E-commerce recommendations more often use NDCG.

How do you interpret a MAP value?

MAP runs from 0 to 1. A value of 0.4 means that on average the relevant positions land at 40% of ideal placement. The absolute number depends on the task, so what matters is comparing two algorithms against each other.