🤖 AI / ML

Normalized Discounted Cumulative Gain

ndcg · normalized discounted cumulative gain · normalised discounted cumulative gain · dcg · idcg · ndcg@k

Definition

NDCG (normalized discounted cumulative gain) is a ranking quality metric that sums the relevance of positions with a logarithmic discount by position and normalises the result against the ideal ranking.

Why a metric needs a positional discount

Simple metrics such as precision@K ignore where the relevant result sits. A shopper shown the right
product in position 10 probably never sees it — yet precision@10 scores that exactly the same as a
hit in position 1.

NDCG fixes this: the contribution of the item at position i is discounted by log2(i + 1). The
item in position 1 contributes in full, the one in position 2 contributes half as much, and so on.

The NDCG formula

DCG@K = Sum (rel_i / log2(i + 1))  for i from 1 to K

NDCG@K = DCG@K / IDCG@K

where IDCG@K is the DCG of the ideal ranking
(every relevant product placed first)

An example for K=4 with relevance values [3, 2, 3, 0]:

DCG@4 = 3/log2(2) + 2/log2(3) + 3/log2(4) + 0/log2(5)
      = 3/1 + 2/1.585 + 3/2 + 0
      = 3 + 1.26 + 1.5 = 5.76

Ideal order [3, 3, 2, 0] → IDCG@4 = 3 + 1.89 + 1 + 0 = 5.89
NDCG@4 = 5.76 / 5.89 = 0.978 approximately

Applications in e-commerce

Evaluating search. How well the search algorithm puts the products shoppers actually buy into
the first positions.

PLP ranking. Personalized category sorting — you need to confirm that the products interesting
to a specific shopper sit higher. NDCG measures that directly.

Offline evaluation of recommendations. Before a new algorithm goes into an A/B test, it is
scored on historical data: NDCG of the new algorithm against the current one.

Tip: use NDCG@K with a K that matches the number of slots actually displayed. For a
recommendation block of 6 products, use NDCG@6. For a catalogue page where 8 to 12 positions are
visible above the fold, use NDCG@10 or NDCG@12.

The limits of NDCG

It requires relevance labels, from experts or from behaviour logs. Without them the metric
cannot be computed.
Offline NDCG is not online conversion. An algorithm with high NDCG on historical data can
still deliver a smaller conversion lift in an A/B test, because of novelty effects or the
behaviour of one particular segment.
It ignores diversity. Showing 10 relevant but near-identical products yields a high NDCG and
poor UX.

Metric	Accounts for position	Accounts for diversity	Needs relevance labels
NDCG	yes	no	yes
MAP	yes (binary)	no	yes
Precision@K	no	no	yes
Coverage	no	yes	no

Frequently asked questions

What is the difference between DCG and NDCG?

DCG is an absolute sum of weighted relevance values and depends on list length, so comparing DCG across queries of different lengths is not valid. NDCG normalises DCG against the ideal DCG (IDCG) of the same query, which keeps the value inside the range 0 to 1 and makes the metric comparable.

How is relevance defined in e-commerce for NDCG?

Usually from behaviour logs: a purchased product scores relevance 3, an add to cart scores 2, a view scores 1, and an ignored item scores 0. The exact scale depends on the business question.

When is NDCG better than precision and recall?

NDCG is preferable wherever the order of results matters — search, PLP ranking. Precision and recall ignore position, NDCG does not. For recommendation widgets with a fixed number of slots, NDCG@K is the usual form, for example NDCG@10.