🤖 AI / ML

Matrix Factorization

matrix factorization · matrix factorisation · mf · latent factors · svd · als recommendations

Definition

Matrix factorization is a class of algorithms that decompose the user-item interaction matrix into the product of two lower-dimensional matrices, making it possible to model hidden preferences and surface relevant products.

The idea behind the method

E-commerce has an interaction matrix: rows are customers, columns are products, values are ratings,
clicks or purchases. The matrix is sparse — most shoppers have interacted with only a tiny
fraction of the catalogue.

Matrix factorization decomposes that matrix R into the product of two smaller matrices:

R = U x V^T (approximately)

U — the user matrix [N x K]
V — the item matrix [M x K]
K — the number of latent factors (10 to 200)

Prediction for user u and item i:
r_hat(u,i) = U[u] . V[i]

The model is trained so that the dot product of the vectors reproduces the known interactions as
closely as possible.

The main algorithms

Algorithm	Data type	Characteristic
SVD / SVD++	Explicit ratings (1–5)	The classic, the basis of the Netflix Prize
ALS	Implicit data (clicks, views)	Parallelises well, strong on large data
NMF	Any non-negative values	Interpretable factors
BPR (Bayesian Personalized Ranking)	Implicit data	Optimises ranking directly

In e-commerce the data is predominantly implicit — a shopper does not leave ratings, they browse
and buy — so ALS and BPR are used more often than SVD.

How to read the result

After training, every user and every product is represented by a vector of K numbers. Products with
similar vectors are similar in their interaction patterns. A shopper is recommended the products
whose vectors give the largest dot product with theirs.

Important: latent factors carry no explicit semantics. You cannot say that factor number 3 is
an interest in electronics. The algorithm finds abstract dimensions that describe the data, and
nothing labels them.

Where MF sits among modern algorithms

Matrix factorization remains a strong baseline: it works well, it is interpretable enough, and it is
computationally predictable. Modern neural approaches — two-tower models, Item2Vec, transformers —
beat it on accuracy given enough data, but demand more resources for training and inference. In most
e-commerce scenarios, MF with ALS is a sensible starting point before the architecture gets more
complicated.

Frequently asked questions

How is matrix factorization different from collaborative filtering?

Collaborative filtering is the broader concept, a methodology. Matrix factorization is one way of implementing it. User-based collaborative filtering looks for similar users directly; factorization works through hidden vector representations.

What are latent factors?

Hidden dimensions extracted from the data automatically. In e-commerce they can be read loosely as abstract preferences — a leaning toward premium products, an interest in a particular category — although the algorithm never assigns them explicit names.

How does matrix factorization handle cold start?

In its basic form it does not: for a new user with no interaction history the preference vector is unknown. Cold start is solved separately, through popular products, content information or hybrid models.

Which factorization algorithm should you choose?

ALS (alternating least squares) works well on implicit data such as clicks and views and parallelises across GPUs and clusters. SVD is best suited to explicit ratings. NMF yields non-negative vectors, which makes interpretation easier.