🤖 AI / ML

AI Hallucination

Hallucination · LLM Hallucination · Model Hallucination

Definition

An AI hallucination is a language model producing factually incorrect or invented statements with high confidence, unsupported by any real data.

Why this matters in e-commerce

In a general chatbot a hallucination is annoying. In an AI shopping assistant it hits the business
directly:

A shopper learns a specification that does not exist, buys, is disappointed → a return plus lost
trust
The assistant states a wrong price confidently → a conflict at checkout
The bot invents compatibility, say a charger for a laptop → an incompatible order is placed

Causes and mechanics

An LLM does not know facts in the ordinary sense — it holds patterns from a training corpus. When
generating, it follows the most likely continuation rather than a verified fact.

Hallucinations cluster around three situations:

Questions about very specific, rare or recent data (a new SKU, a niche product)
Requests for numeric facts (prices, specifications, dimensions)
Conflicts between several sources in the training data

Mitigations

RAG

The most effective approach for e-commerce. Before generating, the system retrieves relevant
documents from a vector database — catalogue, FAQ, category descriptions — and adds them to the
context. The model answers from those documents rather than from memory.

Prompt engineering

System prompt instructions reduce the rate: answer only from the provided catalogue; if the
information is absent, say so rather than inventing it.

Output verification

Post-processing: a separate model or deterministic logic checks that the SKUs and prices in the
answer really exist in the catalogue. References that do not resolve are removed or replaced.

Important: RAG does not eliminate hallucination entirely — when the context lacks the needed
information the model can still fill the gap. A correct architecture handles the not found case
explicitly and says so to the shopper.

Frequently asked questions

Why does an LLM hallucinate?

Language models are trained to predict the most likely next token, not the most accurate answer. When the model does not know, it generates plausible-sounding text from patterns in its training data. The confidence of the delivery does not correlate with factual accuracy.

How does RAG reduce hallucination?

Retrieval-augmented generation forces the model to work from specific documents in a knowledge base rather than from remembered facts. In e-commerce the assistant first retrieves the relevant product records from the catalogue and then answers strictly from what it found. A hallucination is only possible when the needed information is missing from the retrieved context.

How do you measure the hallucination rate in a shopping assistant?

Through automated evaluation: build a question set with known correct answers (product specifications from the catalogue), run the assistant and compare against the reference. The metrics are factual accuracy, faithfulness (how closely the answer follows the source) and hallucination rate.

Gravity Field

Hallucinations and the Gravity Field AI Shopping Assistant

The Gravity Field AI Shopping Assistant runs on a RAG architecture grounded in the client's product catalogue, which minimises hallucinations in answers about the assortment.

✦RAG over the real product catalogue — answers built on current data

✦Grounding — answers reference specific product records

✦Out-of-catalogue control — the assistant does not invent products that do not exist

Explore the platform →