From prompt to context

While LLMs were used for one-off questions, answer quality depended on how the prompt was worded. In assistants and agents, the model receives far more than an instruction at every step:

Context layer What it contains Who owns it
System instructions Role, tone, restrictions, answer rules Product team
Data Product records retrieved through RAG Search and product feed
History Turns of the current conversation Application
Tools Descriptions of functions available through function calling Engineering
Memory Shopper preferences from past sessions Profile and storage

Context engineering means working across all the layers at once: what to include, how much, in what order and what to leave out.

Core techniques

In July 2025 LangChain grouped the practices into four categories; ordering is worth adding to them.

  • Write. Save information outside the window so it can be brought back later: agent notes, session summaries, agent memory.
  • Select. Pull only what is needed into the window: 10–20 relevant product records instead of a whole category, 2–3 tools instead of all of them.
  • Compress. Keep only the tokens the task requires: summaries of old turns, short product records without HTML markup.
  • Isolate. Split the task between sub-agents with separate, clean windows — for example, searching for compatible accessories apart from the main conversation.
  • Order. Models make worse use of the middle of a long input (“Lost in the Middle”, Liu et al., 2023), so rules go at the start and the current question at the end of the context window.

In practice: a shopping assistant’s context

Put into the context Leave out
Answer and merchandising rules: priority brands, no out-of-stock items The whole catalogue or an entire category
The current request and the latest turns The full history of every past session
10–20 product records: title, price, availability, 3–5 key attributes Full HTML descriptions and SEO copy
The basket and the selected filters Raw event logs
A short preference summary: size, budget, favourite categories Phone number, address and other personal data the answer does not need
Descriptions of the tools needed at this step Prices and availability from earlier turns without a refresh

Important: every change to the context is a change to the product. New rules, a different set of product fields or a different amount of history are compared in an A/B test on conversion and revenue per visit, not by eye.