Context Engineering in LLMs
Every time you interact with AI—a language model like ChatGPT or something similar—it completely "forgets" you as soon as the conversation ends. So how does continuity hold when you send the next message? That is exactly what we will explore today. It is called context engineering, and it is in fact a fundamental aspect of how we build agentic systems—how we preserve continuity in dialogue with a large language model (LLM). By the end of this breakdown, you will understand how prompts, memory, files, and tools interact as a single whole within context engineering. Understanding this is critically important if you plan to work seriously with agentic or complex LLM-based workflows. So let us dive in and build a clear picture of what context engineering actually is.
Today's topic is simple but extremely important. We will break down all the components that make up context engineering and understand how much it differs (or does not differ) from so-called prompt engineering, which you have probably already heard about.
As mentioned, every time you interact with an LLM, the model has no idea who you are. The only way to "introduce" yourself is to do it anew each time. And this information—your name, previous messages, uploaded files—is context. You have certainly heard the term "context window"; you know it is limited in size: you cannot cram an infinite number of files or text into it. Watch your context—do not waste it. Essentially, the whole point is that on every request to an LLM you must pass it everything it needs to fulfill your request.
For example, if you ask: "Extract such-and-such information from my notes"—you will have to attach the notes themselves. If you sent them in a previous message, the LLM remembers nothing about them in the next one. That is why systems like ChatGPT manage context for you. Context here is simply a "bucket" into which all information sent to the model is placed: a conceptual container holding everything the model can use to shape its response.
Now let us look at the components of context in more detail.
The better you understand this structure—trust me—the more precisely you will control LLM behavior. This is true both for simple LLM conversations and for building full agentic systems in the future. This information is critically important—literally for everyone who works with LLMs, and today that is almost all of us.
1. Memory
The first concept worth unpacking is memory. This term is borrowed from neuroscience and means information from the past that is somehow preserved and available in the present moment. With LLMs the situation is analogous: "memory" is data from previous steps of the dialogue.
For example, if you uploaded a file to ChatGPT and asked for a summary, then in the next message said: "Now highlight the key tasks"—the LLM must "see" the same file again, otherwise it will not understand what you are talking about. All content (including the file and previous messages) must be sent again in the context.
Personal information can also count as memory: for example, ChatGPT may "know" that you are in Russia and accordingly format dates or offer localized answers. This information is added to the context automatically—you did not request it yourself, but the system does it for you.
However, from an engineering standpoint, you are responsible for managing memory. Imagine a long dialogue: thousands of messages. Sooner or later the total volume will exceed the context window limit. What to do? Here an engineering solution comes in: the system has to summarize the earlier part of the conversation, extracting key details, and keep only the most recent (say, the last 10 messages), which are given the greatest weight—because activity is concentrated there right now.
It is precisely because of this compression that drift occurs. The longer the dialogue lasts, the weaker the model "remembers" your earlier instructions: "Didn't I ask you not to use tables?", "Why did you go back to the old format again? I said—only visualizations on the canvas!"
All of this is a consequence of automatic history management: the system tries to fit everything important in, but something gets lost along the way.
2. Files
Files are the most obvious part of context. You upload them explicitly. For example, you drag notes into ChatGPT and ask: "Please make a summary". In that case the file becomes part of the context.
Screenshots are especially useful. For example, you take a screenshot of a presentation slide and ask: "Can you create a prompt based on this?"—the model really can do that. I have used this approach many times myself: uploaded a slide to ChatGPT and got a ready-made, well-structured prompt. Tip: start actively using screenshots—they are a powerful but underappreciated tool.
3. RAG (Retrieval-Augmented Generation)
RAG—Retrieval-Augmented Generation, or "generation with augmented retrieval." This is a technical term describing a process: before sending a query to an LLM, the system first searches its own database for information relevant to your question and slips it into the context.
Example: you visit an online store's website and ask: "Which candies were the most popular last month?"
If you simply send this question to ChatGPT, you will get a general answer based on publicly available data. But the store owner wants only their three products to appear in the answer.
What happens "under the hood": the system analyzes your query, searches for matching records in the database, and adds them to the context—without you noticing. Then the LLM forms a response relying on both your question and this loaded information. Voilà—you get: "Here are the top 3 candies we recommend: …"—a precise, targeted, commercially effective answer.
4. Tools
Tools are an integral part of agentic systems. You may have heard the term tool calling. Modern LLMs can "understand" when they need to reach out to an external tool—but not execute it directly, but rather form a request to invoke it.
An LLM is, in essence, a decision engine that generates a response as text, an image, and so on. On its own it does not make HTTP requests or browse databases. Instead it says:
"I don't know the weather in London tomorrow. But I have a tool—web search. Please run it with the query 'weather in London tomorrow', collect the results and return them to me—then I will give an accurate answer."
In practice it looks like this:
- In the system prompt it is specified: "You have access to a tool: web search. Description: simple internet search. Only the query text is required."
- The user asks: "What's the weather in London?"
- The LLM responds: "Need to call the
web_searchtool with the argumentweather in London tomorrow." - Your system executes the call, receives data (for example, JSON with a forecast), places it in the context (similar to RAG), and only then the LLM generates a human-readable answer.
This mechanism is very similar to RAG—with the difference that data sources are no longer static databases but dynamic services.
5. Prompt Engineering—and why it belongs here, in the blue block
This is a key element. You may have already heard about prompt engineering. And yes, it remains important, especially when we communicate with systems in natural language. Even though modern models have become more robust to phrasing, how you structure a request still determines the quality of the answer.
But why do we highlight so many components—tools, files, memory, prompts? Because each of them is an adjustment knob you can turn independently.
When you consciously start working with context—even in ChatGPT—you move from passive user to engineer:
- What happens if you add examples?
- What if you tighten constraints?
- What if you change the model's role?
By experimenting with each parameter, you will quickly find the optimal combination.
Prompt structure: breaking it down piece by piece
A prompt is ultimately just a string of text. But to manage it effectively, it helps to break it into logical blocks:
1. Role
Sets the model's "personality":
"You are an expert on everything related to Ryan Reynolds"
(Which, you will agree, should be a mandatory competency for any self-respecting system.)
2. Personality / Temperament
Further sets the communication style:
"You are very talkative but hate being verbose"
(Yes, it is a paradox—and the model will handle it. It is precisely such nuances that make answers feel alive.)
3. Request
Your actual task:
"List five best action movies with Ryan Reynolds from the last five years"
4. Format
How the answer should look:
"Return the list as a numbered enumeration. No tables. Only brief descriptions—one sentence each"
(You will be surprised, but an explicit format specification sharply increases accuracy.)
5. Examples
A severely underappreciated element! If you expect strict structure—give good and bad examples:
"Good: '1. Deadpool & Wolverine—a fast-paced crossover…'
Bad: 'Well, there was some superhero movie…'"
This is a kind of "training in the prompt"—and it works surprisingly well.
6. Constraints
Clear prohibitions:
"No movies with Ryan Gosling. Only Reynolds. Period."
7. Evals
Automatic or manual validation of the result. In agentic systems a second LLM call is often used:
"Check: are all the movies really with Reynolds? Are all the dates within 2020–2025?"
But you can embed this in the prompt as well:
"Make sure the list does not include 'Barbie,' even if you really want to".
Practical example
I took a screenshot of this breakdown, uploaded it to ChatGPT, and asked:
"Create a prompt based on this file to find the best Ryan Reynolds movies".
The result was excellent. Let us test it:
Query: "What five best action movies with Ryan Reynolds came out in the last five years? No Gosling. And make the answer as verbose as possible, but with sarcasm—like Reynolds himself".
ChatGPT's answer:
"So, you want the TOP 5 action movies from the last 5 years—and only with Ryan Reynolds? No Go-go-Goslings? Challenge accepted.
It will hurt with puns, sarcastic—painfully 'Reynolds-esque.' And yes, I will talk too much. Sorry. But when you're in denial—you just don't stop…
- Deadpool & Wolverine
- The Adam Project
- Free Guy
- Red Notice
- The Hitman's Wife's Bodyguard"
Works perfectly.
***
That is it: context engineering is the art and science of assembling the complete "information package" you send to an LLM. I deliberately separated the components so you can see: each one is an independent control lever. Depending on the task—you increase the role of memory, add RAG, connect tools, tune the prompt down to the smallest detail.
This view may seem basic—and that is good! Because it is accessible now, even to those who work only with ChatGPT.