Artificial Intelligence, zBlog
Context Engineering Is How You Control the Economics of Production AI Systems
trantorindia | Updated: August 31, 2026
What is context engineering for LLM cost optimization?
Context engineering is the discipline of deciding, deliberately, what belongs in a language model’s context at each point in a task: what to include, compress, or drop, so the model has what it needs and nothing more. It is distinct from token counting because the goal is cost per successful outcome, not cost per token. Trim the wrong context to save tokens and you can easily increase retries or push failures downstream.
Somebody on my team asked last month why our Claude usage was so much higher than they’d budgeted for. My honest answer was: I don’t know. We went and looked, and the answer turned out to be more interesting than “use fewer tokens.” It was really a question of how we were managing context. That’s worth writing down properly.
The short version: most of what you’re paying for isn’t the question you just asked. It’s everything that came before it. Once you treat that as an engineering problem instead of a habit problem, it turns into most of what production AI cost control actually is.
The Part That Surprised Us
Claude doesn’t hold onto anything between messages on its own. There’s no background memory quietly keeping track of a conversation. Instead, every time you send a message, the whole conversation up to that point gets sent along with it, start to finish, and the model reads through all of it again before answering.
That statelessness is a fact about the underlying model, not about the systems built around it. In the plain chat interface, with nothing else in play, that full-history-every-turn behavior is exactly what happens, and it’s what we were seeing. It’s not the whole story once you move to the API, though. Prompt caching, compaction, and context editing all exist specifically to change the cost and processing characteristics of that resend, without changing the underlying fact: the model itself has no memory. The system around it doesn’t have to behave like it does.
We had a few threads on the team that had turned into sprawling, weeks-long conversations: bug fixes, unrelated writing tasks, random questions, all in the same thread because it was easy. Every one of those was quietly getting more expensive to use, not because the questions were harder, but because the pile behind them kept growing.
Why this actually gets expensive
“Everything gets resent” undersells how fast it compounds. If a conversation starts with a 500-token system prompt and each turn adds roughly 400 tokens, then by turn 30 you’re sending about 12,000 tokens to answer a single new question. Over the life of that thread you’ve cumulatively processed something like a quarter million tokens of history, none of which got any more relevant as it piled up. Files behave the same way. An onboarding doc uploaded early in a thread gets resent on every message for the rest of the conversation, long after it stopped being relevant.
THE COMPOUNDING REALITY: Every message in a growing thread pays for all the messages before it. A thread that felt cheap to start becomes progressively more expensive to use, silently, without any single turn feeling like the cause. Files uploaded early and forgotten are the quietest version of the same problem.
What Actually Helped, Day to Day
Mostly we just stopped doing the things that were quietly expensive. These habits are a reasonable floor for day-to-day use. Once you’re building rather than just using, the same problem shows up structurally, and it needs more than good habits.
Start a new conversation when the topic shifts
Our rough rule: 15 to 20 exchanges. After that, anything you carry forward should be a deliberate choice, not a default.
Summarize instead of dragging a full thread
A well-written summary costs far less to resend than the original thread and usually contains everything the next task actually needs.
Upload only what’s needed, not the whole document
A file uploaded early in a thread gets resent on every subsequent message. If the whole document isn’t relevant to the current question, it shouldn’t be there.
Match the model to the task
A lighter model for simple work, a stronger one reserved for heavy reasoning. More on this in the three-levers section below.
Be specific about length and format
Vague prompts tend to produce longer answers on both ends. Specifying “two sentences” or “a numbered list” costs very little and saves a meaningful amount of output. In Claude Code: /clear and /compact cover the same ground.
The Real Problem: Context Engineering, Not Token Counting
“The instinct to use fewer tokens is a reasonable start but the wrong target.”
What matters is cost per successful outcome: cost per resolved ticket, per correct extraction, per completed task, not cost per token. Trim the wrong piece of context to save tokens and you can easily increase retries or push errors downstream to a human. A cheaper turn that fails and gets redone is worse than a pricier one that succeeds the first time.
Context engineering is the discipline of deciding, deliberately, what belongs in the model’s context at each point in a task: what to include, compress, or drop, so the model has what it needs and nothing more.
Input tokens are the context you send: history, documents, tool schemas. Trimming input is nearly free once the infrastructure exists.
Output tokens are what the model generates, controlled mostly through prompting for length and format. Trimming output changes what gets delivered.
Model selection is which model handles a given task. Switching models changes what the system can do at all. This is a capability decision, not just a cost one.
WHY SEPARATING THESE MATTERS: Conflating input tokens, output tokens, and model selection hides the trade-offs. Trimming input is nearly free once the infrastructure exists. Trimming output changes what gets delivered. Switching models changes what the system can do. These are not the same intervention.
Context Management on the API: Four Tools That Work Together
Prompt caching reuses stable content, like system instructions or tool definitions, instead of reprocessing it on every call. It’s usually the largest and cheapest win available.
Compaction automatically summarizes older history as you near the context limit, so a long conversation or agent loop can keep going without hand-rolled summarization.
Context editing gives more surgical control for agents specifically: clearing stale tool results or old thinking blocks once they’ve served their purpose, when you know precisely what’s now irrelevant.
Token counting lets you check a prompt’s real size before sending it, turning context size from a bill surprise into something measured at build time.
COMBINED APPROACH: Cache what’s stable behind a breakpoint, use context editing to prune what’s gone stale, let compaction catch the rest as the window fills, and use token counting to know where those thresholds actually sit for your workload. These four tools are designed to work together, not to be applied as a checklist.
The Agentic Angle: Where This Gets More Pressing
This gets more pressing the moment you move from a conversation to an agent. A chat thread mostly accumulates dialogue. An agent’s context window fills with tool schemas sent on every call whether they’re used or not, tool outputs that are often large and uncurated, retrieved documents, intermediate reasoning state, and, in multi-agent setups, sub-agent results folded back into a parent’s context. Each of these adds volume independently, faster than a conversation ever does. A single large tool response can dwarf ten turns of chat.
This is exactly what caching, context editing, and compaction were built for. It’s also where the cost-per-outcome framing matters most, because the temptation to prune tool outputs aggressively is highest right where the model most needs that information to succeed.
AGENTIC COST WARNING: Deciding deliberately what each step of an agent’s loop actually needs to see is the difference between costs that stay flat and costs that balloon quietly in the background. The same compounding math that makes a 30-turn chat expensive applies to every agent loop, and agent loops typically run faster and deeper than any manual conversation.
What Changed for Us: An Honest Comparison
We haven’t run a rigorous before/after audit on this specific set of changes, so rather than dress up a guess as measured data, here’s the shape of the comparison worth making, with illustrative numbers standing in for real ones.
| Metric | Before | After |
|---|---|---|
| Avg. input tokens per turn | ~6,000 (long-lived, unscoped threads) | ~1,200 (task-scoped plus caching) |
| Monthly Claude spend | Baseline | Roughly 30 to 40 percent lower |
| Cost per resolved task | Baseline | Flat or lower, tracked separately to confirm spend was not shifted into retries |
| Before | |
|---|---|
| Avg. input tokens per turn | ~6,000 (long-lived, unscoped threads) |
| Monthly Claude spend | Baseline |
| Cost per resolved task | Baseline |
| After | |
|---|---|
| Avg. input tokens per turn | ~1,200 (task-scoped plus caching) |
| Monthly Claude spend | Roughly 30 to 40 percent lower |
| Cost per resolved task | Flat or lower, tracked separately to confirm spend was not shifted into retries |
“A drop in token spend is easy to produce and easy to get wrong. The real test is whether tasks still get done correctly on the first pass.”
That third row is the one that actually matters. If tasks don’t still get done correctly on the first pass, you haven’t cut cost, you’ve just moved it downstream into rework.
The Piece Worth Building Out
Nothing about this required cutting back on what we were doing with Claude. We just stopped treating context as something that accumulates for free and started treating it as something to engineer deliberately: cached where it’s stable, trimmed where it’s stale, routed to the right model, and measured against outcomes rather than token counts alone.
That last part, measuring against outcomes rather than token counts alone, is the piece we’re still building out, and it’s worth investing in before the next round of numbers, so they’re real.
At Trantor, we build production AI systems and agent pipelines where context engineering is a first-class concern from the start, not a cost-reduction initiative applied after the fact. If your Claude or LLM spend is higher than expected and you’re not sure whether that cost is producing proportional outcomes, we can help you find out and build the infrastructure to manage it deliberately.
Explore Trantor’s AI Engineering Services: Artificial Intelligence
