Managing Claude Code costs and context comes down to architecture, not prompting. Use tiered model routing to match task complexity to the right model, implement prompt caching on static system prompts, compress conversation history rather than passing raw transcripts, and set hard spending limits in the Anthropic Console before a runaway loop surprises you at billing time.
The way your app stores state, passes context, and decides when to call the API determines the majority of your costs before you write a single prompt. Most tutorials teach you how to talk to Claude; almost none teach you how to build systems around it. [1] The difference between a prototype that costs $40 in a weekend and a production agent that costs $40 a month is almost entirely structural.
This question came up in the r/ClaudeAI community from developers who'd been using Claude Code for months and loved it — but found token consumption, cost unpredictability, and context degradation felt like a second job. It's the exact inflection point where casual experimenters part ways with practitioners who actually ship production systems.
I've built production AI agents on top of Claude's API — including Buddy, an open-source Google Ads agent — and the lessons here come from real usage, real billing surprises, and real refactoring sessions when something wasn't scaling.
Most developers are surprised to find that the majority of their token spend is in the input (context they're sending), not the completion (output the model returns). The culprit is almost always naive context management.
The "build a chatbot in 10 minutes" pattern is: append every user message and assistant response to a growing array, then send the entire array on every API call. By turn 20 of a conversation averaging 500 tokens per exchange, you're sending roughly 10,000 tokens of history — even if turns 1 through 15 are completely irrelevant to the current question. Multiply that across hundreds of users or automated pipeline runs and you're burning money on context that adds nothing to response quality.
| Claude Model | Input (per 1M tokens) | Output (per 1M tokens) | Context Window |
|---|---|---|---|
| Claude 3.5 Haiku | ~$0.80 | ~$4.00 | 200K tokens |
| Claude 3.5 Sonnet | ~$3.00 | ~$15.00 | 200K tokens |
| Claude Opus 4 | ~$15.00 | ~$75.00 | 200K tokens |
Routing every task through Opus when Haiku would do the job means spending roughly 19x more on input tokens alone. [2] That's the difference between a sustainable product and a money pit.
Build a routing layer that assigns model tiers based on task complexity — even a simple rule-based router saves significant money. You don't need ML to do this; a lookup table of task types to model tiers is a perfectly valid starting point.
In building Buddy, roughly 60–70% of tasks in a typical marketing automation workflow run fine on Haiku without meaningful quality degradation. The remaining 30–40% benefit from Sonnet. True Opus-tier work is rare — around 5% of tasks if pipelines are designed thoughtfully.
If you're interested in how this plays out in a real marketing automation workflow with Claude Code, the routing decisions look different depending on whether you're doing bulk data processing or nuanced campaign analysis.
---The 200K context window is genuinely impressive, but it creates a dangerous mindset: the feeling that you can dump everything in and let the model sort it out. Practitioners who've shipped real production systems will tell you this approach fails in predictable ways. [3]
Large language models tend to weight information at the beginning and end of context more heavily than content buried in the middle — sometimes called the "lost in the middle" problem. [3] If your most critical instructions are buried in the middle of a large context, don't be surprised when they're ignored. The fix isn't a larger context window; it's better information architecture:
Instead of passing raw conversation history, implement a summarization step that compresses older turns into a compact state object:
This pattern typically reduces context token spend by 40–70% in long-running conversations or multi-step agentic pipelines, with minimal quality loss for most use cases.
A concrete example: if you're analyzing 500 ad headlines for quality signals, chunk them into batches of 50 and run parallel Haiku calls. If you're reviewing a full campaign strategy document for internal contradictions, keep it together in a single Sonnet call.
The anxiety about runaway costs isn't irrational — it's a real operational concern, especially for indie developers and small teams. These are the systems that make a measurable difference.
Anthropic's prompt caching feature lets you cache large, static portions of your prompt — system prompts, long reference documents, tool definitions — so they're only charged at full rate once. [4] Subsequent calls using the same cached prefix are charged at roughly a 90% discount on those cached tokens. For apps with large system prompts or reference documentation, prompt caching alone can reduce total costs by 50–80% on repeat calls.
If you're using Claude Code with tool use, every round-trip adds tokens: the tool definition, the model's tool call output, and the tool result you pass back. Bloated tool definitions are a silent cost driver. Keep tool descriptions tight — one precise sentence per tool plus typed parameters is sufficient. Tool definition bloat can add 800–1,200 tokens per call in poorly designed schemas, which compounds fast in agentic loops.
Log every API call with: timestamp, model used, input token count, output token count, task type, and cost estimate. This isn't just for billing — it's your diagnostic tool when something starts misbehaving. In production agents like Buddy, this kind of logging has caught everything from accidental infinite loops to a tool definition bug that was passing a 15K-token document on every call when only a 200-token summary was needed.
---Beyond immediate cost concerns, the deeper question is how to build apps that don't become maintenance nightmares as models change and use cases evolve. [1] A few structural principles make an outsized difference.
Never call the Claude API directly from business logic. Always go through an abstraction layer — a class, a module, a service — that owns the API interaction. This lets you swap models without touching business logic, add logging and retry logic in one place, implement model routing cleanly, and mock the API layer in tests. If you're adopting Claude Code in an enterprise setting, this abstraction layer becomes even more critical for governance and auditability.
System prompts are part of your application logic. They should be version-controlled, tested when changed, and reviewed with the same care as code. [5] Prompt changes that look harmless can have significant behavioral effects — if you're not versioning them, you have no way to roll back when something breaks.
What happens when the API is down, rate limited, or returns an unexpected response? Your app should have answers baked in: retry with exponential backoff, fallback to cached responses where appropriate, and surface failures gracefully to users rather than crashing silently. For teams building more complex workflows, dynamic workflow patterns in Claude Code offer one approach to making pipelines resilient by design.
---The practitioners who get the most out of Claude Code long-term aren't the ones who prompt best — they're the ones who build the tightest systems around it. If you're looking for a reference workflow, the best Claude Code workflow patterns cover how to structure agentic pipelines from the ground up.
---In a typical production pipeline, roughly 60–70% of tasks are suitable for Haiku (classification, extraction, simple Q&A), 25–35% warrant Sonnet (code generation, multi-step reasoning, document analysis), and only around 5% genuinely need Opus (complex synthesis, high-stakes judgment calls where quality variance is costly). Start with this as a default and adjust based on your logged quality outcomes.
Implement prompt caching on your system prompt first — it requires minimal code changes and can cut repeat-call costs by 50–80% if you have a large static system prompt. Second, set a hard spending limit in the Anthropic Console so any runaway behavior gets capped automatically. These two changes can be done in a few hours and deliver results immediately.
Yes, in practice. Large language models tend to weight content at the beginning and end of context more heavily than content in the middle — often called the "lost in the middle" problem. Critical instructions buried deep in a large context are more likely to be missed or underweighted. Design your app so most calls stay well under the maximum; use retrieval to pull in only what's relevant rather than stuffing entire documents.
Start simple: a lookup table that maps task types to model tiers is perfectly adequate and takes an hour to implement. You only need a learned or ML-based router if your task distribution is highly variable and you've already exhausted the gains from a rule-based approach. Most teams never need to go beyond a well-maintained rule-based router.
Keep each tool description to one precise sentence, use strict typing (enums, specific formats) for parameters, and audit your definitions regularly. Verbose tool schemas can add 800–1,200 tokens per call, which compounds quickly in agentic loops. Strict typing also reduces the model's need to ask clarifying questions, cutting round-trips and total token spend simultaneously.
Absolutely. System prompts are application logic — a change that looks minor can meaningfully alter model behavior in production. Store prompts in version control alongside your code, review changes the same way you'd review a code PR, and maintain the ability to roll back quickly. Teams that treat prompts as informal notes rather than code consistently run into hard-to-diagnose regressions.
More answers in this topic: Claude & Anthropic