Debugging Cost and Rate Limits in AI Coding

When the Bug Is the Bill

You ask Claude Code to do something simple. Forty minutes later it's still going, you're hitting rate limits, and your monthly budget evaporated. This is not a code bug. It's a cost bug. They share a debugging discipline.

The 2026 cost landscape (rough orders of magnitude)

The five expensive habits to spot in your own workflow

1. Loading entire codebases into context every session — dramatically more expensive than indexed search
2. Running the heaviest model for trivial tasks (Opus to fix a typo)
3. Long sessions without compaction — every turn re-bills the whole history
4. 10 subagents in parallel for a 30-minute task — 10x cost for no real win
5. Letting agents loop on failing tests for an hour without intervention

Prompt caching is the single biggest cost lever

# Without caching: # - Every turn re-bills the full system prompt + project context # - 200k tokens * 30 turns = 6M input tokens billed # With caching (Anthropic, OpenAI, AI Gateway): # - System prompt + project context cached after first turn # - Subsequent turns pay 10% of cached portion + full new tokens # - Same workload: ~1M billed tokens vs 6M # Claude Code uses caching automatically. # Custom apps via the API: set cache_control: ephemeral on stable system blocks. from anthropic import Anthropic client = Anthropic() response = client.messages.create( model="claude-sonnet-4-6", system=[ {"type": "text", "text": LONG_STABLE_INSTRUCTIONS, "cache_control": {"type": "ephemeral"}}, ], messages=[{"role": "user", "content": ""}], )Caching the stable parts of your prompt is the single most cost-effective change in any LLM application.

Model routing: not every task needs the flagship

• Boilerplate, formatting, comments — Haiku, GPT-5 Mini, Gemini Flash
• Code generation in known patterns — Sonnet, GPT-5, Gemini 2.5 Pro
• Architectural decisions, novel algorithms — Opus, GPT-5.5
• Debugging hard bugs — Opus or Sonnet with Extended Thinking

Diagnosing rate-limit failures

The cost-aware coding loop

# Daily checklist (30 seconds): 1. Check yesterday's spend — anomalies? 2. Confirm no zombie cloud agents are running 3. Review your model defaults — should this project use Sonnet or Haiku? # Per-session checklist (10 seconds): 1. /compact when the session crosses ~50k tokens 2. /clear when the session is done — don't keep stale context for tomorrow 3. Spawn subagents only when the work is truly parallel # Per-prompt checklist (instant): 1. Did I include unnecessary context (paste of files I don't need)? 2. Could a smaller model handle this?Cost discipline is a habit, not a tool. Practice these checks until they're automatic.

When you're hitting limits hard

1. Switch to a flat-rate plan if your usage is steady (Cursor, Windsurf, Copilot)
2. Use API access through AI Gateway with caching enabled — most cost-efficient at scale
3. Move heavy explore-and-summarize work to a cheaper model with long context (Gemini)
4. Reserve flagship models for the actual hard reasoning steps
5. If you genuinely need more capacity, the cost of upgrade is usually less than the cost of throttled engineering time

If you can't measure it, you can't optimize it. Watch the meter.

— A finops engineer turned AI specialist

The big idea: cost and rate limits are part of the AI coding craft. Cache stable prompts, route by task complexity, compact long sessions, and audit background agents. The engineers who stay productive at scale are the ones who treat tokens as a real resource, not an unlimited well.