Prompt Caching and Cost Optimization

Long system prompts are expensive. Prompt caching lets you reuse the prefix at up to 90% cost reduction and much lower latency. Here's how to architect prompts for caching.

34 min · Reviewed 2026

Why caching exists

In production AI apps, the same long preamble — system prompt, few-shot examples, long retrieved documents — is often sent on every request while only a short user message changes. Processing that preamble every time is wasteful. Prompt caching lets the provider store a processed version of the prefix and reuse it.

Prompt structure for caching

// Anthropic Messages API
{
  "model": "claude-sonnet-4-5",
  "system": [
    {
      "type": "text",
      "text": "<long system prompt, role, rules, policies...>",
      "cache_control": {"type": "ephemeral"}
    },
    {
      "type": "text",
      "text": "<knowledge base, 50KB of docs>",
      "cache_control": {"type": "ephemeral"}
    }
  ],
  "messages": [
    {"role": "user", "content": "What's our refund policy for digital goods?"}
  ]
}Marking two prefix chunks as cacheable. Only the final user message varies per request.

What to cache

System prompts longer than ~1K tokens.
Few-shot example blocks that don't change between requests.
Retrieved documents that are reused across a session.
Tool definitions and schemas.
Large style guides, glossaries, or brand-voice references.

What NOT to cache

The user's per-request message (defeats the purpose).
Content under ~1K tokens (overhead > benefit).
Personalized data that changes every request.
Anything behind the cache's expiration window if requests are sparse.

Cache hierarchy

Caching works best when prefixes are layered in order from most-stable to least-stable. Put static rules first, then slowly-changing context (user profile, session info), then the per-request message last. That way the maximum possible prefix hits the cache.

[STABLE SYSTEM]       <- cache forever (or until product change)
[KNOWLEDGE BASE]      <- cache for the day
[USER PROFILE]        <- cache for the session
[CONVERSATION HISTORY]<- cache up to last turn
[CURRENT USER MSG]    <- never cacheLayered cache strategy — deeper layers reused more often.

Measuring impact

Baseline: log tokens-in / tokens-out / latency / cost per request for a representative day.
Add cache_control markers.
Re-measure. Compare per-request cost and P50/P95 latency.
Track cache hit rate (providers return it in responses). Aim for >80%.
Revisit when the preamble changes — cache misses spike after a prompt update.

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-prompting-caching-creators

What is the core idea behind "Prompt Caching and Cost Optimization"?
1. Long system prompts are expensive. Prompt caching lets you reuse the prefix at up to 90% cost reduction and much lower latency. Here's how to architect prompts for caching.
2. 'Pretend you are a soccer coach giving me a pep talk before the big game.'
3. Attending well to information at the start and end of context
4. vocabulary
Which term best describes a foundational idea in "Prompt Caching and Cost Optimization"?
1. cache_control
2. prompt caching
3. cache hit rate
4. prefix reuse
A learner studying Prompt Caching and Cost Optimization would need to understand which concept?
1. prompt caching
2. cache hit rate
3. cache_control
4. prefix reuse
Which of these is directly relevant to Prompt Caching and Cost Optimization?
1. prompt caching
2. cache_control
3. prefix reuse
4. cache hit rate
Which of the following is a key point about Prompt Caching and Cost Optimization?
1. System prompts longer than ~1K tokens.
2. Few-shot example blocks that don't change between requests.
3. Retrieved documents that are reused across a session.
4. Tool definitions and schemas.
Which of these does NOT belong in a discussion of Prompt Caching and Cost Optimization?
1. Few-shot example blocks that don't change between requests.
2. 'Pretend you are a soccer coach giving me a pep talk before the big game.'
3. System prompts longer than ~1K tokens.
4. Retrieved documents that are reused across a session.
Which statement is accurate regarding Prompt Caching and Cost Optimization?
1. Content under ~1K tokens (overhead > benefit).
2. Personalized data that changes every request.
3. The user's per-request message (defeats the purpose).
4. Anything behind the cache's expiration window if requests are sparse.
Which of these does NOT belong in a discussion of Prompt Caching and Cost Optimization?
1. The user's per-request message (defeats the purpose).
2. Personalized data that changes every request.
3. 'Pretend you are a soccer coach giving me a pep talk before the big game.'
4. Content under ~1K tokens (overhead > benefit).
What is the key insight about "The numbers (Anthropic, 2026)" in the context of Prompt Caching and Cost Optimization?
1. Cache writes cost ~25% more than normal input tokens. Cache reads cost ~10% of normal input tokens.
2. 'Pretend you are a soccer coach giving me a pep talk before the big game.'
3. Attending well to information at the start and end of context
4. vocabulary
What is the key insight about "Caches have TTLs" in the context of Prompt Caching and Cost Optimization?
1. 'Pretend you are a soccer coach giving me a pep talk before the big game.'
2. Anthropic's default ephemeral cache is ~5 minutes. If your traffic is bursty, you may not see hits.
3. Attending well to information at the start and end of context
4. vocabulary
What is the recommended tip about "Practitioner tip" in the context of Prompt Caching and Cost Optimization?
1. 'Pretend you are a soccer coach giving me a pep talk before the big game.'
2. Attending well to information at the start and end of context
3. Treat every prompt as a spec: role → context → task → format. Review your first output as a draft, not a final.
4. vocabulary
Which statement accurately describes an aspect of Prompt Caching and Cost Optimization?
1. 'Pretend you are a soccer coach giving me a pep talk before the big game.'
2. Attending well to information at the start and end of context
3. vocabulary
4. In production AI apps, the same long preamble — system prompt, few-shot examples, long retrieved documents — is often sent on every request …
What does working with Prompt Caching and Cost Optimization typically involve?
1. Caching works best when prefixes are layered in order from most-stable to least-stable.
2. 'Pretend you are a soccer coach giving me a pep talk before the big game.'
3. Attending well to information at the start and end of context
4. vocabulary
Which best describes the scope of "Prompt Caching and Cost Optimization"?
1. It is unrelated to prompting workflows
2. It focuses on Long system prompts are expensive. Prompt caching lets you reuse the prefix at up to 90% cost reduct
3. It applies only to the opposite beginner tier
4. It was deprecated in 2024 and no longer relevant
Which section heading best belongs in a lesson about Prompt Caching and Cost Optimization?
1. 'Pretend you are a soccer coach giving me a pep talk before the big game.'
2. Attending well to information at the start and end of context
3. Prompt structure for caching
4. vocabulary

← Back to interactive lesson

Tendril · Creators · Prompting

Prompt Caching and Cost Optimization

Long system prompts are expensive. Prompt caching lets you reuse the prefix at up to 90% cost reduction and much lower latency. Here's how to architect prompts for caching.

34 min · Reviewed 2026

Why caching exists

Prompt structure for caching

// Anthropic Messages API
{
  "model": "claude-sonnet-4-5",
  "system": [
    {
      "type": "text",
      "text": "<long system prompt, role, rules, policies...>",
      "cache_control": {"type": "ephemeral"}
    },
    {
      "type": "text",
      "text": "<knowledge base, 50KB of docs>",
      "cache_control": {"type": "ephemeral"}
    }
  ],
  "messages": [
    {"role": "user", "content": "What's our refund policy for digital goods?"}
  ]
}Marking two prefix chunks as cacheable. Only the final user message varies per request.

What to cache

System prompts longer than ~1K tokens.
Few-shot example blocks that don't change between requests.
Retrieved documents that are reused across a session.
Tool definitions and schemas.
Large style guides, glossaries, or brand-voice references.

What NOT to cache

The user's per-request message (defeats the purpose).
Content under ~1K tokens (overhead > benefit).
Personalized data that changes every request.
Anything behind the cache's expiration window if requests are sparse.

Cache hierarchy

[STABLE SYSTEM]       <- cache forever (or until product change)
[KNOWLEDGE BASE]      <- cache for the day
[USER PROFILE]        <- cache for the session
[CONVERSATION HISTORY]<- cache up to last turn
[CURRENT USER MSG]    <- never cacheLayered cache strategy — deeper layers reused more often.

Measuring impact

Baseline: log tokens-in / tokens-out / latency / cost per request for a representative day.
Add cache_control markers.
Re-measure. Compare per-request cost and P50/P95 latency.
Track cache hit rate (providers return it in responses). Aim for >80%.
Revisit when the preamble changes — cache misses spike after a prompt update.

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-prompting-caching-creators

What is the core idea behind "Prompt Caching and Cost Optimization"?
1. Long system prompts are expensive. Prompt caching lets you reuse the prefix at up to 90% cost reduction and much lower latency. Here's how to architect prompts for caching.
2. 'Pretend you are a soccer coach giving me a pep talk before the big game.'
3. Attending well to information at the start and end of context
4. vocabulary
Which term best describes a foundational idea in "Prompt Caching and Cost Optimization"?
1. cache_control
2. prompt caching
3. cache hit rate
4. prefix reuse
A learner studying Prompt Caching and Cost Optimization would need to understand which concept?
1. prompt caching
2. cache hit rate
3. cache_control
4. prefix reuse
Which of these is directly relevant to Prompt Caching and Cost Optimization?
1. prompt caching
2. cache_control
3. prefix reuse
4. cache hit rate
Which of the following is a key point about Prompt Caching and Cost Optimization?
1. System prompts longer than ~1K tokens.
2. Few-shot example blocks that don't change between requests.
3. Retrieved documents that are reused across a session.
4. Tool definitions and schemas.
Which of these does NOT belong in a discussion of Prompt Caching and Cost Optimization?
1. Few-shot example blocks that don't change between requests.
2. 'Pretend you are a soccer coach giving me a pep talk before the big game.'
3. System prompts longer than ~1K tokens.
4. Retrieved documents that are reused across a session.
Which statement is accurate regarding Prompt Caching and Cost Optimization?
1. Content under ~1K tokens (overhead > benefit).
2. Personalized data that changes every request.
3. The user's per-request message (defeats the purpose).
4. Anything behind the cache's expiration window if requests are sparse.
Which of these does NOT belong in a discussion of Prompt Caching and Cost Optimization?
1. The user's per-request message (defeats the purpose).
2. Personalized data that changes every request.
3. 'Pretend you are a soccer coach giving me a pep talk before the big game.'
4. Content under ~1K tokens (overhead > benefit).
What is the key insight about "The numbers (Anthropic, 2026)" in the context of Prompt Caching and Cost Optimization?
1. Cache writes cost ~25% more than normal input tokens. Cache reads cost ~10% of normal input tokens.
2. 'Pretend you are a soccer coach giving me a pep talk before the big game.'
3. Attending well to information at the start and end of context
4. vocabulary
What is the key insight about "Caches have TTLs" in the context of Prompt Caching and Cost Optimization?
1. 'Pretend you are a soccer coach giving me a pep talk before the big game.'
2. Anthropic's default ephemeral cache is ~5 minutes. If your traffic is bursty, you may not see hits.
3. Attending well to information at the start and end of context
4. vocabulary
What is the recommended tip about "Practitioner tip" in the context of Prompt Caching and Cost Optimization?
1. 'Pretend you are a soccer coach giving me a pep talk before the big game.'
2. Attending well to information at the start and end of context
3. Treat every prompt as a spec: role → context → task → format. Review your first output as a draft, not a final.
4. vocabulary
Which statement accurately describes an aspect of Prompt Caching and Cost Optimization?
1. 'Pretend you are a soccer coach giving me a pep talk before the big game.'
2. Attending well to information at the start and end of context
3. vocabulary
4. In production AI apps, the same long preamble — system prompt, few-shot examples, long retrieved documents — is often sent on every request …
What does working with Prompt Caching and Cost Optimization typically involve?
1. Caching works best when prefixes are layered in order from most-stable to least-stable.
2. 'Pretend you are a soccer coach giving me a pep talk before the big game.'
3. Attending well to information at the start and end of context
4. vocabulary
Which best describes the scope of "Prompt Caching and Cost Optimization"?
1. It is unrelated to prompting workflows
2. It focuses on Long system prompts are expensive. Prompt caching lets you reuse the prefix at up to 90% cost reduct
3. It applies only to the opposite beginner tier
4. It was deprecated in 2024 and no longer relevant
Which section heading best belongs in a lesson about Prompt Caching and Cost Optimization?
1. 'Pretend you are a soccer coach giving me a pep talk before the big game.'
2. Attending well to information at the start and end of context
3. Prompt structure for caching
4. vocabulary

← Back to interactive lesson