Using Prompt Caching to Cut Cost and Latency

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-tools-prompt-cache-r12a1-creators

1. A developer notices their AI application costs have doubled even though the total number of user requests stayed the same. What is the most likely reason?

   1. The system prompt was shortened to reduce token count
   2. The cache TTL is shorter than the average gap between user requests
   3. The AI provider increased their per-token pricing
   4. The application switched to a different model

2. In a prompt structured for caching, where should system instructions and few-shot examples be placed?

   1. Only in the API system message field, not the user message
   2. Mixed throughout the prompt in multiple sections
   3. At the beginning, before any variable content
   4. At the very end, after user input

3. A team wants to cache their prompt but the content includes user-specific data that changes with every request. What should they do?

   1. Separate static instructions into a cacheable prefix, keep user data in the variable section
   2. Cache the entire prompt including user data for faster processing
   3. Request a longer TTL from the provider to accommodate the changes
   4. Store user data in a database and reference it via ID in the prompt

4. What happens to latency when a cached prefix is reused on a subsequent API call?

   1. Latency becomes zero for all cached requests
   2. Latency increases because the cache must be looked up
   3. Latency stays the same regardless of cache status
   4. Latency decreases because prefix tokens don't need reprocessing

5. Which metric provides the clearest signal that prompt caching is actually working?

   1. Average response time per request
   2. Cache hit rate percentage
   3. Total monthly API costs
   4. Number of requests per day

6. A developer sets their cache TTL to 60 minutes but notices many cache misses during what should be active usage periods. What might explain this?

   1. The model is too fast to need caching
   2. The provider has a maximum TTL of 5 minutes
   3. The static content was placed after the variable content
   4. User requests are coming in bursts with gaps shorter than the TTL

7. What is required to enable prompt caching for a provider that supports it?

   1. Paying for a premium API tier
   2. Marking the static content block as cacheable in the API call
   3. Upgrading to a larger model
   4. Installing additional caching software

8. Why is monitoring total cost alone insufficient for evaluating prompt caching?

   1. Costs are calculated differently for cached versus uncached requests
   2. Costs fluctuate due to model version changes unrelated to caching
   3. Costs don't distinguish between cache hits and misses—they only show the final bill
   4. The provider randomly adds fees that don't correlate with caching

9. What limitation exists on cache duration that cannot be overridden by developers?

   1. The provider sets a maximum TTL that cannot be exceeded
   2. Developers can set any TTL they want up to 1 year
   3. Cache duration is unlimited for paid accounts
   4. Cache duration depends on the model size being used

10. What does the term "prefix" refer to in prompt caching?

    1. The first word of every user message
    2. The system message field in the API request
    3. The static beginning portion of a prompt that gets cached
    4. The variable content that changes between calls

11. If a developer wants to maximize cost savings from prompt caching, what should they avoid including in the cacheable prefix?

    1. Company branding guidelines
    2. Few-shot examples that demonstrate the desired output format
    3. User-specific data that changes per request
    4. System instructions that apply to all interactions

12. What occurs when the cache TTL expires between two requests that are otherwise identical?

    1. The request is rejected as invalid
    2. The provider automatically extends the TTL for repeated content
    3. The cache is renewed automatically without any cost
    4. The full prompt is reprocessed and charged at full price

13. A developer tests two prompts: one with caching enabled and one without, using identical system instructions and user input. What should they observe?

    1. The cached version should have lower cost and latency on repeated calls
    2. The uncached version should be faster because it doesn't check cache
    3. There should be no difference in behavior between the two
    4. Both prompts should cost exactly the same

14. What happens to uncached content in a prompt that contains both static and variable sections?

    1. It causes the entire prompt to be rejected
    2. It is processed normally and charged at the full rate
    3. It is automatically added to the cache for next time
    4. It is ignored by the API entirely

15. What is the relationship between prompt caching and API pricing models?

    1. Cached requests are free regardless of content
    2. Caching eliminates all per-token charges
    3. Cached prefix tokens are charged once, variable tokens are charged per call
    4. Caching changes the model from per-token to per-request pricing