AI tools: cost-control patterns for LLM features

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-tools-cost-control-patterns-r7a1-creators

1. A developer implements response caching for an AI feature. What determines whether a newly received user query can use a cached response?

   1. Whether the server has available GPU memory
   2. Whether the query contains any numbers
   3. Whether the query's input hash matches a previously stored hash
   4. Whether the user has premium status

2. A team builds a caching system that uses only the prompt text as the cache key. What critical security/privacy problem emerges when this cache returns results to different users?

   1. The system will consume more GPU memory over time
   2. The cache will reject queries containing special characters
   3. Cache entries will expire after 24 hours automatically
   4. One user's personalized response may be returned to a different user with the same query

3. In a tiered model architecture, what determines whether a user's conversation turn is routed to a small model versus a large model?

   1. The time of day the request is made
   2. The complexity and difficulty of the specific conversation turn
   3. The user's subscription tier level
   4. The total number of tokens used so far

4. An AI feature's daily spending suddenly spikes to three times its normal amount. According to best practices for cost control, what should automatically happen?

   1. A global kill switch should trigger and halt further processing
   2. The system should switch to using only premium models
   3. The billing system should automatically refund all charges
   4. User authentication should be disabled to reduce traffic

5. A startup notices their LLM costs increasing unboundedly each month. How should the team primarily view this problem?

   1. As a marketing problem caused by too many free trials
   2. As a billing issue that requires renegotiating vendor contracts
   3. As a product bug requiring engineering work, not just an operations issue
   4. As a security vulnerability that needs firewall rules

6. Which of the following tasks can an AI model autonomously perform without any additional infrastructure or external systems?

   1. None of these — AI cannot independently cache responses, route between models, or throttle users
   2. The AI can decide when to use a smaller model based on its own judgment
   3. The AI can store its responses in a database for future use
   4. The AI can automatically detect and block abusive users

7. A developer implements per-user daily token caps with 'friendly degradation.' What does this phrase most likely describe?

   1. The system automatically reduces prices for users who spend less
   2. The AI becomes more concise in its responses as limits approach
   3. Premium users get faster responses while free users wait longer
   4. When a user hits their limit, they receive a polite message and reduced functionality rather than a harsh error

8. What is the primary purpose of implementing hard caps on a per-user basis for AI feature usage?

   1. To ensure all users receive equal response times
   2. To prevent any single user from causing unlimited cost exposure
   3. To comply with government data retention regulations
   4. To encourage users to upgrade to premium tiers

9. A team wants to reduce LLM costs for their AI feature. Which approach directly addresses the 'cost spiral' problem described in cost-control patterns?

   1. Switching to a different cloud hosting provider
   2. Implementing caching, tiered models, and per-user caps together
   3. Running marketing campaigns to acquire more users
   4. Hiring more customer support staff

10. The 30-day average spending metric is used as a baseline for which cost-control mechanism?

    1. Triggering alerts and kill switches when spending exceeds the threshold
    2. Setting the default token limit for new accounts
    3. Calculating refund amounts for unsatisfied customers
    4. Determining which users qualify for premium features

11. What information must be included in a cache key to ensure user privacy while still benefiting from response caching?

    1. User-specific identifiers like account ID or locale plus the prompt text
    2. Only a timestamp of when the request was made
    3. The user's email address in plain text
    4. A random UUID generated for each request

12. In the context of LLM cost control, what does the term 'cache hit' refer to?

    1. When the AI model successfully generates a response on the first attempt
    2. When a newly received query matches a previously cached query and returns the stored response
    3. When a user experiences slow response times due to server load
    4. When a user clicks on a cached link in a web interface

13. Who or what is responsible for deciding whether an AI model should cache its own responses?

    1. The AI model itself, through self-improvement
    2. The cloud provider, based on available resources
    3. The engineering team — caching requires infrastructure they must build
    4. The end users, through their subscription choices

14. A user is sending many requests that appear intentionally abusive, consuming excessive resources. According to cost-control best practices, who should handle throttling this user?

    1. The marketing team should reach out to the user
    2. External access control systems, not the AI model itself
    3. The AI model should naturally ignore abusive requests
    4. The billing system should automatically pause the account

15. Why is monitoring the 30-day average spending important for AI feature operations?

    1. It provides a baseline to detect abnormal spending spikes that trigger alerts
    2. It decides which features to enable for free trial users
    3. It determines how much to bill customers each month
    4. It calculates how many active users the system can support