AI Cost Engineering: Where the Money Actually Goes

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-ai-foundations-cost-engineering-final1-creators

1. A developer notices their AI feature costs $500/month. They implement a system where simple questions go to a cheap model and complex questions go to an expensive model. What is this technique called?

   1. Token pooling
   2. Prompt chaining
   3. Model routing
   4. Load balancing

2. Which of the following is TRUE about AI token billing?

   1. You only pay for output tokens, not input tokens
   2. You pay a flat monthly fee regardless of usage
   3. Input tokens are free; output tokens cost money
   4. Every token generated by the model is billed, regardless of whether the user sees it

3. A team implements caching for their AI feature but finds costs have NOT decreased significantly. What is the MOST likely reason?

   1. The cache is too small
   2. The cache is stored on a slow hard drive
   3. Most user requests are unique and rarely repeat
   4. The AI model is too fast

4. A developer compresses their system prompt from 500 words to 300 words, removing filler language but keeping all rules. The AI still follows the rules correctly. What is the primary benefit of this approach?

   1. The AI becomes more creative
   2. The model runs faster on older hardware
   3. More users can access the feature simultaneously
   4. Input token costs decrease

5. A startup chooses the cheapest AI model for their customer service chatbot. After one month, costs are HIGHER than expected despite the low per-token price. What probably happened?

   1. The cheap model generated longer responses
   2. The cheap model couldn't handle the request volume
   3. The cheap model was down for maintenance
   4. The cheap model required more retries to get acceptable answers

6. What does 'streaming' refer to in AI cost optimization?

   1. Running multiple AI models simultaneously
   2. Splitting a large prompt across multiple API calls
   3. Generating and displaying tokens incrementally as they are created
   4. Sending requests in small packets over the internet

7. A developer adds 10 example conversations to their AI prompt to show the model the desired format. These examples are called:

   1. Context windows
   2. System directives
   3. Prompt templates
   4. Few-shot examples

8. What is a key limitation of caching in AI systems?

   1. Caches cannot store text responses
   2. Caches make responses slower
   3. Caches can become stale and serve outdated results
   4. Caches increase API latency

9. A team wants to compress their prompt by 40% but are unsure if quality will suffer. What should they do?

   1. Test the compressed version against an evaluation set of known good outputs
   2. Remove all examples from the prompt
   3. Use a more expensive model to compensate
   4. Guess and hope it works

10. What does 'early-stopping' mean in AI cost engineering?

    1. Stopping the API call before sending the full prompt
    2. Cancelling requests after 1 second
    3. Using the cheapest model available
    4. Terminating generation once sufficient output has been received

11. Why might caching identical requests be more effective for a FAQ bot than for a creative writing assistant?

    1. FAQ bots use cheaper models
    2. FAQ requests are more likely to repeat exactly
    3. Creative writing requires more tokens
    4. FAQ bots don't use AI

12. What is 'cost engineering' in the context of AI systems?

    1. The practice of systematically reducing AI costs while maintaining quality
    2. Designing AI that costs less to train
    3. Calculating the price of AI hardware
    4. Building AI models that generate cheap content

13. A team tracks their AI cost per API call but finds it's NOT a useful metric. Why might per-call cost be misleading?

    1. Per-call cost is always accurate
    2. A cheap model might require multiple calls to get a good result
    3. They should track cost per user instead
    4. API calls are free

14. What happens to a cache as it grows larger over time?

    1. It becomes faster automatically
    2. It consumes more memory and may store outdated responses
    3. It automatically compresses itself
    4. It deletes old responses

15. Which of the following is NOT something AI cost engineering can achieve?

    1. Compress prompts without losing meaning
    2. Reduce costs by routing easy queries to cheaper models
    3. Cache repeated or similar requests
    4. Eliminate the need to pay for generated tokens