What It Actually Costs to Run a Big AI Model

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-builders-foundations-ai-cost-of-running-models-r9a10-teen

1. What does the term 'inference' refer to in AI systems?

   1. The process of designing the architecture of a neural network
   2. The method of measuring how accurately a model performs on test data
   3. The process of collecting and organizing training data before building a model
   4. The process of running a trained model to generate outputs for users

2. A company adds AI writing assistance to their free tier product. Based on the trends in the lesson, what is the most likely reason they can afford to do this?

   1. Government regulations now require AI features to be free
   2. Companies have found a way to make users pay for the electricity instead
   3. Free users don't actually trigger AI responses — they get pre-written answers
   4. The cost of running AI queries has dropped by approximately 99% since 2022

3. What is 'quantization' in the context of AI model optimization?

   1. A method for converting text data into numerical format the model can process
   2. A system for measuring how much compute a model uses
   3. A technique that reduces the precision of numbers used in the model to make it smaller and faster
   4. A process that increases the amount of training data to improve accuracy

4. Which combination of factors contributed to the 99% drop in AI query costs from 2022 to 2026?

   1. Fewer users and reduced demand for AI services
   2. Government subsidies and tax breaks for AI companies
   3. Lower electricity prices and reduced data center rents
   4. Better hardware, smaller specialized models, and engineering optimizations like quantization

5. The lesson mentions DeepSeek V3 trained for under $6 million in late 2024. What was a key reason they achieved this relatively low cost?

   1. They received government funding to offset the costs
   2. They used more GPUs than typical for that budget
   3. They used cleverer methods on cheaper hardware
   4. They only trained a very small model with minimal capabilities

6. If an H100 GPU rents for $3 per hour on a cloud service, approximately how much would it cost to run 100 such GPUs continuously for one full day?

   1. About $300
   2. About $72
   3. About $7,200
   4. About $3,000

7. What does the term 'scaling' typically refer to in AI development?

   1. Limiting the number of users who can access the AI
   2. Making the text font larger in the user interface
   3. Reducing the number of features in a product
   4. Increasing the size of models, data, and compute to improve capabilities

8. In the AI industry, what does 'compute' refer to?

   1. The amount of storage space used to save model files
   2. The physical computer a user types on to interact with AI
   3. The mathematical calculations performed by GPUs to run AI models
   4. The network bandwidth required to send prompts to AI servers

9. Why does the lesson advise builders to track per-call AI costs early in a project?

   1. To impress investors with detailed expense reports
   2. Because costs drop quickly — in 6 months the same capability costs about one-quarter of current prices
   3. Because AI companies will charge more if you don't track usage
   4. To determine which employees should have access to the AI

10. What is a GPU and why is it important for AI?

    1. A Global Positioning Unit; it helps AI models understand location context
    2. A Ground Processing Unit; it processes data sent from satellites to AI systems
    3. A Graphics Processing Unit; it can perform many simple calculations simultaneously, making it ideal for AI work
    4. A General Processing Unit; it controls all computer functions including AI

11. The lesson states that DeepSeek V3 achieved lower training costs than typical frontier models. What does this suggest about the future of AI development?

    1. AI development will only work with massive budgets like the $100M+ for GPT-4
    2. All future AI models will be free to train
    3. Innovative training methods can reduce costs without sacrificing capability
    4. GPU costs will no longer matter for AI development

12. Why might a company choose to use a smaller specialized model instead of a large frontier model?

    1. Smaller models can run without any computer hardware
    2. Smaller models are always more accurate than larger ones
    3. Smaller models require no training data
    4. Smaller models cost less to run and can be optimized for specific tasks

13. The lesson notes that inference happens 'billillions of times per day across all users.' Why is this fact significant for AI companies?

    1. It means companies should charge users per inference call
    2. It proves that AI is only used by a small number of people
    3. It means inference is no longer a concern for AI companies
    4. Even small per-query costs add up to enormous total expenses when multiplied by billions of calls

14. Based on the pricing trends described in the lesson, what pattern do you observe in AI model costs over time?

    1. Costs have remained roughly stable since 2022
    2. Costs are increasing as models get more capable
    3. Costs are steadily decreasing due to hardware improvements and optimization techniques
    4. Costs only change when companies decide to raise prices

15. How do improvements in hardware (like newer GPUs) reduce AI operating costs?

    1. By allowing more calculations to be performed per dollar spent on electricity and hardware
    2. By making AI models require less training data
    3. By reducing the need for any software optimization
    4. By making GPUs last longer physically