AI and self-hosted LLM deployment tools

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-creators-tools-AI-and-self-hosted-LLM-deployment-tools-r9a1-creators

1. What is the primary trade-off when choosing to self-host an LLM instead of using an API service?

   1. Self-hosting eliminates the need for GPU hardware
   2. Self-hosting automatically improves model accuracy
   3. Self-hosting trades lower cost-per-token for higher operational complexity
   4. Self-hosting reduces latency but increases storage costs

2. Which of the following is explicitly listed as a criterion for comparing different serving stacks in the material?

   1. Batching support
   2. Community forum size
   3. License cost
   4. GitHub star count

3. A serving stack's 'model coverage' refers to what?

   1. The number of concurrent users supported
   2. The number of API endpoints available
   3. Which model architectures and sizes the framework can run
   4. The geographic distribution of deployment regions

4. Why might a single-GPU demonstration of a serving stack be misleading for production decisions?

   1. Multi-GPU and multi-node setups introduce additional complexity and failure modes
   2. Single-GPU demos are too fast to measure accurately
   3. GPU count does not affect KV-cache behavior
   4. Single-GPU demos show the maximum potential performance

5. Why can AI tools not fully replace actual load testing when evaluating serving stacks?

   1. AI cannot simulate the actual concurrent requests, memory pressure, and hardware behavior of real traffic
   2. Load testing is illegal in most jurisdictions
   3. Load testing provides no useful data
   4. AI has already tested all serving stacks exhaustively

6. What does 'throughput' measure in the context of LLM serving?

   1. The latency of a single token generation
   2. The total memory used by the model
   3. The number of requests completed per second
   4. The time to generate the first token

7. What does GPU memory ceiling refer to?

   1. The largest model size that can fit in GPU memory for inference
   2. The maximum number of GPUs you can purchase
   3. The highest price you should pay for GPU rental
   4. The maximum electricity consumption of a GPU

8. Why would an AI be unable to accurately predict performance after swapping GPU hardware?

   1. Predictions are always accurate
   2. GPU hardware cannot be swapped
   3. AI systems do not understand hardware
   4. Hardware changes affect memory bandwidth, interconnect speed, and CUDA kernel optimization in ways that require empirical testing

9. In the context of KV-cache, what role does it play in LLM inference?

   1. It encrypts the input prompts
   2. It balances load across multiple servers
   3. It caches previously computed key-value pairs to avoid redundant computation
   4. It stores the compiled model weights

10. What operational role can an AI not substitute when self-hosting LLMs?

    1. A marketing specialist
    2. A data scientist
    3. An SRE (Site Reliability Engineer) on call
    4. A software developer

11. Batching in LLM serving is used to:

    1. Split a single request into multiple smaller requests
    2. Process multiple requests together to improve GPU utilization
    3. Encrypt requests for security
    4. Reduce the model size for faster loading

12. Why might selecting a serving stack based on GitHub stars be problematic?

    1. Stars cause increased latency
    2. GitHub stars measure popularity, not throughput, model coverage, or operational fit for your specific use case
    3. GitHub stars are fake
    4. GitHub stars indicate nothing useful

13. What should you test at before committing to a serving stack for production?

    1. Your actual concurrency level
    2. Only the demo environment
    3. The license agreement
    4. The marketing materials

14. vLLM and TGI are examples of what?

    1. LLM models themselves
    2. LLM serving frameworks
    3. GPU hardware vendors
    4. Cloud API providers

15. A bursty traffic pattern with sudden spikes would most likely stress which aspect of your serving infrastructure?

    1. Hard drive storage capacity
    2. Keyboard response time
    3. GPU memory allocation and KV-cache management
    4. Network cable thickness