Modal: Serverless GPUs for AI Without Kubernetes

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-tools-modal-serverless-gpu-r7a4-creators

1. What type of AI workloads is Modal specifically designed to handle efficiently?

   1. Spiky workloads that have quiet periods between bursts of activity
   2. Continuous high-throughput inference with steady request rates
   3. Long-running training jobs that need 24/7 GPU availability
   4. Applications that require single-digit millisecond latency

2. What happens to a Modal function when it receives no requests for a period of time?

   1. It is automatically deleted to save storage space
   2. It scales to zero and must cold-start on the next request
   3. It switches to a lower-power CPU-only mode
   4. It remains warm and ready to respond immediately

3. At approximately what hourly GPU utilization rate does the lesson suggest reserved GPUs become more cost-effective than serverless?

   1. 50%
   2. 70%
   3. 10%
   4. 30%

4. Why can loading a 70B parameter model from blob storage on each cold start become expensive?

   1. Blob storage automatically deletes large models after 24 hours
   2. The model must be loaded into GPU memory on every request, adding latency and compute costs
   3. 70B models require special licensing fees per cold start
   4. Blob storage has unlimited bandwidth that causes throttling

5. Which statement best describes a cold start in serverless GPU computing?

   1. The initial training phase of a machine learning model
   2. The process of initializing and loading a model when a function wakes from an idle state
   3. The time it takes for a GPU to heat up to operating temperature
   4. A backup mechanism that activates when the primary GPU fails

6. Why is Modal not ideal for ultra-low-latency inference requirements?

   1. Serverless functions may have variable latency due to cold starts
   2. Python functions cannot execute faster than 100ms
   3. Modal only supports CPU inference, not GPU
   4. Network routing adds exactly 10 seconds to every request

7. What is a key benefit of Modal's Python-native approach to deployment?

   1. It eliminates the need for any Python programming knowledge
   2. Python code runs faster than CUDA on GPUs
   3. Developers can deploy GPU-backed functions using standard Python decorators without managing infrastructure
   4. It automatically converts Python code to CUDA kernels

8. What happens to your costs when you run inference at very high QPS (queries per second) continuously on Modal?

   1. Modal automatically provisions dedicated GPUs for high traffic
   2. You receive a discount for consistent high usage
   3. Costs become negligible because the function stays warm
   4. Costs may exceed dedicated GPU infrastructure due to per-request overhead

9. What does it mean that Modal enables 'scale to zero'?

   1. The function can only handle zero input data
   2. The function automatically deletes itself after exactly 1000 requests
   3. GPU resources are released when no requests are coming in
   4. Modal reduces your bill to zero dollars automatically

10. Why might a keep-warm trick help improve Modal performance?

    1. It allows the function to run without any Python dependencies
    2. It prevents cold starts by periodically invoking the function to keep it warm
    3. It reduces the hourly cost of reserved instances
    4. It automatically upgrades the GPU to a faster model

11. For which scenario would Modal be the WORST choice?

    1. An image processing endpoint that gets 50 requests per day
    2. A chatbot that receives 10,000 requests per second around the clock
    3. A batch inference job that runs once per month with varying dataset sizes
    4. A periodic data transformation task that runs overnight

12. What capability does Modal provide for running batch inference jobs?

    1. It requires all batch jobs to be submitted manually through a web UI
    2. It can only process one image at a time
    3. It cannot run batch jobs, only interactive endpoints
    4. It can distribute work across hundreds of GPUs on demand

13. What is the main trade-off when choosing Modal over dedicated GPU infrastructure?

    1. You gain convenience and flexibility but may pay more at high utilization
    2. Dedicated GPUs cannot run Python code
    3. Modal automatically trains your models for you
    4. Modal requires writing code in a special proprietary language

14. What should guide the decision between serverless GPUs and dedicated infrastructure?

    1. Mathematical calculation of utilization and costs
    2. The phase of the moon
    3. Random selection
    4. Vibes and general impressions

15. Which of these is listed as something AI (and Modal) CANNOT do?

    1. Deploy GPU-backed functions from Python
    2. Scale to zero between requests
    3. Eliminate cold starts on huge models without keep-warm tricks
    4. Run batch inference across multiple GPUs