AI Tools: Decide Between Local Models and Hosted APIs With a Real Workload

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-tools-local-vs-hosted-models-r8a1-creators

1. A company needs to process 100,000 customer support chats per day. The chats contain sensitive personal data, and responses must be consistent with company policy. Which deployment approach is most appropriate?

   1. Hybrid with local model and API fallback
   2. Serverless function with on-demand scaling
   3. Hosted API from a major provider
   4. Local LLM deployed on company servers

2. What is the MOST important consideration when deciding between a local LLM and a hosted API?

   1. The brand reputation of the AI provider
   2. Your team's preference for open-source tools
   3. The characteristics of your specific workload
   4. The latest model benchmarks

3. Which cost component is MOST often underestimated in local LLM deployments?

   1. GPU purchase price
   2. Model licensing fees
   3. Operational overhead (monitoring, on-call, upgrades)
   4. Electricity for running the server

4. A startup is building a prototype for a new product feature. They need to iterate quickly and don't yet know how many users will adopt it. Which deployment choice makes the most sense?

   1. Purchase GPUs and deploy a local model
   2. Build a custom local infrastructure from scratch
   3. Train a model from scratch on company data
   4. Use a hosted API on a pay-per-token basis

5. What does TCO stand for, and why is it important for local LLM deployment?

   1. Technical Configuration Option; it describes model settings
   2. Total Cost of Ownership; it captures all expenses including hardware, ops, and maintenance
   3. Token Computation Output; it measures API usage
   4. Total Cost of Operation; it measures only electricity and cooling costs

6. Which of these is a key limitation of AI when recommending infrastructure choices?

   1. AI cannot predict model quality on your specific data without testing
   2. AI always recommends the most expensive option
   3. AI cannot understand natural language queries
   4. AI cannot calculate costs accurately

7. A healthcare company needs to summarize patient notes. The summaries must be medically accurate, and privacy regulations are strict. What should be part of their deployment strategy?

   1. Implement a local model with a fallback to hosted API for complex cases
   2. Rely solely on human review for all summaries
   3. Use any public hosted API since they all have security
   4. Only use local models with no alternatives

8. Which statement about local LLMs is TRUE according to the decision framework?

   1. Local models are cheaper for low-volume applications
   2. Local models are always faster than hosted APIs
   3. Local models require no maintenance once deployed
   4. Local models provide privacy by default since data stays on premises

9. When evaluating a workload, what three factors should you score?

   1. Speed, cost, and popularity
   2. Volume, capability needs, and privacy requirements
   3. Color, size, and brand
   4. Language, format, and storage

10. A company runs 10 million inference requests per month. They have ML ops expertise on staff. What is likely the most cost-effective choice?

    1. Local LLM on company GPUs
    2. Hosted API with pay-per-token pricing
    3. Cloud-based virtual machines
    4. Human reviewers for all requests

11. What is a hybrid deployment, and when is it useful?

    1. Running both local and hosted models, using each for appropriate tasks
    2. Using two different hosted providers for redundancy
    3. A load balancer for distributing requests
    4. A single model that runs on both cloud and edge

12. Which scenario BEST demonstrates appropriate use of a hosted API?

    1. Occasional ad-hoc analysis where capability needs are broad and usage is unpredictable
    2. A fixed dataset that needs processing once per year
    3. A standalone offline application with no internet
    4. Processing 5,000 financial transactions per day with zero latency requirements

13. What operational skills are required for successful local LLM deployment?

    1. Only data entry skills
    2. Marketing and sales
    3. Graphic design and UI development
    4. GPU management, monitoring, on-call support, and infrastructure upgrades

14. A retailer wants to generate product descriptions for their catalog. They have 50,000 products and update weekly. Descriptions must match brand voice exactly. Which approach?

    1. Human writers for each product
    2. Hosted API for flexibility
    3. Generic template with no AI
    4. Local LLM fine-tuned on brand examples

15. What is a critical factor that AI cannot account for when making infrastructure recommendations?

    1. Your team's operational skills
    2. The current weather
    3. The CEO's favorite color
    4. The phase of the moon