AI and Ollama Local Model Routing for Mixed Workloads

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-creators-tools-AI-and-ollama-local-model-routing-r11a4-creators

1. What is the primary problem with using a single default Ollama model for all tasks?

   1. The default model automatically updates itself and breaks compatibility with your code
   2. The default model cannot be changed once Ollama is installed
   3. One model uses too much VRAM when handling multiple simultaneous requests
   4. One model cannot perform optimally across all different types of tasks

2. Which task would be LEAST suitable for routing to a lightweight local model with limited context window?

   1. Summarizing a 50-page technical document
   2. Answering yes/no questions about public facts
   3. Generating short creative writing prompts
   4. Translating single sentences between common languages

3. What does a 'fallback chain' mean in the context of model routing?

   1. A list of all models ranked by popularity
   2. A backup model that runs automatically when the primary model fails
   3. A sequence showing which models were tried in previous sessions
   4. A method to reduce VRAM usage by chaining models together

4. Why can't AI accurately predict how long a cold model will take to load?

   1. Cold loading depends on hardware, current VRAM state, and disk speed, which vary per system
   2. AI models cannot read system clocks
   3. AI only predicts server-based model latency, not local
   4. The lesson states AI cannot predict latency on cold starts

5. When routing tasks across multiple local models, what is the primary benefit of matching task type to model capability?

   1. It automatically updates all models to compatible versions
   2. It ensures each task uses the most appropriate model for better results
   3. It reduces the total number of models you need to download
   4. It allows all models to share the same VRAM pool

6. A developer wants to route 6 different task types across 4 local models with fallback chains. What information should the AI help draft?

   1. A list of all Ollama compatible operating systems
   2. Routing rules that assign each task type to a primary model and fallback models
   3. Installation instructions for each of the four models
   4. A ranking of which models are most popular online

7. What limitation specifically prevents AI from running certain local models on your machine?

   1. Ollama blocks AI from directly running models
   2. Your internet connection is too slow to load models
   3. The models require more VRAM than your GPU provides
   4. AI lacks permission to access your local files

8. Which scenario BEST demonstrates the value of a model routing system?

   1. Keeping all four models loaded in VRAM simultaneously
   2. Running the same 7B model for code, math, and creative writing
   3. Using a code-specialized model for programming tasks, a summarization model for long documents, and a translation model for language tasks
   4. Manually selecting a different model for each task through a command-line menu

9. Why might a benchmark plan be useful when setting up model routing?

   1. Benchmarks measure each model's speed and quality on your specific hardware, helping you assign the right model to the right task
   2. Benchmarks automatically configure your VRAM settings
   3. Benchmarks determine which models are legally licensed for local use
   4. Benchmarks are required by Ollama's terms of service

10. What happens when a routed task reaches a model that has been evicted from VRAM?

    1. The model must be loaded back into memory, causing a delay of 20+ seconds for cold start
    2. The task waits indefinitely until the model is manually reloaded
    3. The task automatically fails with an error message
    4. The routing system downgrades the task to use a text-only mode

11. In a routing configuration with a fallback chain, what determines the order of fallback models?

    1. Random selection to ensure variety
    2. Alphabetical order by model name
    3. Priority based on which model is most likely to succeed given the task type and hardware constraints
    4. The order in which models were downloaded

12. If you have a 4GB VRAM GPU and want to run local models, which model size strategy would AI recommend?

    1. Use a mix of model sizes and implement pinning for the largest ones
    2. Avoid local models entirely and use API-based models
    3. Use only 7B or smaller models that fit in available VRAM
    4. Use only 70B models since they offer the best quality

13. When designing routing rules, what information should each rule ideally contain?

    1. The preferred model, acceptable fallback models, and conditions that trigger each choice
    2. A list of all available models in the system
    3. Instructions for how to install the model
    4. Only the name of the preferred model

14. What is a key difference between routing tasks to local models versus sending tasks to a cloud API?

    1. Cloud APIs cannot handle complex tasks
    2. Local models have predictable latency that never changes
    3. Local models run on your own hardware, so you control them but must manage VRAM and loading times
    4. Local models are always faster than cloud APIs

15. A user reports their routing system is slow even though they have a powerful GPU. What might be the cause?

    1. Models are being loaded from VRAM every time (cold starts), possibly because the wrong models are kept hot
    2. Too many models are pinned in VRAM simultaneously, causing thrashing
    3. The Ollama server is connected to the internet
    4. The routing system is using an outdated version of Python