Local AI Models: When to Run Llama or Mistral on Your Laptop

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-tools-ai-local-models-ollama-r13a2-creators

1. What is a primary privacy benefit of running AI models locally instead of using cloud-based services?

   1. Local models delete all conversation history immediately after each session
   2. Your prompts and data are processed entirely on your own machine without being sent to external servers
   3. Local models automatically encrypt all your files stored on the computer
   4. Running models locally prevents any software from accessing your internet connection

2. Which of these tasks is a local 8B parameter model most likely to handle effectively?

   1. Solving advanced physics problems requiring multi-step reasoning
   2. Summarizing a long document into key bullet points
   3. Generating original research hypotheses for a new scientific field
   4. Creating detailed artistic images from text descriptions

3. If you generate code with a local model and then paste it into a cloud-based code hosting platform, what happens to your privacy?

   1. The privacy benefit is lost because the output now exists on external servers
   2. Your local model becomes more accurate due to cloud feedback
   3. The local model can now access your entire repository automatically
   4. Your data remains private because the model ran locally initially

4. Why do most consumer laptops struggle to run 70 billion parameter models smoothly?

   1. Consumer laptops lack the necessary audio drivers for running AI models
   2. Such large models require significantly more RAM and VRAM than typical consumer hardware provides
   3. Larger models need constant internet connectivity to function properly
   4. 70B models are incompatible with M-series Apple silicon processors

5. Which tool is mentioned in the lesson for running open-weight models on your local machine?

   1. Docker
   2. PyTorch
   3. TensorFlow
   4. Ollama

6. After you have set up a local AI model on your computer, what is the cost per token for using it?

   1. Zero dollars — you only pay for the initial hardware and electricity
   2. Free for the first 100,000 tokens, then a small monthly fee
   3. Calculated based on your internet service provider rates
   4. Exactly one cent per token regardless of model size

7. Which model is recommended in the lesson for coding tasks on an M-series Mac with 16GB+ RAM?

   1. Qwen2.5-Coder 7B
   2. GPT-4o
   3. Mistral 7B
   4. Llama 3.1 8B

8. What does 'open-weight' mean in the context of local AI models?

   1. Open-weight models are free to use for commercial purposes without any restrictions
   2. The model's internal parameters are publicly available and can be downloaded and run by anyone
   3. The model automatically opens any file you request without permission
   4. The model requires an open internet connection to function

9. Which statement best describes the privacy situation when using a local model completely offline?

   1. No data leaves your machine at any point during the entire process
   2. All your prompts are stored in a cloud database but never processed there
   3. Your data is only shared when you explicitly click a 'share' button
   4. Offline models automatically delete all conversation history every hour

10. What is a limitation of local models when it comes to reasoning tasks?

    1. They can only process text in English and one other language
    2. They lose accuracy after generating more than 1,000 tokens
    3. They cannot match frontier model quality on complex reasoning problems
    4. They are unable to perform any type of mathematical calculation

11. What should you audit to ensure your AI pipeline maintains privacy?

    1. The temperature setting you use when generating responses
    2. Just the amount of RAM your computer has available
    3. Only the model file itself to verify it wasn't tampered with
    4. The entire pipeline from input to output, including where you paste results

12. Why might someone choose to use LM Studio or Ollama despite having access to cloud AI?

    1. To automatically generate higher-quality images than dedicated image generators
    2. To share their local model with other users over the internet
    3. To access models that are more advanced than any available through cloud APIs
    4. To keep all their data and prompts completely private while avoiding per-token charges

13. What is the primary reason local models lag behind frontier models in quality?

    1. Local models use older algorithms that were superseded years ago
    2. Frontier models have access to your local files for context
    3. Local models are typically smaller and cannot run the most advanced architectures on consumer hardware
    4. Local models intentionally limit their accuracy to encourage cloud adoption

14. Which of the following is explicitly listed as a simple task local models can handle?

    1. Writing novel-length fiction with consistent characters
    2. Translating between 50+ languages with human-level accuracy
    3. Creating photorealistic video from text descriptions
    4. Classification of text into categories

15. What hardware configuration is specifically recommended in the lesson as a starting point?

    1. A server with more than 1TB of storage
    2. A desktop computer with at least four hard drives
    3. Any laptop with a dedicated NVIDIA RTX graphics card
    4. An M-series Mac with 16GB or more of RAM