AI On-Device: Phi, Gemma, and When Tiny Models Make Sense

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-model-families-AI-tiny-models-on-device-r13a3-creators

1. What approximate parameter count enables AI models to run on typical consumer laptops and phones?

   1. 100 billion parameters
   2. 1 trillion parameters
   3. 500 million parameters
   4. 4 billion parameters

2. What is the primary trade-off when using compact on-device AI models instead of large cloud-based models?

   1. Trading speed for larger context windows and better reasoning
   2. Trading memory usage for access to the internet
   3. Trading accuracy for compatibility with older smartphones
   4. Trading some quality for privacy, offline use, and no per-call costs

3. Which of these is identified as a suitable use case for on-device AI models?

   1. Analyzing a 500-page legal document for nuanced arguments
   2. Classifying emails as spam or not spam in real-time
   3. Writing long-form novels with complex character arcs
   4. Answering ambiguous philosophical questions

4. A student wants to use an on-device model to solve a complex logic puzzle that requires multi-step reasoning. What limitation will they encounter?

   1. The model will automatically update itself to improve performance
   2. The model will require an internet connection to access additional data
   3. The model cannot match frontier models on hard reasoning tasks
   4. The model will refuse to run on battery power

5. A user installs an on-device AI app that claims to be fully private. Under what condition might their data still not be truly private?

   1. If the operating system or app still transmits data to external servers
   2. If they use the model while wearing headphones
   3. If they have more than 10GB of free storage
   4. If they use the model while charging their device

6. In which scenario would using a tiny on-device model make the most sense compared to calling a large cloud API?

   1. When summarizing meeting notes offline during a flight with no WiFi
   2. When translating a 50,000-word document in real-time
   3. When generating creative fiction with nuanced character development
   4. When needing the absolute highest accuracy on a critical medical diagnosis

7. What is quantization in the context of on-device AI models?

   1. A process for connecting models to cloud servers
   2. A method to increase model accuracy by adding more parameters
   3. A technique to compress model size by reducing numerical precision
   4. A security protocol that encrypts model outputs

8. An AI application needs to work inside a smart refrigerator that has no internet connection. Which model type would be most appropriate?

   1. A cloud-based model with streaming capabilities
   2. A small on-device model quantized for edge deployment
   3. A model requiring 100 billion parameters
   4. A frontier model accessed via API

9. What cost advantage do on-device models offer compared to cloud-based APIs?

   1. They cost less per user but more per interaction
   2. They are free to use forever with no limitations
   3. They eliminate per-call or per-token costs once the model is downloaded
   4. They require no initial investment but charge annual fees

10. A developer wants to process a 200-page document through an on-device model for analysis. What limitation will likely prevent this?

    1. On-device models require subscription fees for long documents
    2. On-device models cannot process text at all
    3. On-device models have limited context window handling
    4. On-device models cannot be used for analysis tasks

11. How do on-device models stay current with new knowledge compared to cloud models?

    1. They learn from user interactions over time
    2. They sync with search engines in real-time
    3. They automatically fetch updates whenever connected to WiFi
    4. They do not auto-update and require manual model replacement

12. Which two model families are specifically named in the lesson as examples of small on-device models?

    1. GPT and Claude
    2. Phi and Gemma
    3. Llama and Mistral
    4. BERT and RoBERTa

13. What type of device would most benefit from running a tiny AI model locally rather than calling a cloud API?

    1. A smartphone app requiring offline语音 assistance
    2. A supercomputer running scientific simulations
    3. A desktop workstation with multiple GPUs
    4. A server cluster in a data center

14. A developer needs to automatically categorize support tickets by topic. Why might a tiny on-device model be suitable for this task?

    1. Because classification is a simple, well-defined task tiny models can handle
    2. Because classification needs the latest knowledge from the internet
    3. Because classification requires complex multi-step reasoning
    4. Because classification requires emotional intelligence

15. What functionality would a tiny on-device model likely NOT be able to provide effectively?

    1. Answering vague or ambiguous questions about abstract topics
    2. Performing basic spelling and grammar checks
    3. Generating short replies based on context
    4. Quick text autocomplete in a messaging app