Lesson 1515 of 1596
AI On-Device: Phi, Gemma, and When Tiny Models Make Sense
4B-parameter models run on your laptop and phone. They're not GPT-5 — but they're surprisingly useful.
Creators · Model Families · ~7 min read
The premise
Small models running locally trade peak quality for privacy, offline capability, and zero per-call cost.
What AI does well here
- Privacy-sensitive text processing
- Offline summarization and classification
- Local autocomplete and quick assistants
- Edge devices and mobile apps
What AI cannot do
- Match frontier models on hard reasoning
- Handle very long contexts comfortably
- Replace cloud models for ambiguous, complex prompts
- Stay current — they don't auto-update
Key terms in this lesson
Practice this safely
Use a small project example from your own work. The useful move is to compare the AI's draft against your goal, sources, and constraints before you trust it.
- 1Ask AI to explain on-device in plain language, then underline anything that sounds uncertain or too broad.
- 2Give it one detail from "AI On-Device: Phi, Gemma, and When Tiny Models Make Sense" and ask for two possible next steps plus one reason each step might be wrong.
- 3Check Phi against a trusted source, teacher, adult, expert, or original document before you use it.
End-of-lesson quiz
Check what stuck
10 questions · Score saves to your progress.
Tutor
Curious about “AI On-Device: Phi, Gemma, and When Tiny Models Make Sense”?
Ask anything about this lesson. I’ll answer using just what you’re reading — short, friendly, grounded.
Progress saved locally in this browser. Sign in to sync across devices.
Related lessons
Keep going
Creators · 11 min
Small Language Models on Device: Phi, Gemma, Llama 3.2 in Production
When a 3B-7B model on-device wins over an API call to a frontier model.
Creators · 11 min
AI On-Device Models: Phi, Gemma, and the Edge Tradeoff
What current on-device AI models can do — and where edge inference falls short.
Creators · 9 min
Quantization Tradeoffs (Q4 Vs Q8) For Hermes
Quantization is the dial between model quality and what fits on your hardware. With Hermes, the right setting depends entirely on the task — there is no universal answer.