Choosing a Local Model: Llama, Mistral, Hermes, Qwen, DeepSeek, and Friends

There are too many open-weight models. A short, opinionated tour of the major families and what each is actually good at.

10 min · Reviewed 2026

The open-weight skyline

Open-weight models cluster into a few families with distinct personalities and strengths. Within each family, sizes range from 1B (laptop-friendly) to 70B+ (small cluster). Knowing the families saves you from drowning in Hugging Face.

Family	Origin	Sweet spot	Reputation
Llama	Meta	General purpose, broad ecosystem	The default — well-supported
Mistral / Mixtral	Mistral AI (France)	Efficient, strong reasoning per parameter	European, MoE-friendly
Qwen	Alibaba	Coding, multilingual, long context	Often best-in-class at small sizes
DeepSeek	DeepSeek (China)	Reasoning and coding	Punches well above its size
Hermes / Nous	Community fine-tunes	Chattier, less refusal-y	Fine-tunes of base models
Phi	Microsoft Research	Tiny but capable	Great for embedded / edge
Gemma	Google	Light, well-tuned	Polished, conservative

How to pick — by job, not by hype

Coding assistant on a laptop: Qwen-coder, Llama-code, or DeepSeek-coder at 7-8B
General chat with broad world knowledge: Llama 3.x or 4.x at the largest size that fits
Long-context document analysis: Qwen long variants — strong tokenization for non-English
Small device or embedded: Phi or Gemma at 1-3B
Fewer refusals (research-only, not production): a Hermes or Nous fine-tune

Reading a model card before you commit

License: not all open-weight models are commercially usable
Context length: advertised vs. effective often differ
Tokenizer: matters for non-English performance and cost estimation
Base vs Instruct: are you getting the chat-tuned version?
Eval scores: take vendor numbers with salt; trust independent leaderboards more

Apply this

Pick three models from three different families that fit your hardware
Run the same five-prompt eval set on each
Pick the one that wins on your real task — not the one with the highest leaderboard rank

The big idea: the right local model is the one that wins on your prompts, on your hardware. Family names are a starting filter, not an answer.

End-of-lesson check

8 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-local-choosing-a-model-creators

What is the main idea of "Choosing a Local Model: Llama, Mistral, Hermes, Qwen, DeepSeek, and Friends"?
1. There are too many open-weight models. A short, opinionated tour of the major families and what each is actually good at.
2. Use AI as the final authority for the whole decision
3. Avoid checking the answer once it sounds polished
4. Focus only on speed instead of judgment
Which concept is most central to "Choosing a Local Model: Llama, Mistral, Hermes, Qwen, DeepSeek, and Friends"?
1. model family
2. open weights
3. fine-tune
4. instruction tuning
Which use of AI fits this topic best?
1. Let the AI decide what matters without your review
2. Use the answer before checking whether it fits the situation
3. Coding assistant on a laptop: Qwen-coder, Llama-code, or DeepSeek-coder at 7-8B
4. Treat the AI output as automatically correct
What should a careful learner remember about "Base vs Instruct vs Chat"?
1. Use AI to draft or organize ideas about open weights, then verify before acting.
2. Skip the context so the tool can guess faster
3. Treat the output as private even after sharing it online
4. Use the answer without checking the source
You want to use AI after this lesson. What is the safest next step?
1. Act immediately because the AI answer is written clearly
2. Use AI for drafting and comparison, but verify before publishing or relying on it.
3. Hide uncertainty so the final answer looks cleaner
4. Use private or sensitive details before checking permission
How should AI output about open weights be treated?
1. As proof that no other source is needed
2. As a replacement for context, consent, or expert review
3. As a draft or helper output that still needs human judgment and verification
4. As something that becomes correct when it sounds confident
Name one way to verify an AI answer about open weights.
Which action would help you apply "Choosing a Local Model: Llama, Mistral, Hermes, Qwen, DeepSeek, and Friends" responsibly?
1. Use the tool to avoid thinking through the tradeoff
2. Keep going even if the output conflicts with a trusted source
3. Treat the AI output as automatically correct
4. General chat with broad world knowledge: Llama 3.x or 4.x at the largest size that fits

← Back to interactive lesson

Tendril · Creators · Model Families