Local Model Family: Gemma

Gemma is a useful local-model lesson because it makes one trade-off visible: small local assistants, education demos, research baselines, and comparing Google-style open models to Qwen, Mistral, and Llama. The point is not to crown a permanent winner. The point is to learn how to match a model family to hardware, task, license, and risk.

Current source signal

Build the small version

Create a Gemma model card reader: students extract size, license terms, intended uses, unsafe uses, and runtime requirements.

1. 1Pick one exact model file or runtime tag from the current model card.
2. 2Run three short prompts: one easy, one task-specific, and one likely failure case.
3. 3Record load time, response speed, memory pressure, answer quality, and one surprising failure.
4. 4Write a one-paragraph recommendation: use it, do not use it, or use it only for a narrow job.

model_card_notes:
  family: Gemma
  size: check_current_card
  quantized_available: yes_or_no
  intended_use: classroom_demo_or_app
  license_terms: summarize_before_use
  safety_notes: copy_key_limits
  runtime: ollama_lmstudio_or_transformers

The big idea: remember model card reader. Local model work is product design under constraints, not just downloading the model with the loudest leaderboard score.

Gemma sizing is a useful local-model lesson because it makes one trade-off visible: teaching students how to match a model to RAM, VRAM, latency, and task difficulty. The point is not to crown a permanent winner. The point is to learn how to match a model family to hardware, task, license, and risk.

Current source signal

Build the small version

Run a small Gemma variant and a larger one, then score speed, memory pressure, and answer quality on the same five prompts.

1. 1Pick one exact model file or runtime tag from the current model card.
2. 2Run three short prompts: one easy, one task-specific, and one likely failure case.
3. 3Record load time, response speed, memory pressure, answer quality, and one surprising failure.
4. 4Write a one-paragraph recommendation: use it, do not use it, or use it only for a narrow job.

sizing_test:
  prompts: 5
  models:
    - gemma_small_quantized
    - gemma_larger_quantized
  measure:
    - load_time
    - tokens_per_second
    - memory_used
    - quality_score

choose: smallest model that passes the task rubric

The big idea: remember smallest passing model. Local model work is product design under constraints, not just downloading the model with the loudest leaderboard score.

Gemma variants is a useful local-model lesson because it makes one trade-off visible: discussing when a specialized local model beats a general chat model at a narrow job. The point is not to crown a permanent winner. The point is to learn how to match a model family to hardware, task, license, and risk.

Current source signal

Build the small version

Compare a general chat prompt, an image prompt, and a domain prompt. For each one, decide whether a general model or specialized model is appropriate.

1. 1Pick one exact model file or runtime tag from the current model card.
2. 2Run three short prompts: one easy, one task-specific, and one likely failure case.
3. 3Record load time, response speed, memory pressure, answer quality, and one surprising failure.
4. 4Write a one-paragraph recommendation: use it, do not use it, or use it only for a narrow job.

specialized_model_decision:
  if input.type == image:
    consider vision_variant
  if domain == medical_or_legal:
    require expert_review
  if task == normal_chat:
    use general_instruct

rule: specialization changes evaluation, not just capability

The big idea: remember specialization changes evaluation. Local model work is product design under constraints, not just downloading the model with the loudest leaderboard score.