Quantization Explained: GGUF, AWQ, GPTQ, and the Q4 vs Q8 vs FP16 Decision

A model file's quantization decides how big it is, how fast it runs, and how good it sounds. Learn the formats, the trade-offs, and how to pick the right one.

11 min · Reviewed 2026

What quantization is doing

An LLM's weights are originally floating-point numbers — typically FP16 or BF16. Quantization replaces those with lower-precision integers (often 4 or 8 bits per weight). The model gets smaller and faster. Quality drops a little. The question is always: how little, and is the drop worth the savings?

Precision	Bits / weight	Size for a 7B model	Quality vs FP16	Speed vs FP16
FP16 / BF16	16	~14GB	Reference	1.0x
Q8	~8	~7GB	Effectively identical	~1.3x
Q5	~5	~5GB	Very close	~1.6x
Q4	~4	~4GB	Slight degradation	~2.0x
Q3 / Q2	<4	~3GB or less	Noticeable degradation	~2x+

The format zoo: GGUF, AWQ, GPTQ

GGUF: the format llama.cpp (and therefore Ollama, LM Studio) uses. CPU-and-GPU friendly, the local-model default
AWQ: Activation-aware Weight Quantization. Common for GPU inference servers like vLLM. Good 4-bit quality
GPTQ: an older but still common GPU-targeted quantization. Often on Hugging Face for Linux/CUDA workflows
Native FP16 / BF16: the unquantized weights. Reference quality, large size, GPU only

What the suffixes like Q4_K_M actually mean

GGUF quants come with cryptic suffixes (Q4_K_M, Q5_K_S, Q6_K). The number is the bit width. 'K' means k-quants, a smarter quantization scheme than the original. 'S', 'M', 'L' are size variants — small, medium, large — that trade a tiny bit more space for a bit more quality. Q4_K_M is the most common 'good default' you will see in the wild.

Measuring quality, not vibes

Run llama-perplexity (in llama.cpp) on a representative text sample at FP16 vs your candidate quant
Run a short eval set of your real prompts and compare answers manually
Plot tokens-per-second on your hardware at each quant — the speed gain may surprise you
Pick the highest quant that fits your memory comfortably with your target context size

Apply this

Pick a 7B and download both Q4_K_M and Q8 versions of the same model
Compare answers on five representative prompts side by side
Compare tokens-per-second on your hardware and decide which one wins for your workload

The big idea: quantization is not a tax — it is a slider. Find the highest setting your hardware allows, and only drop further when you have to.

End-of-lesson check

8 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-local-quantization-explained-creators

What is the main idea of "Quantization Explained: GGUF, AWQ, GPTQ, and the Q4 vs Q8 vs FP16 Decision"?
1. A model file's quantization decides how big it is, how fast it runs, and how good it sounds.
2. Use AI as the final authority for the whole decision
3. Avoid checking the answer once it sounds polished
4. Focus only on speed instead of judgment
Which concept is most central to "Quantization Explained: GGUF, AWQ, GPTQ, and the Q4 vs Q8 vs FP16 Decision"?
1. GGUF
2. quantization
3. AWQ
4. GPTQ
Which use of AI fits this topic best?
1. Let the AI decide what matters without your review
2. Use the answer before checking whether it fits the situation
3. GGUF: the format llama.cpp (and therefore Ollama, LM Studio) uses. CPU-and-GPU friendly, the local-model default
4. Treat the AI output as automatically correct
What should a careful learner remember about "How to decide"?
1. Use "How to decide" as a reminder to verify the AI output before anyone relies on it.
2. Skip the context so the tool can guess faster
3. Treat the output as private even after sharing it online
4. Use the answer without checking the source
You want to use AI after this lesson. What is the safest next step?
1. Act immediately because the AI answer is written clearly
2. Use AI for drafting and comparison, but verify before publishing or relying on it.
3. Hide uncertainty so the final answer looks cleaner
4. Use private or sensitive details before checking permission
How should AI output about quantization be treated?
1. As proof that no other source is needed
2. As a replacement for context, consent, or expert review
3. As a draft or helper output that still needs human judgment and verification
4. As something that becomes correct when it sounds confident
Name one way to verify an AI answer about quantization.
Which action would help you apply "Quantization Explained: GGUF, AWQ, GPTQ, and the Q4 vs Q8 vs FP16 Decision" responsibly?
1. Use the tool to avoid thinking through the tradeoff
2. Keep going even if the output conflicts with a trusted source
3. Treat the AI output as automatically correct
4. AWQ: Activation-aware Weight Quantization. Common for GPU inference servers like vLLM. Good 4-bit quality

← Back to interactive lesson

Tendril · Creators · Model Families

Quantization Explained: GGUF, AWQ, GPTQ, and the Q4 vs Q8 vs FP16 Decision

A model file's quantization decides how big it is, how fast it runs, and how good it sounds. Learn the formats, the trade-offs, and how to pick the right one.

11 min · Reviewed 2026

What quantization is doing

Precision	Bits / weight	Size for a 7B model	Quality vs FP16	Speed vs FP16
FP16 / BF16	16	~14GB	Reference	1.0x
Q8	~8	~7GB	Effectively identical	~1.3x
Q5	~5	~5GB	Very close	~1.6x
Q4	~4	~4GB	Slight degradation	~2.0x
Q3 / Q2	<4	~3GB or less	Noticeable degradation	~2x+

The format zoo: GGUF, AWQ, GPTQ

GGUF: the format llama.cpp (and therefore Ollama, LM Studio) uses. CPU-and-GPU friendly, the local-model default
AWQ: Activation-aware Weight Quantization. Common for GPU inference servers like vLLM. Good 4-bit quality
GPTQ: an older but still common GPU-targeted quantization. Often on Hugging Face for Linux/CUDA workflows
Native FP16 / BF16: the unquantized weights. Reference quality, large size, GPU only

What the suffixes like Q4_K_M actually mean

Measuring quality, not vibes

Run llama-perplexity (in llama.cpp) on a representative text sample at FP16 vs your candidate quant
Run a short eval set of your real prompts and compare answers manually
Plot tokens-per-second on your hardware at each quant — the speed gain may surprise you
Pick the highest quant that fits your memory comfortably with your target context size

Apply this

Pick a 7B and download both Q4_K_M and Q8 versions of the same model
Compare answers on five representative prompts side by side
Compare tokens-per-second on your hardware and decide which one wins for your workload

The big idea: quantization is not a tax — it is a slider. Find the highest setting your hardware allows, and only drop further when you have to.

End-of-lesson check

8 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-local-quantization-explained-creators

What is the main idea of "Quantization Explained: GGUF, AWQ, GPTQ, and the Q4 vs Q8 vs FP16 Decision"?
1. A model file's quantization decides how big it is, how fast it runs, and how good it sounds.
2. Use AI as the final authority for the whole decision
3. Avoid checking the answer once it sounds polished
4. Focus only on speed instead of judgment
Which concept is most central to "Quantization Explained: GGUF, AWQ, GPTQ, and the Q4 vs Q8 vs FP16 Decision"?
1. GGUF
2. quantization
3. AWQ
4. GPTQ
Which use of AI fits this topic best?
1. Let the AI decide what matters without your review
2. Use the answer before checking whether it fits the situation
3. GGUF: the format llama.cpp (and therefore Ollama, LM Studio) uses. CPU-and-GPU friendly, the local-model default
4. Treat the AI output as automatically correct
What should a careful learner remember about "How to decide"?
1. Use "How to decide" as a reminder to verify the AI output before anyone relies on it.
2. Skip the context so the tool can guess faster
3. Treat the output as private even after sharing it online
4. Use the answer without checking the source
You want to use AI after this lesson. What is the safest next step?
1. Act immediately because the AI answer is written clearly
2. Use AI for drafting and comparison, but verify before publishing or relying on it.
3. Hide uncertainty so the final answer looks cleaner
4. Use private or sensitive details before checking permission
How should AI output about quantization be treated?
1. As proof that no other source is needed
2. As a replacement for context, consent, or expert review
3. As a draft or helper output that still needs human judgment and verification
4. As something that becomes correct when it sounds confident
Name one way to verify an AI answer about quantization.
Which action would help you apply "Quantization Explained: GGUF, AWQ, GPTQ, and the Q4 vs Q8 vs FP16 Decision" responsibly?
1. Use the tool to avoid thinking through the tradeoff
2. Keep going even if the output conflicts with a trusted source
3. Treat the AI output as automatically correct
4. AWQ: Activation-aware Weight Quantization. Common for GPU inference servers like vLLM. Good 4-bit quality

← Back to interactive lesson