Hardware Sizing for Local Models: VRAM, Unified Memory, and CPU-Only Realities

Whether a model runs well — or at all — depends on the hardware you put under it. Here is the practical map of what hardware can run which class of model.

10 min · Reviewed 2026

The first question is always memory

An LLM has to fit into memory before it can run. On a discrete GPU, that means VRAM. On Apple Silicon, that means unified memory shared between CPU and GPU. On a CPU-only machine, that means RAM and a lot of patience. Whatever runs is whatever fits. So the buying decision is really a memory-sizing decision.

Hardware	Useful memory	Realistic model class	Vibe
8GB integrated GPU laptop	~6GB usable	Up to ~7B at Q4	Toy projects, learning
16GB Apple Silicon Mac	~10-12GB usable	Up to ~13B at Q4	Solid daily driver
24GB consumer GPU (e.g. high-end RTX class)	~22GB usable	Up to ~30B at Q4 or 13B at Q8	Comfortable workhorse
48GB+ Mac Studio class	~40GB+ usable	Up to ~70B at Q4	Power user / small team server
80GB+ datacenter GPU	~78GB+	70B at Q8 or 405B at low quant	Serious self-host

Apple Silicon's unfair advantage

Apple's unified memory architecture means a 64GB Mac Studio can hold a 70B-class model that a 24GB consumer GPU simply cannot. Throughput is not as high as a top-end discrete GPU, but the ceiling on model size is dramatically higher per dollar. For local inference, M-series Macs punch far above their weight.

CPU-only is a thing — barely

A modern desktop CPU can run a 7-8B model at a few tokens per second
Useful for batch processing where latency does not matter
Dramatically slower than even a modest GPU
Good for running a coding assistant in the background — bad for chat UX

How to size before you buy

Decide which model class you actually need (7B, 13B, 30B, 70B)
Pick the quantization you can tolerate quality-wise (Q4 is the sweet spot for most)
Add 25% for KV cache, runtime, and the rest of your OS
Buy hardware whose usable memory comfortably exceeds that number — not just barely matches it

Apply this

Look up the unified-memory or VRAM number on your current hardware
Compute the largest model you can comfortably run at Q4 with 8k context
Identify the smallest hardware upgrade that would unlock the next class up

The big idea: pick the model first, then size the memory, then pick the hardware. Reversing that order is how teams end up with great GPUs that cannot run the model they actually want.

End-of-lesson check

8 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-local-hardware-sizing-creators

What is the main idea of "Hardware Sizing for Local Models: VRAM, Unified Memory, and CPU-Only Realities"?
1. Whether a model runs well — or at all — depends on the hardware you put under it.
2. Use AI as the final authority for the whole decision
3. Avoid checking the answer once it sounds polished
4. Focus only on speed instead of judgment
Which concept is most central to "Hardware Sizing for Local Models: VRAM, Unified Memory, and CPU-Only Realities"?
1. unified memory
2. VRAM
3. Apple Silicon
4. CPU inference
Which use of AI fits this topic best?
1. Let the AI decide what matters without your review
2. Use the answer before checking whether it fits the situation
3. A modern desktop CPU can run a 7-8B model at a few tokens per second
4. Treat the AI output as automatically correct
What should a careful learner remember about "The math: parameters times bytes"?
1. Use AI to draft or organize ideas about VRAM, then verify before acting.
2. Skip the context so the tool can guess faster
3. Treat the output as private even after sharing it online
4. Use the answer without checking the source
You want to use AI after this lesson. What is the safest next step?
1. Act immediately because the AI answer is written clearly
2. Use AI for drafting and comparison, but verify before publishing or relying on it.
3. Hide uncertainty so the final answer looks cleaner
4. Use private or sensitive details before checking permission
How should AI output about VRAM be treated?
1. As proof that no other source is needed
2. As a replacement for context, consent, or expert review
3. As a draft or helper output that still needs human judgment and verification
4. As something that becomes correct when it sounds confident
Name one way to verify an AI answer about VRAM.
Which action would help you apply "Hardware Sizing for Local Models: VRAM, Unified Memory, and CPU-Only Realities" responsibly?
1. Use the tool to avoid thinking through the tradeoff
2. Keep going even if the output conflicts with a trusted source
3. Treat the AI output as automatically correct
4. Useful for batch processing where latency does not matter

← Back to interactive lesson

Tendril · Creators · Model Families

Hardware Sizing for Local Models: VRAM, Unified Memory, and CPU-Only Realities

Whether a model runs well — or at all — depends on the hardware you put under it. Here is the practical map of what hardware can run which class of model.

10 min · Reviewed 2026

The first question is always memory

Hardware	Useful memory	Realistic model class	Vibe
8GB integrated GPU laptop	~6GB usable	Up to ~7B at Q4	Toy projects, learning
16GB Apple Silicon Mac	~10-12GB usable	Up to ~13B at Q4	Solid daily driver
24GB consumer GPU (e.g. high-end RTX class)	~22GB usable	Up to ~30B at Q4 or 13B at Q8	Comfortable workhorse
48GB+ Mac Studio class	~40GB+ usable	Up to ~70B at Q4	Power user / small team server
80GB+ datacenter GPU	~78GB+	70B at Q8 or 405B at low quant	Serious self-host

Apple Silicon's unfair advantage

CPU-only is a thing — barely

A modern desktop CPU can run a 7-8B model at a few tokens per second
Useful for batch processing where latency does not matter
Dramatically slower than even a modest GPU
Good for running a coding assistant in the background — bad for chat UX

How to size before you buy

Decide which model class you actually need (7B, 13B, 30B, 70B)
Pick the quantization you can tolerate quality-wise (Q4 is the sweet spot for most)
Add 25% for KV cache, runtime, and the rest of your OS
Buy hardware whose usable memory comfortably exceeds that number — not just barely matches it

Apply this

Look up the unified-memory or VRAM number on your current hardware
Compute the largest model you can comfortably run at Q4 with 8k context
Identify the smallest hardware upgrade that would unlock the next class up

The big idea: pick the model first, then size the memory, then pick the hardware. Reversing that order is how teams end up with great GPUs that cannot run the model they actually want.

End-of-lesson check

8 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-local-hardware-sizing-creators

What is the main idea of "Hardware Sizing for Local Models: VRAM, Unified Memory, and CPU-Only Realities"?
1. Whether a model runs well — or at all — depends on the hardware you put under it.
2. Use AI as the final authority for the whole decision
3. Avoid checking the answer once it sounds polished
4. Focus only on speed instead of judgment
Which concept is most central to "Hardware Sizing for Local Models: VRAM, Unified Memory, and CPU-Only Realities"?
1. unified memory
2. VRAM
3. Apple Silicon
4. CPU inference
Which use of AI fits this topic best?
1. Let the AI decide what matters without your review
2. Use the answer before checking whether it fits the situation
3. A modern desktop CPU can run a 7-8B model at a few tokens per second
4. Treat the AI output as automatically correct
What should a careful learner remember about "The math: parameters times bytes"?
1. Use AI to draft or organize ideas about VRAM, then verify before acting.
2. Skip the context so the tool can guess faster
3. Treat the output as private even after sharing it online
4. Use the answer without checking the source
You want to use AI after this lesson. What is the safest next step?
1. Act immediately because the AI answer is written clearly
2. Use AI for drafting and comparison, but verify before publishing or relying on it.
3. Hide uncertainty so the final answer looks cleaner
4. Use private or sensitive details before checking permission
How should AI output about VRAM be treated?
1. As proof that no other source is needed
2. As a replacement for context, consent, or expert review
3. As a draft or helper output that still needs human judgment and verification
4. As something that becomes correct when it sounds confident
Name one way to verify an AI answer about VRAM.
Which action would help you apply "Hardware Sizing for Local Models: VRAM, Unified Memory, and CPU-Only Realities" responsibly?
1. Use the tool to avoid thinking through the tradeoff
2. Keep going even if the output conflicts with a trusted source
3. Treat the AI output as automatically correct
4. Useful for batch processing where latency does not matter

← Back to interactive lesson