Hermes On A Mac: Apple Silicon Performance Notes

M-series Macs combine CPU, GPU, and large unified memory in one chip. For LLM inference, that means models can use the full RAM as VRAM — a 32GB Mac can comfortably run a 13B model in higher precision than a 12GB Nvidia consumer card. Apple's Metal stack and the MLX framework give well-optimized inference, especially on M3/M4 Pro/Max chips.

Hardware in plain English

Runtime choice on Mac

• Ollama runs llama.cpp under Metal — broadly fast, well-supported, no fuss.
• LM Studio supports both Metal and an MLX backend. MLX often delivers materially higher tokens-per-second on M-series, particularly on M3/M4.
• Native MLX projects (mlx-lm and friends) give the best per-watt performance for users comfortable with Python.
• Docker is generally a bad fit for LLM inference on macOS — you lose Metal acceleration.

Things to do on day one

1. 1Plug in. Battery throttling reduces inference performance noticeably; serious work happens plugged in.
2. 2Set 'Prevent automatic sleeping' on while a long inference runs — silent sleep eats your batch.
3. 3Close Chrome with a hundred tabs. Memory pressure will swap and tank inference speed.
4. 4Watch Activity Monitor's GPU column on the first few runs to confirm the model is actually using the GPU, not falling back to CPU.
5. 5Cool the laptop — sustained inference will throttle a hot MacBook within minutes.

Applied exercise

1. 1Run the same Hermes model and prompt through Ollama and through LM Studio's MLX backend.
2. 2Note tokens/sec for each. The difference may be 20-60% on M-series chips.
3. 3Pick whichever is faster for your daily workflow.
4. 4Set your inference workflow to launch automatically at login if you'll use it daily.

The big idea: Macs are unusually good at this work. The right runtime, an unplugged battery plan, and decent thermals get you most of the way.