Ollama Basics: Running a Model Yourself

Ollama turns 'I want to run an LLM locally' into a one-line install and a two-word command. Here's the stack, the key commands, and the models worth pulling first.

32 min · Reviewed 2026

What Ollama is

Ollama is a command-line tool that downloads, manages, and serves local LLMs. Under the hood it uses llama.cpp (the fastest open-source runtime) and ships models in GGUF format. You get a localhost API that any app can call — including Claude Code, CrewAI, LangGraph, and OpenClaw.

Install and first model

# macOS: one-line install brew install ollama # Or download the app from ollama.com # Start the background server ollama serve & # Pull and run Llama 4 (8B — works on most modern laptops) ollama run llama4:8b # You're now chatting with a local model. Ctrl-D to exit.From zero to local model in three commands.

The commands worth knowing

ollama list # what's installed ollama pull qwen3.5:8b # download (no run) ollama run gemma4:4b # download if needed, then chat ollama rm llama4:70b # free disk space ollama ps # what's loaded in memory ollama show qwen3.5:8b # details: params, context, quant # API access (OpenAI-compatible) curl http://localhost:11434/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "qwen3.5:8b", "messages": [{"role": "user", "content": "Hello"}] }'Core Ollama commands and the OpenAI-compatible API.

Models to try (April 2026)

Model	Size	Best for
llama4:8b	~5 GB	General chat, balanced speed/quality.
qwen3.5:8b	~5 GB	Strong function calling + coding.
gemma4:12b	~8 GB	Google's frontier-at-size model, reasoning-tuned.
qwen3.5:32b	~20 GB	Near-frontier quality on a 32 GB Mac.
deepseek-coder:16b	~10 GB	Code-focused. Fast on a laptop GPU.
llama4:70b	~40 GB	Highest quality. Needs a Mac Studio or a real GPU box.

Using Ollama with agents

Because Ollama exposes an OpenAI-compatible API at localhost:11434, it drops into any agent framework that supports OpenAI. Point CrewAI, LangGraph, OpenClaw, or AutoGen at that URL and they'll happily run against your local model. As of March 2026, llama.cpp merged full MCP client support — meaning you can plug MCP servers (GitHub, Notion, Supabase) into a local Qwen or Llama too.

# OpenClaw talking to local Ollama — no cloud, no real credentials # Pull once, then run against the local server at localhost:11434. ollama pull qwen3.5:8b openclaw config set backend local-ollama openclaw config set model qwen3.5:8b openclaw run "organize my Downloads folder" # All traffic stays on localhost. Nothing leaves the machine.Point OpenClaw at a local Ollama model. No cloud provider involved; no API key required.

You now have the full local stack: Ollama for the model, OpenClaw or your framework of choice for the agent, MCP for the tools. Next is the builder capstone — design (not code) an agent for your own life.

End-of-lesson check

6 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-agentic-ollama-basics-builders

What is the main idea of "Ollama Basics: Running a Model Yourself"?
1. Ollama turns 'I want to run an LLM locally' into a one-line install and a two-word command.
2. Use AI as the final authority for the whole decision
3. Avoid checking the answer once it sounds polished
4. Focus only on speed instead of judgment
Which concept is most central to "Ollama Basics: Running a Model Yourself"?
1. local models
2. Ollama
3. GGUF
4. llama.cpp
What should a careful learner remember about "LM Studio is the GUI cousin"?
1. Use AI to draft or organize ideas about Ollama, then verify before acting.
2. Skip the context so the tool can guess faster
3. Treat the output as private even after sharing it online
4. Use the answer without checking the source
You want to use AI after this lesson. What is the safest next step?
1. Act immediately because the AI answer is written clearly
2. Use the AI answer as a draft, then check it against a reliable source.
3. Hide uncertainty so the final answer looks cleaner
4. Use private or sensitive details before checking permission
How should AI output about Ollama be treated?
1. As proof that no other source is needed
2. As a replacement for context, consent, or expert review
3. As a draft or helper output that still needs human judgment and verification
4. As something that becomes correct when it sounds confident
Name one way to verify an AI answer about Ollama.

← Back to interactive lesson

Tendril · Builders · Agentic AI