Ollama: The Complete Guide to Running LLMs Locally in 2026

Ollama is a free, open-source tool that downloads and runs large language models on your own computer, so your prompts and data never leave your machine unless you choose a cloud model. It installs with one command on macOS, Windows, and Linux, serves every model through a local API on port 11434, and as of July 2026 its GitHub repository holds 175,608 stars, making it the default way most people run models like Qwen, Gemma, DeepSeek, and gpt-oss locally.

What Is Ollama?

Ollama is an open-source model runner that packages a large language model, its weights, and its configuration into a single artifact you can download and run with one command. It was built on top of the llama.cpp inference engine, is released under the MIT license, and ships as both a desktop app and a command-line tool for macOS, Windows, and Linux.

The project moves fast. As of July 7, 2026, the ollama/ollama repository sits at 175,608 stars and 16,860 forks, and the latest release, v0.31.1, landed on June 30, 2026. The official model library at ollama.com/library covers the major open-weight families: Qwen, Gemma, DeepSeek, gpt-oss, GLM, Kimi, MiniMax, and hundreds more.

Three things explain why Ollama beat heavier alternatives to become the default local runner. First, install and first model run take minutes, not hours. Second, every model is served through a consistent local API, so tools built against Ollama work with any model you pull. Third, since 2025 it has leaned into agent integrations, letting you connect local models to OpenClaw, Claude Code, Codex, Copilot CLI, Droid, and OpenCode from the CLI itself.

How to Install Ollama

Ollama installs with a single shell command on macOS and Linux, a PowerShell one-liner on Windows, or a standard installer download from ollama.com/download. All three platforms get the same CLI and the same local API.

macOS

curl -fsSL https://ollama.com/install.sh | sh

Or download the Ollama.dmg installer directly. Ollama requires macOS 14 Sonoma or later, and Apple Silicon Macs are the sweet spot because model weights load into unified memory.

Windows

irm https://ollama.com/install.ps1 | iex

Run that in PowerShell, or download OllamaSetup.exe from the download page if you prefer a clickable installer.

Linux

curl -fsSL https://ollama.com/install.sh | sh

The script detects NVIDIA and AMD GPUs and configures Ollama as a systemd service. Manual install instructions live in the official docs if you want to place binaries yourself.

Docker

The official ollama/ollama image on Docker Hub is the standard route for servers and homelab setups, and it is how most people run Ollama on a VPS next to an agent runtime.

Pulling and Running Models

Running a model in Ollama is one command: ollama run gemma4 downloads Gemma 4 on first run and drops you into an interactive chat. Typing plain ollama opens an interactive menu that prompts you to run a model or connect Ollama to an existing agent or app.

# download and chat with a model
ollama run gemma4

# download without running
ollama pull qwen3-coder:30b

# see what is installed and what is loaded in memory
ollama list
ollama ps

The commands you will actually use day to day:

Model tags matter. qwen3.5:9b and qwen3.5:27b are different downloads of the same family, and appending :cloud (for example ollama run gemma4:cloud) runs the model on Ollama's hosted infrastructure instead of your hardware, which is the escape hatch when a model is bigger than your machine.

Hardware Requirements

Ollama's practical hardware floor is set by one rule: the model's download size is roughly the minimum memory it needs, plus headroom for context. A Q4-quantized 9B model is a 5 to 6GB download and runs comfortably in 12 to 16GB of VRAM or Apple unified memory; 27 to 30B models want 24GB or more for usable context lengths.

• Apple Silicon Macs: the easiest path. 16GB of unified memory handles 7 to 9B models well; 32GB and up opens 27 to 30B models.
• NVIDIA GPUs: a used RTX 3090 or an RTX 4090 with 24GB VRAM fits most agent-suitable models at Q4 with 32 to 64K context.
• CPU only: works, but token speed drops enough that interactive agent use gets painful beyond small models.

Context length is the hidden memory cost people miss: the KV cache grows with your context window, so a model that fits at 4K context can overflow at 64K. Our GPU optimization guide for Ollama covers quantization levels, context sizing, and specific GPU recommendations in depth.

Using Ollama With AI Agents

Ollama connects to coding and assistant agents two ways: the built-in ollama launch command, or its OpenAI-compatible API. Since the launch integrations shipped, wiring a local model into an agent went from a config-file exercise to a single command.

# turn Ollama into a personal AI assistant via OpenClaw
ollama launch openclaw

# use a local model inside Claude Code
ollama launch claude

Supported integrations as of July 2026 include OpenClaw, Claude Code, Codex, Copilot CLI, Droid, and OpenCode. For OpenClaw specifically, our OpenClaw with Ollama setup guide walks through the full configuration, including model selection and memory settings.

For everything else, Ollama exposes an OpenAI-compatible endpoint at http://localhost:11434/v1/. Any tool that accepts a custom OpenAI base URL can use local models; the API key is required by client libraries but ignored by Ollama, so the convention is to pass the string ollama:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:11434/v1/", api_key="ollama")
resp = client.chat.completions.create(
    model="qwen3-coder:30b",
    messages=[{"role": "user", "content": "Explain this stack trace"}],
)

The compatibility layer supports chat completions, streaming, JSON mode, vision, tools, and reasoning, though it does not yet support logprobs or the tool_choice parameter. There is also a native REST API at /api/chat plus official Python (pip install ollama) and JavaScript (npm i ollama) libraries.

The Best Models to Run on Ollama

The best model to run on Ollama in mid-2026 depends on the job: qwen3-coder:30b for serious coding work, qwen3.5:9b when hardware is tight, and glm-4.7-flash as the fast generalist. Those picks, with the reasoning and hardware pairings behind them, live in our full best Ollama models for 2026 breakdown.

If your endgame is an always-on agent rather than a chat window, model choice changes: agents punish weak tool calling and short context harder than chat does. Our best Ollama models for OpenClaw guide ranks models specifically for agent workloads, and the Gemma 4 free setup guide is the fastest zero-cost starting point.

Ollama vs Cloud APIs

Ollama trades raw model quality for privacy, price, and control: a local 30B model still loses to frontier cloud models on hard reasoning, but it costs nothing per token, works offline, and never sends your data anywhere. The decision is a workload question, not a loyalty question.

Most operators land on a hybrid: a local model for high-volume, low-stakes work and a cloud model for hard tasks. We compared the three routing options for agents in Ollama vs OpenRouter vs local models for OpenClaw, and Ollama's own :cloud tags now blur the line further by serving big open models through the same local API.

Limitations and Tradeoffs

Ollama is not the right tool for every job, and three limitations come up repeatedly. First, quality ceiling: nothing you can run on a 24GB GPU matches frontier cloud models on complex, multi-step reasoning, so agents doing high-stakes work usually still need a cloud fallback. Second, quantized defaults: library models default to Q4 quantization, which trims quality versus full-precision weights; that tradeoff is invisible in chat and occasionally visible in exacting code generation. Third, concurrency: Ollama is built for one user on one machine, not for serving many parallel users, where dedicated inference servers like vLLM are the standard answer.

Also be precise about privacy claims: local models are fully private, but :cloud models run on Ollama's hosted infrastructure, so treat them like any other cloud API for sensitive data.

Related Guides

• Best Ollama Models in 2026: Recommendations by Speed, Context, and Hardware
• Best Ollama Models for OpenClaw: What to Run and Why
• Ollama vs OpenRouter vs Local Models for OpenClaw
• GPU Optimization Guide for Ollama Models in OpenClaw