Local LLM Hardware Checker

How this works

The checker first works out how much memory your machine can actually dedicate to model weights. A discrete GPU can use its full VRAM, so an RTX 4090 with 24 GB gives you 24 GB of usable model memory. Apple silicon shares one pool of unified memory between the system and the GPU, so macOS realistically lets a model use about 75% of total RAM. A CPU-only machine still needs memory for the operating system and your apps, so roughly 60% of RAM is usable, and generation speed will be much slower without a GPU.

That usable number is then compared against approximate Q4 quantization sizes for popular Ollama-runnable models, from tiny 2 GB models like Llama 3.2 3B up to roughly 65 GB for GPT-OSS 120B. Q4 is the default quantization level Ollama ships for most models and is the sweet spot between quality and memory use.

A model gets a "Runs well" verdict when it fits inside 70% of your usable memory, which leaves headroom for the KV cache that grows with your context window. "Tight" means the weights technically fit but you will be limited to short contexts and may see swapping. "Won't fit" means the weights alone exceed your usable memory, so the model will either fail to load or crawl while paging to disk.

Frequently asked questions

How much VRAM do I need to run a 70B model locally?

At Q4 quantization, 70B models like Llama 3.3 70B and DeepSeek-R1 70B weigh roughly 40 GB. That means no single consumer GPU runs them comfortably today, including the RTX 5090 at 32 GB. In practice you need a Mac with 64 GB or more of unified memory, a multi-GPU rig, or a workstation card. On a 64 GB Mac they run but are tight, and 128 GB gives them real headroom.

Why does Apple silicon only get 75% of its RAM?

Apple silicon uses unified memory shared between the CPU and GPU. macOS reserves a portion for the system and caps how much the GPU can allocate, so a 36 GB machine gives a model roughly 27 GB to work with. It is still one of the best ways to run large models locally because unified memory is cheaper per GB than stacking GPU VRAM.

Can I run local LLMs without a GPU at all?

Yes. Ollama and llama.cpp both run models on CPU, and small models in the 3B to 8B range are genuinely usable this way. The catch is speed: CPU inference typically produces single-digit tokens per second on mid-size models, so anything above roughly 14B parameters becomes frustrating for interactive use.

What does Q4 quantization mean?

Quantization compresses model weights from 16-bit precision down to about 4 bits per parameter, cutting memory use to roughly a quarter with only a small quality loss. It is the default format Ollama serves for most models, which is why this tool uses Q4 sizes. Higher precision variants like Q8 exist but roughly double the memory requirement.

My hardware is too weak. What are my options?

You do not need local hardware to run an AI agent. Most people run the agent on a cheap VPS and point it at a cloud API like Claude, which is far cheaper than buying a GPU and works from any laptop. Use the AI Agent VPS Calculator to size a server, or stick to the small models that got a "Runs well" verdict above.