Ollama: “model requires more system memory than is available”

Ollama refuses before it loads anything, with two numbers in parentheses. The refusal is a prediction, not an observation — and understanding which of the two numbers is wrong is the whole of the fix.

The message

The text, as it appears in the server log and as the client relays it:

Error: model requires more system memory (17.7 GiB) than is available (13.6 GiB)

Reported figures vary wildly across issue threads on the Ollama repository — 164.8 GiB against 13.4 GiB for a large model on a laptop, 446.3 GiB against a workstation for a 671B at Q4_K_M, and 69.0 GiB against 65.6 GiB for cases where the two numbers are almost the same. That last shape is the interesting one, because it is the one where the model plainly fits on disk and the reader concludes that Ollama is simply wrong.

It is not wrong, but it is not measuring what most people assume. The check happens before any weights are read. Ollama estimates what the run will need, compares it to what it believes is free, and declines rather than letting the operating system kill the runner mid-load. That is a better failure than the alternative — see what happens when the runner is killed instead — but it means both numbers are estimates and both can be moved.

What the first number counts

The requirement is not the size of the blob on disk. It is roughly four things added together:

• The weights, at whatever quantization the tag you pulled specifies. This is the number closest to the file size, and it is the part people expect.
• The KV cache, sized for the full context length the model or your num_ctx asks for. This is usually the term that surprises people, and it is the one you have the most control over.
• The compute graph — the scratch buffers for activations and logits during a forward pass. It scales with batch size and with vocabulary size, not with context.
• Headroom, so the runner is not sitting on the last free page of the machine.

The KV cache term is worth doing by hand once, because it explains why a 5 GB file asks for 17 GB. The cache stores a key and a value vector per layer per token, and its size is 2 × n_layers × n_kv_heads × head_dim × bytes per token. Llama 3 8B publishes 32 layers, 8 key/value heads under grouped-query attention, and a head dimension of 128. At 16-bit cache entries that is 2 × 32 × 8 × 128 × 2 = 131,072 bytes, or 128 KiB, for every single token of context. Multiply by an 8,192-token window and the cache alone is exactly 1 GiB; ask for 131,072 tokens and it is 16 GiB, on top of the weights. Nothing about that arithmetic is Ollama-specific — it is the same cache described in the general treatment of the KV cache, sized for your model’s own published config.

Run the same arithmetic for the model in front of you. A model with more layers, or without grouped-query attention (so n_kv_heads equals the full head count), produces a much larger per-token figure, and that is the usual reason two models of similar file size have very different requirements.

Where the second number comes from

The availability figure is read from the host, and on Linux it comes from the kernel’s notion of available memory rather than from total memory minus what you can see in a task manager. Three environments routinely make it read low:

• Containers. Inside Docker or Kubernetes the limit that matters is the cgroup limit, not the host’s RAM. A container started with a memory cap will report that cap, and the same ollama run that works on the host fails inside it.
• Filesystem caches that do not release quickly. ZFS in particular holds a large adaptive replacement cache, and while that memory is reclaimable, it is not always counted as available at the moment Ollama looks. This is a recurring report from NAS-style installs.
• A GPU that was expected to take the load. If the model cannot be fully offloaded, the remainder lands in system RAM, and it is the system-RAM shortfall that gets reported even though the real constraint was VRAM. The GPU-side version of this failure is the CUDA allocation error at load time.

To see which term dominates, start the server with debug logging enabled. It prints its estimate broken out by weights, cache and graph alongside what it believes is available, which turns a guess into a reading:

# stop the service first, then run the server in the foreground
OLLAMA_DEBUG=1 ollama serve

# in another shell
ollama run llama3.1:8b

The four levers, in order of effect

1. Cut the context. This is almost always the largest single win and the one people skip. Set it per request in the API, or interactively before the first prompt:

   # interactively
   /set parameter num_ctx 4096
   
   # via the API
   curl http://localhost:11434/api/generate -d '{"model":"llama3.1:8b","prompt":"hi","options":{"num_ctx":4096}}'

   Going from a 128K window to 4K on the Llama 3 8B numbers above takes the cache from 16 GiB to 512 MiB and changes nothing else about the model.
2. Pull a smaller quantization. The tag suffix selects it — llama3.1:8b-instruct-q4_K_M instead of an eight-bit or half-precision tag. Which level to accept is a quality decision, not a memory one; see how to choose a quantization level.
3. Quantize the cache itself. Ollama exposes an environment variable for the KV cache type, which halves or quarters the cache term at some cost in fidelity, and it generally requires flash attention to be enabled as well. Treat the variable names as version-specific and check the current documentation.
4. Fix the availability reading. Raise the container memory limit, or free the cache the host is holding. This is the only lever that helps when the two numbers are close together.

When the numbers look wrong

There is a known class of report where the requirement is inflated because memory-mapped loading is disabled or unavailable. With mmap, weights are paged in from the file and the kernel can evict them again; without it, the whole file is read into anonymous memory and must fit. If your requirement is close to weights-plus-cache and you have disabled mmap — directly, or by using an option that forces a full read — re-enabling it changes the arithmetic rather than the hardware.

The other genuine mismatch is a multi-GPU machine where the estimate only accounts for one device. If the numbers make no sense against hardware you can see, check the debug log for how many devices the scheduler enumerated before you go shopping. More generally, the decision of what to keep local at all is covered in the self-host versus API trade-off and the general Ollama guide.