TL;DR: If Ollama unloads between requests, check the server’s keep-alive policy before buying hardware. A longer keep-alive can reduce repeated loading, but it still requires enough memory. Confirm residency with ollama ps; prompt processing and generation can remain slow even when the model stays loaded.

What you’ll be able to do after this guide:

• Request continued model residency (indefinitely or for a set window) to avoid repeated model-loading work
• Read ollama ps to confirm what’s actually loaded and when it’ll unload
• Stop two models from fighting over VRAM and thrashing each other out of memory

Honest take: Check whether the model unloads between requests before changing hardware. Keep-alive can reduce repeated loading, but it does not remove prompt processing, generation time, or memory-capacity limits.

What’s actually happening

By default, Ollama keeps a model loaded in memory for 5 minutes after its last request, then unloads it and frees the VRAM (Ollama FAQ). That idle timeout is deliberate: it returns GPU memory to the system so other workloads (or other models) can use it.

A request after the model unloads includes loading work before generation. Measure that work on your own server using the API response’s load_duration, documented in Ollama usage metrics, rather than borrowing cold-start timings from another machine. Keep the model, context, and prompt the same when comparing a cold request with a resident request. This separates loading time from prompt evaluation and token generation; keeping weights resident does not make those other stages disappear.

So before you blame your GPU, confirm the symptom. Open two terminals. In one, run your model. In the other, watch what’s resident: The CLI tables below are illustrative layouts, not measurements from this update; your model, size, digest, and remaining time will differ.

$ ollama ps
NAME               ID              SIZE      PROCESSOR    UNTIL
llama3.1:8b        42182419e950    6.7 GB    100% GPU     4 minutes from now

Three columns matter here:

• PROCESSOR — 100% GPU means the model was loaded entirely into the GPU; mixed CPU/GPU percentages report split placement. Placement alone is not a speed benchmark. Investigate an unexpected CPU-only or mixed result with the Ollama GPU troubleshooting guide.
• SIZE — how much memory this model is holding.
• UNTIL — the countdown to unload. 4 minutes from now is the default 5-minute timer ticking down. This is the column that explains your cold starts.

If ollama ps shows no loaded models, none was reported resident at that moment. A later request may need to load the model. That can be normal idle behavior, not a server bug.

Fix 1: Set OLLAMA_KEEP_ALIVE on the service (the real fix)

OLLAMA_KEEP_ALIVE controls how long a model stays resident after its last request. It accepts (Ollama FAQ):

• A duration string: "10m", "24h"
• A number in seconds: 3600
• Any negative value to keep it loaded forever: -1
• 0 to unload immediately after each response

The trap that wastes the most time: setting it in your shell does nothing. Ollama usually runs as a background service (systemd on Linux, a launch agent on macOS, a tray app on Windows) with its own environment. Exporting OLLAMA_KEEP_ALIVE=-1 in .bashrc is invisible to that service (Ollama server configuration). You have to set it where the service can see it.

Linux (systemd):

sudo systemctl edit ollama.service

Add, under the [Service] section:

[Service]
Environment="OLLAMA_KEEP_ALIVE=-1"

Then reload and restart:

sudo systemctl daemon-reload
sudo systemctl restart ollama

-1 pins the model in VRAM permanently — ideal for a dedicated home AI box that only ever runs one model. If you’d rather it free memory overnight, use "8h" instead.

macOS: set it for the launch environment, then restart the Ollama app:

launchctl setenv OLLAMA_KEEP_ALIVE "-1"

Windows: quit Ollama from the system tray, open Settings → System → About → Advanced system settings → Environment Variables, add a user variable OLLAMA_KEEP_ALIVE with value -1, then relaunch Ollama.

Verify it took effect — UNTIL should now read Forever:

$ ollama ps
NAME               ID              SIZE      PROCESSOR    UNTIL
llama3.1:8b        42182419e950    6.7 GB    100% GPU     Forever

Fix 2: Override per request with keep_alive

If you don’t want a global policy — say a script that should load a big model, do one batch job, and release the VRAM — pass keep_alive directly in the API call. The request-level parameter overrides the OLLAMA_KEEP_ALIVE environment variable for that call (Ollama FAQ).

Keep a model loaded for this session:

curl http://localhost:11434/api/generate -d '{
  "model": "llama3.1:8b",
  "prompt": "Summarize this changelog...",
  "keep_alive": -1
}'

Unload immediately when the job is done (frees VRAM the instant the response finishes):

curl http://localhost:11434/api/generate -d '{
  "model": "llama3.1:8b",
  "keep_alive": 0
}'

Sending a request with an empty prompt just loads (or unloads) the model without generating — handy for preloading.

Fix 3: Preload before the user shows up

If your real complaint is “the first request of the day is slow,” preload the model at boot so the cold start happens before anyone is waiting. The cleanest way is an empty generate request that pins the model:

ollama run llama3.1:8b ""

Or via the API in a startup script / systemd unit:

curl http://localhost:11434/api/generate -d '{"model": "llama3.1:8b", "keep_alive": -1}'

If you intentionally automate preloading, run it after the Ollama service is ready and check the response for success. An empty generate request can load a model before an interactive request, as documented in the generate API. A negative keep-alive requests continued residency; it does not guarantee instantaneous responses or prevent eviction under competing memory demand. Verify actual residency after the preload rather than treating a startup command as proof.

Fix 4: Stop two models from thrashing

A subtler version of the same problem: you switch between, say, a coding model and a general chat model, and every switch is slow. That’s not the keep-alive timer — it’s model swapping.

Two environment variables govern this (Ollama FAQ, envconfig source):

Variable	Default	What it does
OLLAMA_MAX_LOADED_MODELS	3 × GPU count (3 for CPU)	How many distinct models can be resident at once
OLLAMA_NUM_PARALLEL	1 (official FAQ and source checked November 11, 2026)	Maximum parallel requests per model; memory requirements also depend on configured context

Concurrent residency is constrained by available memory, not just the configured model-count limit (Ollama FAQ). Parameter count alone cannot establish whether two models fit together: quantization, context, parallelism, and other workloads also matter. Capture the loaded-model report while reproducing your normal switching pattern. An eviction followed by a new load is evidence of reloading; an arbitrary pair of model names is not a measured capacity test.

Practical rules:

• Plenty of VRAM, a few small models you alternate between? Raise OLLAMA_MAX_LOADED_MODELS and let them all stay resident.
• Tight on VRAM? Leave it at default and standardize on one model. Forcing two big models into a card that can’t hold both just guarantees thrashing.
• Watch the KV cache. Bumping OLLAMA_NUM_PARALLEL multiplies KV-cache VRAM by the context length, which can quietly push you into a CUDA out-of-memory error. If you raised parallelism and things got less stable, lower it back.

Check the actual loaded allocation before raising any of these settings. Use the disk-size versus runtime-memory diagnostic to keep those measurements separate, and the system RAM and VRAM planning guide for the broader hardware decision. Do not assume that a larger configured residency limit creates additional memory.

Honest take: when keep-alive is the wrong lever

Keeping a model resident holds memory that another workload might need, including your display or image-generation tools. For a service with concurrent users, residency is only one control: queueing, context allocation, and serving-engine scheduling also matter. Compare the actual workload before changing engines; the vLLM versus Ollama guide discusses that separate decision. A longer keep-alive is not evidence of higher throughput.

For a personal server whose problem is confirmed idle unloading, a suitable keep-alive and a verified preload can address repeated loading. If the model fails to fit or the backend is misconfigured, resolve that independently. Building a local coding setup where cold starts ruin the autocomplete experience? The same keep-alive tuning applies — pair it with the editor integration in our sister site’s local AI coding tool guides on aicoderscope.com.

FAQ

Why does Ollama unload my model after 5 minutes? That’s the default OLLAMA_KEEP_ALIVE value. After 5 minutes with no requests, Ollama frees the VRAM so other workloads can use it. Set OLLAMA_KEEP_ALIVE=-1 on the service to keep it loaded indefinitely.

I set OLLAMA_KEEP_ALIVE in my terminal and nothing changed. Why? Ollama runs as a background service with its own environment. A variable exported in your shell isn’t visible to it. On Linux use systemctl edit ollama.service; on macOS use launchctl setenv; on Windows set a user environment variable and restart the app.

How do I check whether the model is currently in VRAM? Run ollama ps. The UNTIL column reports the residency deadline, and PROCESSOR reports CPU/GPU placement. Full GPU placement does not establish a particular token rate.

How do I free the VRAM right now without restarting Ollama? Send a request with "keep_alive": 0, or ollama stop <model>. The model unloads immediately.

Does keeping a model loaded slow down my GPU for other tasks? It reserves that VRAM for as long as the model is pinned. If you also game or run image generation on the same card, use a finite timeout like "30m" instead of -1 so the memory frees when idle.

Sources

• Runtime configuration and concurrency defaults — Ollama FAQ
• Environment defaults — Ollama source
• Preload and keep-alive request fields — Ollama generate API
• Load duration and other timings — Ollama usage metrics
• Loaded-model inventory — Ollama ps API
• Context allocation and memory — Ollama documentation

Last updated November 11, 2026. Parallelism default and residency guidance were rechecked against official documentation and source. Removed unsupported timing and instant-response claims; no new inference benchmark was performed.

Get the numbers before you buy

New GPU and mini-PC benchmarks, VRAM thresholds, and price checks — sent when there's something worth acting on, not on a schedule. No spam, unsubscribe anytime.

Email address Subscribe