Ollama API timeouts: diagnose before retrying requests

ollamaapitroubleshootingpython

TL;DR: An Ollama client timeout means the client stopped waiting; it does not prove the server stopped working. Send one short diagnostic request, distinguish HTTP errors from transport failures, and inspect server activity before retrying. Increase a timeout only after you know which part of the request is slow.

What you’ll be able to do:

  • Capture one Ollama request without an automatic retry loop.
  • Separate a server error, connection failure, and incomplete response.
  • Decide whether to adjust waiting time, reduce concurrency, or investigate model loading.

Honest take: Start with a single request and a visible error category; a retry loop is a poor first fix for an overloaded local server.

What does an Ollama timeout actually establish?

An Ollama timeout establishes a failure to obtain the expected response within a client or intermediary’s waiting policy. It does not identify the slow component. The request may have failed before reaching Ollama, waited behind other work, spent time loading a model, or started generating while a proxy stopped waiting.

Separate observations from explanations in your incident notes. “The client raised a timeout” is an observation. “The model needs more VRAM” is an explanation requiring additional evidence. Restarting the server before collecting that evidence removes useful information about the original request.

As checked on November 10, 2026, the Ollama generation endpoint supports a nonstreaming response through stream: false. That is convenient for a short diagnostic because the client expects one completed JSON document. It is a different response contract from consuming incremental streamed records; use the NDJSON parsing guide if your application intentionally streams output.

A nonstreaming request also makes a long wait more visible: the caller receives the completed result rather than a sequence of progress records. That does not mean streaming would make the model finish sooner. Treat response delivery and inference completion as separate questions.

Which failure should you investigate first?

Classify the failure before changing any settings. Keep the endpoint, model label, client environment, and approximate request time with the category. Use synthetic text while testing so the diagnostic record does not become another copy of a private document.

ObservationWhat you have establishedNext check
Connection refusedA connection could not be established at that addressHost, port, service, and container network
HTTP error responseAn HTTP server answered unsuccessfullyStatus and bounded error body
Waiting operation times outThe client did not finish that operation in timeServer logs, queue, loading, intermediary
HTTP response contains invalid JSONThe body does not match the selected response contractProxy response, truncation, endpoint
JSON has done: false or no completion flagCompletion has not been establishedResponse contract and server behavior
Completed response arrives slowlyThe request completed but latency is a problemLoading and generation timing separately

The Ollama error reference documents HTTP errors and an error field. A gateway can also generate an HTTP error, so a status code alone does not identify Ollama as the sender. Save a small, sanitized excerpt privately; do not paste a full diagnostic body into a public issue without reviewing it.

How do you make one bounded diagnostic request?

Use an already installed model and the same local server your application should reach. Save the following as probe_ollama_once.py. It requires Python 3’s standard library and deliberately performs no retries. The example accepts an endpoint argument for an isolated loopback test; use it only with a server you trust.

import argparse
import json
import time
import urllib.error
import urllib.request

parser = argparse.ArgumentParser()
parser.add_argument("model")
parser.add_argument("--url", default="http://127.0.0.1:11434/api/generate")
parser.add_argument("--timeout", type=float, default=30)
args = parser.parse_args()
if not args.model.strip() or args.timeout <= 0:
    parser.error("model must be nonempty and timeout must be positive")
limit = 1024 * 1024
payload = {"model": args.model, "prompt": "Reply with OK.",
           "stream": False, "options": {"num_predict": 16}}
request = urllib.request.Request(
    args.url, data=json.dumps(payload).encode("utf-8"),
    headers={"Content-Type": "application/json"}, method="POST")
# Direct connection for this local diagnostic; no environment proxy.
opener = urllib.request.build_opener(urllib.request.ProxyHandler({}))
started = time.monotonic()
try:
    with opener.open(request, timeout=args.timeout) as response:
        status = response.status
        raw = response.read(limit + 1)
except urllib.error.HTTPError as error:
    raise SystemExit(f"HTTP_ERROR status={error.code}; inspect server logs")
except urllib.error.URLError:
    raise SystemExit("TRANSPORT_ERROR; no HTTP completion established")
except (TimeoutError, OSError):
    raise SystemExit("IO_ERROR; no HTTP completion established")
if len(raw) > limit:
    raise SystemExit("BODY_TOO_LARGE; response not accepted")
try:
    result = json.loads(raw)
except (ValueError, UnicodeError):
    raise SystemExit("INVALID_JSON; response not accepted")
if not isinstance(result, dict):
    raise SystemExit("INVALID_SHAPE; expected an object")
if "error" in result:
    raise SystemExit("API_ERROR; inspect server logs")
if result.get("done") is not True:
    raise SystemExit("INCOMPLETE; done=true was not observed")
print(f"HTTP={status} done=true elapsed_s={time.monotonic()-started:.3f}")

Run it with the exact installed model label:

python3 probe_ollama_once.py YOUR_INSTALLED_MODEL --timeout 30

For a completed response, the output has the shape HTTP=200 done=true elapsed_s=...; the elapsed value comes from your request. The 30-second setting, 16-token output cap, and 1 MiB response cap are diagnostic policies chosen here, not Ollama service guarantees or measured performance figures. The script does not print generated text, prompts, or server error bodies.

Python documents the urlopen timeout in terms of blocking operations. This helper uses the same opener interface. It is not a hard total wall-clock deadline: connection handling and repeated reads can make that distinction relevant. A server that slowly keeps sending data can occupy the operation longer than an assumed overall budget. Use an appropriate supervised process or application deadline if your workflow needs a strict total limit.

What did the local test actually verify?

The helper was exercised on November 10 using a loopback HTTP fixture, without running Ollama or a language model. The fixture returned completed JSON, an incomplete object, malformed JSON, a JSON array, an API error object, an HTTP 503 response, and an oversized body. The client accepted only the completed object. A delayed fixture and a closed local port exercised failure handling as well.

Fixture responseExpected diagnostic
Object with done: trueHTTP completion line
Object with done: falseINCOMPLETE
HTML instead of JSONINVALID_JSON
JSON arrayINVALID_SHAPE
Object containing errorAPI_ERROR
HTTP 503HTTP_ERROR status=503
Body over the local capBODY_TOO_LARGE

These are client-control tests, not model benchmarks. They establish that the helper keeps response categories separate and issues one POST per invocation. They do not establish your GPU’s latency, the cancellation behavior of your installed Ollama version, or whether a particular model answers correctly. done: true means completion was observed; it does not grade the generated answer.

The chosen fixture includes deliberately unexpected responses because a local deployment can have more than one HTTP component. A reverse proxy’s HTML error page should not be mistaken for an Ollama JSON message just because both arrived on the expected URL.

Why can retrying make the problem harder to diagnose?

A new POST is another request. Without a verified cancellation or deduplication contract for the whole path, do not assume the previous operation disappeared when the client stopped waiting. Retrying while the first operation may still be active can add work and obscure which response belongs to which attempt.

This is an operational caution, not a claim that every disconnected Ollama request continues indefinitely. Disconnect handling depends on the endpoint, version, client, and intermediaries. The one-request helper intentionally avoids making a cancellation promise it cannot verify.

Record one attempt, inspect activity, and retry only after deciding why another attempt is useful. If multiple clients are involved, pause your own batch submission before comparing latency. A short synthetic prompt sent during a large background job measures a different situation from the same prompt on an idle server.

The Ollama FAQ describes request queueing and concurrency settings. Read the current defaults for your version before changing them. Increasing queue capacity gives more requests a place to wait; it is not evidence of more inference capacity. Change one setting at a time and preserve the previous configuration.

Should you raise the timeout or reduce the workload?

Raise waiting time when a legitimate operation completes successfully but your current client policy is shorter than the observed wait you are willing to tolerate. Reduce the workload when a short diagnostic works and your actual request fails under a larger prompt, longer output, or concurrent load. Neither response requires buying hardware first.

Use the server’s loading and generation metrics when available, alongside client elapsed time. Our Ollama timing breakdown explains why loading, prompt processing, and decoding should not be collapsed into one throughput number. A slow cold load and slow generation call for different fixes.

If the actual problem is memory capacity, estimate the workload with the VRAM calculator and then compare it with observed behavior. That calculator is a planning tool, not a measurement of the failing request. Do not turn a timeout message into a purchase recommendation without checking server logs and a smaller request first.

Honest take: when should you skip this helper?

Skip this helper for a production endpoint that requires authentication, a mandatory proxy, or a streaming-only contract. It deliberately bypasses environment proxies for a local diagnostic and has no credential management. Do not add a private bearer token to the article or a command you intend to share.

A successful local probe does not validate a browser application’s CORS configuration, a container’s network path, or a reverse proxy’s timeout. Run the next test from the actual caller’s environment and keep its endpoint explicit. The useful comparison is the same small request across the two paths, with the differing network layer documented.

FAQ

Q: Does HTTP 200 prove my Ollama task succeeded? A: No. Inspect the response contract and completion flag, then validate the task’s actual output. The helper checks a narrow completion condition and does not assess correctness.

Q: Is a timeout value a complete request deadline? A: Not in this helper. Python’s network timeout applies to blocking operations; use a separate overall deadline when the application needs one.

Q: Should I automatically retry every HTTP 503? A: No. First determine whether the server is busy and whether the prior attempt is still relevant. A controlled retry policy belongs after that diagnosis.

Sources

Last updated November 10, 2026. Official API behavior was checked on this date; local fixture tests did not run model inference or measure hardware performance.

Was this article helpful?

Get the numbers before you buy

New GPU and mini-PC benchmarks, VRAM thresholds, and price checks — sent when there's something worth acting on, not on a schedule. No spam, unsubscribe anytime.