TL;DR: Calculate generation speed from eval_count and eval_duration, then inspect loading and prompt evaluation separately. A slow first response does not prove your GPU is slow. Save the final native API response and compare the same model, prompt, and settings before spending money.

What you’ll be able to do:

• Distinguish model loading, prompt processing, and output generation.
• Calculate rates without confusing nanoseconds with seconds or cached input with fresh work.
• Decide whether to change the workload, investigate residency, or benchmark another machine.

Honest take: Diagnose the slow phase before shopping for a faster card.

Which Ollama speed number should you trust?

Ollama’s native API exposes separate counts and durations for different phases. The official usage documentation, checked November 7, 2026, defines those fields and identifies their timing unit as nanoseconds. The useful generation calculation is therefore:

output tokens per second = eval_count / (eval_duration / 1_000_000_000)

That rate describes output generation inside the reported evaluation interval. It is not your application’s full response latency, a guarantee about answer quality, or a measurement of how quickly the browser renders text. Keep those questions separate when deciding what to optimize.

Question	Relevant evidence	Common mistake
Why is the first request slow?	load_duration, residency before the call	Treating loading time as slow decoding
Why does a long document delay the answer?	Prompt count, cache count, prompt duration	Comparing unlike prompt lengths
How quickly are output tokens generated?	eval_count and eval_duration	Dividing by total request duration
Why does the interface feel slow?	Client elapsed time and streaming behavior	Assuming server metrics include every UI delay

A model that answers briefly can have good generation speed while still feeling sluggish because loading dominates the request. Conversely, a warm model can start quickly and then generate slowly. Those are different problems with different fixes.

How do you save one complete response?

Use a native non-streaming request for the first diagnosis. The generate endpoint documents the request and response contract. Choose an installed local model from your own inventory; the placeholder below is deliberately not a model recommendation.

ollama list
ollama --version
# Replace YOUR_INSTALLED_MODEL with an exact local model name.
curl --fail-with-body --silent --show-error \
  http://localhost:11434/api/generate \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "YOUR_INSTALLED_MODEL",
    "prompt": "Explain why a cold model request can be slow.",
    "stream": false,
    "keep_alive": "5m",
    "options": {"num_predict": 128}
  }' > response.json

The expected shape is one JSON object containing the answer, a completed-response marker, and usage metrics. The output limit above is a chosen diagnostic setting, not a measured model capability. Do not substitute an invented response for the file returned by your own server.

If the request fails, inspect the HTTP error before calculating speed. A saved error object cannot tell you how quickly a model generated. The wrong-server and model-tag checklist is a better next step when the server cannot find the requested model.

Keep this first capture outside a public repository if its prompt or answer contains private material. Timing analysis needs the counts and durations; it rarely needs the entire confidential document that produced them.

How can you calculate rates without accepting broken data?

Validate the completed response and the metric types before dividing. Save this standard-library program as ollama_metrics.py:

import json
import math
import sys


def number(data, key):
    value = data.get(key)
    if (isinstance(value, bool)
            or not isinstance(value, (int, float))
            or not math.isfinite(value)
            or value < 0):
        raise ValueError(f"invalid or missing metric: {key}")
    return value


def summarize(data):
    if data.get("error"):
        raise ValueError("server returned an error")
    if data.get("done") is not True:
        raise ValueError("expected a completed native API response")
    count = number(data, "eval_count")
    duration = number(data, "eval_duration")
    if duration == 0:
        raise ValueError("eval_duration is zero; no usable decode rate")
    total = number(data, "total_duration")
    load = number(data, "load_duration")
    prompt = number(data, "prompt_eval_count")
    prompt_ns = number(data, "prompt_eval_duration")
    cached = data.get("prompt_eval_cached_count")
    result = {
        "decode_tps": round(count * 1_000_000_000 / duration, 2),
        "total_seconds": round(total / 1_000_000_000, 3),
        "load_seconds": round(load / 1_000_000_000, 3),
        "prompt_seconds": round(prompt_ns / 1_000_000_000, 3),
        "prompt_tokens": prompt,
    }
    if cached is None:
        result["uncached_prompt_tps"] = None
        result["cache_note"] = "cache count absent; rate not calculated"
    else:
        cached = number(data, "prompt_eval_cached_count")
        if cached > prompt:
            raise ValueError("cached prompt count exceeds total prompt count")
        fresh = prompt - cached
        result["uncached_prompt_tps"] = (
            round(fresh * 1_000_000_000 / prompt_ns, 2)
            if fresh > 0 and prompt_ns > 0 else None
        )
    return result


if __name__ == "__main__":
    try:
        with open(sys.argv[1], encoding="utf-8") as handle:
            print(json.dumps(summarize(json.load(handle)), indent=2))
    except (ValueError, OSError, IndexError) as exc:
        raise SystemExit(str(exc))

Run python ollama_metrics.py response.json. A failure such as invalid or missing metric: eval_duration means the input does not support this calculation. It does not mean the GPU produced zero tokens per second. Check whether you saved the native endpoint response, an intermediate chunk, a client wrapper, or an error.

This helper intentionally leaves a prompt rate unavailable when the cached-token field is absent. Guessing that the missing field means zero cached tokens would create a precise-looking number from an unknown input. Older responses and compatibility layers may expose a different contract; preserve that uncertainty.

What did we actually test here?

We tested the parser and arithmetic with constructed response objects on November 7, 2026. We did not run model inference or benchmark a GPU for this article. The following is an arithmetic fixture, not a claimed Ollama performance result:

{
  "done": true,
  "eval_count": 100,
  "eval_duration": 2000000000,
  "total_duration": 5000000000,
  "load_duration": 1000000000,
  "prompt_eval_count": 300,
  "prompt_eval_cached_count": 100,
  "prompt_eval_duration": 1000000000
}

Save it as fixture.json. The helper returns decode_tps: 50.0, total_seconds: 5.0, load_seconds: 1.0, and uncached_prompt_tps: 200.0. Output count divided by total time would instead produce 20 tokens per second. Both calculations can be arithmetically correct, but they answer different questions.

Our local checks also rejected a missing duration, a zero decode duration, a Boolean masquerading as a count, an unfinished response, and a cached count larger than the total prompt count. These tests establish the helper’s behavior. They establish nothing about a specific model’s quality, memory footprint, or speed.

Why can prompt processing look impossibly fast?

Prompt caching changes how much fresh work happens during prompt evaluation. The current usage reference distinguishes total prompt tokens, cached prompt tokens, and the duration for uncached prompt evaluation. Dividing the entire prompt count by that duration can exaggerate fresh-input throughput when much of the prompt was reused.

Record both counts when the server supplies them. If your question is how fast a fresh document can be processed, compare fresh input under a documented cache condition. If your actual workload repeatedly reuses a long system prompt, warm-cache latency is useful too. It simply belongs in a separate result column.

Do not concatenate the same test text repeatedly and call the result a hardware limit. That experiment changes prompt length, repetition, cache opportunity, and potentially model behavior at once. A useful comparison changes one relevant variable while keeping the rest visible in the record.

Does a streaming response need different handling?

Ollama streams responses by default, and its streaming documentation describes newline-delimited response objects. Usage metrics belong to the final completed object, rather than every text fragment. A measurement tool must retain that final object and distinguish it from partial output.

For this tutorial, stream: false avoids that collection problem. It does not necessarily make inference faster. It changes how the result is delivered to your client. If your application requires streaming, use the NDJSON parsing guide and feed the completed object into the same calculation.

An interrupted connection can leave visible answer text without final metrics. Report that request as incomplete for this analysis. Do not fill the missing duration with a previous request’s value, and do not silently drop failed requests when evaluating reliability.

What should you compare before replacing hardware?

Start with a cold-versus-warm record for the same model. The Ollama FAQ documents model residency controls, while the running-model endpoint exposes what is currently loaded. Residency is evidence about the server’s state, not proof that every request will use the same execution path.

Record the exact model tag, available digest, Ollama version, endpoint, prompt, output count, context settings, concurrency, and whether another job was running. Keep a client stopwatch alongside the server metrics if user-facing latency is the actual complaint. A model update or a different quantization makes yesterday’s result a different experiment.

Dominant symptom	First investigation	Purchase decision to postpone
Large loading interval on occasional requests	Residency and repeated model switching	Buying a GPU solely for cold starts
Long fresh-prompt evaluation	Input length and preprocessing	Assuming decode bandwidth explains the delay
Slow generation with warm residency	Placement, workload settings, comparable measurements	Buying more capacity without checking the workload
Fast server metrics, slow interface	Client buffering and application path	Replacing server hardware based on UI delay alone

When the evidence does reveal a capacity problem, use the VRAM calculator to organize the memory question. Capacity and throughput are related constraints, but extra memory alone does not establish how fast your chosen task will finish.

Honest verdict: when is this method insufficient?

Native timing fields are an inexpensive first diagnosis for a home lab that already runs Ollama. The helper needs only Python and a small JSON file; analyzing the file requires no GPU or paid API. Collecting a real inference response still uses your existing machine’s resources and electricity, which this article has not measured.

Use a broader benchmark when you need repeated trials, queueing behavior under load, energy measurements, or a quality comparison between models. Server-reported generation speed cannot tell you whether the answer is correct, whether a tool call succeeds, or whether users finish their work sooner. Keep the diagnostic focused: identify the slow phase, verify the same workload, then decide whether a hardware comparison is justified.

FAQ

Q: Can I use the helper with /api/chat? A: The native chat endpoint documents the same core usage fields. Supply its completed response and verify which optional cache fields your server returns.

Q: Is total request time useless? A: No. Total latency matters to users. Keep it as its own measurement rather than labeling output count divided by total time as pure decode speed.

Q: Does the fixture prove 50 tokens per second on this server? A: No. The fixture is constructed data used to test arithmetic. No inference benchmark is claimed.

Sources

• Usage metrics and timing units — Ollama
• Generate endpoint — Ollama
• Chat endpoint — Ollama
• Streaming response format — Ollama
• List running models — Ollama
• Model residency and configuration FAQ — Ollama

Last updated November 7, 2026. Official API contracts checked today; local tests cover the parser and synthetic arithmetic, not model inference or hardware performance. Verify your installed server’s response fields before comparing results.

Get the numbers before you buy

New GPU and mini-PC benchmarks, VRAM thresholds, and price checks — sent when there's something worth acting on, not on a schedule. No spam, unsubscribe anytime.

Email address Subscribe