TL;DR: Ollama’s REST API expects encoded image bytes, while its SDK examples can accept local file paths. Build and inspect a request locally before changing models or buying hardware. A valid base64 round trip proves transport encoding, not image validity or successful vision inference.

What you’ll be able to do:

• Build a native chat request from an image on your own computer.
• Check that the encoded field recovers the original file exactly.
• Separate request mistakes from model capability, decoding, and memory problems.

Honest take: Fix the request boundary first; a larger GPU cannot turn a filename in JSON into image bytes.

Why does a working SDK example fail when copied into REST JSON?

The SDK and REST interfaces accept different image representations. The official vision documentation, checked November 9, 2026, explicitly distinguishes SDK convenience inputs from the base64 data expected by REST. A path that makes sense to a Python client is just text when sent directly in a JSON request.

Suppose a desktop application sends "images": ["/home/me/photo.png"] to an Ollama server. The server has not received that file. Even if both programs run on the same computer, the request has crossed an HTTP boundary rather than invoking a filesystem reader in your process. If the server lives in a container or on another machine, there may not even be a matching path in its filesystem.

Use the representation supported by the interface you actually call. This article uses native REST requests and Python’s standard library, avoiding any dependency on an SDK’s preprocessing behavior. The examples do not call a paid service. The local payload checks require no GPU, installed model, or running Ollama server. Actual image interpretation still requires a compatible model and sufficient resources on your inference machine.

Where does the images array belong?

For a native chat request, put images on the user message. For a native generate request, put images at the request’s top level. The vision example and generate reference show those respective layouts.

Interface	Text field	Image field	What to check
Native /api/chat	messages[].content	messages[].images	Image belongs to the intended user message
Native /api/generate	prompt	Top-level images	Image and prompt are in the same request
SDK call	Client-specific arguments	Client may process paths or bytes	Follow that SDK’s documentation

Do not mix a native Ollama example with an OpenAI-compatible request copied from another application. Similar endpoint names do not guarantee identical message schemas. Keep one known request shape during diagnosis, then adapt it to your application after it works.

A minimal native chat body has this structure. The marker below is explanatory text, not a usable encoded image:

{
  "model": "YOUR_INSTALLED_VISION_MODEL",
  "messages": [
    {
      "role": "user",
      "content": "Describe the visible objects.",
      "images": ["BASE64_IMAGE_BYTES_GO_HERE"]
    }
  ],
  "stream": false
}

The model marker must be replaced with a real installed tag that supports image input. It is not a recommendation to download a particular model. If that tag is missing, use the separate model-not-found diagnostic before debugging vision behavior.

How do you build a request without shell quoting problems?

Read the local file as bytes, encode it once, and let a JSON serializer create the body. This avoids putting a large encoded string into shell interpolation, where line wrapping and quoting can obscure the original problem.

Save this as build_vision_request.py. It writes a request file but sends nothing. The file-size limit is an example policy of 8 MiB, chosen to bound this helper’s local memory use; it is not an Ollama upload limit or a recommended model limit.

import base64
import json
import sys
from pathlib import Path

MAX_FILE_BYTES = 8 * 1024 * 1024  # Tutorial policy, not an API limit.


def make_request(image_path, model):
    image = Path(image_path)
    if not model.strip():
        raise ValueError("model tag is empty")
    with image.open("rb") as handle:
        raw = handle.read(MAX_FILE_BYTES + 1)
    if not raw:
        raise ValueError("image file is empty")
    if len(raw) > MAX_FILE_BYTES:
        raise ValueError("image exceeds this helper's file-size policy")
    encoded = base64.b64encode(raw).decode("ascii")
    recovered = base64.b64decode(encoded, validate=True)
    if recovered != raw:
        raise ValueError("base64 round trip changed the bytes")
    body = {
        "model": model,
        "messages": [{
            "role": "user",
            "content": "Describe the visible objects.",
            "images": [encoded],
        }],
        "stream": False,
    }
    return body, len(raw)


if __name__ == "__main__":
    if len(sys.argv) != 3:
        raise SystemExit("usage: build_vision_request.py IMAGE MODEL")
    body, byte_count = make_request(sys.argv[1], sys.argv[2])
    output = Path("vision-request.json")
    if output.exists():
        raise SystemExit("refusing to replace vision-request.json")
    with output.open("x", encoding="utf-8") as handle:
        json.dump(body, handle)
    print(f"wrote {output}; input bytes={byte_count}")

Run it from a private scratch directory containing an image you are permitted to process:

python3 build_vision_request.py photo.png YOUR_INSTALLED_VISION_MODEL

The success line begins wrote vision-request.json; input bytes= and ends with your actual file size. A missing file raises FileNotFoundError; an empty file raises ValueError: image file is empty. Fix the named file problem before attempting inference. The helper deliberately refuses to overwrite an existing request so that reruns do not quietly replace evidence from an earlier attempt.

Keep the generated JSON private. Base64 is reversible encoding, not redaction or encryption. A screenshot containing a password or a customer record remains sensitive after encoding. Avoid pasting the file into a public issue or committing it alongside application code.

What does the local check actually prove?

The helper proves that its encoded value can be decoded back to the file it read. Python documents both the encoding functions and strict decoding behavior in its base64 reference. That is a narrow but useful property: it catches mistakes introduced between reading bytes and constructing a request.

On November 9, 2026, we ran the article’s helper against a locally generated PNG and verified byte-for-byte recovery after JSON serialization. We also checked missing, empty, and oversized input rejection. These were CPU-only file and encoding tests. We did not run a vision model or measure GPU memory, inference speed, recognition accuracy, or maximum supported image size.

A text file can also survive a base64 round trip. Therefore the success message does not prove that the input is an image. Open it with a trusted image viewer or your normal image decoder before sending it. A file extension alone is not validation: renaming a broken download from .html to .png does not convert its contents.

The helper bounds bytes read from the file, but this is not a comprehensive untrusted-image processing service. It does not constrain decoded pixel dimensions or protect every downstream decoder. For an upload service, use appropriate image validation and request limits in that service rather than treating this small utility as its security boundary.

How do you send the prepared body to a local server?

Send the file unchanged to your own native chat endpoint. The hostname below refers to the machine where the command runs. If your application and server are on different machines, first confirm which endpoint you intend to contact.

curl --silent --show-error --fail-with-body \
  http://localhost:11434/api/chat \
  -H 'Content-Type: application/json' \
  --data-binary @vision-request.json

This command requests a nonstreaming result because the prepared body contains "stream": false. The chat reference documents that control. Keeping one response body makes an initial diagnostic easier to inspect; it is not a claim that nonstreaming improves inference quality or speed. If your curl predates --fail-with-body, use its supported HTTP-error reporting option and inspect the status and response separately.

A successful native chat response contains a message object. An HTTP success status does not establish that the description is correct. Ask about a simple visible feature you can independently verify, and keep the original image available for comparison. Do not mistake a fluent answer about an unseen object for evidence that the request wiring is correct.

If you later enable streaming, use a parser for the actual response format. The NDJSON parsing tutorial handles that separate boundary. Changing image encoding and stream parsing simultaneously makes failures harder to localize.

Which errors should change your next action?

Classify failures by the earliest layer with evidence. The Ollama errors reference describes HTTP status reporting and JSON error bodies; preserve both when recording a failed request.

Observation	Next action	What it does not establish
File cannot be opened locally	Correct the client-side path and permissions	Anything about the GPU
Local byte round trip fails in your application	Fix serialization or encoding	Model compatibility
Server reports an image decoding problem	Check the original image and the exact transmitted field	That more VRAM is needed
Server reports a missing model	Verify installed tag and endpoint	That the image is malformed
Logs identify allocation failure	Investigate that model and workload’s memory use	That every request failure is an OOM
Response is plausible but wrong	Check image attachment, model choice, and task quality	A successful recognition test

Do not repeatedly retry the same body while changing unrelated settings. Record the server version, model tag, input file size, endpoint, HTTP status, and error text. Keep image contents out of shared logs. That gives a maintainer useful evidence without turning a private image into a public debugging artifact.

Honest take: When should you resize an image or upgrade hardware?

Resize only after you have confirmed that the server receives a decodable image and that the model supports the task. Test one representative input at a time and record the effect. A smaller file on disk is not a universal predictor of inference memory: pixel dimensions, preprocessing, architecture, context, and other active workloads also matter.

If resource exhaustion is confirmed, the VRAM calculator can help with preliminary hardware planning. Treat it as a planning tool, then validate the exact vision model and image workload before a purchase. This tutorial supplies no new GPU price or memory measurement, so it cannot justify a particular upgrade.

For a working local setup, keep the smallest change that fixes the failing layer: correct the payload, select a supported installed model, or adjust a documented workload limit. Do not buy hardware to solve a filename, endpoint, or encoding mistake.

Sources

• Vision inputs and client examples — Ollama
• Native chat request and response — Ollama
• Native generate image field — Ollama
• HTTP status and error bodies — Ollama
• Base64 encoding and strict decoding — Python
• Filesystem paths and byte access — Python
• JSON serialization — Python

Last updated November 9, 2026. Local validation covers file handling and payload encoding only; no model inference or hardware benchmark was performed.

Get the numbers before you buy

New GPU and mini-PC benchmarks, VRAM thresholds, and price checks — sent when there's something worth acting on, not on a schedule. No spam, unsubscribe anytime.

Email address Subscribe