Ollama embeddings: catch truncated input before indexing
TL;DR: Set truncate: false while validating an Ollama embedding pipeline. A successful request can otherwise represent only the beginning of an oversized chunk. Fix the chunking contract before re-embedding documents or buying more hardware.
What you’ll be able to do:
- Send an individual extracted chunk with truncation explicitly disabled.
- Separate an input-length rejection from a dimension or transport failure.
- Define an ingestion gate that keeps missing document tails visible.
Honest take: Reject oversized chunks during ingestion; silently shortening the evidence is a poor default for a document-search system.
Why can an embedding succeed while retrieval misses the end?
Ollama’s embedding API documents truncate as enabled by default. When an input exceeds the embedding context, the server can shorten it instead of rejecting the request. Your vector database may receive a normal-looking vector even though the end of the extracted text was never represented.
That distinction matters when an application stores the full original chunk alongside the vector. A reader sees the whole passage in the search result and assumes the entire passage influenced retrieval. The stored text and the text represented by the vector can have different boundaries. A question about a detail near the beginning may work, while a question about a detail near the end fails to retrieve the passage.
This is a diagnostic hypothesis, not proof that every missed answer comes from truncation. Extraction errors, retrieval filters, an unsuitable embedding model, or stale vectors can produce similar symptoms. Start with one known document and one exact extracted chunk. Do not change the model, vector dimensions, chunk size, and retrieval settings together and then attribute the result to a single change.
The official embedding endpoint reference defines the switch. The embedding capability guide explains where embedding models fit in retrieval. Neither source promises that a successful vector guarantees useful retrieval for every part of a document.
What does disabling truncation actually change?
Setting truncate: false changes oversized-input handling into a visible failure. It does not increase the model’s context window, reduce its memory needs, or automatically split the input into chunks.
| Request policy | Oversized input | What your pipeline should do |
|---|---|---|
Omit truncate | Default permits truncation | Do not assume the entire input was represented |
Set truncate: true | Explicitly permits truncation | Record that this is an intentional data-loss policy |
Set truncate: false | Returns an error | Hold the chunk, revise chunking, then retry deliberately |
Current Ollama server source contains the error string the input length exceeds the context length in the non-truncating embedding path. That branch constructs an HTTP 400 status error. Source was checked November 12, 2026, at revision 8a89b76cc7359805e48357ba7e5b99e845a4c211; the exact path taken can vary with the installed server and runner.
Save the status and error body from your actual request. An HTTP 400 is not enough by itself to diagnose an input-length failure: other invalid request fields can also be rejected. Conversely, a timeout or an unavailable model is not a reason to shorten the text automatically. The API error documentation is the starting point for distinguishing these cases.
How do you check one real chunk without a framework?
Export the exact UTF-8 text that the ingestion application sends to the embedding model, then submit that file through a small direct client. Use an already installed embedding model supported by your Ollama server; replace YOUR_INSTALLED_EMBEDDING_MODEL with its actual name.
python3 embed_check.py YOUR_INSTALLED_EMBEDDING_MODEL chunk.txt
Save the following as embed_check.py. The example targets the local native endpoint, sends one input, and never falls back to a different model or enables truncation after an error. Its 60-second timeout is a client blocking-operation setting, not a model latency claim or total ingestion deadline.
import json
import sys
from urllib.error import HTTPError
from urllib.request import Request, urlopen
def embed_one(base, model, text):
payload = {"model": model, "input": [text], "truncate": False}
request = Request(base.rstrip("/") + "/api/embed",
data=json.dumps(payload).encode(),
headers={"Content-Type": "application/json"})
try:
with urlopen(request, timeout=60) as response:
data = json.load(response)
except HTTPError as error:
detail = error.read(4096).decode("utf-8", "replace")
raise RuntimeError(f"HTTP {error.code}: {detail}") from error
vectors = data.get("embeddings")
if not isinstance(vectors, list) or len(vectors) != 1:
raise ValueError("expected one embedding for one input")
vector = vectors[0]
if not isinstance(vector, list) or not vector:
raise ValueError("expected a nonempty vector")
return len(vector), data.get("prompt_eval_count")
if __name__ == "__main__":
if len(sys.argv) != 3:
raise SystemExit("usage: embed_check.py MODEL UTF8_TEXT_FILE")
with open(sys.argv[2], encoding="utf-8") as source:
text = source.read()
if not text.strip():
raise SystemExit("empty input: inspect extraction before embedding")
dimensions, tokens = embed_one("http://127.0.0.1:11434",
sys.argv[1], text)
print(json.dumps({"dimensions": dimensions,
"prompt_eval_count": tokens,
"truncate": False}))
On success the output contains the vector width, the server’s reported input-token count when present, and the explicit truncation policy. A failure stops before reporting success. The helper does not connect to a vector database or write an embedding, so it cannot accidentally insert a diagnostic vector into your production collection.
The client checks response shape only far enough to make the diagnostic readable. It is not a complete numerical-vector validator. Before production insertion, your normal ingestion layer must also enforce the collection’s dimension, numerical type, and finite-value requirements. See the separate embedding dimension mismatch guide for that contract; changing truncation policy does not repair a mismatched index.
What was actually tested for this client?
The published Python client was exercised against a local synthetic HTTP server on November 12, 2026. No embedding model was loaded, and no Ollama inference was measured. The fixture tests request construction and response/error handling, not the model’s tokenizer or context capacity.
| Synthetic response | Input sent by client | Observed result |
|---|---|---|
| Success with one vector | truncate: false | Width 3 and reported token count 7 returned |
| HTTP 400 with input-length error | truncate: false | Runtime error includes HTTP status and body |
The width 3, count 7, and short fixture text are deliberately invented test inputs and outputs. They are not an embedding model specification. The error text is taken from the verified upstream source, but receiving it from our mock server does not prove a particular installed Ollama version will take the same branch.
For your real check, keep a record of the installed server version, model name, file identity, status code, and error body. Keep private document text local. A source citation verifies the public API contract; only your own request can show whether your particular chunk is accepted.
How should you split an oversized chunk?
Split the document at a meaningful structural boundary, then re-run the same non-truncating check on every resulting chunk. Paragraphs, headings, table sections, and code blocks are better starting points than a blind cut through the middle of a sentence.
A character limit is a chunking policy, not a reliable token count. Different languages, identifiers, markup, and extracted PDF text can tokenize differently. A long source-code identifier or table may behave differently from ordinary prose of the same character length. Do not publish a universal characters-per-token conversion as a guaranteed model limit.
Inspect the model-details endpoint for the metadata your installed model exposes. Then use the embedding endpoint itself as the final acceptance gate. Metadata is useful context, but a successful metadata request does not demonstrate that an arbitrary chunk fits the runtime request configuration.
Choose an overlap policy for the content rather than copying a magic number. If a paragraph refers to a definition under the preceding heading, carry that heading or definition into the next chunk deliberately. Record the policy so that future ingestion runs reproduce the same boundaries. Do not keep shrinking only the failing final chunk while leaving the rest of the corpus under a different undocumented rule.
A rejected chunk should enter a visible hold list with its document ID, chunk ID, and failure reason. An ingestion run that silently skips rejected chunks merely exchanges hidden truncation for hidden omissions. Count accepted, rejected, and intentionally excluded chunks separately before marking the document searchable.
What should happen when only one item in a batch fails?
Isolate the failing item before rebuilding the entire corpus. The native endpoint accepts either a text input or an array of texts, but a batch-level error does not identify a safe subset that your application may assume was inserted.
During diagnosis, send each suspect chunk individually using the same model and options. Keep the original document-to-chunk mapping intact. Do not concatenate all chunks into one input to simplify the request: that changes both the context requirement and the retrieval unit.
Once the failing chunk is split and all replacement chunks pass, validate the resulting ingestion transaction. If the database already contains vectors from an earlier truncated run, replacing the chunking configuration alone does not replace those vectors. Plan an explicit rebuild or a versioned collection migration according to your database’s own documented behavior.
Use stable identifiers for replacement records and check for duplicate ingestion. A network retry after an uncertain insertion result can create duplicate records unless your application defines replacement semantics. The Ollama timeout diagnostic explains why a client timeout should not trigger a blind repeated request. Database write confirmation is a separate boundary from embedding generation.
Does a bigger GPU solve the embedding context limit?
A larger GPU does not by itself change the embedding model’s accepted input length. Hardware capacity and the model/runtime context contract are separate constraints. Diagnose the input-length error before spending money on a memory upgrade.
This diagnostic uses an existing computer, Python’s standard library, and your local Ollama installation. The helper adds no paid API requirement and downloads no weights. Running a real model still consumes your machine’s resources; this article does not assign an electricity price or claim a universal RAM or VRAM minimum.
If the actual server logs instead show a memory allocation failure, investigate the model and runtime configuration separately. The VRAM calculator is a planning aid, while the Ollama memory-report guide explains why disk size and loaded memory are different observations. Neither is evidence that your overlong document now fits an embedding context.
Honest take: when should truncation stay enabled?
Intentional truncation can be acceptable when the application explicitly needs only a prefix and communicates that boundary. It is a poor fit for an ingestion process that claims to search the full content of an uploaded document.
Keep truncate: false during acceptance testing and when missing tails would change the meaning of the evidence. If you later enable truncation for a specific workload, record that choice with the ingestion configuration and test questions aimed at the retained and discarded regions. Do not call those questions a retrieval benchmark unless you define and actually run an evaluation.
The practical stopping condition is simple: every intended chunk is either accepted under the declared policy or explicitly held for correction, and the stored text corresponds to the text you chose to embed. That is a more useful milestone than a green request counter.
Sources
- Embedding endpoint and truncation contract — Ollama
- Embedding models and retrieval — Ollama
- Show model details — Ollama
- API error handling — Ollama
- Embedding input validation, pinned server source — Ollama
Last updated November 12, 2026. Local fixtures validate the client only; no model inference, hardware benchmark, or retrieval-quality measurement is claimed.
Was this article helpful?
Thanks for the feedback — it helps improve future articles.
Need hands-on help?
I offer 1-on-1 technical consulting for local AI setup, GPU selection, and AI coding tool configuration — same topics covered on this site.
Book a session — $49 / hour →Get the numbers before you buy
New GPU and mini-PC benchmarks, VRAM thresholds, and price checks — sent when there's something worth acting on, not on a schedule. No spam, unsubscribe anytime.