TL;DR: If your local RAG application breaks after changing its embedding model, compare the new vectors with the index configuration before changing anything else. Matching dimensions are necessary but do not prove that old and new vectors are compatible. Build a separate index from the original documents, validate retrieval, then switch the application.

What you’ll be able to do:

• Check an Ollama embedding response without installing a Python SDK.
• Separate a dimension mismatch from a missing model or an oversized input.
• Plan a reversible reindex that preserves the working collection.

Honest take: Keep the current embedding model until a replacement passes a small retrieval test; a newer chat model is not a reason to rebuild your document index.

What actually needs to match in a local RAG pipeline?

A local RAG pipeline needs a consistent embedding configuration for stored documents and incoming questions. Ollama explicitly recommends using the same embedding model for indexing and querying. Its embedding endpoint returns normalized vectors, and accepts a batch of input strings. That makes the direct API a useful place to isolate a broken integration from the model server. See the official embedding guide.

Think of an index as a saved interpretation of your documents, rather than a folder containing interchangeable arrays. Changing the interpreter does not update the saved interpretation. An application can connect successfully, produce valid JSON, and still retrieve the wrong passages because the query side and document side have drifted apart.

Record these settings together: the embedding model, its local digest, requested output dimensions, document splitting settings, and any text prefixes added by your application. Also record the collection name. Keep the chat model in a separate field: it writes the answer after retrieval and is not the same component that creates the search vectors.

Symptom	First comparison	Sensible next action
Vector insertion fails	Response vector length versus collection size	Create a compatible replacement collection
Queries run but find unrelated passages	Indexing model versus query model	Restore the old configuration or re-embed documents
Only long documents fail	Short input versus the failing chunk	Inspect splitting and truncation
Model request returns HTTP 404	Installed model name versus requested name	Check the target Ollama instance
Direct request works, application fails	Direct request versus application payload	Inspect the adapter and selected endpoint

This table is a diagnostic order, not a list of universal error strings. Libraries and databases wrap failures differently. Save the actual response body before searching for a fix.

How do you confirm which model Ollama has installed?

Ask the same Ollama server that your application uses. A model installed on your desktop does not establish that a container or another home-lab machine has it.

For the example below, the Ollama model library documents embeddinggemma, a 300M-parameter embedding model requiring Ollama v0.11.10 or later. The download is a preparation step; this article does not claim a local inference benchmark or measured memory requirement.

ollama --version
ollama pull embeddinggemma
ollama list

Expected behavior: the pull completes, and the installed-model list includes the requested model. Do not substitute an invented success transcript for the output from your machine. If the model is already present, start by recording its identity before refreshing it.

The list-models API returns model names, sizes, and digests:

curl http://localhost:11434/api/tags

Save the relevant entry with your indexing notes. A digest gives you something concrete to compare after an update. The model label alone is an incomplete change record when the underlying download changes.

Run this request from the environment that actually makes the application’s network calls. Inside a container, localhost identifies that container. If connectivity itself is broken, use our Open WebUI-to-Ollama connection troubleshooting first. Rebuilding a database cannot fix a request that reaches the wrong server.

How can you measure vector length without another dependency?

Use Python’s standard library to call the embedding endpoint and inspect the returned arrays. Save this as check_embeddings.py, then run it with your installed Python interpreter. It sends synthetic text, not your private document collection.

import json
import math
import urllib.error
import urllib.request

texts = ["A spare key is kept in the workshop.",
         "The garden gate needs a new hinge."]
payload = {
    "model": "embeddinggemma",
    "input": texts,
    "truncate": False,
}
request = urllib.request.Request(
    "http://localhost:11434/api/embed",
    data=json.dumps(payload).encode("utf-8"),
    headers={"Content-Type": "application/json"},
    method="POST",
)
try:
    with urllib.request.urlopen(request, timeout=120) as response:
        result = json.load(response)
except urllib.error.HTTPError as error:
    raise SystemExit(error.read().decode("utf-8", errors="replace"))

vectors = result["embeddings"]
if len(vectors) != len(texts):
    raise SystemExit("Unexpected number of vectors")
lengths = [len(vector) for vector in vectors]
if not lengths or min(lengths) == 0 or len(set(lengths)) != 1:
    raise SystemExit("Empty or inconsistent vector lengths")
if not all(math.isfinite(value) for vector in vectors for value in vector):
    raise SystemExit("Non-finite vector value")
print("vectors:", len(vectors))
print("dimensions:", lengths)

Expected structure: vectors: 2, followed by equal, nonzero dimensions for both inputs. The timeout is an example client setting, not a speed claim. Google’s EmbeddingGemma model card lists a native output size of 768 and smaller representations of 512, 256, and 128. Check the actual response rather than copying a size from an unrelated model’s example.

The embedding API reference documents input, the embeddings response array, optional dimensions, and truncate. Start without a dimension override so that you can observe the model’s normal response. An adapter that silently requests a smaller output is an easy discrepancy to miss when comparing application settings with a manual test.

Now compare the printed length with your database’s configured vector field. For example, Qdrant collections define vector size and distance configuration; named vectors can have separate configurations. Inspect the particular field your application uses, rather than assuming every vector in a collection shares the same purpose.

Why is padding or trimming a vector the wrong quick fix?

Padding an arbitrary vector to satisfy a database schema does not make it compatible with embeddings created by another model. It only changes its shape. A clean insert is a weaker result than a useful search.

EmbeddingGemma has documented support for smaller representations through Matryoshka Representation Learning, including renormalization after truncation. That is a specific model capability, not permission to trim every embedding model. If you choose a supported smaller representation, apply the same policy to documents and queries and validate the retrieval behavior before migrating.

Do not repair individual failed rows by guessing their intended length. Quarantine the failed batch with its source-document IDs, inspect the configuration, and rerun it consistently. A mixed collection is harder to diagnose than an indexing job that stops with a clear error.

The most useful question is: can you reproduce this vector from a known document and a recorded configuration? If the answer is no, preserve the source text and rebuild into a new collection. Treat the current vectors as a cache whose provenance is uncertain, not as the only copy of your knowledge base.

What if a short input works but a document fails?

Test the failing chunk itself with automatic truncation disabled. Ollama documents that truncate defaults to true and that an overlong input returns an error when truncation is disabled. This makes a hidden input-length problem visible instead of silently accepting a shortened representation.

Google lists EmbeddingGemma’s maximum input context as 2K. That is a token limit, not a character allowance. Avoid claiming that a fixed number of characters always fits: code, punctuation, and different languages tokenize differently.

Keep a copy of the text after extraction and after splitting. A PDF can yield repeated headers, empty sections, or an entire table as a single awkward chunk. Before changing the model, check whether the supposedly searchable sentence survived extraction and landed inside the chunk you actually sent.

For a practical diagnosis, choose a document containing a distinctive sentence near its end. Search for that information and inspect the retrieved text directly. If the retrieved passage stops before the relevant section, fix the extraction or splitting path. Asking the chat model to be more confident will only conceal the missing evidence.

How do you rebuild the index without losing a working setup?

Use a separate collection name and keep the original collection available until the replacement passes your checks. This is an operational recommendation: it gives you a rollback point without depending on a destructive database reset.

1. Save the current application’s embedding and collection settings.
2. Confirm that you still have the source documents and any required metadata.
3. Create an empty replacement collection with the measured vector size.
4. Index a small, representative document subset using the new configuration.
5. Run the same known-answer questions against both collections.
6. Finish the replacement index, check document coverage, then switch the reader.

Use a manifest to distinguish intentional exclusions from missing documents. A successful indexing process can still omit a file that failed extraction. Track source identifiers and failure reasons, and reconcile them before declaring the new collection complete.

For a household document library, useful test cases include a model number in a manual, a date in an invoice, and an answer that requires the paragraph after a heading. These are suggested checks, not measured results. Include a question whose answer is absent: the application should not present a plausible unrelated passage as proof.

Keep ingestion and query configuration changes together. If one service starts using the replacement model while another still writes old embeddings, you can recreate the original mismatch immediately after a seemingly successful migration. Pause ingestion during the final switch if your application cannot coordinate those settings.

Honest take: when is this actually a hardware problem?

A dimension error is a configuration problem, not evidence that you need a larger GPU. First establish that a short direct embedding request succeeds. Only investigate capacity after the server reports loading or allocation failures, or after a controlled batch-size change reveals a resource issue.

Start with a small batch and keep the chat model idle during diagnosis. Once the basic request works, increase ingestion work gradually while watching the server logs. Separate model loading from repeated requests so that startup time does not masquerade as steady indexing performance. No universal throughput estimate is justified without the actual hardware, model, input lengths, and batching policy.

If your home server genuinely cannot hold the selected workload, use the VRAM planning calculator and the model reloading guide to investigate capacity and residency. Buying hardware before resolving incompatible vectors adds expense while preserving the bug.

FAQ

Q: Can I change the chat model without re-embedding everything? A: Keep the retrieval embedding configuration unchanged and test the new answer-generation model separately. Reindexing is relevant when the representation of your documents changes, not simply because you selected another chat model.

Q: Does HTTP 200 prove the index is repaired? A: It proves that a request succeeded at the HTTP layer. You still need matching vector configuration, document coverage, and useful retrieved passages. Ollama’s error reference distinguishes HTTP failures such as 400 and 404 and documents JSON error bodies; preserve those bodies when troubleshooting.

Q: Should I delete the original collection after one successful query? A: Keep it through the validation period you choose. Test different document formats and confirm the next ingestion run writes into the intended replacement collection before removing your rollback option.

Sources

• Embedding workflow — Ollama
• Generate embeddings API — Ollama
• EmbeddingGemma model package — Ollama
• EmbeddingGemma model card — Google
• List local models API — Ollama
• Collections and vector configuration — Qdrant
• API errors — Ollama

Last updated November 2, 2026; official documentation checked on that date. This is a documentation-based troubleshooting workflow, not a report of a measured local inference run.

Get the numbers before you buy

New GPU and mini-PC benchmarks, VRAM thresholds, and price checks — sent when there's something worth acting on, not on a schedule. No spam, unsubscribe anytime.

Email address Subscribe