You need to migrate a production semantic-search system to a new embedding model. What compatibility, normalization, distance-metric, indexing, recall, and rollout issues would you check before replacing the existing vectors?

How to think about it

Short answer

I would treat a new embedding model as a new retrieval space, not a drop-in vector replacement: verify the encoder contract, normalization, distance metric, dimensions, index structure, recall, latency, and operating cost. I would re-embed documents and queries with versioned data, run old and new indexes side by side, shadow and canary traffic, and keep the old path available until rollback and consistency checks pass.

Why the migration is not a vector swap

An embedding model maps text into a coordinate system. The individual coordinates have no useful meaning to us; the geometry between vectors is what retrieval uses. Change the model and you change that geometry.

A 768-dimensional vector from model A and a 768-dimensional vector from model B may have the same shape in memory, but their coordinates are not interchangeable. A query encoded by A must search documents encoded by A. Encoding the query with B and searching A’s vectors is not a clever bridge between models. It is a broken comparison.

The contract includes more than the model name:

• model version and tokenizer
• document and query preprocessing
• truncation length and chunking rules
• language and modality support
• query and document instructions or prefixes
• output dimension and data type
• whether the output is already normalized
• whether a reranker expects features from the old model

This last point catches teams that did everything right in the vector store and still lost quality. Some retrieval models use different instructions for queries and passages. Others are trained for symmetric inputs. The model’s documented encoding recipe is part of the model, not optional decoration.

Common trap: equal dimensions do not imply compatibility. The same is true of equal-looking scores. A cosine score from one model is not a meaningful threshold for another model.

Normalization and the metric

First establish what the old system actually ranks by. Common choices are cosine similarity, dot product, and Euclidean distance.

Cosine similarity divides the dot product by both vectors’ lengths. It therefore cares about direction, not magnitude. Dot product cares about both direction and magnitude. Euclidean distance measures straight-line separation.

If both vectors are unit length, these rankings become closely related:

• cosine similarity equals the dot product
• squared Euclidean distance equals 2 - 2 * cosine_similarity

That equivalence disappears when vectors are not normalized. A dot-product model may deliberately use vector length as a signal. Normalizing it can remove information and lower relevance. Conversely, using dot product on unnormalized outputs when the model expects cosine can make vector magnitude dominate the result.

For a cosine-style system, row normalization looks like this:

import numpy as np

def normalize_rows(x):
    norms = np.linalg.norm(x, axis=1, keepdims=True)
    return x / np.maximum(norms, 1e-12)

docs = normalize_rows(docs)
queries = normalize_rows(queries)

scores = queries @ docs.T
top10 = np.argsort(-scores, axis=1)[:, :10]

The matrix multiplication produces cosine scores because both sets of rows have unit length. In production, an approximate nearest-neighbor index replaces the full sort; the mathematics does not change.

I would verify whether normalization happens in the encoder, ingestion pipeline, or vector database. Some stores accept a cosine metric and normalize internally; others do not. I would not rely on a name like cosine without checking current behavior and testing a few hand-computed vectors.

Score thresholds must also be retuned. A rule such as “show results only above 0.78” belongs to a particular model, preprocessing recipe, and metric. It is not a universal measure of relevance.

Rebuild the index for the new geometry

A new dimension usually requires a new collection or index schema. Even if the dimensions happen to match, the old approximate-nearest-neighbor structure is built around the old vector distribution.

For example, an HNSW index stores links chosen using distances in the old space. Replacing the payload vectors while keeping those links leaves a graph whose neighborhood decisions were made using obsolete geometry. An IVF index has centroids trained on the old distribution. Product-quantization codebooks are also distribution-specific. The result may return vectors and still have poor recall.

The safe pattern is:

1. Create a versioned index for the new model and metric.
2. Backfill every current document into it.
3. Rebuild any centroids, graph links, quantizers, and replicas.
4. Measure approximate search against exact search using the same new vectors.
5. Tune search parameters for the new index rather than copying old settings blindly.

That fourth step separates two failures that otherwise look identical. If exact search is bad, the model, input formatting, chunking, or labels are the likely problem. If exact search is good but the ANN index is bad, the index configuration or compression is the likely problem.

Filtering matters too. A vector index that retrieves 100 candidates globally may return only three useful candidates after tenant, language, permissions, or document-status filters. Test the real filter combinations. Pre-filtering, post-filtering, and shard-level filtering can have very different recall behavior.

A concrete migration check

Suppose a help-center search serves 10 million document chunks. The old model emits 768-dimensional float32 vectors, already normalized, and the index uses cosine similarity. The proposed model emits 1,024-dimensional vectors.

The raw vector storage rises from:

• 10,000,000 * 768 * 4 bytes = 30.72 GB
• 10,000,000 * 1,024 * 4 bytes = 40.96 GB

That is an extra 10.24 decimal GB before HNSW links, replicas, metadata, and temporary rebuild storage. Query encoding may also cost more. Capacity planning needs to include the old and new indexes existing simultaneously during migration.

I would take 5,000 real queries, including zero-result searches, tail queries, multiple languages, long queries, fresh documents, and the queries that generated support complaints. For each query, I would record human judgments or a trusted relevance set. Then I would compare:

• Recall at the candidate depth, such as Recall@100 if a reranker sees 100 results
• final ranking metrics such as nDCG@10 or MRR
• exact-search results using the new vectors
• ANN results using the new index
• p50 and p95 latency
• empty-result rate and filter-specific failures
• embedding throughput, index build time, memory, and cost

Imagine the service contract requires p95 search latency under 200 ms and no more than a one-percentage-point drop in Recall@100 on each important customer slice. Those are acceptance criteria for this service, not laws of nature. A new model that wins on an academic benchmark but misses product names, internal acronyms, or non-English tickets is not an upgrade for this system.

I would also compare top-result overlap, but only as a diagnostic. Low overlap is not automatically bad: two lists can differ while the new list is better. Human judgments and downstream outcomes decide that.

Roll out without creating a mixed-space outage

Store the embedding model version, dimension, normalization status, metric, preprocessing version, and index version with the index metadata. Put the model version into cache keys. Otherwise an apparently successful cutover can serve old cached results or reuse query vectors from the wrong space.

During backfill, dual-write new and changed documents to both versions. Track a watermark showing how far the new index has caught up. Handle updates and deletes explicitly; a new index containing yesterday’s document text is not ready merely because its document count looks correct.

Then:

• send shadow queries to the new index without showing its results
• compare quality proxies, latency, errors, score distributions, and filter behavior
• canary a small, stable slice of users or tenants
• expand gradually behind a feature flag
• switch atomically once the gates pass
• retain the old index and encoder until the rollback window closes

Do not send a new-model query vector to the old index as a fallback. The fallback must use the old encoder as well. If both systems need to operate during a transition, keep them as separate spaces and choose one route per request, or combine them through a deliberate hybrid-ranking strategy. Never compare raw scores from the two models as though 0.81 means the same thing in both.

What they’ll ask next

“Can I update the old vectors in place?”
Usually not safely. A store may permit updates, but the ANN graph, centroids, or quantizers can still reflect the old distribution. Build and validate a fresh index; use an in-place update only if the index documentation and measurements show that its structure is rebuilt or remains valid.

“How do I tell whether the model or the ANN index caused a recall drop?”
Run exact nearest-neighbor search with the new vectors first. Compare that result with labeled relevance, then compare the ANN result with the exact result. The first comparison tests semantic quality; the second tests approximate-index recall.

“What if we cannot re-embed all 10 million documents at once?”
Keep two versioned indexes and backfill incrementally. Route queries consistently to one version, or use a measured dual-search strategy. Do not silently mix old and new vectors in one index and hope similar text will hide the problem.

One line to say in the room

“I would migrate the retrieval space, not just the vectors: validate the encoder and metric, rebuild and separately measure the index, then cut over through a versioned shadow-and-canary rollout with a real rollback.”