Vector Search medium
Your team wants to switch to a new embedding model with better retrieval quality. The live index holds 40 million documents embedded with the old model and serves production traffic. How should you roll out the change?
- A Point queries at the new model now and re-embed documents lazily as they change, since both models encode the same text.
- B Swap the model in config and rebuild the HNSW graph in place overnight, since the stored vectors remain valid graph inputs.
- C Learn a linear mapping from old vectors into the new space so the existing index can be reused without any re-embedding.
- D Build a second versioned index with the new model, dual-write new documents, backfill the rest, then cut queries over.
Answer: D. Build a second versioned index with the new model, dual-write new documents, backfill the rest, then cut queries over.
Vectors from different models live in **incompatible spaces**, so changing the model invalidates the entire index. A **versioned index with dual-write and backfill** costs double storage and compute during the migration, but it allows a safe cutover and rollback. Comparing new-model queries against old-model documents produces meaningless distances, and rebuilding the graph does not change the old vectors. A learned mapping is lossy and is only worth considering as a stopgap when re-embedding is impossible.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- You are building an IVF index on 20 million vectors. A teammate asks why the build needs a 'training' step before any vectors can be added. What is the reason?
- An HNSW index serves a marketplace where about 30% of listings are updated or removed each week. Each update is applied as a delete plus an insert, and deletes mark tombstones. Over a month, recall@10 drifts from 97% to 89% and p99 latency rises at the same ef_search. What is happening, and what should you do?
- Users delete documents in your primary database, but the deleted documents keep appearing in vector search results for hours. The vector index is rebuilt nightly. What is the best fix?
- Your embedding model outputs unnormalized vectors whose magnitudes vary widely. A teammate wants to switch the index from cosine similarity to dot product because it is cheaper to compute. What is the consequence?
- You are setting an SLO for a new ANN-backed recommendation search over 50 million vectors. A product manager asks for 100% recall@10 with single-digit millisecond latency. What is a realistic response?
- A B2B SaaS app serves 500 customers, and every query searches only one customer's documents. One large customer runs heavy batch queries that slow down search for everyone. Which sharding approach fits best?