Vector Search medium
An IVF index was trained on a sample of product embeddings 18 months ago. Since then the catalog has shifted heavily into a new category. Recall at a fixed nprobe has dropped, and a few inverted lists now hold most of the vectors. What should you do?
- A Raise nprobe permanently, since probing more lists recovers recall and the list imbalance has no effect on query latency.
- B Retrain the coarse quantizer on a fresh representative sample and rebuild, since old centroids no longer fit the data.
- C Switch the metric from L2 to cosine so vectors in the new category spread more evenly across the existing centroids.
- D Add product quantization so the oversized lists are compressed, which restores both recall and a balanced scan cost.
Answer: B. Retrain the coarse quantizer on a fresh representative sample and rebuild, since old centroids no longer fit the data.
IVF's coarse quantizer is **trained on a sample**, and when the data distribution drifts, the lists become imbalanced and the centroids stop partitioning the data well. **Retraining and rebuilding** costs a full index build but restores balanced lists and good recall. Raising nprobe hides the problem, and probing oversized lists inflates latency. Raising nprobe is a reasonable short-term fix only when the drift is mild and the lists are still balanced.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- You are building an IVF index on 20 million vectors. A teammate asks why the build needs a 'training' step before any vectors can be added. What is the reason?
- An HNSW index serves a marketplace where about 30% of listings are updated or removed each week. Each update is applied as a delete plus an insert, and deletes mark tombstones. Over a month, recall@10 drifts from 97% to 89% and p99 latency rises at the same ef_search. What is happening, and what should you do?
- Users delete documents in your primary database, but the deleted documents keep appearing in vector search results for hours. The vector index is rebuilt nightly. What is the best fix?
- Your embedding model outputs unnormalized vectors whose magnitudes vary widely. A teammate wants to switch the index from cosine similarity to dot product because it is cheaper to compute. What is the consequence?
- You are setting an SLO for a new ANN-backed recommendation search over 50 million vectors. A product manager asks for 100% recall@10 with single-digit millisecond latency. What is a realistic response?
- A B2B SaaS app serves 500 customers, and every query searches only one customer's documents. One large customer runs heavy batch queries that slow down search for everyone. Which sharding approach fits best?