Vector Search medium
You moved a 200M-vector index to IVF-PQ to cut memory by about 30x. Recall@10 dropped from 96% to 78%. Full-precision vectors are still stored on SSD. How do you recover most of the recall without restoring the original RAM footprint?
- A Raise the number of PQ subquantizers until codes approach the original vector size, which removes compression error.
- B Fetch a larger shortlist, such as top-200, by PQ distance, then rerank it with exact distances from full vectors on SSD.
- C Switch the metric from L2 to inner product, since PQ distance errors tend to cancel out when vectors are compared by dot product.
- D Move the PQ codes to SSD and keep the full vectors in RAM, so the coarse search computes exact distances on the hot path.
Answer: B. Fetch a larger shortlist, such as top-200, by PQ distance, then rerank it with exact distances from full vectors on SSD.
PQ distances are approximate, so the true neighbors usually land in a wider shortlist but in the wrong order. **Reranking with full vectors** fixes the order at the cost of a few hundred SSD reads per query. Growing the PQ codes toward full size recovers recall only by giving up the memory savings that justified PQ. Larger codes make sense only when memory is no longer the binding constraint.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- You are building an IVF index on 20 million vectors. A teammate asks why the build needs a 'training' step before any vectors can be added. What is the reason?
- An HNSW index serves a marketplace where about 30% of listings are updated or removed each week. Each update is applied as a delete plus an insert, and deletes mark tombstones. Over a month, recall@10 drifts from 97% to 89% and p99 latency rises at the same ef_search. What is happening, and what should you do?
- Users delete documents in your primary database, but the deleted documents keep appearing in vector search results for hours. The vector index is rebuilt nightly. What is the best fix?
- Your embedding model outputs unnormalized vectors whose magnitudes vary widely. A teammate wants to switch the index from cosine similarity to dot product because it is cheaper to compute. What is the consequence?
- You are setting an SLO for a new ANN-backed recommendation search over 50 million vectors. A product manager asks for 100% recall@10 with single-digit millisecond latency. What is a realistic response?
- A B2B SaaS app serves 500 customers, and every query searches only one customer's documents. One large customer runs heavy batch queries that slow down search for everyone. Which sharding approach fits best?