Vector Search medium
An IVF index with 16,384 lists serves search with nprobe=8. Recall@10 measured against ground truth is 82%, below the 95% target. Meanwhile p99 latency is 4 ms against a 30 ms budget. What is the most direct lever?
- A Increase the list count to 65,536 and rebuild, since finer clusters place each query closer to its true nearest neighbors.
- B Add product quantization to shrink vectors, freeing memory bandwidth so the index can scan its current lists faster.
- C Replace the index with flat brute-force search, since an IVF index cannot reach 95% recall without a full retrain.
- D Raise nprobe step by step while measuring recall and p99, spending some of the latency headroom to buy recall.
Answer: D. Raise nprobe step by step while measuring recall and p99, spending some of the latency headroom to buy recall.
**nprobe** directly trades latency for recall, because probing more lists finds neighbors that sit just across a cluster boundary. With a 26 ms headroom, this is a query-time change that needs no rebuild. Adding more lists at the same nprobe actually *lowers* recall, because each probe covers less of the data. PQ cuts memory but adds quantization error, which further reduces recall.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- You are building an IVF index on 20 million vectors. A teammate asks why the build needs a 'training' step before any vectors can be added. What is the reason?
- An HNSW index serves a marketplace where about 30% of listings are updated or removed each week. Each update is applied as a delete plus an insert, and deletes mark tombstones. Over a month, recall@10 drifts from 97% to 89% and p99 latency rises at the same ef_search. What is happening, and what should you do?
- Users delete documents in your primary database, but the deleted documents keep appearing in vector search results for hours. The vector index is rebuilt nightly. What is the best fix?
- Your embedding model outputs unnormalized vectors whose magnitudes vary widely. A teammate wants to switch the index from cosine similarity to dot product because it is cheaper to compute. What is the consequence?
- You are setting an SLO for a new ANN-backed recommendation search over 50 million vectors. A product manager asks for 100% recall@10 with single-digit millisecond latency. What is a realistic response?
- A B2B SaaS app serves 500 customers, and every query searches only one customer's documents. One large customer runs heavy batch queries that slow down search for everyone. Which sharding approach fits best?