Vector Search medium
You must index 100 million 1,536-dimension fp32 embeddings. The cluster budget allows roughly 256 GB of RAM in total for the index, and recall@10 of about 90% to 95% is acceptable. Which index design fits?
- A HNSW with default parameters, since it gives the best recall-latency curve and its graph overhead is small next to the vectors.
- B Flat brute-force search sharded across nodes, since exact search avoids recall loss and needs no extra graph structure in memory.
- C IVF with product quantization, compressing each vector to tens of bytes and reranking a shortlist to recover recall.
- D HNSW with M lowered to 4, which shrinks the neighbor lists enough to bring the whole index under the memory budget.
Answer: C. IVF with product quantization, compressing each vector to tens of bytes and reranking a shortlist to recover recall.
Each raw vector is about 6 KB, so the vectors alone take about **600 GB**, and HNSW adds a further 1.5x to 2x on top of that. **IVF-PQ** compresses vectors by one to two orders of magnitude, trading some recall for memory, which fits a 90% to 95% target. Lowering M only shrinks the graph and leaves the 600 GB of raw vectors in place. HNSW is the better choice when RAM is plentiful and you need recall close to 99%.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- You are building an IVF index on 20 million vectors. A teammate asks why the build needs a 'training' step before any vectors can be added. What is the reason?
- An HNSW index serves a marketplace where about 30% of listings are updated or removed each week. Each update is applied as a delete plus an insert, and deletes mark tombstones. Over a month, recall@10 drifts from 97% to 89% and p99 latency rises at the same ef_search. What is happening, and what should you do?
- Users delete documents in your primary database, but the deleted documents keep appearing in vector search results for hours. The vector index is rebuilt nightly. What is the best fix?
- Your embedding model outputs unnormalized vectors whose magnitudes vary widely. A teammate wants to switch the index from cosine similarity to dot product because it is cheaper to compute. What is the consequence?
- You are setting an SLO for a new ANN-backed recommendation search over 50 million vectors. A product manager asks for 100% recall@10 with single-digit millisecond latency. What is a realistic response?
- A B2B SaaS app serves 500 customers, and every query searches only one customer's documents. One large customer runs heavy batch queries that slow down search for everyone. Which sharding approach fits best?