Vector Search hard
You must serve 100M documents with 1,536-dimension fp32 embeddings from an in-memory HNSW index. Each node has 256 GB of RAM. Which capacity estimate and plan are correct?
- A About 60 GB total, so a single node holds the index with ample headroom for replicas and growth.
- B About 600 GB; graph links are tiny next to the vectors, so three nodes are enough for one full copy.
- C About 150 GB after converting to fp16, so a single node can hold the full HNSW index plus its graph.
- D About 0.9 to 1.2 TB with graph overhead, so shard across 5 or more nodes or use PQ or a disk index.
Answer: D. About 0.9 to 1.2 TB with graph overhead, so shard across 5 or more nodes or use PQ or a disk index.
Each vector is 1,536 dims x 4 bytes is about 6 KB, so 100M vectors take about 600 GB raw. HNSW neighbor lists and metadata typically push the total footprint to **1.5x to 2x** that. The tradeoff is HNSW's recall and latency against its memory bill; if the bill is unaffordable, **PQ** or an SSD-resident graph gives up some recall for far less RAM. Option B fails because ignoring graph overhead under-provisions by hundreds of GB, and fp16 (C) halves raw size to about 300 GB, not 150 GB.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- You are building an IVF index on 20 million vectors. A teammate asks why the build needs a 'training' step before any vectors can be added. What is the reason?
- An HNSW index serves a marketplace where about 30% of listings are updated or removed each week. Each update is applied as a delete plus an insert, and deletes mark tombstones. Over a month, recall@10 drifts from 97% to 89% and p99 latency rises at the same ef_search. What is happening, and what should you do?
- Users delete documents in your primary database, but the deleted documents keep appearing in vector search results for hours. The vector index is rebuilt nightly. What is the best fix?
- Your embedding model outputs unnormalized vectors whose magnitudes vary widely. A teammate wants to switch the index from cosine similarity to dot product because it is cheaper to compute. What is the consequence?
- You are setting an SLO for a new ANN-backed recommendation search over 50 million vectors. A product manager asks for 100% recall@10 with single-digit millisecond latency. What is a realistic response?
- A B2B SaaS app serves 500 customers, and every query searches only one customer's documents. One large customer runs heavy batch queries that slow down search for everyone. Which sharding approach fits best?