Vector Search medium
A 50M-vector HNSW index is rebuilt nightly, and each rebuild takes about 6 hours. Product now requires new documents to be searchable within one minute of creation, and about 200K new documents arrive per day. What design meets this?
- A Search a small fresh segment by brute force alongside the main index, merge the results, and fold it in on the next rebuild.
- B Rebuild the full HNSW graph every minute on a larger machine, since graph build time drops linearly with added CPU cores.
- C Shorten the rebuild interval to every hour on a larger machine, since one hour is close to the one-minute target.
- D Serve fresh documents from a keyword index only, since embeddings for new content are unreliable until a model is retrained.
Answer: A. Search a small fresh segment by brute force alongside the main index, merge the results, and fold it in on the next rebuild.
A **small fresh segment** of at most a few hundred thousand vectors is cheap to scan exactly. Merging its results with the main index gives near-real-time visibility without frequent HNSW builds, which are slow. The cost is a second search path and a merge step on every query. An hourly rebuild still misses the one-minute target by a wide margin and burns six machine-hours of build for every hour of service. When ingest is low and the library supports concurrent inserts, writing straight into the live graph can be simpler.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- You are building an IVF index on 20 million vectors. A teammate asks why the build needs a 'training' step before any vectors can be added. What is the reason?
- An HNSW index serves a marketplace where about 30% of listings are updated or removed each week. Each update is applied as a delete plus an insert, and deletes mark tombstones. Over a month, recall@10 drifts from 97% to 89% and p99 latency rises at the same ef_search. What is happening, and what should you do?
- Users delete documents in your primary database, but the deleted documents keep appearing in vector search results for hours. The vector index is rebuilt nightly. What is the best fix?
- Your embedding model outputs unnormalized vectors whose magnitudes vary widely. A teammate wants to switch the index from cosine similarity to dot product because it is cheaper to compute. What is the consequence?
- You are setting an SLO for a new ANN-backed recommendation search over 50 million vectors. A product manager asks for 100% recall@10 with single-digit millisecond latency. What is a realistic response?
- A B2B SaaS app serves 500 customers, and every query searches only one customer's documents. One large customer runs heavy batch queries that slow down search for everyone. Which sharding approach fits best?