Vector Search hard
A news search service must make articles searchable within 5 seconds of publication. The main HNSW index holds 300M vectors. Inserting directly into it at peak causes p99 spikes from insert cost and lock contention, and a full rebuild takes hours. Which design meets the freshness target?
- A Send new articles to a small delta segment, search it with the main index, merge, and compact on a schedule.
- B Rebuild the main index every few seconds on a standby replica and swap it in, so readers never see inserts in flight.
- C Insert directly into the main index with a much lower ef_construction so each insert finishes fast enough at peak.
- D Batch new articles and insert into the main index every 5 seconds, since batching removes most per-insert graph cost.
Answer: A. Send new articles to a small delta segment, search it with the main index, merge, and compact on a schedule.
A small **delta segment** absorbs writes cheaply, since flat search over a few hours of articles is fast and exact, and compaction into the main index happens off the hot path. The cost is an extra search per query, plus merge and dedup logic, plus deletes reconciled across segments. Lowering ef_construction (C) permanently degrades graph quality and recall for every future query. Direct inserts are fine only when the write rate is low enough not to contend with reads.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- You are building an IVF index on 20 million vectors. A teammate asks why the build needs a 'training' step before any vectors can be added. What is the reason?
- An HNSW index serves a marketplace where about 30% of listings are updated or removed each week. Each update is applied as a delete plus an insert, and deletes mark tombstones. Over a month, recall@10 drifts from 97% to 89% and p99 latency rises at the same ef_search. What is happening, and what should you do?
- Users delete documents in your primary database, but the deleted documents keep appearing in vector search results for hours. The vector index is rebuilt nightly. What is the best fix?
- Your embedding model outputs unnormalized vectors whose magnitudes vary widely. A teammate wants to switch the index from cosine similarity to dot product because it is cheaper to compute. What is the consequence?
- You are setting an SLO for a new ANN-backed recommendation search over 50 million vectors. A product manager asks for 100% recall@10 with single-digit millisecond latency. What is a realistic response?
- A B2B SaaS app serves 500 customers, and every query searches only one customer's documents. One large customer runs heavy batch queries that slow down search for everyone. Which sharding approach fits best?