Vector Search medium
A news search index uses HNSW and expires articles after 30 days by marking them as deleted. Months later, recall has dropped and latency has risen, even though the number of live vectors has stayed roughly constant. What is happening, and what fixes it?
- A Deleted articles are leaking into results and lowering precision; add a post-filter that drops expired IDs before returning.
- B HNSW graphs decay as the embedding model ages; re-embed all live articles with the same model every month to refresh them.
- C Cosine distances drift as more articles are added; switch the metric to L2 so distances in the graph stay stable over time.
- D Tombstoned nodes stay in the graph and are traversed then discarded; periodically rebuild or compact to remove them.
Answer: D. Tombstoned nodes stay in the graph and are traversed then discarded; periodically rebuild or compact to remove them.
HNSW deletes are usually **soft**: tombstoned nodes still occupy memory and take up traversal steps, so the search spends effort on dead nodes and finds fewer live neighbors. **Periodic rebuild or compaction** trades build cost for restored recall and latency. Adding a post-filter does nothing, because tombstones are already excluded from results; the damage happens during traversal. Soft deletes are fine when churn is a small fraction of the index.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- You are building an IVF index on 20 million vectors. A teammate asks why the build needs a 'training' step before any vectors can be added. What is the reason?
- An HNSW index serves a marketplace where about 30% of listings are updated or removed each week. Each update is applied as a delete plus an insert, and deletes mark tombstones. Over a month, recall@10 drifts from 97% to 89% and p99 latency rises at the same ef_search. What is happening, and what should you do?
- Users delete documents in your primary database, but the deleted documents keep appearing in vector search results for hours. The vector index is rebuilt nightly. What is the best fix?
- Your embedding model outputs unnormalized vectors whose magnitudes vary widely. A teammate wants to switch the index from cosine similarity to dot product because it is cheaper to compute. What is the consequence?
- You are setting an SLO for a new ANN-backed recommendation search over 50 million vectors. A product manager asks for 100% recall@10 with single-digit millisecond latency. What is a realistic response?
- A B2B SaaS app serves 500 customers, and every query searches only one customer's documents. One large customer runs heavy batch queries that slow down search for everyone. Which sharding approach fits best?