Vector Search medium
A 50M-vector index supports filtering on user_id, and each user owns only about 500 vectors. With filter-aware HNSW traversal, these queries are slow and sometimes miss results because few graph neighbors match the filter. What should the query planner do?
- A Post-filter instead and raise top-k to 10,000, since over-fetching that many candidates guarantees enough matches survive.
- B Fetch the ~500 matching vectors via the metadata index and brute-force score them, which is exact and cheap at that size.
- C Raise efSearch until traversal finds enough matching nodes, since a higher efSearch always restores recall under any filter.
- D Rebuild HNSW with a larger M so the graph stays connected for every user_id value, accepting the extra memory it costs.
Answer: B. Fetch the ~500 matching vectors via the metadata index and brute-force score them, which is exact and cheap at that size.
When a filter is **extremely selective**, the matching subgraph becomes disconnected, so filtered traversal wanders through nodes that don't match. **Brute-forcing the small matching set** is both exact and fast. Post-filtering with top-10,000 expects only about 0.1 matches, because 500 out of 50M is 0.001%. Filter-aware ANN remains the better path when the filter keeps a large set of vectors, too many to scan.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- You are building an IVF index on 20 million vectors. A teammate asks why the build needs a 'training' step before any vectors can be added. What is the reason?
- An HNSW index serves a marketplace where about 30% of listings are updated or removed each week. Each update is applied as a delete plus an insert, and deletes mark tombstones. Over a month, recall@10 drifts from 97% to 89% and p99 latency rises at the same ef_search. What is happening, and what should you do?
- Users delete documents in your primary database, but the deleted documents keep appearing in vector search results for hours. The vector index is rebuilt nightly. What is the best fix?
- Your embedding model outputs unnormalized vectors whose magnitudes vary widely. A teammate wants to switch the index from cosine similarity to dot product because it is cheaper to compute. What is the consequence?
- You are setting an SLO for a new ANN-backed recommendation search over 50 million vectors. A product manager asks for 100% recall@10 with single-digit millisecond latency. What is a realistic response?
- A B2B SaaS app serves 500 customers, and every query searches only one customer's documents. One large customer runs heavy batch queries that slow down search for everyone. Which sharding approach fits best?