Vector Search medium
A multi-tenant document search runs HNSW for the top-10 nearest vectors and then drops any results that don't belong to the requesting tenant. Small tenants, each holding about 0.5% of all vectors, often get 0 to 2 results even though they have relevant documents. What is the right fix?
- A Raise top-k from 10 to 50 before applying the tenant filter so that enough candidates survive the filtering step.
- B Apply the tenant filter during traversal, via pre-filtering or filter-aware search, so only matching vectors become candidates.
- C Switch the metric from cosine to dot product so that the requesting tenant's vectors score higher than other tenants' vectors.
- D Rebuild the HNSW graph with a higher M so each node links to more neighbors and the final top-10 contains more of the tenant's data.
Answer: B. Apply the tenant filter during traversal, via pre-filtering or filter-aware search, so only matching vectors become candidates.
**Post-filtering** is cheap and simple, but with a filter that matches only 0.5% of vectors, the expected number of matches in a top-10 is about 0.05. **Pre-filtering or filter-aware traversal** fixes this at the cost of more complex, somewhat slower queries. Over-fetching to top-50 is tempting, but it still yields about 0.25 expected matches. Over-fetching only works when the filter keeps a large share of the data, for example 30% or more.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- You are building an IVF index on 20 million vectors. A teammate asks why the build needs a 'training' step before any vectors can be added. What is the reason?
- An HNSW index serves a marketplace where about 30% of listings are updated or removed each week. Each update is applied as a delete plus an insert, and deletes mark tombstones. Over a month, recall@10 drifts from 97% to 89% and p99 latency rises at the same ef_search. What is happening, and what should you do?
- Users delete documents in your primary database, but the deleted documents keep appearing in vector search results for hours. The vector index is rebuilt nightly. What is the best fix?
- Your embedding model outputs unnormalized vectors whose magnitudes vary widely. A teammate wants to switch the index from cosine similarity to dot product because it is cheaper to compute. What is the consequence?
- You are setting an SLO for a new ANN-backed recommendation search over 50 million vectors. A product manager asks for 100% recall@10 with single-digit millisecond latency. What is a realistic response?
- A B2B SaaS app serves 500 customers, and every query searches only one customer's documents. One large customer runs heavy batch queries that slow down search for everyone. Which sharding approach fits best?