Vector Search medium
Your embedding model was trained with cosine similarity but outputs unnormalized vectors. A teammate configures the index with inner product because it is cheaper to compute. Retrieval quality drops, and long documents with large-magnitude vectors dominate the results. What fixes this?
- A Inner product is approximate in HNSW while cosine is exact; switch to flat search to restore exact cosine scoring.
- B Cosine and inner product always rank identically, so the drop comes from HNSW tuning; raise efSearch to restore recall.
- C Vector magnitudes inflate inner-product scores; L2-normalize at write and query time, then inner product equals cosine.
- D L2 distance on the raw vectors produces the same ranking as cosine for any input, so switch the metric to L2 instead.
Answer: C. Vector magnitudes inflate inner-product scores; L2-normalize at write and query time, then inner product equals cosine.
Cosine and dot product are interchangeable **only for unit-length vectors**. On raw vectors, magnitude leaks into the inner-product score and biases results toward large-norm items. **Normalizing once at write and query time** keeps the cheaper inner product while matching the metric the model was trained with. Raising efSearch only improves ANN recall against the wrong metric, so it cannot fix the ranking. Raw dot product is correct only for models trained with it, where magnitude carries signal.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- You are building an IVF index on 20 million vectors. A teammate asks why the build needs a 'training' step before any vectors can be added. What is the reason?
- An HNSW index serves a marketplace where about 30% of listings are updated or removed each week. Each update is applied as a delete plus an insert, and deletes mark tombstones. Over a month, recall@10 drifts from 97% to 89% and p99 latency rises at the same ef_search. What is happening, and what should you do?
- Users delete documents in your primary database, but the deleted documents keep appearing in vector search results for hours. The vector index is rebuilt nightly. What is the best fix?
- Your embedding model outputs unnormalized vectors whose magnitudes vary widely. A teammate wants to switch the index from cosine similarity to dot product because it is cheaper to compute. What is the consequence?
- You are setting an SLO for a new ANN-backed recommendation search over 50 million vectors. A product manager asks for 100% recall@10 with single-digit millisecond latency. What is a realistic response?
- A B2B SaaS app serves 500 customers, and every query searches only one customer's documents. One large customer runs heavy batch queries that slow down search for everyone. Which sharding approach fits best?