Vector Search easy
You run semantic search over one public corpus of 300 million web pages. Every query searches the whole corpus, and traffic is fairly uniform. How should you shard the index?
- A Shard by the topic of each page so that queries only need to visit the single most relevant shard.
- B Shard by hash of document ID to spread data and load evenly, then merge top-k from all shards.
- C Shard by the date of each page so that recent content sits on the fastest nodes in the cluster.
- D Shard by the user who issued the query so each user's searches always land on the same node.
Answer: B. Shard by hash of document ID to spread data and load evenly, then merge top-k from all shards.
**Hash sharding** spreads data and load evenly, at the cost of a scatter-gather on every query, where tail latency is set by the slowest shard. Sharding by topic is tempting, but queries near topic boundaries miss results and popular topics create hot shards. When queries are naturally scoped, for example to one tenant, partitioning by that scope is the better choice.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- You are building an IVF index on 20 million vectors. A teammate asks why the build needs a 'training' step before any vectors can be added. What is the reason?
- An HNSW index serves a marketplace where about 30% of listings are updated or removed each week. Each update is applied as a delete plus an insert, and deletes mark tombstones. Over a month, recall@10 drifts from 97% to 89% and p99 latency rises at the same ef_search. What is happening, and what should you do?
- Users delete documents in your primary database, but the deleted documents keep appearing in vector search results for hours. The vector index is rebuilt nightly. What is the best fix?
- Your embedding model outputs unnormalized vectors whose magnitudes vary widely. A teammate wants to switch the index from cosine similarity to dot product because it is cheaper to compute. What is the consequence?
- You are setting an SLO for a new ANN-backed recommendation search over 50 million vectors. A product manager asks for 100% recall@10 with single-digit millisecond latency. What is a realistic response?
- A B2B SaaS app serves 500 customers, and every query searches only one customer's documents. One large customer runs heavy batch queries that slow down search for everyone. Which sharding approach fits best?