Vector Search medium
A B2B search product serves 2,000 tenants. One tenant holds 30% of all vectors and runs heavy batch queries that spike p99 latency for everyone. Every query is scoped to a single tenant. Which sharding strategy fits?
- A Shard by tenant and give the large tenant dedicated shards, so its load is isolated and queries touch only its shards.
- B Shard by hash of document ID, which spreads every tenant evenly across nodes and so prevents any single tenant hotspot.
- C Keep one shared index and add read replicas behind the load balancer, since replicas absorb spikes without resharding.
- D Shard by each vector's nearest IVF centroid so similar documents colocate and most queries touch only a single shard.
Answer: A. Shard by tenant and give the large tenant dedicated shards, so its load is isolated and queries touch only its shards.
**Tenant sharding** isolates noisy neighbors and lets single-tenant queries hit only that tenant's shards. The cost is uneven shard sizes, which you handle by giving large tenants dedicated shards. Hash sharding balances data, but every shard then holds part of the heavy tenant and serves its scatter-gather queries, so the noise spreads to everyone. Hash sharding is the better choice when queries span tenants or when all tenants are small and similar.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- You are building an IVF index on 20 million vectors. A teammate asks why the build needs a 'training' step before any vectors can be added. What is the reason?
- An HNSW index serves a marketplace where about 30% of listings are updated or removed each week. Each update is applied as a delete plus an insert, and deletes mark tombstones. Over a month, recall@10 drifts from 97% to 89% and p99 latency rises at the same ef_search. What is happening, and what should you do?
- Users delete documents in your primary database, but the deleted documents keep appearing in vector search results for hours. The vector index is rebuilt nightly. What is the best fix?
- Your embedding model outputs unnormalized vectors whose magnitudes vary widely. A teammate wants to switch the index from cosine similarity to dot product because it is cheaper to compute. What is the consequence?
- You are setting an SLO for a new ANN-backed recommendation search over 50 million vectors. A product manager asks for 100% recall@10 with single-digit millisecond latency. What is a realistic response?
- A B2B SaaS app serves 500 customers, and every query searches only one customer's documents. One large customer runs heavy batch queries that slow down search for everyone. Which sharding approach fits best?