Rag And Retrieval hard
An internal engineering assistant uses dense-only retrieval. A query like 'what causes ERR_QUOTA_4471 in billing-svc' returns conceptually related quota articles but rarely the runbook that mentions the exact code. Recall on natural-language questions is fine. Which fix addresses the root cause?
- A Add a BM25 lexical index alongside the vector index and fuse both result lists, so exact tokens like error codes match directly.
- B Fine-tune the embedding model on paraphrase pairs from support tickets, so that error codes cluster closer to the runbooks that reference them.
- C Raise first-stage k from 50 to 500, so the exact-match runbook eventually appears somewhere in the candidate set handed to the reranker.
- D Re-chunk the runbooks into smaller 150-token pieces, so each error code dominates the embedding of the chunk that contains it.
Answer: A. Add a BM25 lexical index alongside the vector index and fuse both result lists, so exact tokens like error codes match directly.
Dense embeddings capture meaning but blur rare tokens such as error codes, SKUs and proper nouns. **Hybrid retrieval** adds exact lexical matching at the cost of running and fusing a second index. Fine-tuning is tempting, but it is expensive and still does not guarantee exact-string matching for codes the model never saw. Fine-tuning pays off when the gap is domain vocabulary rather than literal identifiers.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- You must index 100M chunks using a 3,072-dimension embedding in fp32, served from RAM via HNSW. The raw vectors alone take about 1.2 TB before graph overhead. You need to cut memory several-fold while losing at most a point or two of recall@10. Which approach fits?
- A multi-tenant SaaS app stores all tenants in one vector index with a tenant_id field. Queries fetch the top 20 chunks by similarity and then drop chunks from other tenants. Small tenants often get 0-2 results. Why does this happen, and what is the fix?
- Runbooks are split into fixed 300-token chunks with no overlap. When a procedure step starts near the end of one chunk and finishes in the next, answers come back incomplete. What is the lowest-cost fix?
- Your offline eval shows recall@5 of 92% on a labeled set, so the gold passage is in the context for nearly every question. Yet human graders mark 35% of answers wrong, and the wrong answers often ignore or contradict the gold passage. Where should the team invest next?
- Your product requires every RAG answer to include clickable source citations that users can verify. How should those citations be produced?
- A runbook RAG system uses fixed 500-token chunks with no overlap. Users who ask for a multi-step recovery procedure get answers that stop at step 4 of 9, and retrieval logs show the top hit ends mid-procedure. What is the most targeted fix?