Rag And Retrieval medium
Your hardware support bot uses dense-vector retrieval only. Users asking 'What does error E-4012 mean?' get chunks about generic error handling and about E-4021, a different code. Natural-language questions work well. What is the best fix?
- A Increase the embedding dimension from 768 to 3,072 so the vectors capture finer-grained token distinctions.
- B Raise first-stage top-k from 20 to 200 so the exact error-code chunk is more likely to appear somewhere.
- C Add a BM25 lexical index alongside the vector index and merge the two ranked lists with reciprocal rank fusion.
- D Fine-tune the generator model on the error-code manual so it can answer code lookups from memory without retrieval.
Answer: C. Add a BM25 lexical index alongside the vector index and merge the two ranked lists with reciprocal rank fusion.
Dense embeddings capture meaning, so near-identical identifiers like E-4012 and E-4021 land close together. **Lexical (BM25) retrieval** matches exact tokens, and **hybrid retrieval** combines both at the cost of a second index and a fusion step. Raising k only adds noise and tokens without fixing the ranking. Dense-only retrieval is enough when queries are mostly paraphrased natural language with no codes or rare names.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- You must index 100M chunks using a 3,072-dimension embedding in fp32, served from RAM via HNSW. The raw vectors alone take about 1.2 TB before graph overhead. You need to cut memory several-fold while losing at most a point or two of recall@10. Which approach fits?
- A multi-tenant SaaS app stores all tenants in one vector index with a tenant_id field. Queries fetch the top 20 chunks by similarity and then drop chunks from other tenants. Small tenants often get 0-2 results. Why does this happen, and what is the fix?
- Runbooks are split into fixed 300-token chunks with no overlap. When a procedure step starts near the end of one chunk and finishes in the next, answers come back incomplete. What is the lowest-cost fix?
- Your offline eval shows recall@5 of 92% on a labeled set, so the gold passage is in the context for nearly every question. Yet human graders mark 35% of answers wrong, and the wrong answers often ignore or contradict the gold passage. Where should the team invest next?
- Your product requires every RAG answer to include clickable source citations that users can verify. How should those citations be produced?
- A runbook RAG system uses fixed 500-token chunks with no overlap. Users who ask for a multi-step recovery procedure get answers that stop at step 4 of 9, and retrieval logs show the top hit ends mid-procedure. What is the most targeted fix?