Rag And Retrieval hard
Auditors find that 12% of citations in your RAG answers point to real internal documents that were not in the retrieved context for that query. Some cited passages do not even contain the claimed fact. What design enforces grounded citations?
- A Fine-tune the generator on the internal corpus, so its memory of documents is accurate enough that citing them from memory becomes reliable.
- B After generation, embed each answer sentence and attach the nearest document in the whole index as its citation, replacing the model's picks.
- C Give each retrieved chunk an ID, require citations to reference only those IDs, and verify each cited span supports its claim before returning.
- D Lower the sampling temperature to zero, so the model deterministically reproduces citation strings it has seen and stops inventing new ones.
Answer: C. Give each retrieved chunk an ID, require citations to reference only those IDs, and verify each cited span supports its claim before returning.
Citations must point at **retrieved spans**, never at the model's memory. The verification step costs extra latency and may force the system to drop or hedge unsupported claims. Post-hoc nearest-neighbor attribution is tempting, but it cites whatever resembles the generated text, not the evidence the model actually read. It can also attach sources the user cannot access.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- You must index 100M chunks using a 3,072-dimension embedding in fp32, served from RAM via HNSW. The raw vectors alone take about 1.2 TB before graph overhead. You need to cut memory several-fold while losing at most a point or two of recall@10. Which approach fits?
- A multi-tenant SaaS app stores all tenants in one vector index with a tenant_id field. Queries fetch the top 20 chunks by similarity and then drop chunks from other tenants. Small tenants often get 0-2 results. Why does this happen, and what is the fix?
- Runbooks are split into fixed 300-token chunks with no overlap. When a procedure step starts near the end of one chunk and finishes in the next, answers come back incomplete. What is the lowest-cost fix?
- Your offline eval shows recall@5 of 92% on a labeled set, so the gold passage is in the context for nearly every question. Yet human graders mark 35% of answers wrong, and the wrong answers often ignore or contradict the gold passage. Where should the team invest next?
- Your product requires every RAG answer to include clickable source citations that users can verify. How should those citations be produced?
- A runbook RAG system uses fixed 500-token chunks with no overlap. Users who ask for a multi-step recovery procedure get answers that stop at step 4 of 9, and retrieval logs show the top hit ends mid-procedure. What is the most targeted fix?