Rag And Retrieval hard
A multi-tenant enterprise RAG product retrieves the top-20 chunks from a shared index and generates an answer. It then strips any citation the user cannot access before returning the response. A security review flags this design. What is the actual flaw, and what is the fix?
- A The model can spoof citations, so the fix is to sign each citation ID server-side and verify the signatures before rendering the response.
- B The filter runs too late for latency reasons. Moving the same ACL check to a cache layer in front of the response makes it both safe and fast.
- C The shared index is the problem. The only safe fix is a physically separate vector index per user, so ACLs never need evaluating at query time.
- D The model already read restricted chunks and can paraphrase them. ACLs must be enforced as a filter inside the retrieval query, before generation.
Answer: D. The model already read restricted chunks and can paraphrase them. ACLs must be enforced as a filter inside the retrieval query, before generation.
Stripping citations hides the source but not the content, because restricted text has already shaped the answer. Permissions must constrain **retrieval itself**. Per-user indexes are safe, but they multiply storage and ingestion cost. ACL metadata filters, often combined with per-tenant partitions, are the usual middle ground. Fully separate indexes are justified only when compliance demands hard physical isolation.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- You must index 100M chunks using a 3,072-dimension embedding in fp32, served from RAM via HNSW. The raw vectors alone take about 1.2 TB before graph overhead. You need to cut memory several-fold while losing at most a point or two of recall@10. Which approach fits?
- A multi-tenant SaaS app stores all tenants in one vector index with a tenant_id field. Queries fetch the top 20 chunks by similarity and then drop chunks from other tenants. Small tenants often get 0-2 results. Why does this happen, and what is the fix?
- Runbooks are split into fixed 300-token chunks with no overlap. When a procedure step starts near the end of one chunk and finishes in the next, answers come back incomplete. What is the lowest-cost fix?
- Your offline eval shows recall@5 of 92% on a labeled set, so the gold passage is in the context for nearly every question. Yet human graders mark 35% of answers wrong, and the wrong answers often ignore or contradict the gold passage. Where should the team invest next?
- Your product requires every RAG answer to include clickable source citations that users can verify. How should those citations be produced?
- A runbook RAG system uses fixed 500-token chunks with no overlap. Users who ask for a multi-step recovery procedure get answers that stop at step 4 of 9, and retrieval logs show the top hit ends mid-procedure. What is the most targeted fix?