Rag And Retrieval hard
After moving ACL enforcement into retrieval, you implement it by fetching the top 100 from an HNSW index and then dropping chunks the user cannot see. Users who can access only about 0.5% of the corpus often get zero results. What is the most robust fix?
- A Raise the fetch size to the top 2,000 before dropping unauthorized chunks, so restricted users still have enough survivors after the filter.
- B Apply the ACL predicate during the ANN search itself (filtered traversal or pre-filter), and partition by tenant for highly selective users.
- C Skip filtering for low-privilege users and instead instruct the model, in the system prompt, to ignore any chunks outside their permissions.
- D Rebuild HNSW with higher efConstruction and M values, so the graph's improved connectivity surfaces more authorized chunks in the top 100.
Answer: B. Apply the ACL predicate during the ANN search itself (filtered traversal or pre-filter), and partition by tenant for highly selective users.
**Post-filtering** is secure, but recall collapses under highly selective filters: at 0.5% access, only about 0.5 of the top 100 survive on average. Fetching 2,000 helps somewhat, at a large latency cost, and still fails for narrower users. It is acceptable only when filters are mildly selective. Tuning the graph does not change the fact that the true nearest neighbors are mostly unauthorized, so the filter must act during the search.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- You must index 100M chunks using a 3,072-dimension embedding in fp32, served from RAM via HNSW. The raw vectors alone take about 1.2 TB before graph overhead. You need to cut memory several-fold while losing at most a point or two of recall@10. Which approach fits?
- A multi-tenant SaaS app stores all tenants in one vector index with a tenant_id field. Queries fetch the top 20 chunks by similarity and then drop chunks from other tenants. Small tenants often get 0-2 results. Why does this happen, and what is the fix?
- Runbooks are split into fixed 300-token chunks with no overlap. When a procedure step starts near the end of one chunk and finishes in the next, answers come back incomplete. What is the lowest-cost fix?
- Your offline eval shows recall@5 of 92% on a labeled set, so the gold passage is in the context for nearly every question. Yet human graders mark 35% of answers wrong, and the wrong answers often ignore or contradict the gold passage. Where should the team invest next?
- Your product requires every RAG answer to include clickable source citations that users can verify. How should those citations be produced?
- A runbook RAG system uses fixed 500-token chunks with no overlap. Users who ask for a multi-step recovery procedure get answers that stop at step 4 of 9, and retrieval logs show the top hit ends mid-procedure. What is the most targeted fix?