Ai Coding And Product Systems hard
A workspace AI does RAG over user pages. ACLs are copied into the vector index as metadata at ingest time and re-synced nightly. A user removed from a confidential page can still get answers that quote it 10 minutes later. What fix closes the leak without rebuilding the index on every permission change?
- A Pass the user's group memberships into the system prompt and instruct the model to refuse to quote any document the user cannot access.
- B Re-embed the revoked page immediately so its vectors drift away from that user's likely queries in the embedding space.
- C Move the ACL sync from nightly to hourly and document up to one hour of exposure as an accepted eventual-consistency window.
- D Check every retrieved chunk against the live authorization service at query time, using index ACLs only as a recall prefilter.
Answer: D. Check every retrieved chunk against the live authorization service at query time, using index ACLs only as a recall prefilter.
**Permission enforcement belongs in the retrieval layer.** A query-time check against the source of truth adds latency to every request but makes revocation take effect at once, and the stale index ACL still narrows candidates cheaply. Prompt-based enforcement (A) is the tempting wrong answer: the model sees the forbidden text, and prompt injection or plain error will leak it. Faster sync (C) only shrinks the window, which is acceptable only for low-sensitivity content.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- Your assistant uses a classifier to route queries to a small or large model, which saves 60% of GPU spend. Analysis shows about 20% of hard queries are misrouted to the small model, which then gives confident wrong answers, and regenerate clicks on those threads have doubled. What should you do?
- Your workspace AI retrieves the top-k chunks from a shared vector index and passes them to the model with ACL metadata and a system prompt saying 'only use documents this user can access.' A user just received an answer quoting a private HR page they cannot open. What is the correct fix?
- Your support bot answers billing questions. One user has asked the same refund question three times, and retrieval confidence for that topic is low. What should the bot do next?
- Your AI review system receives a 4,000-line pull request spanning 60 files. Sending the whole diff in one prompt yields shallow, generic comments, and real bugs in files near the middle of the diff are missed. How should you structure the review?
- In your ChatGPT-style assistant, power users keep single threads running for weeks, reaching about 150K tokens. Cost per turn keeps climbing, latency is rising, and the model increasingly misses facts from early in the thread. What should you do?
- Your workspace AI filters retrieval using ACLs stored in the vector index, which is rebuilt nightly. An admin removes an employee from a confidential project page, but for the rest of the day the employee can still get AI answers drawn from it. What is the right fix?