Ai Coding And Product Systems medium
Your AI code review system supports team-specific guidelines, and one customer has about 800 rules covering style, security, and framework conventions across several languages. Putting all of them in every prompt is costly, and the model applies Python rules to Go files while ignoring most of the others. What should you do?
- A Put all 800 guidelines into every review prompt, since a long context window lets the model pick the rules that apply to each diff.
- B Drop the custom guidelines entirely and rely on the model's general knowledge of best practices, which covers most team conventions.
- C Fine-tune a review model on the guideline document so it internalizes all 800 rules and needs no rules in the prompt at review time.
- D Index the rules, retrieve the few relevant to each chunk's language, paths, and APIs, and cite the matching rule in each comment.
Answer: D. Index the rules, retrieve the few relevant to each chunk's language, paths, and APIs, and cite the matching rule in each comment.
**Rule retrieval** trades some risk of missing an applicable rule for focused prompts and comments that cite their rule, which helps keep false positives low. You can mitigate the recall risk by always including a small set of critical rules. Stuffing every rule into the prompt (A) is tempting with long context windows, but it dilutes attention and causes exactly the cross-language misapplication described. Putting all rules in the prompt is fine when the rule set is only a dozen or so entries.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- Your assistant uses a classifier to route queries to a small or large model, which saves 60% of GPU spend. Analysis shows about 20% of hard queries are misrouted to the small model, which then gives confident wrong answers, and regenerate clicks on those threads have doubled. What should you do?
- Your workspace AI retrieves the top-k chunks from a shared vector index and passes them to the model with ACL metadata and a system prompt saying 'only use documents this user can access.' A user just received an answer quoting a private HR page they cannot open. What is the correct fix?
- Your support bot answers billing questions. One user has asked the same refund question three times, and retrieval confidence for that topic is low. What should the bot do next?
- Your AI review system receives a 4,000-line pull request spanning 60 files. Sending the whole diff in one prompt yields shallow, generic comments, and real bugs in files near the middle of the diff are missed. How should you structure the review?
- In your ChatGPT-style assistant, power users keep single threads running for weeks, reaching about 150K tokens. Cost per turn keeps climbing, latency is rising, and the model increasingly misses facts from early in the thread. What should you do?
- Your workspace AI filters retrieval using ACLs stored in the vector index, which is rebuilt nightly. An admin removes an employee from a confidential project page, but for the rest of the day the employee can still get AI answers drawn from it. What is the right fix?