All Questions
Ai Coding And Product Systems hard

Your answer engine has a 3-second end-to-end latency budget for each answer. Today it spends 0.3 s on search, 0.2 s on fetching, and 2.5 s generating with a large model that receives 30 full pages. Users report factual errors. Where should the budget go?

  1. A Move to a larger generator and keep all 30 full pages in context, since generation quality is the main driver of factual accuracy.
  2. B Cut retrieval to the top 3 results and spend the freed budget on a longer reasoning pass with the large generator model.
  3. C Skip page fetching, generate from search snippets only, and use the saved time for a second self-consistency generation pass.
  4. D Spend more on retrieval: rerank passages and extract relevant spans, then give a smaller generator a compact, cited context.

One system design question a day, with spaced repetition so the ones you miss come back.

Get the app

Related questions