All Questions
Ai Coding And Product Systems medium

Your answer engine has a fixed latency budget of about 3 seconds and a per-query cost cap. Answers are fluent but often wrong because the supporting sources are weak or off-topic. Where should the budget go?

  1. A Move more of the budget into generation with a larger model, since better reasoning can compensate for weak or partially relevant sources.
  2. B Cut retrieval to the top 3 search results to free latency for longer, more detailed generated answers with richer explanations.
  3. C Spend more on retrieval: fetch 10-30 results, rerank passages, and let a modest model synthesize from the best evidence.
  4. D Skip live search and answer from the model's parametric knowledge, citing sources afterward by searching for pages that match the text.

One system design question a day, with spaced repetition so the ones you miss come back.

Get the app

Related questions