All Questions
Rag And Retrieval medium

To 'never miss anything,' a team raised the number of chunks sent to the generator from 5 to 40. Answer accuracy dropped slightly, while latency and token cost rose about 6x. What best explains the accuracy drop?

  1. A Extra low-relevance chunks add noise that distracts the model and dilutes attention, while every chunk costs tokens.
  2. B Reranker scores are less reliable past rank 10, so the best chunks end up ordered lower in the context.
  3. C Retrieval latency rose, so more requests hit the generator timeout and returned partial answers.
  4. D Forty chunks pushed the question out of the model's recent-token window, so it attended only to the last chunks.

One system design question a day, with spaced repetition so the ones you miss come back.

Get the app

Related questions