All Questions
Rag And Retrieval hard

A team retrieves the top 40 chunks and passes all of them to the model because 'the context window can hold it.' Compared with top-8, quality dropped and latency rose, and answers often miss facts from chunks placed mid-context. What assembly strategy is best?

  1. A Keep all 40 but sort them by document date, so the model sees a coherent chronological narrative and can locate facts anywhere in the context.
  2. B Rerank to a small set, drop near-duplicates, and put the strongest evidence at the start or end of a budgeted context.
  3. C Keep all 40 and first summarize each one with a smaller model, so the same breadth of evidence fits in fewer tokens without losing any facts.
  4. D Raise k to 80 and switch to a model with a larger context window, since the quality drop means relevant evidence was still being left out.

One system design question a day, with spaced repetition so the ones you miss come back.

Get the app

Related questions