All Questions
Ai Coding And Product Systems medium

Your inline code completion client sends a request on every keystroke. GPU load is spiking during busy hours, and users sometimes see suggestions that reflect text they typed several characters ago. What should you change?

  1. A Fire a request on every keystroke but queue them strictly in order, so each suggestion reflects every intermediate state of the file.
  2. B Debounce briefly, cancel in-flight requests when the user keeps typing, and reuse cached suggestions whose prefix still matches.
  3. C Raise the debounce delay to several seconds so far fewer requests are sent, accepting that suggestions appear only after long pauses.
  4. D Move completion to a larger model with higher batch throughput so the extra keystroke requests are absorbed without adding servers.

One system design question a day, with spaced repetition so the ones you miss come back.

Get the app

Related questions