All Questions
Llm Gateway Cost And Caching easy

Your primary LLM provider is fully down for an hour, and no tested fallback exists for your agent feature. The product also has an FAQ answer feature backed by a semantic cache. What is the best degradation mode?

  1. A Queue all requests indefinitely and replay them once the primary provider recovers.
  2. B Retry each failed call in a tight loop until the primary provider starts responding.
  3. C Serve cached or reduced answers where possible and show a degraded-mode notice.
  4. D Send all traffic to whichever provider is up and assume the prompts behave the same.

One system design question a day, with spaced repetition so the ones you miss come back.

Get the app

Related questions