The client SDK retries failed calls 3 times, and the gateway also retries each upstream call 3 times. The provider becomes overloaded and returns 429s and timeouts. What happens, and what is the right fix?

1. A One user request can become up to 16 provider attempts, deepening overload; retry at a single layer with a budget, jitter, and Retry-After.
2. B Retries are harmless because rejected 429 calls are never billed, so the only cost is latency; raising retry counts improves the success rate.
3. C Remove gateway retries and fail over to the cheapest provider on the first error, since a different provider never experiences correlated load.
4. D Add an idempotency key to every call so the provider deduplicates retried prompts and bills them once, which makes layered retries safe.