Api Design hard
Your API serves both batch data pipelines and real-time user requests. Batch jobs consume all rate limit quota, starving real-time requests. How do you ensure fairness?
- A Establish prioritized processing queues with isolated rate limit buckets for each request class, ensuring critical real-time traffic has reserved capacity during high batch load.
- B Horizontally scale the global API server cluster and increase total request throughput quotas to accommodate the combined peak load of both real-time and background workloads.
- C Implement a mandatory 'cooperative throttling' policy where batch pipeline clients are expected to self-limit their request frequency based on the server's CPU utilization metrics.
- D Deploy a time-based scheduling policy that restricts batch processing jobs to low-traffic off-peak hours, using a centralized cron service to coordinate the activation of batch API keys.
Answer: A. Establish prioritized processing queues with isolated rate limit buckets for each request class, ensuring critical real-time traffic has reserved capacity during high batch load.
To prevent low-priority batch jobs from starving high-priority real-time traffic, you should implement **Quality of Service (QoS)** mechanisms like weighted fair queuing or separate rate limit buckets. For example, reserve 80% of capacity for real-time traffic and let batch jobs use the remaining 20% (plus any unused real-time capacity). Increasing global limits (B) or blocking (D) are crude solutions that waste capacity or degrade UX.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- You're designing a REST API for a blog. Which endpoint best represents 'get all comments for post 123'?
- A client wants to update only the 'status' field of an order, leaving other fields unchanged. Which HTTP method is most appropriate?
- A client calls POST /users with valid data, and the user is created. What status code should the server return?
- A client retries a failed 'POST /payments' request due to network timeout. The server receives the request twice. Without idempotency handling, what can happen?
- An API returns 10,000 products. Returning all in one response is impractical. Which pagination approach is simple and stateless?
- Your API serves both mobile apps and third-party integrations. Which authentication method is most appropriate?