Agents And Tool Use medium
Your agent platform implements approval gates by having the worker thread poll an approvals table until a human clicks Approve. Median approval time is 12 minutes. Under load, the worker pool is exhausted and new agent runs queue for a long time. What is the right redesign?
- A Increase the worker pool size and polling interval so that waiting agents consume fewer resources while blocked on humans.
- B Persist the agent's transcript and pending call, release the worker, and resume the loop from that checkpoint on approval.
- C Let the model continue executing subsequent steps optimistically while approval is pending, rolling back if it is denied.
- D Set a short approval timeout so that requests without a quick human response are auto-approved and the worker is freed.
Answer: B. Persist the agent's transcript and pending call, release the worker, and resume the loop from that checkpoint on approval.
Human approval takes minutes, so the loop should **suspend to durable state** and resume on an event instead of holding a worker. A bigger pool (A) only postpones exhaustion, because blocked workers scale with pending approvals rather than with actual compute. The cost is checkpointing and resume logic. Blocking is acceptable only when the gate is automated and resolves in milliseconds.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- You are defining a 'schedule_meeting' tool for an agent. Which schema decision most reduces wrong or malformed calls?
- A model emits three parallel tool calls: `get_inventory`, `get_price`, and `get_shipping`. To cut latency, the host appends each result to the transcript as it arrives and immediately re-invokes the model. Runs start failing with malformed-request errors, and some produce hallucinated shipping costs. What should the host do?
- A team splits a coding agent into planner, coder, tester, and reviewer agents, and each receives the full transcript of the previous agent. The expected quality gains did not appear. Cost tripled, and the reviewer often fixates on approaches that earlier agents abandoned. When is a multi-agent split actually justified?
- A support agent should remember each customer's preferences across months of sessions. The team appends every past session summary to the system prompt. After six months, prompts exceed 80,000 tokens, latency has doubled, and the agent sometimes ships to an address the customer replaced weeks ago. What redesign fixes both problems?
- A teammate proposes splitting a single coding agent into five agents, arguing that more agents means smarter results. When is a multi-agent split actually justified?
- A data-analysis agent runs model-generated Python inside the same container as your API service. That container has database admin credentials in environment variables. A security review flags this as critical. What is the best mitigation?