Agents And Tool Use hard
An outbound-email agent runs on a pool of 50 async workers. Every customer-facing email requires human approval, and the median approval takes 20 minutes. Under peak load the pool saturates and new tasks queue for hours. What is the right architecture?
- A At the gate, persist the transcript and pending tool call durably, release the worker, and resume the loop when an approval event arrives.
- B Scale the worker pool to match peak concurrency and raise the approval timeout to two hours so blocked tasks are never dropped mid-wait.
- C Replace the human with a reviewer model call inside the same loop so approval resolves in seconds and workers are no longer held open.
- D Send the email optimistically and queue a human review afterward, since reviewers can flag problems and the team can follow up with corrections.
Answer: A. At the gate, persist the transcript and pending tool call durably, release the worker, and resume the loop when an approval event arrives.
Approval takes minutes, so the loop must **suspend and resume** from durable state instead of holding a worker. This costs checkpointing complexity, but capacity no longer depends on how fast reviewers respond. A bigger pool only works when approvals are fast and rare. A reviewer model puts the gate back inside the model, and post-hoc review fails because a sent email cannot be recalled.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- You are defining a 'schedule_meeting' tool for an agent. Which schema decision most reduces wrong or malformed calls?
- A model emits three parallel tool calls: `get_inventory`, `get_price`, and `get_shipping`. To cut latency, the host appends each result to the transcript as it arrives and immediately re-invokes the model. Runs start failing with malformed-request errors, and some produce hallucinated shipping costs. What should the host do?
- A team splits a coding agent into planner, coder, tester, and reviewer agents, and each receives the full transcript of the previous agent. The expected quality gains did not appear. Cost tripled, and the reviewer often fixates on approaches that earlier agents abandoned. When is a multi-agent split actually justified?
- A support agent should remember each customer's preferences across months of sessions. The team appends every past session summary to the system prompt. After six months, prompts exceed 80,000 tokens, latency has doubled, and the agent sometimes ships to an address the customer replaced weeks ago. What redesign fixes both problems?
- A teammate proposes splitting a single coding agent into five agents, arguing that more agents means smarter results. When is a multi-agent split actually justified?
- A data-analysis agent runs model-generated Python inside the same container as your API service. That container has database admin credentials in environment variables. A security review flags this as critical. What is the best mitigation?