Agents And Tool Use hard
A model emits three parallel tool calls: `get_inventory`, `get_price`, and `get_shipping`. To cut latency, the host appends each result to the transcript as it arrives and immediately re-invokes the model. Runs start failing with malformed-request errors, and some produce hallucinated shipping costs. What should the host do?
- A Keep streaming each result back as it lands, tagging each with its call ID so the model can match results to requests on its own turn by turn.
- B Execute the calls sequentially in emitted order so results arrive in a predictable sequence and the transcript stays consistent.
- C Apply a short timeout and return only the results that finished in time, omitting slow calls so the turn is never blocked.
- D Run them concurrently with per-call timeouts, then return all three results in one turn, keyed by call ID, with errors as explicit results.
Answer: D. Run them concurrently with per-call timeouts, then return all three results in one turn, keyed by call ID, with errors as explicit results.
Parallel tool calls form one model turn, so every call ID needs a matching result before the model runs again. The cost is that the turn waits for the slowest call, and per-call timeouts bound that wait. Returning a timeout as an explicit error result keeps the pairing intact and tells the model the data is missing. Silently omitting the result invites the model to guess the shipping cost, and sequential execution is only right when calls depend on each other's outputs.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- You are defining a 'schedule_meeting' tool for an agent. Which schema decision most reduces wrong or malformed calls?
- A team splits a coding agent into planner, coder, tester, and reviewer agents, and each receives the full transcript of the previous agent. The expected quality gains did not appear. Cost tripled, and the reviewer often fixates on approaches that earlier agents abandoned. When is a multi-agent split actually justified?
- A support agent should remember each customer's preferences across months of sessions. The team appends every past session summary to the system prompt. After six months, prompts exceed 80,000 tokens, latency has doubled, and the agent sometimes ships to an address the customer replaced weeks ago. What redesign fixes both problems?
- A teammate proposes splitting a single coding agent into five agents, arguing that more agents means smarter results. When is a multi-agent split actually justified?
- A data-analysis agent runs model-generated Python inside the same container as your API service. That container has database admin credentials in environment variables. A security review flags this as critical. What is the best mitigation?
- An internal agent has grown to 120 tools, many with overlapping names like search_docs, find_docs, and query_docs. Wrong-tool errors are rising. What is the best next step?