Agents And Tool Use hard
A customer-support agent calls a `create_refund` tool. The payment API times out after 30 seconds. The host retries the HTTP request, and separately the model, seeing an error, emits a fresh `create_refund` call on its next step. Some customers are refunded two or three times. Which fix closes both duplicate paths?
- A Pass the model-generated tool_call_id to the payment API as the idempotency key, since it uniquely identifies each requested action.
- B Add a system-prompt rule telling the model to inspect the transcript and never re-issue a refund that might already have succeeded.
- C Have the host derive the key from business intent (task, order, action) so both host retries and model re-emissions dedupe.
- D Disable automatic retries for side-effecting tools and return the timeout to the model as a hard failure it must reason about.
Answer: C. Have the host derive the key from business intent (task, order, action) so both host retries and model re-emissions dedupe.
A timeout is ambiguous because the refund may have succeeded, so every retry path must carry the same **idempotency key**. A key derived on the host from business intent dedupes both the HTTP retry and the model's re-emitted call. A **tool_call_id** changes on every emission, so it only covers host retries. Disabling retries gives up reliability and still fails, because the model itself re-issues the action after it sees an error.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- You are defining a 'schedule_meeting' tool for an agent. Which schema decision most reduces wrong or malformed calls?
- A model emits three parallel tool calls: `get_inventory`, `get_price`, and `get_shipping`. To cut latency, the host appends each result to the transcript as it arrives and immediately re-invokes the model. Runs start failing with malformed-request errors, and some produce hallucinated shipping costs. What should the host do?
- A team splits a coding agent into planner, coder, tester, and reviewer agents, and each receives the full transcript of the previous agent. The expected quality gains did not appear. Cost tripled, and the reviewer often fixates on approaches that earlier agents abandoned. When is a multi-agent split actually justified?
- A support agent should remember each customer's preferences across months of sessions. The team appends every past session summary to the system prompt. After six months, prompts exceed 80,000 tokens, latency has doubled, and the agent sometimes ships to an address the customer replaced weeks ago. What redesign fixes both problems?
- A teammate proposes splitting a single coding agent into five agents, arguing that more agents means smarter results. When is a multi-agent split actually justified?
- A data-analysis agent runs model-generated Python inside the same container as your API service. That container has database admin credentials in environment variables. A security review flags this as critical. What is the best mitigation?