Agents And Tool Use hard
An invoice pipeline processes 10,000 invoices a day, always with the same four steps: extract fields, validate totals, look up the vendor, post to the ledger. It runs as a ReAct loop in which the model chooses each next step. Cost and latency vary widely, and about 1 in 500 runs skips validation. What should you change?
- A Keep the ReAct loop but add few-shot examples showing the four steps in order, so the model follows the sequence reliably.
- B Encode the fixed sequence as workflow code, calling the model only for judgment steps like extraction, and keep ReAct for open tasks.
- C Switch to plan-and-execute, where the model drafts the four-step plan up front for each invoice and then executes that plan step by step.
- D Split into four step agents coordinated by an orchestrator agent that decides which specialist should handle each invoice next.
Answer: B. Encode the fixed sequence as workflow code, calling the model only for judgment steps like extraction, and keep ReAct for open tasks.
**ReAct** is worth its cost when the next step depends on what was just observed. A known, fixed path should be deterministic code, so validation can never be skipped and cost per invoice is predictable. The loss is flexibility: unusual invoices need an explicit escalation path instead of improvisation. Few-shot examples lower the skip rate but cannot make it zero, and plan-and-execute still lets the model write a path you already know.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- You are defining a 'schedule_meeting' tool for an agent. Which schema decision most reduces wrong or malformed calls?
- A model emits three parallel tool calls: `get_inventory`, `get_price`, and `get_shipping`. To cut latency, the host appends each result to the transcript as it arrives and immediately re-invokes the model. Runs start failing with malformed-request errors, and some produce hallucinated shipping costs. What should the host do?
- A team splits a coding agent into planner, coder, tester, and reviewer agents, and each receives the full transcript of the previous agent. The expected quality gains did not appear. Cost tripled, and the reviewer often fixates on approaches that earlier agents abandoned. When is a multi-agent split actually justified?
- A support agent should remember each customer's preferences across months of sessions. The team appends every past session summary to the system prompt. After six months, prompts exceed 80,000 tokens, latency has doubled, and the agent sometimes ships to an address the customer replaced weeks ago. What redesign fixes both problems?
- A teammate proposes splitting a single coding agent into five agents, arguing that more agents means smarter results. When is a multi-agent split actually justified?
- A data-analysis agent runs model-generated Python inside the same container as your API service. That container has database admin credentials in environment variables. A security review flags this as critical. What is the best mitigation?