Agents And Tool Use medium
An agent's `search_logs` tool sometimes returns around 200K tokens of matching lines. Those results overflow the context window and crash the run. Which approach best balances completeness and stability?
- A Return the full output but instruct the model to read only the first part, relying on it to ignore the remaining log lines.
- B Truncate results to a few thousand tokens with a note on what was cut, and offer pagination or filter params for more.
- C Automatically switch to a model with a larger context window whenever a tool result exceeds the current model's limit.
- D Silently drop any tool result over the limit and return an empty success so the agent loop continues without interruption.
Answer: B. Truncate results to a few thousand tokens with a note on what was cut, and offer pagination or filter params for more.
**Bounded results plus a way to ask for more** keep the context stable and let the model narrow its search, at the cost of extra steps when it needs deeper data. Switching to a larger model (C) postpones the overflow and raises cost on every later step that carries the huge result. Silent dropping (D) is worse, because the model reasons from a false empty result. Returning the full output is reasonable only when results are reliably small.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- You are defining a 'schedule_meeting' tool for an agent. Which schema decision most reduces wrong or malformed calls?
- A model emits three parallel tool calls: `get_inventory`, `get_price`, and `get_shipping`. To cut latency, the host appends each result to the transcript as it arrives and immediately re-invokes the model. Runs start failing with malformed-request errors, and some produce hallucinated shipping costs. What should the host do?
- A team splits a coding agent into planner, coder, tester, and reviewer agents, and each receives the full transcript of the previous agent. The expected quality gains did not appear. Cost tripled, and the reviewer often fixates on approaches that earlier agents abandoned. When is a multi-agent split actually justified?
- A support agent should remember each customer's preferences across months of sessions. The team appends every past session summary to the system prompt. After six months, prompts exceed 80,000 tokens, latency has doubled, and the agent sometimes ships to an address the customer replaced weeks ago. What redesign fixes both problems?
- A teammate proposes splitting a single coding agent into five agents, arguing that more agents means smarter results. When is a multi-agent split actually justified?
- A data-analysis agent runs model-generated Python inside the same container as your API service. That container has database admin credentials in environment variables. A security review flags this as critical. What is the best mitigation?