allsrc.dev
Go back

Your Tool Definitions Are Prompts, And Most Are Bad Ones

Every other pattern in this series guards the boundary around a tool. This one is about the tool itself, because a badly designed tool cannot be governed into being useful. No before_tool hook fixes a function that returns 40,000 tokens, and no system prompt fixes an error message that says 400.

Here is what a tool layer looks like when it starts as a thin wrapper over an existing REST API — which is how nearly all of them start:

get_data(id, status, type, format, page, sort, filter) -> str   # "Gets data"
list_invoices(customer) -> str                                 # returns all 40 rows
get_invoice(id) -> str
get_invoice_details(id) -> str

Run the auditor over that and you get 30 findings, 6 of them errors.

TL;DR

Four mistakes, in order of what they cost you:

  1. Unbounded responses. The single most expensive mistake in agent engineering, and invisible in code review because the function looks fine. Measured on 40 rows of a realistic API payload: ~3,530 tokens raw, ~350 shaped to three fields, ~201 shaped and paginated. 17× cheaper for the same answer.
  2. Errors that are not instructions. Error: 400 teaches the model nothing, so it tries variations until your budget guard stops it. Unknown status 'pending'. Use one of: open, paid, overdue. is repaired on the next turn.
  3. Free-form parameters where a closed set exists. status: str invites "in progress", "In-Progress", "WIP". An enum makes the wrong call unrepresentable rather than merely discouraged — the same principle as fail-closed policy, applied to arguments.
  4. Fragmented tools forcing multi-call choreography. list then get then get_details is three round trips and three chances to lose the thread.

Runnable code: patterns/harness/tool_design/

Run it like a linter

uv run python -m patterns.harness.tool_design.demo
=== BEFORE: a thin wrapper over the REST API ===
30 finding(s), 6 error(s):
  [error] get_data: thin-description — 9 characters. The description is the only
          thing telling the model when to reach for this...
  [warn]  get_data: unconstrained-enum — 'status' is a free-form string but almost
          certainly has a closed set of values...
  [error] get_data: unbounded-response — no declared bound on response size...
  [warn]  get_invoice, get_invoice_details, list_invoices: fragmented-tools —
          3 separate read tools for 'invoice'...

=== AFTER: tools designed for a model to use ===
No findings.

=== what unbounded, unshaped responses actually cost ===
  raw API payload, 40 rows:        ~ 3,530 tokens
  shaped to 3 fields:              ~   350 tokens  (10x cheaper)
  shaped + first page of 20:       ~   201 tokens  (17x cheaper)

The pattern

from patterns.harness.tool_design.pattern import audit, report
print(report(my_registry))          # run it in CI, like a linter

And the helpers for writing tools that pass:

@registry.tool(
    "Search invoices for a customer, optionally filtered by status. Returns up to "
    "20 invoices with id, total and status, newest first, plus a cursor for the "
    "next page. Use this instead of listing everything.",
    bounded_response=True, actionable_errors=True, schema_documented=True,
)
def search_invoices(customer_id: str, status: str = "open", offset: int = 0) -> str:
    if status not in ("open", "paid", "overdue"):
        return actionable_error(f"Unknown status {status!r}.",
                                "Use one of: open, paid, overdue.",
                                example="status='overdue'")
    page = paginate(matching, offset=offset, limit=20)
    ...

Note what the good description does that the bad one does not: it says what comes back and when to prefer this tool. The model cannot plan the next step without knowing the shape of the result, so if you do not tell it, it calls the tool to find out — and you pay for that turn.

When to use it

When NOT to use it

Trade-offs and failure modes

Frequently asked questions

How long should a tool description be?

Long enough to say what it does, when to prefer it over alternatives, and what it returns — typically two to three sentences. The failure mode is not verbosity, it is omission: a nine-character description like “Gets data” is the actual problem.

Should I return JSON or prose from a tool?

Shaped lines, usually. Raw JSON spends tokens on braces, nulls, and nested objects nobody will use — that is most of the 10× in the measurement above. Return the fields the agent needs, compactly, and keep JSON for when the structure genuinely matters.

How does this relate to MCP?

Directly: an MCP server is a tool layer someone else wrote, and the audit is how you find out what it will cost you before you mount it. The fragmentation and unbounded- response findings are extremely common in third-party servers.

Does this matter less with cheaper models or bigger windows?

It matters differently. Cheaper tokens reduce the cost penalty but not the reliability penalty — a model given 40,000 tokens to answer a three-row question is more likely to lose the thread, whatever the price per token.

References

Part of the agent harness and governance series. Next: agent evaluations — why scoring the final answer passes an agent that leaked your credentials.

Share this post:
Share this post on LinkedinShare this post on XShare this post on ThreadsShare this post on WhatsappShare this post on Facebook
Previous Post
"I've Fixed The Failing Test" Is A Claim, Not A Completion Signal
Next Post
Output Scoring Passes The Agent That Leaked Your Credentials