allsrc.dev
Go back

Pattern: The Tool Privilege Broker

A support agent gets read access to the orders database so it can answer “where is my refund?” Reasonable scope, reviewed, approved, shipped.

Weeks later a customer writes: “I was double-charged, sort it out.”

And the model — doing exactly what it was trained to do — escalates from SELECT to DELETE.

I want to be precise about what happened, because the instinctive reading is wrong. Nothing malicious occurred. The tool was legitimate. The credential was valid. The instruction was a reasonable interpretation of what the customer asked for. The failure is architectural: the only thing standing between a proposed action and an executed action was the model’s judgment.

TL;DR

A tool privilege broker is a deterministic policy engine that sits between “the model asked to run a tool” and “the harness ran it,” answering allow or deny from the principal, the environment, and the arguments — never from the prompt.

Policy that lives in a system prompt is a suggestion. Policy that lives in the broker is a fact. The difference matters most in exactly the situation where you need it: when the model is under prompt injection and is being instructed to ignore your instructions.

OWASP: ASI02 (Tool Misuse and Exploitation), ASI03 (Identity and Privilege Abuse).

Runnable code: patterns/governance/tool_privilege_broker/

The failure, then the fix

uv run python -m patterns.governance.tool_privilege_broker.demo
=== WITHOUT the pattern: the DELETE reaches the database ===
  tool result -> 2 rows: order 90310 ($49), order 90311 ($49) — duplicate charge
  tool result -> OK, statement executed: 'DELETE FROM orders WHERE id = 90311'

=== WITH the pattern: same model, same script — the DELETE is denied ===
  tool result -> 2 rows: order 90310 ($49), order 90311 ($49) — duplicate charge
  tool result -> DENIED by policy: write operations are not permitted (DELETE detected)

Same model. Same trajectory. The only difference is a deterministic policy boundary.

The pattern

broker = ToolPrivilegeBroker([
    PolicyRule(
        tool="execute_sql",
        environments=("dev", "staging", "prod"),
        roles=("support",),
        arg_guards=(read_only_sql,),
    ),
])
harness = Harness(model, tools, hooks=[broker])

The code is about eighty lines. The design decisions are the part worth arguing about.

1. Fail closed. A tool with no rule is denied, not allowed. New tools are inert until someone writes policy for them, because registration is not authorization. This one line prevents the most common real-world regression: somebody adds a tool, forgets the policy, and it ships with implicit full access.

2. Deny in code, not in prompt. “You must never modify data” in a system prompt is advisory. Models under pressure ignore it. Models under injection are told to ignore it. The broker cannot be argued with, because it does not read natural language.

3. Denials are visible to the model. The refusal comes back as a tool result, so the agent re-plans — escalates to a human, tries a read-only route, or reports honestly. Silent refusal produces an agent that stalls or invents success.

4. Policy keys off the principal, not the agent. The rule asks “may this user’s delegated run do this,” which is what stops one over-privileged service account becoming every user’s privileges. That is the confused deputy, and the broker is where the two patterns meet.

5. Argument guards are where the real policy lives. execute_sql being allowed is the easy decision. Whether this SQL is allowed is the interesting one:

def read_only_sql(arguments, key="query"):
    statement = str(arguments.get(key, "")).lower()
    for keyword in ("insert", "update", "delete", "drop", "truncate", "alter", "grant"):
        if keyword in statement.split():
            return f"write operations are not permitted ({keyword.upper()} detected)"
    return None

The repository ships three reusable guards: read_only_sql, max_value for numeric ceilings, and recipient_domain — which restricts who an outbound message may be addressed to. That last one exists because composing all the patterns together revealed a gap: redacting an email body does nothing if an injection gets to choose the recipient. A scrubbed message to an attacker-controlled address is still a channel.

When to use it

When NOT to use it

Trade-offs and failure modes

Frequently asked questions

Why not just use the model’s own refusal behaviour?

Because refusal is a property of the model’s training, not of your architecture, and it changes when you change models. A control you cannot test and did not write is not a control. The broker gives you a deterministic check with a unit test next to it.

Where does this sit relative to my API gateway?

The gateway enforces coarse network and identity policy; the broker enforces per-call, per-argument policy that the gateway cannot see — a gateway cannot tell SELECT from DELETE inside a JSON body without becoming an application. Use both. The gateway is your perimeter, the broker is your semantics.

Should the broker be a separate service?

For a single application, no — in-process is faster, simpler, and easier to test. For a fleet of agents sharing policy, yes, and that is essentially what Microsoft’s Agent Governance Toolkit built with its stateless sub-millisecond policy engine. The pattern is the same either way; only deployment changes.

How do I write argument guards that are not brittle?

Prefer structural constraints over textual ones. An enum on the parameter beats a regex on the value, because it makes the wrong call unrepresentable rather than merely detected — see tool design. Use text screening only where the schema genuinely cannot express the rule.

References

Part of the agent harness and governance series. Next: goal integrity — what to do when the content your agent reads starts giving it orders.

Share this post:
Share this post on LinkedinShare this post on XShare this post on ThreadsShare this post on WhatsappShare this post on Facebook
Previous Post
The Confused Deputy Is In Your Agent Right Now
Next Post
Prompt Injection Defences Are Losing. Bind Capabilities Instead