One Redaction Function Is Always Wrong

A billing agent handles a duplicate charge. It looks up the customer record, opens a chargeback with the payment processor, and files a summary ticket with the outsourced helpdesk.

The processor legitimately needs the card number. The helpdesk absolutely does not — but the model has both in context, and “the ticket summary” is just “whatever’s relevant,” so the card and the SSN go out to a third-party SaaS.

The instinctive fix is a global scrubber. That is worse than it looks. Mask the card everywhere and the chargeback breaks, because the processor needs the real value. Mask nothing and you are one helpful summary away from a breach.

The same value has four different correct representations depending on where it is going, and one function cannot produce all four.

TL;DR

Four boundaries, four policies:

The second idea is the one that makes this practical: tokenise rather than destroy. Keep the original in a vault the model cannot reach and give the model a stable token. The agent can then dispute “the card on file” without ever knowing the card — and the harness rehydrates the real value only for tools explicitly authorised to receive that data type.

The model can only leak what it holds. That is a structural property, not a detection race.

OWASP: ASI02, ASI06. Runnable code: patterns/governance/redaction_boundary/

Four boundaries, one record

uv run python -m patterns.governance.redaction_boundary.demo

=== WITHOUT the pattern ===
  payment-processor <- 4111111111111111
  helpdesk-vendor   <- Duplicate charge for card 4111111111111111, SSN 123-45-6789
  ^ the card AND the SSN just left for a third-party helpdesk

=== WITH the pattern ===
  what the MODEL saw:
    Card on file: [CARD:tok_f8a684d7]
    SSN: [SSN:tok_72800d81]

  what each vendor received:
    payment-processor <- 4111111111111111

  DENIED by policy: tool 'file_helpdesk_ticket' is not authorized to receive
  CARD values (argument 'summary' contains a CARD token)

  what the HUMAN sees:    the card ending **** 1111
  what the AUDIT retains: Card on file: [CARD]

  vault rehydrations (each one logged):
    [('[CARD:tok_f8a684d7]', 'open_dispute.card'), ('[CARD:tok_f8a684d7]', 'display')]

The pattern

vault = Vault()
guard = RedactionBoundary(
    vault=vault,
    rehydrate_for={"open_dispute": frozenset({"CARD"})},   # the ONLY card-authorised tool
)

1. Tokenise, don’t destroy. Destroying data makes the agent useless for half of what it is for. Redaction that enables work gets adopted; redaction that blocks it gets disabled by the first engineer under deadline pressure.

2. Rehydration is per-tool and per-kind. open_dispute is authorised for CARD and nothing else — hand it an SSN token and it is denied. Authority is about the pair, not the tool.

3. Tokens are deterministic per value, so the same card reads as the same token across turns and runs. The agent can reason about identity (“this is the card from earlier”) without the value. There is a real trade-off here, below.

4. Every rehydration is logged. vault.reads records which tool dereferenced which token and why. The vault is now the highest-value target in your system; treat its access log as a security feed.

5. Raw values in model-authored arguments are scrubbed, not forwarded. If a card reaches the model by some other path — memory, the user’s own message — the outbound boundary still catches it.

While building this I hit a bug worth mentioning because it is the kind that survives code review: the card detector consumed trailing separators, so 4111111111111111 and 4111111111111111, tokenised differently. Token stability is the whole premise, and a regex quantifier silently broke it. There is now a test named test_tokens_are_deterministic_and_reversible.

When to use it

• Any agent touching regulated data: PCI card data, PHI, PII, credentials.
• Any agent talking to more than one downstream system with different trust levels. That asymmetry is the entire problem.
• Any agent whose transcript is retained, shown to support staff, or fed into evaluations. Those are three more boundaries with three more policies.

When NOT to use it

• Single-boundary systems. If the agent reads and writes exactly one system that already holds the data, tokenising on the way in and rehydrating on the way out is overhead with a new failure mode.
• When you can avoid the data entirely. The strongest version of this pattern is not tokenising the SSN — it is a tool that never returns it. Ask what the agent actually needs before building a vault.
• As a compliance checkbox over a design mistake. If the answer to “why is PHI in the context window?” is “because the retriever returned it,” fix the retriever — see RAG access control.
• Do not tokenise free-text prose wholesale. Detectors on unstructured text have false negatives. Where the schema is known, redact at the source by column, and use detectors only as a backstop.

Trade-offs and failure modes

• Detectors miss things. Every regex here is a floor: international formats, spelled-out numbers, values split across lines. The serious implementation redacts structurally at the data source.
• Deterministic tokens leak equality. Same token means same value, so an attacker who can submit values and observe tokens can confirm guesses. Use per-tenant salts when that matters, and accept losing cross-tenant identity.
• The vault is now the crown jewels. You have concentrated every sensitive value behind one interface. That is an improvement in auditability and a concentration of risk — it needs its own access control, key management, and monitoring, none of which is in the pattern file.
• Tokens occasionally confuse models. Some will try to reformat or explain a token, or claim they cannot proceed. Keep token syntax boring and stable, and mention it in the system prompt.
• Partial masks are still identifying. **** 1111 plus a name plus a postcode is often enough to re-identify someone. “Partial” is a usability choice, not an anonymisation claim.

Frequently asked questions

Is this the same as a PII vault product?

Conceptually yes, and if you have one, use it — Skyflow, Very Good Security, or a KMS-backed store will handle key rotation and access control properly. This pattern is the harness-side half: deciding which boundary each value is crossing and which tools may dereference a token. That decision does not come with the vault.

Why not just avoid putting sensitive data in the context at all?

That is the better answer when you can get it, and I say so above. The reason this pattern exists is that agents often need to act on a value they should not hold — dispute this card, refund this account — and tokenisation is what makes that possible without a redesign of the downstream system.

How does this interact with prompt caching?

Deterministic tokens help: the same record produces the same context bytes, so prefixes stay cacheable. Random per-run tokens would break caching on every run, which is another reason the tokens here are content-derived.

What about data in the model’s output?

Handled at the DISPLAY boundary via for_display(), which turns tokens into partial values for humans. The important part is that the model never held the raw value, so there is far less to catch on the way out — which is the whole argument for doing this inbound rather than as output filtering.

References

• OWASP GenAI Security Project, Top 10 for Agentic Applications 2026 — ASI02, ASI06
• OpenAI, Agentic Governance Cookbook — output-stage guardrails and redaction as a distinct pipeline stage
• Microsoft, Agent Governance Toolkit — policy-enforced data handling at the runtime boundary
• Runnable code and tests: patterns/governance/redaction_boundary/

Part of the agent harness and governance series. Next: RAG access control and provenance — authorising retrieval, and catching an answer that cites a document it never saw.