Skip to content
allsrc.dev
Go back

Your Framework Choice Matters Less Than You Think

I have been making a claim throughout this series: your choice between agent frameworks will not determine whether you have an incident, and the governance layer will.

That is easy to assert and worth testing. So I took the four flagship patterns and mounted them on two real frameworks — LangGraph and Microsoft Agent Framework — with their own dependencies, their own test suites, and one hard constraint: the pattern objects must be imported unchanged.

They were. Here is what that took, and the two footguns I hit on the way.

TL;DR

LangGraphMicrosoft Agent Framework
Bridge code~200 lines (mount.py)~200 lines (middleware.py)
Patterns modifiedzerozero
ALLOWexecute inside the tool nodeawait call_next()
DENYappend a ToolMessage the model readsset context.result, return without call_next
PAUSEinterrupt() — durable via the checkpointerapproval request as the result
HALTroute to ENDraise MiddlewareTermination()
Biggest footgunthe interrupted node is replayed on resumeMiddlewareTermination is HALT, not DENY

Both test suites assert parity: the denial strings match core.Harness byte-for-byte, because they come from the same code.

assert envelope_denial(langgraph_denials) == envelope_denial(reference_denials)

Runnable: adapters/

The same output, three runtimes

The same poisoned-ticket scenario on the reference harness, on LangGraph, and on Agent Framework produces the same audit trail — including the same approval fingerprint, because the same HITLApprovalGate computed it:

  proposed read_ticket(ticket_id='T-666')
  executed read_ticket (result a9930db4f4eb6eea)
  proposed notify_customer(to='[EMAIL]', body='API_KEY=[KEY]')
  DENIED send_email — tool 'send_email' is outside the capability envelope for this run
  proposed execute_sql(query='DELETE FROM orders WHERE id = 90311')
  DENIED execute_sql — write operations are not permitted (DELETE detected)
  PAUSED issue_refund — awaiting human approval: amount 4900 exceeds the auto-approve
                        limit of 100 (ref 35ce961b27a628bd)
  chain verification: True (chain intact)

To add a pattern to either adapter you write no framework code — you append it to the hooks list. That is the whole return on doing this.

Footgun 1: MiddlewareTermination is HALT, not DENY

The natural reading of Agent Framework’s middleware docs is that you deny a tool call by setting context.result and raising MiddlewareTermination.

That terminates the entire agent loop. The model never sees the denial, never re-plans, and your run ends early — which looks like a governance success and is actually a broken agent. You will ship it, because the tool did not execute and the test you wrote checks exactly that.

To deny one call and let the run continue, set the result and return without calling call_next():

if decision.action == "deny":
    context.result = f"DENIED by policy: {decision.reason}"
    return                       # no call_next, no exception

There is a test pinning it: tool a is denied, tool b still executes afterwards, and the run reaches its final message. Reserve MiddlewareTermination for genuine HALT — budget exhaustion, kill switches — where ending the run is the intent.

Footgun 2: a chat client must mix in FunctionInvocationLayer

Subclassing only BaseChatClient produces a warning — “does not support function invoking” — and no tool call ever reaches your middleware. The governance layer silently does nothing.

class FakeChatClient(FunctionInvocationLayer, BaseChatClient):  # both, in this order

This is the highest-severity footgun in the set, because everything looks fine until you check whether the tool actually ran. A warning in a log is not an adequate signal for “your entire policy layer is bypassed.”

Footgun 3: LangGraph replays the interrupted node

On resume, LangGraph re-executes the whole node from its checkpoint, so before_tool is called twice for the same tool call. You can see it in the audit trail as PAUSED issue_refund appearing twice.

Patterns that only decide are unaffected. Patterns that record on decide — audit trails, budget metering, thrash counters — must tolerate the repeat. There is a test pinning this so it cannot regress silently, and if you need exact-once accounting, apply the idempotency keying from durable execution to the hook, not just the tool.

Where the frameworks genuinely differ

I said framework choice matters less than you think. Less, not none. One difference is real and worth deciding on:

LangGraph’s interrupt() plus a checkpointer gives you durable human-in-the-loop for free. The parked run is persisted; resuming is Command(resume="approved") against the same thread id. The reference harness needs a replay trick to achieve the same thing, and the approval gate article documents that workaround precisely because most harnesses lack this.

If human approval is central to your product, that is a legitimate argument for LangGraph.

Agent Framework’s counter-argument is that its middleware pipeline is the cleanest mapping to these patterns of anything I tested — the hook points line up almost one-to-one, and it does not replay, so recording hooks need no idempotency work.

What this does and does not prove

Does prove: the patterns are portable, the bridge is small and writable once, and identical governance decisions come out of three different runtimes because the decision logic is shared.

Does not prove: that every pattern ports equally well. I mounted four of eighteen. The before_model patterns — compaction especially — involve message translation that is lossy at the edges, and I would expect more friction there than the tool-boundary patterns had.

Also does not prove that frameworks are interchangeable for other reasons. Streaming, observability integration, deployment story, and multi-agent orchestration differ substantially. My claim is narrow and specific: the governance layer is not where that choice bites you.

Frequently asked questions

Should I use the reference harness in production?

No. It exists so the patterns can be read and tested in isolation — about 150 lines with no dependencies. In production, mount the same patterns on whatever you already run.

Does this work with CrewAI, Google ADK, or the OpenAI Agents SDK?

I have not tested those, so I will not claim it. The requirement is that the framework exposes a seam where you can intercept a tool call before execution and substitute a result. If it does, the mapping is the same shape as the two here. If it does not, that is a genuinely important finding about that framework.

How much work is a new adapter?

Roughly a day, most of it in message translation and in discovering the footguns above. The bridge is about 200 lines and you write it once for all eighteen patterns.

What if my framework calls tool functions directly?

Then you do not have a governance boundary and no pattern in this series can be mounted. That is the single most important thing to check before choosing a framework for an agent that touches anything real: can you intercept a tool call and refuse it?

References


This closes the agent harness and governance series. The open gaps I have not solved — inter-agent delegation, kill switches, tool supply-chain pinning, and streaming output guardrails — are documented in the repository README, and I would rather name them than imply the list is complete.



Previous Post
Twelve Governance Patterns On One Agent: Do They Compose?