Frameworks & harnesses
Agent = Model + Harness. The scaffolding decides whether the agent works.
A model answers questions; an agent does work, and everything between the two is harness — tools, the loop, memory, error handling, permissions. LangChain moved 52.8% to 66.5% on Terminal-Bench 2.0 with the model held fixed, purely by changing prompts, tooling and middleware. That is the whole argument: you rarely have a model problem, you have a harness problem.
You need this when
- You have a loop with memory — multi-step work, tool calls, retries — not a single prompt.
- You need audit trails, permission gates and human approval before an agent touches money or customers.
- More than one person maintains the automation, so the logic must live somewhere other than a prompt in someone's head.
You can skip it when
- It is one prompt in, one answer out. Call the model API directly — a framework adds tokens and dependencies for nothing.
- You are prototyping to see if the idea works at all. Write 200 lines yourself first; port to a framework only if it survives.
- The vendor's 'agent builder' is a drag-and-drop wrapper with no export. Use n8n or a plain SDK so your logic stays portable.
The long version
What it is — in one coffee-break
A model answers questions. An agent does work. Everything between those two things is the harness: the tool definitions, the loop that decides which tool to call and when to stop, the memory that survives between steps, the retry logic when an API returns a 500, and the permission gate that stops it emailing 4,000 customers at 3am. RXed writes this as Agent = Model + Harness. The element Fw covers both ends of that scaffolding — the heavy end (SDKs, visual builders, orchestration graphs) and the thin end (three hundred lines you wrote yourself). The applicability test is blunt: does the vendor ship a builder, SDK or harness for composing AI systems? A chat box plus an API key does not count.
How much of the outcome actually lives in the harness? LangChain reported moving from 52.8% to 66.5% on Terminal-Bench 2.0 — a jump of 13.7 points — with the model held constant, purely from changes to system prompts, tooling and middleware (explainX, May 2026). Same brain, better cockpit. That is why swapping to a newer model rarely rescues an agent that keeps failing.
When you actually need it (and when you don't)
The useful rule of thumb from practitioners: use a raw SDK for a chain, use a real harness when there is a loop with memory. One document in, one summary out, no state — call the API directly and skip the dependency tree. A quote agent that reads an email, checks stock, prices three options, asks a human to approve and then writes to your ERP — that is a loop with memory, and hand-rolling all of it means rebuilding retries, tracing and permissions badly.
Be honest about churn. Heavy frameworks age fast. The 12-factor agents argument is that teams hit roughly 70-80% quality quickly, then have to reverse-engineer the framework's own prompts and control flow to get further — and often rewrite. Abstraction is not free either: one review found a hand-written RAG pipeline used about 2.7x fewer tokens than the LangChain equivalent for the same task (agentailor, July 2026). A thin harness plus a good model beats a fat framework more often than vendors admit.
How to recognize good vs bad implementations
Good harnesses make the loop visible and the failures boring. You can see every tool call, its arguments, its result and how many times it retried. Tools are narrow and typed, not one do_anything() function. Permissions are explicit — some actions run automatically, some require a human click. Context is managed, not just appended until it overflows. The Claude Agent SDK is a fair reference shape: built-in tools, hooks, subagents, sessions, MCP connections and a permission layer, all inspectable.
Three questions that expose a weak demo. Show me the full trace of one run — every tool call, every retry, every token. Break a tool on purpose right now: what does the agent do when the CRM returns a 500? Export my agent's logic and show me what I keep if I cancel next month. A vendor who can only replay a happy path, or whose logic cannot leave their UI, is selling you a demo and a lock-in, not a harness.
What this costs
Open-source and self-hosted is genuinely cheap. n8n Community Edition is free software with unlimited executions; a suitable VPS runs roughly $4-7 (about €4-6) per month (No Code MBA, June 2026). You pay in your own sysadmin time and in model tokens. n8n Cloud is $20/month annual for 2,500 executions, $50/month for 10,000, and $667/month for 40,000 — and when you hit the cap, workflows stop immediately. SDKs like the Claude Agent SDK and LangGraph cost nothing as libraries; you pay for tokens and for observability (LangSmith or Langfuse).
The managed end is a different order of magnitude. Sierra runs custom enterprise contracts with annual commitments around $150,000 plus $50,000-$200,000 implementation and three to seven month deployments; Decagon is roughly a $50,000 platform fee plus about $0.50 per resolution; Fin charges $0.99 per resolution with no platform fee; Agentforce charges $2.00 per conversation regardless of outcome (witn, June 2026). For a 30-person company, the honest path is self-hosted n8n or a thin SDK harness at under €50/month all-in, and only move up when volume or compliance forces it.
Where to see it scored
RXed audited 62 tools, 57 of them scored on this element. The top of the table is Sierra (9), Gemini / Google (9) and Claude / Anthropic (9), with n8n (8.5) just behind and by far the cheapest way in. The bottom is where the element does its real work: Cognism (2.5) and Midjourney (2) are fine products that ship almost no composable harness — you cannot build a system on them, only use them. That is not a criticism of the tool, it is a warning about the architecture diagram you were about to draw.
Flashcards
Check yourself
1. LangChain's Terminal-Bench 2.0 jump from 52.8% to 66.5% came from what?
2. You need a single summary generated from one document, once. Best choice?
3. According to the 12-factor agents argument, where do frameworks typically fail?
4. Roughly what does self-hosted n8n Community Edition cost to run?
5. Which is the strongest signal of a bad harness in a vendor demo?
Cheat sheet
- Agent = Model + Harness. Swapping models rarely fixes a broken harness.
- Chain? Raw SDK. Loop with memory? Framework or a deliberate thin harness.
- Frameworks get you to 70-80%; the last 20% means reverse-engineering them.
- Same model, better harness: +13.7 pts on Terminal-Bench 2.0 (LangChain, 2026).
- Heavy abstraction costs tokens — one hand-written RAG used ~2.7x fewer.
- n8n self-hosted: free software, €4-7/mo VPS, unlimited runs. Start there.
Who actually does this well
| Best on this element | Score | Why it scored that |
|---|---|---|
| Claude / Anthropic | 9 | Agent SDK (Python/TS) with hooks, subagents, skills and plugins, plus the hosted Managed Agents harness — Anthropic's explicit thesis is that the harness is the product. |
| Gemini / Google | 9 | The open-source, model-agnostic Agent Development Kit plus Agent Engine, Agent Garden samples and an Agents CLI — a full build-deploy-govern harness. |
| Sierra | 9 | Agent SDK (code) plus Agent Studio 2.0 (no-code Journeys, Workspaces, versioned releases) is a real harness with GitHub-style collaboration and instant rollback. |
| Decagon | 8.5 | AOPs plus Duet plus Agent Workbench form a genuine natural-language-and-code scaffolding system with Git versioning and CI/CD integration — arguably Decagon's core architectural co |
| n8n | 8.5 | The product IS a harness: visual canvas plus real JS/Python code nodes, sub-workflows, error workflows, Git-based version control (Business+), queue-mode scaling and an AI Workflow |
| Base44 | 8 | The product is itself a full harness: GitHub-backed codebase access, workspace-level custom OpenAPI integrations, per-app backend functions, and a visual IDE for direct edits along |
| Clay | 8 | Claygent Builder is a genuine harness (prompts, variables, output schemas, test suites, versioning, deploy-once-propagate-everywhere) and the Agent plugin ships an API and CLI for |
| Glean | 8 | Agent Builder (no-code, conversational and headless modes), a documented Agent Development Lifecycle, public APIs and an MCP Gateway exposing 2,000+ governed tools through one endp |
And the other end of the same column:
| Weakest | Score | Why it scored that |
|---|---|---|
| Synthesia | 4 | Python and Node.js SDKs plus a REST API exist for programmatic video generation, but there is no orchestration harness, builder or SDK for composing AI systems. |
| Sora (OpenAI) | 3 | A clean Videos API with official SDKs and a Batch tier, but both models are flagged Legacy in OpenAI's own model pages and the endpoint shuts down on 24/09/2026. The score reflects |
| Cognism | 2.5 | A browser extension and a Data-as-a-Service API/bulk-delivery pipe move data in and out, but there is no SDK, builder or agent harness for composing AI systems on top of Cognism. |
| Midjourney | 2 | No public API, SDK or builder — a deliberate 'humans, not pipelines' choice, but every serious competitor (FLUX, OpenAI, Ideogram) ships a production API and third-party wrappers v |
Scored on 57 of 62 audited tools. Every score links to the full audit and its reasoning.