RXed AI News

We x-ray the AI industry.
@RXed_EU
Fw Deployment

Frameworks & harnesses

Agent = Model + Harness. The scaffolding decides whether the agent works.

Deployment · OrchestrationScored on 57/62 audited toolsMedian 7/10Best Claude / Anthropic 9

A model answers questions; an agent does work, and everything between the two is harness — tools, the loop, memory, error handling, permissions. LangChain moved 52.8% to 66.5% on Terminal-Bench 2.0 with the model held fixed, purely by changing prompts, tooling and middleware. That is the whole argument: you rarely have a model problem, you have a harness problem.

You need this when

  • You have a loop with memory — multi-step work, tool calls, retries — not a single prompt.
  • You need audit trails, permission gates and human approval before an agent touches money or customers.
  • More than one person maintains the automation, so the logic must live somewhere other than a prompt in someone's head.

You can skip it when

  • It is one prompt in, one answer out. Call the model API directly — a framework adds tokens and dependencies for nothing.
  • You are prototyping to see if the idea works at all. Write 200 lines yourself first; port to a framework only if it survives.
  • The vendor's 'agent builder' is a drag-and-drop wrapper with no export. Use n8n or a plain SDK so your logic stays portable.

The long version

What it is — in one coffee-break

A model answers questions. An agent does work. Everything between those two things is the harness: the tool definitions, the loop that decides which tool to call and when to stop, the memory that survives between steps, the retry logic when an API returns a 500, and the permission gate that stops it emailing 4,000 customers at 3am. RXed writes this as Agent = Model + Harness. The element Fw covers both ends of that scaffolding — the heavy end (SDKs, visual builders, orchestration graphs) and the thin end (three hundred lines you wrote yourself). The applicability test is blunt: does the vendor ship a builder, SDK or harness for composing AI systems? A chat box plus an API key does not count.

How much of the outcome actually lives in the harness? LangChain reported moving from 52.8% to 66.5% on Terminal-Bench 2.0 — a jump of 13.7 points — with the model held constant, purely from changes to system prompts, tooling and middleware (explainX, May 2026). Same brain, better cockpit. That is why swapping to a newer model rarely rescues an agent that keeps failing.

When you actually need it (and when you don't)

The useful rule of thumb from practitioners: use a raw SDK for a chain, use a real harness when there is a loop with memory. One document in, one summary out, no state — call the API directly and skip the dependency tree. A quote agent that reads an email, checks stock, prices three options, asks a human to approve and then writes to your ERP — that is a loop with memory, and hand-rolling all of it means rebuilding retries, tracing and permissions badly.

Be honest about churn. Heavy frameworks age fast. The 12-factor agents argument is that teams hit roughly 70-80% quality quickly, then have to reverse-engineer the framework's own prompts and control flow to get further — and often rewrite. Abstraction is not free either: one review found a hand-written RAG pipeline used about 2.7x fewer tokens than the LangChain equivalent for the same task (agentailor, July 2026). A thin harness plus a good model beats a fat framework more often than vendors admit.

How to recognize good vs bad implementations

Good harnesses make the loop visible and the failures boring. You can see every tool call, its arguments, its result and how many times it retried. Tools are narrow and typed, not one do_anything() function. Permissions are explicit — some actions run automatically, some require a human click. Context is managed, not just appended until it overflows. The Claude Agent SDK is a fair reference shape: built-in tools, hooks, subagents, sessions, MCP connections and a permission layer, all inspectable.

Three questions that expose a weak demo. Show me the full trace of one run — every tool call, every retry, every token. Break a tool on purpose right now: what does the agent do when the CRM returns a 500? Export my agent's logic and show me what I keep if I cancel next month. A vendor who can only replay a happy path, or whose logic cannot leave their UI, is selling you a demo and a lock-in, not a harness.

What this costs

Open-source and self-hosted is genuinely cheap. n8n Community Edition is free software with unlimited executions; a suitable VPS runs roughly $4-7 (about €4-6) per month (No Code MBA, June 2026). You pay in your own sysadmin time and in model tokens. n8n Cloud is $20/month annual for 2,500 executions, $50/month for 10,000, and $667/month for 40,000 — and when you hit the cap, workflows stop immediately. SDKs like the Claude Agent SDK and LangGraph cost nothing as libraries; you pay for tokens and for observability (LangSmith or Langfuse).

The managed end is a different order of magnitude. Sierra runs custom enterprise contracts with annual commitments around $150,000 plus $50,000-$200,000 implementation and three to seven month deployments; Decagon is roughly a $50,000 platform fee plus about $0.50 per resolution; Fin charges $0.99 per resolution with no platform fee; Agentforce charges $2.00 per conversation regardless of outcome (witn, June 2026). For a 30-person company, the honest path is self-hosted n8n or a thin SDK harness at under €50/month all-in, and only move up when volume or compliance forces it.

Where to see it scored

RXed audited 62 tools, 57 of them scored on this element. The top of the table is Sierra (9), Gemini / Google (9) and Claude / Anthropic (9), with n8n (8.5) just behind and by far the cheapest way in. The bottom is where the element does its real work: Cognism (2.5) and Midjourney (2) are fine products that ship almost no composable harness — you cannot build a system on them, only use them. That is not a criticism of the tool, it is a warning about the architecture diagram you were about to draw.

Flashcards

Check yourself

1. LangChain's Terminal-Bench 2.0 jump from 52.8% to 66.5% came from what?

+13.7 points with the model held constant. The harness carried the gain — the clearest case for Agent = Model + Harness.

2. You need a single summary generated from one document, once. Best choice?

That is a chain, not a loop with memory. Frameworks add tokens and dependencies with no benefit at this shape.

3. According to the 12-factor agents argument, where do frameworks typically fail?

Teams reach 70-80% fast, then have to dig into the framework's own prompts and control flow to go further — often rewriting.

4. Roughly what does self-hosted n8n Community Edition cost to run?

Community Edition is free software with unlimited executions; the real cost is a small VPS and your own maintenance time.

5. Which is the strongest signal of a bad harness in a vendor demo?

No per-step trace means no debugging, no audit and no way to prove what the agent did. That is the first thing to demand.
5 questions · nothing is tracked, nothing is sent anywhere.

Cheat sheet

  • Agent = Model + Harness. Swapping models rarely fixes a broken harness.
  • Chain? Raw SDK. Loop with memory? Framework or a deliberate thin harness.
  • Frameworks get you to 70-80%; the last 20% means reverse-engineering them.
  • Same model, better harness: +13.7 pts on Terminal-Bench 2.0 (LangChain, 2026).
  • Heavy abstraction costs tokens — one hand-written RAG used ~2.7x fewer.
  • n8n self-hosted: free software, €4-7/mo VPS, unlimited runs. Start there.

Who actually does this well

Best on this elementScoreWhy it scored that
Claude / Anthropic9Agent SDK (Python/TS) with hooks, subagents, skills and plugins, plus the hosted Managed Agents harness — Anthropic's explicit thesis is that the harness is the product.
Gemini / Google9The open-source, model-agnostic Agent Development Kit plus Agent Engine, Agent Garden samples and an Agents CLI — a full build-deploy-govern harness.
Sierra9Agent SDK (code) plus Agent Studio 2.0 (no-code Journeys, Workspaces, versioned releases) is a real harness with GitHub-style collaboration and instant rollback.
Decagon8.5AOPs plus Duet plus Agent Workbench form a genuine natural-language-and-code scaffolding system with Git versioning and CI/CD integration — arguably Decagon's core architectural co
n8n8.5The product IS a harness: visual canvas plus real JS/Python code nodes, sub-workflows, error workflows, Git-based version control (Business+), queue-mode scaling and an AI Workflow
Base448The product is itself a full harness: GitHub-backed codebase access, workspace-level custom OpenAPI integrations, per-app backend functions, and a visual IDE for direct edits along
Clay8Claygent Builder is a genuine harness (prompts, variables, output schemas, test suites, versioning, deploy-once-propagate-everywhere) and the Agent plugin ships an API and CLI for
Glean8Agent Builder (no-code, conversational and headless modes), a documented Agent Development Lifecycle, public APIs and an MCP Gateway exposing 2,000+ governed tools through one endp

And the other end of the same column:

WeakestScoreWhy it scored that
Synthesia4Python and Node.js SDKs plus a REST API exist for programmatic video generation, but there is no orchestration harness, builder or SDK for composing AI systems.
Sora (OpenAI)3A clean Videos API with official SDKs and a Batch tier, but both models are flagged Legacy in OpenAI's own model pages and the endpoint shuts down on 24/09/2026. The score reflects
Cognism2.5A browser extension and a Data-as-a-Service API/bulk-delivery pipe move data in and out, but there is no SDK, builder or agent harness for composing AI systems on top of Cognism.
Midjourney2No public API, SDK or builder — a deliberate 'humans, not pipelines' choice, but every serious competitor (FLUX, OpenAI, Ideogram) ships a production API and third-party wrappers v

Scored on 57 of 62 audited tools. Every score links to the full audit and its reasoning.