RXed AI News

What it is — in one coffee-break

Every serious vendor now ships a ladder, not one model. At the top is the flagship from the demos; below it sit smaller siblings trained to do most of the work for a fraction of the price. Google runs from Gemini 3.1 Pro at $2.00 per million input tokens and $12.00 output down to Gemini 3.5 Flash-Lite at $0.30 and $2.50. Mistral runs from Mistral Large at $2/$6 down to Ministral 3B at $0.04 both ways — a fiftyfold spread inside one catalogue.

The second half of this element is the part your accountant cares about: some small models ship as open weights you can download and run on hardware you already own. Quantisation is what makes that practical — the model's numbers are rounded from 16 bits to 4, cutting memory to roughly a quarter for a small accuracy loss. That is why an 8-billion-parameter model is about 5 GB on disk and runs on a laptop.

When you actually need it (and when you don't)

You need the cheap tier when volume is high and each task is narrow: classifying inbound email, pulling fields off invoices, tagging tickets, drafting first-pass replies. Push those down the ladder and the bill falls five- to fiftyfold with no measurable quality loss, because the task never needed frontier reasoning. You need local models when data cannot leave the building — an open-weight model on your own box is the only setup where the data never left is a fact rather than a contract clause.

You do not need them for work that decides something. On 2026 management-task comparisons, local 8B models scored about 2.8-3.3 out of 5 against 4.8 for a frontier model. Agents are worse: benchmarks put Llama 3.3 70B near 97% well-formed tool calls and Gemma 4 27B near 95%, while sub-7B models emit malformed calls on multi-step plans — and at 95% per call an eight-step workflow finishes about 66% of the time. Long context is the other cliff; frontier models already shed 30-60 points of multi-fact retrieval past 200K tokens, and small ones fall off sooner.

How to recognize good vs bad implementations

A good implementation makes the tier a choice you control: a visible model selector, published per-tier pricing, one API shape across tiers so switching is a single string, and at best open weights you can pull down yourself. A bad one hides routing behind a smart toggle, bills a blended rate, and quietly serves the cheap model on your most important prompt.

Ask three things on the demo. Which tier served this exact response, and can you show me that in the response metadata? If I pin the cheapest tier to my highest-volume task, what is my monthly bill — in writing? Do you publish per-tier numbers for tool calling and long-context retrieval, or only for the flagship? Silent routers deflect the first, vendors with no real ladder bundle their way out of the second, and almost nobody survives the third — how they fail there tells you how honest the rest of the pitch is.

What this costs

Cloud, cheap tier: Ministral 3B at $0.04 per million both ways, Mistral Small at $0.10 in and $0.30 out, Gemini 3.5 Flash-Lite at $0.30 and $2.50. A shop pushing 20M input and 5M output tokens a month pays roughly $19 on Flash-Lite against about $100 on the Pro tier for identical traffic; batch endpoints cut another 50%.

Self-hosted: licence €0. A used RTX 3060 12 GB in a PC you already own costs €150-200 and runs a quantised 8-12B model. A Mac mini with 32 GB of unified memory is about €1,200 and gives 55-82 tokens/second on Qwen3 8B at Q4_K_M, or 32-52 on Gemma 3 12B; a 27B model wants 48 GB and a €3,000 Mac Studio. Electricity is the line people forget: a laptop-class machine drawing ~100W for eight hours a day is roughly 200 kWh, or €60 a year at €0.30/kWh, while a 650W GPU rig is nearer €390. Do the arithmetic first — €1,200 of hardware against a €17 monthly API bill takes about six years to pay back. Self-hosting wins on data residency and predictability, not on price.

Where to see it scored

Of 62 tools audited, 39 carry a score here. Gemini / Google (9) leads on the clearest published ladder and per-tier pricing. Mistral AI (8.5) matches it on substance and beats everyone on the self-host path, since its small models ship under Apache 2.0. Decagon (8.5) is the applied case — tier choice exposed inside a vertical product. Krea (8) earns its score for offering real speed and cost tiers instead of one price. At the bottom, Gamma (5) and Base44 (5) give you one opaque model and one bill — fine until your volume triples. Check any vendor's cost story against Google's published price list.