RXed AI News

Opik (Comet)

Apache-2.0 with the whole product in the free self-host and the cheapest managed tier in the category at 19 dollars, and the only platform here that will block a PII leak before it reaches a user, as long as you self-host, because the guardrails run nowhere else.

PRICING

Three things to model before you commit. First, the free self-host is real and unusually complete: Apache-2.0 across the board including the Agent Optimizer SDK, unlimited spans, unlimited retention, unlimited members, and it is the only place the guardrails run at all. Second, the metering unit on cloud is the span, not the request, so an agent that makes six tool calls per user action burns six spans, and a chatty agent will hit the 25,000-span free tier in days. Third, Pro is listed per user per month and cloud retention defaults to 60 days, so the two costs that grow are headcount and history: a team of ten is $190 a month before overage, and pushing 100,000 spans to 400-day retention costs $29 on top of the $5 for the spans themselves.

checked 2026-08-14 · vendor pricing page

Element scores

Strengths

Opik is the most honest open-source deal in this category, and the licence is the reason. Apache-2.0 across the whole repository including the Agent Optimizer SDK, with unlimited spans, unlimited retention and unlimited team members on the self-host, and no feature flags to untangle before you find out what you actually got. Evaluation is deep and free everywhere: 30+ built-in LLM-as-judge and heuristic metrics, custom Python metrics, test suites with assertions that produce a plain pass or fail for regression testing, annotation queues with a purpose-built reviewer UI for subject-matter experts who are not engineers, thread-level conversation evaluation, and online evaluation rules that score live traffic as it lands. The Agent Optimizer is the genuine differentiator and nobody else here ships it: eight or more algorithms including GEPA, MIPRO, few-shot Bayesian and a hierarchical reflective optimizer that root-causes failed outputs, tuning not just prompts but tool and MCP schemas from traces you already logged. Guardrails are the second differentiator and they are the real thing rather than a scoring dashboard, blocking PII and off-topic content in the request path with a raised exception before your application can ship the response. Prompt management quietly beats the nearest competitor too, with a red-green version diff and deploy-to environment labels that an independent hands-on test could not find in Langfuse. Add multimedia logging on every tier, a native OpenTelemetry path, 40+ integrations, an MCP server that also points at self-hosted instances, and a company that has been building developer tooling since 2017 rather than since the last funding cycle.

Honest dings

The operational load is the price of the free licence, and it is heavier than the marketing suggests. The production Helm chart brings MySQL, ClickHouse via the Altinity operator, Redis, MinIO and ZooKeeper, which is five stateful services, one more than Langfuse, and the documented gotchas are real: ClickHouse cluster macros are required even on a single node, high-availability installs have to be done in two phases because the earliest analytics migrations predate cluster-aware DDL and fail on replicas that never received the base schema, and the chart migrated off Bitnami images with a one-shot migration flag you must set and then unset. Free is a licence statement, not an effort estimate. Second, the AI layer is inverted against the people most likely to choose Opik: OpikAssist and Opik Connect are unavailable on the open-source build, so teams self-hosting for data control get the least assisted product, while the guardrails are the exact opposite and run only on self-host and not on any cloud tier. That split is confusing and both halves should be stated plainly at purchase. Third, the guardrails have hard limits that are easy to miss: English only, topic checks capped at 1,024 tokens of input, and a GPU node strongly recommended in production, which is a cost line nobody puts in the pricing table. Fourth, cloud pricing is per user per month with a 60-day retention default, so the two things that grow are headcount and history. Fifth, RBAC and SSO are Enterprise-only, not merely Pro-gated, which is stricter than Langfuse where organisation RBAC and SSO are in the free build. Finally, two measured caveats from independent hands-on testing: prompt templating is flat variable substitution with no conditional logic, and the Python track_openai wrapper added about 102 ms per call in that tester's session. The tester was explicit that this number came from a separate session to their Langfuse measurement and should not be read as a head-to-head, so benchmark it against your own baseline before instrumenting a latency-sensitive path.