Bifrost
The fastest self-hosted AI gateway you can run, and the numbers hold up: 11 microseconds of added latency at 5,000 requests per second, one Go binary, Apache-2.0, MCP client and server in the free tier. It covers 23 providers where LiteLLM covers 100+, and guardrails, clustering, RBAC and audit logs are all behind a quote-only enterprise licence, so check your provider list and your compliance list before you fall for the benchmark.
PRICING
| OSS | Free forever | Apache-2.0, self-hosted; gateway, governance primitives, MCP gateway, caching, observability |
| Enterprise | Custom | quote-only, 14-day trial; guardrails, clustering, adaptive load balancing, SSO/RBAC, audit logs, vault, VPC and air-gapped deployment |
Nothing about Bifrost is metered per token: it never sits between you and your provider invoice. What the pricing page does not price is the enterprise tier, which is where every compliance-grade control lives. If you need guardrails, audit logs, RBAC or SSO, budget for a sales conversation and treat the free tier as the developer edition.
checked 2026-09-06 · vendor pricing page
Element scores
Strengths
The performance claim survives independent testing, which is rare. The vendor publishes 11 microseconds of gateway overhead at a sustained 5,000 RPS on t3.xlarge; one engineer who measured it himself on his own hardware landed at tens of microseconds at p50 and concluded the honest cost was the network hop, not the code, and a second measured roughly 0.08ms on c5.xlarge while disagreeing about whether it matters. The free tier is unusually generous for an Apache-2.0 project: MCP client and server, Code Mode, semantic caching, virtual keys with hierarchical budgets, the prompt repository, Prometheus and OTLP all ship without a licence. A six-week factory pilot near Bologna, written up by someone stating no commercial relationship with the vendor, reported weekly VLM spend down from EUR 4,640 to EUR 1,920 on a 41% semantic cache hit rate, 112 outage minutes down to zero, and per-station cost attribution they did not have before. One Go binary that deploys on a small VM is a real advantage in an OT network that cannot reach a hosted dashboard.
Honest dings
23 providers against LiteLLM's 100+ is the first filter that fails, and one production evaluator dropped Bifrost at exactly that step because some of his providers simply were not supported. The performance argument is also eroding: LiteLLM shipped a Rust gateway path benchmarked at 0.05ms overhead and 6,782 RPS, which closes the single-instance gap the pitch depends on. Everything a regulated buyer needs — guardrails, clustering, RBAC, SSO, audit logs, log exports, air-gapped deployment — is enterprise-only and quote-only, and one competitor comparison notes Bifrost itself is not SOC 2, HIPAA or ITAR certified while a hosted rival is. Deep tracing and evaluation route to Maxim AI, a separate product on a separate bill. 975 open issues on 7,820 stars is a young project moving fast. And the semantic cache has a real failure mode: the factory pilot logged two false-negative defect reports traced to a cache hit on a near-identical SKU before they added a variant ID to the cache key.
Sources (12) — every claim traceable
Every audit lists the research it rests on — transparency and traceability are the product. Tools evolve: each audit is a snapshot of its audit date, and re-audits supersede older versions (kept below for reference).
- github.com/maximhq/bifrost — Official repository, measured live 06/09/2026 via the GitHub API: 7,820 stars, 1,166 forks, 975 open issues, 30 watchers, 169 contributor pages, created 19/03/2025, last push 05/09/2026, not archived, Apache-2.0. Source of the vendor's own one-line self-description and of the 5,000 RPS overhead table (11 microseconds on t3.xlarge, 59 on t3.medium, 100% success rate) (accessed 2026-09-06)
- getmaxim.ai/bifrost/pricing — Official pricing page, read 06/09/2026. Primary source for the OSS/Enterprise split: free forever under self-hosting with MCP gateway, Code Mode, semantic caching, virtual keys, prompt repository, Prometheus and OTel; enterprise custom-priced with a 14-day trial adding guardrails, clustering, adaptive load balancing, SAML/OIDC, vault, RBAC, audit logs, log exports and federated-auth MCP (accessed 2026-09-06)
- docs.getbifrost.ai/overview — Official documentation overview. 20+ providers behind a unified API, 11 microseconds of overhead at 5,000 RPS, and the free-tier feature list including budget and rate limits, MCP tool filtering, semantic caching, Prometheus and OTLP (accessed 2026-09-06)
- docs.getbifrost.ai/features/semantic-caching — Official docs. A vector store is required for both caching modes; Redis/Valkey, Weaviate, Qdrant and Pinecone are the supported backends; Qdrant, Pinecone and Weaviate fail in direct-only mode; caching only activates on a request carrying a cache key (accessed 2026-09-06)
- docs.getbifrost.ai/providers/supported-providers/op… — Official per-provider operation matrix. Evidence that embeddings, TTS, transcription, image generation/edit/variation, files, batch, video generation and context compaction are all proxied for OpenAI — and, read against the Anthropic and xAI pages, that unsupported operations return UnsupportedOperationError rather than being silently dropped (accessed 2026-09-06)
- docs.getbifrost.ai/deployment-guides/config-json/st… — Official docs. The three stores a deployment needs: config_store on SQLite or PostgreSQL, logs_store with optional S3/GCS offload, and vector_store on Weaviate, Redis/Valkey, Qdrant or Pinecone for semantic caching (accessed 2026-09-06)
- dev.to/marcorinaldi_ai/six-weeks-of-bifrost-in-a-fa… — Independent six-week production pilot, 27/05/2026, author states no commercial relationship. Weekly VLM spend EUR 4,640 to EUR 1,920 at a 41% semantic cache hit rate, 112 outage minutes to zero, per-virtual-key cost attribution, and the two false-negative defect reports caused by a cache hit on a near-identical SKU (accessed 2026-09-06)
- dev.to/marcuswwchen/the-latency-tax-of-an-llm-gatew… — Independent measurement, June 2026. In-process overhead in the tens of microseconds at p50 and under a millisecond at p99, with the honest conclusion that the real cost is the extra network hop; also the note that semantic caching is kept off for eval determinism (accessed 2026-09-06)
- dev.to/paultwist/litellm-vs-bifrost-i-tested-both-i… — Independent 50/50 production traffic split on c5.xlarge at 200-400 RPS, 23/06/2026. Measured roughly 0.08ms Bifrost overhead against 7-8ms for the LiteLLM Python proxy, dropped Bifrost on the 23-vs-100+ provider filter, found no cost-based routing option, and reports LiteLLM-Rust at 0.05ms and 6,782 RPS (accessed 2026-09-06)
- truefoundry.com/blog/bifrost-alternative-mcp-gateway — Competitor comparison, used for the compliance ceiling rather than the verdict: Bifrost self-hosted only with no managed option, and not SOC 2, HIPAA or ITAR certified, with deeper tracing and evals routing to Maxim AI as a separate product and billing relationship (accessed 2026-09-06)
- flotorch.ai/blogs/llm-gateway-comparison-2026 — Third-party 2026 gateway comparison table. Independent confirmation of native MCP server support, built-in semantic caching and guardrails, four-tier hierarchical budgets, and the two stated weaknesses: 20+ providers against LiteLLM's 100+, and less mature workspace isolation and SSO/SCIM than established players (accessed 2026-09-06)
- mcpmanager.ai/blog/best-mcp-gateway-ai-ml-teams — Third-party MCP gateway round-up, updated 27/04/2026. Confirms the single-binary LLM-plus-MCP gateway shape, virtual keys carrying per-consumer budgets, rate limits and tool-level access controls, and states plainly that the open-source core is Apache-2.0 while enterprise features require a commercial agreement (accessed 2026-09-06)