LiteLLM
The default open-source AI gateway, and it earns that on reach: 140+ providers, 1,800+ models, the best MCP and A2A surface in the category. But the governance features people adopt it for leak. Sixteen silent-failure modes are documented against public GitHub issues, including a $50 budget that let $764.78 through. Route traffic with it. Do not trust its budget number as your cost control.
PRICING
| Open source | $0 | MIT licence on everything outside the enterprise/ directory, which carries its own commercial licence. Python SDK plus self-hosted proxy. Includes virtual keys, per-key/user/team budgets, spend tracking, RPM and TPM limits, load balancing, fallbacks, retries, exact and semantic caching, guardrail integrations, MCP gateway and A2A agent gateway. Key and budget features require a PostgreSQL database; semantic caching requires Redis. You pay infrastructure and provider tokens, nothing to LiteLLM |
| Enterprise trial | $0 for 30 days | Full Enterprise features, self-hosted, key emailed instantly. No sales call, no credit card. The most honest evaluation path in this category |
| Enterprise Basic | ~$250/mo (third-party estimate, not vendor-published) | SSO, RBAC, audit logs, JWT authorization, Prometheus metrics, custom callbacks, enterprise guardrails, SLA-backed support. Delivered as a Docker licence key. Confirm the figure with the vendor: LiteLLM publishes no price page |
| Enterprise Premium | ~$30,000/yr (third-party estimate, not vendor-published) | Adds SCIM, air-gapped deployment, multi-region control plane, live upgrade assistance, dedicated onboarding, enterprise roadmap access, 24/7 support with Sev 0 at 1 hour, Sev 1 at 6 hours, Sev 2-3 at 24 hours and security patches at 72 hours. SOC 2 Type 2 and ISO 27001 |
Free is real here in a way it usually is not: MIT, no token markup, no seat count, no telemetry, and the governance features most vendors gate (virtual keys, budgets, rate limits, MCP, A2A) are in the open-source build. What you pay for instead is operations. Production LiteLLM is a Python proxy plus Postgres plus Redis plus your own monitoring, and third-party TCO analyses put the run cost around €280-640/month in infrastructure before a single engineer-hour, which for a small team exceeds the Enterprise Basic licence itself. Two numbers to price honestly before you commit: the budget enforcement you are adopting this for is documented as unreliable under concurrency, so treat the provider invoice as the source of truth and keep a hard cap at the provider; and the Rust migration lands the router on 15/09/2026 and the full server on 01/12/2026, so a deployment started now will cross a runtime change inside a quarter. Euro figures converted from published USD at roughly 0.92.
checked 2026-09-02 · vendor pricing page
Element scores
Strengths
Reach, and it is not close. 140+ providers and 1,800+ models behind one OpenAI-compatible key, day-zero support when a model ships, and the same key reaching Bedrock, Azure, Vertex, a self-hosted vLLM and Ollama on a laptop. Netflix's staff engineer says on the record that new models reach their users within a day of release and it saved his team months; AT&T's VP of the Data Office is quoted at up to 56% cost reduction on some coding tasks. The licence is MIT on everything outside enterprise/, which is cleaner than the Elastic-style licences its self-hosted rivals use, and the features vendors normally paywall (virtual keys, budgets, RPM and TPM limits, spend tracking, semantic caching, MCP, A2A) are in the free build. The protocol work is the standout: an MCP Gateway with one endpoint for every tool, permissions by key, team and organisation, all three transports, a REST route that calls tools without a model in the loop, OAuth 2.0 discovery with dynamic client registration and PKCE by default, AWS SigV4 for Bedrock AgentCore servers, protocol version 2025-11-25 with SEP-986 name enforcement on new servers, and an A2A agent gateway that fetches, curates and re-serves agent cards with virtual keys and spend applied. Nobody else in the open-source tier is close on that. The Rust migration is being run the right way, one function at a time behind the existing config, database and API, with a published route-by-route timeline and a reproducible benchmark harness in the repo rather than a slide. And the reasoning-model handling deserves credit for naming the Anthropic thinking-blocks incompatibility with OpenAI-shaped clients instead of pretending it away.
Honest dings
The governance layer fails open. Theory Delta published sixteen silent-failure modes in March 2026, every one traced to a public GitHub issue with a reproduction, and the pattern is consistent: the feature does not throw, it just does not work. Budget and rate-limit counters use unsynchronised read-modify-write, so under concurrency increments are lost. A user configured a $50 budget and reached $764.78 in actual spend with no alert and no exception (#12977). Five concurrent requests against a 100 TPM limit consumed 663 tokens because all five read current_tpm=0 (#18730). Team-scoped virtual keys skip user-level budget enforcement entirely (#12905), and pass-through routes are not budget-tracked at all (#10750). Guardrails are worse, because the failure is a compliance failure: nine distinct guardrail bugs in one issue including intermittent passport and SSN masking (#19637), UI-configured guardrails that never execute (#15584), post_call skipped on passthrough (#20270), and Bedrock guardrails whose blocked output is computed and then discarded in favour of the original input (#22949). The same unsynchronised counter breaks the cooldown mechanism, so providers that should be marked unhealthy keep receiving traffic (#20977). Staging passes because staging is not concurrent. Production does not. Beyond that: 4,871 open issues against 57,772 stars is a real signal about maintenance load, not a badge of popularity; two of the fallback bugs above are closed not-planned rather than fixed; there is no evaluation product at all, so quality measurement is always somebody else's tool; and Enterprise pricing appears nowhere on the vendor's site, so every published figure in this audit is a third-party estimate that has to be confirmed in writing.
Sources (21) — every claim traceable
Every audit lists the research it rests on — transparency and traceability are the product. Tools evolve: each audit is a snapshot of its audit date, and re-audits supersede older versions (kept below for reference).
- github.com/BerriAI/litellm — Official repository, measured live 02/09/2026 via the GitHub API: 57,772 stars, 11,084 forks, 4,871 open issues, 233 watchers, created 27/07/2023, last push 02/09/2026, not archived. Source of the vendor's own one-line self-description naming the Rust core, and of the 4,871-open-issue figure used in the dings (accessed 2026-09-02)
- github.com/BerriAI/litellm/blob/main/LICENSE — Official licence file and the basis for the open_source tier call: MIT on all content outside the enterprise/ directory, which carries its own separate commercial licence. Explains why the GitHub API reports the licence as NOASSERTION rather than MIT (accessed 2026-09-02)
- theorydelta.com/findings/llm-gateway-silent-failures — The single most important independent source in this audit, published 01/03/2026 and last verified 22/03/2026 against LiteLLM 1.55+. Sixteen silent-failure modes, every one linked to a public GitHub issue with a reproduction or maintainer confirmation: budget overshoot $50 to $764.78 (#12977), 6.6x TPM overshoot from unsynchronised counters (#18730), team-key budget bypass (#12905), untracked pass-through routes (#10750), nine guardrail failures (#19637), UI guardrails never invoked (#15584), model-level guardrails attached after pre_call_hook (#18363), post_call skipped on passthrough (#20270), Bedrock guardrail output discarded (#22949), 0% vs 80% cache hits through the proxy (#18219), cache_hit metric always zero (#6229), load balancing defeating cache warmup (#6784), cascading fallback failure (#17729), cooldown race condition (#20977), mid-stream fallback injecting a default system prompt (#18229, no fix planned), multimodal fallback dropping images (#15803, closed not-planned). Drives the Guardrails, Tracing and Small-models scores and the verdict (accessed 2026-09-02)
- litellm.ai/enterprise — Official Enterprise page verified live 02/09/2026. Full Enterprise feature list (SSO plus SCIM, OIDC/JWT, secret managers and key rotation, RBAC by key/team/org, air-gapped and multi-region deployment, live upgrade assistance), SOC 2 Type 2 and ISO 27001, published support SLAs (Sev 0 one hour, Sev 1 six hours, Sev 2-3 24 hours, vulnerability 72 hours), and the instant 30-day trial key with no sales call. Confirms no price is published anywhere on the vendor's site (accessed 2026-09-02)
- litellm.ai — Official homepage verified live 02/09/2026. Source of the 140+ providers and 1,800+ models figure, the day-zero model support claim, and every named-customer quote used in this audit: NVIDIA (Ajay Dogra, 100+ model endpoints), Netflix (David Leen, models within a day of release, months of work saved), Okta (Dennis Henry, backend model swap as a config update), Lemonade (Mark Koltnuk), AT&T (Mark Austin, VP Data Office, up to 56% cost reduction on some advanced AI tasks) (accessed 2026-09-02)
- docs.litellm.ai/docs/proxy/enterprise — Official Enterprise features documentation: which capabilities sit behind the licence (SSO for the admin UI, audit logs with retention, JWT auth, public/private route control, secret managers, IP-based ACLs, max request and file size, enforced params, key rotation, team-based logging, GDPR logging opt-out per team, tag budgets, GCS and Azure Blob log export, /spend/report). Basis for splitting the Tracing and pricing narratives between free and paid (accessed 2026-09-02)
- docs.litellm.ai/docs/mcp — Official MCP Gateway documentation and the primary evidence for the 9/10 Protocols score: one fixed endpoint for all MCP tools, list/call/prompts/resources, /mcp-rest/tools/list and /mcp-rest/tools/call for model-free curl testing, Streamable HTTP plus SSE plus stdio transports, permissions by key, team and organisation, OAuth 2.0 Authorization Server Discovery with dynamic client registration and PKCE by default plus manual credential and URL overrides, AWS SigV4 signing for Bedrock AgentCore, static headers and server variables. Protocol version 2025-11-25 from v1.80.18, with SEP-986 server-name compliance enforced on new servers and warnings on legacy ones (accessed 2026-09-02)
- docs.litellm.ai/docs/a2a — Official Agent Gateway (A2A) documentation: supported agent providers (A2A, Vertex AI Agent Engine, LangGraph, Azure AI Foundry, Bedrock AgentCore, Pydantic AI), logging, load balancing, streaming and iteration budgets, agents declarable in config.yaml for read-only ConfigMap deployments, message/send and message/stream routed through LiteLLM's own A2A client so logging, guardrails and spend apply while other methods forward upstream (accessed 2026-09-02)
- docs.litellm.ai/docs/a2a_agent_card — Official A2A agent card documentation: LiteLLM fetches the upstream card from /.well-known/agent-card.json, lets the operator curate which skills and fields are exposed, serves the curated card at /a2a/{agent_id}/.well-known/agent.json and accepts JSON-RPC 2.0 invocation at /a2a/{agent_id}, with protocol version 1.0 or 0.3 selectable per agent. Field-by-field support table against A2A v1.0 §4.4 (accessed 2026-09-02)
- docs.litellm.ai/docs/proxy/guardrails/quick_start — Official guardrails documentation: the full integration catalogue (Presidio with per-entity actions and confidence thresholds, Aporia, Lakera, Cato, Pillar, generic guardrail API), pre_call / during_call / post_call modes including lists, and the system-message and tool-message skip flags. Establishes how wide the configured surface is, which is what makes the documented enforcement failures consequential (accessed 2026-09-02)
- docs.litellm.ai/docs/fine_tuning — Official fine-tuning documentation and the basis for that score: OpenAI-compatible /fine_tuning/jobs and /files endpoints, explicitly marked Enterprise-only, supported providers limited to OpenAI, Azure OpenAI and Vertex AI, cost tracking flagged amber with 'let us know if you need this' (accessed 2026-09-02)
- docs.litellm.ai/docs/reasoning_content — Official reasoning-content documentation, source of the Thinking-models score: reasoning_effort and standardised reasoning_content across DeepSeek, Anthropic, Bedrock, OpenAI Responses, Vertex, OpenRouter, xAI, Google AI Studio, Perplexity, Mistral Magistral and Groq, plus the candid architecture table explaining why Anthropic extended thinking with tool calling breaks on OpenAI-compatible clients and the modify_params workaround (accessed 2026-09-02)
- docs.litellm.ai/blog/litellm-rust-launch — Official Rust migration announcement by CTO Ishaan Jaffer, last updated June 2026. Four-stage plan and the dated route-by-route timeline used in this audit: litellm.ocr() by 15/08/2026, /messages and /chat/completions by 01/09/2026, the router (load balancing, fallbacks, retries, cooldowns) by 15/09/2026, the full axum server by 01/12/2026. Same config.yaml, database, client API and providers throughout (accessed 2026-09-02)
- docs.litellm.ai/blog/rust-ai-gateway-benchmarks — Official AIGatewayBench results, July 2026, treated as a vendor benchmark: Rust at about 0.7ms p99 added latency and 21.8MB peak memory against Portkey 2.3ms/90.4MB, Bifrost 4.5ms/199.1MB and LiteLLM Python 257.7ms/329.5MB, with all four pointed at the same deterministic local mock and the harness published for reproduction. Cited as a vendor claim with a reproducible method, not as an independent result (accessed 2026-09-02)
- truefoundry.com/blog/litellm-pricing-guide — Independent (competitor-published, so read with that bias) pricing and TCO analysis, 2026. Source of the Enterprise Basic ~$250/month and Enterprise Premium ~$30,000/year figures that the vendor does not publish, and of the operational-cost breakdown ($300-700/month infrastructure plus DevOps time) behind the pricing note. Corroborated on the numbers by two further sources below (accessed 2026-09-02)
- seaflux.tech/blogs/explore-litellm-effortless-ai-pr… — Independent 2026 feature and pricing guide corroborating the same Enterprise Basic $250/month and Premium ~$30,000/year figures, the AWS and Azure Marketplace procurement route, and the semantic-caching mechanism (Redis backend, vector similarity, tuneable threshold) used in the Small-models score (accessed 2026-09-02)
- api7.ai/litellm-vs-vercel-ai-gateway — Independent 2026 comparison and the third corroborating source on licensing mechanics: MIT core including proxy, virtual keys, per-key/user/team budgets, spend tracking, exact and semantic caching and the MCP gateway, with PostgreSQL required for key and budget features; SSO free up to five users; larger SSO, SCIM, audit logs and enterprise guardrails behind the paid licence; deployment by Docker, Kubernetes/Helm or Terraform (accessed 2026-09-02)
- kosmoy.com/resources/blog/litellm-alternatives — Independent competitive analysis, July 2026, notable for handling vendor benchmark claims properly: flags Maxim AI's '50x faster than LiteLLM' Bifrost claim as vendor-run with no independent benchmark existing, and confirms the Rust migration is staged August to December 2026 targeting sub-1ms overhead. Also corroborates that LiteLLM's OSS core includes more than rival OSS cores and that its MCP gateway with production OAuth 2.0 is ahead of Bifrost's (accessed 2026-09-02)
- braintrust.dev/articles/ai-gateway-comparison-2026 — Independent (competitor-published) 2026 gateway comparison placing LiteLLM as the open-source pick for teams with DevOps capacity, and confirming what the free plan includes versus what Enterprise adds (cloud or self-hosted deployment, support, custom SLAs, JWT auth, SSO, audit logs) (accessed 2026-09-02)
- docs.litellm.ai/docs/providers — Official provider index, used to verify the breadth claims behind the LLM, Embeddings and Multimodal scores rather than take the marketing number on trust: OpenAI, Azure, Vertex, Google AI Studio, Anthropic (including tool search for 10,000+ tool catalogues), Bedrock, SageMaker, Cohere, Deepgram, ElevenLabs, Fal AI, Black Forest Labs image generation and editing, Groq, Fireworks, DeepInfra, Databricks, Cerebras, LangGraph, Docker Model Runner, Lemonade and many more, plus the JSON-file route for adding OpenAI-compatible providers (accessed 2026-09-02)
- pypi.org/project/litellm — PyPI registry checked 02/09/2026 to confirm the published package version matches the GitHub release tag: litellm 1.99.0, corroborating the 01/09/2026 release date and an actively shipping project (accessed 2026-09-02)