Discover great AI tools
📢 NoticeWebsite launched:DeepToSeek AI tools directory is live. Submit great tools and get featured!
Home / Open Source Radar / Inference & Training / litellm

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

Stars63.4k
Forks13.4k
LanguagePython
LicenseMIT

About this project

litellm is an open-source inference and training project on GitHub: The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]. It is infrastructure rather than a finished product: the layer that serves, quantises, trains or orchestrates models. It is maintained by BerriAI and published as a public repository you can read, fork and run yourself.

On GitHub the repository is tagged ai-gateway, anthropic, azure-openai, bedrock, gateway, langchain, litellm, llm, llm-gateway, llmops, mcp-gateway and openai, which is a fair summary of what it covers. That generally means an API surface, batching and memory management, and often an OpenAI-compatible endpoint so existing clients keep working unchanged.

It is written mainly in Python and released under no standard SPDX license identifier — the repository uses a custom or modified license, so read it before commercial use. It is actively maintained: the most recent commit landed on November 17, 2026.

To try it, clone the repository and follow its README; that file is the canonical install guide. Because it runs on your own infrastructure, budget time for environment setup: dependencies, model weights and configuration usually take longer than the first successful run.

Key facts as recorded on GitHub: 63,414 stars; 13,354 forks; primary language Python; last commit on November 17, 2026; topics ai-gateway, anthropic, azure-openai, bedrock, gateway, langchain.