← All repos

litellm

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

GitHubdocs.litellm.ai/docs/
ai-gatewayanthropicazure-openaibedrockgatewaylangchainlitellmllmllm-gatewayllmopsmcp-gatewayopenaiopenai-proxyrustrust-aivertex-ai
Browse cluster: Claude AI Agent Frameworks & MCP Tools →
42,848commits
1,572contributors
15languages

Tech stack & purpose

LiteLLM is an AI gateway built with a Rust core and Python SDK that enables calling over 100 large language model APIs through a unified OpenAI-compatible interface. The project addresses the challenge of managing multiple LLM providers by offering cost tracking, guardrails, load balancing, and logging across services like Bedrock, Azure, OpenAI, Anthropic, VertexAI, vLLM, and Nvidia NIM. The repository includes practical examples such as a cookbook demonstrating how to use Claude's Agent SDK with LiteLLM as a proxy, and a benchmarking tool for comparing LLM response quality, cost, and latency across different providers and models.

Community & reference links

Languages

Python
84.5%
TypeScript
12.6%
HTML
1.5%
Rust
0.6%
HCL
0.2%
Go
0.2%
Shell
0.1%
JavaScript
0.1%
CSS
0.0%
Dockerfile
0.0%
Go Template
0.0%
Makefile
0.0%
PLpgSQL
0.0%
PowerShell
0.0%
Ruby
0.0%

Contributors (top 30 of 1,572)