BricksLLM
🔒 Enterprise-grade API gateway that helps you monitor and impose cost or rate limits per API key. Get fine-grained access control and monitoring per user, application, or environment. Supports OpenAI, Azure OpenAI, Anthropic, vLLM, and open-source LLMs.
Tech stack & purpose
BricksLLM is a cloud-native API gateway written in Go that helps organizations deploy large language models in production by providing enterprise-grade monitoring, cost control, and access management. It natively supports OpenAI, Anthropic, Azure OpenAI, vLLM, and other LLM providers, enabling features such as per-user rate limiting, cost tracking and limits, PII detection and masking, request caching, failover capabilities, and granular access control. The project was part of Y Combinator's 2022 batch and uses PostgreSQL and Redis for its backend infrastructure, with optional integrations for Datadog monitoring and AWS for PII detection.
Community & reference links
Languages
Contributors
- Spike Lu688 commits
- Donovan So14 commits
- Lei Lei7 commits
- XiangWang6 commits
- Andrew Rothstein5 commits
- Minho Ryang4 commits
- 0xdaydream4 commits
- Phuong Le3 commits
- James Hoekzema2 commits
- Donald Buczek1 commits
- Ikko Eltociear Ashimine1 commits
- Mark Percival1 commits
- Tomasz Wysocki1 commits
- Christoph Thiede1 commits
- Amirhesam Adibinia1 commits