Aqueduct AI Gateway is an actively developing project designed to create a self-hosted AI gateway with robust features. Built on the LiteLLM Router SDK and utilizing Django, it aims to streamline AI request routing without reimplementing existing APIs. The gateway supports local deployment via Docker Compose, setting up necessary services like Django, PostgreSQL, and an authentication mock OIDC provider.
Key features include a user-friendly administrative interface for managing teams and organizations, a gateway server for relaying requests to various LLM providers, and detailed usage tracking. Future enhancements target role-based access control (RBAC), advanced metrics, and improved request handling capabilities, along with database optimizations for performance.
The project encourages contributions and provides extensive documentation, focusing on continuous testing and feature expansion, such as API rate limitations and model-specific access controls. For those preferring a simpler solution without user self-service, Envoy AI Gateway is recommended.
Source link