Introduction
AI Gateway provides a unified interface to manage your organization’s LLM usage. Below are the key features that enhance functionality and security:
-
Unified API: Access multiple LLM providers through a single OpenAI compatible interface, requiring no code changes.
-
API Key Security: Secure and centralized management of credentials for all providers.
-
Governance & Control: Set limits, enforce access controls, and apply content filtering to manage how LLMs are used within your organization.
-
Rate Limiting: Implement measures to prevent abuse and ensure fair usage across users.
-
Observability: Track and analyse usage, costs, latency, and overall performance.
-
Cost Management: Monitor spending and configure budget alerts to keep expenses under control.
-
Audit Trails: Maintain logs of all interactions with LLMs to support compliance and auditing requirements.
Call 1000+ LLMs with a single API
Truefoundry AI Gateway integrated with the following LLM providers. If you don’t see the provider you need, please reach out to us support@truefoundry.com and we will be happy to add it.
Gateway can be deployed on the edge, close to your applications
Gateway can be deployed on the edge, close to your applications
Unified API
The Unified API in the TrueFoundry AI Gateway provides a standardised interface for accessing and interacting with different language models from various providers. This means you can seamlessly switch between models and providers without changing your application’s code structure. By abstracting the underlying complexities, the Unified API simplifies the process of integrating multiple models and ensures consistency in how they are accessed and utilised.
Supported Providers
Below is a comprehensive list of popular LLM providers that is supported by TrueFoundry AI Gateway.
Provider | Streaming Supported | How to add models from this provider? | |
---|---|---|---|
![]() GCP Logo | GCP | ✅ | Click here |
![]() AWS Logo | AWS | ✅ | Click here |
![]() Azure Logo | Azure OpenAI | ✅ | Click here |
![]() Azure Logo | Azure AI Foundry | ✅ | Click here |
![]() OpenAI Logo | OpenAI | ✅ | Click here |
![]() Cohere Logo | Cohere | ✅ | Click here |
![]() AI21 Logo | AI21 | ✅ | Click here |
![]() Anthropic Logo | Anthropic | ✅ | Click here |
![]() Together AI Logo | Together AI | ✅ | Click here |
![]() DeepInfra Logo | DeepInfra | ✅ | Click here |
![]() Ollama Logo | Ollama | ✅ | Click here |
![]() Palm Logo | Palm | ✅ | Click here |
![]() Perplexity AI Logo | Perplexity AI | ✅ | Click here |
![]() Mistral AI Logo | Mistral AI | ✅ | Click here |
![]() Groq Logo | Groq | ✅ | Click here |
![]() Nomic Logo | Nomic | ✅ | Click here |
![]() TrueFoundry Logo | Self Hosted Models on TrueFoundry | ✅ | Click here |