AI Gateway provides a unified interface to manage your organization’s LLM usage. Below are the key features that enhance functionality and security:

  • Unified API: Access multiple LLM providers through a single OpenAI compatible interface, requiring no code changes.

  • API Key Security: Secure and centralized management of credentials for all providers.

  • Governance & Control: Set limits, enforce access controls, and apply content filtering to manage how LLMs are used within your organization.

  • Rate Limiting: Implement measures to prevent abuse and ensure fair usage across users.

  • Observability: Track and analyse usage, costs, latency, and overall performance.

  • Cost Management: Monitor spending and configure budget alerts to keep expenses under control.

  • Audit Trails: Maintain logs of all interactions with LLMs to support compliance and auditing requirements.

Call 1000+ LLMs with a single API

Truefoundry AI Gateway integrated with the following LLM providers. If you don’t see the provider you need, please reach out to us support@truefoundry.com and we will be happy to add it.

Gateway can be deployed on the edge, close to your applications

Gateway can be deployed on the edge, close to your applications

🚀Learn more on AI Gateway Benchmarks here: Read more

Unified API

The Unified API in the TrueFoundry AI Gateway provides a standardised interface for accessing and interacting with different language models from various providers. This means you can seamlessly switch between models and providers without changing your application’s code structure. By abstracting the underlying complexities, the Unified API simplifies the process of integrating multiple models and ensures consistency in how they are accessed and utilised.

Supported Providers

Below is a comprehensive list of popular LLM providers that is supported by TrueFoundry AI Gateway.

ProviderStreaming SupportedHow to add models from this provider?

GCP Logo

GCPClick here

AWS Logo

AWSClick here

Azure Logo

Azure OpenAIClick here

Azure Logo

Azure AI FoundryClick here

OpenAI Logo

OpenAIClick here

Cohere Logo

CohereClick here

AI21 Logo

AI21Click here

Anthropic Logo

AnthropicClick here

Together AI Logo

Together AIClick here

DeepInfra Logo

DeepInfraClick here

Ollama Logo

OllamaClick here

Palm Logo

PalmClick here

Perplexity AI Logo

Perplexity AIClick here

Mistral AI Logo

Mistral AIClick here

Groq Logo

GroqClick here

Nomic Logo

NomicClick here

TrueFoundry Logo

Self Hosted Models on TrueFoundryClick here