Introduction

AI Gateway provides a unified interface to manage your organization’s LLM usage. Below are the key features that enhance functionality and security:

• Unified API: Access multiple LLM providers through a single OpenAI compatible interface, requiring no code changes.

• API Key Security: Secure and centralized management of credentials for all providers.

• Governance & Control: Set limits, enforce access controls, and apply content filtering to manage how LLMs are used within your organization.

• Rate Limiting: Implement measures to prevent abuse and ensure fair usage across users.

• Observability: Track and analyse usage, costs, latency, and overall performance.

• Cost Management: Monitor spending and configure budget alerts to keep expenses under control.

• Audit Trails: Maintain logs of all interactions with LLMs to support compliance and auditing requirements.

​

Call 1000+ LLMs with a single API

Truefoundry AI Gateway integrated with the following LLM providers. If you don’t see the provider you need, please reach out to us support@truefoundry.com and we will be happy to add it.

Gateway can be deployed on the edge, close to your applications

​

Unified API

The Unified API in the TrueFoundry AI Gateway provides a standardised interface for accessing and interacting with different language models from various providers. This means you can seamlessly switch between models and providers without changing your application’s code structure. By abstracting the underlying complexities, the Unified API simplifies the process of integrating multiple models and ensures consistency in how they are accessed and utilised.

​

Supported Providers

Below is a comprehensive list of popular LLM providers that is supported by TrueFoundry AI Gateway.

	Provider	Streaming Supported	How to add models from this provider?
GCP Logo	GCP	✅	Click here
AWS Logo	AWS	✅	Click here
Azure Logo	Azure OpenAI	✅	Click here
Azure Logo	Azure AI Foundry	✅	Click here
OpenAI Logo	OpenAI	✅	Click here
Cohere Logo	Cohere	✅	Click here
AI21 Logo	AI21	✅	Click here
Anthropic Logo	Anthropic	✅	Click here
Together AI Logo	Together AI	✅	Click here
DeepInfra Logo	DeepInfra	✅	Click here
Ollama Logo	Ollama	✅	Click here
Palm Logo	Palm	✅	Click here
Perplexity AI Logo	Perplexity AI	✅	Click here
Mistral AI Logo	Mistral AI	✅	Click here
Groq Logo	Groq	✅	Click here
Nomic Logo	Nomic	✅	Click here
TrueFoundry Logo	Self Hosted Models on TrueFoundry	✅	Click here