
Key Features
Unified API Interface
API Keys Management
Multimodal Inputs
Access Control
Rate Limiting
Load Balancing
Budget Limiting
Guardrails
Observability & Metrics
Prompt Playground
Batch Predictions
MCP Registry
Centralized Authn/Authz for all MCP Servers
Virtual MCP Servers
Agent Playground
Build Agents with unified API for all MCP servers
Rate Limiting and Observability for Tools
Supported Model Providers
We integrate with 1000+ LLMs through the following providers.














Supported APIs
The following accordions show which features are supported for each provider across different endpoints:- ✅ Supported by Provider and Truefoundry
- ❌ Provider by provider, but not by Truefoundry
- - Provider does not support this feature
Chat Completion (/chat/completions)
Chat Completion (/chat/completions)
| Provider | Stream | Non Stream | Tools | JSON Mode | Schema Mode | Prompt Caching | Reasoning | Structured Output |
|---|---|---|---|---|---|---|---|---|
| OpenAI | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | - |
| Azure OpenAI | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | - |
| Anthropic | ✅ | ✅ | ✅ | - | ✅ | ✅ | ✅ | - |
| Bedrock | ✅ | ✅ | ✅ | - | ✅ | ✅ | ✅ | - |
| Vertex | ✅ | ✅ | ✅ | - | ✅ | ✅ | ✅ | - |
| Cohere | ✅ | ✅ | ✅ | ✅ | ✅ | - | ✅ | - |
| Gemini | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | - |
| Groq | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | - |
| AI21 | ✅ | ✅ | - | ✅ | - | - | - | - |
| Cerebras | ✅ | ✅ | - | ✅ | - | - | ✅ | - |
| SambaNova | ✅ | ✅ | - | ✅ | - | - | ✅ | - |
| Perplexity-AI | ✅ | ✅ | - | ✅ | - | - | ✅ | ✅ |
| Together-AI | ✅ | ✅ | ✅ | ✅ | - | ✅ | ✅ | ✅ |
| DeepInfra | ✅ | ✅ | ✅ | ✅ | - | ✅ | ✅ | - |
Embedding (/embeddings)
Embedding (/embeddings)
| Provider | String | List of String |
|---|---|---|
| OpenAI | ✅ | ✅ |
| Azure OpenAI | ✅ | ✅ |
| Anthropic | - | - |
| Bedrock | ✅ | ✅ |
| Vertex | ✅ | ✅ |
| Cohere | ✅ | ✅ |
| Gemini | - | - |
| Groq | - | - |
| SambaNova | ❌ | ❌ |
| Together-AI | ✅ | ✅ |
| DeepInfra | ❌ | ❌ |
Image Generation (/images/generations)
Image Generation (/images/generations)
| Provider | Generate |
|---|---|
| OpenAI | ✅ |
| Azure OpenAI | ✅ |
| Bedrock | ✅ |
| Vertex | ✅ |
| Anthropic | - |
| Cohere | - |
| Gemini | ❌ |
| Groq | - |
| Together-AI | ❌ |
| DeepInfra | ❌ |
Image Edit (/images/edits)
Image Edit (/images/edits)
| Provider | Edit |
|---|---|
| OpenAI | ✅ |
| Azure OpenAI | ✅ |
| Bedrock | ✅ |
| Vertex | ✅ |
| Anthropic | - |
| Cohere | - |
| Gemini | ❌ |
| Groq | - |
| Together-AI | ❌ |
| DeepInfra | ❌ |
Image Variation (/images/variations)
Image Variation (/images/variations)
| Provider | Variation |
|---|---|
| OpenAI | ✅ |
| Azure OpenAI | - |
| Bedrock | ✅ |
| Vertex | - |
| Anthropic | - |
| Cohere | - |
| Gemini | ❌ |
| Groq | - |
| Together-AI | ❌ |
| DeepInfra | ❌ |
Audio Transcription (/audio/transcriptions)
Audio Transcription (/audio/transcriptions)
| Provider | Transcription |
|---|---|
| OpenAI | ✅ |
| Azure OpenAI | ✅ |
| Anthropic | - |
| Bedrock | - |
| Vertex | ❌ |
| Cohere | - |
| Gemini | ❌ |
| Groq | ✅ |
| Together-AI | ❌ |
| DeepInfra | ❌ |
Audio Translation (/audio/translations)
Audio Translation (/audio/translations)
| Provider | Translation |
|---|---|
| OpenAI | ✅ |
| Azure OpenAI | ✅ |
| Anthropic | - |
| Bedrock | - |
| Vertex | ❌ |
| Cohere | - |
| Gemini | ❌ |
| Groq | ✅ |
| Together-AI | ❌ |
| DeepInfra | ❌ |
Text To Speech (/audio/speech)
Text To Speech (/audio/speech)
| Provider | Text To Speech |
|---|---|
| OpenAI | ✅ |
| Azure OpenAI | ✅ |
| Anthropic | - |
| Bedrock | - |
| Vertex | ❌ |
| Cohere | - |
| Gemini | ❌ |
| Groq | ❌ |
| Together-AI | ❌ |
| DeepInfra | ❌ |
Rerank (/rerank)
Rerank (/rerank)
| Provider | Rerank |
|---|---|
| OpenAI | - |
| Azure OpenAI | - |
| Anthropic | - |
| Bedrock | ✅ |
| Vertex | - |
| Cohere | ✅ |
| Gemini | - |
| Groq | - |
| Together-AI | ❌ |
| DeepInfra | ❌ |
Batch (/batches)
Batch (/batches)
| Provider | Batch |
|---|---|
| OpenAI | ✅ |
| Azure OpenAI | ❌ |
| Anthropic | ❌ |
| Bedrock | ✅ |
| Vertex | ✅ |
| Cohere | ❌ |
| Gemini | ❌ |
| Groq | ✅ |
| Cerebras | - |
| Together-AI | ❌ |
| DeepInfra | ❌ |
Fine Tune
Fine Tune
| Provider | Fine Tune |
|---|---|
| OpenAI | ✅ |
| Azure OpenAI | - |
| Anthropic | - |
| Bedrock | ❌ |
| Vertex | ❌ |
| Cohere | ❌ |
| Gemini | - |
| Groq | ❌ |
| Cerebras | - |
| Together-AI | ❌ |
| DeepInfra | ❌ |
Files (/files)
Files (/files)
| Provider | Files |
|---|---|
| OpenAI | ✅ |
| Azure OpenAI | ❌ |
| Anthropic | ❌ |
| Bedrock | ✅ |
| Vertex | ✅ |
| Cohere | ❌ |
| Gemini | ❌ |
| Groq | ✅ |
| Cerebras | - |
| Together-AI | ❌ |
| DeepInfra | ❌ |
Moderation (/moderations)
Moderation (/moderations)
| Provider | Moderation |
|---|---|
| OpenAI | ✅ |
| Azure OpenAI | - |
| Anthropic | - |
| Bedrock | - |
| Vertex | - |
| Cohere | ❌ |
| Gemini | - |
| Groq | ✅ |
| Cerebras | - |
| Together-AI | ❌ |
| DeepInfra | ❌ |
Model Response (/responses)
Model Response (/responses)
| Provider | Model Response |
|---|---|
| OpenAI | ✅ |
| Azure OpenAI | ✅ |
| Anthropic | - |
| Bedrock | - |
| Vertex | - |
| Cohere | - |
| Gemini | - |
| Groq | ❌ |
| Cerebras | - |
| Together-AI | - |
| DeepInfra | - |
Completion (/completions)
Completion (/completions)
| Provider | Completion |
|---|---|
| OpenAI | - |
| Azure OpenAI | - |
| Anthropic | - |
| Bedrock | - |
| Vertex | - |
| Cohere | - |
| Gemini | - |
| Groq | - |
| Cerebras | ❌ |
| Together-AI | ✅ |
| DeepInfra | ✅ |
Ecosystem & Integrations
Discover how TrueFoundry connects with your favorite AI frameworks and tools to streamline your ML development workflow.- AI Frameworks
- Coding Assistants
- Agent Builder
- Guardrails
- Others
Deployment Options
The Truefoundry AI Gateway can either be used as a SaaS offering or deployed on-premise.- SaaS Offering: You can directly use the gateway as a SaaS offering by signing up on our website, you can find the instructions here.
- Enterprise Deployment for enterprise security and control. You can deploy the gateway in your cloud or on-premise. You can find the architecture and deployment instructions here.
Frequently Asked Questions
What's the performance impact of using the gateway?
What's the performance impact of using the gateway?
is closer to your users.

AI Gateway on the edge, close to your applications for optimal performance
Can I deploy the gateway on-premise?
Can I deploy the gateway on-premise?
How do I integrate my self-hosted models?
How do I integrate my self-hosted models?
Can I use the gateway without the full MLOps platform?
Can I use the gateway without the full MLOps platform?
Does the Gateway support text, vision, audio, image generation, and embeddings through a single interface?
Does the Gateway support text, vision, audio, image generation, and embeddings through a single interface?
What parameters are supported (temperature, max tokens, etc.)?
What parameters are supported (temperature, max tokens, etc.)?
Can I stream responses?
Can I stream responses?
How do I send files to models?
How do I send files to models?
How do I manage sessions for multi-turn interactions?
How do I manage sessions for multi-turn interactions?
How do I use structured outputs or Pydantic validation?
How do I use structured outputs or Pydantic validation?
Prompt management and versioning?
Prompt management and versioning?
Batch predictions and embeddings?
Batch predictions and embeddings?
Gemini (agentic AI) through TrueFoundry?
Gemini (agentic AI) through TrueFoundry?
Fiddler / Palo Alto AIRS / Pangea / Patronus integrations
Fiddler / Palo Alto AIRS / Pangea / Patronus integrations
Are Workday, Salesforce, SAP, etc. supported?
Are Workday, Salesforce, SAP, etc. supported?
Can outputs be automatically evaluated using tools like Arize, Galileo?
Can outputs be automatically evaluated using tools like Arize, Galileo?






