Quick Start
Here’s a simple example to get you started immediately:About this example: This image generation request would work perfectly fine through TrueFoundry’s standard inference API. This example specifically demonstrates how to use the proxy route to make requests with provider-native formats while proxying requests via TrueFoundry AI Gateway.
Endpoint Structure
The proxy API supports multiple endpoint patterns for maximum flexibility:Standard Proxy Endpoints
Model and Provider Configuration
The proxy API supports multiple ways to specify your model and provider name:1. URL Query Parameters
tfyModelName
- Complete TrueFoundry model name (e.g.,provider-name/model-name
)tfyProviderName
- TrueFoundry provider name for routing (only required when using actual model IDs)
2. Request Headers
x-tfy-model-name
- Complete TrueFoundry model namex-tfy-provider-name
- TrueFoundry provider name (only required when using actual model IDs)
3. Request Body
model
- Can be either TrueFoundry model name (provider-name/model-name
) or actual model ID
Model and Provider Configuration Guide:When is Provider Name Required?
- Not required: When using TrueFoundry model names (
provider-name/model-name
) - provider is auto-identified - Required: When using actual model IDs (
gpt-4o
,claude-3-sonnet
) - specify provider via headers/query params
- Self-hosted models: Always required - base URL and configuration are model-specific
- Full feature endpoints (
Anthropic Messages, Claude Code, NIM Rerank
): Required for logging, rate limiting, budget tracking - Other endpoints: Provider name alone is sufficient for basic routing
- TrueFoundry format (
provider-name/model-name
) → automatically converted to actual model ID - Actual model ID format → passed through unchanged to provider
Endpoint Support
Endpoints with Full Feature Support
These specific endpoints have complete logging, rate limiting, and budget management:- Anthropic Messages API -
/messages
- Claude Code - Available from Vertex, Bedrock, and Anthropic providers
- NIM Rerank API -
/ranking
Universal Proxy Support
You can use any provider endpoint through the TrueFoundry Proxy API, even if it doesn’t have specific feature support.Code Examples
Endpoints with Full Feature Support
These endpoints provide complete TrueFoundry feature integration including logging, rate limiting, and budget management.Anthropic Messages API
NIM Rerank API
Standard Proxy Examples
All provider endpoints can be proxied through TrueFoundry AI Gateway for basic routing, but won’t include advanced features like detailed logging, rate limiting, and budget management.Custom Provider Endpoints
For self-hosted models, always provide the complete TrueFoundry model name since the base URL and configuration are model-specific.
Image Generation
For all other proxy endpoints: You can simply pass the TrueFoundry provider name via headers (
x-tfy-provider-name
) or query parameters (tfyProviderName
) along with the actual model ID (if required), and the proxy will route your request correctly without requiring the full TrueFoundry model name format.