​
Overview

The Proxy API allows you to route requests directly to AI provider endpoints through TrueFoundry AI Gateway without any translation logic. This means you can use provider-native request and response formats while still benefiting from TrueFoundry’s features like logging, rate limiting, and budget management. TrueFoundry Proxy Endpoint routes your request to the appropriate provider and returns the provider’s response without modifying it, preserving the native request/response format.

​
Endpoint Structure

All proxy endpoints follow the pattern: 
https://{controlPlaneUrl}/api/llm/v1/{provider-endpoint-path}

​
Supported Endpoints

TrueFoundry Proxy API supports all provider endpoints. However, the following endpoints have additional features like full logging, rate limiting, and budget management:
  • Anthropic Messages API - /v1/messages
  • Claude Code - Available from Vertex, Bedrock, and Anthropic providers
  • NIM Rerank - /v1/ranking

​
Requirements

To use the Proxy API, you must include one of the following:
  1. TrueFoundry Model Name in the request body 
    {
  "model": "your-truefoundry-model-name"
  // other request parameters
}
  2. Provider name in the headers 
    x-tfy-provider-name: tfy-provider-name
For endpoints with logging support, the TrueFoundry model name must be included in the request body.

​
Examples

​
Anthropic Messages API

from anthropic import Anthropic

BASE_URL = "https://{controlPlaneUrl}/api/llm"
API_KEY = "your-truefoundry-api-key"

# Configure Anthropic client with TrueFoundry settings
client = Anthropic(
    api_key=API_KEY,
    base_url=BASE_URL
)

# Make requests as usual with your TrueFoundry model name
response = client.messages.create(
    model="your-truefoundry-model-name",
    max_tokens=1024,
    messages=[
        {"role": "user", "content": "Tell me a joke about programming"}
    ]
)

print(response.content)

​
NIM Rerank API

import requests
import json

BASE_URL = "https://{controlPlaneUrl}/api/llm/v1/ranking"
API_KEY = "your-truefoundry-api-key"

headers = {
    "Content-Type": "application/json",
    "Authorization": f"Bearer {API_KEY}",
    "X-TFY-METADATA": '{"tfy_log_request":"true"}'
}

payload = {
    "model": "your-truefoundry-model-name",
    "query": {
        "text": "Where is New Delhi?"
    },
    "passages": [
        {"text": "New Delhi is capital of India"},
        {"text": "The sky is blue"}
    ]
}

response = requests.post(BASE_URL, headers=headers, json=payload)
print(response.json())