​
Overview

The Proxy API allows you to route requests directly to AI provider endpoints through TrueFoundry AI Gateway without any translation logic. This means you can use provider-native request and response formats while still benefiting from TrueFoundry’s features like logging, rate limiting, and budget management. TrueFoundry Proxy Endpoint routes your request to the appropriate provider and returns the provider’s response without modifying it, preserving the native request/response format.

​
Endpoint Structure

All proxy endpoints follow the pattern: 
https://{controlPlaneUrl}/api/llm/v1/{provider-endpoint-path}

​
Endpoint Support

You can use any provider endpoint through the TrueFoundry Proxy API.

​
Endpoints with Full Feature Support

These specific endpoints have complete logging, rate limiting, and budget management:
  • Anthropic Messages API - /v1/messages
  • Claude Code - Available from Vertex, Bedrock, and Anthropic providers
  • NIM Rerank - /v1/ranking

​
Requirements

To use the Proxy API, you must include one of the following:
  1. TrueFoundry Model Name in the request body 
    {
  "model": "your-truefoundry-model-name"
  // other request parameters
}
  2. Provider name in the headers 
    x-tfy-provider-name: tfy-provider-name
For endpoints with logging support, the TrueFoundry model name must be included in the request body.

​
Examples

​
Anthropic Messages API

from anthropic import Anthropic

BASE_URL = "https://{controlPlaneUrl}/api/llm"
API_KEY = "your-truefoundry-api-key"

# Configure Anthropic client with TrueFoundry settings
client = Anthropic(
    api_key=API_KEY,
    base_url=BASE_URL
)

# Make requests as usual with your TrueFoundry model name
response = client.messages.create(
    model="anthropic-main/claude-3.7",
    max_tokens=1024,
    messages=[
        {"role": "user", "content": "Tell me a joke about programming"}
    ]
)

print(response.content)

​
NIM Rerank API

import requests
import json

URL = "https://{controlPlaneUrl}/api/llm/v1/ranking"
API_KEY = "your-truefoundry-api-key"

headers = {
    "Content-Type": "application/json",
    "Authorization": f"Bearer {API_KEY}",
    "X-TFY-LOGGING-CONFIG": '{"enabled": true}'
}

payload = {
    "model": "self-hosted/nim-rerank", # Your TrueFoundry Model name
    "query": {
        "text": "Where is New Delhi?"
    },
    "passages": [
        {"text": "New Delhi is capital of India"},
        {"text": "The sky is blue"}
    ]
}

response = requests.post(BASE_URL, headers=headers, json=payload)
print(response.json())