Overview

This provides universal API for all the supported models via standard OpenAI /chat/completions endpoint.

​

Universal OpenAI Compatible API

TrueFoundry AI Gateway allows you to use any chat based LLM model via standard OpenAI /chat/completions endpoint. You can use standard OpenAI client to send requests to the gateway. Here is a sample code snippet for the same:

from openai import OpenAI

client = OpenAI(
    api_key="your_truefoundry_api_key",
    base_url="<truefoundry-base-url>/api/llm/api/inference/openai" # e.g. https://my-company.truefoundry.cloud/api/llm/api/inference/openai
)

response = client.chat.completions.create(
    model="openai-main/gpt-4o-mini", # this is the truefoundry model id
    messages=[{"role": "user", "content": "Hello, how are you?"}]
)

print(response.choices[0].message.content)

You will need to configure the following:

1. base_url: This is the base URL of the TrueFoundry dashboard.
2. api_key: This is the API key which can be generated from Personal Access Tokens
3. model: This is the TrueFoundry model ID. This is of the format provider_account/model_name. You can find this on TrueFoundry LLM playground UI.

​

API Reference

For detailed API specifications, parameters, and response schemas, see the Chat Completions API Reference.