The chat completions API supports all OpenAI compatible parameters to fine-tune model behavior and response characteristics.
Basic Parameters
Here’s how to use common parameters:
from openai import OpenAI
client = OpenAI(
api_key="your_truefoundry_api_key",
base_url="<truefoundry-base-url>/api/llm/api/inference/openai"
)
response = client.chat.completions.create(
model="openai-main/gpt-4o-mini",
messages=[{"role": "user", "content": "Hello, how are you?"}],
temperature=0.7, # Controls randomness (0.0 to 1.0)
max_tokens=100, # Maximum number of tokens to generate
top_p=0.9, # Nucleus sampling parameter
frequency_penalty=0.0, # Reduces repetition
presence_penalty=0.0, # Encourages new topics
stop=["\n", "Human:"] # Stop sequences
)
print(response.choices[0].message.content)
You can pass any of the parameters supported by OpenAI’s API. You can refer to the OpenAI API Reference for more details.
Some models from different vendors (or maybe even same vendors) do not support all the parameters supported by OpenAI’s API. For example, temperature is not supported by o series models like o3-mini.
