from openai import OpenAIclient = OpenAI( api_key="your_truefoundry_api_key", base_url="<truefoundry-base-url>/api/llm/api/inference/openai")response = client.chat.completions.create( model="openai-main/gpt-4o-mini", messages=[{"role": "user", "content": "Hello, how are you?"}], temperature=0.7, # Controls randomness (0.0 to 1.0) max_tokens=100, # Maximum number of tokens to generate top_p=0.9, # Nucleus sampling parameter frequency_penalty=0.0, # Reduces repetition presence_penalty=0.0, # Encourages new topics stop=["\n", "Human:"] # Stop sequences)print(response.choices[0].message.content)
You can pass any of the parameters supported by OpenAI’s API. You can refer to the OpenAI API Reference for more details.
Some models from different vendors (or maybe even same vendors) do not support all the parameters supported by OpenAI’s API. For example, temperature is not supported by o series models like o3-mini.