Here is a quick guide to get started with TrueFoundry AI Gateway.
Sign Up
Follow this if you don't have an account on TrueFoundry already
Welcome Page
Signup Page
Landing Page
Add your models
Add your provider accounts, API credentials, and models
Go to the AI Gateway Tab from the left sidebar and click on Models tab and select your provider from the list. You can add multiple API keys from the same provider by creating multiple provider accounts.
AI Gateway Page
A Provider Account refers to one account of a model provider like OpenAI, Anthropic, Bedrock (AWS Account) etc.
You can add multiple OpenAI, AWS, Anthropic accounts each with their own API keys / auth tokens as different provider accounts. Each provider account can have multiple models that are available with that provider. For example, if the provider account type is OpenAI, you will have models like GPT-4, GPT-4o, GPT-3.5-turbo etc.
Provider Account Page
In case the model you want to add is not available in the list, you can still add the models by clicking on +Add Model at the bottom of the form and filling up the details.
To add additional models in the future, you can do so by traversing to AI Gateway tab and then Models tab. You will see list of provider accounts on the left side, select the provider account and then click on Add Model button on top right corner.
Access Models in Playground
The LLM Playground is a UI for the AI Gateway where you can tryout different models you’ve added from across providers like OpenAI, Mistral, Cohere etc. Below is an overview of the features:
AI Gateway Playground Page
AI Gateway Select Model Page
Access Gateway Models via Code
You can access your models via code snippets.
AI Gateway Code Snippets Page
Integrate Gateway Models In Code
OpenAI client
, Langchain Client
, Python (Streaming and Non-Streaming)
, Node.js
, LangGraph
, LLamaIndex
, Google ADK
, Curl
.If you have any questions, please reach out to us at support@truefoundry.com or book a demo with our team here.