Adding Models to LLM Gateway

Understanding the concept of a Provider and a Model -

What is a Provider?

A provider is a company or organization that offers access to various language models. These providers offer different models, each with unique capabilities, performance characteristics, and pricing structures. Examples of popular providers include OpenAI and Anthropic.

What is a Model?

A model is a specific instance of an AI language model offered by a provider. Each model has distinct features and is trained on different datasets, leading to variations in performance and suitability for different tasks. For instance, OpenAI's GPT-3 and GPT-4 are different models, each with its own set of capabilities.

Steps to Add Providers and Models

This section captures how to add models along with setting up required access controls.

1. Navigate to the Integrations Page: Start by accessing the Integrations page in your TrueFoundry dashboard.
2. Choose a Provider: Select the provider you would like to integrate. Options might include OpenAI, Anthropic, and others.
3. Add Models: After selecting a provider, you will be able to add specific models offered by that provider. For example:
   1. If you choose OpenAI, you might add models such as GPT-3, GPT-3.5, or GPT-4
4. Configure Integration Settings: Each model may require specific configurations. Ensure you enter the necessary API keys, access tokens, or any other required credentials and settings.

Save and Test the Integration: After adding the models, save the integration and test it to ensure everything is functioning correctly. You can typically do this by running sample queries or using the Playground within the TrueFoundry LLM Gateway.