Anthropic

TrueFoundry offers a secure and efficient gateway to seamlessly integrate various Large Language Models (LLMs) into your applications, including models available on Anthropic.

​

Adding Models

This section explains the steps to add Anthropic models and configure the required access controls.

• From the TrueFoundry dashboard, navigate to AI Gateway > Models and select Anthropic.
• Click Add Anthropic Account

• Give a unique name to your Anthropic account and complete the form with your Anthropic authentication details (API Key).
• Add collaborators to your account, this will give access to the account to other users/teams. Learn more about access control here.
• Select the model from the list. If you see the model you want to add in the list of checkboxes, we support public model cost for these models.

​

Supported Models

A list of models supported by Anthropic, along with their corresponding model IDs, can be found here: View Full List

The TrueFoundry AI Gateway supports all text and image models in Anthropic. We are also working on adding support for additional modalities, including speech, in the near future.

​

Inference

After adding the models, you can perform inference using an Anthropic-compatible API via the TrueFoundry AI Gateway. For instance, you can directly utilize the Anthropic library as well.

If you want to test out the added models on our playground, you can do so by clicking on the Try in Playground button next to the model you want to test.

from anthropic import Anthropic

client = Anthropic(api_key="Enter your API Key here", base_url="https://{controlPlaneUrl}/api/inference/openai")
stream = client.messages.create(
    messages = [
            {"role": "user", "content": "Enter your prompt here"},
    ],
    model= "anthropic-provider/claude-3-sonnet",
)