Skip to main content

​
Adding Models

This section explains the steps to add Google Vertex AI models and configure the required access controls.
1

Navigate to Google Vertex Models in AI Gateway

From the TrueFoundry dashboard, navigate to AI Gateway > Models and select Google Vertex.
Navigating to Google Vertex Provider Account in AI Gateway

Navigate to Google Vertex Models

2

Add Google Vertex Account and Authentication

Give a unique name to your Google Vertex account. This will be used to refer to the models later. You will also need to provide your Google Cloud Project ID, Service Account JSON, and a default Region. Add collaborators to your account, this will give access to the account to other users/teams. Learn more about access control here.
1. Project ID
  • You can find your Project ID in the top-right corner of your Google Cloud Console.
Google Cloud Console header showing project ID location in the dropdown menu

Finding your Project ID in Google Cloud Console

2. Service Account JSON
  • Generate a Service Account JSON key by following the official Google Cloud documentation here.
  • The service account must have the Vertex AI User role, which includes the aiplatform.endpoints.predict permission required by the gateway.
3. Region
  • Specify a default region for all models under this account. You can override this region for individual models later.
Google Vertex account configuration form with fields for name, project ID, service account JSON, and region

Add Vertex Provider Account

3

Add Models

You can either select available models from the list or add them manually by clicking + Add Model. When adding a model manually, the Model ID format depends on the provider.
Select a Gemini model from the list or add it manually.
  • Model ID Format: google/<vertex-model-id>
  • Example: google/gemini-1.5-pro
You can find the Model ID in the Google Cloud Console.
Google Cloud Console showing Gemini model details with model ID highlighted

Find Gemini Model ID in Google Console

Select a Claude model from the list or add it manually.
  • Model ID Format: anthropic/<vertex-model-id>
  • Example: anthropic/claude-3-5-sonnet-v2@20241022
Google Cloud Console showing Anthropic Claude model details with model ID highlighted

Find Anthropic Model ID in Google Console

Select a Mistral model from the list or add it manually.
  • Model ID Format: mistralai/<vertex-model-id>
  • Example: mistralai/mistral-large-2411@001
Google Cloud Console showing Mistral AI model details with model ID highlighted

Find Mistral Model ID in Google Console

When adding any model manually, you can specify a Region to override the default one set at the account level.

​
Inference

After adding the models, you can perform inference using an OpenAI-compatible API via the Playground or by integrating it with your own application.
Code Snippet and Try in Playgroud Buttons for each Google Vertex model

Infer Model in Playground or Get Code Snippet

​
FAQs

No. You can set a default region at the account level and override it for each individual model if needed. This allows you to use models from different regions with a single provider account.