Google Vertex
Overview
TrueFoundry offers a secure and efficient gateway to seamlessly integrate various Large Language Models (LLMs) into your applications, including models hosted on Google Vertex.
Authentication Setup
To enable access to Google Vertex models, you must authenticate using the following credentials:1. Project ID
You can find your Project ID in the top-right corner of your Google Cloud Console.Finding your Project ID in Google Cloud Console
2. Service Account JSON
- Generate a Service Account JSON key by following these steps.
- This key enables TrueFoundry to securely access your Vertex AI models.
3. Required IAM Permissions
The service account must have the following permission:Adding a Vertex Provider Account
To register your Google Vertex account in TrueFoundry:1. Navigate to AI Gateway
- Open the TrueFoundry Dashboard.
-
Go to AI Gateway → Models.
2. Enter Authentication Details
- Paste your Service Account JSON.
- Enter the Project ID and Region.
-
Add Collaborators who can access or manage this provider account.
The Region specified here will be the default for all models added under this provider account. You can override this default region for individual models during model configuration if needed.
Add Vertex Provider Screenshot
Supported Models
Currently TrueFoundry AI Gateway supports text models from Vertex AI. Some of these models support function calling and sending files like images or documents attached to their messages and TrueFoundry AI Gateway also supports the same. Support for Vertex AI models like image generation, audio are coming soon on TrueFoundry AI Gateway.FAQs
Do I need to add multiple provider accounts for different regions?
Do I need to add multiple provider accounts for different regions?
No, it is not required to create separate provider accounts for different regions. Each model can have its own region configuration:
- When adding a model, you can specify a region that overrides the default region set in the provider account
- The default region (specified during provider account creation) will be used for any models where you don’t explicitly set a region
- This allows you to use models from different regions while maintaining a single provider account