TrueFoundry allows you to connect and use models from providers like OpenAI, Anthropic, AWS Bedrock, Google Vertex AI, and many other providers from a single place. Once added, these models can be used in TrueFoundry AI Gateway and your applications.
The models are saved as provider integrations in TrueFoundry. The group of models are saved as a provider account. You can add multiple models to a single provider account; the models in the same provider account are called integrations.
Replace YOUR_CONTROL_PLANE_URL with your actual TrueFoundry control plane
URL and YOUR_API_KEY with your API key throughout this guide.
Add Models API Overview Here is the curl command example to add a model as a provider account:
curl -X PUT "YOUR_CONTROL_PLANE_URL/api/svc/v1/apply" \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "manifest": { "type": "provider-account/openai", "name": "my-openai-account", "auth_data": { "type": "openai-api-key", "api_key": "sk-..." }, "integrations": [ { "type": "openai-model", "name": "gpt-4", "model_name": "gpt-4", "model_type": "chat" } ] }, "dryRun": false }' Request Schema The Apply API accepts a TrueFoundryApplyRequest object with the following structure:
manifest
ModelProviderAccount | ModelManifest | ArtifactManifest | Service | ...
required
The manifest object defining the resource to be created or updated. For model
provider accounts, this should be a ModelProviderAccount object.
When set to true, performs a dry run without actually creating or updating
the resource. Useful for validation.
ModelProviderAccount Schema For creating model provider accounts, the manifest should be a ModelProviderAccount object. This is a union type that can be one of any of the following provider account types:
Supported Provider Account Types: Common Fields for All Provider Accounts: The provider account type (e.g., “provider-account/openai”,
“provider-account/anthropic”, etc.)
Name of the provider account. Must be 3-32 characters, lowercase alphanumeric
with hyphens, cannot start with a number.
Authentication data specific to the provider (API keys, credentials, etc.)
List of model integrations available through this provider account
Optional list of users who have access to this provider account
Response Schema { "id" : "provider-account-123" , "name" : "my-openai-account" , "fqn" : "tenant-name/openai/my-openai-account" , "provider" : "openai" , "manifest" : { "type" : "provider-account/openai" , "name" : "my-openai-account" , "auth_data" : { "type" : "openai-api-key" , "api_key" : "sk-..." } }, "integrations" : [ { "id" : "integration-456" , "name" : "gpt-4" , "fqn" : "tenant-name/openai/my-openai-account/model/gpt-4" , "type" : "model" , "manifest" : { "type" : "openai-model" , "name" : "gpt-4" , "model_name" : "gpt-4" , "model_type" : "chat" }, "providerAccountFqn" : "tenant-name/openai/my-openai-account" , "createdBySubject" : { "subjectSlug" : "user@example.com" }, "createdAt" : "2024-01-15T10:30:00Z" , "updatedAt" : "2024-01-15T10:30:00Z" } ], "createdBySubject" : { "subjectSlug" : "user@example.com" }, "createdAt" : "2024-01-15T10:30:00Z" , "updatedAt" : "2024-01-15T10:30:00Z" , "action" : "CREATE" } Response Fields Unique identifier for the provider account.
Name of the provider account.
Fully qualified name of the provider account (tenant/provider/name).
The provider type (e.g., “openai”, “anthropic”, “aws-bedrock”).
The provider account manifest that was applied, excluding integrations.
List of provider integrations (models) associated with this account.
Information about the user who created the provider account.
Timestamp when the provider account was created.
Timestamp when the provider account was last updated.
The action performed: “CREATE” for new resources, “UPDATE” for existing ones.
Examples Show OpenAI Provider Account
For connecting to OpenAI’s API: { "type" : "provider-account/openai" , "name" : "my-openai-account" , "auth_data" : { "type" : "openai-api-key" , "api_key" : "sk-your-openai-api-key" }, "integrations" : [ { "type" : "openai-model" , "name" : "gpt-4" , "model_name" : "gpt-4" , "model_type" : "chat" } ] } Show Anthropic Provider Account
For connecting to Anthropic’s Claude API: { "type" : "provider-account/anthropic" , "name" : "my-anthropic-account" , "auth_data" : { "type" : "anthropic-api-key" , "api_key" : "sk-ant-your-anthropic-api-key" }, "integrations" : [ { "type" : "anthropic-model" , "name" : "claude-3-sonnet" , "model_name" : "claude-3-sonnet-20240229" , "model_type" : "chat" } ] } Show AWS Bedrock Provider Account
For connecting to AWS Bedrock: { "type" : "provider-account/aws-bedrock" , "name" : "my-aws-bedrock-account" , "region" : "us-east-1" , "auth_data" : { "type" : "aws-access-key" , "access_key_id" : "your-access-key" , "secret_access_key" : "your-secret-key" }, "integrations" : [ { "type" : "bedrock-model" , "name" : "claude-3-sonnet" , "model_name" : "anthropic.claude-3-sonnet-20240229-v1:0" , "model_type" : "chat" } ] } Error Handling The Apply API may return various error responses. Common scenarios include:
400 Bad Request : Invalid manifest structure or missing required fields401 Unauthorized : Invalid or missing API key403 Forbidden : Insufficient permissions for the operation404 Not Found : Provider account not found409 Conflict : Provider account with the same name already exists422 Unprocessable Entity : Invalid provider credentials or configuration
List Models You can list and retrieve model integrations using the List Provider Integrations API. This allows you to query existing model integrations with filtering and pagination support.
API Overview Here is the curl command example to list model integrations:
curl -X GET "YOUR_CONTROL_PLANE_URL/api/svc/v1/provider-integrations?type=model" \ -H "accept: application/json" \ -H "Authorization: Bearer YOUR_API_KEY" Query Parameters Filter integrations by type. For models, use model.
Filter by fully qualified name (FQN) of a specific integration. Cannot be used with id.
Filter by integration ID. Cannot be used with fqn.
Pagination offset - number of items to skip.
Maximum number of items to return. If not specified, returns all items.
Response Schema { "data" : [ { "id" : "cmfpjat8x04yg01oe1jng7l5l" , "name" : "gpt-4" , "fqn" : "truefoundry:openai:my-openai-account:model:gpt-4" , "type" : "model" , "manifest" : { "name" : "gpt-4" , "type" : "integration/model/openai" , "model_id" : "gpt-4" , "model_types" : [ "chat" ], "cost" : { "metric" : "public_cost" } }, "metadata" : {}, "providerAccountFqn" : "truefoundry:openai:my-openai-account" , "providerAccount" : { "id" : "cmc93zjju01h001pfdr3l437d" , "name" : "my-openai-account" , "fqn" : "truefoundry:openai:my-openai-account" , "provider" : "openai" , "manifest" : { "name" : "my-openai-account" , "type" : "provider-account/openai" , "auth_data" : { "type" : "api-key" , "api_key" : "dummy-key" }, "collaborators" : [ { "role_id" : "provider-account-manager" , "subject" : "user:user@example.com" }, { "role_id" : "provider-account-access" , "subject" : "team:everyone" } ] }, "createdBySubject" : { "subjectId" : "cm7t30kps08zz01s78jsgdtru" , "subjectSlug" : "user@example.com" , "subjectType" : "user" , "subjectPatName" : "user-pat-1" , "subjectDisplayName" : "John Doe" }, "createdAt" : "2024-01-15T10:30:00.000Z" , "updatedAt" : "2024-01-15T10:30:00.000Z" , "createdBy" : "user@example.com" }, "createdBySubject" : { "subjectId" : "cm7t30kps08zz01s78jsgdtru" , "subjectSlug" : "user@example.com" , "subjectType" : "user" , "subjectPatName" : "user-pat-1" , "subjectDisplayName" : "John Doe" }, "createdAt" : "2024-01-15T10:30:00.000Z" , "updatedAt" : "2024-01-15T10:30:00.000Z" , "createdBy" : "user@example.com" } ], "pagination" : { "total" : 1 , "offset" : 0 , "limit" : 10 } } Response Fields Array of model integration objects. Show Integration Object Properties
Unique identifier for the model integration.
Name of the model integration.
Fully qualified name of the integration in the format: tenant:provider:account:model:name (colon-separated).
Type of the integration (e.g., “model”).
The model integration manifest (auth_data is excluded for security).
Additional metadata associated with the integration.
FQN of the provider account this integration belongs to.
Full provider account object that contains this integration. Show Provider Account Properties
Unique identifier for the provider account.
Name of the provider account.
Fully qualified name of the provider account.
Provider type (e.g., “openai”, “anthropic”, “aws-bedrock”).
Provider account manifest including auth_data and collaborators.
Information about the user who created the provider account.
Timestamp when the provider account was created.
Timestamp when the provider account was last updated.
ID of the user who created the provider account.
Information about the user who created the integration. Unique identifier for the subject.
Type of subject (e.g., “user”).
Slug identifier for the subject (e.g., email address).
Personal Access Token name if the action was performed using a PAT, or null.
Display name of the subject, or null if not available.
Timestamp when the integration was created.
Timestamp when the integration was last updated.
ID of the user who created the integration.
Pagination metadata. Show Pagination Properties
Total number of integrations matching the query.
Current pagination offset.
Maximum number of items returned per page.
Examples To list all model integrations in your tenant: curl -X GET "YOUR_CONTROL_PLANE_URL/api/svc/v1/provider-integrations?type=model" \ -H "accept: application/json" \ -H "Authorization: Bearer YOUR_API_KEY" Show List Models with Pagination
To retrieve models with pagination (10 items starting from offset 20): curl -X GET "YOUR_CONTROL_PLANE_URL/api/svc/v1/provider-integrations?type=model&limit=10&offset=20" \ -H "accept: application/json" \ -H "Authorization: Bearer YOUR_API_KEY" Show Get Specific Model by FQN
To retrieve a specific model by its fully qualified name: curl -X GET "YOUR_CONTROL_PLANE_URL/api/svc/v1/provider-integrations?type=model&fqn=truefoundry:openai:my-openai-account:model:gpt-4" \ -H "accept: application/json" \ -H "Authorization: Bearer YOUR_API_KEY" Show Get Specific Model by ID
To retrieve a specific model by its integration ID: curl -X GET "YOUR_CONTROL_PLANE_URL/api/svc/v1/provider-integrations?type=model&id=integration-456" \ -H "accept: application/json" \ -H "Authorization: Bearer YOUR_API_KEY" Error Handling The List API may return various error responses. Common scenarios include:
400 Bad Request : Invalid query parameters (e.g., both fqn and id provided)401 Unauthorized : Invalid or missing API key404 Not Found : Integration with specified id or fqn not found 
