Skip to main content

​
Adding Models

This section explains the steps to add Databricks models and configure the required access controls.
1

Navigate to Databricks Models in AI Gateway

From the TrueFoundry dashboard, navigate to AI Gateway > Models and select Databricks.
Navigating to Databricks Provider Account in AI Gateway

Navigate to Databricks Models

2

Add Databricks Account and Authentication

Give a unique name for the Databricks account. This will be used to refer to the models later. Provide the authentication details for the gateway to access your Databricks models. Truefoundry supports both Service Principal and Personal Access Token (PAT) based authentication.
Using Service Principal (Recommended):Service Principal authentication is the recommended approach for production environments as it provides better security and access control.
  • Choose Service Principal Auth.
  • Enter your Databricks Service Principal Client ID and OAuth Secret.
Databricks Account Configuration Form with Service Principal Client ID and OAuth Secret Fields

Service Principal Authentication

Using Personal Access Token (PAT):Personal Access Tokens are suitable for development and testing environments.
  • Choose Databricks API Key Based Auth.
  • Enter your PAT.
Databricks Account Configuration Form with PAT Field

Personal Access Token (PAT) Authentication

Finally, enter your Databricks workspace URL (e.g., https://<workspace_id>.databricks.com).
3

Add Models

Click + Add Model to add a new model configuration. The Model ID in TrueFoundry must exactly match the serving endpoint name in your Databricks workspace.
  1. Access Databricks Serving: In your Databricks workspace, navigate to Serving in the left sidebar and click Create serving endpoint.
    Databricks workspace interface showing the Serving option in the left sidebar menu
  2. Configure Endpoint:
    • Endpoint name: Choose a descriptive name. This name will be your Model ID in TrueFoundry.
    • Served Entity: Choose from Foundation Models or your custom models.
    Databricks serving endpoint configuration form with fields for endpoint name and model selection
  3. Deploy and Verify: Click Create and wait for the deployment to become Ready.
Databricks model configuration form in TrueFoundry with display name and model ID fields

Add Databricks Model in TrueFoundry

​
Inference

After adding the models, you can perform inference using an OpenAI-compatible API via the Playground or by integrating it with your own application.
Code Snippet and Try in Playgroud Buttons for each model

Infer Model in Playground or Get Code Snippet

⌘I