Open WebUI is an open-source LLM application interface that enables users to interact with large language models through a user-friendly web UI. TrueFoundry integrates seamlessly with Open WebUI, allowing you to route all Open WebUI LLM requests through TrueFoundry’s Gateway for enhanced security, load balancing, rate limiting, cost management, and more.

TrueFoundry’s AI Gateway provides robust integration with Open WebUI, ensuring all LLM calls are routed through the Gateway and benefit from its built-in features.

​
Prerequisites

Before integrating Open WebUI with TrueFoundry, ensure you have the following:

  1. Authentication Token: Create a Personal Access Token or Virtual Account in TrueFoundry by following the instructions in Generating Tokens. This token will authenticate your Open WebUI application to the TrueFoundry Gateway.

  2. Gateway Base URL: Locate your TrueFoundry Gateway base URL, which follows the format <control plane url>/api/llm/api/inference/openai. The control plane URL is where your TrueFoundry dashboard is hosted (e.g., https://company.truefoundry.cloud/api/llm/api/inference/openai).

  3. Open WebUI Instance: Deploy Open WebUI by following the official documentation for local or cloud deployment.

​
Configuration Steps

This guide assumes you have an active Open WebUI instance and have obtained your TrueFoundry AI Gateway base URL and authentication token.

​
Step 1: Configure Open WebUI to Use TrueFoundry Gateway

  1. Open the Open WebUI settings page.

  2. Navigate to the “Admin Panel” section from the top right icon.

Open WebUI Navigate to Admin Panel

  1. Add a new connection from Settings → Connections → Manage OpenAI API Connections (+).

Open WebUI Admin Panel

  1. Add the following details:
  • Base URL: Set this to your TrueFoundry Gateway base URL (e.g., https://company.truefoundry.cloud/api/llm/api/inference/openai).
  • API Key: Enter your PAT (Personal Access Token) or Virtual Account Token generated in Generating Tokens.
  • Model ID: Add the model IDs of the models you want to use. (Press the + button on the right side to add a model.)

You can find the model IDs in AI Gateway → Models Page. Hover over the model and click the copy button to copy the model ID.

Copy Model ID

Connection Form

  1. Click the “Save” button.

​
Step 2: Test the Integration

  1. Open a new chat in Open WebUI.

  2. Select the model you want to use from the dropdown on the top left corner.

  3. Send a prompt to verify that requests are routed through TrueFoundry’s Gateway.

Your Open WebUI instance is now integrated with TrueFoundry’s AI Gateway and ready for secure, compliant LLM usage.