Flowise is an open-source LLM application builder that enables users to create LLM applications using drag-and-drop tools. TrueFoundry integrates seamlessly with Flowise, allowing you to deploy your Flowise applications through TrueFoundry’s Gateway. TrueFoundry’s AI Gateway provides comprehensive integration with Flowise, routing all LLM calls through the Gateway to ensure your applications are secure, compliant, and cost-effective through the Gateway’s built-in features.

​
Prerequisites

Before integrating Flowise with TrueFoundry, ensure you have:
  1. TrueFoundry Account: Create a Truefoundry account with atleast one model provider and generate a Personal Access Token by following the instructions in Generating Tokens
  2. Flowise Account: Set up a Flowise account using either the open-source version for local deployment or the SaaS version

​
Integration Guide

  1. Flowise Account: Set up a Flowise account using either the open-source version for local deployment or the SaaS version.
This guide will walk you through connecting Flowise with TrueFoundry’s AI Gateway step by step.

​
Step 1: Configure Flowise Credentials

First, you need to set up credentials in Flowise to connect to TrueFoundry’s Gateway:
  1. Navigate to Credentials: In your Flowise dashboard, look for the “Credentials” section in the left sidebar or main menu
  2. Add New Credential: Click on “Add Credential” or the ”+” button to create a new credential

Flowise Credentials

​
Step 2: Configure OpenAI Custom Credential

Since TrueFoundry’s Gateway is OpenAI-compatible, you’ll use the OpenAI Custom credential type:
  1. Select Credential Type: From the dropdown menu, choose “OpenAI API”

OpenAI Custom

  1. Enter Credential Details:
    • API Key: Paste your TrueFoundry authentication token here
    • Name: Give your credential a descriptive name (e.g., “TrueFoundry Gateway”)
  2. Save the Credential: Click “Save” to store your TrueFoundry Gateway credentials

​
Step 3: Create Your First Agent Workflow

Now you’ll create an agent that uses TrueFoundry’s Gateway for LLM calls:
  1. Navigate to AgentFlow: In the left sidebar, click on “AgentFlow” to access the workflow builder

AgentFlow

  1. Add a New Node: Click the “Add Node” button to start building your workflow

Add Node

  1. Connect Start Node: Make sure to create a connection from the Start node to your AgentFlow node. This is crucial for the workflow to function properly.

Edit Node

​
Step 4: Configure the LLM Model

This is the most important step where you connect your agent to TrueFoundry’s models:
  1. Configure the Agent Node:
    • Click on your agent node to open the configuration panel
    • Select Credential: Choose the TrueFoundry credential you created in Step 2
    • Base URL and Model Name: You will get both the base URL and model name from the unified code snippet in our playground (ensure you use the same model name as written)

Get Base URL and Model Name from Unified Code Snippet

  • Additional Settings: Configure any other parameters like temperature, max tokens, etc., as needed

Edit Node

​
Step 5: Save and Test Your Integration

  1. Save Your Workflow: Click the “Save” button in the top-right corner of the Flowise interface
  2. Test the Connection:
    • Open the chat interface by clicking the chat icon
    • Send a test message to verify that your agent is working correctly
    • The messages should now be processed through TrueFoundry’s Gateway

Save and Test the Agent

​
Troubleshooting Tips

  • Authentication Errors: Double click on the LLM node you will see the settings ensure api key, base url, model name are correctly configured
  • Model Not Found: Ensure the model name from the unified code snippet is exact and the model is deployed
  • Connection Issues: Verify your Gateway base URL format is correct: https://your-control-plane.truefoundry.cloud/api/llm
  • Workflow Not Responding: Check that all nodes are properly connected, especially the connection from the Start node
Your Flowise application is now integrated with TrueFoundry’s AI Gateway and ready for use.