Key Components

To install the complete control plane on your own infrastructure, you need to install the following components:

Truefoundry Control Plane + Gateway (Shipped as a single helm chart called truefoundry)
Postgres Database (Managed or Self-Hosted with Postgres >= 13)
Blob Storage (S3, GCS, Azure Container or any other S3 compatible storage)

Compute Requirements

Truefoundry ships as a helm chart (https://github.com/truefoundry/infra-charts/tree/main/charts/truefoundry) that has configurable options to either deploy both Deployment and AI Gateway feature or just choose the one of them according to your needs. The compute requirements change based on the set of features and the scale of the number of users and requests. Here are a few scenarios that you can choose from based on your needs.

The small tier is recommended for development purposes. Here all the components are deployed on Kubernetes and in non HA mode (single replica). This is suitable if you are just testing out the different features of Truefoundry.

This setup brings up 1 replica of the services and is not highly-available. It can enable you to test the features but we do not recommend this for production mode.

Component	CPU	Memory	Storage	Min Nodes	Remarks
Helm-Chart (AI Gateway Control Plane components)	2 vCPU	8GB	60GB _{Persistent Volumes (Block Storage) On Kubernetes}	2 _{Pods should be spread over min 2 nodes}	_{Cost: ~ $120 pm}
Helm-Chart (AI Gateway component only)	1 vCPU	512Mi	-	1 _{Pods should be spread over min 1 node}	_{Cost: ~ $10 pm}
Postgres (Deployed on Kubernetes)	0.5 vCPU	0.5GB	5GB _{Persistent Volumes (Block Storage) On Kubernetes}		_{PostgreSQL version >= 13} _{IOPS: Default (suitable for dev/testing)} _{For PostgreSQL 17+: Disable SSL, for AWS: by setting force_ssl parameter to 0 in the parameter group, for Azure: by setting require_secure_transport parameter to false in the parameter group}
Blob Storage (S3 Compatible)			20GB

Prerequisites for Installation

Kubernetes Cluster: K8s cluster 1.27+
Egress access to TrueFoundry Central Auth Server: https://auth.truefoundry.com & https://login.truefoundry.com
Domain to map the ingress of the Control Plane dashboard and AI Gateway along with certificate for the domain.
This Domain will be referred as Control Plane URL in our documentation.
Tenant Name, Licence key, and image pull secret from TrueFoundry team. If you have not registered yet, please visit TrueFoundry to register.
Postgres database. We usually recommend managed postgres database (For e.g. AWS RDS, or Google Cloud SQL, or Azure Database for PostgreSQL) for production environments. For instance requirements, refer to the Compute Requirements section.
In case, you do not have a managed database just for testing purposes, set devMode to true in the values file to spin up a local postgres database.
Blob Storage to store the AI Gateway request logs (either S3, GCS, Azure Blob Storage, or any other S3 compatible storage). You can find the instructions in the guide below.

Installation Instructions

Setup Control Plane Platform IAM Role

Creating AWS IAM Role for Control Plane

Control Plane IAM Role needs to have permission to assume any other IAM role in or cross account to provide access to different cloud services like blob, secret, models, etc.

Create a new IAM role for Control Plane with a suitable name like tfy-control-plane-platform-deps
Following is the IAM policy that needs to be attached to the Control Plane IAM Role:

{
    "Statement": [
        {
            "Action": "sts:AssumeRole",
            "Effect": "Allow",
            "Resource": "*"
        }
    ],
    "Version": "2012-10-17"
}

Here ”*” is used to allow the Control Plane IAM Role to assume any other IAM role in or cross account. In place of ”*” you can also give specific ARNs of other IAM roles

Add the following trust policy to the Control Plane IAM Role:

{
  "Version": "2012-10-17",
  "Statement": [
      {
          "Effect": "Allow",
          "Principal": {
              "Federated": "arn:aws:iam::<ACCOUNT_ID>:oidc-provider/oidc.eks.<AWS_REGION>.amazonaws.com/id/<OIDC_ID>"
          },
          "Action": "sts:AssumeRoleWithWebIdentity",
          "Condition": {
              "StringEquals": {
                  "oidc.eks.<AWS_REGION>.amazonaws.com/id/<OIDC_ID>:sub": [
                      "system:serviceaccount:truefoundry:truefoundry",
                  ]
              }
          }
      }
  ]
}

In place of <ACCOUNT_ID>, <AWS_REGION>, and <OIDC_ID> you can also give the values from your EKS cluster. You can find the OIDC_ID from the EKS cluster. Also, here we are assuming that the service account is truefoundry and the namespace is truefoundry, you can change it as per your needs.

Allow your IAM role to be assumed by Control Plane IAM Role

Add the following trust policy to your IAM Role to allow it to be assumed by the Control Plane IAM Role:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Principal": {
                "AWS": "<CONTROL_PLANE_IAM_ROLE_ARN>"
            },
            "Action": "sts:AssumeRole"
        }
      ]
}

Create S3 Bucket

Create a S3 Bucket with following config:

Make sure the bucket has lifecycle configuration to abort multipart upload set for 7 days.
Make sure CORS is applied on the bucket with the below configuration:

[
  {
    "AllowedHeaders": ["*"],
    "AllowedMethods": ["GET", "POST", "PUT"],
    "AllowedOrigins": ["*"],
    "ExposeHeaders": ["ETag"],
    "MaxAgeSeconds": 3000
  }
]

Create a IAM Policy to allow access to the S3 Bucket with following config:

{
  "Sid": "S3",
  "Effect": "Allow",
  "Action": ["s3:*"],
  "Resource": [
    "arn:aws:s3:::<YOUR_S3_BUCKET_NAME>",
    "arn:aws:s3:::<YOUR_S3_BUCKET_NAME>/*"
  ]
}

Attach the IAM Policy to the Control Plane Platform IAM Role

Create Postgres RDS Database

Create a Postgres RDS instance of size db.t3.medium with storage size of 30GB.

Important Configuration Notes: - For PostgreSQL 17: Disable SSL by setting force_ssl parameter to 0 in the parameter group - Security Group: Ensure your RDS security group has inbound rules allowing traffic from EKS node security groups

In case you want to setup Postgres on Kubernetes and not use RDS for testing purposes, skip this step and set devMode to true in the values file below

Create Kubernetes Secrets

We will create two secrets in this step:

Store the License Key and DB Credentials
Store the Image Pull Secret

Create Kubernetes Secret for License Key and DB Credentials

We need to create a Kubernetes secret containing the licence key and db credentials.

If you are using Postgres on Kubernetes in the dev mode, the values will be as follows:DB_HOST: <HELM_RELEASE_NAME>-postgresql.<NAMESPACE>.svc.cluster.local // eg. truefoundry-postgresql.truefoundry.svc.cluster.localDB_NAME: truefoundryDB_USERNAME: postgres # In order to use custom username, please update the same at postgresql.auth.usernameDB_PASSWORD: randompassword # You can change this to any value here.

truefoundry-creds.yaml

apiVersion: v1
kind: Secret
metadata:
  name: truefoundry-creds
type: Opaque
stringData:
  TFY_API_KEY: <TFY_API_KEY> # Provided by TrueFoundry team
  DB_HOST: <DB_HOST>
  DB_NAME: <DB_NAME>
  DB_USERNAME: <DB_USERNAME>
  DB_PASSWORD: <DB_PASSWORD>

Apply the secret to the Kubernetes cluster (Assuming you are installing the control plane in the truefoundry namespace)

kubectl apply -f truefoundry-creds.yaml -n truefoundry

Create Kubernetes Secret for Image Pull Secret

We need to create a Image Pull Secret to enable pulling the truefoundry images from the private registry.

truefoundry-image-pull-secret.yaml

apiVersion: v1
kind: Secret
metadata:
  name: truefoundry-image-pull-secret
type: kubernetes.io/dockerconfigjson
stringData:
  .dockerconfigjson: <IMAGE_PULL_SECRET> # Provided by TrueFoundry team

Apply the secret to the Kubernetes cluster (Assuming you are installing the control plane in the truefoundry namespace)

kubectl apply -f truefoundry-image-pull-secret.yaml -n truefoundry

Create HelmChart Values file

Create a values file as given below and replace the following values:

Control Plane URL: URL that you will map to the control plane dashboard.
Tenant Name: Tenant name provided by TrueFoundry team.
AWS S3 Bucket Name: Name of the S3 bucket you created in the previous step.
AWS Region: Region of the S3 bucket you created in the previous step.
Control Plane IAM Role ARN: ARN of the IAM role you created in the previous step.

truefoundry-values.yaml

global:
  # Domain to map the platform to
  controlPlaneURL: https://example.com

  # Ask TrueFoundry team to provide these
  tenantName: <TENANT_NAME>

  # Choose the resource tier as per your needs
  resourceTier: medium # or small or large

  # This is the reference to the secrets we created in the previous step
  existingTruefoundryCredsSecret: "truefoundry-creds"
  existingTruefoundryImagePullSecretName: "truefoundry-image-pull-secret"

  config:
    defaultCloudProvider: "aws"
    storageConfiguration:
      awsS3BucketName: "<AWS_S3_BUCKET_NAME>"
      awsRegion: "<AWS_REGION>"

  serviceAccount:
    annotations:
      eks.amazonaws.com/role-arn: <CONTROL_PLANE_IAM_ROLE_ARN>

# In case, you want to spin up postgres on kubernetes, enable this
# Please add creds and host details in the secret `truefoundry-creds` in the previous step
devMode:
  enabled: false

truefoundryFrontendApp:
  ingress:
    hosts:
      - example.com
    enabled: false
    annotations: {}
    ingressClassName: nginx # Replace with your ingress class name
tags:
  llmGateway: true
  llmGatewayRequestLogging: true

# Disable few depenencies for only LLM Gateway setup
tfyBuild:
  enabled: false
sfyManifestService:
  enabled: false
tfyController:
  enabled: false
mlfoundryServer:
  enabled: false
tfy-buildkitd-service:
  enabled: false

Install Helm chart

helm upgrade --install truefoundry oci://tfy.jfrog.io/tfy-helm/truefoundry -n truefoundry --create-namespace -f truefoundry-values.yaml

FAQ

Can I use the images from my own private registry?

If you want to use private image registry for the LLM Gateway, you can do so by following the instructions below.

Get the list of images for AI Gateway installation.

Replace $TRUEFOUNDRY_HELM_CHART_VERSION with the version of the Truefoundry helm chart you want to use. You can find the latest version here.

export TRUEFOUNDRY_HELM_CHART_VERSION=<TRUEFOUNDRY_HELM_CHART_VERSION>
# Get the list of images for AI Gateway installation.
curl -s https://raw.githubusercontent.com/truefoundry/infra-charts/main/scripts/extract_images_from_helm_chart.sh | bash -s -- --repo oci://tfy.jfrog.io/tfy-helm --chart truefoundry --version $TRUEFOUNDRY_HELM_CHART_VERSION --values truefoundry-values.yaml

Push the images to your registry. We provide you a script that makes it easy to push the images to your registry.

curl -s https://raw.githubusercontent.com/truefoundry/infra-charts/main/scripts/clone_images_to_your_registry.sh | bash -s -- --input truefoundry-images-$TRUEFOUNDRY_HELM_CHART_VERSION --dest-registry <YOUR_REGISTRY>

The <YOUR_REGISTRY> should be your registry URL like docker.io or registry.example.com. The script assumes you are logged into your registry.

Update the truefoundry-values.yaml file to use your custom images.

global:
  image:
    registry: <YOUR_REGISTRY>
postgresql:
  image:
    registry: <YOUR_REGISTRY> # Replace with your registry
tfy-clickhouse:
  altinity-clickhouse-operator:
    metrics:
      image:
        repository: <YOUR_REGISTRY>/tfy-mirror/altinity/metrics-exporter
    operator:
      image:
        repository: <YOUR_REGISTRY>/tfy-mirror/altinity/clickhouse-operator
  zookeeper:
    image:
      registry: <YOUR_REGISTRY>

Can I use my Artifactory as a mirror to pull images?

Do we need any NFS volumes in Kubernetes for the AI Gateway or Control Plane?

What is the structure of access logs

We log access information in standard output with the following format:

logfmt
json

These can be switched with the help of an environment variable to the AI Gateway installation. (Default: logfmt)

Log format

Standard log format structure:

time="%START_TIME%" level=%LEVEL% ip=%IP_ADDRESS% tenant=%TENANT_NAME% user=%SUBJECT_TYPE%:%SUBJECT_SLUG% model=%MODEL_ID% method=%METHOD% path=%PATH% status=%STATUS_CODE% time_taken=%DURATION%ms trace_id=%TRACE_ID%

Log operator	Details
START_TIME	ISO timestamp for request start. eg. 2025-08-12 13:34:50
LEVEL	info\|warn\|error
IP_ADDRESS	IP address of the caller. eg. ::ffff:10.99.55.142
TENANT_NAME	Name of the tenant. eg. truefoundry
SUBJECT_TYPE	user\|virtualaccount
SUBJECT_SLUG	Email or virtual account name. eg. tfy-user@truefoundry.com\|demo-virtualaccount
MODEL_ID	Model ID. eg. openai-default/gpt-5
METHOD	GET\|POST\|PUT
PATH	Path of the request. eg. /api/inference/openai/chat/completions
STATUS_CODE	200\|400\|401\|403\|429\|500
DURATION	Duration of the request. eg. 12
TRACE_ID	Trace ID of the request

Examples:

time="2025-08-12 13:34:50" level=info ip=::ffff:10.99.55.142 tenant=truefoundry user=virtualaccount:demo-virtualaccount model=openai-default/gpt-5 method=POST path=/api/inference/openai/chat/completions status=200 time_taken=53ms trace_id=587b2a946c13f62f9160674a8c983ce3

Get Started

Developer Guide

MCP Registry and Gateway

Observability

Integrations

Deployment

API Reference

Chat

Agent

MCP

Embeddings

Rerank

Responses

Image

Audio

Batch

Files

Moderations

Models

Deploy AI Gateway

Key Components

Compute Requirements

Prerequisites for Installation

Installation Instructions

Allow your IAM role to be assumed by Control Plane IAM Role

FAQ

Log format

Get Started

Developer Guide

MCP Registry and Gateway

Observability

Integrations

Deployment

API Reference

Chat

Agent

MCP

Embeddings

Rerank

Responses

Image

Audio

Batch

Files

Moderations

Models

​Key Components

​Compute Requirements

​Prerequisites for Installation

​Installation Instructions

​FAQ

Key Components

Compute Requirements

Prerequisites for Installation

Installation Instructions

FAQ