Compute Requirements

To install the control-plane, we need a Kubernetes cluster and a managed Postgres database. Truefoundry ships as a helm chart (https://github.com/truefoundry/infra-charts/tree/main/charts/truefoundry) that has configurable options to either deploy both Deployment and AI Gateway feature or just choose the one of them according to your needs. The compute requirements change based on the set of features and the scale of the number of users and requests.

Here are a few scenarios that you can choose from based on your needs.

The small tier is recommended for development purposes. Here all the components are deployed on Kubernetes and in non HA mode (single replica). This is suitable if you are just testing out the different features of Truefoundry.

This setup brings up 1 replica of the services and is not highly-available. It can enable you to test the features but we do not recommend this for production mode.

ComponentCPUMemoryStorageMin NodesRemarks
Helm-Chart
(AI Gateway Control Plane components)
2 vCPU8GB60GB
Persistent Volumes (Block Storage) On Kubernetes
2
Pods should be spread over min 2 nodes
Cost: ~ $120 pm
Helm-Chart
(AI Gateway component only)
1 vCPU512Mi-1
Pods should be spread over min 1 node
Cost: ~ $10 pm
Postgres
(Deployed on Kubernetes)
0.5 vCPU0.5GB5GB
Persistent Volumes (Block Storage) On Kubernetes
PostgreSQL version >= 13
Blob Storage
(S3 Compatible)
20GB

Prerequisites for Installation

  1. Egress access to auth.truefoundry.com and analytics.truefoundry.com
  2. Domain to map the ingress of the frontend and gateway
  3. Tenant, Licence key, and image pull secret from TrueFoundry team

Installation Instructions

  1. Create a values file as given below and replace relevant values:
    global:
      # Ask TrueFoundry team to provide these
      tenantName: <TENANT_NAME>
      tfyApiKey: <LICENCE_KEY>
      # Pass the secret name containing the tfy api key. Use the key 'TFY_API_KEY' for the key name
      # existingTruefoundryCredsSecret: ""
      truefoundryImagePullConfigJSON: <TRUEFOUNDRY_IMAGE_PULL_SECRET>
      # If you are using an existing image pull secret, set the name here
      # existingTruefoundryImagePullSecretName: ""
      
      # Domain to map the platform to
      controlPlaneURL: https://example.com
      resourceTier: small # or medium or large
    
    truefoundryFrontendApp:
      # Choose the service mesh: we support both istio and k8s ingress, enable as per your setting
      # Replace `example.com` with your Domain
      istio:
        virtualservice:
          hosts:
            - example.com
          enabled: false
          gateways:
            - istio-system/tfy-wildcard
      ingress:
        hosts:
          - example.com
        enabled: false
        annotations: {}
        ingressClassName: nginx
    tags:
      llmGateway: true
      llmGatewayRequestLogging: true
    # Enable if you want to use devMode
    devMode:
      enabled: false
    # Disable few depenencies for only LLM Gateway setup
    tfyBuild:
      enabled: false
    tfyController:
      enabled: false
    mlfoundryServer:
      enabled: false
    tfy-buildkitd-service:
      enabled: false
    
  2. Add helm repo
    helm repo add truefoundry https://truefoundry.github.io/infra-charts
    helm repo update truefoundry
    
  3. Install the helm chart with your values file
    helm upgrade --install truefoundry truefoundry/truefoundry -n truefoundry --create-namespace -f truefoundry-values.yaml