Add TLS Certificates

Adding TLS Certificates to Your TrueFoundry Deployment

This guide explains how to configure TLS certificates to enable secure HTTPS access to your TrueFoundry deployment. We’ll cover multiple approaches based on your cloud provider.

Quick Reference Guide

Cloud Provider	Recommended Method	Alternative Methods	Reference Guide
AWS	AWS Certificate Manager	cert-manager with DNS validation	AWS DNS & TLS Setup
GCP	cert-manager with Cloud DNS	Manual certificate files	GCP DNS & TLS Setup
Azure	cert-manager with Azure DNS	Manual certificate files	Azure DNS & TLS Setup
Generic	Manual certificate files	cert-manager with Let’s Encrypt	Generic Cluster Setup

AWS: Using Certificate Manager

When running TrueFoundry on AWS EKS, you have two options for TLS termination:

Terminate TLS at the Network Load Balancer (recommended)
Terminate TLS at the Istio ingress layer

For production AWS deployments, terminating TLS at the Network Load Balancer using AWS Certificate Manager (ACM) is recommended for best performance and manageability.

Step-by-Step Guide for AWS Certificate Manager

Create a certificate in ACM:
- Navigate to AWS Certificate Manager in the AWS console
- Request a public certificate
- Specify your domain (e.g., *.example.com)
- Choose DNS validation (recommended)
Validate domain ownership:
- Add the CNAME records provided by ACM to your DNS provider
  - Follow the official AWS guide for DNS validation
  - For detailed steps on adding CNAME records, see AWS documentation on DNS validation
- Wait for the certificate to change to “Active” status (this may take 30 minutes or longer)
- Copy the certificate ARN for the next step (format will be like: arn:aws:acm:region:account:certificate/certificate-id)
Apply the certificate to your TrueFoundry deployment:
- In the TrueFoundry platform, navigate to Deployments > Helm
- Filter to find the helm chart for your cluster
- Select tfy-istio-ingress
- Click Edit and update the configuration:
gateway: annotations: service.beta.kubernetes.io/aws-load-balancer-ssl-cert: <your-certificate-arn-here>

Configure domain routing:

In the same tfy-istio-ingress configuration, update the gateway configuration with your custom host:

tfyGateway:
  name: 'tfy-wildcard'
  spec:
    selector:
      istio: 'tfy-istio-ingress'
    servers:
      - hosts:
        - "<EDIT HERE *.example.com>"
        port:
          name: http-tfy-wildcard
          number: 80
          protocol: HTTP
        tls:
          httpsRedirect: true
      - hosts: 
        - "<EDIT HERE *.example.com>"
        port:
          name: https-tfy-wildcard
          number: 443
          protocol: HTTPS

GCP: Using cert-manager with Cloud DNS

For GCP deployments, we recommend using cert-manager with Let’s Encrypt and GCP Cloud DNS for automatic certificate issuance and renewal.

PrerequisitesThis approach requires workload identity to be enabled on your GKE cluster. For clusters created through Terraform code generated by TrueFoundry, this is enabled by default.

Step 1: Configure GCP Service Account and Permissions

# Set your variables
export CLUSTER_NAME="<your-gke-cluster>"
export CLUSTER_LOCATION="<your-cluster-zone>" #(us-central1)

# Get your GCP project ID (if you don't know it)
export PROJECT_ID=$(gcloud config get-value project)
# Or list available projects and select one
# gcloud projects list

# This is a suggested role name that you can customize
export DNS_ROLE_NAME="<cert_manager_dns_role>"
export GCP_SERVICEACCOUNT_NAME="cert-manager-dns"
export MAIL_ID="<your-email@example.com>"

# Get load balancer IP (once cluster is set up)
export LOAD_BALANCERIP=$(kubectl get svc \
-n istio-system tfy-istio-ingress \
-ojsonpath='{.status.loadBalancer.ingress[0].ip}')

# Verify values
echo "PROJECT_ID: ${PROJECT_ID}"
echo "CLUSTER_NAME: ${CLUSTER_NAME}"
echo "CLUSTER_LOCATION: ${CLUSTER_LOCATION}"
echo "LOAD_BALANCERIP: ${LOAD_BALANCERIP}"

# Set project
gcloud config set project $PROJECT_ID

# Create service account
gcloud iam service-accounts create $GCP_SERVICEACCOUNT_NAME \
  --display-name "$GCP_SERVICEACCOUNT_NAME"

# Create custom role with minimal DNS permissions
gcloud iam roles create custom_dns_role \
  --project=$PROJECT_ID \
  --title=$DNS_ROLE_NAME \
  --permissions=dns.resourceRecordSets.create,dns.resourceRecordSets.delete,dns.resourceRecordSets.get,dns.resourceRecordSets.list,dns.resourceRecordSets.update,dns.changes.create,dns.changes.get,dns.changes.list,dns.managedZones.list

# Bind role to service account
gcloud projects add-iam-policy-binding $PROJECT_ID \
  --condition=None \
  --member=serviceAccount:$GCP_SERVICEACCOUNT_NAME@$PROJECT_ID.iam.gserviceaccount.com \
  --role="projects/$PROJECT_ID/roles/custom_dns_role"

# Allow cert-manager to use the service account through workload identity
gcloud iam service-accounts add-iam-policy-binding \
  --role roles/iam.workloadIdentityUser \
  --member "serviceAccount:$PROJECT_ID.svc.id.goog[cert-manager/cert-manager]" \
  $GCP_SERVICEACCOUNT_NAME@$PROJECT_ID.iam.gserviceaccount.com

Step 2: Configure DNS for Your Domain

# Set DNS zone name
export DNS_HOSTED_ZONE="example.com"
export DNS_ZONE_RESOURCE_GROUP="${PROJECT_ID}"

# Create DNS zone (if it doesn't exist)
gcloud dns managed-zones create example-com \
  --description="DNS zone for ${DNS_HOSTED_ZONE}" \
  --dns-name=${DNS_HOSTED_ZONE} \
  --visibility=public
  
# Get nameservers
gcloud dns managed-zones describe example-com \
  --format="json" | jq -r '.nameServers[]'

Add the nameservers from the output to your domain registrar (GoDaddy, Namecheap, etc.). If using an existing DNS zone, you can list all zones and find yours:

# List all DNS zones
gcloud dns managed-zones list

# Get details of your specific zone
gcloud dns managed-zones describe YOUR_ZONE_NAME

Step 3: Install and Configure cert-manager (Skip for clusters Created through TrueFoundry-generated Terraform)

Note: If you installed your cluster using Terraform code generated by the TrueFoundry platform, cert-manager is already installed and configured correctly. You can skip this step and proceed to Step 4.

In the TrueFoundry platform, navigate to Integrations
Select your cluster
Click the three dots and select Manage Applications
Install cert-manager if not already installed
If already installed, edit its configuration:
- Go to Deployments > Helm
- Filter for your cluster
- Find and edit cert-manager
- Ensure the configuration includes:
extraArgs: - --issuer-ambient-credentials serviceAccount: create: true annotations: iam.gke.io/gcp-service-account: $GCP_SERVICEACCOUNT_NAME@$PROJECT_ID.iam.gserviceaccount.com
Replace placeholders with your actual values.

Step 4: Create Certificate Issuer and Request Certificate

Get your cluster credentials:

gcloud container clusters get-credentials $CLUSTER_NAME --zone $CLUSTER_LOCATION --project $PROJECT_ID

Create an issuer for Let’s Encrypt:

kubectl apply -f - <<EOF
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: letsencrypt-issuer
  namespace: istio-system
spec:
  acme:
    email: $MAIL_ID
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-key
    solvers:
    - dns01:
        cloudDNS:
          project: $PROJECT_ID
EOF

Request a certificate:

kubectl apply -f - <<EOF
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: example-com-cert
  namespace: istio-system
spec:
  secretName: example-com-tls
  duration: 2160h # 90d
  renewBefore: 360h # 15d
  issuerRef:
    name: letsencrypt-issuer
  dnsNames:
  - "example.com"
  - "*.example.com"
EOF

Check certificate status:
kubectl get certificates -n istio-system
Troubleshooting
If the certificate remains in a non-ready state for more than 10 minutes, check the cert-manager logs:

kubectl logs -n cert-manager -l app=cert-manager

Step 5: Configure TLS in Your Ingress Gateway

In the TrueFoundry platform, navigate to Deployments > Helm
Filter for your cluster and select tfy-istio-ingress

Update the configuration:

tfyGateway:
  name: tfy-wildcard
  spec:
    selector:
      istio: tfy-istio-ingress
    servers:
      - hosts:
          - "*.example.com"
          - "example.com"
        port:
          name: http-tfy-wildcard
          number: 80
          protocol: HTTP
        tls:
          httpsRedirect: true
      - hosts:
          - "*.example.com"
          - "example.com"
        port:
          name: https-tfy-wildcard
          number: 443
          protocol: HTTPS
        tls:
          mode: SIMPLE
          credentialName: example-com-tls

Step 6: Update Cluster Metadata with Domain

Navigate to Integrations in the TrueFoundry platform
Select your cluster and click Edit
Enable Show advanced fields
Enable Base Domain URL
Add your domain (e.g., example.com)

Azure: Using cert-manager with Azure DNS

For Azure deployments, we use cert-manager with Let’s Encrypt and Azure DNS for automated certificate management.

Step 1: Configure Variables and Workload Identity

# Set your variables
export CLUSTER_NAME="<your-aks-cluster>"
export RESOURCE_GROUP="<your-resource-group>"
export AZURE_SUBSCRIPTION_ID="<your-subscription-id>"
export SERVICE_ACCOUNT_NAME=cert-manager
export SERVICE_ACCOUNT_NAMESPACE=cert-manager
export MAIL_ID="<your-email@example.com>"

# Get OIDC issuer URL and load balancer IP
export OIDC_ISSUER_URL=$(az aks show \
--resource-group $RESOURCE_GROUP \
--name $CLUSTER_NAME \
--query "oidcIssuerProfile.issuerUrl" -o tsv)

export LOAD_BALANCERIP=$(kubectl get svc \
-n istio-system tfy-istio-ingress \
-ojsonpath='{.status.loadBalancer.ingress[0].ip}')

# Create and get managed identity
export IDENTITY_NAME="$CLUSTER_NAME"

# Create managed identity and get IDs
PRINCIPAL_ID=$(az identity create \
--name "${IDENTITY_NAME}" \
--resource-group "${RESOURCE_GROUP}" \
--query principalId -otsv)

IDENTITY_CLIENT_ID=$(az identity show \
--name "${IDENTITY_NAME}" \
--resource-group "${RESOURCE_GROUP}" \
--query 'clientId' -otsv)

# Verify values
echo "PRINCIPAL_ID: ${PRINCIPAL_ID}"
echo "IDENTITY_CLIENT_ID: ${IDENTITY_CLIENT_ID}"
echo "OIDC_ISSUER_URL: ${OIDC_ISSUER_URL}"
echo "LOAD_BALANCERIP: ${LOAD_BALANCERIP}"

If needed, enable workload identity on your AKS cluster:

az aks update \
  --name ${CLUSTER_NAME} \
  --resource-group ${RESOURCE_GROUP} \
  --enable-oidc-issuer \
  --enable-workload-identity

Step 2: Configure DNS Zone and Permissions

Create a DNS zone (or use existing one):

# Set DNS zone name
export DNS_HOSTED_ZONE="example.com"
export DNS_ZONE_RESOURCE_GROUP="${RESOURCE_GROUP}"

# Create DNS zone
az network dns zone create \
  --name ${DNS_HOSTED_ZONE} \
  --resource-group ${DNS_ZONE_RESOURCE_GROUP} \
  --query nameServers

Add the nameservers from the output to your domain registrar. If using an existing DNS zone:

# Get the DNS zone ID
DNS_ZONE_ID=$(az network dns zone show \
  --name ${DNS_HOSTED_ZONE} \
  --resource-group ${DNS_ZONE_RESOURCE_GROUP} \
  --query id -otsv)

# Assign permissions
az role assignment create \
  --assignee $PRINCIPAL_ID \
  --role "DNS Zone Contributor" \
  --scope $DNS_ZONE_ID

# Set up federated credentials
az identity federated-credential create \
  --name "cert-manager" \
  --identity-name "${IDENTITY_NAME}" \
  --issuer "${OIDC_ISSUER_URL}" \
  --resource-group "${RESOURCE_GROUP}" \
  --subject "system:serviceaccount:${SERVICE_ACCOUNT_NAMESPACE}:${SERVICE_ACCOUNT_NAME}"

Step 3: Install and Configure cert-manager

In TrueFoundry, navigate to Integrations > [Your Cluster] > Manage Applications
Install cert-manager if not already installed

Configure with these values:

installCRDs: true
extraArgs:
  - --issuer-ambient-credentials
podLabels:
  azure.workload.identity/use: "true"
serviceAccount:
  labels:
    azure.workload.identity/use: "true"

Step 4: Create Certificate Issuer and Request Certificate

Get your cluster credentials:

az aks get-credentials --name $CLUSTER_NAME --resource-group $RESOURCE_GROUP

Create an issuer:

kubectl apply -f - <<EOF
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: azure-issuer
  namespace: istio-system
spec:
  acme:
    email: $MAIL_ID
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: azure-issuer-key
    solvers:
    - dns01:
        azureDNS:
          hostedZoneName: $DNS_HOSTED_ZONE
          resourceGroupName: $DNS_ZONE_RESOURCE_GROUP
          subscriptionID: $AZURE_SUBSCRIPTION_ID
          environment: AzurePublicCloud
          managedIdentity:
            clientID: $IDENTITY_CLIENT_ID
EOF

Request a certificate:

kubectl apply -f - <<EOF
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: example-cert
  namespace: istio-system
spec:
  secretName: example-com-tls
  duration: 2160h # 90d
  renewBefore: 360h # 15d
  issuerRef:
    name: azure-issuer
  dnsNames:
  - "example.com"
  - "*.example.com"
EOF

Check certificate status:

kubectl get certificates -n istio-system

Step 5: Configure TLS in Your Ingress Gateway

In the TrueFoundry platform, navigate to Deployments > Helm
Filter for your cluster and select tfy-istio-ingress

Update the configuration:

tfyGateway:
  name: tfy-wildcard
  spec:
    selector:
      istio: tfy-istio-ingress
    servers:
      - hosts:
          - "*.example.com"
          - "example.com"
        port:
          name: http-tfy-wildcard
          number: 80
          protocol: HTTP
        tls:
          httpsRedirect: true
      - hosts:
          - "*.example.com"
          - "example.com"
        port:
          name: https-tfy-wildcard
          number: 443
          protocol: HTTPS
        tls:
          mode: SIMPLE
          credentialName: example-com-tls

Step 6: Update Cluster Metadata with Domain

Navigate to Integrations in the TrueFoundry platform
Select your cluster and click Edit
Enable Show advanced fields
Enable Base Domain URL
Add your domain (e.g., example.com)

Generic Cluster: Using cert-manager with Let’s Encrypt

For generic Kubernetes clusters, you can use cert-manager with Let’s Encrypt to automatically issue and manage certificates.

Step 1: Install cert-manager

To install cert-manager through TrueFoundry’s Addon/Helm section:

Navigate to Clusters in the TrueFoundry platform
Select your cluster
Click the three dots (⋮) and select Manage Addons
Look for cert-manager in the list of available applications
Click Install

Step 2: Create a ClusterIssuer for Let’s Encrypt

# Set your email address
export EMAIL="<your-email@example.com>"

# Create HTTP01 issuer for Let's Encrypt (production)
kubectl apply -f - <<EOF
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: ${EMAIL}
    privateKeySecretRef:
      name: letsencrypt-prod-key
    solvers:
    - http01:
        ingress:
          class: istio
EOF

Step 2.5: Add VirtualService for ACME HTTP-01 Challenges

Add the following VirtualService to ensure ACME HTTP-01 challenge requests are routed correctly to the cert-manager solver:

YAML

# VirtualService for ACME challenges
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: cm-acme-http-solver
  namespace: istio-system
spec:
  gateways:
    - istio-system/tfy-wildcard
  hosts:
    - "<your-domain-here>"
  http:
    - match:
        - uri:
            prefix: "/.well-known/acme-challenge/"
      route:
        - destination:
            host: cm-acme-http-solver
            port:
              number: 8089
    - match:
        - uri:
            prefix: "/"
      redirect:
        scheme: https
        redirectCode: 308

Notes:

Replace <your-domain-here> with your actual domain (e.g., example.com or *.example.com).
The host under destination should match the name of the cert-manager solver service. It is usually cm-acme-http-solver, but you can confirm this by running:
```
kubectl get svc -n istio-system | grep cm-acme-http-solver
```
This VirtualService should be created in the same namespace as your Istio ingress gateway (commonly istio-system).

Using Your Own Certificate Files

If you have your own certificate files (e.g., from another certificate provider or self-signed), you can use them directly with TrueFoundry.

Note for Generic Kubernetes Clusters

This method is particularly useful for generic Kubernetes deployments or when you want to use certificates issued by your organization’s certificate authority.

Option 1: Import Existing Certificate Files

If you already have certificate and key files:

Create a Kubernetes secret with your certificate and key:

# Create secret from local certificate files
kubectl create secret tls my-tls-secret \
  --cert=path/to/cert/file \
  --key=path/to/key/file \
  -n istio-system

Alternatively, you can create the secret using a YAML definition:

apiVersion: v1
kind: Secret
metadata:
  name: my-tls-secret
  namespace: istio-system
type: kubernetes.io/tls
data:
  tls.crt: <base64-encoded-certificate>
  tls.key: <base64-encoded-private-key>

TipTo encode your certificate and key files in base64:

cat path/to/cert/file | base64 -w 0
cat path/to/key/file | base64 -w 0

Configure the ingress gateway to use your certificate:

tfyGateway:
  name: tfy-wildcard
  spec:
    selector:
      istio: tfy-istio-ingress
    servers:
      - hosts:
          - "*.example.com"
          - "example.com"
        port:
          name: http-tfy-wildcard
          number: 80
          protocol: HTTP
        tls:
          httpsRedirect: true
      - hosts:
          - "*.example.com"
          - "example.com"
        port:
          name: https-tfy-wildcard
          number: 443
          protocol: HTTPS
        tls:
          mode: SIMPLE
          credentialName: my-tls-secret

Option 2: Generate Self-Signed Certificates

For testing or internal use, you can generate self-signed certificates:

# Generate a self-signed certificate
openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
  -keyout tls.key -out tls.crt \
  -subj "/CN=*.example.com" \
  -addext "subjectAltName = DNS:example.com,DNS:*.example.com"

# Create a secret using the generated files
kubectl create secret tls self-signed-tls \
  --cert=tls.crt \
  --key=tls.key \
  -n istio-system

Then configure your gateway to use the self-signed-tls secret as shown in Option 1.

⚠️WarningSelf-signed certificates will cause browser warnings. They should only be used for testing or internal systems.

Troubleshooting

Certificate not issued: Check cert-manager logs in the cert-manager namespace
HTTPS not working: Verify your secret name matches the credential name in the gateway configuration
DNS errors: Make sure your nameservers are correctly configured at your domain registrar

Getting Started

Train and Deploy Models

Service Deployment

Job Deployment

Workflow Deployment

Async Service Deployment

Volumes

ML Repository

Platform

Deploying On Your Own Cloud

Adding TLS Certificates to Your TrueFoundry Deployment

Quick Reference Guide

AWS: Using Certificate Manager

Step-by-Step Guide for AWS Certificate Manager

GCP: Using cert-manager with Cloud DNS

Step 1: Configure GCP Service Account and Permissions

Step 2: Configure DNS for Your Domain

Step 3: Install and Configure cert-manager (Skip for clusters Created through TrueFoundry-generated Terraform)

Step 4: Create Certificate Issuer and Request Certificate

Troubleshooting

Step 5: Configure TLS in Your Ingress Gateway

Step 6: Update Cluster Metadata with Domain

Azure: Using cert-manager with Azure DNS

Step 1: Configure Variables and Workload Identity

Step 2: Configure DNS Zone and Permissions

Step 3: Install and Configure cert-manager

Step 4: Create Certificate Issuer and Request Certificate

Step 5: Configure TLS in Your Ingress Gateway

Step 6: Update Cluster Metadata with Domain

Generic Cluster: Using cert-manager with Let’s Encrypt

Step 1: Install cert-manager

Step 2: Create a ClusterIssuer for Let’s Encrypt

Step 2.5: Add VirtualService for ACME HTTP-01 Challenges

Using Your Own Certificate Files

Note for Generic Kubernetes Clusters

Option 1: Import Existing Certificate Files

Option 2: Generate Self-Signed Certificates

Troubleshooting

Getting Started

Train and Deploy Models

Service Deployment

Job Deployment

Workflow Deployment

Async Service Deployment

Volumes

ML Repository

Platform

Deploying On Your Own Cloud

​Adding TLS Certificates to Your TrueFoundry Deployment

​Quick Reference Guide

​AWS: Using Certificate Manager

​Step-by-Step Guide for AWS Certificate Manager

​GCP: Using cert-manager with Cloud DNS

​Step 1: Configure GCP Service Account and Permissions

​Step 2: Configure DNS for Your Domain

​Step 3: Install and Configure cert-manager (Skip for clusters Created through TrueFoundry-generated Terraform)

​Step 4: Create Certificate Issuer and Request Certificate

​Troubleshooting

​Step 5: Configure TLS in Your Ingress Gateway

​Step 6: Update Cluster Metadata with Domain

​Azure: Using cert-manager with Azure DNS

​Step 1: Configure Variables and Workload Identity

​Step 2: Configure DNS Zone and Permissions

​Step 3: Install and Configure cert-manager

​Step 4: Create Certificate Issuer and Request Certificate

​Step 5: Configure TLS in Your Ingress Gateway

​Step 6: Update Cluster Metadata with Domain

​Generic Cluster: Using cert-manager with Let’s Encrypt

​Step 1: Install cert-manager

​Step 2: Create a ClusterIssuer for Let’s Encrypt

​Step 2.5: Add VirtualService for ACME HTTP-01 Challenges

​Using Your Own Certificate Files

​Note for Generic Kubernetes Clusters

​Option 1: Import Existing Certificate Files

​Option 2: Generate Self-Signed Certificates

​Troubleshooting

Adding TLS Certificates to Your TrueFoundry Deployment

Quick Reference Guide

AWS: Using Certificate Manager

Step-by-Step Guide for AWS Certificate Manager

GCP: Using cert-manager with Cloud DNS

Step 1: Configure GCP Service Account and Permissions

Step 2: Configure DNS for Your Domain

Step 3: Install and Configure cert-manager (Skip for clusters Created through TrueFoundry-generated Terraform)

Step 4: Create Certificate Issuer and Request Certificate

Troubleshooting

Step 5: Configure TLS in Your Ingress Gateway

Step 6: Update Cluster Metadata with Domain

Azure: Using cert-manager with Azure DNS

Step 1: Configure Variables and Workload Identity

Step 2: Configure DNS Zone and Permissions

Step 3: Install and Configure cert-manager

Step 4: Create Certificate Issuer and Request Certificate

Step 5: Configure TLS in Your Ingress Gateway

Step 6: Update Cluster Metadata with Domain

Generic Cluster: Using cert-manager with Let’s Encrypt

Step 1: Install cert-manager

Step 2: Create a ClusterIssuer for Let’s Encrypt

Step 2.5: Add VirtualService for ACME HTTP-01 Challenges

Using Your Own Certificate Files

Note for Generic Kubernetes Clusters

Option 1: Import Existing Certificate Files

Option 2: Generate Self-Signed Certificates

Troubleshooting