Azure

Provisioning Control Plane Infrastructure on Azure

🚧

There are steps in this guide where Truefoundry team will have to be involved. Please reach out to [email protected] to get the credentials

Setting up Truefoundry control plane on your own cloud involves creating the infrastructure to support the platform and then installing the platform itself.

Setting up Infrastructure

Requirements

These are the infrastructure components required to set up a production grade Truefoundry control plane.

📘

If you have the below requirements already set up then skip directly to the Installation section

RequirementsDescriptionReason for Requirement
Kubernetes ClusterAny Kubernetes cluster will work here - we can also choose the compute-plane cluster itself to install Truefoundry helm chart. The Truefoundry helm chart will be installed here.
Azure Flexible Server for PostgreSQLPostgres >= 13The database is used by Truefoundry control plane to store all its metadata.
Container in Azure Storage AccountAny container bucket reachable from control-plane.This is used by control-plane to store the intermediate code while building the docker image.
Egress Access for TruefoundryAuthEgress access to https://auth.truefoundry.comThis is needed to validate the users logging into Truefoundry so that licensing can be maintained.
Egress access For Docker Registry1 public.ecr.aws
2. quay.io
3. ghcr.io
4. docker.io/truefoundrycloud
5. docker.io/natsio
6. nvcr.io
7. registry.k8s.io
This is to download docker images for Truefoundry, ArgoCD, NATS, ArgoRollouts, ArgoWorkflows, Istio.
DNS with TLS/SSLOne endpoint to point to the control plane service (something like platform.example.com where example.com is your domain. There should also be a certificate with the domain so that the domains can be accessed over TLS.

The control-plane url should be reachable from the compute-plane so that compute-plane cluster can connect to the control-plane
The developers will need to access the Truefoundry UI at domain that is provided here.
User/ServiceAccount to provision the infrastructure- azure subscription with billing enabled
- Contributor Role to the above Subscription.
- Role Based Access Administrator to the above subscription
These are the permissions required by the IAM user in GCP to create the entire control plane components.

Run Infra Provisioning using OCLI

Prerequisites

  1. Install git if not already present.

  2. Setup az CLI

    1. Install azure cli >= 2.50

    2. Log in and set a subscription. Please ensure that the user has Contributor and RBAC admin roles in the Subscription

      # login
      az login
      
      # setting the subscription
      az account set --subscription $SUBSCRIPTION_ID
      

Installing OCLI

  1. Download the binary using the below command.
    curl -H 'Cache-Control: max-age=0' -s "https://releases.ocli.truefoundry.tech/binaries/ocli_$(curl -H 'Cache-Control: max-age=0' -s https://releases.ocli.truefoundry.tech/stable.txt)_darwin_arm64" -o ocli
    
    curl -H 'Cache-Control: max-age=0' -s "https://releases.ocli.truefoundry.tech/binaries/ocli_$(curl -H 'Cache-Control: max-age=0' -s https://releases.ocli.truefoundry.tech/stable.txt)_darwin_amd64" -o ocli
    
    curl -H 'Cache-Control: max-age=0' -s "https://releases.ocli.truefoundry.tech/binaries/ocli_$(curl -H 'Cache-Control: max-age=0' -s https://releases.ocli.truefoundry.tech/stable.txt)_linux_arm64" -o ocli
    
    curl -H 'Cache-Control: max-age=0' -s "https://releases.ocli.truefoundry.tech/binaries/ocli_$(curl -H 'Cache-Control: max-age=0' -s https://releases.ocli.truefoundry.tech/stable.txt)_linux_amd64" -o ocli
    
  2. Make the binary executable and move it to $PATH
    sudo chmod +x ./ocli
    sudo mv ocli /usr/local/bin
    
  3. Confirm by running the command
    ocli --version
    

Configuring input config file

  1. To create a new cluster, you would require your Azure Subscription, Location, Resource Group.
  2. Run the following command to fill in the inputs interactively
    ocli infra-init
    
  3. For networking, there are the following possible configurations:
    1. New resource group & network (Recommended) - This will create a new resource group and a new Virtual network.
    2. Existing resource group with existing network - You can use an existing resource group and an existing Virtual network.
    3. Existing resource group with new network - You can use an existing resource group while creating a new Virtual network
  4. Once all the inputs are filled, an input config file with the nametfy-config.yaml would be generated in your current directory
  5. Modify the file to enable control plane installation by setting azure.tfy_control_plane.enabled:true. Below is the sample for the same:
aws: null
azure:
  cluster:
    name: CLUSTER_NAME
    node_pools:
      sku_ondemand: Standard_D4ds_v5
      sku_spot: Standard_D2ds_v5
  location: eastus
  network:
    existing: true
    subnet_cidr: ""
    subnet_id: "/subscriptions/xxxxx-xxxxx-xxxxx-xxxxxxxxx/resourceGroups/RESOURCE_GROUP/providers/Microsoft.Network/virtualNetworks/VNET/subnets/SUBNET"
    vnet_cidr: ""
    vnet_id: "/subscriptions/xxxxx-xxxxx-xxxxx-xxxxxxxxx/resourceGroups/RESOURCE_GROUP/providers/Microsoft.Network/virtualNetworks/VNET"
    vnet_name: ""
  platform_features:
    blob_storage:
      container_enable_override: false
      container_override_name: ""
      enabled: true
      storage_account_enable_override: false
      storage_account_override_name: ""
    cloud_integration:
      azuread_application_enable_override: false
      azuread_application_override_name: ""
      enabled: true
    container_registry:
      container_registry_enable_override: false
      container_registry_override_name: ""
      enabled: true
    enabled: true
  resource_group:
    existing: true
    name: RESOURCE_GROUP
  state:
    container_name: tfy-tfstate-CLUSTER_NAME-cn-1714629250
    resource_group: tfy-tfstate-CLUSTER_NAME-rg-1714629250
    storage_account_name: tfytfstateCLUSTER_NAMEsa
  subscription:
    id: SUBSCRIPTION_ID
    name: SUBSCRIPTION_NAME
  tags: {}
  tfy_control_plane:
    enabled: false
binaries:
  terraform:
    binary_path: null
  terragrunt:
    binary_path: null
gcp: null
provider: azure
aws: null
azure:
  cluster:
    name: CLUSTER_NAME
    node_pools:
      sku_ondemand: Standard_D4ds_v5
      sku_spot: Standard_D2ds_v5
  location: eastus
  network:
    existing: false
    subnet_cidr: 10.10.0.0/16
    subnet_id: ""
    vnet_cidr: 10.0.0.0/8
    vnet_id: ""
    vnet_name: ""
  platform_features:
    blob_storage:
      container_enable_override: false
      container_override_name: ""
      enabled: true
      storage_account_enable_override: false
      storage_account_override_name: ""
    cloud_integration:
      azuread_application_enable_override: false
      azuread_application_override_name: ""
      enabled: true
    container_registry:
      container_registry_enable_override: false
      container_registry_override_name: ""
      enabled: true
    enabled: true
  resource_group:
    existing: true
    name: RESOURCE_GROUP
  state:
    container_name: tfy-tfstate-CLUSTER_NAME-cn-1714629250
    resource_group: tfy-tfstate-CLUSTER_NAME-rg-1714629250
    storage_account_name: tfytfstateCLUSTER_NAMEsa
  subscription:
    id: SUBSCRIPTION_ID
    name: SUBSCRIPTION_NAME
  tags: {}
  tfy_control_plane:
    enabled: false
binaries:
  terraform:
    binary_path: null
  terragrunt:
    binary_path: null
gcp: null
provider: azure
aws: null
azure:
  cluster:
    name: CLUSTER_NAME
    node_pools:
      sku_ondemand: Standard_D4ds_v5
      sku_spot: Standard_D2ds_v5
  location: eastus
  network:
    existing: false
    subnet_cidr: 10.10.0.0/16
    subnet_id: ""
    vnet_cidr: 10.0.0.0/8
    vnet_id: ""
    vnet_name: ""
  platform_features:
    blob_storage:
      container_enable_override: false
      container_override_name: ""
      enabled: true
      storage_account_enable_override: false
      storage_account_override_name: ""
    cloud_integration:
      azuread_application_enable_override: false
      azuread_application_override_name: ""
      enabled: true
    container_registry:
      container_registry_enable_override: false
      container_registry_override_name: ""
      enabled: true
    enabled: true
  resource_group:
    existing: false
    name: RESOURCE_GROUP
  state:
    container_name: tfy-tfstate-CLUSTER_NAME-cn-1714629250
    resource_group: tfy-tfstate-CLUSTER_NAME-rg-1714629250
    storage_account_name: tfytfstateCLUSTER_NAMEsa
  subscription:
    id: SUBSCRIPTION_ID
    name: SUBSCRIPTION_NAME
  tags: {}
  tfy_control_plane:
    enabled: false
binaries:
  terraform:
    binary_path: null
  terragrunt:
    binary_path: null
gcp: null
provider: azure

Create the cluster

Run the following command to create the GKE cluster and IAM roles needed to provide access to various infrastructure components as per the inputs configured above.

ocli infra-create --file tfy-config.yaml

This command may take around 30-45 minutes to complete.

In the last step the database credentials will be printed. Make sure to note them down.

Installation

Installation steps

  1. Make sure that kubectl context is set to the newly created cluster.
  2. Install argoCD -
    kubectl create namespace argocd
    kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/core-install.yaml
    
  3. Add the truefoundry helm repo
    helm repo add truefoundry https://truefoundry.github.io/infra-charts/
    helm repo update
    
  4. We will create a values.yaml for the helm chart installation -
    1. Download the values.yaml from helm chart repo -
      curl https://raw.githubusercontent.com/truefoundry/infra-charts/main/charts/tfy-k8s-azure-aks-inframold/values-cp.yaml > values.yaml
      
    2. Fill in the tenant_name, cluster_name, truefoundry_image_pull_config_json and tfy_api_key in the downloaded file. You can get these from the Truefoundry team
    3. Also fill in the database section with database creds. If you created the infrastructure using OCLI, you can get the credentials by running ocli output
  5. Apply the helm chart with the values.yaml
    helm install -n argocd inframold truefoundry/tfy-k8s-azure-aks-inframold --version 0.0.16 -f values.yaml
    

Test the installation

  1. Port forward the frontend application to access the Truefoundry dashboard -
    kubectl port-forward svc/truefoundry-truefoundry-frontend-app -n truefoundry 5000
    
  2. Access the truefoundry dashboard from a browser by opening http://localhost:5000. You can login with the username and password provided by the Truefoundry team.
  3. Now you are ready to connect a cluster to the Truefoundry platform and get deploying. Go here for the directions. You can also onboard the same cluster as the control plane