Documentation
BlogsCreate Account

Deploy Truefoundry in an Air-gapped Environment

Suggest Edits

Overview

An air-gapped environment simply means an environment that has been isolated from other environments or networks or more commonly from the internet.

When deploying Truefoundry components, by default there would be a need to access the internet to get the artifacts (Images and Helm Charts) that will be needed for a successful deployment. Due to the nature of an air-gapped systems, it will not be possible to get these artifacts. In order to circumvent this, we have come up with a way to get the necessary artifacts into your systems or environmnets; The following steps will explain how this will be done or achieved.

Deploying Inframold charts from a private registry would require having the necessary artifacts required to successfully deploy the chart to be available in the registry. In order to achieve this, we have a manifest file for each charts that details the required artifacts needed and a script that makes it easy to deploy those artifacts to your own private registry.

Requirements

In order to be setup Truefoundry in an air-gapped environment, the following are needed

  1. An OCI compatible registry which is accessible by the cluster.
  2. The registry should also be able to serve helm charts from an unauthenticated endpoint.
  3. A secure environment that has access to the internet and can also upload artifacts to the secure registry. This is for initial setup.
  4. The following should also be install in the environment;
    1. Docker
    2. Helm
    3. Kubectl

Uploading artifacts to a private registry

Deploying Inframold charts from a private registry would require having the necessary artifacts required to successfully deploy the chart to be available in the registry. In order to achieve this, we have an artifact-manifest file which details the required artifacts needed for each Truefoundry chart and a script that makes it easy to deploy these artifacts to your own private registry. The infra-chart repo contains a list of 3rd party chart dependencies alongside Truefoundry helm charts that can be deployed, here is a list of Truefoundry Helm Charts that can be deployed depending on the environment you wish to deploy Truefoundry into;

Steps to deploy artifacts to a private registry

  1. To start download the artifacts manifest file of the helm chart from the list above, python script and requirements.txt files locally
  2. Create a python virtual environment so as to have an isolated environment for the upload_artifact.py script. You can follow the virtualenv guide for installation.
  3. Install the python dependencies 
    pip install -r requirements.txt
  4. The upload_artifact.py is a sample python scripts which handles deployment to JFrog and ECR registries, it takes in the following arguments;
    • artifact_type - this takes in the artifact type which can be either image or helm
    • file_path - this is the location of the artefacts-manifest.json file that contains the artefacts information.
    • destination_registry - this is the registry you plan to use in your air-gapped environment.
    • registry_type the type of registry. The script currently take ecr for AWS registry and jfrog
  5. Before uploading images or charts to your registry, make sure you're authenticated. When setting up jfrog, you have to create a token the script will use. Follow this link to generate your token. Once you get the token, pass it in as an environment variable - JFROG_TOKEN before running the script.
    When deploying to ECR, the following are needed to be passed as environment variables - AWS_PROFILE and AWS_REGION.
    To update your registry with the required images to a JFROG registry, run the following command
python upload_artifacts.py image artifacts-manifest.json <jfrog-registry-tenant>.jfrog.io jfrog

python upload_artifacts.py image artifacts-manifest.json <jfrog-registry-tenant>.jfrog.io jfrog

Deploying Inframold Charts

  1. Create a namespace for kyverno
kubectl create ns kyverno

  1. If your registry needs to be authenticated before you can pull images, then you have to authenticate the pods to pull the images. Create a secret with the registry auth information

    kubectl create secret docker-registry regcred \
--docker-server=tfyregistry.jfrog.io \
--docker-username="" \ 
--docker-password="" \
-n kyverno

  2. Create a file called kyverno-values.yaml with the following content

    global:  
  image:  
    registry: tfyregistry.jfrog.io/truefoundry

## only add this section if you need to authenticate against the registry

existingImagePullSecrets:

- regcred

webhooksCleanup:  
  imagePullSecrets:  
    - name: regcred

policyReportsCleanup:  
  imagePullSecrets:  
    - name: regcred
# Cleanup cronjobs to prevent internal resources from stacking up in the cluster

cleanupJobs:  
    imagePullSecrets:  
      - name: regcred

# Cluster admission reports configuration

clusterAdmissionReports:  
    imagePullSecrets:  
      - name: regcred

# Admission controller configuration

admissionController:  
  imagePullSecrets:  
    - name: regcred

# Background controller configuration

backgroundController:  
  imagePullSecrets:  
    - name: regcred

# Cleanup controller configuration

cleanupController:  
  imagePullSecrets:  
    - name: regcred

# Reports controller configuration

reportsController:  
  imagePullSecrets:  
    - name: regcred

  3. Install kyverno helm chart

helm repo add kyverno https://kyverno.github.io/kyverno/
helm repo update  
helm install kyverno kyverno/kyverno --version 3.2.5 -n kyverno -f kyverno-values.yaml
  1. Install argocd crds, this needs to be installed first because the Kyverno ClusterPolicy references the ArgoCD CRD
kubectl apply -f https://github.com/argoproj/argo-cd/manifests/crds\?ref\=stable

  1. Install Kyverno Config Chart to patch your images

    1. Create a file called kyverno-config.yaml with the following content

      ## @section replaceImageRegistry Configuration options for replacing the image registry
replaceImageRegistry:
## @param replaceImageRegistry.enabled Enable or disable replacing the image registry
  enabled: true
## @param replaceImageRegistry.newRegistry The new image registry to use
  newRegistry: "<registry-url>"

## @section replaceArgoHelmRepo Configuration options for replacing the Argo Helm repository
replaceArgoHelmRepo:
## @param replaceArgoHelmRepo.enabled Enable or disable replacing the Argo Helm repository
  enabled: true
## @param replaceArgoHelmRepo.newRegistry The new Argo Helm registry URL
  newRegistry: "<regsitry-url>"

    2. Install the kyverno-config chart

      helm repo add truefoundry https://truefoundry.github.io/infra-charts/ 
helm repo update  
helm install kyverno-config truefoundry/tfy-kyverno-config --version 0.1.3 -n kyverno -f kyverno-config.yaml

  2. Install argoCD

    kubectl create namespace argocd  
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/core-install.yaml

  3. We will create a values.yaml for the helm chart installation

    1. Download the values.yaml from helm chart repo 
      curl https://raw.githubusercontent.com/truefoundry/infra-charts/main/charts/tfy-k8s-aws-eks-inframold/values-cp.yaml > values.yaml
    2. Fill in the tenant_name, cluster_name, truefoundry_image_pull_config_json and tfy_api_key in the downloaded file. You can get these from the Truefoundry team.
    3. Also fill in the database section with database creds. If you created the infrastructure using OCLI, you can get the credentials by running ocli output

  4. Apply the helm chart with the values.yaml

    helm install -n argocd inframold truefoundry/tfy-k8s-aws-eks-inframold --version 0.0.16 -f values.yaml

Updated about 1 month ago