AWS

Provisioning Control Plane Infrastructure on AWS

Dev Mode

Infrastructure Requirements

RequirementsDescriptionReason for Requirement
Kubernetes ClusterAny Kubernetes cluster will work here - we can also choose the compute-plane cluster itself to install Truefoundry helm chart. The Truefoundry helm chart will be installed here. Need to add the size of cluster, ebs is needed for NATS
Egress Access for TruefoundryAuthEgress access to https://auth.truefoundry.comThis is needed to verify the users logging into the Truefoundry platform for licensing purposes
Egress access For Docker Registry1. public.ecr.aws
2. quay.io
3. ghcr.io
4. docker.io/truefoundrycloud
5. docker.io/natsio
6. nvcr.io
7. registry.k8s.io
This is to download docker images for Truefoundry, ArgoCD, NATS, ArgoRollouts, ArgoWorkflows, Istio

Installation steps

  1. Create truefoundry namespace
    kubectl create ns truefoundry
    
  2. Create an imagePullSecret for the Truefoundry images in the truefoundry namespace
    cat <<EOF | kubectl apply -f -
    apiVersion: v1
    kind: Secret
    metadata:
      name: truefoundry-image-pull-secret
      namespace: truefoundry
    data:
      .dockerconfigjson: <image pull secret provided by truefoundry team>
    type: kubernetes.io/dockerconfigjson
    EOF
    
  3. Install argoCD -
    kubectl create namespace argocd
    kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/core-install.yaml
    
  4. Add the truefoundry helm repo
    helm repo add truefoundry https://truefoundry.github.io/infra-charts/
    helm repo update
    
  5. We will create a values.yaml for the helm chart installation -
    1. Download the values.yaml from helm chart repo -
      curl https://raw.githubusercontent.com/truefoundry/infra-charts/main/charts/tfy-k8s-aws-eks-inframold/values.yaml > values.yaml
      
    2. Fill in the tenantName, clusterName and sfyApiKey in the downloaded file
  6. Apply the helm chart with the values.yaml. Replace the chart_name with the correct one from step 3
    helm install -n argocd inframold truefoundry/tfy-k8s-aws-eks-inframold --version 0.0.9 -f values.yaml
    

Test the installation

  1. Port forward the frontend application to access the Truefoundry dashboard -
    kubectl port-forward svc/truefoundry-truefoundry-frontend-app -n truefoundry 5000
    
  2. Access the truefoundry dashboard from a browser by opening http://localhost:5000. You can login with the username and password provided by the Truefoundry team.
  3. Now you are ready to connect a cluster to the Truefoundry platform and get deploying. Go here for the directions. You can also onboard the same cluster as the control plane

Prod mode


Infrastructure Requirements

RequirementsDescriptionReason for Requirement
Kubernetes ClusterAny Kubernetes cluster will work here - we can also choose the compute-plane cluster itself to install Truefoundry helm chart. The Truefoundry helm chart will be installed here.
Postgres RDSPostgres >= 13The database is used by Truefoundry control plane to store all its metadata.
S3 bucketAny S3 bucket reachable from control-plane.This is used by control-plane to store the intermediate code while building the docker image.
Egress Access for TruefoundryAuthEgress access to https://auth.truefoundry.comThis is needed to verify the users logging into the Truefoundry platform for licensing purposes
Egress access For Docker Registry1. public.ecr.aws
2. quay.io
3. ghcr.io
4. docker.io/truefoundrycloud
5. docker.io/natsio
6. nvcr.io
7. registry.k8s.io
This is to download docker images for Truefoundry, ArgoCD, NATS, ArgoRollouts, ArgoWorkflows, Istio.
DNS with TLS/SSLOne endpoint to point to the control plane service (something like platform.example.com where example.com is your domain. There should also be a certificate with the domain so that the domains can be accessed over TLS.

The control-plane url should be reachable from the compute-plane so that compute-plane cluster can connect to the control-plane
The developers will need to access the Truefoundry UI at domain that is provided here.
User/ServiceAccount to provision the infrastructureThis is the set of permissions needed to provision the infrastructure for Truefoundry control-plane.

Permissions required to create the infrastructure

export REGION="" # us-east-1
export SHORT_REGION="" #usea1
export ACCOUNT_ID="" #123524493244
export NAME="" 
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "iam:CreateInstanceProfile",
                "iam:DeleteInstanceProfile",
                "rds:AddTagsToResource",
                "iam:GetInstanceProfile",
                "iam:RemoveRoleFromInstanceProfile",
                "rds:DeleteTenantDatabase",
                "iam:AddRoleToInstanceProfile",
                "rds:CreateDBInstance",
                "rds:DescribeDBInstances",
                "rds:RemoveTagsFromResource",
                "rds:CreateTenantDatabase",
                "iam:TagInstanceProfile",
                "rds:DeleteDBInstance"
            ],
            "Resource": [
                "arn:aws:iam::$ACCOUNT_ID:instance-profile/*",
                "arn:aws:rds:$REGION:$ACCOUNT_ID:db:tfy-$SHORT_REGION-$NAME-*"
            ]
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": [
                "rds:AddTagsToResource",
                "rds:DeleteDBSubnetGroup",
                "rds:DescribeDBSubnetGroups",
                "iam:DeleteOpenIDConnectProvider",
                "iam:GetOpenIDConnectProvider",
                "rds:CreateDBSubnetGroup",
                "rds:ListTagsForResource",
                "rds:RemoveTagsFromResource",
                "iam:TagOpenIDConnectProvider",
                "iam:CreateOpenIDConnectProvider",
                "rds:CreateDBInstance",
                "rds:DeleteDBInstance"
            ],
            "Resource": [
                "arn:aws:rds:$REGION:$ACCOUNT_ID:subgrp:tfy-$SHORT_REGION-$NAME-*",
                "arn:aws:iam::$ACCOUNT_ID:oidc-provider/*"
            ]
        },
        {
            "Sid": "VisualEditor9",
            "Effect": "Allow",
            "Action": [
                "rds:DescribeDBInstances"
            ],
            "Resource": [
                "arn:aws:rds:$REGION:$ACCOUNT_ID:db:*"
            ]
        },
        {
            "Sid": "VisualEditor2",
            "Effect": "Allow",
            "Action": [
                "iam:CreatePolicy",
                "iam:GetPolicyVersion",
                "iam:GetPolicy",
                "iam:ListPolicyVersions",
                "iam:DeletePolicy",
                "iam:TagPolicy"
            ],
            "Resource": [
                "arn:aws:iam::$ACCOUNT_ID:policy/tfy-*",
                "arn:aws:iam::$ACCOUNT_ID:policy/truefoundry-*",
                "arn:aws:iam::$ACCOUNT_ID:policy/AmazonEKS_Karpenter_Controller_Policy*",
                "arn:aws:iam::$ACCOUNT_ID:policy/AmazonEKS_CNI_Policy*",
                "arn:aws:iam::$ACCOUNT_ID:policy/AmazonEKS_AWS_Load_Balancer_Controller*",
                "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryFullAccess"
            ]
        },
        {
            "Sid": "VisualEditor3",
            "Effect": "Allow",
            "Action": [
                "iam:ListPolicies",
                "elasticfilesystem:*",
                "iam:GetRole",
                "s3:ListAllMyBuckets",
                "kms:*",
                "ec2:*",
                "s3:ListBucket",
                "route53:AssociateVPCWithHostedZone",
                "sts:GetCallerIdentity",
                "eks:*"
            ],
            "Resource": "*"
        },
        {
            "Sid": "VisualEditor4",
            "Effect": "Allow",
            "Action": "dynamodb:*",
            "Resource": "arn:aws:dynamodb:$REGION:$ACCOUNT_ID:table/$NAME-$REGION-tfy-ocli-table"
        },
        {
            "Sid": "VisualEditor5",
            "Effect": "Allow",
            "Action": "iam:*",
            "Resource": [
                "arn:aws:iam::$ACCOUNT_ID:role/tfy-*",
                "arn:aws:iam::$ACCOUNT_ID:role/initial-*"
            ]
        },
        {
            "Sid": "VisualEditor6",
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::tfy-$SHORT_REGION-$NAME-*/*",
                "arn:aws:s3:::$NAME-$REGION-tfy-ocli-bucket/*",
                "arn:aws:s3:::tfy-$SHORT_REGION-$NAME*",
                "arn:aws:s3:::$NAME-$REGION-tfy-ocli-bucket",
                "arn:aws:s3:::tfy-$SHORT_REGION-$NAME-truefoundry*",
                "arn:aws:s3:::tfy-$SHORT_REGION-$NAME-truefoundry*/*"
            ]
        },
        {
            "Sid": "VisualEditor7",
            "Effect": "Allow",
            "Action": "events:*",
            "Resource": "arn:aws:events:$REGION:$ACCOUNT_ID:rule/tfy-$SHORT_REGION-$NAME*"
        },
        {
            "Sid": "VisualEditor8",
            "Effect": "Allow",
            "Action": "sqs:*",
            "Resource": "arn:aws:sqs:$REGION:$ACCOUNT_ID:tfy-$SHORT_REGION-$NAME-karpenter"
        }
    ]
}

Installation steps

  1. Create truefoundry namespace

    kubectl create ns truefoundry
    
  2. Create an imagePullSecret for the Truefoundry images in the truefoundry namespace

    cat <<EOF | kubectl apply -f -
    apiVersion: v1
    kind: Secret
    metadata:
      name: truefoundry-image-pull-secret
      namespace: truefoundry
    data:
      .dockerconfigjson: <image pull secret provided by truefoundry team>
    type: kubernetes.io/dockerconfigjson
    EOF
    
  3. The helm charts to use for the three cloud providers are as follows -

  4. Add the truefoundry helm repo

      helm repo add truefoundry https://truefoundry.github.io/infra-charts/
    
  5. Create a values.yaml and fill in the values. There are two different values.yaml for dev and production mode. The can be found over here -

    • Dev installation - link
    • Prod installation - link

    Fill in the values for the specific setup.

  6. Apply the helm chart with the values.yaml. Replace the chart_name with the correct one from step 3

    helm install inframold truefoundry/<chart_name> --version 0.0.8 -f values.yaml