AWS
Provisioning Control Plane Infrastructure on AWS
Dev Mode
Infrastructure Requirements
Requirements | Description | Reason for Requirement |
---|---|---|
Kubernetes Cluster | Any Kubernetes cluster will work here - we can also choose the compute-plane cluster itself to install Truefoundry helm chart. | The Truefoundry helm chart will be installed here. Need to add the size of cluster, ebs is needed for NATS |
Egress Access for TruefoundryAuth | Egress access to https://auth.truefoundry.com | This is needed to verify the users logging into the Truefoundry platform for licensing purposes |
Egress access For Docker Registry | 1. public.ecr.aws 2. quay.io 3. ghcr.io 4. docker.io/truefoundrycloud 5. docker.io/natsio 6. nvcr.io 7. registry.k8s.io | This is to download docker images for Truefoundry, ArgoCD, NATS, ArgoRollouts, ArgoWorkflows, Istio |
Installation steps
- Create
truefoundry
namespacekubectl create ns truefoundry
- Create an imagePullSecret for the Truefoundry images in the
truefoundry
namespacecat <<EOF | kubectl apply -f - apiVersion: v1 kind: Secret metadata: name: truefoundry-image-pull-secret namespace: truefoundry data: .dockerconfigjson: <image pull secret provided by truefoundry team> type: kubernetes.io/dockerconfigjson EOF
- Install argoCD -
kubectl create namespace argocd kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/core-install.yaml
- Add the truefoundry helm repo
helm repo add truefoundry https://truefoundry.github.io/infra-charts/ helm repo update
- We will create a
values.yaml
for the helm chart installation -- Download the values.yaml from helm chart repo -
curl https://raw.githubusercontent.com/truefoundry/infra-charts/main/charts/tfy-k8s-aws-eks-inframold/values.yaml > values.yaml
- Fill in the
tenantName
,clusterName
andsfyApiKey
in the downloaded file
- Download the values.yaml from helm chart repo -
- Apply the helm chart with the values.yaml. Replace the
chart_name
with the correct one from step 3helm install -n argocd inframold truefoundry/tfy-k8s-aws-eks-inframold --version 0.0.9 -f values.yaml
Test the installation
- Port forward the frontend application to access the Truefoundry dashboard -
kubectl port-forward svc/truefoundry-truefoundry-frontend-app -n truefoundry 5000
- Access the truefoundry dashboard from a browser by opening
http://localhost:5000
. You can login with the username and password provided by the Truefoundry team. - Now you are ready to connect a cluster to the Truefoundry platform and get deploying. Go here for the directions. You can also onboard the same cluster as the control plane
Prod mode
Infrastructure Requirements
Requirements | Description | Reason for Requirement |
---|---|---|
Kubernetes Cluster | Any Kubernetes cluster will work here - we can also choose the compute-plane cluster itself to install Truefoundry helm chart. | The Truefoundry helm chart will be installed here. |
Postgres RDS | Postgres >= 13 | The database is used by Truefoundry control plane to store all its metadata. |
S3 bucket | Any S3 bucket reachable from control-plane. | This is used by control-plane to store the intermediate code while building the docker image. |
Egress Access for TruefoundryAuth | Egress access to https://auth.truefoundry.com | This is needed to verify the users logging into the Truefoundry platform for licensing purposes |
Egress access For Docker Registry | 1. public.ecr.aws 2. quay.io 3. ghcr.io 4. docker.io/truefoundrycloud 5. docker.io/natsio 6. nvcr.io 7. registry.k8s.io | This is to download docker images for Truefoundry, ArgoCD, NATS, ArgoRollouts, ArgoWorkflows, Istio. |
DNS with TLS/SSL | One endpoint to point to the control plane service (something like platform.example.com where example.com is your domain. There should also be a certificate with the domain so that the domains can be accessed over TLS. The control-plane url should be reachable from the compute-plane so that compute-plane cluster can connect to the control-plane | The developers will need to access the Truefoundry UI at domain that is provided here. |
User/ServiceAccount to provision the infrastructure | This is the set of permissions needed to provision the infrastructure for Truefoundry control-plane. |
Permissions required to create the infrastructure
export REGION="" # us-east-1
export SHORT_REGION="" #usea1
export ACCOUNT_ID="" #123524493244
export NAME=""
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"iam:CreateInstanceProfile",
"iam:DeleteInstanceProfile",
"rds:AddTagsToResource",
"iam:GetInstanceProfile",
"iam:RemoveRoleFromInstanceProfile",
"rds:DeleteTenantDatabase",
"iam:AddRoleToInstanceProfile",
"rds:CreateDBInstance",
"rds:DescribeDBInstances",
"rds:RemoveTagsFromResource",
"rds:CreateTenantDatabase",
"iam:TagInstanceProfile",
"rds:DeleteDBInstance"
],
"Resource": [
"arn:aws:iam::$ACCOUNT_ID:instance-profile/*",
"arn:aws:rds:$REGION:$ACCOUNT_ID:db:tfy-$SHORT_REGION-$NAME-*"
]
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": [
"rds:AddTagsToResource",
"rds:DeleteDBSubnetGroup",
"rds:DescribeDBSubnetGroups",
"iam:DeleteOpenIDConnectProvider",
"iam:GetOpenIDConnectProvider",
"rds:CreateDBSubnetGroup",
"rds:ListTagsForResource",
"rds:RemoveTagsFromResource",
"iam:TagOpenIDConnectProvider",
"iam:CreateOpenIDConnectProvider",
"rds:CreateDBInstance",
"rds:DeleteDBInstance"
],
"Resource": [
"arn:aws:rds:$REGION:$ACCOUNT_ID:subgrp:tfy-$SHORT_REGION-$NAME-*",
"arn:aws:iam::$ACCOUNT_ID:oidc-provider/*"
]
},
{
"Sid": "VisualEditor9",
"Effect": "Allow",
"Action": [
"rds:DescribeDBInstances"
],
"Resource": [
"arn:aws:rds:$REGION:$ACCOUNT_ID:db:*"
]
},
{
"Sid": "VisualEditor2",
"Effect": "Allow",
"Action": [
"iam:CreatePolicy",
"iam:GetPolicyVersion",
"iam:GetPolicy",
"iam:ListPolicyVersions",
"iam:DeletePolicy",
"iam:TagPolicy"
],
"Resource": [
"arn:aws:iam::$ACCOUNT_ID:policy/tfy-*",
"arn:aws:iam::$ACCOUNT_ID:policy/truefoundry-*",
"arn:aws:iam::$ACCOUNT_ID:policy/AmazonEKS_Karpenter_Controller_Policy*",
"arn:aws:iam::$ACCOUNT_ID:policy/AmazonEKS_CNI_Policy*",
"arn:aws:iam::$ACCOUNT_ID:policy/AmazonEKS_AWS_Load_Balancer_Controller*",
"arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryFullAccess"
]
},
{
"Sid": "VisualEditor3",
"Effect": "Allow",
"Action": [
"iam:ListPolicies",
"elasticfilesystem:*",
"iam:GetRole",
"s3:ListAllMyBuckets",
"kms:*",
"ec2:*",
"s3:ListBucket",
"route53:AssociateVPCWithHostedZone",
"sts:GetCallerIdentity",
"eks:*"
],
"Resource": "*"
},
{
"Sid": "VisualEditor4",
"Effect": "Allow",
"Action": "dynamodb:*",
"Resource": "arn:aws:dynamodb:$REGION:$ACCOUNT_ID:table/$NAME-$REGION-tfy-ocli-table"
},
{
"Sid": "VisualEditor5",
"Effect": "Allow",
"Action": "iam:*",
"Resource": [
"arn:aws:iam::$ACCOUNT_ID:role/tfy-*",
"arn:aws:iam::$ACCOUNT_ID:role/initial-*"
]
},
{
"Sid": "VisualEditor6",
"Effect": "Allow",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::tfy-$SHORT_REGION-$NAME-*/*",
"arn:aws:s3:::$NAME-$REGION-tfy-ocli-bucket/*",
"arn:aws:s3:::tfy-$SHORT_REGION-$NAME*",
"arn:aws:s3:::$NAME-$REGION-tfy-ocli-bucket",
"arn:aws:s3:::tfy-$SHORT_REGION-$NAME-truefoundry*",
"arn:aws:s3:::tfy-$SHORT_REGION-$NAME-truefoundry*/*"
]
},
{
"Sid": "VisualEditor7",
"Effect": "Allow",
"Action": "events:*",
"Resource": "arn:aws:events:$REGION:$ACCOUNT_ID:rule/tfy-$SHORT_REGION-$NAME*"
},
{
"Sid": "VisualEditor8",
"Effect": "Allow",
"Action": "sqs:*",
"Resource": "arn:aws:sqs:$REGION:$ACCOUNT_ID:tfy-$SHORT_REGION-$NAME-karpenter"
}
]
}
Installation steps
-
Create
truefoundry
namespacekubectl create ns truefoundry
-
Create an imagePullSecret for the Truefoundry images in the
truefoundry
namespacecat <<EOF | kubectl apply -f - apiVersion: v1 kind: Secret metadata: name: truefoundry-image-pull-secret namespace: truefoundry data: .dockerconfigjson: <image pull secret provided by truefoundry team> type: kubernetes.io/dockerconfigjson EOF
-
The helm charts to use for the three cloud providers are as follows -
-
Add the truefoundry helm repo
helm repo add truefoundry https://truefoundry.github.io/infra-charts/
-
Create a values.yaml and fill in the values. There are two different
values.yaml
for dev and production mode. The can be found over here -Fill in the values for the specific setup.
-
Apply the helm chart with the values.yaml. Replace the
chart_name
with the correct one from step 3helm install inframold truefoundry/<chart_name> --version 0.0.8 -f values.yaml
Updated about 2 hours ago