Skip to main content
Setting up workflows in a Workload Cluster (already connected to truefoundry) requires the following configuration to be done:

​
Requirements:

  1. Cloud Storage Bucket (S3/GCS/AzureBlob)
  2. Bucket Access - IAM permissions to access the bucket
If you have created the compute plane cluster using TrueFoundry, a bucket and the required IAM roles are created and attached as an integration provider in the platform.Ideally the Blob Storage and the cluster should be in the same region, if created manually.

​
Step 1: Create an integration provider for the cloud storage bucket

  1. Follow this document to create an integration provider for your cloud provider. You need to create an IAM role with the right trust relationship for the workflow propeller to access the storage bucket
  2. In the “Clusters” page, edit your cluster from the three dots and update the “Workflow Storage Integration” to use your cloud storage bucket.

​
Step 2: Install Workflow Propeller in the cluster

To install the workflow propeller, you need to follow the steps below depending on the cloud provider.
  1. From the cluster page, click on the “Add-Ons” tab and scroll to the “Tfy Workflow Propeller” section.
  2. Click on “Install” and proceed with workspace creation.
  3. The values for the helm chart are given below depending on the cloud provider.
  4. There are few common values that are common to all the cloud providers. These can be found from the three dots against your cluster and then clicking on “Show Cluster Token”.
    • Tenant name - This is the tenant name of your TrueFoundry account.
    • Control Plane URL - This is the URL of your TrueFoundry control plane.
    • TFY agent token - This is the token of the cluster you connected.
  • AWS
  • GCP
  • Azure
For AWS we need the following values
  1. AWS S3 bucket name - This is the name of the S3 bucket used in the integration provider.
  2. AWS region - This is the region of the S3 bucket.
  3. AWS IAM role ARN - AWS IAM role ARN used in the integration provider.
Final values file should like this. Ensure to replace the placeholders with the actual values.
global:
  tenantName: <TENANT_NAME>
  controlPlaneUrl: <CONTROL_PLANE_URL_IN_HTTPS_FORMAT>
flyte-core:
  storage:
    type: s3
    bucketName: <AWS_S3_BUCKET_NAME>
    connection:
      region: <AWS_REGION>
      auth-type: iam
    enable-multicontainer: true
  configmap:
    core:
      propeller:
        metadata-prefix: s3://<AWS_S3_BUCKET_NAME>/tfy-workflow-propeller/metatdata
        rawoutput-prefix: s3://<AWS_S3_BUCKET_NAME>/tfy-workflow-propeller/raw_data
    admin:
      admin:
        Command:
          - echo
          - <TFY_AGENT_TOKEN>
        endpoint: <CONTROL_PLANE_URL>:443
  flyteadmin:
    serviceAccount:
      alwaysCreate: true
  flytepropeller:
    serviceAccount:
      annotations:
        eks.amazonaws.com/role-arn: <AWS_IAM_ROLE_ARN>
tfySignedURLServer:
  env:
    AWS_REGION: <AWS_REGION>
    S3_BUCKET_NAME: s3://<AWS_S3_BUCKET_NAME>
    DEFAULT_CLOUD_PROVIDER: aws
  enabled: true
⌘I