Access Cloud Services Like S3
We often have a requirement to access cloud managed services like blob storage, queues, databases etc from our services. A most common use case is access S3 or GCS bucket from our services to read or write data. To enable this in Truefoundry, the process is the same as you would do in a normal Kubernetes cluster. In this example below we explain the process to read and write data to blob storage - however, the concept remains roughly the same in case you are connecting to other cloud services like SQS. The key steps are:
1. Add the cloud specific code access the service in your code
2. Authenticate your service to access the cloud service
We need to provide the correct credentials to our code so that it can authenticate and connect to the cloud services. The exact approach depends on the cloud provider. Here’s how you can do it for the most common cloud providers:
There are two ways to authenticate your service to access AWS services:
1. Access Key and Secret Access Key
This involves setting the AWS_ACCESS_KEY_ID
and AWS_SECRET_ACCESS_KEY
environment variables. AWS SDKs will automatically pick these up from the environment variables and authenticate with the corresponding AWS service. The accesskey and secretaccesskey can be found in the AWS console and can be generated by your Infra team.
2. IAM Role-Based Access through Service Account (Recommended)
This approach typically involves creating IAM roles, associating them with Kubernetes service accounts, and configuring your deployments to use those service accounts. Here’s a detailed breakdown:
Key Concepts
- Kubernetes Service Accounts (SA): These are identities for processes running inside a pod. They provide a way to authenticate your pods with other Kubernetes services and external resources.
- IAM Roles: IAM roles are sets of permissions that define what actions an AWS entity (like a user, application, or service) can perform.
- IAM Roles for Service Accounts (IRSA): This is the key technology that allows you to map a Kubernetes service account to an IAM role. It uses AWS’s OpenID Connect (OIDC) provider capability.
Using IRSA (IAM Role for Service Accounts), you can securely grant Kubernetes deployments access to cloud services using service accounts and IAM roles, leveraging the power of IRSA for authentication and authorization. This is recommended and well understood by the Infrastructure / Devops teams. Please reach out to them to get the IAM role and service account created.
How to create an IAM role and service account?
How to create an IAM role and service account?
Get cluster and accountdetails
We will need the name of the cluster, AWS account ID, region, namespace (workspace in Truefoundry) in which the application is to be deployed. Set the following variables:
Get Cluster's OIDC Provider URL
The value of OIDC_ISSUER_URL will be the OIDC provider URL. It will be something like: oidc.eks.YOUR_REGION.amazonaws.com/id/YOUR_OIDC_ID
Create an IAM Policy
Create an IAM policy with the required permissions. This example grants full access to S3. It’s strongly recommended to scope down the permissions to only what’s necessary for security best practices.
Replace your-bucket-name with the actual name of your S3 bucket. You can also use wildcards to specify multiple buckets or prefixes within a bucket.
Create the policy using the AWS CLI:
Create an IAM Role
Create the assume role policy file.
Create an IAM role using this assume role policy:
Attach the IAM Policy to the Role
Create and Apply the Kubernetes Service Account
Create a Kubernetes service account in your desired namespace. You can apply this either via Kubectl or using the Truefoundry UI.
This is a simple YAML file (e.g., service-account.yaml
):
Apply using Kubectl:
or using the Truefoundry UI:
TODO: Add supademo link here
Verify the Service Account
Run a pod and test if you are able to perform operations on the AWS S3 bucket
Kubectl exec into the pod and test if you are able to perform operations on the AWS S3 bucket
Run the command aws s3 ls
to verify if you are able to access the S3 bucket.
There are two ways to authenticate your service to access AWS services:
1. Access Key and Secret Access Key
This involves setting the AWS_ACCESS_KEY_ID
and AWS_SECRET_ACCESS_KEY
environment variables. AWS SDKs will automatically pick these up from the environment variables and authenticate with the corresponding AWS service. The accesskey and secretaccesskey can be found in the AWS console and can be generated by your Infra team.
2. IAM Role-Based Access through Service Account (Recommended)
This approach typically involves creating IAM roles, associating them with Kubernetes service accounts, and configuring your deployments to use those service accounts. Here’s a detailed breakdown:
Key Concepts
- Kubernetes Service Accounts (SA): These are identities for processes running inside a pod. They provide a way to authenticate your pods with other Kubernetes services and external resources.
- IAM Roles: IAM roles are sets of permissions that define what actions an AWS entity (like a user, application, or service) can perform.
- IAM Roles for Service Accounts (IRSA): This is the key technology that allows you to map a Kubernetes service account to an IAM role. It uses AWS’s OpenID Connect (OIDC) provider capability.
Using IRSA (IAM Role for Service Accounts), you can securely grant Kubernetes deployments access to cloud services using service accounts and IAM roles, leveraging the power of IRSA for authentication and authorization. This is recommended and well understood by the Infrastructure / Devops teams. Please reach out to them to get the IAM role and service account created.
How to create an IAM role and service account?
How to create an IAM role and service account?
Get cluster and accountdetails
We will need the name of the cluster, AWS account ID, region, namespace (workspace in Truefoundry) in which the application is to be deployed. Set the following variables:
Get Cluster's OIDC Provider URL
The value of OIDC_ISSUER_URL will be the OIDC provider URL. It will be something like: oidc.eks.YOUR_REGION.amazonaws.com/id/YOUR_OIDC_ID
Create an IAM Policy
Create an IAM policy with the required permissions. This example grants full access to S3. It’s strongly recommended to scope down the permissions to only what’s necessary for security best practices.
Replace your-bucket-name with the actual name of your S3 bucket. You can also use wildcards to specify multiple buckets or prefixes within a bucket.
Create the policy using the AWS CLI:
Create an IAM Role
Create the assume role policy file.
Create an IAM role using this assume role policy:
Attach the IAM Policy to the Role
Create and Apply the Kubernetes Service Account
Create a Kubernetes service account in your desired namespace. You can apply this either via Kubectl or using the Truefoundry UI.
This is a simple YAML file (e.g., service-account.yaml
):
Apply using Kubectl:
or using the Truefoundry UI:
TODO: Add supademo link here
Verify the Service Account
Run a pod and test if you are able to perform operations on the AWS S3 bucket
Kubectl exec into the pod and test if you are able to perform operations on the AWS S3 bucket
Run the command aws s3 ls
to verify if you are able to access the S3 bucket.
Authenticate to GCP using IAM serviceaccount
Authenticate to GCP using IAM serviceaccount
In Google Kubernetes Engine (GKE), applications leverage Workload Identity to securely connect with Google Cloud Platform (GCP) services through IAM service accounts. This seamless integration enables fine-grained access control and eliminates the need for managing credentials within the application code, enhancing both security and operational efficiency in the GKE environment.
Step 1 (A) - Pre-requisites
-
Export important variables
-
Authenticate using
gcloud
-
Set your project ID
Step 1 (B) - Enabling workload identity for your cluster
Workload identity needs to be enabled for your GKE cluster so that pods can leverage this to authenticate to GCP services without credentials.
-
If workload identity is not enabled for your GKE cluster, run the below command to enable it
-
The above step will enable workload identity only in the new node pool. To enable the workload identity in the existing node pool
Step 2 - Create a kubernetes workspace
-
Export the namespace and the
serviceaccount
. TrueFoundry’s workspace is analogous to Kubernetes namespace. -
Go to Workspaces tab from the left panel of the portal and create the workspace with same name as of your namespace $APP_NS
- Click on
+ New Workspace
to create a new workspace. If you already have a workspace created click on the Edit section from the right side of the workspace card. - Select the cluster where you want to create the serviceaccount and enter the name of the workspace (namespace).
- Click on
Step 3 - Create IAM serviceaccount in GCP
In this section we will create an IAM serviceaccount
which has access to buckets. We will try to use this to access the bucket files in GCP
-
Export these variables and enter the name of the google serviceaccount you want to give in the variable
GSA_NAME
. We are assigning this serviceaccount Storage admin permission. You can assign the permissions that you want for accessing your GCP application. -
Create the IAM serviceaccount and assign the role using the below command. We are also assigning
roles/iam.workloadIdentityUser
role to the IAM serviceaccount on itself so that it can be accessed from inside GKE.
The policy contains bindings with conditions
When you are trying to run the command gcloud projects add-iam-policy-binding
you might get the below output
You can enter a condition if you want to restrict the GCP IAM serviceaccount to a certain bucket or you can use option 2 and continue.
Step 4 - Create Kubernetes Serviceaccount in your workspace
-
Go to Workspaces tab from the left panel of the portal and click on the pencil icon to edit your workspace.
-
Click on
Show Advanced fields
on bottom of the screen and enableService accounts
field. -
Click on
+ Add Service Accounts
to add a Serviceaccount-
Enter the name of your Kubernetes Serviceaccount which is in the variable $APP_SA
-
Enter the IAM Serviceaccount name which will be
$GSA_NAME@$PROJECT_ID.iam.gserviceaccount.com
-
-
Click on update to continue\
Test the serviceaccount
-
Create the below pod
-
Check if you are able to list buckets without passing creds
Coming Soon
3. Select the service account for the service
Once you’ve configured the Service Account in Kubernetes following the steps above, you can select the service account for the service in the Truefoundry UI. This can be viewed after switching on the advanced options in the service deployment form.