Kustomize Support
Patch or Add resources using Kustomize
TrueFoundry allows you to tweak the most common parameters of the deployment through the service spec. However, there might be situations in which you might want to override some fields that are not exposed in the TrueFoundry Service spec. You can then use Kustomize to add, patch or delete the Kubernetes resources that TrueFoundry deploys on the cluster.
Kustomize enables you to
- Patch the rendered Kubernetes resources generated by the TrueFoundry Application. E.g. Adding extra annotations for Prometheus / Datadog
- Add extra Kubernetes resources along with your TrueFoundry Application. E.g. Adding extra ConfigMap, Secret, Istio VirtualService, etc
Truefoundry doesn’t allow you to use Kustomize to create cluster level resources like ClusterRole
, ClusterRoleBinding
or non-namespace scoped resources like EnvoyFilter
and WasmPlugin
. You can create these resources only if you are cluster admin for the cluster, else the deployment will fail. This prevents the scenario where a user who has access to a certain workspace can create cluster level resources and impact other workloads in the cluster.
Using Kustomize for your application
You can add patches and resources using the kustomize
field in the service deployment form.
There are two sections:
- Patch: We define an array of patches to be applied to the rendered Kubernetes resources generated by the TrueFoundry Application. For e.g. this Kustomize patch adds Prometheus scraping annotations to the pod template metadata of a Deployment named “my-service”. Specifically, it configures Prometheus to scrape metrics from port 8000 by setting prometheus.io/port to “8000” and enabling scraping with prometheus.io/scrape set to “true”.
- Additional Manifest: This section allows you to add new Kubernetes resources to the deployment. The example below adds a new ConfigMap to the deployment.
This is how it looks after filling up the Kustomize section:
Once you deploy the application, you can view the generated Kubernetes resources in the Application Spec
Tab and then selecting Applied K8s Manifest
This should reflect the kustomized resources after your Kustomize patches and additions are applied.
Commonly Used Kustomize Patches
Here are some commonly used Kustomize patches that you can use to customize your application:
Enable Prometheus to scrape metrics from the service
Enable Prometheus to scrape metrics from the service
Enter this in the Patch
section:
Enable Autoscaling on memory usage
Enable Autoscaling on memory usage
Enter the Keda ScaledObject spec in the Additional Manifest
section:
To add any custom autoscaling like based on queue length, you can checkout Keda documentation to get the ScaledObject spec.
Add an additional container to the deployment
Add an additional container to the deployment
Enter this in the Patch
section:
Patch Security Context
Patch Security Context
Enter this in the Patch
section: