Cost Optimisation FAQs

You can also go through this product tour to understand how TF helps in cost optimization - https://www.truefoundry.com/product-tour?optimize-infra-costs.

Sagemaker and other cloud providers' internal platforms charge a mark up of 20-30% as compared to bare kubernetes clusters which we use.
Spot instances - TrueFoundry leverages AWS Spot Instances to run workloads at a lower cost. If a Spot Instance is interrupted, TrueFoundry can automatically switch to On-Demand instances, ensuring that the workload continues without manual intervention. This ensures a balance between cost savings and reliability.
Autoscaling: TrueFoundry dynamically adjusts the number of running instances based on the current workload. This means resources are only used when needed, preventing over-provisioning and reducing costs.
Reducing manual errors and enhancing productivity - TrueFoundry automates many aspects of the ML pipeline along with figuring out optimal infrastructure,reducing the chance of manual errors.
Fractional GPUs - TrueFoundry allows users to allocate fractional GPUs for their workloads. This is particularly useful for tasks that do not require the full capacity of a GPU, leading to better utilization and cost savings.
Additionally, the platform comes with a lot of built in hooks such as auto-shutdown of notebooks after certain time of inactivity, infra overprovisioning alerts etc which further enable you to optimize costs.