Changelog
Hi, and welcome! Let’s talk about what’s new with TrueFoundry in this release.
New Feature: Export LLM Playground Metrics to CSV
Good news! You can now export your LLM Playground metrics as a handy CSV file. It’s perfect for bringing into your favorite analytics tools and diving deeper into your data. Happy analyzing!
More updates:
- Fixed bug: The temperature parameter now supports decimal values in prompt schema
Hi, and welcome! Let’s talk about what’s new with TrueFoundry in this release.
New Deployment Deletion Status
We’ve introduced a new status indicator to show when a deployment is in the process of being deleted. Once the deletion is complete, you’ll receive a notification confirming its success. If the deletion fails, the status will update to deletion_failed, along with a message explaining the reason for the failure.
More updates:
- Support for non-OpenAI models from Azure on per API basis.
- Removing mandate to pass a “valid” model name is request. You can now do full header based routing. [ just need to comply with openai standards]
- Support for Responses API [Beta]. You can now use codium in your terminal via Truefoundry LLM Gateway
Hi, and welcome! Let’s talk about what’s new with TrueFoundry in this release.
Gateway Metrics Just Got a Big Upgrade! 🎉
We’ve given Gateway metrics a full makeover — and it’s better than ever! In addition to the metrics you already know and love, you can now explore brand new insights like Request Latency, Requests per Second (RPS), Time to First Token, and Inter-Token Latency. More visibility = more power!
But wait, there’s more! You now have dedicated views for User Activity and Model Usage, making it super easy to track input/output tokens, total requests, and total cost per user and model.
And yes — we’ve got your config metrics covered too! TrueFoundry Gateway continues to support settings like rate limits, fallbacks, and load balancing. Now, you can actually see which configs were applied to requests over time, complete with clear visuals and data.
In short: more clarity, better control, and a smoother experience! 🚀
Hi, and welcome! Let’s talk about what’s new with TrueFoundry in this release.
Finetuning Just Got Smarter – Say Hello to Dora!
We’re excited to share that finetuning now supports Dora by default! Dora is a big step up from LoRA — offering improved performance that gets you much closer to full finetuning, without the extra complexity.
It’s faster, smarter, and more efficient — what’s not to love?
👉 Learn more about Dora and how to get started here.
LLM Deployment Just Got a Boost with TRT-LLM + NVIDIA Triton!
Great news — you can now deploy LLM models using the TRT-LLM backend with the NVIDIA Triton Inference Server!
We’ve got you covered with prebuilt engines for Llama models, so you can get started right away. Have another model in mind? No problem — you can also request custom engines for any model of your choice!
Faster inference, smoother deployment — let’s go! 💥
Improved Retry Information in Workflows
We’ve given the info panel a fresh new look—for both map and non-map tasks! Now, whenever a task gets retried, you can simply expand the row to dive into all the details. See the retry history, track the duration(for non-map tasks), and easily visualize logs, events, and metrics for each retry attempt.
It’s all right there, just a click away—cleaner, clearer, and way more insightful!
Hi, and welcome! Let’s talk about what’s new with TrueFoundry in this release.
Auto-pilot insights on cost savings and reliability fixes
Exciting update! 🎉 We’ve added two new metrics to help you easily spot the amazing impact of autopilot! Head over to the audit logs page, switch to the autopilot filter, and you’ll be able to see the total cost saved and the number of reliability issues fixed by autopilot in your chosen time range. How cool is that?
Want to enable autopilot too? It’s super simple! Just go to your cluster details view to enable autopilot for all deployments within the cluster, or visit the specific deployment page to turn it on for individual deployments. Enjoy the benefits! 😊🚀
And here’s a fun tidbit—on our internal dev setup, autopilot saved us an amazing $2590 and fixed 470 reliability issues!
Other Features, Fixes & Improvements:
- Search functionality added to all integration select dropdowns
- Moved readme tab to the beginning for SSH server details page for improved discoverability
- Minor Bug fixes and UI/UX improvements
Hi, and welcome! Let’s talk about what’s new with TrueFoundry in this release.
Setup Custom OAuth for Service Deployment
Great news! 🎉 TrueFoundry now makes it super easy to integrate custom JWT authentication, giving you the ability to set up JWT-based authentication using JWKS. Once you’ve added your custom integration, you’ll be able to use it right in your deployment settings for endpoint login and authentication. Plus, it supports a variety of providers like Google, Okta, AzureAD, Cognito, and many more — including TrueFoundry! 😊
Filter deployments by Auto-pilot Recommendations
This was a highly requested functionality. Now you will be able to filter all deployments based on if they have any recommendations by auto-pilot.
Support for fractional GPU in On-Prem clusters
This for our enterprise clients. If you setup on-prem clusters then now you can enjoy the benefits of fractional GPUs!
Other Features, Fixes & Improvements:
- Added new menu items to simplify creation and updation of model versions
- Minor Bug fixes and UI/UX improvements
Hi, and welcome! Let’s talk about what’s new with TrueFoundry in this release.
Audit Logs
As an admin, you can now monitor all user and virtual account activity across the platform. You’ll be able to view the specific actions performed, such as “Create Deployment” or “Start Notebook,” along with detailed metadata, including the associated resource and the user who triggered each action.
Other Features, Fixes & Improvements:
-
Workflow node enhancement for map nodes – You can now view the status of underlying map tasks directly within the map node, without needing to open the detailed view.
-
On LLM Gateway v0.27.0, the rate limit config now supports virtualaccount instead of serviceaccount under subjects. To reduce the risk during the upgrade:
- Before upgrade, We can add new subects with virtualaccountwhere ever serivceaccount.
- Complete the upgrade
- After the upgrade, remove sericeaccount subjects.
-
Minor Bug fixes and UI/UX improvements
Hi, and welcome! Let’s talk about what’s new with TrueFoundry in this release.
Automated Upgrades for Infrastructure Add-ons.
Staying up to date just got a whole lot easier. We’re rolling out automated upgrades for infrastructure add-on components like tfy-agent, argo, istio etc, ensuring your stack is always running the latest compatible versions —without the manual hassle. Whenever a new stable release drops, we’ll handle the upgrade seamlessly, so you can focus on shipping instead of maintaining.
Of course, we’re not just throwing updates over the fence. Our system checks for compatibility before rolling anything out, minimising disruptions and keeping things smooth. You get the latest features and security patches without lifting a finger. Set it, forget it, and let your infra take care of itself.
To enable this for your cluster simply visit the cluster drawer and enable “Auto-pilot” for add-ons.
Other Features, Fixes & Improvements:
- Enhancements to autopilot: Added recommendations for CPU throttling, capacity type selection based on environment, and a volume size fix for Prometheus/Loki.
- LLM Gateway: Introduced support for real-time models.
- Minor Bug fixes and UI/UX improvements
New Features & Improvements:
- Added additional metadata to audit logs.
- Implemented a context window for logs with a search feature.
- Configured fallback mechanisms and load balancing for the LLM gateway.
- Implemented “tfy apply” for teams.
- Implemented “tfy apply” for rate limit configuration on the LLM gateway.
- Enabled manifest-driven creation of data directories.
Bug fixes:
- Minor Bug fixes and UI/UX improvements
New Features & Improvements:
- Added support for running Spark jobs.
- Added support for deploying RStudio.
- Improved the application listing page by adding filters for device type and last deployed by.
- Implemented Autopilot for clusters to optimize performance and resource management.
- Added a guided tour to enhance the onboarding experience for new users.
- Enabled pause and resume functionality in the application set for better control.
Bug fixes:
- Minor Bug fixes and UI/UX improvements
New Features & Improvements:
- Improvements in node selector experience for Azure and Generic clusters
- Easy deployment for sklearn and xgboost models
- Users can now log sklearn and xgboost models with added input/output schema (or automatically inferred) to TrueFoundry Model Registry and enable low-code / no-code inference deployments
- Added Cluster Insights
- Cluster auto sync for syncing nodepools and other monitoring settings
- Addon auto-upgrade one-click flow
- Compare diff for multiple deployment versions
- New cluster onboarding flow revamped to a single command
- Autopilot button to allow TrueFoundry to redeploy applications after calculating CPU / Memory Recommendations
Deprecated
- The username field is now deprecated from the system
Bug fixes:
- Minor Bug fixes and UI/UX improvements
New Features & Improvements:
- Adds support for deploying vision, image, and speech-to-text models
- Enables recommendations on cluster infra addons for enhanced reliability
- Adds
Upgrade availablealerts for cluster infra addons - with diff to be applied - Enhances
Nodelevel details and metrics including CPU, memory and GPU usage - Improves
Clusterlevel metrics - Experience
Node selectorexperience now on Azure and Generic clusters too. - Redesigned pod scheduling experience in UI
- Adds support for custom s3-compatible blob storage in integrations
- Supports model download in GPU nodes for faster bootstrap time
- Adds dry-run option for cluster, workspace, deployment, integrations, and virtual-account
- Introduced manifest-driven creation for Artifacts and Models in ML repo
Changes
- Depreciated integration id in git URl for build source
- Combined logs, metrics, and events buttons into a single monitoring button
New Features & Improvements:
- Added Insights for deployed service to help tune your resources, rollout strategy, probes, etc
- Increased events data retention to 7 days and improved graphs to visualize
- Enhances ML Repo tables and metrics page
- Enhanced adding deployed service to LLM Gateway experience
- Added support for workflows via UI
- Added more metrics to track in LLM Gateway
- Added support for docker registry, git integration, and secret store in virtual account
- Support OAuth type authentication for deployed service
- Added copy button in Environment tab to copy value under application details
- Automatically generate Python code snippet and curl request for triggering Job
- Support deployment of private helm charts
Bug fixes:
- Minor UI bug fixes
Truefoundry Workflows, ML Repo GitOps
New Features & Improvements:
-
Truefoundry Workflows:
- Enables users to run sequence of tasks (DAGs) to manage and scale their complex pipelines. This is powered by Flyte (an open-source project) in the backend.
-
ML Repo
- Gitops enabled for ML Repo
- Now you can add or update Collaborators from the form itself
-
LLM Gateway Improvements
Bug fixes:
- Minor UI bug fixes
Auto-shutdown, Rate-limiting in LLM Gateway, Enterprise SSO
New Features & Improvements:
- Auto-shutdown/scale to zero in services
- Virtual account extended support for llm-models access
- Rate-limiting in llm-gateway
- Support for enterprise SSO
- Support for build-script in the workbench
- Show volume usage metrics for the home directory in the workbench
Bug fixes:
- Minor UI bug fixes
Application Set Components, Autopilot recommendation, UI Improvements and Bug Fixes
New Features & Improvements:
-
‘Add another component’ feature in the custom application set deployment edit form. Now users can edit their custom application set to add components such as services, jobs, and helms using the edit form
-
Added autopilot recommendation button on the services listing page
-
Integrations improvements -
- Clicking on authorized users from the integration listing page will open the specific integration in the edit form, other integrations will stay collapsed so that it will be easier to edit integration based on selection.
- Access Control user search input rendering performance is improved
Bug fixes:
- LLM Gateway UI bug fixes
- Cluster UI bug fixes and improvements
Audit Logs, Cluster UI Revamp, LLM Gateway Improvements
New Features & Improvements:
-
Added Audit Logs feature; can be accessed on the settings page
-
Cluster Listing and details page UI is now more user-friendly
-
LLM Gateway Improvements
- Fixed Metrics graph size and animations
- Fixed the layout color of metrics
Bug fixes:
- Minor UI bug fixes
Multiple OpenAI Integrations, Liveness and Readiness Prob UI Revamp, Event charts, and Bug fixes
New Features & Improvements:
- Added support for multiple OpenAI integration types
- The UI for Liveness and Readiness prob form has become more user-friendly
- Added event charts for all application types with the events dashboard
Bug fixes:
- Bug fixes in docker registry integrations
- Minor UI bug fixes
Statically Provisioned Volumes, CI/CD, ENV based Optimization
New Features & Improvements:
- Added support to deploy Statically Provisioned Volumes
- Added new methods to enable CI/CD for deployments
- Improved ENVs to Optimize the deployments based on the use case
Bug fixes:
- Bug fixes and UX improvements in the deployment form
- Other minor UI bug fixes and improvements
Fractional GPUs, AMQP autoscaling, Search & Pagination for the app listing
New Features & Improvements:
- Added search & pagination for deployments listing
- Added support for fractional GPUs
- Added support for tfy-secrets in build-args
- Added Python Spec viewer for deployments
- Enabled AMQP-based autoscaling in async services
- Enabled Pause/Resume feature for services with Autoscaling
- Enabled Bulk addition of ENVs in the deployment form and secrets in the Secret Group
- Updated RPS-based autoscaling to include Inflight requests
Bug fixes:
- Minor UI bug fixes and improvements
Cloud Account Integration, HuggingFace Models, UI improvements, and bug fixes
New Features & Improvements:
- Added support for Cloud account integration
- Added support for more models from HuggingFace
- Improved the UI for App Catalogue and Model Catalogue Deployment
Bug fixes:
- Minor UI bug fixes
- Fixed artifact file preview in case of multiple versions and for long-width CSVs
Model Catalogue, Deploy using Huggingface URL, Support for GPU devices and AWS inferential
New Features & Improvements:
- Added support for GPU devices and AWS inferential.
- Added support for an OpenAPI Tab across all services.
- Added support to copy both Username and Password from the Notebook deployment page.
- Improved usability for artifacts and endpoints in the Services Listing.
- Improved the Resources section within the deployment form.
- Improved the logs and metrics for better usability
- Removed the Model Deployment tab for a cleaner interface; models can now be seamlessly deployed directly from the Model Catalogue.
- Enhanced Model Catalogue functionality with the ability to deploy models using HuggingFace URLs.
Bug fixes:
- Minor UI bug fixes
Enhanced Secrets, Deployments, Improved UX, and Bug Fixes
New Features & Improvements:
- Added the ability to search secrets and introduced pagination for better navigation.
- Implemented a confirmation layer to prevent accidental deployment deletions.
- Implemented new listing filters on the workspace page for a more user-friendly experience.
- Users can now export the metrics dashboard directly to Grafana for enhanced analysis.
Bug fixes:
- Refined the deployment flow and monitoring with minor bug fixes and UX enhancements.
Mirror Requests using Intercepts, Chat Model in LLM Gateway, UI Improvements and Bug fixes
New Features & Improvements:
- Added support for chat type models in LLM Gateway
- Added support for mirroring request using intercepts
- Added support for artifact cache and download in service form
- Removed delete user functionality
- Removed last used column from notebooks
Bug fixes:
- Fixed images and plots loading issue in ML Repos
- Minor UI bug fixes
Data Directories in ML Repos, UI Improvements and Bug fixes
New Features & Improvements:
- Added support for data directories in UI
- Added a feature to pause and resume async services
- Added support to deploy examples for services
Bug fixes:
- Fixed a bug where the app listing filter gets reset on opening logs
- Minor UI bugs fixes
Pause Service Feature, UI Improvements and Bug fixes
New Features & Improvements:
- Added a feature to pause and resume a service
- Added model metadata and benchmarking information in model catalogue
- Added a refresh button in deployment form to refresh env variables
- Added a message on edit form if active and latest version differ to tell users they are editing an older version.
- Added a delete option in models and artifacts in ML Repo tabs
- Added a button to open secrets in new tab in env var section of deployment form
- Added a validation to make value in secret creation mandatory
- Added a search in filter dropdown for workspace in deployment page
- Added a warning on secret deletion to show that the secret is being used in a deployment
- Removed “Show cluster token” menu item from cluster card expanded menu
Bug fixes:
- Fixed validation message in resource section for request/limit
- Fixed bug where changing name from dropdown doesn’t change the image in notebook deployment
- Fixed condition if no nodepools in cluster and nodepool checkbox is enabled then deployment form submission is not allowed
- Fixed zipped downloaded artifacts are corrupted issue
- Fixed 500 error on cluster onboarding step-3
- Fixed redirection issue where cancel deployment button is redirecting to the deployment details page.
Pause Service Feature, UI Improvements and Bug fixes
New Features & Improvements:
- Added a feature to pause and resume a service
- Added model metadata and benchmarking information in model catalogue
- Added a refresh button in deployment form to refresh env variables
- Added a message on edit form if active and latest version differ to tell users they are editing an older version.
- Added a delete option in models and artifacts in ML Repo tabs
- Added a button to open secrets in new tab in env var section of deployment form
- Added a validation to make value in secret creation mandatory
- Added a search in filter dropdown for workspace in deployment page
- Added a warning on secret deletion to show that the secret is being used in a deployment
- Removed “Show cluster token” menu item from cluster card expanded menu
Bug fixes:
- Fixed validation message in resource section for request/limit
- Fixed bug where changing name from dropdown doesn’t change the image in notebook deployment
- Fixed condition if no nodepools in cluster and nodepool checkbox is enabled then deployment form submission is not allowed
- Fixed zipped downloaded artifacts are corrupted issue
- Fixed 500 error on cluster onboarding step-3
- Fixed redirection issue where cancel deployment button is redirecting to the deployment details page.
Deploy using YAML, Application deployment like Qdrant & Postgre, UI Improvements and Bug fixes
New Features & Improvements:
- Added an option in the UI to allow users to deploy by pasting YAML
- Added UI support to deploy specific applications like Qdrant, Postgres etc
- Added a file uploader to allow users to upload artifacts in ML Repos
- Added support to upload/download artifact from CLI
- Added support for multipart upload in ml-foundry artifacts
- Added parallel upload of artifact/model contents in ml-foundry
- Changed notebook resources for sudo access ephemeral storage
Bug fixes:
- Fixed bugs in bootstrap script for workload cluster
- Fixed bug where command is not copied when converting from local build to image build
Patch & Additions Support, Setup Account Flow, UI Improvements and Bug fixes
New Features & Improvements:
- Added “patch” and “additions” fields in service deployment form
- Added support for parallel upload of artifact/model contents in ML repos
- Added filter by cluster in deployments listing page
- Added setup account screen for new users
- Added python code to load the model from model catalogue
- Changes generic cluster type icon to Kubernetes
- Changed catalog model deployments type from service to model
Bug fixes:
- Fixed the issue where training loss is not coming up on ML Repos when finetune script is running
- Fixed data overflow issue in secrets
Nodepools, apt-packages for Notebooks, UI Improvements and Bug fixes
New Features & Improvements:
- Added apt build options for Notebook
- Added nodepools in deployment forms - Now you can configure and use your pre-existing nodepools with applications
- Added a message to prompt the user to fill up values with <to_be_provided> in the helm deployment
- Added a button highlight on events button if the latest event is an error.
- Added validation on image type in the deployment form
- Added task-based filtering in the model catalogue
- Added support to allow replicas to go to 0 for a service deployment
- Changed intercepts position to the advanced section in the service deployment form
Bug fixes:
- Fixed the issue where GPU metrics legend repeats when changing time windows
- Fixed crashing issue post installation of application on clusters
Generic Cluster Support, GPUs on Azure, UI Improvements and Bug fixes
New Features & Improvements:
- Added support for Generic Cluster integration.
- Added support for GPUs on Azure.
- Added support to resize volumes.
- Added support for HuggingFace image and audio pipelines in model servers.
- Added a badge in deployment listing table to show if the deployment is using GPUs.
Bug fixes:
- Fixed GPU metrics on GCP Standard clusters.
- Fixed an issue where users were unable to set the cluster as default.
- Fixed an issue where some other chart version automatically gets selected while installing application.
- Fixed icons for volumes and notebooks in deployment listing page
- Fixed issue in cluster application install where it would ask to create workspace even when a workspace is selected.
Nodepools for Azure, Basic Auth in Notebook Deployment, UI Improvements and Bug fixes
New Features & Improvements:
- Added Nodepools for Azure clusters.
- Added basic auth in notebook deployment.
- Added workspace info in the volume usage dropdown on the volume deployment list page.
- Added the default Python version in the notebook image description in the notebook deployment form.
- Added workspace name in the subdomain of volume endpoint in volume deployments form.
- Added a way to add collaborators in GIT integrations if no collaborators exist.
- Changed volume size limit from 200Gi to 2000Gi.
- Change “Stop After” documentation in the notebook deployment form.
Bug fixes:
- Fixed service deletion error where deletion fails even if “allow_intercepts” is enabled and no intercepts are added.
- Fixed expose check issue where turning off expose in port on an already exposed service fails validation on a host when using a non-wildcard domain.
- Fixed collaborator issue for teams where a user is able to add some teams in collaborator but not others due to permission issues.
- Fixed workspace filter issues where clicking on all would show an infinite loader.
- Fixed the 500 issue on running jobs with array type params
- Fixed pre-filling of dependent field in helm deployment.
Azure Blob Storage Support, UI Improvements and Bug fixes
New Features & Improvements:
- Added support for Azure blob storage in mlfoundry.
- Added a container dropdown in pod logs to view logs for a specific container.
- Added a container column and a checkbox to toggle its visibility in logs.
- Added spot preference to spec and manifest.
- Changed connect cluster view to show tenant name, cluster token, and control plane URL.
- Changed workspace actions to have the delete option inside the dropdown menu.
- Removed the “Create Workspace” popup which would show up on the deployment listing page if there were no deployments.
- Removed “File Mounts” from deployment forms.
Bug fixes:
- Fixed cluster applications status issue on new application installation. Now on installing an application, it will update the status to installed.
- Fixed the endpoint issue on volume redeployment where it would disappear until the deployment was complete.
- Fixed notebook volume size issue. It was showing 2000 MB for all notebooks.
- Fixed volume unit issue on the deployment details page to show Gi instead of GB.
Notebooks, Volume Browser, UI Improvements and Bug fixes
New Features & Improvements:
- Added notebook deployment from UI
- Added volume browser for TFY Volumes.
- Added metrics for notebook deployments.
- Added full-screen button in string mount text field in the deployment form.
- Added line numbers in the string mount text field in deployment form.
- Added disc usage and capacity in cluster nodes table.
- Added allocatable capacity info and requests for resources in the nodes table.
- Added demo deployments for model and helm.
- Added support to upload/download files between local machine and TFY Volumes.
- Added a dropdown in deployment step-1 to select the deployment type.
- Added deployment count and list in the volume table to show where a volume is being used.
- Changed “Volume Storage” position and moved it under the resource section.
- Changed volume dropdown behavior in volume mount in deployment form. Now it will show the volume list when the field is focused.
- Changed the ML Repo icon in the side drawer.
- Removed “Ephemeral Storage” from notebook deployment form.
Bug fixes:
- Fixed endpoint field in notebook deployment form. Now it will pre-fill with the notebook name.
- Fixed metrics loading issue for notebook pods.
- Fixed endpoint generation issue in the deployment form where it was not generating the correct URL dynamically.
- Fixed random refresh issue in the deployment version table.
- Fixed GPU dropdown bug which prevented deselection of GPUs from a deployment.
UI Improvements and Bug fixes
New Features & Improvements:
- Added an option to filter the workspace and deployment list based on environments.
- Added pagination on ML Repos page.
- Added a “Used in” info for volumes to show which deployments are using the volume.
- Added a confirmation popup to confirm if the user really wants to close the form when they close a form drawer that has data in the fields.
- Added “tfy-volume://” prefix on the volumes details page for Volume FQN.
- Added an info icon in version status pills on the deployment details page to show progress timeline on hover.
- Added labels for the “Complete Rollout” and “Skip Rollout Step” buttons on the deployment progress tooltip.
- Changed blob-storage deletion flow to accept the storage name for deletion confirmation.
- Changed protocol dropdown width in deployment form to improve description readability.
- Changed rollout field position to advanced fields in the deployment form.
- Changed “No deployment” page to add a description of service and job deployment types.
- Changed description for “Storage Root” field in blob storage form to show example values.
- Changed button labels for integrations from “Create” to “Connect” to represent a more accurate action.
- Removed “escape” key click to hide side drawers.
Bug fixes:
- Fixed description for the “Max surge” field in deployment form.
- Fixed file mounts deletion in deployment form.
- Fixed the issue where “Latest Version” info was still visible on the deployment details page after the deployment was complete.
- Fixed alignment of filter popup on compare screen for Job runs.
- Fixed model action overlay on schema drawer.
- Fixed 500 error when the users click on compare button for ML repo with no runs.
- Fixed word wrapping for docker registry error messages.
- Fixed the flow of workspace creation to redirect to page one of the workspace table after successful creation.
- Fixed created date issue on the environment and team listing page where all created dates were showing as the current date.
- Fixed error message for logs tab when Loki is not configured for a cluster
- Fixed error message for metrics tab when Prometheus is not configured for a cluster