Jump to Content

Documentation API Reference Changelog

Blogs Create Account

Documentation

Blogs Create Account

Documentation API Reference Changelog

All

Pages

Start typing to search…

Getting Started

About TrueFoundry
Create and Setup your account
TrueFoundry Key Concepts
Create Your First Deployment
Setup for CLI
Generating TrueFoundry API Keys

Service

Introduction to a Service
Deploy your first Service
- Deploy Service from a public Github repository
- Deploy Service using Python SDK
Interacting with your Service
Configuring your service
Update, Rollback, Promote
Monitor your Service
Set up Alerts
Log & Monitor Custom Metrics
Set up CI/CD

Job

Introduction to a Job
Running your first Job
- Deploy and Run Job from a public GitHub repository
- Deploy and Run Job using Python SDK
Interacting with your Job
Monitor your Job
Configuring your Job

ML Repository

Introduction to ML Repo
Quick Start
Guides
API Reference

Workflow

Introduction to Workflow
Creating your first workflow
Interacting with workflow
Workflow Concepts
Guides
Infra Set-Up for Workflows

Large Language Models (LLMs)

Introduction to LLMOps
Deploying LLMs
Deploy LLMs using Nvidia TensorRT-LLM (TRT-LLM)
Finetuning LLMs

LLM Gateway

Intro to LLM Gateway
Adding Models to LLM Gateway
- AWS Bedrock
Interacting with LLM Gateway
Configure Gateway

Workbenches: Notebooks and VSCode

Launch Jupyter Notebook
Launch an SSH Server

Volumes

Introduction to Volume
Creating and Utilizing Volume
Interacting with your Volume
Creating Statically Provisioned Volumes

Async Service

Introduction to Async Service
Deploy your first Async Service
Configure async service
Queue Integrations
- AWS SQS
Monitor your Async Service

Secret Management

Introduction to Secrets
Manage Secrets

Collaboration and Access Control

Collaboration and Access Control

Deploying On Your Own Cloud

Modes of Deployment
Deploy Compute Plane
Deploy Control Plane Only
Advanced Configuration
Integrations
SSO Integration
Deploy Truefoundry in an Air-gapped Environment

HOW TOs

Migrate Pytorch Sagemaker Endpoints to TrueFoundry Platform
Create custom K8s objects
Model Deployment
- Scikit Learn
- XGBoost
Setup Gitops using Truefoundry

References

Service
Job
AsyncService
Helm
Image and Build
Deployment Additional Configuration
Guide to inference model on truefoundry

RateLimiting

Rate Limit requests for specified providers or models

Updated about 11 hours ago

Interacting with LLM Gateway