Jump to Content
TruefoundryDocumentation
HomeDocumentationRecipesAPI ReferenceChangelogDiscussions
DocumentationLog InTruefoundry
Log In
HomeDocumentationRecipesAPI ReferenceChangelogDiscussions

OVERVIEW

  • Introduction
  • Infrastructure Onboarding

Get started

  • Setup for CLI
  • Getting started - Dashboard

examples

  • End to End Examples
  • Services, Autoscaling and Rollouts
    • via UI
    • via Python SDK
    • via Servicefoundry CLI
  • Job with Parameters
    • via UI
    • via Python SDK
    • via Servicefoundry CLI
  • Finetuning LLM
  • Docs QA
  • Huggingface Model Deployment

Deploy a service

  • Introduction to service
  • Deploy your first service
    • via UI
    • via Python SDK
  • Interacting with your Service
    • Browse logs for an application
    • Browse metrics for an application
    • Stopping a Service
  • Advanced Guides
    • Deploy a Dockerized Service
    • Deploy a gRPC Service
    • Deploy Tensorflow Serving with gRPC
    • Deploy a LLM (Llama 2 7B) with Huggingface Text-Generation-Inference
    • Deploy a LLM with vLLM
  • Additional Configurations
    • Add health checks
    • Add Authentication to Endpoints
    • Intercepts
  • Scaling and Lifecycle
    • AutoScaling
    • Rollout Strategy

Deploy a Job

  • Introduction to Job
  • Deploy your first job
    • via User Interface
    • via SDK for Python
  • Interacting with your Job
    • Trigger a Job
    • Viewing Job Runs
    • Terminate a Job
    • Browsing Job Logs
    • Browsing Job Metrics
  • Advanced Guides
    • Deploy a Cron Job
    • Deploy a Dockerized Job
    • Deploy a Job with Parameters
  • Additional Configurations
  • Setting up CI/CD using TrueFoundry

ML Repository

  • Introduction to ML Repo
  • Quick Start
  • Guides
    • Installation and setup
    • Creating a run
    • Log and Get Parameters
    • Log and Get Metrics
    • Log and Get Artifacts
    • Log and Get Models
    • Add Tags
    • Log Images
    • Log Plots
    • Creating ML Repo via UI
  • API Reference
    • MlFoundry
    • Runs
    • Artifact
    • Models

Large language models (LLMs)

  • Deploying an LLM
  • Send Requests to the Deployed LLM
  • Model Fine-Tuning
  • Intregration With Langchain
  • TrueFoundry Playground

Deploy model directly

  • Introduction to Model Deployment
  • Deploy from Huggingface Hub
  • Deploy from TrueFoundry Registry
  • Guides
    • Create Service for a live endpoint
    • Caching Huggingface models using volumes
    • Deploy a logged XGBoost model
    • Deploy a logged LGBM model
    • Deploy a logged Keras model
    • Deploy a logged PyTorch model

Running Jupyter notebooks

  • Start a Jupyter Notebook
  • Start a VSCode IDE
  • Guides
    • Gaining Root User Access
    • Adding Apt Packages
    • Configuring Python Environments Inside a Notebook
    • Run notebook with GPU

DEPLOY A HELM APPLICATION

  • Deploy LabelStudio with TrueFoundry

VOLumes

  • Creating a Volume
  • Attaching Volumes to Deployment

ASYNC SERVICE

  • Introduction to Async Service
  • Creating an Async Service
  • Deploy your first Async Service
    • via UI
    • via Python SDK
    • via CLI
  • Async Service Autoscaling

Deployment options

  • Build
  • Resources
    • Adding GPUs to Applications
  • Redeploy
  • Environment Variables
  • Build Docker Image with build-args
  • Ignoring files: .tfyignore
  • Generate API Key
  • Login
  • Endpoints and Routing
    • Specifying host for your service
    • Figuring out your Configured Domain
    • Customize path for a port
    • Setting up Domain
  • Promotion Flow

Concepts

  • Cluster and Workspace
  • Secrets
  • Teams
  • Environments
  • Collaboration
    • Invite User
    • Manage Teams

Deploying on your own cloud

  • Creating your own kubernetes cluster
  • Azure
    • Azure Infra requirements
    • Creating an AKS cluster using onboarding-cli
    • Azure AKS
    • Understanding Azure Node Pools
    • Managing applications
    • SSL and DNS setup in Azure
    • Post Cluster configurations
  • AWS
    • AWS infra requirements
    • Creating an EKS cluster using onboarding-cli
    • EFS in AWS EKS
    • Autoscaling in EKS: Karpenter
    • Post cluster configurations
  • Google Cloud Platform
    • GCP Infra Requirements
    • Setting up DNS and certificates on GKE
    • GCP Post Cluster Configurations
  • Generic
    • kind
    • minikube
    • Setting up SSL using cert-manager through HTTP-01 challenge
  • Connect cluster to control plane
  • SSO
    • Azure AD
  • Integrations
    • Docker Registry
    • Git
    • Blob Storage
    • Secret Store

References

  • Service
  • Job
  • AsyncService
  • Model Deployment
  • Volume
  • Helm
  • Image and Build
  • Deployment Additional Configuration

FALL BACK

  • Cost and Resource Insights
Powered by 

Table of Contents

Suggest Edits
  • Service
    • Service
    • Liveliness/Readiness Probe
      • HttpProbe
      • HealthProbe
    • Port
      • Port
    • Param
      • Param

  • Job
    • Job
    • Triggers
      • Manual
      • Schedule

  • Model Deployment
    • ModelDeployment
    • Model Source
      • TruefoundryModelRegsitry
      • HuggingfaceModelHub

  • Image
    • Image
    • Build
    • Build Spec
      • PythonBuild
      • DockerFileBuild
    • Build Source
      • LocalSource
      • GitSource
      • RemoteSource

  • Deployment - Additional Configuration
    • Resources
    • FileMount
    • Autoscaling

Updated 8 days ago