Overview

There are different ways to deploy the models as an API - depending on the framework / type of model.

Truefoundry doesn’t provide any client side framework for model deployment. We believe there are a lot of great open-source frameworks to build inference services and we don’t want to build yet another one. This also helps avoid any vendor lock-in with Truefoundry since you don’t need to change your code to deploy models on Truefoundry or migrate to another platform.

Truefoundry supports deploying models from different frameworks. You can deploy models from HuggingFace, or any custom models that you have logged in the Truefoundry model registry or your own custom inference code in any of the frameworks. All model API deployments in Truefoundry are abstractions on top of the Service Deployment feature. That’s why its highly recommended to get familiar with service deployment first.

​

Bring your own inference code and model in any framework

Truefoundry can deploy model inference code in any framework that you are using. Here are a few examples of deploying models in most commonly used frameworks:

1. Flask
2. FastAPI
3. PytorchServe
4. TensorFlow Serve
5. Mlflow Serve
6. Litserve
7. Truss
8. BentoML