For some model frameworks, Truefoundry can generate a model deployment package containing the following:

  • Inference code wrapped around a general-purpose web framework like FastAPI or more specialized model servers like Triton.
  • A requirements.txt. We also generate a Dockerfile for model servers like Triton, which are complex to set up on all operating systems.
  • A README file that contains instructions on testing locally and deploying on Truefoundry.

​
This approach gives you the flexibility to:

  • Deploy the package as it is to Truefoundry or anywhere else.
  • Change the inference code to add custom business logic and dependencies.
  • Test the code locally.
  • Maintain the generated code in your version control system (Github, Gitlab, etc.).

​
To create the deployment package:

  • Locate the model you want to deploy in the model registry and click the Deploy button.
  • Select a workspace for deployment, and copy the command.
  • Execute the command in your terminal to generate the model deployment package.
❯ tfy deploy-init model --name 'my-sklearn-model-1' --model-version-fqn 'model:truefoundry/my-classification-project/my-sklearn-model-1:1' --workspace-fqn 'tfy-usea1-devtest:deb-ws' --model-server 'fastapi'
...
Generating application code for 'model:truefoundry/my-classification-project/my-sklearn-model-1:1'

Model Server code initialized successfully!

Code Location: /work/model-deployment/my-sklearn-model-1

Next Steps:
- Navigate to the model server directory:
cd /work/model-deployment/my-sklearn-model-1
- Refer to the README file in the directory for further instructions.

❯ cd /work/model-deployment/my-sklearn-model-1
❯ ls
README.md               deploy.py               infer.py                requirements.txt        server.py
  • Follow the instructions present on theREADME.md.