Deploy Service Using Python SDK

In this guide, we’ll deploy a FastAPI service for solving the Iris classification problem. This problem involves predicting the species of an iris flower based on its sepal length, sepal width, petal length, and petal width. There are three species: Iris setosa, Iris versicolor, and Iris virginica.

Project Setup

We’ve already created a FastAPI service for the Iris classification problem, and you can find the code in our GitHub repository: GitHub Repository.

Clone the GitHub repository with the following command:

git clone https://github.com/truefoundry/getting-started-examples.git

Navigate to the project directory:

cd deploy-ml-model

Please review the service code to become familiar with the code you’ll deploy.

Project Structure

The project files are organized as follows:

.
├── app.py - Contains FastAPI code for inference.
├── iris_classifier.joblib - The model file.
└── requirements.txt - Lists dependencies.

All these files are located in the same directory.

Prerequisties

Before you proceed with the guide, make sure you have the following:

Truefoundry CLI:
Set up and configure the TrueFoundry CLI tool on your local machine by following the Setup for CLI guide.
Workspace:
To deploy your service, you’ll need a workspace. If you don’t have one, you can create it using this guide: Creating a Workspace or seek assistance from your cluster administrator. Also, copy the Workspace FQN as this will be required later.

Deploying the Service

Create a deploy.py file in the same directory as your Service code (app.py). This file will contain the necessary configuration for your Service.

Your directory structure will then appear as follows:

File Structure

.
├── iris_classifier.joblib
├── app.py
├── deploy.py
└── requirements.txt

`deploy.py`

Picking a value for `host`

Providing a host value depends on the base domain urls configured in the cluster settings, you can learn how to find the base domain urls available to you here

For e.g. If your base domain url is *.truefoundry.your-org.com then a valid value can be fastapi-your-workspace-8000.truefoundry.your-org.com.

Alternatively if you have a non wildcard based domain url e.g. truefoundry.your-org.com, then a valid value can be truefoundry.your-org.com/fastapi-your-workspace-8000

import logging
from truefoundry.deploy import Build, PythonBuild, Service, Resources, Port

# Set up logging to display informational messages
logging.basicConfig(level=logging.INFO)

# Create a TrueFoundry **Service** object to configure your service
service = Service(
  	# Specify the name of the service
    name="your-service",
    # Define how to build your code into a Docker image
  	image=Build(
        # `PythonBuild` helps specify the details of your Python Code.
      	# These details will be used to templatize a DockerFile to build your Docker Image
        build_spec=PythonBuild(
          	# Define the command to run the application
            command="uvicorn app:app --port 8000 --host 0.0.0.0",
   		      # Specify the path to requirements file
            requirements_path="requirements.txt",
        )
    ),
  	# Set the ports your server will listen on
    ports=[
        Port(
            port=8000,
          	# Define the host for the service, to learn how to set this view the above callout
            host="your-service-your-workspace-8000.your-organization.ai",
        )
    ],
  	# Define the resource constraints.
    #
    # Requests are the minimum amount of resources that a container needs to run.
    # Limits are the maximum amount of resources that a container can use.
    #
    # If a container tries to use more resources than its limits, it will be throttled or killed.
    resources=Resources(
      # CPU is specified as a number. 1 CPU unit is equivalent to 1 physical CPU core, or 1 virtual core.
      cpu_request=0.2,
      cpu_limit=0.5,

      # Memory is defined as an integer and the unit is Megabytes.
      memory_request=200,
      memory_limit=500,

      # Ephemeral storage is defined as an integer and the unit is Megabytes.
      ephemeral_storage_request=1000,
      ephemeral_storage_limit=2000,
    ),
  	# Define environment variables that your Service will have access to
    env={
        "UVICORN_WEB_CONCURRENCY": "1",
        "ENVIRONMENT": "dev"
    }
)

# Deploy the service to the specified workspace, copy workspace FQN using the following guide
# https://docs.truefoundry.com/docs/key-concepts#creating-a-workspace
service.deploy(workspace_fqn="your-workspace-fqn")

To understand the code, you can click the following recipe:

Deploy FastAPI service via python

Setup the project

First, you need to import the following modules:

argparse from Python’s standard library.
logging from Python’s standard library.
Import the necessary classes (Build, PythonBuild, Service, Resources, and Port) from servicefoundry.

Set up logging to display servicefoundry logs by configuring the log level to INFO.

import argparse
import logging
from servicefoundry import Build, PythonBuild, Service, Resources, Port

logging.basicConfig(level=logging.INFO)

Add the arguments

Create an argument parser using argparse.ArgumentParser(). This parser will handle command-line arguments.

Add required arguments to the parser using the .add_argument() method.

Call .parse_args() on the parser to obtain a Namespace containing the parsed arguments.

parser = argparse.ArgumentParser()
parser.add_argument("--workspace_fqn", required=True, type=str)
parser.add_argument("--host", required=True, type=str)
args = parser.parse_args()

Setup Service

Now, it’s time to define the properties of the Service:

Specify the name of the service, which will be its identifier in TrueFoundry’s deployments dashboard.
Define the image with instructions on how to build the container image.
Set up the ports the server should listen to.
Configure the resources for the application.
Define the env variables that should be available in the server environment.

service = Service(
    name="fastapi",
    image=Build(
        build_spec=PythonBuild(
            command="uvicorn app:app --port 8000 --host 0.0.0.0",
            requirements_path="requirements.txt",
        )
    ),
    ports=[
        Port(
            port=8000,
            host=args.host,
        )
    ],
    resources=Resources(
        cpu_request=0.2,
        cpu_limit=0.5,
        memory_request=200,
        memory_limit=500,
      	ephemeral_storage_request=1000,
      	ephemeral_storage_limit=2000
    ),
    env={
        "UVICORN_WEB_CONCURRENCY": "1",
        "ENVIRONMENT": "dev"
    }
)

Define the Docker Image Build Instructions

For defining how to build your code into a Docker image, use the Build class:

Specify the build_source to determine the source code location. If not provided, the current working directory is used.
Define the build_spec using the PythonBuild class to set up a Python environment.

service = Service(
    name="fastapi",
    image=Build(
        build_spec=PythonBuild(
            command="uvicorn app:app --port 8000 --host 0.0.0.0",
            requirements_path="requirements.txt",
        )
    ),
    ports=[
        Port(
            port=8000,
            host=args.host,
        )
    ],
    resources=Resources(
        cpu_request=0.2,
        cpu_limit=0.5,
        memory_request=200,
        memory_limit=500,
      	ephemeral_storage_request=1000,
      	ephemeral_storage_limit=2000
    ),
    env={
        "UVICORN_WEB_CONCURRENCY": "1",
        "ENVIRONMENT": "dev"
    }
)

Configure the Python Build

In the PythonBuild class, provide the following arguments:

command: The command to start your service.
requirements_path: The path to your dependencies file.

service = Service(
    name="fastapi",
    image=Build(
        build_spec=PythonBuild(
            command="uvicorn app:app --port 8000 --host 0.0.0.0",
            requirements_path="requirements.txt",
        )
    ),
    ports=[
        Port(
            port=8000,
            host=args.host,
        )
    ],
    resources=Resources(
        cpu_request=0.2,
        cpu_limit=0.5,
        memory_request=200,
        memory_limit=500,
      	ephemeral_storage_request=1000,
      	ephemeral_storage_limit=2000
    ),
    env={
        "UVICORN_WEB_CONCURRENCY": "1",
        "ENVIRONMENT": "dev"
    }
)

Specify Port

Set the port your server will listen on. In this case, it’s 8000, matching the port in your uvicorn command.

service = Service(
    name="fastapi",
    image=Build(
        build_spec=PythonBuild(
            command="uvicorn app:app --port 8000 --host 0.0.0.0",
            requirements_path="requirements.txt",
        )
    ),
    ports=[
        Port(
            port=8000,
            host=args.host,
        )
    ],
    resources=Resources(
        cpu_request=0.2,
        cpu_limit=0.5,
        memory_request=200,
        memory_limit=500,
      	ephemeral_storage_request=1000,
      	ephemeral_storage_limit=2000
    ),
    env={
        "UVICORN_WEB_CONCURRENCY": "1",
        "ENVIRONMENT": "dev"
    }
)

Specify resource constraints.

You can specify resource constraints using the Resources class.

cpu_request: Specifies the minimum CPU reserved for the application (0.5 represents 50% of CPU resources).
cpu_limit: Defines the upper limit on CPU usage, beyond which the application is throttled.
memory_request: Specifies the minimum required memory (e.g., 1 means 1 MB).
memory_limit: Sets the maximum memory allowed; exceeding this limit triggers an Out of Memory (OOM) error.
ephemeral storage_request: Specifies the minimum required temporary storage.
ephemeral storage_limit: Sets the maximum temporary storage capacity allowed.

These resource constraints help you control the allocation and utilization of CPU, memory, and temporary storage for your deployments. Temporary storage, often referred to as ephemeral storage, is a short-term storage space for your application’s temporary data.

service = Service(
    name="fastapi",
    image=Build(
        build_spec=PythonBuild(
            command="uvicorn app:app --port 8000 --host 0.0.0.0",
            requirements_path="requirements.txt",
        )
    ),
    ports=[
        Port(
            port=8000,
            host=args.host,
        )
    ],
    resources=Resources(
        cpu_request=0.2,
        cpu_limit=0.5,
        memory_request=200,
        memory_limit=500,
      	ephemeral_storage_request=1000,
      	ephemeral_storage_limit=2000
    ),
    env={
        "UVICORN_WEB_CONCURRENCY": "1",
        "ENVIRONMENT": "dev"
    }
)

Specifying environment variables

You can also provide environment variables using a dictionary of the format {"env_var_name": "env_var_value". This is helpful for configurations like environment type (dev/prod) or model registry links.

service = Service(
    name="fastapi",
    image=Build(
        build_spec=PythonBuild(
            command="uvicorn app:app --port 8000 --host 0.0.0.0",
            requirements_path="requirements.txt",
        )
    ),
    ports=[
        Port(
            port=8000,
            host=args.host,
        )
    ],
    resources=Resources(
        cpu_request=0.2,
        cpu_limit=0.5,
        memory_request=200,
        memory_limit=500,
      	ephemeral_storage_request=1000,
      	ephemeral_storage_limit=2000
    ),
    env={
        "UVICORN_WEB_CONCURRENCY": "1",
        "ENVIRONMENT": "dev"
    }
)

Deploy the service

Use service.deploy() to initiate the deployment. Provide the workspace_fqn (fully qualified workspace name) to specify where the service should be deployed.

service.deploy(workspace_fqn=args.workspace_fqn)

To deploy using Python SDK use:

python deploy.py

Run the above command from the same directory containing the app.py and requirements.txt files.

Exclude files when building and deploying your source code:

To exclude specific files from being built and deployed, create a .tfyignore file in the directory containing your deployment script (deploy.py). The .tfyignore file follows the same rules as the .gitignore file.

If your repository already has a .gitignore file, you don’t need to create a .tfyignore file. Service Foundry will automatically detect the files to ignore.

Place the .tfyignore file in the project’s root directory, alongside deploy.py.

After running the command mentioned above, wait for the deployment process to complete. Monitor the status until it shows DEPLOY_SUCCESS:, indicating a successful deployment.

Once deployed, you’ll receive a dashboard access link in the output, typically mentioned as You can find the application on the dashboard:. Click this link to access the deployment dashboard.

View your deployed service

Your deployment will be successful in a few seconds, and your service will be displayed as active (green), indicating that it’s up and running.

Congratulations! You’ve successfully deployed your FastAPI service.

To learn how to use your service-specific dashboard and send requests to your service, check out this guide:
Interacting with your Service

Deploy Service From A Public Github Repository Interacting With Your Service

On this page

Project Setup
Project Structure
Prerequisties
Deploying the Service
deploy.py
View your deployed service

Getting Started

Service

Job

Workflow

Large Language Models (LLMs)

Workbenches: Notebooks and SSH

Volumes

Async Service

Secret Management

ML Repository

Access Control

Deploying On Your Own Cloud

HOW TOs

Deploy Service Using Python SDK

Project Setup

Project Structure

Prerequisties

Deploying the Service

`deploy.py`

Picking a value for `host`

Exclude files when building and deploying your source code:

View your deployed service

Getting Started

Service

Job

Workflow

Large Language Models (LLMs)

Workbenches: Notebooks and SSH

Volumes

Async Service

Secret Management

ML Repository

Access Control

Deploying On Your Own Cloud

HOW TOs

​Project Setup

​Project Structure

​Prerequisties

​Deploying the Service

​deploy.py

​Picking a value for host

​Exclude files when building and deploying your source code:

​View your deployed service

Project Setup

Project Structure

Prerequisties

Deploying the Service

`deploy.py`

Picking a value for `host`

Exclude files when building and deploying your source code:

View your deployed service