Deploying your first service

👍

What you'll learn

• Creating a FastAPI service to serve an ML model
• Deploying our service via servicefoundry

This is a guide to deploy a scikit-learn model via FastAPI and servicefoundry.

After you complete the guide, you will have a successfully deployed FastAPI Service. Your deployed FastAPI Service will look like this:

Project structure

To complete this guide, you are going to create the following files:

• app.py : contains our inference and FastAPI code
• iris_classifier.joblib : the model file
• deploy.py: contains our deployment code
• requirements.txt : contains our dependencies

Your final file structure is going to look like this:

.
├── app.py
├── iris_classifier.joblib
├── deploy.py 
└── requirements.txt

As you can see, all the following files are created in the same folder/directory

Model details

For this guide, we have already trained a model.
The given model has been trained on iris dataset. Then it is stored as a joblib file in google drive.

Attributes :
sepal length in cm, sepal width in cm, petal length in cm, petal width in cm

Predicted Attribute :
class of iris plant (one of the following - Iris Setosa, Iris Versicolour, Iris Virginica)

Step 1: Fetching the model

Download the model from the following link.
Then move the model in your development directory.

Afterwards, your directory should look like this :

.
└── iris_classifier.joblib

Step 2: Implement the inference service code.

The first step is to create a web API and deploy the model.
For this we are going to use FastAPI for this. FastAPI is a modern, intuitive web framework for building web APIs in python.

Create the app.py and requirements.txt files in the same directory where the model is stored.

.
├── iris_classifier.joblib
├── app.py
└── requirements.txt

app.py

import os
import joblib
import pandas as pd
from fastapi import FastAPI

model = joblib.load("iris_classifier.joblib")

app = FastAPI(root_path=os.getenv("TFY_SERVICE_ROOT_PATH"))

@app.post("/predict")
def predict(
    sepal_length: float, sepal_width: float, petal_length: float, petal_width: float
):
    data = dict(
        sepal_length=sepal_length,
        sepal_width=sepal_width,
        petal_length=petal_length,
        petal_width=petal_width,
    )
    prediction = int(model.predict(pd.DataFrame([data]))[0])
    return {"prediction": prediction}

Click on the Open Recipe below to understand the app.py:

requirements.txt

fastapi
joblib
numpy
pandas
scikit-learn
uvicorn

Step 3: Deploying the inference API

You can deploy services on Truefoundry programmatically via our Python SDK.

Create a deploy.py, after which our file structure will look like this:

File Structure

.
├── iris_classifier.joblib
├── app.py
├── deploy.py
└── requirements.txt

deploy.py

import argparse
import logging
from servicefoundry import Build, PythonBuild, Service, Resources

logging.basicConfig(level=logging.INFO)

parser = argparse.ArgumentParser()
parser.add_argument("--workspace_fqn", required=True, type=str)
args = parser.parse_args()

image = Build(
      build_spec=PythonBuild(
        command="uvicorn app:app --port 8000 --host 0.0.0.0",
        requirements_path="requirements.txt",
      )
)
  
service = Service(
    name="fastapi",
    image=image,
    ports=[{"port": 8000}],
    resources=Resources(
      cpu_request=0.5,
      cpu_limit=1,
      memory_request=1000,
      memory_limit=1500
    ),
    env={
      "UVICORN_WEB_CONCURRENCY": "1",
      "ENVIRONMENT": "dev"
    }
)
service.deploy(workspace_fqn=args.workspace_fqn)

Follow the recipe below to understand the deploy.py file :

To deploy using Python API use:

python deploy.py --workspace_fqn <YOUR WORKSPACE FQN HERE>

Run the above command from the same directory containing the app.py and requirements.txt files.

📘

.tfyignore files

If there are any files you don't want to be copied to the workspace, like a data file, or any redundant files. You can use .tfyignore files in that case.

Interact with the Service

After you run the command given above, you will get a link at the end of the output. The link will take you to your application's dashboard.

Once the build is complete you should get the endpoint for your service :

Copy the following endpoint url, we will need this to make the requests. We will now send the following examples for prediction.

Here is an example Python code snippet to send a request with the above data.

import json
from urllib.parse import urljoin

import requests

# Replace this with the value of your endpoint 
ENDPOINT_URL = "<YOUR_ENDPOINT_URL>" #like https://truefoundry.tfy-ctl-euwe1-production.production.truefoundry.com/fastapi-docs/

response = requests.post(
    urljoin(ENDPOINT_URL, f'predict'),
    json={
            "sepal_length": 5.7,
            "sepal_width": 4.3,
            "petal_width": 3.2,
            "petal_length": 1.5,
    }
)

result = response.json()
print("Predicted Classes:", result["prediction"])

This will give you the following output

Predicted Classes: 0

Next Steps

• Deploying your first service
• Reference Documentation for Services
• Deploying your first Job
• More projects