Deploying your first service

πŸ‘

What you'll learn

  • Creating a FastAPI service to serve an ML model
  • Deploying our service via servicefoundry

This is a guide to deploy a scikit-learn model via FastAPI and servicefoundry.

After you complete the guide, you will have a successfully deployed FastAPI Service. Your deployed FastAPI Service will look like this:

Project structure

To complete this guide, you are going to create the following files:

  • app.py : contains our inference and FastAPI code
  • iris_classifier.joblib : the model file
  • deploy.py: contains our deployment code
  • requirements.txt : contains our dependencies

Your final file structure is going to look like this:

.
β”œβ”€β”€ app.py
β”œβ”€β”€ iris_classifier.joblib
β”œβ”€β”€ deploy.py 
└── requirements.txt

As you can see, all the following files are created in the same folder/directory

Model details

For this guide, we have already trained a model.
The given model has been trained on iris dataset. Then it is stored as a joblib file in google drive.

Attributes :
sepal length in cm, sepal width in cm, petal length in cm, petal width in cm

Predicted Attribute :
class of iris plant (one of the following - Iris Setosa, Iris Versicolour, Iris Virginica)

Step 1: Fetching the model

Download the model from the following link.
Then move the model in your development directory.

Afterwards, your directory should look like this :

.
└── iris_classifier.joblib

Step 2: Implement the inference service code.

The first step is to create a web API and deploy the model.
For this we are going to use FastAPI for this. FastAPI is a modern, intuitive web framework for building web APIs in python.

Create the app.py and requirements.txt files in the same directory where the model is stored.

.
β”œβ”€β”€ iris_classifier.joblib
β”œβ”€β”€ app.py
└── requirements.txt

app.py

import os
import joblib
import pandas as pd
from fastapi import FastAPI

model = joblib.load("iris_classifier.joblib")

app = FastAPI(root_path=os.getenv("TFY_SERVICE_ROOT_PATH"))

@app.post("/predict")
def predict(
    sepal_length: float, sepal_width: float, petal_length: float, petal_width: float
):
    data = dict(
        sepal_length=sepal_length,
        sepal_width=sepal_width,
        petal_length=petal_length,
        petal_width=petal_width,
    )
    prediction = int(model.predict(pd.DataFrame([data]))[0])
    return {"prediction": prediction}

Click on the Open Recipe below to understand the app.py:

requirements.txt

fastapi
joblib
numpy
pandas
scikit-learn
uvicorn

Step 3: Deploying the inference API

You can deploy services on Truefoundry programmatically via our Python SDK.

Create a deploy.py, after which our file structure will look like this:

File Structure

.
β”œβ”€β”€ iris_classifier.joblib
β”œβ”€β”€ app.py
β”œβ”€β”€ deploy.py
└── requirements.txt

deploy.py

import argparse
import logging
from servicefoundry import Build, PythonBuild, Service, Resources

logging.basicConfig(level=logging.INFO)

parser = argparse.ArgumentParser()
parser.add_argument("--workspace_fqn", required=True, type=str)
args = parser.parse_args()

image = Build(
      build_spec=PythonBuild(
        command="uvicorn app:app --port 8000 --host 0.0.0.0",
        requirements_path="requirements.txt",
      )
)
  
service = Service(
    name="fastapi",
    image=image,
    ports=[{"port": 8000}],
    resources=Resources(
      cpu_request=0.5,
      cpu_limit=1,
      memory_request=1000,
      memory_limit=1500
    ),
    env={
      "UVICORN_WEB_CONCURRENCY": "1",
      "ENVIRONMENT": "dev"
    }
)
service.deploy(workspace_fqn=args.workspace_fqn)

Follow the recipe below to understand the deploy.py file :

To deploy using Python API use:

python deploy.py --workspace_fqn <YOUR WORKSPACE FQN HERE>

Run the above command from the same directory containing the app.py and requirements.txt files.

πŸ“˜

.tfyignore files

If there are any files you don't want to be copied to the workspace, like a data file, or any redundant files. You can use .tfyignore files in that case.

Interact with the Service

After you run the command given above, you will get a link at the end of the output. The link will take you to your application's dashboard.

Once the build is complete you should get the endpoint for your service :

Copy the following endpoint url, we will need this to make the requests. We will now send the following examples for prediction.

sepal_lengthsepal_widthpetal_lengthpetal_width
7.03.24.71.4

Here is an example Python code snippet to send a request with the above data.

import json
from urllib.parse import urljoin

import requests

# Replace this with the value of your endpoint 
ENDPOINT_URL = "<YOUR_ENDPOINT_URL>" #like https://truefoundry.tfy-ctl-euwe1-production.production.truefoundry.com/fastapi-docs/

response = requests.post(
    urljoin(ENDPOINT_URL, f'predict'),
    json={
            "sepal_length": 5.7,
            "sepal_width": 4.3,
            "petal_width": 3.2,
            "petal_length": 1.5,
    }
)

result = response.json()
print("Predicted Classes:", result["prediction"])

This will give you the following output

Predicted Classes: 0

Next Steps