Create Gradio demo for a logged model

👍

What you'll learn

• Retrieve model logged during training via mlfoundry
• Creating a Gradio application to serve your model
• Deploying our service via servicefoundry

This is a guide to deploy a model logged in TrueFoundry's model registry via Gradio using servicefoundry

After you complete the guide, you will have a successfully deployed Gradio Service. Your deployed Gradio Service will look like this:

Project structure

To complete this guide, you are going to create the following files:

• app.py: contains our inference and Gradio code
• deploy.py: contains our deployment code
• requirements.txt: contains our dependencies.

Your final file structure is going to look like this:

.
├── app.py
├── deploy.py
└── requirements.txt

As you can see, all the following files are created in the same folder/directory.

Step 1: Copy the logged model's fqn

Firstly you need the logged model's fqn, to ensure you can retrieve it and use it in the application

For that you can go to click on the Runs tab, and click on the logged model name you want to use.

Once you click on it, you will see the run details, click on the Models tab and copy the fqn of the model you want to use. The model-fqn will be in the following format e.g. "model:truefoundry/user/iris-demo/iris-classifier:1"

Step 2: Implement the inference service code.

The first step is to create a web Interface and deploy the model.
For this, we are going to use Gradio for this. Gradio is a python library using which we can quickly create a web interface on top of our model inference functions.

Create the app.py and requirements.txt files in the same directory where the model is stored.

.
├── app.py
└── requirements.txt

app.py

🚧

In the code below ensure to replace "" with the model-fqn you copied above

#Replace <fqn-of-your-model> with your model-fqn you copied in the previous section
import os
import mlfoundry
import joblib
import pandas as pd
import gradio as gr

client = mlfoundry.get_client()
model_version = client.get_model("<fqn-of-your-model>") # e.g. "model:truefoundry/user/iris-demo/iris-classifier:1"
model = model_version.load()

def model_inference(sepal_length: float, sepal_width: float, petal_length: float, petal_width: float) -> int:
    data = dict(
        sepal_length=sepal_length,
        sepal_width=sepal_width,
        petal_length=petal_length,
        petal_width=petal_width,
    )
    prediction = int(model.predict(pd.DataFrame([data]))[0])
    return prediction

sepal_length_input = gr.Number(label = "Enter the sepal length in cm")
sepal_width_input = gr.Number(label = "Enter the sepal width in cm")
petal_length_input = gr.Number(label = "Enter the petal length in cm")
petal_width_input = gr.Number(label = "Enter the petal width in cm")

inputs = [sepal_length_input, sepal_width_input, petal_length_input, petal_width_input]

output = gr.Number()

gr.Interface(
    fn=model_inference,
    inputs=inputs,
    outputs=output,
).launch(server_name="0.0.0.0", server_port=8080)

requirements.txt

pandas
gradio
scikit-learn
joblib
altair
mlfoundry

Step 3: Deploying the inference API

You can deploy services on TrueFoundry programmatically using our Python SDK

Via python SDK

File Structure

.
├── app.py
├── deploy.py
└── requirements.txt

deploy.py

🚧

In the code below, ensure to replace "YOUR_WORKSPACE_FQN" in the last line with your WORKSPACE_FQN

import logging
from servicefoundry import Build, PythonBuild, Service, Resources

logging.basicConfig(level=logging.INFO)

image=Build(
        build_spec=PythonBuild(
          command="python app.py",
          requirements_path="requirements.txt",
        )
)

service = Service(
    name="mlfoundry-model-gradio",
    image=image,
    ports=[{"port": 8080}],
    resources=Resources(memory_limit=1500, memory_request=1000),
    env={"TFY_API_KEY": "YOUR_TFY_API_KEY"}
)
service.deploy(workspace_fqn="YOUR_WORKSPACE_FQN")

To deploy using Python API use:

python deploy.py

End result

You can go to your deployments dashboard here and you will find a new deployment created with the name "mlfoundry-model-gradio".

Afterwards, click on the service you just deployed. On the top-right corner you will see the endpoint of deployed application.

Click on the link here, you will redirected to your deployed application.