Documentation

via SDK for Python

Suggest Edits

👍

What you'll learn

  • Deploy your training code as a Job on Truefoundry
  • Configuring job to not run immediately once it is deployed

This is a guide to deploying training code as a Job via servicefoundry, which can either be triggered automatically on deployment or manually using either the TrueFoundry Dashboard or our Python SDK.

After you complete the guide, you will have successfully deployed a job to train a model on the iris dataset. Your jobs deployment dashboard will look similar to this:

Project structure

To complete this guide, you are going to create the following files:

  • train.py : contains our training code
  • requirements.txt : contains our dependencies
  • deploy.py contains our deployment code (you can also use a deployment configuration for deploying using a YAML file)

Your final file structure is going to look like this:

.
├── train.py
├── deploy.py
└── requirements.txt

As you can see, all the following files are created in the same folder/directory

Step 1: Implement the training code

The first step is to create a job that trains a scikit learn model on iris dataset

We start with a train.py containing our training code and requirements.txt with our dependencies.

.
├── train.py
└── requirements.txt

train.py

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report

X, y = load_iris(as_frame=True, return_X_y=True)
X = X.rename(columns={
        "sepal length (cm)": "sepal_length",
        "sepal width (cm)": "sepal_width",
        "petal length (cm)": "petal_length",
        "petal width (cm)": "petal_width",
})

# NOTE:- You can pass these configurations via command line
# arguments, config file, environment variables.
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)
# Initialize the model
clf = LogisticRegression(solver="liblinear")
# Fit the model
clf.fit(X_train, y_train)

preds = clf.predict(X_test)
print(classification_report(y_true=y_test, y_pred=preds))

Click on the Open Recipe below to understand the components of train.py:

🦉
Training a scikit-learn model
Open Recipe

requirements.txt

numpy==1.24.1
pandas==1.5.2
scikit-learn==1.2.0

Step 2: Deploying as a Job

You can deploy services on Truefoundry programmatically via our Python SDK.

Create the deploy.py, after which our file structure will look like this:

File Structure

.
├── train.py
├── deploy.py
└── requirements.txt

deploy.py

📘

In the code below, ensure to replace "<YOUR_WORKSPACE_FQN>" in the last line with your workspace FQN.

import argparse
import logging
from servicefoundry import Build, Job, PythonBuild, Resources, Manual

logging.basicConfig(level=logging.INFO)

parser = argparse.ArgumentParser()
parser.add_argument("--workspace_fqn", required=True, type=str)
args = parser.parse_args()

# First we define how to build our code into a Docker image
image = Build(
    build_spec=PythonBuild(
        command="python train.py",
        requirements_path="requirements.txt",
    )
)

# Specify job name, resources, environment variables etc here
job = Job(
    name="iris-train-job",
    image=image,
  	resources=Resources(memory_limit=1500, memory_request=100),
)
job.deploy(workspace_fqn=args.workspace_fqn)

In the above deploy.py file, you may specify the trigger = Manual() to deploy a job with a manual trigger. Follow the recipe below to understand the deploy.py file:

🦉
Deploy Training Code as a Job
Open Recipe

To deploy the job using Python API use:

python deploy.py --workspace_fqn <YOUR WORKSPACE FQN HERE>

📘

.tfyignore file

You can mention files and folders you don't want to be included in the deployed source code in a .tfyignore file.

End result

On successful deployment, the Job will be created. To trigger it manually, you can either do so from the TrueFoundry Dashboard or use the trigger_job function in our Python SDK as documented here on the Trigger a Job page.

We can now visit our Applications page to check the Build status, Build Logs, Runs History and monitor the progress of runs. See Monitoring and Debugging guide for more details.

See Also

Updated about 1 month ago