Using dockerfile for python function task

In Truefoundry workflow you can pass the dockerfile path in the task config, this is useful when you want to install a binary and use it in your python function task like using a jq command, etc.

TaskDockerFileBuild

To use dockerfile for python task, we need to import TaskDockerFileBuild and use it in PythonTaskConfig.

from truefoundry.workflow import task, workflow, PythonTaskConfig, TaskDockerFileBuild
from truefoundry.deploy import Resources

task_config = PythonTaskConfig(
    image=TaskDockerFileBuild(
        dockerfile_path="Dockerfile",
    ),
    resources=Resources(cpu_request=0.45),
    service_account="<service-account>",
)

Example

In this guide, we will see how to write a Python function task with a dockerfile. In this example, we will take a string as an input and then will calculate the SHA56 hash from the input. We will be passing the

Prerequisites

Before you proceed with the guide, make sure you have the following:

Truefoundry CLI: Set up and configure the TrueFoundry CLI tool on your local machine by following the Setup for CLI guide.
Workspace: To deploy your workflow, you'll need a workspace. If you don't have one, you can create it using this guide: Creating a Workspace or seek assistance from your cluster administrator.

Creating the workflow

Create a workflow.py where we will write the code for our workflow and then create deploy.py file where we will store the truefoundry configuration for deploying workflow and a Dockerfile.

Your directory structure will then appear as follows:

.  
├── workflow.py  
├── deploy.py
└── Dockerfile

Dockerfile

# Use an official Python runtime as a parent image
FROM python:3.10-slim

# Install jq binary
RUN apt-get update && apt-get install -y jq

# Set the working directory
WORKDIR /app

RUN pip install truefoundry[workflow]==0.3.0rc13

# Copy the current directory contents into the container
COPY . /app

# Set the default command to run Python
CMD ["python"]

workflow.py

from truefoundry.workflow import task, workflow, PythonTaskConfig, TaskDockerFileBuild
from truefoundry.deploy import Resources

task_config = PythonTaskConfig(
    image=TaskDockerFileBuild(
        dockerfile_path="Dockerfile",
    ),
    resources=Resources(cpu_request=0.45),
    service_account="<service-account>",
)


@task(task_config=task_config)
def run_jq(input_json: str) -> str:
    import subprocess

    # Run the jq binary using Python's subprocess module
    process = subprocess.run(
        ["jq", "."],  # jq command to pretty-print JSON
        input=input_json,
        text=True,
        capture_output=True,
    )
    return process.stdout


@workflow
def my_workflow(input_json: str) -> str:
    return run_jq(input_json=input_json)

As you can see, we have given the dockerfile_path argument in PythonTaskConfig where the path to the docker file is used as its value.

deploy.py

from truefoundry.deploy import Workflow, LocalSource


workflow = Workflow(
    name="python-dockerfile-task-example",  # your workflow application name
    workflow_file_path="workflow.py",
)
workflow.deploy(workspace_fqn="<workspace-fqn>")

Now run the below command in the terminal to deploy your workflow, replace with the workspace fqn which you can find on the ui.

python deploy.py

Run the above command from the same directory containing the deploy.py file.