Note
While using ServiceFoundry python SDK
type
is not a required field in any of the imported classes
Job
Description
Describes the configuration for the job
Schema
{
"type": "string",
"name": "string",
"image": {},
"resources": {
"cpu_request": 0.2,
"cpu_limit": 0.5,
"memory_request": 200,
"memory_limit": 500,
"ephemeral_storage_request": 1000,
"ephemeral_storage_limit": 2000,
"instance_family": [
"string"
]
},
"trigger": {
"type": "manual"
},
"retries": 1,
"timeout": 1000,
"successful_jobs_history_limit": 20,
"failed_jobs_history_limit": 20,
"restart": "Never",
"params": [
{
"name": "string",
"description": "string",
"default": "string"
}
],
"env": null,
"service_account": "string",
"file_mounts": [
{
"mount_dir": "string",
"data": {
"property1": "string",
"property2": "string"
}
}
]
}
Properties
Name | Type | Required | Description |
---|---|---|---|
type | string | true | none |
name | string | true | Name of the job |
image | object | true | Specify whether you want to deploy a Docker image or build and deploy from source code |
resources | Resources | false | Describes the resource constraints for the application so that it can be deployed accordingly on the cluster To learn more you can go here |
trigger | object | true | Specify the trigger |
retries | integer | false | Specify the maximum number of attempts to retry a job before it is marked as failed. |
timeout | integer | false | Job timeout in seconds. |
restart | string | true | When to restart the job |
params | [Param] | false | Configure params and pass it to create different job runs |
env | object¦null | false | Configure environment variables to be injected in the service. Docs |
service_account | string | false | Service account that this workload should use |
file_mounts | [FileMount] | false | Files to be mounted +ignore |
Enumerate Values
Python Examples
Building using a Python Build pack
from servicefoundry import Job, Build, PythonBuild, Port
job = Job(
name="my-job",
image=Build(
build_spec=PythonBuild(
python_version="3.9",
requirements_path="requirements.txt",
command="python train.py", # or as a list ["python", "train.py"]
)
)
)
deployment = job.deploy(workspace_fqn='...')
from servicefoundry import (
Job, Build, PythonBuild, LocalSource, Resources,
Manual,
)
job = Job(
name="my-job",
image=Build(
docker_registry="admin-truefoundry:production-euwe1-ecr", # optional, FQN registry to push the built image to
build_source=LocalSource(
project_root_path="./"
),
build_spec=PythonBuild(
python_version="3.9",
build_context_path="./",
requirements_path="requirements.txt",
pip_packages=["scikit-learn==1.0.2", "numpy==1.20.0"],
apt_packages=["ffmpeg", "curl", "wget"],
command="python train.py", # or as a list ["python", "train.py"]
),
),
command="python train.py", # or as a list ["python", "train.py"]
# Note: This command has precedence over the one in build_spec
resources=Resources(
cpu_request=1,
memory_request=1000, # in Megabytes
ephemeral_storage_request=1000, # in Megabytes
cpu_limit=4,
memory_limit=4000,
ephemeral_storage_limit=10000,
instance_family=["c6i", "t3", "m4"],
),
env={
"VERY_SECRET_API_KEY": "tfy-secret://my-user:my-secret-group:VERY_SECRET_API_KEY"
},
trigger=Manual(run=False),
retries=1,
timeout=100,
successful_jobs_history_limit=20,
failed_jobs_history_limit=20,
restart="Never",
service_account="my-k8s-service-account"
)
deployment = job.deploy(workspace_fqn='...')
Building using a Dockerfile
This is useful if you have an already written Dockerfile
from servicefoundry import Job, Build, DockerFileBuild
job = Job(
name="my-job",
image=Build(
build_spec=DockerFileBuild(),
),
)
deployment = job.deploy(workspace_fqn='...')
from servicefoundry import (
Job, Build, PythonBuild, LocalSource, Resources,
Manual,
)
job = Job(
name="my-job",
image=Build(
docker_registry="admin-truefoundry:production-euwe1-ecr", # optional, FQN registry to push the built image to
build_source=LocalSource(
project_root_path="./"
),
build_spec=PythonBuild(
python_version="3.9",
build_context_path="./",
requirements_path="requirements.txt",
pip_packages=["scikit-learn==1.0.2", "numpy==1.20.0"],
apt_packages=["ffmpeg", "curl", "wget"],
command="python train.py", # or as a list ["python", "train.py"]
),
),
resources=Resources(
cpu_request=1,
memory_request=1000, # in Megabytes
ephemeral_storage_request=1000, # in Megabytes
cpu_limit=4,
memory_limit=4000,
ephemeral_storage_limit=10000,
instance_family=["c6i", "t3", "m4"],
),
env={
"VERY_SECRET_API_KEY": "tfy-secret://my-user:my-secret-group:VERY_SECRET_API_KEY"
},
trigger=Manual(run=False),
retries=1,
timeout=100,
successful_jobs_history_limit=20,
failed_jobs_history_limit=20,
restart="Never",
service_account="my-k8s-service-account"
)
deployment = job.deploy(workspace_fqn='...')
Using an already existing Image from an Image Registry
Note: The remote Image Registry should either be public or added to Truefoundry Platform via the Integrations Tab
from servicefoundry import Job, Image
job = Job(
name="my-job",
image=Image(
image_uri="docker/whalesay:latest",
),
)
deployment = job.deploy(workspace_fqn='...')
from servicefoundry import (
Job, Image, Resources, Manual
)
job = Job(
name="my-job",
image=Image(
image_uri="docker/whalesay:latest",
docker_registry="admin-truefoundry:production-euwe1-ecr", # optional, FQN registry to pull the built image from
command=["cowsay", "boo"],
),
resources=Resources(
cpu_request=1,
memory_request=1000, # in Megabytes
ephemeral_storage_request=1000, # in Megabytes
cpu_limit=4,
memory_limit=4000,
ephemeral_storage_limit=10000,
instance_family=["c6i", "t3", "m4"],
),
env={
"VERY_SECRET_API_KEY": "tfy-secret://my-user:my-secret-group:VERY_SECRET_API_KEY"
},
trigger=Manual(run=False),
retries=1,
timeout=100,
successful_jobs_history_limit=20,
failed_jobs_history_limit=20,
restart="Never",
service_account="my-k8s-service-account"
)
deployment = job.deploy(workspace_fqn='...')
Deploying from a Git Repository
Note: The remote Git Server (Github or Bitbucket) needs to be connected via the Integrations Tab
from servicefoundry import (
Job, Build, GitSource, PythonBuild, Resources,
Manual
)
job = Job(
name="my-job",
image=Build(
docker_registry="admin-truefoundry:production-euwe1-ecr", # optional, FQN registry to push the built image to
build_source=GitSource(
repo_url="https://github.com/myorg/myrepo",
branch_name="main", # Optional, by default the default branch configured on the repo
ref="547ea1423267902a6ff98517d4ae205d8f7c47f8", # Optional, by default latest commit on the branch
),
build_spec=PythonBuild(
python_version="3.9",
build_context_path="./",
requirements_path="requirements.txt",
pip_packages=["scikit-learn==1.0.2", "numpy==1.20.0"],
apt_packages=["ffmpeg", "curl", "wget"],
command="python train.py", # or as a list ["python", "train.py"]
),
),
resources=Resources(
cpu_request=1,
memory_request=1000, # in Megabytes
ephemeral_storage_request=1000, # in Megabytes
cpu_limit=4,
memory_limit=4000,
ephemeral_storage_limit=10000,
instance_family=["c6i", "t3", "m4"],
),
env={
"VERY_SECRET_API_KEY": "tfy-secret://my-user:my-secret-group:VERY_SECRET_API_KEY"
},
trigger=Manual(run=False),
retries=1,
timeout=100,
service_account="my-k8s-service-account"
)
deployment = job.deploy(workspace_fqn='...')
from servicefoundry import (
Job, Build, DockerFileBuild, GitSource, Resources,
Manual
)
job = Job(
name="my-job",
image=Build(
docker_registry="admin-truefoundry:production-euwe1-ecr", # optional, FQN registry to push the built image to
build_source=GitSource(
repo_url="https://github.com/myorg/myrepo",
branch_name="main", # Optional, by default the default branch configured on the repo
ref="547ea1423267902a6ff98517d4ae205d8f7c47f8", # Optional, by default latest commit on the branch
),
build_spec=DockerFileBuild(
dockerfile_path='./Dockerfile',
build_context_path='./',
command="python train.py",
),
),
resources=Resources(
cpu_request=1,
memory_request=1000, # in Megabytes
ephemeral_storage_request=1000, # in Megabytes
cpu_limit=4,
memory_limit=4000,
ephemeral_storage_limit=10000,
instance_family=["c6i", "t3", "m4"],
),
env={
"VERY_SECRET_API_KEY": "tfy-secret://my-user:my-secret-group:VERY_SECRET_API_KEY"
},
trigger=Manual(run=False),
retries=1,
timeout=100,
successful_jobs_history_limit=20,
failed_jobs_history_limit=20,
restart="Never",
service_account="my-k8s-service-account"
)
deployment = job.deploy(workspace_fqn='...')
Triggers
The modules below help configure the Triggers for the Job
Manual
Description
Describes that we are going to manually trigger our job.
Schema
{
"type": "string"
}
Properties
Name | Type | Required | Description |
---|---|---|---|
type | string | true | none |
Python Examples
from servicefoundry import Job, Manual
job = Job(
...
trigger=Manual(
run=False
),
...
)
)
Schedule
Description
Describes that we are going to schedule our job to run at a schedule, making our job a cron job.
Schema
{
"type": "string",
"schedule": "string",
"concurrency_policy": "Forbid"
}
Properties
Name | Type | Required | Description |
---|---|---|---|
type | string | true | none |
schedule | string | true | Specify the schedule for this job to be run periodically in cron format. Learn more |
concurrency_policy | string | false | Choose whether to allow this job to run while another instance of the job is running, or to replace the currently running instance. Allow will enable multiple instances of this job to run. Forbid will keep the current instance of the job running and stop a new instance from being run. Replace will terminate any currently running instance of the job and start a new one. |
Enumerate Values
Property | Value |
---|---|
concurrency_policy | Forbid |
concurrency_policy | Allow |
concurrency_policy | Replace |
Python Examples
from servicefoundry import Job, Schedule
job = Job(
...
trigger=Schedule(
schedule="0 8 1 * *", # Cron format
concurrency_policy="Forbid" # Any one of ["Forbid", "Allow", "Replace"]
),
...
)
)
Param
Param
Description
Describes what params are going to be given to the Deployment
Schema
{
"name": "string",
"description": "string",
"default": "string"
}
Properties
Name | Type | Required | Description |
---|---|---|---|
name | string | true | Name of the param |
description | string | false | Description of param |
default | string | false | Default value or placeholder |
Updated 20 days ago