Service

Note

While using ServiceFoundry python SDK type is not a required field in any of the imported classes

Description

Describes the configuration for the service

Schema

{
    "type": "string",
    "name": "string",
    "image": {},
    "resources": {
        "cpu_request": 0.2,
        "cpu_limit": 0.5,
        "memory_request": 200,
        "memory_limit": 500,
        "ephemeral_storage_request": 1000,
        "ephemeral_storage_limit": 2000,
        "instance_family": ["string"]
    },
    "replicas": 1,
    "env": null,
    "ports": [
        {
            "protocol": "TCP",
            "port": 80,
            "expose": true,
            "host": "string",
            "path": "string",
            "auth": {
                "type": "basic_auth",
                "username": "string",
                "password": "string"
            }
        }
    ],
    "liveness_probe": {
        "config": {
            "type": "string",
            "path": "string",
            "port": 65535,
            "host": "string",
            "scheme": "HTTP"
        },
        "initial_delay_seconds": 0,
        "period_seconds": 10,
        "timeout_seconds": 1,
        "success_threshold": 1,
        "failure_threshold": 3
    },
    "readiness_probe": {
        "config": {
            "type": "string",
            "path": "string",
            "port": 65535,
            "host": "string",
            "scheme": "HTTP"
        },
        "initial_delay_seconds": 0,
        "period_seconds": 10,
        "timeout_seconds": 1,
        "success_threshold": 1,
        "failure_threshold": 3
    },
    "service_account": "string",
    "file_mounts": [
        {
            "mount_dir": "string",
            "data": {
                "property1": "string",
                "property2": "string"
            }
        }
    ]
}

Properties

Name	Type	Required	Description
type	string	true	none
name	string	true	Name of the service. This uniquely identifies this service in the workspace. > Name can only contain alphanumeric characters and ’-’ and can be atmost 25 characters long
image	object	true	Specify whether you want to deploy a Docker image or build and deploy from source code
resources	Resources	false	Describes the resource constraints for the application so that it can be deployed accordingly on the cluster To learn more you can go here
replicas	integer	true	Number of service instances/replicas you want to run.
env	object¦null	false	Configure environment variables to be injected in the service.Docs
ports	[Port]	true	Specify the ports you want the service to be exposed to
liveness_probe	HealthProbe	false	Describes the configuration for the Health Probe’s To learn more you can go here
readiness_probe	HealthProbe	false	Describes the configuration for the Health Probe’s To learn more you can go here
service_account	string	false	Service account that this workload should use
file_mounts	[FileMount]	false	Files to be mounted +ignore

Python Examples

Building using a Python Build pack

from servicefoundry import Service, Build, PythonBuild, Port

service = Service(
    name="my-service",
  	image=Build(
      build_spec=PythonBuild(
          python_version="3.9",
          requirements_path="requirements.txt",
          command="uvicorn app:app --port 8080 --host 0.0.0.0", # or as a list ["python", "main.py"]
      )
    ),
    ports=[Port(port=8080)]
)

deployment = service.deploy(workspace_fqn='...')

Building using a Dockerfile

This is useful if you have an already written Dockerfile

from servicefoundry import Service, Build, DockerFileBuild, Port

service = Service(
    name="my-service",
  	image=Build(
      build_spec=DockerFileBuild(),
    ),
    ports=[Port(port=8080)]
)

deployment = service.deploy(workspace_fqn='...')

Using an already existing Image from an Image Registry

Note: The remote Image Registry should either be public or added to TrueFoundry Platform via the Integrations Tab

from servicefoundry import Service, Image, Port

service = Service(
    name="my-service",
  	image=Image(
      image_uri="seldonio/mlserver:1.2.0",
    ),
    ports=[Port(port=8080)]
)

deployment = service.deploy(workspace_fqn='...')

Deploying from a Git Repository

Note: The remote Git server (Github or Bitbucket) needs to be connected via the Integrations Tab

from servicefoundry import (
    Service, Build, GitSource, PythonBuild, Resources,
    Port, HealthProbe, BasicAuthCreds, HttpProbe
)

service = Service(
    name="my-service",
  	image=Build(
        docker_registry="admin-truefoundry:production-euwe1-ecr",  # optional, FQN registry to push the built image to
        build_source=GitSource(
            repo_url="https://github.com/myorg/myrepo",
            branch_name="main", # Optional, by default the default branch configured on the repo
            ref="547ea1423267902a6ff98517d4ae205d8f7c47f8", # Optional, by default latest commit on the branch
        ),
        build_spec=PythonBuild(
            python_version="3.9",
            build_context_path="./",
            requirements_path="requirements.txt",
            pip_packages=["requests", "numpy==1.20.0"],
            apt_packages=["ffmpeg", "curl", "wget"],
            command="uvicorn app:app --port 8000 --host 127.0.0.1", # or as a list ["python", "main.py"]
        ),
    ),
    resources=Resources(
        cpu_request=1,
        memory_request=1000, # in Megabytes
        ephemeral_storage_request=1000, # in Megabytes
        cpu_limit=4,
        memory_limit=4000,
        ephemeral_storage_limit=10000,
        instance_family=["c6i", "t3", "m4"],
    ),
    replicas=2,
    env={
        "NUM_WORKERS": "5",
        "VERY_SECRET_API_KEY": "tfy-secret://my-user:my-secret-group:VERY_SECRET_API_KEY"
    },
    ports=[
        Port(
            port=8080,
            protocol="TCP",  # or "UDP"
            expose=True,
            auth=BasicAuthCreds(
                username="admin",
                password="supersecretpassword",  # Just an example, don't hardcode this!
            )
        )
    ],
    liveness_probe=HealthProbe(
        config=HttpProbe(
            path="v2/health/live",
            port=8080,
            host="0.0.0.0",
            scheme="HTTP"
        ),
        initial_delay_seconds=2,
        period_seconds=5,
        timeout_seconds=5,
        success_threshold=1,
        failure_threshold=3,
    ),
    readiness_probe=HealthProbe(
        config=HttpProbe(
            path="v2/health/ready",
            port=8080,
            host="0.0.0.0",
            scheme="HTTP"
        ),
        initial_delay_seconds=2,
        period_seconds=5,
        timeout_seconds=5,
        success_threshold=1,
        failure_threshold=3,
    ),
    service_account="my-k8s-service-account",
  	file_mount = FileMount(
      mount_dir="./tmp",
      data={"foo":"bar"}
    ),

)

deployment = service.deploy(workspace_fqn='...')

Liveliness/Readiness Probe

The modules below help configuring the Health Probes for the Service. Learn more about Health Probes here.

HttpProbe

Description

Describes the Instructions for assessing container health by executing an HTTP GET request.
To learn more you can go here

Schema

{
    "type": "string",
    "path": "string",
    "port": 65535,
    "host": "string",
    "scheme": "HTTP"
}

Properties

Name	Type	Required	Description
type	string	true	none
path	string	true	The endpoint, relative to the port, to which the HTTP GET request should be directed.
port	integer	true	The TCP socket within the container to which the HTTP GET request should be directed.
host	string	false	none
scheme	string	false	none

Python Examples

from servicefoundry import Service, HealthProbe, HttpProbe

service = Service(
    ...
    liveness_probe=HealthProbe(
        config=HttpProbe(
          path="/livez",
          port=8080,
          host="0.0.0.0",
          scheme="HTTP"
        ),
        ...
    )
)

HealthProbe

Description

Describes the configuration for the Health Probe’s
To learn more you can go here

Schema

{
    "config": {
        "type": "string",
        "path": "string",
        "port": 65535,
        "host": "string",
        "scheme": "HTTP"
    },
    "initial_delay_seconds": 0,
    "period_seconds": 10,
    "timeout_seconds": 1,
    "success_threshold": 1,
    "failure_threshold": 3
}

Properties

Name	Type	Required	Description
config	HttpProbe	true	Describes the Instructions for assessing container health by executing an HTTP GET request. To learn more you can go here
initial_delay_seconds	integer	false	Number of seconds after the container is started before the first probe is initiated.
period_seconds	integer	false	How often, in seconds, to execute the probe.
timeout_seconds	integer	false	Number of seconds after which the probe times out.
success_threshold	integer	false	Minimum consecutive successes for the probe to be considered successful after having failed.
failure_threshold	integer	false	Number of consecutive failures required to determine the container is not alive (liveness probe) or not ready (readiness probe).

Python Examples

from servicefoundry import Service, HealthProbe, HttpProbe

service = Service(
    ...
    liveness_probe=HealthProbe(
        config=HttpProbe(
          path="/readyz",
          port=8080,
          host="0.0.0.0",
          scheme="HTTP"
        ),
        initial_delay_seconds=2,
        period_seconds=5,
        timeout_seconds=5,
        success_threshold=1,
        failure_threshold=3,
    )
)

Port

Description

Describes the ports the service should be exposed to.

Schema

{
    "protocol": "TCP",
    "port": 80,
    "expose": true,
    "host": "string",
    "path": "string",
    "auth": {
        "type": "basic_auth",
        "username": "string",
        "password": "string"
    }
}

Properties

Name	Type	Required	Description
protocol	string	true	Protocol for the port.
port	integer	true	Port number to expose.
expose	boolean	true	Expose the port
host	string	false	Host
path	string	false	Path
auth	BasicAuthCreds	false	+label=Username and password for service auth

Enumerate Values

Property	Value
protocol	TCP
protocol	UDP

Python Examples

from servicefoundry import Service, Port

service = Service(
    ...
    ports=[Port(
        port=8080,
        protocol="TCP",  # or "UDP"
        expose=True,
        auth=BasicAuthCreds(
          username="admin",
          password="supersecretpassword",  # Just an example, don't hardcode this!
        )
    )]
)

On this page

Service
Description
Schema
Properties
Python Examples
Liveliness/Readiness Probe
HttpProbe
Description
Schema
Properties
Python Examples
HealthProbe
Description
Schema
Properties
Python Examples
Port
Port
Description
Schema
Properties
Enumerate Values
Python Examples

Getting Started

Train and Deploy Models

Service Deployment

Job Deployment

Workflow Deployment

Async Service Deployment

Volumes

ML Repository

Secret Management

Access Control

Platform

Deploying On Your Own Cloud

HOW TOs

​Note

​Service

​Description

​Schema

​Properties

​Python Examples

Building using a Python Build pack

Building using a Dockerfile

Using an already existing Image from an Image Registry

Deploying from a Git Repository

​Liveliness/Readiness Probe

​HttpProbe

​Description

​Schema

​Properties

​Python Examples

​HealthProbe

​Description

​Schema

​Properties

​Python Examples

​Port

​Port

​Description

​Schema

​Properties

​Enumerate Values

​Python Examples

Note

Service

Description

Schema

Properties

Python Examples

Liveliness/Readiness Probe

HttpProbe

Description

Schema

Properties

Python Examples

HealthProbe

Description

Schema

Properties

Python Examples

Port

Port

Description

Schema

Properties

Enumerate Values

Python Examples