Advanced Options
Advanced configurations
Configuring resource (CPU/Memory) requirements
You can customize resource (CPU/Memory) request and limit the requirements using the resources
argument in FunctionService
class constructor.
from servicefoundry.function_service import FunctionService
from servicefoundry import Resources
service = FunctionService(
name="t5-small",
resources=Resources(
cpu_request=0.2,
cpu_limit=0.5,
memory_request=128,
memory_limit=512,
),
)
...
Read more about resources here.
Locally running the defined service
You can run the defined service locally using the run
method defined in FunctionService
class.
Using the example here
import logging
from servicefoundry.function_service import FunctionService
from module import normal, uniform
logging.basicConfig(level=logging.INFO)
service = FunctionService(name="func-service")
service.register_function(normal)
service.register_function(uniform)
service.run().join()
Note that,
run()
returns an instance ofthreading.Thread
. We call thejoin()
method so that we wait for the daemon thread's lifetime. To stop the webserver, pressCtrl-C
.
Environment Variables
You can inject environment variables using the env
argument in FunctionService
class constructor.
You can access the value of the environment variables using os.getenv
in the deployed function or class.
from servicefoundry.function_service import FunctionService
from servicefoundry import Resources
service = FunctionService(
name="t5-small",
env={"MODEL_FQN": "t5-small"}
)
...
Updated 19 days ago