When you create a Service, you need to specify the ports that the Service will listen on. You can decide to expose a port in which case you will have to attach a domain to the service.
If you do not expose a service, you get a URL of the form
servicename-workspacename.svc.cluster.local:port which can only be called from services within the cluster.
This will enable the service to be called from anywhere using the endpoint - other users or services living outside your cluster will also be able to access it. In this case, you will need to map a domain URL to your service. TrueFoundry supports both host-based routing and path-based routing.
If you use host-based routing, two services s1 and s2 will have URLs of the form
In the case of path-based routing, the URLs will be of the form
yourdomain.com/s2. If you decide to enable path-based routing, please ensure the application that you have deployed supports path-based routing.
Domains are configured in the cluster. You can view which domains are accessible to you from here.
Host-based routing will be preferred especially if you are deploying some UI-based applications.
Service class has a
ports argument where you can pass a list of
Port objects. Each
Port object represents a port that the service will listen on. In the
Port field you can specify the
Port Number, host for your
Picking a value for
Providing a host value depends on the base domain urls configured in the cluster settings, you can learn how to find the base domain urls available to you here
For e.g. If your base domain url is
*.truefoundry.your-org.comthen a valid value can be
Alternatively if you have a non wildcard based domain url e.g.
truefoundry.your-org.com, then a valid value can be
import logging from servicefoundry import Build, Service, DockerFileBuild, Port logging.basicConfig(level=logging.INFO) service = Service( name="your-service", image=Build(build_spec=DockerFileBuild()), + ports=[ + Port( + port=8080, + # Uses subdomain based routing + host="fastapi-your-workspace-8080.your-organization.com", + # Alternatively we can use path based routing - E.g. + # host="your-site.your-organization.com", + path="/fastapi/v1/" + ) + ] ) service.deploy(workspace_fqn="YOUR_WORKSPACE_FQN")
Usually, most of the commonly used applications will come with a way to define the port. We have listed it for some of the most commonly used frameworks in ML.
FastAPI (Uvicorn) - Default port is 8000. This can be overridden using the command
uvicorn main:app --port 8001
Gradio App - Launch your Gradio app as follows and enter the port number 8080.
Now you need to specify the same port number in the service configuration. In this case
Streamlit - Default port is 8501.
Ideally when deploying a
Service it is a good idea to protect publicly exposed ports using some form of authentication. You can add a username password based authentication to your endpoints.
We can add basic (username and password) authentication to a Port of the Service in the Python definition using
from servicefoundry import Build, Service, Port, BasicAuthCreds, DockerFileBuild service = Service( name="your-service", image=Build(build_spec=DockerFileBuild()), ports=[ Port( port=8080, host="fastapi-my-workspace-8080.truefoundry.my-org.com", + auth=BasicAuthCreds( + username="hello", + password="pass" + ) ) ] ) service.deploy(workspace_fqn="YOUR_WORKSPACE_FQN")
Upon deploying the service, the endpoint will now display a padlock 🔒 icon, indicating that authentication is required for access.
When attempting to access the endpoint from a web browser, you will be prompted to enter your credentials.
When using Python, you can pass the credentials in the following manner:
import requests endpoint = 'https://fastapi-my-workspace-8080.truefoundry.my-org.com/' response = requests.get(endpoint, auth=('hello', 'pass'))
When using cURL, you'll need to include the credentials as part of the command:
curl -v -X GET --user hello:pass https://fastapi-my-workspace-8080.truefoundry.my-org.com/
Updated 2 days ago