Service

A Service is a continuously running process that usually has some APIs to interact with it. Since a Service is continuously running, we incur the cost of the service based on resource usage. Services can be scaled up or down depending on the incoming traffic or resource usage.

Usecases

They are used when the incoming request can come at anytime, and the response must be returned almost instantly.

Some examples of services are:

1. Realtime Model Inference Hosting (Flask, FastAPI)
2. Model Inference on incoming streams like Kafka messages
3. A backend to power dynamic websites
4. Model demos like Streamlit, Gradio

Key things to note while building a service

A Service usually exposes APIs at some port. This port can be mapped to a domain, allowing others to call this Service. There can be multiple port exposed by Service and each can be mapped to a different URL.

It's s usually a good idea to have the API documentation while building a Service since it makes it relatively easy for the consumers of your service to understand API inputs and outputs.

If you are writing your Service in FastAPI, the Swagger UI is automatically rendered in the /docs link.