Services

A Service represents a persistent and continuously running applications that typically provides a set of APIs for interaction. As Services run continuously, costs are incurred based on resource utilization. Services can be dynamically scaled based on incoming traffic or resource demands.

Use Cases

Services are ideal when requests can arrive at any time and prompt responses are crucial. Some typical use cases for services include:

1. Hosting Realtime Model Inference (e.g., Flask, FastAPI)
2. Performing Model Inference on Incoming Data Streams (e.g., Kafka messages)
3. Powering Dynamic Backend for Websites
4. Demonstrating Models (e.g., Streamlit, Gradio)

Key Considerations when Building a Service

A Service typically exposes APIs on designated ports. These ports can be mapped to domains, enabling external calls to the Service. Multiple ports can be exposed by a Service, each possibly mapped to a different URL.