Component | Used For | Description |
---|---|---|
Dashboard | Essential | This is the UI component to view deployments, and other resources. |
Backend Service | Essential | Truefoundry comprises of multiple backend services that handle various aspects like authorization, deployment control flow, CRUD APIs, interaction with database and external services, etc. |
PostgreSQL | Essential | Database to store user information, deployment information, etc. This can be deployed on Kubernetes in dev-mode, however, we recommend using managed database like RDS in production. |
Controller | Essential | The controller is responsible for handling all connections from the multiple tfy-agent components in different compute-plane clusters. |
Queue | Essential | We use NATS as queueing and caching layer to be able to process requests and logging from the LLM Gateway. |
Image Builder | Deployment Only | We build docker images on the Kubernetes cluster using buildkit for our deployments. |
AI Gateway | AI Gateway Only | AI Gateway to unify the request and response format for all LLM providers. |
Clickhouse | AI Gateway Only | Database to store request logs and metrics of AI Gateway. |
Component | Description |
---|---|
Blob Storage | Control-plane needs access to one blob storage to store the code uploaded for building the docker image. This can be backed by AWS S3, Azure Blob Storage, GCS, etc. |
Secret Store | Secret store to store the secrets for the deployment. This can be backed by AWS SSM, Azure Vault, GCS, etc. |
Docker registry | Docker registry to store the docker images for the deployment. This can be backed by AWS ECR, Azure Container Registry, GCR, etc. |
Component | CPU | Memory | Storage | Min Nodes | Remarks |
---|---|---|---|---|---|
Helm-Chart (AI Deployment + AI Gateway) | 2 vCPU | 8GB | 60GB Persistent Volumes (Block Storage) On Kubernetes | 2 Pods should be spread over min 2 nodes | Cost: ~ $120 pm |
Helm-Chart (AI Deployment Only) | 1 vCPU | 4GB | 50GB Persistent Volumes (Block Storage) On Kubernetes | 2 Pods should be spread over min 2 nodes | Cost: ~ $60 pm |
Helm-Chart (AI Gateway Only) | 2 vCPU | 8GB | 60GB Persistent Volumes (Block Storage) On Kubernetes | 2 Pods should be spread over min 2 nodes | Cost: ~ $120 pm |
Postgres (Deployed on Kubernetes) | 0.5 vCPU | 0.5GB | 5GB Persistent Volumes (Block Storage) On Kubernetes | PostgreSQL version >= 13 | |
Blob Storage (S3 Compatible) | 20GB |