Cost Tracking

TrueFoundry AI Gateway supports two types of cost tracking for models:

1. Public Cost: Pre-configured pricing based on provider’s published rates for commonly used models
2. Private Cost: Custom pricing that can be manually configured for specialized models or custom rates (e.g. custom pricing contracts with providers) or for fine-tuned models

​

Public Cost

Public cost refers to the pre-configured pricing information available for commonly used models. This is provided on a best-effort basis and may not be available for all models. When you select a model that has public cost enabled:

• The cost per token is automatically populated based on Provider’s published pricing
• Pricing is kept up-to-date with the provider’s official rates
• No manual configuration needed for supported models

This feature simplifies cost management for standard models from providers like OpenAI, Anthropic, and AWS Bedrock.

When adding a model through the + Add Model form or editing an existing model, you’ll see an option to enable public cost if it’s available for that specific model.

​

Private Cost

For models where we are not able to support public pricing or you need custom pricing (e.g. custom pricing contracts with providers) or for fine-tuned models:

• You can manually configure cost per token for input and output
• Track costs for private or specialized models

To set up private cost:

1. When adding a new model, select Private Cost Metrics
2. Enter your custom input and output token rates
3. Save to apply your custom cost settings

To update private cost for an existing model:

1. Click the “Edit” button next to the model
2. Select Private Cost Metrics
3. Update your custom input and output token rates
4. Save to apply the new cost settings

This flexibility allows you to accurately track costs for any model deployment scenario.