- Enhance Observability: Filter requests to get specific insights and create custom metrics pages for advanced monitoring.
- Apply Conditional Configurations: Use metadata in the
when
block of your gateway configurations to selectively apply rules for rate limiting, model fallbacks, load balancing, and more.
How to Log Custom Metadata
The custom metadata is logged as key value pairs in theX-TFY-METADATA
header. Here’s an example of how to log custom metadata:
You can include multiple metadata keys with each request. Please note that all values must be strings, with a maximum length of 128 characters.
Using Metadata for Observability
Metadata keys are instrumental for monitoring and debugging. You can use them to filter your request logs and build custom metric dashboards.Filter Logs
Filter your logs using one or more metadata keys to quickly isolate specific requests. This is useful for debugging or analyzing usage patterns for a particular feature, environment, or user.
Filtering requests using custom Metadata Keys
Create Custom Metrics
You can also group metrics by metadata keys to create custom visualizations. For example, you can monitor cost and usage per customer by grouping with acustomer_id
key.

Grouping metrics by a custom 'customer_id' metadata key.
Using Metadata in Gateway Configurations
Metadata is powerful for creating conditional rules in your AI Gateway. You can use themetadata
field inside a when
block to apply configurations only when certain metadata is present in the request.
Here are a few examples:
Rate limit
You can rate limit your requests based on metadata. For example, tolimit all requests to the gpt-4 model from the dev environment to 1000 requests per day
.
customer-id
or set up specific fallback models for a feature
.
Learn more in the Load Balancing and Fallback documentation.