gateway-rate-limiting-config
. It helps TrueFoundry identify that this is a rate limiting configuration file.{user}
, {model}
which will be replaced by actual user or model in the request.
{user}-daily-limit
, the system will create a separate rule for each user (for example, alice-daily-limit, bob-daily-limit) and apply the limit individually to each one.X-TFY-METADATA
. The subjects, models and metadata fields are conditioned in an AND fashion - meaning that the rule will only match if all the conditions are met. If an incoming request doesn’t match the when block in one rule, the next rule will be evaluated.
subjects
: Filter based on the list of users / teams / virtual accounts calling the model. User can be specified using user:john-doe
or team:engineering-team
or virtual-account:acct_1234567890
.models
: Rule matches if the model name in the request matches any of the models in the list.metadata
: Rule matches if the metadata in the request matches the metadata in the rule. For e.g. if we specify metadata: {environment: "production"}
, the rule will only match if the request has the metadata key environment
with value production
in the request header X-TFY-METADATA
.