Access Control

Centralised Key Management

Distributing your core OpenAI or other provider keys to all developers is a big concern from a security standpoint. The AI Gateway allows you to add all the keys centrally and each developer/product gets their own API key to interact with the models. This keeps complete accountability of who is using the models without sacrificing the security of the root keys. The gateway can read the keys from your Secret Manager like AWS SSM, Google Secret Store or Azure Vault. You can also revoke permissions dynamically from users or products without affecting other users since everyone gets their own API keys.

In case you don't want to handover the API keys to developers, you can use a simple client side library to make the calls to the Gateway that automatically handles authentication for you using OAuth/OIDC connect without you having to manually copy paste keys. This provides enhanced security by issuing short lived tokens and automatically refreshing them.

Authentication and Authorization

Truefoundry AI Gateway provides a concept of ProviderAccounts which allows us to add different providers and enable or disable some models within them. You can add users or services to have access to certain provider accounts. For example, in the picture below, ProviderAccount4 and ProviderAccount5 are both AzureOpenAI providers but ProviderAccount4 is for dev environment and ProviderAccount5 is for production. We can grant User1, User2 access to ProviderAccount4 and LLMApp1 access to ProviderAccount5.

Authorization Configuration

The authorization configuration is a YAML with the following shape:

authz_rules:
  - models:
      - truefoundry-self-hosted/llama-2-7b-chat-hf-10707
      - openai-main/text-embedding-ada-002
    users:
      - username1
			- username2

The models property is an array of Fully Qualified Names (FQNs) of models. Each FQN comprises the provider account and the model name, separated by a /.

The users property is an array of usernames. This will decide which users can access the models defined in the models property.

You can have multiple rules.