Access Control
Centralised Key Management
Distributing your core OpenAI or other provider keys to all developers is a big concern from a security standpoint. The LLM Gateway allows you to add all the keys centrally and each developer/product/service gets their own API key to interact with the models.
This keeps complete accountability of who is using the models without sacrificing the security of the root keys. The gateway can read the keys from your Secret Manager like AWS SSM, Google Secret Store or Azure Vault. You can also revoke permissions dynamically from users or products without affecting other users since everyone gets their own API keys.
In case you don't want to handover the API keys to developers, you can use a simple client side library to make the calls to the Gateway that automatically handles authentication for you using OAuth/OIDC connect without you having to manually copy paste keys. This provides enhanced security by issuing short lived tokens and automatically refreshing them.
Authentication and Authorization
You can add models to LLM Gateway by adding provider accounts like OpenAI, Anthropic, Bedrock etc through through the Integrations page. Each model provider can have multiple models within and you can configure access control at the model level.
Adding access control through the UI
While adding a model as per the steps in the previous docs, you can configure the users and teams within the organisation who can access this model.
GitOps using YAML spec
If you are configuring TrueFoundry using GitOps, then you can use the YAML spec of provider account to configure access control at the model level. For example, configuring access control on Claude 3 above would mean your provider account spec would look like below. Here, users alice
, bob
and anyone who's part of dev-team
can consume the models through the Gateway.
name: my-anthropic-account
type: provider-account/anthropic
auth_data:
api_key: xxxx
type: api-key
integrations:
- name: claude-3-test
type: integration/model/anthropic
model_id: claude-3-haiku-20240307
model_types:
- chat
authorized_subjects:
- user:alice
- user:bob
- team:dev-team
Updated 29 days ago