Overview

Guardrails in the AI Gateway provide a mechanism to ensure safety, quality, and compliance by validating and transforming inputs and outputs of Large Language Models (LLMs). This document outlines the configuration and application of guardrails within the TrueFoundry AI Gateway.

​

Overview

Guardrails allow you to define rules that are applied to requests and responses processed by the AI Gateway. These rules can be used to mask personally identifiable information (PII), filter topics, and enforce content standards.

​

Configuration

Guardrails are configured using a YAML file that specifies the rules to be applied. Each rule can include input and output guardrails, which define the actions to be taken on the data. To know more about guardrails, please refer to the Configure Guardrails.

​

Use Cases

Guardrails can be used to:

• Mask PII in inputs and outputs.
• Filter out responses containing certain topics.
• Enforce compliance with content standards.

​

Integrations

The TrueFoundry AI Gateway supports various integrations to enhance the functionality and flexibility of guardrails:

• OpenAI Moderations: Integrate with OpenAI’s moderation tools to automatically detect and handle content that may violate usage policies, such as violence, hate speech, or harassment.

• AWS Bedrock Guardrails: Utilize AWS Bedrock’s capabilities to apply guardrails on AI models, ensuring compliance and safety in AI interactions.

• Custom Guardrails Integrations: The AI Gateway allows for custom integrations, enabling organizations to define and implement their own guardrails tailored to specific needs and requirements.

These integrations provide a robust framework for managing and enforcing content standards across different AI models and platforms.