To get started with integrating and using Guardrails, follow the steps below:
1

Create a Guardrails Group

A guardrails group serves as a container for multiple guardrail integrations. You can assign who can add/edit/remove guardrails in a group and who can access the guardrails in a group. There are two type of roles for collaborators:
  • Manager: Can create, edit, delete and use the guardrail integrations in the group.
  • User: Can use the guardrail integrations in the group.
To create a guardrails group, navigate to the AI Gateway -> Guardrail. Give your group a name. In our case, we’ve named it openai-guardrail.

Create a Guardrails Group

Give your group a name

A Guardrail Group is the minimum level of access control for guardrails.A common usage pattern is to create an org-wide global guardrails group which contains the organization wide guardrails. The Platform team is the manager and the entire org are users of the guardrail.For team/product specific guardrails, its better to create individual guardrail groups per product / team.
2

Add Guardrail Integration in Guardrails Group

You can add integrations to your guardrails for content moderation and safety. Let’s create an OpenAI guardrail integration, which we will later use as an output guardrail in our configuration example. This will allow us to automatically moderate and filter responses from the LLM using OpenAI’s moderation capabilities.
  1. Select the OpenAI Moderation from the list of integrations.
    Guardrail integration selection interface showing OpenAI Moderation option

    Select the OpenAI Moderation Integration

  2. Fill in the OpenAI Moderation Guardrail Config form. For this tutorial, we’ve named it tfy-openai-moderation.
    OpenAI Moderation configuration form with fields for name, API key, and operation type

    Fill in the OpenAI Moderation Form

  3. Save the guardrail group.
You are now ready to use this OpenAI guardrail integration in your guardrail configuration to moderate LLM inputs. This means the guardrail will be applied to the input prompts before they are sent to the LLM, helping to ensure that any unsafe or non-compliant content is detected and handled prior to model processing.
To customize moderation sensitivity for specific categories such as harassment, sexual, or hate, enable the Category Thresholds option. You can then adjust the threshold values for each category according to your requirements.
3

Test Guardrails in Playground

  1. Navigate to the Playground
    Navigate to the AI Gateway -> Playground Tab.
  2. Select Guardrails
    On the left side, you’ll see two options: Input Guardrails and Output Guardrails.
    • Click on either “Input Guardrails” or “Output Guardrails” depending on where you want to apply the guardrail.
    • Add the guardrail you want to apply by clicking on the option and selecting the desired guardrail from the list.
    As we have already created an OpenAI guardrail named tfy-openai-moderation in the previous steps, select this guardrail under Input Guardrails.
    AI Gateway playground interface with guardrail selection panel on the left side

    Select Guardrails

  3. Test the Guardrail
    • Enter a prompt that would typically be flagged as unsafe or offensive.
    • Send the request to the model.
    You should see that the prompt is blocked by the guardrail, demonstrating that your configuration is working as expected.
Testing in the Playground is an easy way to validate that your guardrails are correctly configured before deploying them to production.
4

Trigger Guardrails in LLM requests from code

You can pass the guardrails to be associated as headers at a per request level by providing the X-TFY-GUARDRAILS header in the request. You can copy the relevant code snippet from the Playground section.
  1. Select the Guardrail to Apply
    On the left side, choose the Input Guardrails and Output Guardrails section, and select the guardrail you want to apply to your request. For this tutorial, we’ve selected the tfy-openai-moderation guardrail for both input and output.
    Guardrail selector in playground showing selected OpenAI moderation guardrail

    Select the Guardrail to Apply

  2. Get the Code Snippet
    After configuring your request and selecting the desired guardrail(s), click the Code button at the top right of the Playground.
    In the code snippet section, you will see ready-to-use examples for different SDKs and curl.
    Note: The generated code will automatically include the necessary X-TFY-GUARDRAILS header with your selected guardrails.
    Code snippet section showing language options for implementing guardrails

    Code Snippet Section

  3. Copy and Run the Code
    Copy the generated curl command (or code for your preferred SDK) and run it.
    The guardrails you selected in the Playground will be applied automatically, as reflected in the request headers.
    Curl command example with X-TFY-GUARDRAILS header for applying guardrails

    Copy the code snippet

You can use this approach with any HTTP client or SDK by adding the X-TFY-GUARDRAILS header to your request. This allows you to dynamically apply guardrails per request without changing your global configuration.
5

Configure application of guardrails at gateway layer

  1. Navigate to the Guardrails Section in Config
    Go to the AI Gateway dashboard, then select the Config tab. In the left sidebar, click on the Guardrails section.
  2. Create a Guardrail Configuration
    Click on Create/Edit Guardrail Config. Fill in the required details for your guardrail configuration, such as rules, input guardrails, and output guardrails.
  3. Save the Configuration
    After filling out the form, click Save to apply your guardrail configuration.
Gateway-level guardrail configuration interface with YAML editor for rules

Configure application of guardrails at gateway layer

For more details on how to configure guardrails, see the Guardrails Configuration Guide.
Configuring guardrails at the gateway layer is recommended for organization-wide enforcement. This centralizes guardrail management and auditing, eliminating the need to set headers on every request.