Custom Guardrail/Plugins Configuration

Custom Guardrails/Plugins are a way to introduce custom “validation” or “mutations” to the request and response of the LLM. You can implement custom security policies, PII detection, content moderation specific to your use case.

​

Adding Custom Guardrail Integration

To add Custom Guardrail to your TrueFoundry setup, follow these steps:

1. Navigate to AI Gateway

   • Go to AI Gateway in your TrueFoundry dashboard.

2. Access Guardrails

   • Click on Guardrails.

3. Add New Guardrails Group

   • Click on Add New Guardrails Group.

Guardrails groups help manage access control and security policies for your LLM applications. Configure rules to prevent harmful content, ensure compliance, and maintain data privacy. For more details, refer to the Collaborator Section.

4. Fill in the Guardrails Group Form
   • Name: Enter a name for your guardrails group.
   • Collaborators: Add collaborators who will have access to this group.
   • Custom Guardrail Config:
     • Name: Enter a name for the Custom Guardrail configuration.
     • Operation: The operation type to use for the guardrail.
       • Validate: Guardrails with this operation are used to validate requests. These guardrails are run in parallel.
       • Mutate: Guardrails with this operation can both validate and mutate requests. Mutate guardrails are run sequentially.
     • URL: Enter the URL for the Guardrail Server.
     • Auth Data: Provide authentication data for the Guardrail Server. This data will be sent to the Guardrail Server for authorization.
       • Choose between Custom Basic Auth or Custom Bearer Auth.
     • Headers (Optional): Add any headers required for the Guardrail Server. These will be forwarded as is.
     • Config: Enter the configuration for the Guardrail Server. This is a JSON object that will be sent along with the request.

By following these steps, you can set up a custom guardrail in TrueFoundry, ensuring your applications adhere to specific security and compliance requirements.