Custom Guardrails/Plugins are a way to introduce custom “validation” or “mutations” to the request and response of the LLM. You can implement custom security policies, PII detection, content moderation specific to your use case.
We provide a template repository that you can use as a starting point to build your own custom guardrail server. This template includes best practices and example implementations to help you get started quickly.

​
Adding Custom Guardrail Integration

To add Custom Guardrail to your TrueFoundry setup, follow these steps:
  1. Navigate to AI Gateway
    • Go to AI Gateway in your TrueFoundry dashboard.
  2. Access Guardrails
    • Click on Guardrails.
  3. Add New Guardrails Group
    • Click on Add New Guardrails Group.

Navigate to Guardrails

Guardrails groups help manage access control and security policies for your LLM applications. Configure rules to prevent harmful content, ensure compliance, and maintain data privacy. For more details, refer to the Collaborator Section.
  1. Fill in the Guardrails Group Form
    • Name: Enter a name for your guardrails group.
    • Collaborators: Add collaborators who will have access to this group.
    • Custom Guardrail Config:
      • Name: Enter a name for the Custom Guardrail configuration.
      • Operation: The operation type to use for the guardrail.
        • Validate: Guardrails with this operation are used to validate requests. These guardrails are run in parallel.
        • Mutate: Guardrails with this operation can both validate and mutate requests. Mutate guardrails are run sequentially.
      • URL: Enter the URL for the Guardrail Server.
      • Auth Data: Provide authentication data for the Guardrail Server. This data will be sent to the Guardrail Server for authorization.
        • Choose between Custom Basic Auth or Custom Bearer Auth.
      • Headers (Optional): Add any headers required for the Guardrail Server. These will be forwarded as is.
      • Config: Enter the configuration for the Guardrail Server. This is a JSON object that will be sent along with the request.

Fill in the Custom Guardrail Form

By following these steps, you can set up a custom guardrail in TrueFoundry, ensuring your applications adhere to specific security and compliance requirements.

​
Template Repository Overview

The custom guardrails template repository provides a comprehensive FastAPI application with multiple guardrail implementations. It serves as a starting point for building your own custom guardrail server with best practices and example implementations.

​
Architecture

The template follows a modular architecture:
  • main.py: FastAPI application with route definitions
  • guardrail/: Directory containing all guardrail implementations
  • entities.py: Pydantic models for request/response validation
  • requirements.txt: Dependencies and libraries

​
Entities and Data Models

The template defines several Pydantic models that structure the data flow between TrueFoundry AI Gateway and your custom guardrail server.

​
RequestContext

class SubjectType(str, Enum):
    user = 'user'
    team = 'team'
    serviceaccount = 'serviceaccount'

class Subject(BaseModel):
    subjectId: str
    subjectType: SubjectType
    subjectSlug: Optional[str] = None
    subjectDisplayName: Optional[str] = None

class RequestContext(BaseModel):
    user: Subject
    metadata: Optional[dict[str, str]] = None
RequestContext is a Pydantic model that provides structured contextual information for each request processed by your custom guardrail server. It includes details about the user (as a Subject object) and optional metadata relevant to the request lifecycle. This context is automatically populated by the TrueFoundry AI Gateway and can be leveraged for access control, auditing, or custom logic within your guardrail implementations.

​
InputGuardrailRequest

class InputGuardrailRequest(BaseModel):
    requestBody: CompletionCreateParams
    context: RequestContext
    config: Optional[dict] = None
InputGuardrailRequest represents the schema for requests sent to the input guardrail endpoint. It encapsulates the original model input (requestBody), which is OpenAI-compatible and follows the schema from the official OpenAI repository, along with configuration options (config) and contextual information (context) about the request.

​
OutputGuardrailRequest

class OutputGuardrailRequest(BaseModel):
    requestBody: CompletionCreateParams
    responseBody: ChatCompletion
    config: Optional[dict] = None
    context: RequestContext
OutputGuardrailRequest represents the schema for requests sent to the output guardrail endpoint. It encapsulates the original model input (requestBody), the model’s output (responseBody), configuration options (config), and contextual information (context) about the request. Both requestBody and responseBody are OpenAI-compatible and follow the schemas from the official OpenAI repository.

​
Available Guardrails

The template repository includes five pre-implemented guardrails that demonstrate different validation and transformation techniques.

​
Request Examples

​
Running Locally

# Install dependencies
pip install -r requirements.txt

# Run the server
python main.py

# Or using uvicorn directly
uvicorn main:app --host 0.0.0.0 --port 8000 --reload