Truefoundry Docs

Custom Guardrails/Plugins are a way to introduce custom “validation” or “mutations” to the request and response of the LLM. You can implement custom security policies, PII detection, content moderation specific to your use case.

Template Repository Overview

The custom guardrails template repository provides a comprehensive FastAPI application with multiple guardrail implementations. It serves as a starting point for building your own custom guardrail server with best practices and example implementations.

Architecture

The template follows a modular architecture:

main.py: FastAPI application with route definitions
guardrail/: Directory containing all guardrail implementations
entities.py: Pydantic models for request/response validation
requirements.txt: Dependencies and libraries

Entities and Data Models

The template defines several Pydantic models that structure the data flow between TrueFoundry AI Gateway and your custom guardrail server.

RequestContext

class SubjectType(str, Enum):
    user = 'user'
    team = 'team'
    serviceaccount = 'serviceaccount'

class Subject(BaseModel):
    subjectId: str
    subjectType: SubjectType
    subjectSlug: Optional[str] = None
    subjectDisplayName: Optional[str] = None

class RequestContext(BaseModel):
    user: Subject
    metadata: Optional[dict[str, str]] = None

RequestContext is a Pydantic model that provides structured contextual information for each request processed by your custom guardrail server. It includes details about the user (as a Subject object) and optional metadata relevant to the request lifecycle. This context is automatically populated by the TrueFoundry AI Gateway and can be leveraged for access control, auditing, or custom logic within your guardrail implementations.

InputGuardrailRequest

class InputGuardrailRequest(BaseModel):
    requestBody: CompletionCreateParams
    context: RequestContext
    config: Optional[dict] = None

InputGuardrailRequest represents the schema for requests sent to the input guardrail endpoint. It encapsulates the original model input (requestBody), which is OpenAI-compatible and follows the schema from the official OpenAI repository, along with configuration options (config) and contextual information (context) about the request.

OutputGuardrailRequest

class OutputGuardrailRequest(BaseModel):
    requestBody: CompletionCreateParams
    responseBody: ChatCompletion
    config: Optional[dict] = None
    context: RequestContext

OutputGuardrailRequest represents the schema for requests sent to the output guardrail endpoint. It encapsulates the original model input (requestBody), the model’s output (responseBody), configuration options (config), and contextual information (context) about the request. Both requestBody and responseBody are OpenAI-compatible and follow the schemas from the official OpenAI repository.

Available Guardrails

The template repository includes five pre-implemented guardrails that demonstrate different validation and transformation techniques.

1. PII Redaction (Presidio)

Info

Code Snippet

from presidio_analyzer import AnalyzerEngine
from presidio_anonymizer import AnonymizerEngine

# Initialize Presidio
analyzer = AnalyzerEngine()
anonymizer = AnonymizerEngine()

def process_input_guardrail(request: InputGuardrailRequest) -> Optional[dict]:
    messages = request.requestBody.get('messages', [])
    for message in messages:
        # Analyze and anonymize each message
        results = analyzer.analyze(text=message["content"], entities=[], language='en')
        anonymized_content = anonymizer.anonymize(text=message["content"], analyzer_results=results)
        # ... update message content as needed
    # Return modified request if any PII was redacted, else None

Response Behavior

2. NSFW Filtering (Local Model)

Info

Code Snippet

from transformers import pipeline

classifier = pipeline("text-classification", model="unitary/unbiased-toxic-roberta")

def nsfw_filtering(request: OutputGuardrailRequest) -> Optional[dict]:
    transformed_body = request.responseBody.copy()
    for choice in transformed_body.get("choices", []):
        classification_results = classifier(choice["message"]["content"])
        for result in classification_results:
            if (
                (result['label'] == 'toxicity' and result['score'] > 0.2) or
                (result['label'] == 'sexual_explicit' and result['score'] > 0.2) or
                (result['label'] == 'obscene' and result['score'] > 0.2)
            ):
                raise HTTPException(
                    status_code=400, 
                    detail=f"This message is not allowed as it is NSFW"
                )
    return None

Response Behavior

3. Drug Mention Detection (Guardrails AI)

Info

Code Snippet

from guardrails import Guard
from guardrails.hub import MentionsDrugs

guard = Guard().use(MentionsDrugs, on_fail="exception")

def drug_mention(request: OutputGuardrailRequest) -> Optional[dict]:
    try:
        for choice in request.responseBody.get("choices", []):
            guard.validate(choice["message"]["content"])
    except Exception as e:
        raise HTTPException(status_code=400, detail=str(e))
    return None

Response Behavior

4. Web Sanitization (Guardrails AI)

Info

Code Snippet

from guardrails import Guard
from guardrails_grhub_web_sanitization import WebSanitization

guard = Guard().use(WebSanitization, on_fail="exception")

def web_sanitization(request: InputGuardrailRequest) -> Optional[dict]:
    try:
        messages = request.requestBody.get("messages", [])
        for message in messages:
            guard.validate(message["content"])
    except Exception as e:
        raise HTTPException(status_code=400, detail=str(e))
    return None

Response Behavior

5. PII Detection (Guardrails AI)

Info

Code Snippet

from guardrails import Guard
from guardrails.hub import DetectPII

guard = Guard().use(DetectPII, on_fail="exception")

def pii_detection_guardrails_ai(request: InputGuardrailRequest) -> Optional[dict]:
    try:
        messages = request.requestBody.get("messages", [])
        for message in messages:
            guard.validate(message["content"])
    except Exception as e:
        raise HTTPException(status_code=400, detail=str(e))
    return None

Response Behavior

Request Examples

Input Guardrail Request

{
  "requestBody": {
    "messages": [
      {
        "role": "user",
        "content": "Hello, my name is John Doe and my email is john.doe@example.com"
      }
    ],
    "model": "gpt-3.5-turbo",
    "temperature": 0.7
  },
  "config": {
    "check_content": true,
    "transform_input": true
  },
  "context": {
    "user": {
      "subjectId": "123",
      "subjectType": "user",
      "subjectSlug": "john_doe@truefoundry.com",
      "subjectDisplayName": "John Doe"
    },
    "metadata": {
      "ip_address": "192.168.1.1",
      "session_id": "abc123"
    }
  }
}

Output Guardrail Request

{
  "requestBody": {
    "messages": [
      {
        "role": "user",
        "content": "Hello"
      }
    ],
    "model": "gpt-3.5-turbo"
  },
  "responseBody": {
    "id": "chatcmpl-123",
    "object": "chat.completion",
    "created": 1677652288,
    "model": "gpt-3.5-turbo",
    "choices": [
      {
        "index": 0,
        "message": {
          "role": "assistant",
          "content": "Hello! How can I help you today?"
        },
        "finish_reason": "stop"
      }
    ]
  },
  "config": {
    "check_content": true
  },
  "context": {
    "user": {
      "subjectId": "123",
      "subjectType": "user",
      "subjectSlug": "john_doe@truefoundry.com",
      "subjectDisplayName": "John Doe"
    },
    "metadata": {
      "ip_address": "192.168.1.1",
      "session_id": "abc123"
    }
  }
}

Running Locally

# Install dependencies
pip install -r requirements.txt

# Run the server
python main.py

# Or using uvicorn directly
uvicorn main:app --host 0.0.0.0 --port 8000 --reload

Adding Custom Guardrail Integration

To add Custom Guardrail to your TrueFoundry setup, follow these steps:

Navigate to AI Gateway
- Go to AI Gateway in your TrueFoundry dashboard.
Access Guardrails
- Click on Guardrails.
Add New Guardrails Group
- Click on Add New Guardrails Group.

Navigate to Guardrails

Guardrails groups help manage access control and security policies for your LLM applications. Configure rules to prevent harmful content, ensure compliance, and maintain data privacy. For more details, refer to the Collaborator Section.

Fill in the Guardrails Group Form
- Name: Enter a name for your guardrails group.
- Collaborators: Add collaborators who will have access to this group.
- Custom Guardrail Config:
  - Name: Enter a name for the Custom Guardrail configuration.
  - Operation: The operation type to use for the guardrail.
    - Validate: Guardrails with this operation are used to validate requests. These guardrails are run in parallel.
    - Mutate: Guardrails with this operation can both validate and mutate requests. Mutate guardrails are run sequentially.
  - URL: Enter the URL for the Guardrail Server.
  - Auth Data: Provide authentication data for the Guardrail Server. This data will be sent to the Guardrail Server for authorization.
    - Choose between Custom Basic Auth or Custom Bearer Auth.
  - Headers (Optional): Add any headers required for the Guardrail Server. These will be forwarded as is.
  - Config: Enter the configuration for the Guardrail Server. This is a JSON object that will be sent along with the request.

Custom Guardrail configuration form in dashboard

Fill in the Custom Guardrail Form

Get Started

Developer Guide

MCP Registry and Gateway

Observability

Integrations

Deployment

API Reference

Chat

Agent

MCP

Embeddings

Rerank

Responses

Image

Audio

Batch

Files

Moderations

Custom Guardrail/Plugins Configuration

Template Repository Overview

Architecture

Entities and Data Models

RequestContext

InputGuardrailRequest

OutputGuardrailRequest

Available Guardrails

Request Examples

Running Locally

Adding Custom Guardrail Integration

Get Started

Developer Guide

MCP Registry and Gateway

Observability

Integrations

Deployment

API Reference

Chat

Agent

MCP

Embeddings

Rerank

Responses

Image

Audio

Batch

Files

Moderations

​Template Repository Overview

​Architecture

​Entities and Data Models

​RequestContext

​InputGuardrailRequest

​OutputGuardrailRequest

​Available Guardrails

​Request Examples

​Running Locally

​Adding Custom Guardrail Integration

Template Repository Overview

Architecture

Entities and Data Models

RequestContext

InputGuardrailRequest

OutputGuardrailRequest

Available Guardrails

Request Examples

Running Locally

Adding Custom Guardrail Integration