The chat completions API supports structured response formats through the response_format parameter, enabling you to receive consistent, predictable outputs in JSON format. This guide covers both basic JSON mode and advanced JSON schema validation with Pydantic integration.

​
Overview

There are two primary approaches for structured responses:

  1. JSON Mode: Basic JSON formatting without schema validation
  2. JSON Schema Mode: Structured responses with strict schema validation and Pydantic parsing

​
JSON Mode

JSON mode ensures the model’s output is valid JSON without enforcing a specific structure.

​
Basic JSON Mode Usage

from openai import OpenAI

client = OpenAI(
    api_key="your_truefoundry_api_key",
    base_url="<truefoundry-base-url>/api/llm/api/inference/openai"
)

response = client.chat.completions.create(
    model="openai-main/gpt-4o",
    messages=[
        {"role": "system", "content": "You are a helpful assistant designed to output JSON."},
        {"role": "user", "content": "Extract information about the 2020 World Series winner"}
    ],
    response_format={"type": "json_object"}
)

print(response.choices[0].message.content)

Output:

{
  "team": "Los Angeles Dodgers",
  "year": 2020,
  "opponent": "Tampa Bay Rays",
  "games_played": 6,
  "series_result": "4-2"
}

​
JSON Schema Mode

JSON Schema mode provides strict structure validation and type safety using predefined schemas.

​
Method 1: OpenAI Client with JSON Schema

​
Basic Schema Definition

from openai import OpenAI
import json

client = OpenAI(
    api_key="your_truefoundry_api_key",
    base_url="<truefoundry-base-url>/api/llm/api/inference/openai"
)


# Define JSON schema
user_info_schema = {
    "type": "object",
    "properties": {
        "name": {"type": "string"},
        "age": {"type": "integer", "minimum": 0},
        "occupation": {"type": "string"},
        "location": {"type": "string"},
        "skills": {
            "type": "array",
            "items": {"type": "string"}
        }
    },
    "required": ["name", "age", "occupation", "location", "skills"],
    "additionalProperties": False
}

response = client.chat.completions.create(
    model="openai-main/gpt-4o",
    messages=[
        {
            "role": "system", 
            "content": "Extract user information and respond according to the provided JSON schema."
        },
        {
            "role": "user", 
            "content": "My name is Sarah Johnson, I'm 28 years old, and I work as a data scientist in New York. I'm skilled in Python, SQL, and machine learning."
        }
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "user_info",
            "schema": user_info_schema,
            "strict": True
        }
    }
)

# Parse response
result = json.loads(response.choices[0].message.content)

Output:

{
  "name": "Sarah Johnson",
  "age": 28,
  "occupation": "data scientist",
  "location": "New York",
  "skills": ["Python", "SQL", "machine learning"]
}

When using OpenAI’s response_format with JSON schema and strict mode is set to true, all properties defined in the schema must be included in the required array. If any property is defined but not marked as required, the API will return a 400 Bad Request Error”.

​
Method 2: OpenAI Client with Pydantic Parsing

Pydantic provides automatic validation, serialization, and type hints for structured data.

​
Installation

pip install pydantic openai

​
Basic Pydantic Integration

from openai import OpenAI
from pydantic import BaseModel, Field
from typing import List

client = OpenAI(
    api_key="your_truefoundry_api_key",
    base_url="<truefoundry-base-url>/api/llm/api/inference/openai"
)

# Define Pydantic model
class UserInfo(BaseModel):
    name: str = Field(description="Full name of the user")
    age: int = Field(ge=0, description="Age in years")
    occupation: str = Field(description="Job title or profession")
    location: str = Field(description="City or location")
    skills: List[str] = Field(description="List of professional skills")
    
    class Config:
        extra = "forbid"  # Prevent additional fields


response = client.chat.completions.create(
    model="openai-main/gpt-4o",
    messages=[
        {
            "role": "system",
            "content": "Extract user information and respond according to the provided schema."
        },
        {
            "role": "user",
            "content": "Hi, I'm Mike Chen, a 32-year-old software architect from Seattle. I specialize in cloud computing, microservices, and Kubernetes."
        }
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "user_info",
            "schema": UserInfo.model_json_schema(),
            "strict": True
        }
    }
)

# Parse and validate with Pydantic
raw_content = response.choices[0].message.content
user_data = UserInfo.model_validate_json(raw_content)

print(f"Name: {user_data.name}")
print(f"Age: {user_data.age}")
print(f"Skills: {', '.join(user_data.skills)}")

When using OpenAI models with Pydantic Models, there should not be any optional fields in the pydantic model. This limitation exists because the corresponding JSON schema will then have missing field in the “required” section and this is NOT allowed when strict mode is true For models with optional fields, consider using the beta parse client detailed in the following section.

​
Method 3: OpenAI Client’s Beta Parse API

The beta parse client provides the most streamlined approach for Pydantic integration, automatically handling schema conversion and parsing.

from openai import OpenAI
from pydantic import BaseModel, Field
from typing import List, Optional

class UserInfo(BaseModel):
    name: str = Field(description="Full name of the user")
    age: int = Field(ge=0, description="Age in years")
    occupation: str = Field(description="Job title or profession")
    location: Optional[str] = Field(None, description="City or location")
    skills: List[str] = Field(default=[], description="List of professional skills")

client = OpenAI(
    api_key="your_truefoundry_api_key",
    base_url="<truefoundry-base-url>/api/llm/api/inference/openai"
)

completion = client.beta.chat.completions.parse(
    model="openai-main/gpt-4o",
    messages=[
        {
            "role": "system",
            "content": "Extract user information from the provided text."
        },
        {
            "role": "user",
            "content": "Hello, I'm Alex Rodriguez, a 29-year-old product manager from Austin. I have experience in agile methodologies, data analysis, and team leadership."
        }
    ],
    response_format=UserInfo,
)

user_result = completion.choices[0].message.parsed

print(f"Name: {user_result.name}")
print(f"Age: {user_result.age}")
print(f"Occupation: {user_result.occupation}")
print(f"Location: {user_result.location}")
print(f"Skills: {', '.join(user_result.skills)}")