​
Supported Providers

Vertex AI

OpenAI

TrueFoundry supports File APIs compatible with OpenAI and Vertex AI. This guide demonstrates how to set up the client and use file-related endpoints.

Setup OpenAI client first -

from openai import OpenAI

BASE_URL = "https://{controlPlaneUrl}/api/llm"
API_KEY = "your-truefoundry-api-key"

# Configure OpenAI client with TrueFoundry settings
client = OpenAI(
    api_key=API_KEY,
    base_url=BASE_URL,
)

​
Provider Specific Extra Headers

When making requests, you’ll need to specify provider-specific headers based on which LLM provider you’re using:

​
OpenAI Provider Headers

extra_headers = {
    "x-tfy-provider-name": "openai-provider-name"  # name of tfy provider integration
}

​
Vertex AI Provider Headers

extra_headers = {
    "x-tfy-provider-name": "google-provider-name",  # name of tfy provider integration
    "x-tfy-vertex-storage-bucket-name": "your-bucket-name",
    "x-tfy-vertex-region": "your-region",  # e.g., "europe-west4"
    "x-tfy-provider-model": "gemini-2-0-flash"  # or any other supported model
}

​
1. Upload File

Use this to upload files for usage with Batch or Assistants APIs.

file = client.files.create(
    file=open("request.jsonl", "rb"),
    purpose="batch",
    extra_headers=extra_headers
)

print(file.id)

Notes:

  • Max individual file size: 512 MB
  • Max org-wide storage: 100 GB
  • For Batch API: only .jsonl, max 200 MB
  • For Assistants API: up to 2M tokens

Expected Output

{
  "id": "file-abc123",
  "object": "file",
  "bytes": 120000,
  "created_at": 1677610602,
  "filename": "mydata.jsonl",
  "purpose": "fine-tune",
}

​
2. List Files

Returns a list of files

files = client.files.list(
	extra_headers=extra_headers
)

Expected Output

{
  "data": [
    {
      "id": "file-abc123",
      "object": "file",
      "bytes": 175,
      "created_at": 1613677385,
      "filename": "salesOverview.pdf",
      "purpose": "assistants",
    },
    {
      "id": "file-abc123",
      "object": "file",
      "bytes": 140,
      "created_at": 1613779121,
      "filename": "puppy.jsonl",
      "purpose": "fine-tune",
    }
  ],
  "object": "list"
}

​
3. Retrieve File

Returns information about a specific file.

file = client.files.retrieve(
	file.id,
	extra_headers=extra_headers
)

print(file.json())

Expected Output

{
  "id": "file-abc123",
  "object": "file",
  "bytes": 120000,
  "created_at": 1677610602,
  "filename": "mydata.jsonl",
  "purpose": "fine-tune",
}

​
4. Delete File

Delete a file permanently

deleted_file = client.files.delete(
	file.id,
	extra_headers=extra_headers
)

print(deleted_file.json())

Expected Output

{
  "id": "file-abc123",
  "object": "file",
  "deleted": true
}

​
5. Retrieve File Content

Returns the contents of the specified file.

output_content = client.files.content(
    file.id,
    extra_headers=extra_headers
)

print(output_content.json())