Skip to main content
Users can use prompts in two ways:
  1. Prompt Run API - Server-side rendering where the Gateway handles prompt rendering
  2. SDK - Client-side rendering where you fetch and render the prompt locally
In both the cases, you need to get the Prompt Version FQN to use the prompt.
You can get the FQN of the prompt version from the Prompt Registry as shown in the image below.
Getting FQN from prompt registry

​
Prompt Run API (server-side rendering)

Use the AI Gateway by passing the prompt version FQN in the request body. The Gateway will internally render the prompt and execute it. Parameters:
  • prompt_version_fqn - The fully qualified name of your prompt version
  • prompt_variables[Optional] - Variables to substitute in your prompt template
Important considerations:
  • If the prompt version doesn’t have a model configured, pass the model in the request body using the model parameter
  • If you specify a model in the request body and your prompt version already has a model configured, the request body model takes precedence
  • Any messages passed in the request body will be appended to the messages defined in the prompt version
You can get the exact code snippet you need for running the prompt from the playground or the Code Snippet button in the Playground or Prompt Registry.

​
From Playground

  1. Navigate to the Playground in your TrueFoundry dashboard
  2. Select and load the prompt as described in the document here Playground
  3. Click on the “Code Snippet” button to generate the exact code you need
  4. Copy the generated code snippet for your application
Getting code snippet from playground

​
From Prompt Registry

  1. Go to the Prompt Registry in your TrueFoundry dashboard
  2. Find and select the prompt version you want to use
  3. Click on the “Code Snippet” button in the prompt details
  4. Copy the generated code snippet with the correct prompt version FQN
Getting code snippet from prompt registry
from openai import OpenAI

client = OpenAI(
    api_key="your-tfy-api-key",
    base_url="https://{controlPlaneURL}/api/llm"
)

stream = client.chat.completions.create(
    messages=[],
    model="",
    stream=True,
    extra_headers={
        "X-TFY-METADATA": '{"your_custom_key":"your_custom_value"}',
        "X-TFY-LOGGING-CONFIG": '{"enabled": true}',
    },
    extra_body={
        "prompt_version_fqn": "chat_prompt:truefoundry/default/my-second-prompt:1",
        "prompt_variables": {
            "name": "John Doe",
            "age": "30"
        }
    },
)

for chunk in stream:
    if (
        chunk.choices
        and len(chunk.choices) > 0
        and chunk.choices[0].delta.content is not None
    ):
        print(chunk.choices[0].delta.content, end="", flush=True)

​
SDK (client-side rendering)

You can fetch the template and render it client-side and then make the request to the model with whatever OpenAI client you are using.
  1. Go to the Prompt Registry in your TrueFoundry dashboard
  2. Find and select the prompt version you want to use
  3. Click on the “Use Via SDK” button to view the code snippet
Getting code snippet from prompt registry
from truefoundry import client, render_prompt
from openai import OpenAI

# Fetch the prompt template
prompt_template = client.prompt_versions.get_by_fqn(
    fqn="chat_prompt:truefoundry/default/my-second-prompt:1",
).data.manifest

# Render the prompt with input variables
variables = {}  # Add your variables here
prompt = render_prompt(prompt_template, variables=variables)

# Send request to OpenAI directly
response = OpenAI().chat.completions.create(
    messages=prompt["messages"],
    model=prompt.get("model") or "gpt-4o-mini",
    **(prompt.get("parameters") or {}),
)

print(response.choices[0].message.content)
⌘I