Prompt Management

TrueFoundry’s LLM Playground is your go-to space for experimenting, testing, and perfecting prompts for your AI application. Here, you can explore different models, test various variables, review completions, and fine-tune your prompt engineering strategy—all before deploying to production. It’s the ideal environment to ensure your prompts are optimized and ready for real-world use.

​

Writing Prompts on Playground

Writing prompts on Playground is pretty simple.

1. You start by selecting a model that you have integrated.
2. Add your System Message
3. Add your user message or query
4. Hit Run to generate a response

​

Add Variables to Prompts

You can simply create variable by writing {{variable_name}} as part of your messages. Once the variable is added you can provide values which will be reflect in the chat section as shown below:

​

Add Tools to Prompts

Some models support function calling, enabling the AI to request specific information or perform actions. The Playground simplifies experimentation with these features.

Although the gateway does not directly execute calls to external tools, it allows you to describe the tool and simulate the call within the response. This simulation provides a detailed representation of the request and expected response, allowing developers to understand how the language model would interact with external systems.

To define functions that the model can call, simply click the “Add Tool” button and follow the steps as shown below:

​

Save Prompts as Prompt Template

Once you are happy with the prompt. You can click on “Save Template” button on the screen. All prompt templates are stored in ML Repos which is like a storage unit on TrueFoundry.

To update an existing template, follow the same steps.

​

Prompt Template Versions

Whenever you modify a prompt template, TrueFoundry automatically tracks these changes. The versioning system enables you to:

1. Experiment with and test different prompt variations
2. Maintain a full history of all prompt changes
3. Compare different versions

​

Load existing Prompt Template

To load a version of an existing prompt template, you just need to visit playground and follow these steps:

​

Use Prompt Template in Code

TrueFoundry automatically generates a code snippet for you to use specific version of your prompt template. Simply visit the prompt details page and switch to “Use in Code” tab to copy the code snippet.

​

Fetching latest prompt version

You can fetch latest version of a prompt using latest tag instead of the version number

from typing import Dict, Any
from truefoundry import client

def get_latest_prompt() -> str:
	prompt_version_response = client.prompt_versions.get_by_fqn(
	    # Here we are using :latest instead of version number in the fqn
	    # fqn="chat_prompt:my-org/my-ml-repo/test-prompt:1",
	    fqn="chat_prompt:my-org/my-ml-repo/test-prompt:latest",
	)
	prompt_template = prompt_version_response.data.manifest
	return prompt_template


CACHED_PROMPT = ""
CACHED_PROMPT_LAST_FETCHED: float = 0

def get_latest_prompt_cached() -> str:
    """
    Fetches latest prompt and caches it for 10 mins
    """
    global CACHED_PROMPT, CACHED_PROMPT_LAST_FETCHED
    ttl = 600
    if time.time() - CACHED_PROMPT_LAST_FETCHED > ttl:
        CACHED_PROMPT = get_latest_prompt()
        CACHED_PROMPT_LAST_FETCHED = time.time()
    return CACHED_PROMPT

    
prompt_template = get_latest_prompt_cached()