The Unified API in the TrueFoundry LLM Gateway provides a standardised interface for accessing and interacting with different language models from various providers. This means you can seamlessly switch between models and providers without changing your application's code structure. By abstracting the underlying complexities, the Unified API simplifies the process of integrating multiple models and ensures consistency in how they are accessed and utilized.

Key Features of the Unified API

1. Standardization: The Unified API standardizes requests and responses across different models, making it easier to manage and integrate multiple models.
2. Flexibility: Easily switch between different models and providers without altering the core application code.
3. Efficiency: Streamline the development process by using a single API to interact with various models, reducing the need for multiple integrations and bespoke handling.
4. Compatibility: By adhering to the OpenAI request-response format, the Unified API ensures seamless integration with Python libraries like OpenAI and LangChain.

Unified API Code Generation

You can use the Gateway to get the code to call the unified API using various languages and libraries. Code generation in TrueFoundry LLM Gateway refers to the automated creation of boilerplate code necessary for integrating language models into your applications.

The demo below demonstrates how to call any model from any provider using a standardized code through the Gateway.

This feature significantly speeds up development by providing ready-to-use code snippets tailored to your specific needs and the models you are using.

Tool Calling

Tool calling in TrueFoundry LLM Gateway enables language models to simulate interactions with external functions, enhancing their versatility and practical application.

Although the gateway does not directly execute calls to external tools, it allows you to describe the tool and simulate the call within the response. This simulation provides a detailed representation of the request and expected response, allowing developers to understand how the language model would interact with external systems.