If you are looking to experiment with open-source LLMs, getting started with them can be cumbersome because you need to set up GPUs and load the models - hence incurring time and cost. It's much more difficult to test out open-source LLMs compared to trying out OpenAI which already provides the LLM as a ready-to-use API.
To overcome this, Truefoundry provides an experimental playground where you can try out different open-source models as an API without having to go through the trouble of deploying them on your own. Once you figure out which model works best for you the best, you can either host the model on your own cloud environment or ask us to host it in a production-ready setup for you. You can also integrate OpenAI and other providers with the playground so that you can compare the responses between OpenAI and other open-source models.
Choose an LLM from the list and enter prompts to generate responses for those prompts.
You can select up to four models simultaneously assess these different models side by side, helping you make informed decisions for your specific needs.
If you don't see any, you can integrate OpenAI models into the LLM Gateway. To do this however you need to be Tenant admin. If you do not have administrator privileges, please request your Tenant Admini to integrate OpenAI on your behalf following this guide
You can similarly Integrate other providers like Cohere etc.
Updated 3 months ago