What is MCP and how does it work?
MCP is an open protocol that standardizes how applications provide context to LLMs. Think of MCP like a USB-C port for AI applications. Just as USB-C provides a standardized way to connect your devices to various peripherals and accessories, MCP provides a standardized way to connect AI models to different data sources and tools. - From Anthropic docs
| Slack MCP Server | GitHub MCP Server |
|---|---|
| Send a message to a channel | Get the list of repositories |
| Search for messages in a channel | Get the list of issues |
| Get the list of channels | Get the list of pull requests |
| Get the list of users | Create a pull request on a repository |
| More… | More… |
Get open pull requests on my repository test-repo and send me a slack message with the list of pull requests.
You can also create your own MCP servers to expose your data and internal APIs as tools to LLMs.
Democratizing MCP Access and AI agentic workflows
The above concept of MCP servers presents a really powerful system wherein it becomes really easy for everyone inside an organization to build agents really fast and explore a wide range of automations. However, if every developer were to manage their own MCP servers, or authorization, it will result in chaos and security risks. A centralized MCP gateway and registry helps solve these problems.
| Without MCP Gateway | With MCP Gateway | ||
|---|---|---|---|
| AI agents require separate connections to each MCP server | AI agents connect to one gateway, access multiple MCP servers | ||
| Each developer configures VS Code, Cursor, Claude Code individually | Single configuration point for VS Code, Cursor, Claude Code | ||
| Developers must install and manage MCP servers locally | Central IT manages cloud-hosted MCP infrastructure via streamable HTTP | ||
| No standard authentication flow for enterprise tools | Developers use standard OAuth 2LO/3LO flows for enterprise MCP servers | ||
| Scattered API keys and credentials across tools | Centralized credential management with secure vault integration | ||
| No visibility into what tools teams are using | Complete visibility and audit trail for all tool usage | ||
| Security risks from unmanaged tool sprawl | Enterprise-grade security with governed tool access | ||
| No dynamic tool discovery for autonomous agents | Dynamic tool discovery and invocation for autonomous workflows | ||
| No curated tool catalog for multi-tenant environments | Registry provides discoverable, curated MCP servers for multi-tenant use |
Truefoundry MCP Gateway
The Truefoundry AI Gateway is an enterprise-ready platform that centralizes access to AI development tools using the Model Context Protocol (MCP). Instead of managing hundreds of individual tool configurations across your development teams, provide secure, governed access to curated AI tools through a single platform. It provides a MCP registry, centralized authentication and a MCP client built into the gateway that can orchestrate the agentic loop between the LLM and the MCP servers.Architecture
Truefoundry MCP Gateway Architecture
- Centralized MCP Registry: You can add public as well as your self hosted MCP servers which are registered in the Truefoundry Control Plane. The Control plane maintains the centralized registry of all the MCP servers and their authentication mechanisms. It handles user-specific OAuth2 flows, securely storing and refreshing access tokens and ensuring users can only access resources they are authorized for.
- Access Control: While registering a MCP server, you can specify the list of users/teams that have access to the MCP server.
- Unified Key to access all MCP servers: Any user can generate a single Personal Access Token (PAT) using which they can access all the models and MCP servers that they have access to. You can also generate a Virtual Account Token(VAT) to provide access to a specific set of MCP servers to an application.
- Agent Playground: Truefoundry AI gateway provides a playground where in users can play with prompts and different tools of MCP servers to build agents. Truefoundry comes with commonly used tools like Websearch, WebScraping, document extraction, code execution. The gateway comprises of a MCP client that orchestrates executing the tools decided by the LLM providers.
The Gateway also streames the progress of the request back to the UI so that the user can see the LLM responses, tool calls and the tool responses.
- Use MCP Servers in Code: The Gateway also shows the code snippets using which you can start using the MCP servers in your code.
Agentic Loop in the Truefoundry AI Gateway