​
What is MCP and how does it work?

MCP is an open protocol that standardizes how applications provide context to LLMs. Think of MCP like a USB-C port for AI applications. Just as USB-C provides a standardized way to connect your devices to various peripherals and accessories, MCP provides a standardized way to connect AI models to different data sources and tools. - From Anthropic docs
Diagram explaining how MCP connects AI models to tools and data sources MCP Servers are programs that expose data and capabilities to LLMs via the MCP protocol.
Slack MCP ServerGitHub MCP Server
Send a message to a channelGet the list of repositories
Search for messages in a channelGet the list of issues
Get the list of channelsGet the list of pull requests
Get the list of usersCreate a pull request on a repository
More…More…
If an LLM is provided access to a Slack and Github MCP server, we can create agents very easily by prompting to the LLM:
Get open pull requests on my repository test-repo and send me a slack message with the list of pull requests.
You can also create your own MCP servers to expose your data and internal APIs as tools to LLMs.

​
Democratizing MCP Access and AI agentic workflows

The above concept of MCP servers presents a really powerful system wherein it becomes really easy for everyone inside an organization to build agents really fast and explore a wide range of automations. However, if every developer were to manage their own MCP servers, or authorization, it will result in chaos and security risks. A centralized MCP gateway and registry helps solve these problems. Before and after MCP gateway The key advantages of a MCP Gateway are:
Without MCP GatewayWith MCP Gateway
AI agents require separate connections to each MCP server AI agents connect to one gateway, access multiple MCP servers
Each developer configures VS Code, Cursor, Claude Code individually Single configuration point for VS Code, Cursor, Claude Code
Developers must install and manage MCP servers locally Central IT manages cloud-hosted MCP infrastructure via streamable HTTP
No standard authentication flow for enterprise tools Developers use standard OAuth 2LO/3LO flows for enterprise MCP servers
Scattered API keys and credentials across tools Centralized credential management with secure vault integration
No visibility into what tools teams are using Complete visibility and audit trail for all tool usage
Security risks from unmanaged tool sprawl Enterprise-grade security with governed tool access
No dynamic tool discovery for autonomous agents Dynamic tool discovery and invocation for autonomous workflows
No curated tool catalog for multi-tenant environments Registry provides discoverable, curated MCP servers for multi-tenant use

​
Truefoundry MCP Gateway

The Truefoundry AI Gateway is an enterprise-ready platform that centralizes access to AI development tools using the Model Context Protocol (MCP). Instead of managing hundreds of individual tool configurations across your development teams, provide secure, governed access to curated AI tools through a single platform. It provides a MCP registry, centralized authentication and a MCP client built into the gateway that can orchestrate the agentic loop between the LLM and the MCP servers.

​
Architecture

Architecture diagram of MCP Gateway

Truefoundry MCP Gateway Architecture

The key functionalities available are:
  1. Centralized MCP Registry: You can add public as well as your self hosted MCP servers which are registered in the Truefoundry Control Plane. The Control plane maintains the centralized registry of all the MCP servers and their authentication mechanisms. It handles user-specific OAuth2 flows, securely storing and refreshing access tokens and ensuring users can only access resources they are authorized for.
  2. Access Control: While registering a MCP server, you can specify the list of users/teams that have access to the MCP server.
  3. Unified Key to access all MCP servers: Any user can generate a single Personal Access Token (PAT) using which they can access all the models and MCP servers that they have access to. You can also generate a Virtual Account Token(VAT) to provide access to a specific set of MCP servers to an application.
  4. Agent Playground: Truefoundry AI gateway provides a playground where in users can play with prompts and different tools of MCP servers to build agents. Truefoundry comes with commonly used tools like Websearch, WebScraping, document extraction, code execution. The gateway comprises of a MCP client that orchestrates executing the tools decided by the LLM providers.
The Gateway also streames the progress of the request back to the UI so that the user can see the LLM responses, tool calls and the tool responses.
  1. Use MCP Servers in Code: The Gateway also shows the code snippets using which you can start using the MCP servers in your code.
Architecture diagram of TrueFoundry AI Gateway's MCP Registry and integration with LLM providers

Agentic Loop in the Truefoundry AI Gateway