Skip to content

Understanding MCP Gateway Architecture

This guide demonstrates how to explore and understand MCP Gateway's architecture by examining its components, configuration, and request flows in a running cluster.

About MCP Gateway Components

You should read the overview first to understand the high-level components and design.

Prerequisites

Required Setup: This guide assumes you have a running MCP Gateway cluster.

See the installation guide for setup instructions.

Step 1: Understanding Configuration and State

Before observing the system in action, let's understand how it's configured.

MCPServerRegistration Resources

MCPServer custom resources define backend servers:

# View all MCPServerRegistration resources
kubectl get mcpsr -A

# Describe a specific MCPServerRegistration
kubectl describe mcpserver api-key-server -n mcp-test

Example MCPServerRegistration spec: 

spec:
  prefix: test1_
  targetRef:
    group: gateway.networking.k8s.io
    kind: HTTPRoute
    name: mcp-server1-route
    namespace: mcp-test

Key fields:

  • prefix: Prefix added to all tools from this server
  • targetRef: Reference to HTTPRoute that defines the backend

Generated Configuration

The controller generates a ConfigMap that the broker/router consumes:

# View generated configuration
kubectl get configmap mcp-gateway-config -n mcp-system -o yaml

Configuration structure: 

data:
  config.yaml: |
    servers:

      - name: mcp-test/mcp-server1-route
        url: http://mcp-test-server1.mcp-test.svc.cluster.local:9090/mcp
        hostname: server1.mcp.local
        prefix: test1_
        enabled: true

What each server entry contains:

  • name: Unique identifier (namespace/route-name)
  • url: Full URL to backend MCP server
  • hostname: Used for routing decisions by Envoy
  • prefix: Prefix for tool federation
  • enabled: Whether this server is active

HTTPRoute Configuration

HTTPRoutes define how to reach backend services:

# View all routes
kubectl get httproute -A

# Describe a backend server route
kubectl describe httproute mcp-server1-route -n mcp-test

What to examine:

  • spec.hostnames: Used by router for routing decisions (e.g., server1.mcp.local)
  • spec.rules.backendRefs: Service and port to route to
  • status: Whether the route is accepted and programmed

Envoy Configuration

Envoy's configuration is generated by Istio from Gateway and HTTPRoute resources:

# View full Envoy configuration (large output)
make istio-config

# Check specific backend clusters
istioctl proxy-config clusters $(kubectl get pods -l istio=ingressgateway -n gateway-system -o jsonpath='{.items[0].metadata.name}') -n gateway-system

# Check routing configuration
istioctl proxy-config routes $(kubectl get pods -l istio=ingressgateway -n gateway-system -o jsonpath='{.items[0].metadata.name}') -n gateway-system

Key Envoy concepts:

  • Clusters: Backend service definitions (e.g., mcp-test-server1.mcp-test.svc.cluster.local:9090)
  • Listeners: Entry points for traffic (e.g., port 8080)
  • Routes: Mapping from hostnames/paths to clusters
  • ext_proc filter: External processor that calls the router for tools/call requests

Step 2: Understanding Session Management

The router maintains two types of session mappings:

  1. Client sessions: Created by the broker when clients initialize
  2. Backend sessions: Created by the router when first calling a tool on a backend server

Session mapping structure: 

gateway-session-id/backend-server-name → backend-mcp-session-id

Example flow:

  1. Client initializes → broker creates session client-abc-123
  2. Client calls test1_hi with session client-abc-123
  3. Router checks: Does client-abc-123/server1 mapping exist?
  4. If no: Router initializes connection to backend, gets session backend-xyz-789
  5. Router stores: client-abc-123/server1 → backend-xyz-789
  6. Router forwards request with session backend-xyz-789
  7. Future calls from same client to server1 reuse session backend-xyz-789

Step 3: Using MCP Inspector for Interactive Testing

The MCP Inspector provides a web interface for exploring the gateway:

# Open inspector for the gateway
make inspect-gateway

This opens http://localhost:6274/?transport=streamable-http&serverUrl=http://mcp.127-0-0-1.sslip.io:8001/mcp

What you can do:

  1. View all tools: Navigate to Tools → List Tools to see all available tools with their prefixes
  2. Execute tools: Click on any tool (e.g., test1_greet) and provide arguments to execute it
  3. View request/response: See full JSON payloads for each interaction

Step 4: Examining Component Layout and Logs

Now that you understand the configuration, let's observe the running system.

View Cluster Status

# Get overview of all components
make status

This shows:

  • Running components and their status
  • Local processes (if any)

Inspect Component Pods

# MCP Gateway components
kubectl get pods -n mcp-system
kubectl describe pods -n mcp-system

# Test MCP servers
kubectl get pods -n mcp-test

# Gateway infrastructure (Istio)
kubectl get pods -n istio-system

MCP Broker Logs: Tool Aggregation

The broker is responsible for:

  • Connecting to all configured backend MCP servers
  • Aggregating tools from all servers with prefixes
  • Handling client initialize and tools/list requests
  • Caching the aggregated tool list

View broker logs:

# Watch all logs in real-time
kubectl logs -f deployment/mcp-gateway -n mcp-system

# View recent broker activity (filter by key patterns)
kubectl logs deployment/mcp-gateway -n mcp-system --tail=100 | grep -E "(Registering|Discovered|Federating)"

Key log patterns:

Pattern Meaning
Registering server mcpURL=... prefix=... Broker connecting to backend MCP server
Discovered tools mcpURL=... #tools=N Successfully connected and discovered tools
Federating tool mcpURL=... "federated name"=... Adding tool with prefix to aggregated list
Server registered url=... totalServers=N Backend server successfully registered

MCP Router Logs: Tool Call Routing

The router is responsible for:

  • Intercepting tools/call requests via Envoy ext_proc
  • Parsing tool names to determine target server (based on prefix)
  • Setting routing headers (:authority) for Envoy
  • Managing backend MCP server sessions
  • Stripping prefixes from tool names before forwarding

View router logs:

# Watch all logs in real-time
kubectl logs -f deployment/mcp-gateway -n mcp-system

Note: The router operates as an Envoy external processor (ext_proc) and intercepts requests at the proxy level. Router logs are generated when processing tool calls.

To see router activity, use the MCP Inspector (from Step 3) to make tool calls.

For detailed routing logs, enable debug logging: 

kubectl set env deployment/mcp-gateway LOG_LEVEL=-4 -n mcp-system

MCP Gateway Controller Logs: Dynamic Discovery

The controller is responsible for:

  • Watching MCPServerRegistration custom resources
  • Resolving HTTPRoute references to backend URLs
  • Updating ConfigMap with server configuration
  • Triggering broker/router reloads

View controller logs:

# Watch controller logs
kubectl logs -f deployment/mcp-gateway-controller -n mcp-system

Key log patterns:

Pattern Meaning
Reconciling MCPServerRegistration Processing MCPServerRegistration resource change
Updated HTTPRoute status HTTPRoute discovered and status updated
Config push completed Configuration pushed to broker/router
Successfully regenerated aggregated configuration Configuration successfully updated with server count
Successfully got status from endpoint Backend server health check passed

Step 5: Tracing Request Flows

This section demonstrates how different request types flow through the system.

Setup: Log Monitoring

Open two terminals to monitor logs simultaneously:

# Terminal 1 - MCP Gateway logs (broker and router combined)
kubectl logs -f deployment/mcp-gateway -n mcp-system

# Terminal 2 - Envoy access logs (optional)
kubectl logs -f -n gateway-system -l istio=ingressgateway

Flow 1: Initialize Request

The initialize method establishes a client session and returns capabilities.

How to test: Use the MCP Inspector (Step 3) - when you first connect, it sends an initialize request automatically.

What happens:

The initialize request is handled by the broker.

Expected logs with debug logging enabled: 

INFO Processing request method=initialize
INFO [EXT-PROC] Response backend session: mcp-session-xxxxx
INFO [EXT-PROC] Session ID doesn't need reverse mapping

Flow 2: Tools List Request

The tools/list method returns all available tools from all servers.

How to test: In the MCP Inspector (Step 3), navigate to Tools → List Tools.

What happens:

The broker returns the cached aggregated tool list.

Expected logs: 

DEBUG [EXT-PROC] HandleRequestBody None tool call setting method header onlytools/list
INFO Sending MCP routing instructions to Envoy: request_body:{response:{header_mutation:{set_headers:{header:{key:"x-mcp-method" raw_value:"tools/list"}}}}}
INFO [EXT-PROC] Processing response body...

Example tools returned:

  • test1_greet (from server1 with prefix test1_)
  • test2_greet (from server2 with prefix test2_)
  • apikey_greet (from api-key-server with prefix apikey_)

Flow 3: Tool Call Request (The Routing Magic)

The tools/call method is where routing happens - this is the most complex flow.

How to test: In the MCP Inspector (Step 3):

  1. Navigate to Tools → List Tools to see available tools
  2. Click on a tool (e.g., test1_greet)
  3. Provide any required arguments
  4. Click Execute
  5. Watch the logs in your terminal to see the routing activity

What happens:

The router intercepts the request via Envoy ext_proc, strips the prefix, and routes to the appropriate backend server.

Expected logs with debug logging enabled: 

INFO [EXT-PROC] Found matching server toolName=test1_greet serverPrefix=test1_ serverName=mcp-test/mcp-server1-route
INFO Stripped tool name tool=greet originalPrefix=test1_
INFO Completed MCP processing with routing target=mcp-test/mcp-server1-route
INFO Sending MCP routing instructions to Envoy: request_body:{response:{header_mutation:{set_headers:{header:{key:"x-mcp-toolname" raw_value:"test1_greet"}}
INFO [EXT-PROC] Processing response body... (size: 136, end_of_stream: true)
INFO [EXT-PROC] Response body content: event: message
id: 3_0
data: {"jsonrpc":"2.0","id":3,"result":{"content":[{"type":"text","text":"Hi Patryk"}],"structuredContent":{}}}

Step 6: Troubleshooting Common Scenarios

Scenario 1: Tool Not Found

Symptoms: 

{"error": {"code": -32601, "message": "Tool not found"}}

Investigation:

  1. Check if tool exists in aggregated list using MCP Inspector (Step 3) or: 

    kubectl get configmap mcp-gateway-config -n mcp-system -o yaml

  2. Check MCPServerRegistration status: 

    kubectl get mcpsr -A
kubectl describe mcpserver <name> -n mcp-test

  3. Check broker logs for connection issues: 

    kubectl logs deployment/mcp-gateway -n mcp-system | grep -i error

Scenario 2: Backend Server Unreachable

Symptoms:

  • Router logs show connection failures
  • 503 Service Unavailable responses
  • Tools list is missing tools from a specific server

Investigation:

  1. Check backend pod health: 

    kubectl get pods -n mcp-test
kubectl describe pod <backend-pod-name> -n mcp-test

  2. Check service exists: 

    kubectl get svc -n mcp-test
kubectl describe svc <backend-service-name> -n mcp-test

  3. Check HTTPRoute is valid: 

    kubectl get httproute -n mcp-test
kubectl describe httproute mcp-server1-route -n mcp-test

Scenario 3: Routing to Wrong Server

Symptoms:

  • Tool calls timeout or return unexpected results
  • Router logs show incorrect authority headers
  • Tools work when calling backend directly but fail through gateway

Investigation:

  1. Check logs for routing activity: 

    kubectl logs deployment/mcp-gateway -n mcp-system | grep -E "(routing|authority|ext_proc)"

  2. Verify prefix mappings in configuration: 

    kubectl get configmap mcp-gateway-config -n mcp-system -o jsonpath='{.data.config\.yaml}' | grep -A 5 prefix

  3. Check Envoy routing configuration: 

    istioctl proxy-config routes $(kubectl get pods -l istio=ingressgateway -n gateway-system -o jsonpath='{.items[0].metadata.name}') -n gateway-system | grep -A 10 "server1.mcp.local"