Learn MCP - Performance & Optimization

#Introduction

In episode 18 you built custom transports and understood the SDK internals — framing, batching, and error codes. Now we turn to the question that arises as soon as a server starts being used by many hosts: how do you make MCP fast and keep it stable as load rises? Episode 19 is the optimization map from the easiest (payload) to the most architectural (many-to-many scaling and streaming).

#Measure First: Baseline Before Optimization

The first principle of performance is the one you already know from episode 15: don't optimize without data. Before touching any code, install telemetry and record these three numbers:

• Latency per phase — DNS, connection, TLS, up to the first response byte.
• Average payload size for tools/list, resources/read, and tools/call.
• The number of round-trips one agent run consumes, from first request to completion.

With this baseline you can see which problem is actually expensive. Run your server as usual with bun run dev, load it with a testing tool, then compare the numbers. Often the biggest cost isn't tool execution, but giant payloads and excessive round-trips eating the context window.

#Optimizing Payload Size

Large payloads consume transfer time and model context tokens. Here are the main weapons:

• Cursor pagination — for resources/list and long lists, don't return everything at once.
• Resource templates — let clients fetch a specific resource, not an entire directory.
• Concise tool schemas — avoid verbose descriptions; the model needs enough context, not an essay.
• Structured output — constrain the output shape in the schema so the model doesn't send a giant JSON full of unneeded fields.

{
  "jsonrpc": "2.0",
  "id": "r-7",
  "method": "resources/list",
  "params": {
    "cursor": "eyJwYWdlIjoyfQ=="
  }
}

A cursor keeps paging stateful on the client side without storing a session on the server — aligned with the stateless spirit of the 2026-07-28 spec.

#Batching and Reducing Round-Trips

Every round-trip means network latency plus context tokens for storing intermediate results. Reduce their count three ways:

• Dense tool design — instead of three calls get_user, get_orders, get_address, provide a get_user_profile that returns everything in one go.
• MRTR for interactive flows — use multi-round-trip requests only when you truly need confirmation or step-up authorization, not for work that can be summarized.
• Notifications for signals — progress and cancel don't need a reply round-trip.

From episode 18 you know notifications can be batched into a single write cycle. Combine that with economical tool design, and an agent run that used to be 30 calls can drop to half.

#Caching Resources

Data that rarely changes — tool lists, schemas, static resources — is a perfect caching candidate. The simplest pattern is a TTL cache with invalidation via notifications/resources/updated.

const cache = new Map<string, { value: unknown; expiresAt: number }>()
 
async function cachedRead(uri: string, ttlMs = 60_000) {
  const hit = cache.get(uri)
  if (hit && hit.expiresAt > Date.now()) return hit.value
 
  const value = await readResource(uri)
  cache.set(uri, { value, expiresAt: Date.now() + ttlMs })
  return value
}

One important warning: don't cache personal or per-user data without separating the key per user. A cache that leaks across users is a data leak — remember the hardening lesson in episode 14.

#Connection Pooling

Opening an HTTP connection from scratch for every request is expensive — the TCP handshake, TLS, and HTTP/2 get rebuilt over and over. The solution is connection pooling with keep-alive: connections are reused for many requests.

import { Pool } from 'undici'
 
const pool = new Pool('https://mcp.example.com', {
  connections: 20,
  pipelining: 4,
  keepAliveTimeout: 30_000
})

On the server side, make sure the HTTP framework supports keep-alive, and configure the load balancer not to cut idle connections too aggressively — the server-side keep-alive timeout should be larger than the client's usage gaps, so connections don't die in the middle of the pool.

#Scaling Many-to-Many

One host can connect to many servers, and one server serves many hosts — a many-to-many matrix. This is where the stateless architecture from episode 3 pays its debt: without Mcp-Session-Id, requests can be routed to any instance, so:

• Many hosts (IDEs, CLIs, bots, applications) come in through a single MCP gateway as in episode 16.
• The gateway distributes requests to a server pool with round-robin, no sticky sessions.
• Health checks remove failing instances from the pool, and the rolling updates from episode 16 run without sessions being interrupted.

The picture is simple: many hosts, one entry point, a server pool that grows and shrinks with load.

#Handling Streaming Load

Streaming — SSE, tools that stream results, and MRTR — holds connections longer than ordinary requests. Two main enemies: ballooning buffers and streams that stay silent forever. Apply these rules:

• Backpressure — if the consumer is slow, don't accumulate output without limit; stop producing until the consumer is ready.
• Per-stream timeout — a stream that doesn't progress within X seconds is force-closed.
• Stream limit per client — don't let one client open hundreds of streams.

proxy_buffering off;
proxy_read_timeout 3600s;

The combination of disabled buffering and realistic timeouts keeps the streaming server responsive for all clients, not just the fastest ones.

#Conclusion

Episode 19 gave you a full arsenal for MCP performance: measure first with telemetry, trim payload size with pagination and structured output, reduce round-trips through dense tool design, cache resources with TTL, pool connections with keep-alive, a stateless many-to-many architecture, and streaming load management with backpressure.

Key takeaways:

• Optimization starts with measurement — plant the OTel from episode 15 before touching code.
• Payload and round-trips are the most often ignored costs; pagination and dense tool design solve them.
• Cache static data with TTL, but never mix caches across users.
• Connection pooling and keep-alive turn hundreds of small requests into one long-lived connection.
• The stateless architecture is the prerequisite for many-to-many scale without sticky sessions.

In the next episode 20 we step out of the server and into the ecosystem: integration with LangChain and LangGraph, the OpenAI Agents SDK, Claude and Codex, IDE agents, plus RAG patterns, tool automation, and agentic payments. See you there!