Learn gRPC - HTTP/2, Load Balancing & Connection Management

#Introduction

gRPC can't be fully understood without understanding HTTP/2. All of gRPC's advantages — multiplexing, streaming, efficiency — are born from this transport protocol. And once many servers are involved, a question arises: how do we divide traffic fairly?

Episode 10 covers three layers: the HTTP/2 networking characteristics that shape gRPC's behavior, client-side load balancing strategies from pick_first to xDS, and connection management — pooling, keepalive, and stream management — so the connections you built in episode 4 stay healthy under load.

#HTTP/2 Networking Characteristics

#Multiplexing and Stream Limits

A single HTTP/2 connection can carry many streams, and one stream is one RPC. The advantage: many RPCs share one TCP connection without queueing. But there's a limit: the maximum active streams per connection is usually 100 (controlled by SETTINGS_MAX_CONCURRENT_STREAMS).

The practical implication: for thousands of concurrent calls, a client needs more than one connection — that's why gRPC opens multiple connections per channel automatically when needed.

#HPACK and Header Compression

HTTP/2 headers are compressed with HPACK using static and dynamic tables. Repeated headers like content-type: application/grpc are sent as table references, not full strings. This is why gRPC's per-request overhead is so small compared to HTTP/1.1.

#Client-Side Load Balancing

#pick_first

The default strategy: the client picks one address from the resolver's list and uses that connection. If it fails, it moves to the next address:

conn, _ := grpc.NewClient(
    "dns:///catalog-svc:50051",
    grpc.WithTransportCredentials(insecure.NewCredentials()),
)

The pick_first strategy fits when load balancing is handled at another level — for example, a Kubernetes Service already balances across pods.

#round_robin

To distribute calls evenly across instances, enable round_robin via service config:

conn, _ := grpc.NewClient(
    "dns:///catalog-svc:50051",
    grpc.WithDefaultServiceConfig(`{"loadBalancingConfig":[
        {"round_robin": {}}
    ]}`),
    grpc.WithTransportCredentials(insecure.NewCredentials()),
)

With {"round_robin": {}} in the service config, every call rotates to the next instance in sequence. This is the simplest strategy for a pool of homogeneous servers.

#xDS and EDS

For dynamic control, xDS (specifically EDS/Endpoint Discovery Service) lets gRPC clients receive the endpoint list from a control plane like Envoy or Istio, including per-endpoint weight and health. Details in episode 18; what you need to understand now: xDS is centrally managed load balancing, not manually configured on each client.

#Connection Pooling and Keepalive

#Keepalive to Detect Dead Connections

A connection that looks alive can actually be broken at the TCP level. Keepalive sends periodic pings to detect this:

conn, _ := grpc.NewClient("localhost:50051",
    grpc.WithTransportCredentials(insecure.NewCredentials()),
    grpc.WithKeepaliveParams(keepalive.ClientParameters{
        Time:                30 * time.Second,
        Timeout:             10 * time.Second,
        PermitWithoutStream: true,
    }),
)

The Time: 30 * time.Second setting sends a ping every 30 seconds when there's no activity. PermitWithoutStream: true allows pings even without an active RPC — keeping the connection warm.

#Server Keepalive

The server also configures keepalive policy so one client can't dominate:

s := grpc.NewServer(grpc.KeepaliveParams(keepalive.ServerParameters{
    MaxConnectionIdle: 5 * time.Minute,
    Time:              2 * time.Hour,
    Timeout:           20 * time.Second,
}))

MaxConnectionIdle: 5 * time.Minute closes connections idle too long so instances can be scaled down. Balancing client and server parameters avoids dead connections being left hanging.

#Stream Management

#The Impact of Long Streams

Long streaming RPCs — like watch or event feeds — hold one stream slot on the connection. If all streams are taken up by long streams, new unary RPCs queue up. The solution: limit long streams per connection and open separate channels for different workloads, e.g. one channel for long streams and one for unary.

#Backpressure

gRPC applies flow control per stream. When the receiver is slow, the sender is automatically throttled. Don't hold Recv for too long when reading a stream, because that creates backpressure that slows down the whole pipeline.

#Closing

Key takeaways:

• HTTP/2 multiplexing lets many RPCs share one connection, with a per-connection stream limit.
• HPACK shrinks repeated headers so per-request overhead is tiny.
• pick_first for a single address; round_robin for even distribution; xDS for centralized control.
• Keepalive on client and server keeps connections from TCP deadlock.
• Long streams hold slots; separate channels for unary and streaming workloads.
• HTTP/2 flow control handles backpressure automatically when the receiver is slow.

In episode 11 next, we cover TLS, mTLS, and authentication in gRPC — setting up certificates and TLS on server and client, mutual TLS for two-way authentication, and modern integration with JWT, OAuth2, and token-based auth via metadata. A healthy connection without encryption is only half the journey; now it's time to secure it.