Learn WireGuard - Handshake & Lifecycle

#Introduction

Every WireGuard data packet sits behind a session, and every session is born from a process called the handshake. Understanding the handshake means understanding why a connection is a few milliseconds slow at first, why an idle connection can "die," and how WireGuard survives attacks that flood it with handshake packets.

Episode 5 dissects the Noise IK handshake mechanism, the cookie mechanism as an anti-DoS shield, the automatic rekeying policy, and the state transition map of a session from birth to end.

Throughout this episode, remember that all these mechanisms run automatically inside the kernel. Your only role is to ensure the correct initial conditions — keys, endpoint, and firewall — and then let the protocol take care of itself.

#The Noise IK Handshake

#Three Messages, One Round Trip

The WireGuard handshake consists of just three messages: a handshake initiation (from the initiator), a handshake response (from the responder), and after that both parties already have session keys. From the initiator's point of view this is one round trip — unlike TLS, which can take several exchanges.

The process in the Noise IKpsk2 scheme:

• The initiator sends a handshake initiation containing its ephemeral public key.
• The responder validates it, computes a shared secret, and replies with a handshake response.
• Both parties derive the same data encryption keys.
• The responder can already send data; the initiator waits for the response.

wg show wg0 latest-handshakes

The output of wg show wg0 latest-handshakes shows the time of the last handshake as a UNIX timestamp. A value of zero means the peer has never handshaked — usually because the endpoint is wrong or the UDP port is closed.

#Presence Packets

After the handshake, if there is no data, WireGuard sends a keepalive packet roughly every 10 seconds so that NAT mappings and session state stay fresh. These keepalives do not count as transferred traffic.

These keepalives work together with PersistentKeepalive, covered in episode 7: the internal keepalive maintains the protocol's session state, while PersistentKeepalive maintains the NAT mapping so the session remains reachable from the outside.

#The Cookie Mechanism

#Anti-DoS on the Handshake

Handshake initiation is computed with elliptic curve cryptography, which is fairly expensive. If an attacker floods a server with fake handshakes, the server can be exhausted. WireGuard dampens this with the cookie mechanism.

How it works: the server asks the responder to supply a cookie (a MAC value derived from the destination IP address) before accepting the next handshake. Because the cookie is bound to the IP address, an attacker spoofing addresses cannot complete the handshake. The mechanism is efficient: no per-requester session state needs to be stored.

#When Cookies Appear

Cookies appear when the server is under heavy handshake pressure or when the peer table is full. When you see a Handshake was rate limited message in the logs, that is the cookie mechanism working.

Important: the cookie mechanism is not protection against every kind of DoS attack; it is a specific dampener for handshake floods. This fits the WireGuard philosophy — focus on one problem and solve it well.

One practical note: the handshake age is not the only health indicator. The combination of latest-handshakes and a continuously growing transfer gives a more complete picture, and both will be used as the basis for monitoring in episodes 8 and 21.

#Rekeying and State Transitions

#Automatic Rekeying

Session keys are not used forever. WireGuard replaces keys automatically:

• Time-based: rekey roughly every 2 minutes per direction.
• Volume-based: rekey after a large number of packets have been sent.
• Idle-based: when a session is inactive too long, the keys are discarded and the next handshake starts from scratch.

Rekeying preserves forward secrecy — the leak of one session key does not open other sessions.

#The Session State Map

A WireGuard session moves through simple states:

• empty: no handshake yet, session keys are empty.
• handshake-initiation-sent: the initiator has sent the request and waits for a response.
• handshake-complete: the response is valid, session keys are active, and data can flow.
• data-received: the session sends and receives encrypted data.
• timeout / rekey: the session ages, keys are replaced, or the session closes back to empty.

These transitions happen automatically in the kernel. You do not manage this state manually — all you observe is latest-handshakes and the output of wg show.

#Diagnosing Handshake Problems

#Repeated Handshake Failures

If wg show always shows a zero handshake, this is the usual troubleshooting order:

• Does Endpoint in [Peer] point to the correct address and UDP port?
• Does the firewall open UDP on the ListenPort?
• Do both sides know each other's public keys correctly?
• Is one side behind a strict NAT and in need of PersistentKeepalive?

If all of these are correct but the handshake still fails, set PersistentKeepalive = 25 on the client side and watch whether the endpoint on the server changes after the first packet arrives. A changing endpoint indicates the packet is arriving even though the handshake is not complete, so the problem lies in processing, not connectivity.

watch -n 2 "sudo wg show wg0 latest-handshakes"

watch -n 2 refreshes the display every two seconds, so you can watch the handshake appear the moment the first connection is made.

#Closing

Episode 5 dissected the life cycle of a WireGuard session: the one-round-trip Noise IK handshake, the cookie mechanism for holding back handshake floods, automatic rekeying, and the state transition map that runs inside the kernel.

Key takeaways:

• The handshake uses three messages in one round trip (Noise IKpsk2).
• A zero latest-handshakes means no handshake has ever happened.
• The cookie mechanism dampens handshake flood attacks.
• Session keys are rekeyed roughly every 2 minutes.
• Keepalives are sent every few seconds while a session is idle.
• Diagnosing handshakes always starts with the endpoint, firewall, and public keys.

In episode 6 we cover routing and allowed IPs — why AllowedIPs is not a firewall, how to use it for split tunneling, and how wg-quick interacts with ip route to build tunnel routes automatically.