Learn L2TP IPsec - IKEv1 & IKEv2

#Introduction

In episode 2 we learned that IKE is responsible for building keys and Security Associations. Now we dissect both of its versions. Episode 4 compares IKEv1, which forms the foundation of classic L2TP/IPsec, with IKEv2, which is more modern, more efficient, and recommended. The choice between them determines the ike= and ikev2= settings in your configuration files.

Why is this topic important? Because many attacks against L2TP/IPsec (covered in episode 13) exploit IKEv1 weaknesses. Understanding the differences lets you decide deliberately: when you are forced to use IKEv1 for compatibility, and when you can raise security with IKEv2.

#IKEv1: Two Exchange Modes

#Main Mode

IKEv1 defines two ways to build the authentication phase. Main Mode uses six messages in three pairs of round-trips: proposal and transform exchange (SA), Diffie-Hellman key exchange with nonces, then an encrypted identity exchange. Because identities are protected, Main Mode is more secure — especially when using certificates, since peer identities do not leak to eavesdroppers.

#Aggressive Mode

Aggressive Mode compresses the process into just three messages. It is faster and uses fewer round-trips, but both parties' identities are sent before the shared key is established — in plaintext. This leaks identities (for example usernames or PSK hashes) to eavesdroppers and becomes the basis for dictionary attacks. With PSK, Aggressive Mode is considered dangerous and should be avoided.

Summary of the comparison:

Main Mode  : 6 messages, encrypted identities, more secure
Aggressive : 3 messages, plaintext identities, faster but leaks

#IKEv1: Phase 1 and Phase 2

#Phase 1: Main SA

IKEv1 works in two phases. Phase 1 builds the IKE SA (often called the ISAKMP SA) — the keys used to protect the exchange itself. The output of Phase 1 is the main SA and the derived key material. The mode used (Main or Aggressive) determines the number of messages.

#Phase 2: Quick Mode

Phase 2 builds the IPsec SA — the keys ESP actually uses to encrypt data. This process is called Quick Mode and uses the keys from Phase 1 to protect the negotiation. Quick Mode also determines parameters such as SPI, ESP algorithms, and new session keys. This IPsec SA is what ends up in the kernel tables and is used for L2TP traffic.

The sequence:

Phase 1: Main/Aggressive Mode -> IKE SA
Phase 2: Quick Mode -> IPsec SA (used by ESP)

#IKEv2: RFC 7296

#Two Lean Exchanges

IKEv2, standardized in RFC 7296, simplifies everything. It only needs IKE_SA_INIT for the proposal and Diffie-Hellman key exchange, then IKE_AUTH for authentication and establishment of the first SA. In total it is usually four messages — fewer than IKEv1's six-message Main Mode, and with no insecure aggressive mode.

IKEv2 also improves NAT detection and support for users behind NAT, handles rekeying more cleanly, and carries built-in MOBIKE (RFC 4555), which lets a connection survive a device changing networks — for example from Wi-Fi to cellular.

#Better Authentication and EAP

IKEv2 supports the Extended Authentication Protocol (EAP) inside the protocol, so RADIUS and LDAP integration is far cleaner than XAUTH in IKEv1. The combination of EAP-MSCHAPv2 or EAP-TLS makes remote access more secure. We will discuss this integration in detail in episode 17.

#Choosing IKE in Configuration

#The ike and ikev2 Values in Libreswan

In Libreswan, the algorithm sequence in ike= determines the suite offered for Phase 1, while ikev2= determines the protocol version. To force IKEv1 (the classic L2TP/IPsec remote access):

conn L2TP-PSK
    ikev2=never
    ike=aes256-sha2;modp2048,aes128-sha1;modp1024
    phase2alg=aes128-sha1

When all clients support IKEv2, prefer:

conn vpn-ikev2
    ikev2=insist
    ike=aes256-sha2;dhgroup14,aes128-sha1;dhgroup14

Note the ikev2=insist value — it forces IKEv2 and rejects IKEv1 connections, the right choice for a modern environment.

#Practical Selection Rules

Use this simple guide:

• Classic mobile clients (iOS, Android, older Windows) using L2TP/IPsec PSK almost always need IKEv1.
• Internal environments where all clients are modern should use IKEv2.
• Never use Aggressive Mode with PSK — choose Main Mode or IKEv2.
• Step down to AES-256 and modp2048 suites, and check client support before dropping older suites.

#Verifying the IKE Version in Use

After a client connects, make sure the negotiation used the version you expect. In Libreswan, ipsec statusall shows the IKE version on the connection detail line:

sudo ipsec statusall | grep -E "ikev|ISAKMP SA"

A line showing IKEv1 or IKEv2 on the SA confirms the agreed version. If the result is not what you expect, recheck the ikev2= and ike= suites on both sides.

#Closing

Episode 4 explained the differences between IKEv1 and IKEv2 thoroughly: the two IKEv1 modes and the risk of Aggressive Mode, the two-phase IKEv1 flow, the advantages of lean IKEv2 with MOBIKE support, and how to choose the ike= and ikev2= values in configuration.

Key takeaways:

• IKEv1 Main Mode uses six messages and protects identities.
• IKEv1 Aggressive Mode uses three messages but leaks identities.
• IKEv1 has two phases: Phase 1 builds the IKE SA, Phase 2 builds the IPsec SA.
• IKEv2 (RFC 7296) only needs IKE_SA_INIT and IKE_AUTH.
• IKEv2 supports MOBIKE and EAP within the protocol.
• In Libreswan, ikev2=never forces IKEv1 and ikev2=insist forces IKEv2.

In the next episode, episode 5, we will discuss transport mode versus tunnel mode — how ESP protects the payload in the two modes, when each is used, and how both appear in the L2TP/IPsec architecture.