Learn Proxmox VE - Proxmox Cluster & High Availability (HA)

#Introduction

Up to this point, each Proxmox node has been running on its own. Episode 13 unites them: forming a cluster so all nodes are managed as a single unit, complete with high availability (HA) that automatically brings VMs back up when a node fails.

We'll form the cluster with pvecm, understand quorum and voting rules, use a QDevice to keep a two-node cluster safe, and then configure the HA manager, groups, and priorities. By the end of the episode, you'll watch VMs automatically move when a node is shut down.

#Forming a Proxmox Cluster

#Combining Nodes with pvecm

A cluster is formed from one initial node, then other nodes join. Create the cluster on the first node:

pvecm create pve-cluster

Then on the second node, join using the first node's address:

pvecm add 192.168.1.10

The pvecm add command connects the node to the existing cluster. After all nodes have joined, the web UI displays them under Datacenter.

#Corosync Communication

Communication between nodes is handled by Corosync — the service that manages cluster messaging. By default Corosync uses multicast, but on networks that don't support multicast, configure unicast mode so it still works. The quality and latency of the Corosync network greatly determine cluster stability.

pvecm status

The pvecm status command shows the configuration, cluster members, and quorum status. Get used to checking this command whenever there's a cluster problem.

#Quorum & Voting Rules

#Preventing Split-Brain

Quorum is the majority rule that determines whether the cluster remains fit to operate. Every node has one vote, and the cluster only works if more than half the votes are present. The goal: to prevent split-brain — the condition where the network is cut and two parts of the cluster both think they're in charge.

3 nodes : needs 2 votes (majority of 2 out of 3)

That's why an odd number of nodes — 3, 5, or 7 — is highly recommended. With just two nodes, when one dies or the network is cut, only one vote remains out of two — not a majority, so the cluster stops. This is a safe state, but it takes services fully down.

#QDevice: The Tiebreaker for 2 Nodes

For a two-node cluster, Proxmox uses a QDevice — an external device that acts as a tiebreaker. With a QDevice, a two-node cluster gets a third vote from the outside device, so when one node fails, the surviving node has two votes (its own + the QDevice) and keeps running the services.

pve-qdevice install --host 192.168.1.10

This command installs the QDevice on the node chosen as the tiebreaker. The QDevice itself can be a small VM or another stable node.

#Configuring High Availability

#Marking a VM as an HA Resource

Once the cluster is standing, mark a VM as an HA resource:

ha-manager add vm:100

The ha-manager add vm:100 command registers VM 100 for management by the HA manager. From that point on, the HA manager ensures VM 100 is always running on one of the healthy nodes.

#HA Groups and Priority

To control where VMs run, create an HA group and set priority:

• HA group: the set of nodes where HA VMs are allowed to run.
• Priority: determines the preference order — VMs with higher priority are started on the more preferred node when the cluster restarts workloads.

ha-manager groupadd prod --nodes node-a,node-b --priority 100

With this group, HA VMs will tend to run on node-a, and move to node-b if node-a fails.

#Simulating Failover

Test the failure scenario by shutting down one of the nodes:

Node A down -> VM 100 restarts automatically on Node B (seconds)

When node A dies, the HA manager detects the lost resource and brings the VM back up on a healthy node within seconds. Check the status with ha-manager status to see the resources and their locations.

#Closing

Episode 13 brought it all together: forming a cluster with pvecm, understanding quorum and the dangers of split-brain, using a QDevice for two-node clusters, and configuring the HA manager, groups, priorities, and failover simulation.

The key takeaways:

• A cluster is formed with pvecm create then pvecm add.
• Majority quorum prevents split-brain; use an odd number of nodes.
• QDevice becomes the tiebreaker for two-node clusters.
• HA resources are managed by ha-manager.
• HA groups and priorities determine where VMs run.
• Test failover regularly, not just when production has problems.

In the next episode, episode 14, we will cover user management, permissions, and two-factor authentication — getting to know authentication realms, RBAC with built-in and custom roles, permission paths, and enabling TOTP and WebAuthn. Your cluster is standing; now it's time to secure the access to it!