Home VirtualizationHow to Build a Proxmox Cluster with 3 Nodes and Enable High Availability

How to Build a Proxmox Cluster with 3 Nodes and Enable High Availability

by Knowledge Base Linux
How to Build a Proxmox Cluster — How to Build a Proxmox Cluster with 3 Nodes and Enable High Availability

A single Proxmox node is a single point of failure. A Proxmox cluster fixes that: group three (or more) nodes together, give them shared storage, and the built-in HA manager will detect a dead host and restart its virtual machines on the survivors automatically. No shared SAN appliance, no extra license — clustering and high availability are part of Proxmox VE itself. This article covers How to Build a Proxmox Cluster in detail.

This guide builds a three-node Proxmox cluster from scratch: preparing the nodes, creating and joining the cluster with pvecm, adding shared storage, testing live migration, and enabling HA with a real failover test. If your nodes are not installed yet, follow our Proxmox VE 9 install guide and the production best practices first — cluster pain usually starts with shortcuts taken there.

How a Proxmox Cluster Works (and Why Three Nodes) – How to Build a Proxmox Cluster

Cluster communication runs on Corosync, which keeps every node’s view of the cluster identical through a replicated configuration filesystem mounted at /etc/pve. Corosync decides by quorum: a node may only run HA workloads if it can see more than half of the cluster’s votes.

That math is why three nodes is the sane minimum. In a two-node cluster, losing one node leaves the survivor with 1 of 2 votes — not a majority — so HA stops exactly when you need it. Workarounds exist (an external QDevice vote on a Raspberry Pi or small VM), but three real nodes give you quorum and spare capacity. Rule of thumb: an N-node cluster that must survive one failure should be sized so all critical VMs fit on N−1 nodes.

One honest expectation before we start: HA restarts VMs on another host after a failure. It is not zero-downtime — there is a crash, a detection delay, and a cold boot, typically 1–3 minutes. Planned work uses live migration instead, which is seamless.

Proxmox cluster failover automatically moving virtual machines to healthy nodes
When a node fails, the HA manager restarts its protected VMs on the remaining healthy nodes.

Prerequisites for a Three-Node Proxmox Cluster – How to Build a Proxmox Cluster

  • Three Proxmox VE nodes on the same version, freshly updated, with static IPs and resolvable hostnames (add every node to /etc/hosts on all nodes).
  • Synchronized clocks on all nodes (chrony) — Corosync and certificates are unforgiving about time skew.
  • Empty joining nodes: nodes 2 and 3 should not host VMs yet. A node that already has guests cannot join cleanly.
  • A dedicated, low-latency network for Corosync if possible (a separate VLAN or direct links). Corosync tolerates a shared LAN in small setups, but it must never compete with backup or storage floods.
  • Shared storage for HA guests: Ceph, NFS, iSCSI/FC, or — with caveats — ZFS replication between nodes. Local-only storage cannot fail over.
  • Similar CPUs across nodes, or a common guest CPU type (e.g. x86-64-v2-AES) so live migration works in both directions.

Step 1: Prepare the Nodes

On every node, update fully and reboot, then add all cluster members to /etc/hosts:

192.168.10.11  pve1.example.com  pve1
192.168.10.12  pve2.example.com  pve2
192.168.10.13  pve3.example.com  pve3

Verify name resolution and time sync from each node (ping pve2, timedatectl). Five minutes of checking here saves hours of Corosync debugging later — most “cluster bugs” are DNS, hosts files or clock skew in a trench coat.

Step 2: Create the Proxmox Cluster on Node 1

In the web UI of the first node: Datacenter → Cluster → Create Cluster, give it a name, and select the network for the first Corosync link (add a second link later for redundancy if you have the interfaces). The CLI equivalent on pve1:

pvecm create kbcluster
pvecm status

pvecm status should show the cluster name, one node, and Quorate: Yes. The cluster configuration now lives in /etc/pve/corosync.conf — replicated automatically, and a file you should edit only through pvecm or the GUI, never by hand on a live system.

Step 3: Join Nodes 2 and 3

The GUI way: on node 1, open Datacenter → Cluster and click Join Information — copy the block. On each joining node, click Join Cluster, paste it, enter node 1’s root password, and join. The shell equivalent, run on the joining node:

pvecm add 192.168.10.11

Confirm the SSH fingerprint when prompted. After both joins, verify from any node:

pvecm status
pvecm nodes

You want three nodes listed, Votes: 3, Quorate: Yes. From this moment all three web interfaces show the same datacenter: log in to any node and you manage them all.

Step 4: Add Shared Storage and Test Live Migration

HA can only restart a VM where its disks are visible. Add your shared storage under Datacenter → Storage (NFS export, iSCSI LUN, or a Ceph RBD pool) and make sure it is enabled on all nodes. Local ZFS pools stay node-local — fine for non-critical guests, invisible to HA.

Create a test VM on the shared storage and live-migrate it while it pings:

qm migrate 100 pve2 --online

If the migration completes without the ping dropping more than a packet or two, your network, CPU compatibility and storage are all proven — the three things HA depends on. (VMs pinned to local disks can still migrate with --with-local-disks, but they cannot be HA-protected in the same way.)

Step 5: Enable High Availability

Proxmox’s HA stack watches nodes with hardware/software watchdogs: a node that stops reporting is fenced (it must reboot before rejoining) so a merely network-partitioned host can never keep writing to shared storage while its twin boots the same VM elsewhere. That safety-first design is why you should never defeat it.

Optionally organize placement first: under Datacenter → HA → Groups (Proxmox VE 9 also adds HA affinity rules) you can prefer certain nodes for certain workloads and restrict others. Then protect a VM — in the GUI (VM → More → Manage HA) or from the shell:

ha-manager add vm:100 --state started --max_restart 3 --max_relocate 2
ha-manager status

The state started means: keep this VM running somewhere in the cluster. max_restart covers in-place restarts (e.g. a corrupted node that still holds quorum), max_relocate covers moving to another node.

Step 6: Test a Real Failover

Do not trust an HA setup you have not broken on purpose. With the protected test VM running on pve1, simulate a genuine crash — pull the power cable or use the BMC/IPMI to cut power. (In a lab only, echo c > /proc/sysrq-trigger forces an instant kernel crash.)

Watch Datacenter → HA or ha-manager status: after the watchdog and quorum timers expire, the VM is marked for recovery and boots on a surviving node. Time it — most clusters land between one and three minutes. Then restore power; the node rejoins, and you can live-migrate the VM home during business hours, not at 3 a.m.

Troubleshooting: Proxmox Cluster FAQ

The cluster lost quorum after I rebooted a node

Check Corosync first: systemctl status corosync and corosync-cfgtool -s on the affected node, then the basics — hosts file entries, switch/VLAN changes, firewall blocking UDP 5405–5412. If one link is flaky, add a second Corosync link on a different network instead of debugging the same wire forever.

“This node already has guests” or join fails midway

Joining nodes must be empty and must not belong to another cluster. If a previous failed join left configuration behind, reinstalling the joining node is honestly the fastest clean fix. Never delete /etc/pve/corosync.conf files by hand on a production cluster to “reset” things.

How do I remove a node from the cluster?

Migrate its guests away, shut it down, then from a remaining node: pvecm delnode pve3. The removed node keeps its local configuration — reinstall it before reusing it as a standalone host, or its stale cluster config will confuse everyone later.

Can I force quorum with pvecm expected 1?

You can, in a genuine emergency, on a cluster you have verified is not split-brained. As a routine fix it is how split-brain data corruption happens. If you find yourself typing it monthly, the real fix is a third node or a QDevice.

Conclusion

A three-node Proxmox cluster with HA is the point where a virtualization setup grows up: quorum-based decisions, shared storage, live migration for planned work, and automatic restarts for unplanned reality. Build it methodically, test the failover while it is still a rehearsal, and keep Proxmox Backup Server in the picture too — HA protects against dead hardware, but only backups protect against dead data.

You may also like

KbLinux

Typically replies within a day

Hello, Welcome to the site. Please click below button for chatting me through Telegram.