Archived
Initial planning docs and hardening scripts for HA rebuild
Covers node 1 hardware/network layout, LVM-thin -> ZFS migration path, Ceph as the future HA storage upgrade, and baseline SSH/firewall hardening.
This commit is contained in:
@@ -0,0 +1,57 @@
|
||||
# Overview & Roadmap
|
||||
|
||||
## Background
|
||||
|
||||
Current hardware requires disabling KVM hardware virtualization for VMs to
|
||||
start at all (falls back to software emulation — slow). This is a
|
||||
host/BIOS-level issue, not a Proxmox limitation, and shouldn't be needed on
|
||||
new hardware. Before reusing that workaround on the new box:
|
||||
|
||||
- Confirm VT-x (Intel) / AMD-V (AMD) is enabled in BIOS/UEFI.
|
||||
- Confirm IOMMU is enabled if passthrough is planned.
|
||||
- Update BIOS/microcode first.
|
||||
- Rule out running Proxmox nested inside another hypervisor.
|
||||
|
||||
## End goal
|
||||
|
||||
3-node Proxmox VE cluster with HA-managed VMs. Two ways to get VM disks
|
||||
available on more than one node (see `02-storage-zfs-ceph.md`):
|
||||
|
||||
1. **Ceph** — true distributed shared storage, sync replication, needs 3+
|
||||
nodes and a fast dedicated network. Zero/near-zero RPO on failover.
|
||||
2. **ZFS + storage replication** — local ZFS pool per node, Proxmox
|
||||
replicates VM disks between nodes on a schedule (as often as every
|
||||
minute). Lighter weight, async — failover loses whatever changed since
|
||||
last replication.
|
||||
|
||||
Starting point: ZFS + replication (lighter, works from node 1 onward).
|
||||
Ceph is the upgrade path once 3 nodes exist and/or zero-RPO failover
|
||||
matters enough to justify the overhead.
|
||||
|
||||
## Cluster fundamentals (apply from node 1 onward)
|
||||
|
||||
- 3 nodes minimum for real quorum. If starting with 2, add a QDevice
|
||||
(small VM or Raspberry Pi) as tie-breaker.
|
||||
- Dedicated network for corosync (cluster/quorum traffic) — never shared
|
||||
with VM or storage traffic. Needs low, consistent latency (well under
|
||||
5ms); jitter matters more than bandwidth.
|
||||
- All nodes on the same PVE version, NTP-synced, SSH reachable between
|
||||
nodes.
|
||||
- Set VM CPU type to a portable type (e.g. `x86-64-v2-AES` or `kvm64`)
|
||||
rather than `host` if nodes will ever have different CPUs — needed for
|
||||
clean live migration.
|
||||
|
||||
## Rollout sequence
|
||||
|
||||
1. Build node 1 per `01-hardware-node1.md` — fresh PVE install on ZFS boot
|
||||
mirror, local ZFS "tier 2" pool for VM disks, Ceph-earmarked disks left
|
||||
idle or as a temporary ZFS pool.
|
||||
2. Migrate VMs off old hardware via `vzdump` → copy backups → `qmrestore`
|
||||
onto the new ZFS storage (converts disks to ZVOLs).
|
||||
3. Apply hardening (`04-security-hardening.md`) and networking
|
||||
(`03-networking.md`) before exposing the node beyond the LAN.
|
||||
4. Add nodes 2 and 3 identically (same disk/network layout).
|
||||
5. Join cluster, stand up dedicated corosync network, configure
|
||||
replication and HA groups.
|
||||
6. Optionally migrate the Ceph-earmarked disks from temporary ZFS to real
|
||||
Ceph OSDs once 3 nodes are up.
|
||||
Reference in New Issue
Block a user