Archived

This repository has been archived on 2026-08-17. You can view files and clone it. You cannot open issues or pull requests or push a commit.

Files

T

beatzaplenty ba73420350 Document pve-test's wifi-primary network and add Claude node guardrails

pve-test was briefly clustered with pve1 then deliberately de-clustered
so it could move to wifi-primary networking (4addr bridge mode, bonded
with a wired LAN backup) - a change not achievable while clustered given
corosync's latency requirements. Captures that as a reproducible script
plus docs: cluster separation procedure, the wifi network design and the
live-cutover pitfalls hit along the way, and node-role/history context.

Also adds CLAUDE.md guardrails for pve1 (production) vs pve-test
(sandbox) - this repo had none before, despite scripts here being able
to make real changes to both.

Separately: both nodes' mgmt firewalls were dropping ICMP by default
(TCP 22/8006 only), which looked like an outage mid-troubleshooting even
though SSH/web UI were fine. Added an explicit ping-allow rule to the
firewall template, applied it live on both nodes, and added an audit.sh
check so it stays enforced.

2026-07-21 21:20:27 +00:00

2.5 KiB

Raw Permalink Blame History

Networking

pve1 as built (Stage 1, current)

Single NIC (2.5GbE), single bridge vmbr0 on the flat LAN (192.168.2.0/24), no VLANs. There is no corosync or Ceph traffic to separate yet — this node isn't clustered. Segmentation for Stage 1 is done at the firewall, not the network: scripts/deploy-firewall.sh restricts SSH (22) and the web UI (8006) to the management CIDR via the PVE datacenter firewall (default-deny inbound otherwise). That's sufficient until Stage 2 needs actual separate physical/VLAN paths for corosync and Ceph traffic — see below.

pve-test as built (sandbox, current)

Different node, different design, not a Stage 1/2 example to generalize from: pve-test runs vmbr0 bridged over a wifi NIC in 4addr client mode (active-backup bonded with a wired NIC as an automatic fallback). Full detail, including why this is normally impossible and how it was validated before trusting it with the management IP, in 06-pve-test-wifi-network.md. This is intentionally a one-off for a standalone sandbox box — never extend it to a node that's clustered or Ceph-connected (see the Stage 2 note in 00-overview.md).

Target design (Stage 2, future cluster)

Required separation

Keep these on logically separate networks/VLANs, ideally separate NICs:

Management — web UI (8006), SSH
Corosync — cluster quorum. Low, consistent latency (well under 5ms) matters more than bandwidth. Never share with VM/storage traffic.
Ceph public — VM-to-OSD traffic (once Ceph is live)
Ceph cluster/backend — OSD-to-OSD replication, heaviest load

Practical layout

2x 10/25GbE bonded or split:

Link pair A → Ceph (public + backend, or split further if 4 NICs available)
Link pair B → management + corosync + VM traffic, with corosync on its own VLAN even when sharing a physical NIC with the rest

Cluster join requirements

All nodes reachable to each other on SSH (22) and the corosync network
Same PVE version across nodes
NTP-synced clocks

Firewall

Proxmox's built-in firewall operates at datacenter and node level. Default-deny, then whitelist:

SSH from the management network/VLAN only
Web UI (8006) from the management network/VLAN only
Corosync ports between cluster nodes
Ceph ports between cluster nodes (once Ceph is live)

Enforce the network separation above at the firewall — corosync and Ceph traffic shouldn't be reachable from the VM network even if they end up sharing a physical link.

See config/pve-firewall/ for a starting rule set.

2.5 KiB Raw Permalink Blame History