Archived
refactor(ha): renumber cluster→VLAN10, storage→VLAN20; iSCSI/NFS protocol separation
Check NixOS configurations / eval-hosts (push) Failing after 17m31s
Check NixOS configurations / eval-hosts (push) Failing after 17m31s
Renumber internal HA bridges so VLAN ID matches IP third octet: vmbr1 (cluster): 192.168.4.x → 192.168.10.x (VLAN 10, /29) vmbr2 (storage): 192.168.5.x → 192.168.20.x (VLAN 20, /24) Enforce protocol separation at the firewall on both HA nodes: - iSCSI (3260/tcp): storage-client subnet (192.168.20.0/24) only - NFS (111,2049,20048 tcp+udp): LAN subnet (192.168.2.0/24) only - Cluster subnet (192.168.10.224/29) accepted wholesale (DRBD+Corosync) Moves from allowedTCPPorts/allowedUDPPorts to source-restricted extraCommands rules on the nixos-fw chain. NFS exports reduced to LAN-only (drop haClientCidr lines from mkNfsExports). Storage-client network clients use iSCSI only; no NFS on VLAN 20. All HA script VIP/storage IP defaults updated to match. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J8djTWdXVzXZc99iujU6T2
This commit is contained in:
+28
-22
@@ -2,17 +2,22 @@
|
|||||||
|
|
||||||
## Subnets
|
## Subnets
|
||||||
|
|
||||||
| Subnet | CIDR | Purpose | Routed? |
|
| Subnet | VLAN | CIDR | Purpose | Routed? |
|
||||||
|---|---|---|---|
|
|---|---|---|---|---|
|
||||||
| LAN | `192.168.2.0/24` | General LAN — clients and infrastructure | Yes (gateway .254) |
|
| LAN | 2 (native/untagged) | `192.168.2.0/24` | General LAN — clients and infrastructure | Yes (gateway .254) |
|
||||||
| Cluster | `192.168.4.224/29` | HA file server DRBD replication + Corosync heartbeat | No — internal `vmbr1` only, no uplink |
|
| Cluster | 10 | `192.168.10.224/29` | HA file server DRBD replication + Corosync heartbeat | No — internal `vmbr1` only, no uplink |
|
||||||
| Storage client | `192.168.5.0/24` | HA file server iSCSI + NFS — docker and server access VIP here | No — internal `vmbr2` only, no uplink |
|
| Storage client | 20 | `192.168.20.0/24` | HA file server iSCSI — docker and server access VIP here | No — internal `vmbr2` only, no uplink |
|
||||||
|
|
||||||
The cluster and storage-client subnets never leave pve1. `vmbr1` and `vmbr2` are Proxmox Linux
|
The cluster and storage-client subnets never leave pve1. `vmbr1` and `vmbr2` are Proxmox Linux
|
||||||
bridges with no physical port attached; traffic between guests on each bridge stays in-kernel.
|
bridges with no physical port attached; traffic between guests on each bridge stays in-kernel.
|
||||||
|
|
||||||
The host octet is consistent across all subnets — e.g. ha-node1 is always `.228`:
|
VLAN IDs match the third octet of each subnet (VLAN 2 → 192.168.**2**.x, VLAN 10 → 192.168.**10**.x,
|
||||||
`192.168.2.228` (LAN), `192.168.4.228` (cluster), `192.168.5.228` (storage client).
|
VLAN 20 → 192.168.**20**.x). The host octet is consistent across all subnets — e.g. ha-node1
|
||||||
|
is always `.228`: `192.168.2.228` (LAN), `192.168.10.228` (cluster), `192.168.20.228` (storage client).
|
||||||
|
|
||||||
|
**Protocol separation** (enforced by firewall on HA nodes):
|
||||||
|
- iSCSI (port 3260): storage-client subnet (VLAN 20) only — docker and server
|
||||||
|
- NFS (ports 111, 2049, 20048): LAN subnet (VLAN 2) only — pxe-boot and other LAN clients
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -79,14 +84,14 @@ MAC address churn.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Cluster network — 192.168.4.224/29
|
## Cluster network — VLAN 10 — 192.168.10.224/29
|
||||||
|
|
||||||
Internal to pve1 only. Proxmox bridge `vmbr1`, no physical NIC attached.
|
Internal to pve1 only. Proxmox bridge `vmbr1`, no physical NIC attached.
|
||||||
|
|
||||||
| IP | Hostname | Interface role |
|
| IP | Hostname | Interface role |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| `192.168.4.228` | ha-node1 | DRBD replication + Corosync ring0 (primary heartbeat) |
|
| `192.168.10.228` | ha-node1 | DRBD replication + Corosync ring0 (primary heartbeat) |
|
||||||
| `192.168.4.227` | ha-node2 | DRBD replication + Corosync ring0 (primary heartbeat) |
|
| `192.168.10.227` | ha-node2 | DRBD replication + Corosync ring0 (primary heartbeat) |
|
||||||
| — | no gateway | Isolated — not routed to LAN or internet |
|
| — | no gateway | Isolated — not routed to LAN or internet |
|
||||||
|
|
||||||
Corosync ring1 (backup heartbeat only) uses the LAN IPs (`192.168.2.228` / `192.168.2.227`)
|
Corosync ring1 (backup heartbeat only) uses the LAN IPs (`192.168.2.228` / `192.168.2.227`)
|
||||||
@@ -94,21 +99,22 @@ over `vmbr0` — no additional bridge needed, and DRBD traffic never crosses rin
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Storage-client network — 192.168.5.0/24
|
## Storage-client network — VLAN 20 — 192.168.20.0/24
|
||||||
|
|
||||||
Internal to pve1 only. Proxmox bridge `vmbr2`, no physical NIC attached.
|
Internal to pve1 only. Proxmox bridge `vmbr2`, no physical NIC attached.
|
||||||
|
|
||||||
| IP | Hostname | Interface / role |
|
| IP | Hostname | Interface / role |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| `192.168.5.229` | ha-vip | Pacemaker floating VIP — iSCSI portal + NFS endpoint |
|
| `192.168.20.229` | ha-vip-storage | Pacemaker floating VIP — iSCSI endpoint (port 3260) |
|
||||||
| `192.168.5.228` | ha-node1 | Storage-client NIC (ens20 / vmbr2) |
|
| `192.168.20.228` | ha-node1 | Storage-client NIC (ens20 / vmbr2) |
|
||||||
| `192.168.5.227` | ha-node2 | Storage-client NIC (ens20 / vmbr2) |
|
| `192.168.20.227` | ha-node2 | Storage-client NIC (ens20 / vmbr2) |
|
||||||
| `192.168.5.226` | server | Storage-client NIC (ens19 / vmbr2) — for final rsync before cutover |
|
| `192.168.20.226` | server | Storage-client NIC (ens19 / vmbr2) — temporary, for final rsync before decommission |
|
||||||
| `192.168.5.225` | docker | Storage-client NIC (eth1 / vmbr2) — iSCSI initiator + NFS client |
|
| `192.168.20.225` | docker | Storage-client NIC (eth1 / vmbr2) — iSCSI initiator |
|
||||||
| — | no gateway | Isolated — not routed to LAN or internet |
|
| — | no gateway | Isolated — not routed to LAN or internet |
|
||||||
|
|
||||||
iSCSI initiators and NFS clients connect exclusively to the VIP (`192.168.5.229`) so sessions
|
iSCSI initiators connect to the VIP (`192.168.20.229:3260`) so sessions survive failover
|
||||||
survive failover transparently. The portal is bound to the VIP only (not `[::0]`).
|
transparently. Portal listens on `[::0]:3260`; firewall restricts port 3260 to
|
||||||
|
`192.168.20.0/24` source only — LAN clients cannot reach iSCSI.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -123,10 +129,10 @@ Current → target IP for every host being renumbered.
|
|||||||
| pve1 | `192.168.2.250` | `192.168.2.245` | `/etc/network/interfaces` on Proxmox host |
|
| pve1 | `192.168.2.250` | `192.168.2.245` | `/etc/network/interfaces` on Proxmox host |
|
||||||
| pbs | `192.168.2.108` | `192.168.2.244` | static config on PBS host |
|
| pbs | `192.168.2.108` | `192.168.2.244` | static config on PBS host |
|
||||||
| nixos workstation | `192.168.2.119` | `192.168.2.243` | `networking.interfaces` / NetworkManager on guest |
|
| nixos workstation | `192.168.2.119` | `192.168.2.243` | `networking.interfaces` / NetworkManager on guest |
|
||||||
| ha-node1 | — | `192.168.2.228` (LAN), `192.168.4.228` (cluster), `192.168.5.228` (storage) | active |
|
| ha-node1 | — | `192.168.2.228` (LAN), `192.168.10.228` (cluster/VLAN 10), `192.168.20.228` (storage/VLAN 20) | active |
|
||||||
| ha-node2 | — | `192.168.2.227` (LAN), `192.168.4.227` (cluster), `192.168.5.227` (storage) | active |
|
| ha-node2 | — | `192.168.2.227` (LAN), `192.168.10.227` (cluster/VLAN 10), `192.168.20.227` (storage/VLAN 20) | active |
|
||||||
| ha-vip-lan | — | `192.168.2.229` (vmbr0 / Pacemaker `vip-lan`) — LAN iSCSI + NFS | active |
|
| ha-vip-lan | — | `192.168.2.229` (vmbr0 / Pacemaker `vip-lan`) — NFS endpoint for LAN clients | active |
|
||||||
| ha-vip-storage | — | `192.168.5.229` (vmbr2 / Pacemaker `vip-storage`) — storage-client iSCSI + NFS | active |
|
| ha-vip-storage | — | `192.168.20.229` (vmbr2 / Pacemaker `vip-storage`) — iSCSI endpoint for VLAN 20 clients | active |
|
||||||
| server | `192.168.2.252` | `192.168.2.226` | static config on guest |
|
| server | `192.168.2.252` | `192.168.2.226` | static config on guest |
|
||||||
| docker | `192.168.2.249` | `192.168.2.225` | static config on guest |
|
| docker | `192.168.2.249` | `192.168.2.225` | static config on guest |
|
||||||
| nix-cache | `192.168.2.120` | `192.168.2.224` | static config on guest |
|
| nix-cache | `192.168.2.120` | `192.168.2.224` | static config on guest |
|
||||||
|
|||||||
@@ -17,14 +17,12 @@
|
|||||||
{ lib, pkgs, vars, ... }:
|
{ lib, pkgs, vars, ... }:
|
||||||
|
|
||||||
let
|
let
|
||||||
# Generates /etc/exports lines for all nfsShares data entries. Each share
|
# Generates /etc/exports lines for all nfsShares data entries.
|
||||||
# gets two export lines — one for LAN clients (pxe-boot, proxmox host) and
|
# NFS is LAN-only (VLAN 2). Storage-client subnet (VLAN 20) uses iSCSI only.
|
||||||
# one for storage-client-subnet clients (docker, server via vmbr2).
|
|
||||||
mkNfsExports = storageRoot:
|
mkNfsExports = storageRoot:
|
||||||
lib.concatMapStrings
|
lib.concatMapStrings
|
||||||
(share:
|
(share:
|
||||||
" ${storageRoot}/${share.subpath} ${vars.lanCidr}${vars.nfsShares.options}\n" +
|
" ${storageRoot}/${share.subpath} ${vars.lanCidr}${vars.nfsShares.options}\n")
|
||||||
" ${storageRoot}/${share.subpath} ${vars.haClientCidr}${vars.nfsShares.options}\n")
|
|
||||||
(lib.filter builtins.isAttrs (lib.attrValues vars.nfsShares));
|
(lib.filter builtins.isAttrs (lib.attrValues vars.nfsShares));
|
||||||
in
|
in
|
||||||
{
|
{
|
||||||
|
|||||||
@@ -131,27 +131,31 @@ in
|
|||||||
|
|
||||||
networking.firewall = {
|
networking.firewall = {
|
||||||
allowedTCPPorts = [
|
allowedTCPPorts = [
|
||||||
vars.ports.haServerIscsi
|
|
||||||
vars.ports.haServerPacemakerRemoted
|
vars.ports.haServerPacemakerRemoted
|
||||||
vars.ports.haServerPcsd
|
vars.ports.haServerPcsd
|
||||||
vars.ports.haServerDrbd
|
vars.ports.haServerDrbd
|
||||||
vars.ports.nfsRpcbind
|
|
||||||
vars.ports.nfsd
|
|
||||||
vars.ports.nfsMountd
|
|
||||||
];
|
];
|
||||||
allowedUDPPorts = [
|
allowedUDPPorts = [
|
||||||
vars.ports.haServerCorosync1
|
vars.ports.haServerCorosync1
|
||||||
vars.ports.haServerCorosync2
|
vars.ports.haServerCorosync2
|
||||||
vars.ports.haServerCorosyncCrypto
|
vars.ports.haServerCorosyncCrypto
|
||||||
vars.ports.nfsRpcbind
|
|
||||||
vars.ports.nfsd
|
|
||||||
vars.ports.nfsMountd
|
|
||||||
];
|
];
|
||||||
|
# Protocol separation: iSCSI (VLAN 20 / storage clients only),
|
||||||
|
# NFS (VLAN 2 / LAN only). Cluster-internal subnets accepted wholesale
|
||||||
|
# since they are isolated bridges with no external uplink.
|
||||||
extraCommands = ''
|
extraCommands = ''
|
||||||
iptables -A INPUT -s ${vars.haServer1Ip}/32 -j ACCEPT
|
iptables -A nixos-fw -s ${vars.haServer1Ip}/32 -j nixos-fw-accept
|
||||||
iptables -A INPUT -s ${vars.haServer2Ip}/32 -j ACCEPT
|
iptables -A nixos-fw -s ${vars.haServer2Ip}/32 -j nixos-fw-accept
|
||||||
iptables -A INPUT -s ${vars.haStorageCidr} -j ACCEPT
|
iptables -A nixos-fw -s ${vars.haStorageCidr} -j nixos-fw-accept
|
||||||
iptables -A INPUT -s ${vars.haClientCidr} -j ACCEPT
|
|
||||||
|
iptables -A nixos-fw -p tcp -s ${vars.haClientCidr} --dport ${toString vars.ports.haServerIscsi} -j nixos-fw-accept
|
||||||
|
|
||||||
|
iptables -A nixos-fw -p tcp -s ${vars.lanCidr} --dport ${toString vars.ports.nfsRpcbind} -j nixos-fw-accept
|
||||||
|
iptables -A nixos-fw -p udp -s ${vars.lanCidr} --dport ${toString vars.ports.nfsRpcbind} -j nixos-fw-accept
|
||||||
|
iptables -A nixos-fw -p tcp -s ${vars.lanCidr} --dport ${toString vars.ports.nfsd} -j nixos-fw-accept
|
||||||
|
iptables -A nixos-fw -p udp -s ${vars.lanCidr} --dport ${toString vars.ports.nfsd} -j nixos-fw-accept
|
||||||
|
iptables -A nixos-fw -p tcp -s ${vars.lanCidr} --dport ${toString vars.ports.nfsMountd} -j nixos-fw-accept
|
||||||
|
iptables -A nixos-fw -p udp -s ${vars.lanCidr} --dport ${toString vars.ports.nfsMountd} -j nixos-fw-accept
|
||||||
'';
|
'';
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -12,7 +12,7 @@ NODE1="${NODE1:-ha-server-1}"
|
|||||||
NODE2="${NODE2:-ha-server-2}"
|
NODE2="${NODE2:-ha-server-2}"
|
||||||
NODE1_IP="${NODE1_IP:-192.168.2.228}" # vars.haServer1Ip
|
NODE1_IP="${NODE1_IP:-192.168.2.228}" # vars.haServer1Ip
|
||||||
NODE2_IP="${NODE2_IP:-192.168.2.227}" # vars.haServer2Ip
|
NODE2_IP="${NODE2_IP:-192.168.2.227}" # vars.haServer2Ip
|
||||||
VIP="${VIP:-192.168.5.229}" # vars.haServerVip
|
VIP="${VIP:-192.168.20.229}" # vars.haServerVip
|
||||||
XFS_MOUNT="${XFS_MOUNT:-/srv/ha-data}" # vars.haStorageRoot
|
XFS_MOUNT="${XFS_MOUNT:-/srv/ha-data}" # vars.haStorageRoot
|
||||||
ISCSI_IQN="${ISCSI_IQN:-iqn.2026-01.home.sweet:ha-storage}" # vars.haIscsiIqn
|
ISCSI_IQN="${ISCSI_IQN:-iqn.2026-01.home.sweet:ha-storage}" # vars.haIscsiIqn
|
||||||
# ──────────────────────────────────────────────────────────────────────────
|
# ──────────────────────────────────────────────────────────────────────────
|
||||||
|
|||||||
@@ -26,7 +26,7 @@ NODE1="${NODE1:-ha-server-1}"
|
|||||||
NODE2="${NODE2:-ha-server-2}"
|
NODE2="${NODE2:-ha-server-2}"
|
||||||
NODE1_IP="${NODE1_IP:-192.168.2.228}" # vars.haServer1Ip
|
NODE1_IP="${NODE1_IP:-192.168.2.228}" # vars.haServer1Ip
|
||||||
NODE2_IP="${NODE2_IP:-192.168.2.227}" # vars.haServer2Ip
|
NODE2_IP="${NODE2_IP:-192.168.2.227}" # vars.haServer2Ip
|
||||||
VIP="${VIP:-192.168.5.229}" # vars.haServerVip (storage-client, vmbr2)
|
VIP="${VIP:-192.168.20.229}" # vars.haServerVip (storage-client, vmbr2, VLAN 20)
|
||||||
VIP_LAN="${VIP_LAN:-192.168.2.229}" # vars.haServerLanVip (LAN, vmbr0)
|
VIP_LAN="${VIP_LAN:-192.168.2.229}" # vars.haServerLanVip (LAN, vmbr0)
|
||||||
XFS_MOUNT="${XFS_MOUNT:-/srv/ha-data}" # vars.haStorageRoot
|
XFS_MOUNT="${XFS_MOUNT:-/srv/ha-data}" # vars.haStorageRoot
|
||||||
ISCSI_IQN="${ISCSI_IQN:-iqn.2026-01.home.sweet:ha-storage}" # vars.haIscsiIqn
|
ISCSI_IQN="${ISCSI_IQN:-iqn.2026-01.home.sweet:ha-storage}" # vars.haIscsiIqn
|
||||||
@@ -419,7 +419,7 @@ cibadmin --replace --scope resources --xml-text '<resources>
|
|||||||
</primitive>
|
</primitive>
|
||||||
<primitive id="vip-storage" class="ocf" type="IPaddr2" provider="heartbeat">
|
<primitive id="vip-storage" class="ocf" type="IPaddr2" provider="heartbeat">
|
||||||
<instance_attributes id="vip-storage-attrs">
|
<instance_attributes id="vip-storage-attrs">
|
||||||
<nvpair id="vip-storage-ip" name="ip" value="192.168.5.229"/>
|
<nvpair id="vip-storage-ip" name="ip" value="192.168.20.229"/>
|
||||||
<nvpair id="vip-storage-cidr" name="cidr_netmask" value="24"/>
|
<nvpair id="vip-storage-cidr" name="cidr_netmask" value="24"/>
|
||||||
<nvpair id="vip-storage-nic" name="nic" value="ens20"/>
|
<nvpair id="vip-storage-nic" name="nic" value="ens20"/>
|
||||||
</instance_attributes>
|
</instance_attributes>
|
||||||
|
|||||||
@@ -84,9 +84,9 @@ NODE1_HOST="ha-server-1"
|
|||||||
NODE2_HOST="ha-server-2"
|
NODE2_HOST="ha-server-2"
|
||||||
NODE1_IP="192.168.2.228"
|
NODE1_IP="192.168.2.228"
|
||||||
NODE2_IP="192.168.2.227"
|
NODE2_IP="192.168.2.227"
|
||||||
STORAGE_IP1="192.168.4.228"
|
STORAGE_IP1="192.168.10.228"
|
||||||
STORAGE_IP2="192.168.4.227"
|
STORAGE_IP2="192.168.10.227"
|
||||||
STORAGE_CIDR="192.168.4.0/29"
|
STORAGE_CIDR="192.168.10.224/29"
|
||||||
SSH_USER="${PROXMOX_SSH_USER:-wayne}"
|
SSH_USER="${PROXMOX_SSH_USER:-wayne}"
|
||||||
|
|
||||||
# ── Argument parsing ──────────────────────────────────────────────────────────
|
# ── Argument parsing ──────────────────────────────────────────────────────────
|
||||||
@@ -452,7 +452,7 @@ if ! $SKIP_CLUSTER_INIT; then
|
|||||||
ssh -i ~/.ssh/id_ed25519 -o StrictHostKeyChecking=no "${HA_USER}@${NODE1_IP}" \
|
ssh -i ~/.ssh/id_ed25519 -o StrictHostKeyChecking=no "${HA_USER}@${NODE1_IP}" \
|
||||||
"sudo env NODE1=${NODE1_HOST} NODE2=${NODE2_HOST} \
|
"sudo env NODE1=${NODE1_HOST} NODE2=${NODE2_HOST} \
|
||||||
NODE1_IP=${NODE1_IP} NODE2_IP=${NODE2_IP} \
|
NODE1_IP=${NODE1_IP} NODE2_IP=${NODE2_IP} \
|
||||||
VIP=192.168.5.229 XFS_MOUNT=/srv/ha-data \
|
VIP=192.168.20.229 XFS_MOUNT=/srv/ha-data \
|
||||||
ISCSI_IQN=iqn.2026-01.home.sweet:ha-storage \
|
ISCSI_IQN=iqn.2026-01.home.sweet:ha-storage \
|
||||||
VMID_NODE1=${VMID1} VMID_NODE2=${VMID2} \
|
VMID_NODE1=${VMID1} VMID_NODE2=${VMID2} \
|
||||||
HA_USER=${HA_USER} HA_KEY=/root/.ssh/cluster-init-key \
|
HA_USER=${HA_USER} HA_KEY=/root/.ssh/cluster-init-key \
|
||||||
@@ -491,7 +491,7 @@ if ! $SKIP_TESTS; then
|
|||||||
else
|
else
|
||||||
NODE1="$NODE1_HOST" NODE2="$NODE2_HOST" \
|
NODE1="$NODE1_HOST" NODE2="$NODE2_HOST" \
|
||||||
NODE1_IP="$NODE1_IP" NODE2_IP="$NODE2_IP" \
|
NODE1_IP="$NODE1_IP" NODE2_IP="$NODE2_IP" \
|
||||||
VIP="192.168.5.229" \
|
VIP="192.168.20.229" \
|
||||||
bash "${REPO_ROOT}/scripts/ha/acceptance-tests.sh"
|
bash "${REPO_ROOT}/scripts/ha/acceptance-tests.sh"
|
||||||
fi
|
fi
|
||||||
fi
|
fi
|
||||||
|
|||||||
@@ -19,7 +19,7 @@ NODE1="${NODE1:-ha-server-1}"
|
|||||||
NODE2="${NODE2:-ha-server-2}"
|
NODE2="${NODE2:-ha-server-2}"
|
||||||
NODE1_IP="${NODE1_IP:-192.168.2.228}"
|
NODE1_IP="${NODE1_IP:-192.168.2.228}"
|
||||||
NODE2_IP="${NODE2_IP:-192.168.2.227}"
|
NODE2_IP="${NODE2_IP:-192.168.2.227}"
|
||||||
VIP="${VIP:-192.168.5.229}"
|
VIP="${VIP:-192.168.20.229}"
|
||||||
XFS_MOUNT="${XFS_MOUNT:-/srv/ha-data}"
|
XFS_MOUNT="${XFS_MOUNT:-/srv/ha-data}"
|
||||||
HA_USER="${HA_USER:-nixos}"
|
HA_USER="${HA_USER:-nixos}"
|
||||||
# ──────────────────────────────────────────────────────────────────────────
|
# ──────────────────────────────────────────────────────────────────────────
|
||||||
|
|||||||
@@ -11,7 +11,7 @@ NODE1="${NODE1:-ha-server-1}"
|
|||||||
NODE2="${NODE2:-ha-server-2}"
|
NODE2="${NODE2:-ha-server-2}"
|
||||||
NODE1_IP="${NODE1_IP:-192.168.2.228}" # vars.haServer1Ip
|
NODE1_IP="${NODE1_IP:-192.168.2.228}" # vars.haServer1Ip
|
||||||
NODE2_IP="${NODE2_IP:-192.168.2.227}" # vars.haServer2Ip
|
NODE2_IP="${NODE2_IP:-192.168.2.227}" # vars.haServer2Ip
|
||||||
VIP="${VIP:-192.168.5.229}" # vars.haServerVip
|
VIP="${VIP:-192.168.20.229}" # vars.haServerVip
|
||||||
XFS_MOUNT="${XFS_MOUNT:-/srv/ha-data}" # vars.haStorageRoot
|
XFS_MOUNT="${XFS_MOUNT:-/srv/ha-data}" # vars.haStorageRoot
|
||||||
HA_USER="${HA_USER:-nixos}"
|
HA_USER="${HA_USER:-nixos}"
|
||||||
# ──────────────────────────────────────────────────────────────────────────
|
# ──────────────────────────────────────────────────────────────────────────
|
||||||
|
|||||||
+17
-15
@@ -103,27 +103,29 @@
|
|||||||
|
|
||||||
# HA file server cluster
|
# HA file server cluster
|
||||||
# LAN IPs (vmbr0 / ens18) — management only after storage migration.
|
# LAN IPs (vmbr0 / ens18) — management only after storage migration.
|
||||||
# Cluster IPs (vmbr1 / ens19) — isolated internal bridge, DRBD replication
|
# Cluster IPs (vmbr1 / ens19) — VLAN 10 (192.168.10.x), isolated internal bridge,
|
||||||
# and Corosync heartbeat only; never leaves pve1.
|
# DRBD replication and Corosync heartbeat only; never leaves pve1.
|
||||||
# Storage-client IPs (vmbr2 / ens20) — isolated internal bridge for iSCSI
|
# Storage-client IPs (vmbr2 / ens20) — VLAN 20 (192.168.20.x), isolated internal
|
||||||
# and NFS; docker and server VMs connect here instead of crossing vmbr0.
|
# bridge for iSCSI; docker and server VMs connect here instead of crossing vmbr0.
|
||||||
# haServerVip: floating virtual IP on vmbr2, managed by Pacemaker IPaddr2;
|
# haServerVip: floating virtual IP on vmbr2, managed by Pacemaker IPaddr2;
|
||||||
# all iSCSI and NFS clients connect here regardless of which node is Active.
|
# iSCSI clients connect here regardless of which node is Active.
|
||||||
|
# Protocol separation: iSCSI on storage-client subnet (VLAN 20) only;
|
||||||
|
# NFS on LAN subnet (VLAN 2) only. Enforced by firewall on the HA nodes.
|
||||||
haServer1Host = "ha-server-1";
|
haServer1Host = "ha-server-1";
|
||||||
haServer2Host = "ha-server-2";
|
haServer2Host = "ha-server-2";
|
||||||
haServer1Ip = "192.168.2.228"; # LAN IP, node 1 (vmbr0 / ens18)
|
haServer1Ip = "192.168.2.228"; # LAN IP, node 1 (vmbr0 / ens18)
|
||||||
haServer2Ip = "192.168.2.227"; # LAN IP, node 2 (vmbr0 / ens18)
|
haServer2Ip = "192.168.2.227"; # LAN IP, node 2 (vmbr0 / ens18)
|
||||||
haServer1StorageIp = "192.168.4.228"; # cluster-net IP, node 1 (vmbr1 / ens19)
|
haServer1StorageIp = "192.168.10.228"; # cluster-net IP, node 1 (vmbr1 / ens19, VLAN 10)
|
||||||
haServer2StorageIp = "192.168.4.227"; # cluster-net IP, node 2 (vmbr1 / ens19)
|
haServer2StorageIp = "192.168.10.227"; # cluster-net IP, node 2 (vmbr1 / ens19, VLAN 10)
|
||||||
haStorageCidr = "192.168.4.224/29"; # cluster subnet — internal to pve1 only
|
haStorageCidr = "192.168.10.224/29"; # cluster subnet — VLAN 10, internal to pve1 only
|
||||||
haStoragePrefixLength = 29; # cluster subnet prefix length (/29)
|
haStoragePrefixLength = 29; # cluster subnet prefix length (/29)
|
||||||
haServer1ClientIp = "192.168.5.228"; # storage-client IP, node 1 (vmbr2 / ens20)
|
haServer1ClientIp = "192.168.20.228"; # storage-client IP, node 1 (vmbr2 / ens20, VLAN 20)
|
||||||
haServer2ClientIp = "192.168.5.227"; # storage-client IP, node 2 (vmbr2 / ens20)
|
haServer2ClientIp = "192.168.20.227"; # storage-client IP, node 2 (vmbr2 / ens20, VLAN 20)
|
||||||
haServerVip = "192.168.5.229"; # storage-client floating VIP on vmbr2 (Pacemaker IPaddr2 vip-storage)
|
haServerVip = "192.168.20.229"; # storage-client floating VIP on vmbr2 (Pacemaker IPaddr2 vip-storage, VLAN 20)
|
||||||
haServerLanVip = "192.168.2.229"; # LAN floating VIP on vmbr0 (Pacemaker IPaddr2 vip-lan) — LAN iSCSI/NFS access
|
haServerLanVip = "192.168.2.229"; # LAN floating VIP on vmbr0 (Pacemaker IPaddr2 vip-lan) — NFS access
|
||||||
dockerStorageIp = "192.168.5.225"; # docker CT storage-client IP (vmbr2 / eth1)
|
dockerStorageIp = "192.168.20.225"; # docker CT storage-client IP (vmbr2 / eth1, VLAN 20)
|
||||||
serverStorageIp = "192.168.5.226"; # server VM storage-client IP (vmbr2 / ens19)
|
serverStorageIp = "192.168.20.226"; # server VM storage-client IP (vmbr2 / ens19, VLAN 20) — temporary, for final rsync
|
||||||
haClientCidr = "192.168.5.0/24"; # storage-client subnet — internal to pve1 only
|
haClientCidr = "192.168.20.0/24"; # storage-client subnet — VLAN 20, internal to pve1 only
|
||||||
haClientPrefixLength = 24; # storage-client subnet prefix length (/24)
|
haClientPrefixLength = 24; # storage-client subnet prefix length (/24)
|
||||||
haStorageRoot = "/srv/ha-data"; # XFS-over-DRBD mount point on the Active node
|
haStorageRoot = "/srv/ha-data"; # XFS-over-DRBD mount point on the Active node
|
||||||
haIscsiIqn = "iqn.2026-01.home.sweet:ha-storage";
|
haIscsiIqn = "iqn.2026-01.home.sweet:ha-storage";
|
||||||
|
|||||||
Reference in New Issue
Block a user