beatzaplenty/nixos
Archived
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
This repository has been archived on 2026-07-30. You can view files and clone it. You cannot open issues or pull requests or push a commit.
Files
8c861446946c6498aa3b891d31709c835c32fe33
nixos/docs/ip-addressing.md
T
beatzaplentyandClaude Sonnet 4.6 8c86144694
Check NixOS configurations / eval-hosts (push) Successful in 10m21s
Details
docs(ha): update docs to reflect implemented network segmentation 
ip-addressing.md:
  - Storage-client network purpose updated: NFS primary, iSCSI available
  - Protocol separation box updated: NFS on both VIPs (each CIDR-restricted);
    iSCSI on VLAN 20 only but not in active use
  - docker role updated to NFS client (not iSCSI initiator)
  - vip-storage description updated to NFS + iSCSI endpoint

ha-network-audit.md:
  - Implementation status header added noting all audit items resolved
  - Records decisions that diverged from original recommendations
    (VLAN renumbering, dual VIP, NFS-over-iSCSI for docker/swarm)
  - Notes deferred items (STONITH, iSCSI ACLs)

network-cutover.md:
  - New Stage 6: HA storage cutover — docker NFS remount from
    server.sweet.home to vip-storage (192.168.20.229), final rsync
    steps, and server.sweet.home decommission procedure
  - Former Stage 6 (final cleanup) renumbered to Stage 7

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8djTWdXVzXZc99iujU6T2
2026-07-29 19:04:42 +10:00

8.2 KiB
Raw Blame History

IP Addressing Scheme

Subnets

Subnet VLAN CIDR Purpose Routed?
LAN 2 (native/untagged) 192.168.2.0/24 General LAN — clients and infrastructure Yes (gateway .254)
Cluster 10 192.168.10.224/29 HA file server DRBD replication + Corosync heartbeat No — internal vmbr1 only, no uplink
Storage client 20 192.168.20.0/24 HA file server NFS (and iSCSI if needed) — docker and swarm nodes mount from VIP here No — internal vmbr2 only, no uplink

The cluster and storage-client subnets never leave pve1. vmbr1 and vmbr2 are Proxmox Linux bridges with no physical port attached; traffic between guests on each bridge stays in-kernel.

VLAN IDs match the third octet of each subnet (VLAN 2 → 192.168.2.x, VLAN 10 → 192.168.10.x, VLAN 20 → 192.168.20.x). The host octet is consistent across all subnets — e.g. ha-node1 is always .228: 192.168.2.228 (LAN), 192.168.10.228 (cluster), 192.168.20.228 (storage client).

Protocol separation (enforced by firewall on HA nodes):

  • NFS (ports 111, 2049, 20048): both subnets, each restricted to its own CIDR
    • VLAN 2 only → vip-lan (192.168.2.229) — pxe-boot and other LAN clients
    • VLAN 20 only → vip-storage (192.168.20.229) — docker, future swarm nodes
  • iSCSI (port 3260): VLAN 20 only — available but not in active use; NFS is preferred for multi-host access (shared volumes across a Docker Swarm require a shared filesystem, not per-host block devices)

LAN — 192.168.2.0/24

Address map

Range Purpose
.1–.9 Reserved, never assign
.10–.59 Client DHCP pool (router-assigned)
.60–.219 Unallocated buffer
.220–.229 Virtual nodes (VMs / LXC containers)
.230–.239 Expansion buffer (reserved, unallocated)
.240–.249 Physical nodes (bare-metal hosts)
.250–.253 Network services
.254 Router / gateway

Network services (.250–.253)

IP Hostname Role
192.168.2.254 router Gateway (TP-Link)
192.168.2.253 domain-controller FreeIPA — authoritative DNS for sweet.home, Kerberos, LDAP
192.168.2.250–.252 — Reserved for future network services

Physical nodes (.240–.249)

IP Hostname Role
192.168.2.245 pve1 Proxmox VE hypervisor
192.168.2.244 pbs Proxmox Backup Server
192.168.2.243 nixos Bare-metal workstation (baremetal-gui)
192.168.2.246–.249 — Reserved — second Proxmox node and associated services
192.168.2.240–.242 — Reserved

pve1 sits mid-range deliberately so a second Proxmox node can slot in on either side.

Virtual nodes (.220–.229)

All VMs and LXC containers run on pve1.

IP Hostname Role Status
192.168.2.229 ha-vip-lan HA file server LAN floating VIP (Pacemaker vip-lan) — LAN iSCSI + NFS Active
192.168.2.228 ha-node1 HA file server node 1 — management NIC Active
192.168.2.227 ha-node2 HA file server node 2 — management NIC Active
192.168.2.226 server Current NFS/ZFS file server — retires when HA is live Retiring
192.168.2.225 docker Docker / Traefik stack Active
192.168.2.224 nix-cache Nix binary cache + remote builder Active
192.168.2.223 pxe-boot PXE / TFTP / HTTP netboot server Active
192.168.2.222 tailscale-router Tailscale exit node / router Active
192.168.2.221 tor-relay Tor relay Active
192.168.2.220 pdm Proxmox Deploy Manager Active

Client DHCP pool (.10–.59)

Assigned by the router. DNS option points to 192.168.2.253 (domain-controller).

Devices in this range: phones, laptops, IoT, Canon printer, any non-infrastructure host. No static reservations for infrastructure hosts — all infra uses static IP configuration on the guest itself (not DHCP reservations), so IPs survive VM recreation regardless of MAC address churn.

Cluster network — VLAN 10 — 192.168.10.224/29

Internal to pve1 only. Proxmox bridge vmbr1, no physical NIC attached.

IP Hostname Interface role
192.168.10.228 ha-node1 DRBD replication + Corosync ring0 (primary heartbeat)
192.168.10.227 ha-node2 DRBD replication + Corosync ring0 (primary heartbeat)
— no gateway Isolated — not routed to LAN or internet

Corosync ring1 (backup heartbeat only) uses the LAN IPs (192.168.2.228 / 192.168.2.227) over vmbr0 — no additional bridge needed, and DRBD traffic never crosses ring1.

Storage-client network — VLAN 20 — 192.168.20.0/24

Internal to pve1 only. Proxmox bridge vmbr2, no physical NIC attached.

IP Hostname Interface / role
192.168.20.229 ha-vip-storage Pacemaker floating VIP — NFS + iSCSI endpoint
192.168.20.228 ha-node1 Storage-client NIC (ens20 / vmbr2)
192.168.20.227 ha-node2 Storage-client NIC (ens20 / vmbr2)
192.168.20.226 server Storage-client NIC (ens19 / vmbr2) — temporary, for final rsync before decommission
192.168.20.225 docker Storage-client NIC (eth1 / vmbr2) — NFS client
— no gateway Isolated — not routed to LAN or internet

NFS clients mount from 192.168.20.229 (surviving failover transparently via the VIP). Firewall on each HA node restricts NFS and iSCSI ports to 192.168.20.0/24 — LAN hosts cannot reach either service on this VIP. The vip-storage endpoint is not reachable from the workstation directly (internal bridge only); health checks proxy through the active HA node.

Migration reference

Current → target IP for every host being renumbered.

Host Current IP New IP Config location
router 192.168.2.254 192.168.2.254 unchanged
domain-controller 192.168.2.138 192.168.2.253 /etc/sysconfig/network-scripts/ifcfg-eth0 on guest
pve1 192.168.2.250 192.168.2.245 /etc/network/interfaces on Proxmox host
pbs 192.168.2.108 192.168.2.244 static config on PBS host
nixos workstation 192.168.2.119 192.168.2.243 networking.interfaces / NetworkManager on guest
ha-node1 — 192.168.2.228 (LAN), 192.168.10.228 (cluster/VLAN 10), 192.168.20.228 (storage/VLAN 20) active
ha-node2 — 192.168.2.227 (LAN), 192.168.10.227 (cluster/VLAN 10), 192.168.20.227 (storage/VLAN 20) active
ha-vip-lan — 192.168.2.229 (vmbr0 / Pacemaker vip-lan) — NFS endpoint for LAN clients active
ha-vip-storage — 192.168.20.229 (vmbr2 / Pacemaker vip-storage) — iSCSI endpoint for VLAN 20 clients active
server 192.168.2.252 192.168.2.226 static config on guest
docker 192.168.2.249 192.168.2.225 static config on guest
nix-cache 192.168.2.120 192.168.2.224 static config on guest
pxe-boot 192.168.2.247 192.168.2.223 static config on guest; update vars.pxeServerIp in variables.nix ✓
tailscale-router 192.168.2.121 192.168.2.222 static config on guest
tor-relay 192.168.2.107 192.168.2.221 static config on guest
pdm 192.168.2.248 192.168.2.220 static config on guest

Cutover notes

  • Do domain-controller first — it becomes the DNS server; everything else depends on it having its new IP and FreeIPA DNS configured before Pi-hole is retired.
  • pve1 last among physical hosts — changing the Proxmox management IP drops the web UI briefly; all guests keep running.
  • Update Pi-hole custom.list / FreeIPA DNS A records to new IPs before flipping any host, so name resolution stays valid throughout the migration.
  • variables.nix already updated for pxeServerIp (.247→.223), pbsIp (.108→.244), and new domainControllerIp (.253). Rebuild affected hosts after renumbering.
  • Router DHCP: once domain-controller is at .253 and FreeIPA DNS is serving sweet.home, switch router DHCP on with pool .10–.59 and DNS option pointing to .253; retire Pi-hole CT.
  • Pi-hole's iPXE dnsmasq config (99-ipxe-chainload.conf) moves to the pxe-boot CT as a dnsmasq proxy-mode config before Pi-hole is decommissioned.
Reference in New Issue View Git Blame Copy Permalink