Archived
Add Docker Swarm HA cluster: ha-docker-1 and ha-docker-2
Two new NixOS Proxmox VMs (VMIDs 202/203) forming a dual-manager Docker
Swarm on dedicated vmbr3 (192.168.30.0/24, VLAN 30) for gossip and VXLAN,
with NFS via the storage-client network (vmbr2) from the existing HA cluster.
- nixos/variables.nix: add ha-docker IP/interface/port vars and swarm CIDR
- nixos/modules/build-types/ha-docker.nix: new build type — Docker 29,
NFS mounts, beszel-agent, health monitoring, swarm firewall rules with
checkReversePath = "loose" for VXLAN routing mesh
- nixos/hosts/ha-docker-{1,2}/host.nix: per-host identity — three NICs
(LAN, storage, swarm), IPA dyndns pinned to LAN interface
- nixos/flake.nix: add proxmox-ha-docker-{1,2} targets; build-validated
with nix build --dry-run (169 derivations, no errors)
- nixos/docs/ip-addressing.md: document VLAN 30 / swarm.home zone,
ha-docker IP allocations across all three subnets
- nixos/scripts/docker-swarm/deploy.sh: 10-phase lifecycle script
(bridge, keys, IPA, VMs, swarm init, DNS, verify); modelled on
scripts/ha/deploy.sh with --destroy mode
- nixos/docs/internal/docker-swarm-cutover.md: service-by-service
migration guide covering Traefik log rotation, Nextcloud cron sidecar,
docker-health-to-gotify swarm awareness updates, Passbolt/Gitea steps,
DNS cutover, and CT 105 decommission checklist
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DASH15okNvWeY1rVJmyJoJ
This commit is contained in:
@@ -188,6 +188,46 @@ rec {
|
||||
# the data disk; drive-scsi0 is the OS disk.
|
||||
haServerDrbdDisk = "/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi1";
|
||||
|
||||
# ── Docker Swarm cluster ──────────────────────────────────────────────────
|
||||
#
|
||||
# Three network segments, all internal to pve1:
|
||||
# LAN vmbr0 192.168.2.0/24 — management; SSH + external service traffic
|
||||
# Storage-client vmbr2 192.168.20.0/24 — NFS from HA cluster VIP (shared with HA nodes)
|
||||
# Swarm cluster vmbr3 192.168.30.0/24 — Docker Swarm gossip + VXLAN overlay
|
||||
#
|
||||
# Host octet consistent across subnets: node1 = .230, node2 = .231.
|
||||
# IPs from the .230–.239 expansion buffer documented in docs/ip-addressing.md.
|
||||
#
|
||||
# Docker Swarm uses --advertise-addr and --data-path-addr on the swarm NIC
|
||||
# (ens20/vmbr3) so all inter-node cluster traffic stays on the isolated
|
||||
# internal bridge and never crosses the LAN.
|
||||
#
|
||||
# When expanding to a second Proxmox node, vmbr3 (VLAN 30) and vmbr1
|
||||
# (VLAN 10) share the same inter-node trunk NIC via VLAN tagging — same
|
||||
# physical wire, different VLAN IDs.
|
||||
|
||||
haDocker1Host = "ha-docker-1";
|
||||
haDocker2Host = "ha-docker-2";
|
||||
|
||||
haDocker1Ip = "192.168.2.230"; # LAN management NIC (ens18, vmbr0)
|
||||
haDocker2Ip = "192.168.2.231";
|
||||
|
||||
haDocker1StorageIp = "192.168.20.230"; # storage-client NIC (ens19, vmbr2)
|
||||
haDocker2StorageIp = "192.168.20.231";
|
||||
|
||||
haDocker1SwarmIp = "192.168.30.230"; # swarm cluster NIC (ens20, vmbr3)
|
||||
haDocker2SwarmIp = "192.168.30.231";
|
||||
|
||||
haDockerSwarmCidr = "192.168.30.0/24";
|
||||
haDockerSwarmPrefixLength = 24;
|
||||
|
||||
# NIC names for ha-docker VMs. ens19/ens20 occupy the same guest bus
|
||||
# positions as vmStorageInterface/vmStorageClientInterface on ha-server VMs
|
||||
# but are attached to different bridges — storage (vmbr2) and swarm (vmbr3)
|
||||
# respectively. Kept as named variables to avoid bare literals in modules.
|
||||
haDockerStorageInterface = "ens19"; # vmbr2 — NFS client
|
||||
haDockerSwarmInterface = "ens20"; # vmbr3 — Docker Swarm gossip + VXLAN
|
||||
|
||||
# ── Storage / NFS ─────────────────────────────────────────────────────────
|
||||
|
||||
# NFS share definitions — used by ha-server.nix (exports), docker/mount-data.nix,
|
||||
@@ -263,6 +303,13 @@ rec {
|
||||
dockerHttps = 443;
|
||||
dockerExtra = 8080;
|
||||
|
||||
# Docker Swarm inter-node ports (modules/build-types/ha-docker.nix).
|
||||
# Firewalled to haDockerSwarmCidr only — vmbr3 is an isolated bridge
|
||||
# with no physical uplink, so these ports are unreachable from LAN.
|
||||
dockerSwarmMgmt = 2377; # TCP — Raft consensus + cluster management
|
||||
dockerSwarmDisc = 7946; # TCP+UDP — Serf gossip (container network discovery)
|
||||
dockerSwarmVxlan = 4789; # UDP — VXLAN overlay data path
|
||||
|
||||
# Beszel monitoring hub on docker.sweet.home, reached by every agent
|
||||
# (modules/beszel/enable-agent.nix, hosts/nixos/home.nix)
|
||||
beszelHub = 8090;
|
||||
|
||||
Reference in New Issue
Block a user