Add Docker Swarm HA cluster: ha-docker-1 and ha-docker-2

Two new NixOS Proxmox VMs (VMIDs 202/203) forming a dual-manager Docker
Swarm on dedicated vmbr3 (192.168.30.0/24, VLAN 30) for gossip and VXLAN,
with NFS via the storage-client network (vmbr2) from the existing HA cluster.

- nixos/variables.nix: add ha-docker IP/interface/port vars and swarm CIDR
- nixos/modules/build-types/ha-docker.nix: new build type — Docker 29,
  NFS mounts, beszel-agent, health monitoring, swarm firewall rules with
  checkReversePath = "loose" for VXLAN routing mesh
- nixos/hosts/ha-docker-{1,2}/host.nix: per-host identity — three NICs
  (LAN, storage, swarm), IPA dyndns pinned to LAN interface
- nixos/flake.nix: add proxmox-ha-docker-{1,2} targets; build-validated
  with nix build --dry-run (169 derivations, no errors)
- nixos/docs/ip-addressing.md: document VLAN 30 / swarm.home zone,
  ha-docker IP allocations across all three subnets
- nixos/scripts/docker-swarm/deploy.sh: 10-phase lifecycle script
  (bridge, keys, IPA, VMs, swarm init, DNS, verify); modelled on
  scripts/ha/deploy.sh with --destroy mode
- nixos/docs/internal/docker-swarm-cutover.md: service-by-service
  migration guide covering Traefik log rotation, Nextcloud cron sidecar,
  docker-health-to-gotify swarm awareness updates, Passbolt/Gitea steps,
  DNS cutover, and CT 105 decommission checklist

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DASH15okNvWeY1rVJmyJoJ
This commit is contained in:
2026-07-30 19:01:17 +10:00
co-authored by Claude Sonnet 4.6
parent 43314b6aa3
commit c96752e5d0
8 changed files with 1162 additions and 5 deletions
+84
View File
@@ -0,0 +1,84 @@
# Docker Swarm node build type.
#
# Produces NixOS hosts that form a Docker Swarm manager cluster. Two nodes
# (ha-docker-1, ha-docker-2) are both managers so either can accept Docker
# API and `docker stack` commands.
#
# Key differences from the existing `docker` build type (used by CT 105):
# - nextcloud-cron-job.nix is EXCLUDED — `docker exec` breaks in swarm
# because the target container may be on the other node. The cron job
# is replaced by a nextcloud-cron sidecar in the Nextcloud stack.
# See docs/internal/docker-swarm-cutover.md.
# - traefik/rotate-logs.nix is EXCLUDED — log rotation moves to Docker's
# json-file log driver (max-size/max-file on the Traefik service
# definition). See docs/internal/docker-swarm-cutover.md.
# - raspi/mount-data.nix is EXCLUDED — specific to CT 105's backup role.
# - Swarm firewall ports (2377/tcp, 7946/tcp+udp, 4789/udp) are opened
# on the swarm NIC (ens20/vmbr3) only.
# - checkReversePath = "loose" is required for the Swarm ingress routing
# mesh: VXLAN return traffic is asymmetric (arrives ens20, exits ens18).
# - beszel-agent is enabled for host-level monitoring.
{ pkgs, vars, ... }:
{
# Pin Docker Engine to version 29, matching CT 105, so image layers cached
# on NFS volumes remain compatible across old and new hosts.
nixpkgs.overlays = [
(final: prev: {
docker = prev.docker_29;
docker_cli = prev.docker_29;
})
];
imports = [
../docker/enable-service.nix
../docker/mount-data.nix
../docker/docker-health-to-gotify.nix
../beszel/enable-agent.nix
../services/enable-rpcbind.nix
];
environment.systemPackages = with pkgs; [
nfs-utils
];
boot.supportedFilesystems = [ "nfs" ];
systemd.tmpfiles.rules = [
# Symlink ~/docker → NFS config mount so the docker-health-to-gotify
# script (and operator convenience) resolves ~/docker/... correctly.
"L+ /home/${vars.primaryUser}/docker - - - - ${vars.nfsShares.dockerConfig.mountpoint}"
"d /mnt/docker 0755 ${vars.primaryUser} users -"
];
users.users.${vars.primaryUser}.extraGroups = [ "docker" ];
networking.firewall = {
# LAN-facing service ports — same as the existing docker build type.
allowedTCPPorts = [
vars.ports.dockerHttp
vars.ports.dockerHttps
vars.ports.dockerExtra
vars.ports.beszelHub
];
# Swarm inter-node ports restricted to the swarm NIC (ens20/vmbr3).
# vmbr3 is an isolated internal bridge — no LAN reachability.
interfaces.${vars.haDockerSwarmInterface} = {
allowedTCPPorts = [
vars.ports.dockerSwarmMgmt # 2377 — Raft + cluster management
vars.ports.dockerSwarmDisc # 7946 — Serf gossip (TCP half)
];
allowedUDPPorts = [
vars.ports.dockerSwarmDisc # 7946 — Serf gossip (UDP half)
vars.ports.dockerSwarmVxlan # 4789 — VXLAN overlay data path
];
};
# Docker Swarm ingress routing mesh creates asymmetric routes: a request
# arrives on ens18 (LAN) for a container that lives on ens20's VXLAN
# overlay; the return path differs from the incoming interface. Strict
# rp_filter drops these packets. "loose" allows them.
checkReversePath = "loose";
};
}