- Document bidirectional LAN ↔ Tailscale routing via tailscale-router
(static route on router for 100.64.0.0/10, subnet advertisement into Tailnet)
- Document DNS proxy chain: FreeIPA conditional forwarder for *.ts.net
→ tailscale-router → MagicDNS; reverse for *.sweet.home from Tailscale clients
- Remove stale pihole.sweet.home from IP table (decommissioned)
- Fix freeipa role: remove Pi-hole IP from ipa_dns_forwarder, document
ts.net conditional forwarder as a post-install step
- Fix inventory: remove Pi-hole forwarder IP, add ts.net forwarder note
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvNjoxTWEDkhXsd1Dq2ETP
Consolidates nixos, docker, raspi, and debian-configuration into a single
infrastructure-as-code repo. Includes:
- ansible/: full inventory + proxmox-hardening, freeipa, and raspberrypi
roles (converted from debian-configuration bash scripts)
- terraform/: Proxmox VMs, Dynu DNS, Pi-hole (decommissioned stub),
Docker container catalog — migrated from docker/infrastructure/terraform/
- stacks/docker/, stacks/raspi/, nixos/: placeholder READMEs pending
git subtree population (see implementation plan)
- docs/: internal MkDocs site with architecture, network topology, runbooks,
and drift-detection guide; external sanitized site
- scripts/: drift-detect.sh, docs-build.sh, install-hooks.sh, check-secrets.sh
- CI: secret-scan (push/PR), drift-detect (daily), docs-build (on change)
- Pi-hole removed throughout — DNS is FreeIPA, DHCP is router
See docs/internal/implementation-plan.md for the phased rollout after
pushing to Gitea.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvNjoxTWEDkhXsd1Dq2ETP
- variables.nix: switch to rec {}, extract giteaDomain/giteaRepoPath,
extraAdminSshKeys, haLanNfsFqdn, tailscaleResolverIp, ports.dhcp,
ports.dns; ipaServer now derives from homeDomain ref; section headers
- modules: use new vars throughout (pxe-boot, ts-dns-forwarder,
cluster-config, configuration.nix, mount-pxe-images) — eval unchanged
- docs: delete ephemeral planning docs (AUDIT_REPORT, ha-network-audit,
network-cutover); add docs/ha.md; drop migration reference table from
ip-addressing.md; remove stale server example from beszel.md
- CLAUDE.md/README.md/AGENTS.md: fix build types (tailscale-router,
ha-server, drop server); document scripts/ha/, scripts/ipa/, and
all previously undocumented top-level and lib scripts
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Updates mount-pxe-images.nix to mount pxe-boot/images from
ha-vip-lan.sweet.home (Pacemaker vip-lan, 192.168.2.229) instead of
server.sweet.home. Storage root changes from /tank to haStorageRoot
(/srv/ha-data). NFSv3+nolock options for LXC are unchanged.
Removes nfsServerHost and storageRoot from variables.nix — all NFS
clients now reference the HA cluster directly (docker via haStorageNfsFqdn
on VLAN 20, pxe-boot via ha-vip-lan on VLAN 2). Updates the nfsShares
comment to reflect the new single-source-of-truth role without the
dead server.nix references.
DNS records for server.sweet.home and server.storage.home were already
cleaned up automatically by IPA dynamic DNS when the VM was deleted.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8djTWdXVzXZc99iujU6T2
Removes all flake artifacts for the retired NFS/ZFS file server:
- flake.nix: drop linode-server, proxmox-server, lxc-server targets
- hosts/server/: delete host identity
- modules/build-types/server.nix: delete build type (ZFS pool + NFS exports)
- vars/per-machine/{linode,lxc,proxmox}-server/: delete clan vars
- secrets/server.keytab: delete IPA keytab
- .sops.yaml: remove server key anchors, strip from common.yaml recipients,
delete server.yaml and server.keytab creation rules
- variables.nix: remove serverIp, serverStorageIp (nfsServerHost/storageRoot
retained — pxe-boot/mount-pxe-images.nix still references them)
- docs: mark server decommissioned in ip-addressing.md
Docker NFS mounts already pointed at nfs.storage.home (previous commit).
VM destruction and DNS cleanup (server.sweet.home, server.storage.home
A/PTR records) to follow once the VM is confirmed stopped.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8djTWdXVzXZc99iujU6T2
Updates mount-data.nix to mount all docker shares from
nfs.storage.home (Pacemaker vip-storage, 192.168.20.229) over docker's
eth1/vmbr2 interface instead of server.sweet.home over the LAN.
Adds haStorageNfsFqdn variable to variables.nix for the storage.home
zone FQDN so mounts survive a future VIP renumber without a rebuild.
Storage root changes from /tank (server ZFS pool) to /srv/ha-data
(HA cluster XFS-over-DRBD). Share subpaths are unchanged.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8djTWdXVzXZc99iujU6T2
Adds a DNS Zones section to ip-addressing.md covering the three FreeIPA
zones (sweet.home / cluster.home / storage.home) with all A records and PTR
records as configured. Updates Stage 6 in network-cutover.md to reference
nfs.storage.home by DNS name rather than the raw VIP IP, and marks DNS as a
completed prerequisite for the HA storage cutover.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8djTWdXVzXZc99iujU6T2
ip-addressing.md:
- Storage-client network purpose updated: NFS primary, iSCSI available
- Protocol separation box updated: NFS on both VIPs (each CIDR-restricted);
iSCSI on VLAN 20 only but not in active use
- docker role updated to NFS client (not iSCSI initiator)
- vip-storage description updated to NFS + iSCSI endpoint
ha-network-audit.md:
- Implementation status header added noting all audit items resolved
- Records decisions that diverged from original recommendations
(VLAN renumbering, dual VIP, NFS-over-iSCSI for docker/swarm)
- Notes deferred items (STONITH, iSCSI ACLs)
network-cutover.md:
- New Stage 6: HA storage cutover — docker NFS remount from
server.sweet.home to vip-storage (192.168.20.229), final rsync
steps, and server.sweet.home decommission procedure
- Former Stage 6 (final cleanup) renumbered to Stage 7
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8djTWdXVzXZc99iujU6T2
vip-storage (192.168.20.229) now serves NFS as well as iSCSI, firewalled
to haClientCidr (192.168.20.0/24) only. This allows docker and future
swarm nodes to NFS-mount shared volumes from the storage network rather
than the LAN, keeping storage traffic off vmbr0.
LAN NFS (vip-lan 192.168.2.229) remains for pxe-boot and other LAN
clients. Protocol and subnet boundaries enforced by firewall on both nodes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8djTWdXVzXZc99iujU6T2
Storage VIP (192.168.20.229) is on vmbr2 — an internal-only bridge with no
physical uplink, unreachable from the LAN. Test it via SSH to the active node
(which has VLAN 20 connectivity) instead of directly. LAN VIP (192.168.2.229)
continues to be tested directly from wherever the script runs.
Also adds VIP_LAN variable to config section.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8djTWdXVzXZc99iujU6T2
Renumber internal HA bridges so VLAN ID matches IP third octet:
vmbr1 (cluster): 192.168.4.x → 192.168.10.x (VLAN 10, /29)
vmbr2 (storage): 192.168.5.x → 192.168.20.x (VLAN 20, /24)
Enforce protocol separation at the firewall on both HA nodes:
- iSCSI (3260/tcp): storage-client subnet (192.168.20.0/24) only
- NFS (111,2049,20048 tcp+udp): LAN subnet (192.168.2.0/24) only
- Cluster subnet (192.168.10.224/29) accepted wholesale (DRBD+Corosync)
Moves from allowedTCPPorts/allowedUDPPorts to source-restricted extraCommands
rules on the nixos-fw chain.
NFS exports reduced to LAN-only (drop haClientCidr lines from mkNfsExports).
Storage-client network clients use iSCSI only; no NFS on VLAN 20.
All HA script VIP/storage IP defaults updated to match.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8djTWdXVzXZc99iujU6T2
mkTarget computed flakeTarget as "${platform}-${buildType}" which gives
"proxmox-ha-server" for both nodes, but the actual nixosConfigurations
keys are "proxmox-ha-server-1" / "proxmox-ha-server-2". This caused
Switch-nix to fail immediately with an "unknown flake attribute" error.
Add an optional nameSuffix parameter to mkTarget and pass "-1" / "-2"
for the ha-server entries so /etc/flake-target matches the real
attribute name on each node.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8djTWdXVzXZc99iujU6T2
Both VIPs fail over together in ha-group:
vip-lan: 192.168.2.229/24 on ens18 (vmbr0) — LAN iSCSI + NFS access
vip-storage: 192.168.5.229/24 on ens20 (vmbr2) — storage-client iSCSI + NFS
variables.nix: haServerLanVip = "192.168.2.229"
cluster-init.sh: both VIP primitives in ha-group XML; VIP_LAN var added
ip-addressing.md: both VIP entries documented
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Infrastructure changes already applied to pve1:
- vmbr2 internal bridge created (192.168.5.0/24, no physical uplink)
- VM 200 (ha-server-1): net2 added → vmbr2 (ens20)
- VM 201 (ha-server-2): net2 added → vmbr2 (ens20)
- CT 105 (docker): net1 added → vmbr2 (eth1)
- VM 101 (server): net1 added → vmbr2 (ens19) — needs reboot to activate
NixOS config (deploy to ha nodes to complete; docker/server at cutover):
- ha-server-{1,2}/host.nix: ens20 with 192.168.5.{228,227}/24
- docker/host.nix: eth1 with 192.168.5.225/24
- server/host.nix: ens19 with 192.168.5.226/24
- cluster-config.nix: corosync ring1 on LAN IPs as backup heartbeat path
- cluster-config.nix: allow haClientCidr (192.168.5.0/24) in iptables
- ha-server.nix: NFS exports now allow both lanCidr and haClientCidr
- VIP moves from 192.168.2.229 (vmbr0/LAN) to 192.168.5.229 (vmbr2)
- iSCSI portal to be rebound from [::0] to 192.168.5.229 at cutover
variables.nix: haStorageCidr corrected to 192.168.4.224/29; new vars:
vmStorageClientInterface, lxcStorageInterface, haServer{1,2}ClientIp,
dockerStorageIp, serverStorageIp, haClientCidr/PrefixLength; haServerVip
updated to 192.168.5.229.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Read-only discovery of pve1 bridges, HA cluster VMs (200/201), and
Docker CT (105) — current state, risk flags, recommended vmbr2 storage
bridge layout, gap list, and migration notes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
pve1 runs non-NixOS guests (FreeIPA domain-controller, Proxmox Data Manager, etc.)
alongside NixOS ones. Validate each discovered hostname against the set of
hostnames defined in nixosConfigurations before adding it to the gc list.
Brings back the flake eval but uses it correctly: extract all hostname values
(not target-name keys) to build a filter set, then only include pve1 guests
whose name matches a flake-managed NixOS host.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The by-id lookup is read-only so it's safe to run in dry-run mode.
Previously it was gated behind `if ! $DRY_RUN`, which always triggered
the sdb fallback warning in dry-run — making it look like the device
path wasn't reliable when the symlink actually exists on both servers.
Now the lookup always runs and the script errors out with a clear message
if the by-id symlink is genuinely missing, instead of silently falling
back to a guessed /dev/sd* name.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The flake hostname map approach was wrong: create-proxmox-resource.sh names
every guest after its NixOS hostname (pct create --hostname <nixos-hostname>,
qm create --name <nixos-hostname>), so pct/qm list already output the final
hostname directly. Looking those names up in a map keyed by flake target names
(lxc-docker, proxmox-server, ...) found nothing and the dynamic list was
always empty.
Drop the flake eval entirely; use the pct/qm list output as-is, filtering
only nix-cache and already-seen entries.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
qm lives at /usr/sbin/qm, which is not in the default PATH for
non-interactive SSH sessions as a non-root user. Add PVE_SUDO (set to
"sudo" when PVE_SSH_USER != root, matching create-proxmox-resource.sh's
own sudo_prefix pattern) and prepend it to all three qm invocations in
the script.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two bugs:
1. Dynamic discovery used BatchMode=yes for the pct/qm list SSH call, which
silently suppressed any sudo password prompt and returned empty output.
Fix: split SSH_OPTS into SSH_OPTS (BatchMode, for gc) and SSH_QUERY_OPTS
(no BatchMode, for discovery) so sudo can prompt when needed.
2. pve1 gc used a non-login SSH session which doesn't source /etc/profile,
so nix-collect-garbage was never on PATH. The Nix installer registers
itself via /etc/profile.d/nix-daemon.sh which only runs in login shells.
Fix: use "bash -l -c 'nix-collect-garbage -d'" instead.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
BatchMode SSH sessions don't source /etc/profile on non-NixOS hosts, so
nix-collect-garbage isn't on PATH for the wayne user. Source the nix-daemon
profile script explicitly, matching the pattern in scripts/lib/nix-bootstrap.sh.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace the static host list with dynamic discovery: workstation (nixos)
and pve1 are hard-wired first and second; remaining hosts are discovered
on every run by SSHing to pve1, listing running VMs/containers via
pct/qm list, and resolving their NixOS hostnames from a single flake eval.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Runs nix-collect-garbage -d on all deployed NixOS hosts and pve1 in
parallel, skipping nix-cache to avoid evicting shared cache paths.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Generated /home/wayne/.ssh/id_ed25519 on pve1 (single-user Nix install
owned by wayne, no daemon — builds run as wayne not root). Adds the
public key to remoteBuilderAuthorizedKeys so nix-cache accepts it.
After deploying nix-cache, re-run configure-nix-cache-client.sh as
wayne on pve1 to wire in the builder line and known_hosts entry.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
nfs-utils changed ConditionPathExists from !/etc/krb5.keytab to
/etc/krb5.keytab, so on IPA-joined hosts the service now starts instead
of skipping. Exports use standard auth (no sec=krb5) so the nfs/ Kerberos
principal is never provisioned and the service fails.
enable = false masks the unit so nfs-server's Wants= can't pull it in.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
KEY (hub's SSH public key) is now stored in variables.nix as beszelHubKey
and set by enable-agent.nix — no host file ever needs to touch it.
TOKEN was already universal; this removes the last per-host beszel config.
Importing enable-agent.nix in a build type is now the only step needed to
add a new host to beszel monitoring.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
