Compare commits

..
Author SHA1 Message Date
beatzaplenty 55ba283c82 Merge branch 'main' into worktree-docker-gui-fix
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m24s
2026-07-28 00:32:06 +00:00
beatzaplenty 2d46d25a67 Merge pull request 'fix(ipa): harden script and update module docs' (#82) from worktree-ipa-client-module into main
Check NixOS configurations / eval-hosts (push) Failing after 9m45s
Reviewed-on: #82
2026-07-28 00:25:13 +00:00
beatzaplentyandClaude Sonnet 4.6 f5d29be041 fix(ipa): harden script and update module docs
Check NixOS configurations / eval-hosts (pull_request) Failing after 9m53s
Script fixes:
- Rename HOSTNAME variable to TARGET (shadowed the bash builtin)
- Fix ipa-getkeytab -s to always use IPA_SERVER, not DC_HOST (diverge if
  --dc is overridden to a jump host)
- Remove dead REALM variable
- Add EXIT trap to delete the plaintext keytab if the script aborts before
  sops encryption completes; cleared after successful encrypt
- Distinguish real ipa host-add failures from "already exists" instead of
  swallowing all errors with || true
- Warn explicitly when no platform age keys exist for the target (keytab
  would be admin-only and the host couldn't decrypt it at boot)
- Fix sops fallback from pinned nixos-25.11 channel to nixpkgs (uses the
  repo's own flake.lock)
- Expand "next steps" output to include networking.domain and nameservers
  lines that host.nix requires for IPA membership

Module docs:
- Point to the script as the primary setup path; move manual steps to a
  fallback section
- Note that certs/ipa-ca.crt is already committed (no need to re-fetch)
- Document the networking.domain and nameservers requirements in the header
- Add sync-host-keys.sh as explicit step 0

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-28 10:24:23 +10:00
beatzaplenty e10a1572c3 Merge branch 'main' of https://gitea.lan.ddnsgeek.com/beatzaplenty/nixos
Check NixOS configurations / eval-hosts (push) Successful in 10m25s
2026-07-28 10:23:34 +10:00
beatzaplenty 578ef70aa9 updated flake.lock 2026-07-28 10:23:23 +10:00
beatzaplenty e9fcbbbbcb Merge pull request 'Worktree ipa client module' (#80) from worktree-ipa-client-module into main
Check NixOS configurations / eval-hosts (push) Failing after 9m47s
Reviewed-on: #80
2026-07-28 00:13:08 +00:00
beatzaplentyandClaude Sonnet 4.6 f46ae18672 fix(ipa): work around OpenSSH 10 AuthorizedKeysCommand path check
Check NixOS configurations / eval-hosts (pull_request) Failing after 9m43s
OpenSSH 10.0 tightened AuthorizedKeysCommand security by checking every
path component of the command binary for group/world-write permission.
/nix/store is 1775 (group-writable by nixbld), so sshd silently skips
the command for any binary in the Nix store — causing IPA pubkey auth to
silently fail with no diagnostic.

Fix: copy sss_ssh_authorizedkeys to /usr/local/bin via systemd tmpfiles
(C+ copies the file rather than symlinking, so the path at runtime is
root-owned/755 throughout), and point AuthorizedKeysCommand at the copy.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-28 10:08:57 +10:00
beatzaplenty fc277294f3 add vars to module
Check NixOS configurations / eval-hosts (push) Successful in 10m32s
2026-07-28 10:02:58 +10:00
beatzaplenty f3e5ea67a0 add docker group to user in enable docker service module
Check NixOS configurations / eval-hosts (push) Successful in 10m33s
2026-07-28 09:59:48 +10:00
beatzaplenty 7a8aebf679 Merge branch 'worktree-docker-gui-fix' 2026-07-28 09:59:40 +10:00
beatzaplenty a8d95aad02 enable docker
Check NixOS configurations / eval-hosts (push) Failing after 9m45s
2026-07-28 09:49:14 +10:00
beatzaplentyandClaude Sonnet 4.6 dac5fbd574 feat(gui): enable docker service on workstation
Check NixOS configurations / eval-hosts (pull_request) Failing after 9m45s
Imports docker/enable-service.nix and pins virtualisation.docker to
docker_29 via a nixpkgs overlay. Fixes indentation on `imports` block
and `programs.direnv.enable` line so nixpkgs-fmt passes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-28 09:48:59 +10:00
beatzaplenty da0c651c60 Merge branch 'worktree-ipa-client-module' 2026-07-28 09:48:55 +10:00
beatzaplentyandClaude Sonnet 4.6 97019205da fix(ipa): create home dir on first login + AuthorizedKeysCommand
Check NixOS configurations / eval-hosts (pull_request) Failing after 9m45s
- security.pam.services.sshd.makeHomeDir: IPA users have no pre-created
  home directory on the host; without this, sshd opens a session to a
  missing directory and resets the connection immediately after auth
- AuthorizedKeysCommand was already added in previous commit

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-28 09:47:11 +10:00
beatzaplentyandClaude Sonnet 4.6 5487490b8e feat(ipa): add AuthorizedKeysCommand + enroll tailscale-router
- modules/ipa/client.nix: add AuthorizedKeysCommand so sshd fetches SSH
  public keys from IPA via sss_ssh_authorizedkeys, enabling pubkey login
  without per-host authorized_keys files

- hosts/tailscale-router/host.nix: add IPA client module + networking.domain
  so SSSD runs and wayne can authenticate on this host

- secrets/tailscale-router.keytab: sops-encrypted keytab for
  tailscale-router.sweet.home (generated by create-nixos-ipa-host-account.sh)

- .sops.yaml: creation rule for secrets/tailscale-router.keytab

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-28 09:42:38 +10:00
beatzaplenty 543ea432f0 Merge pull request 'fix(tailscale-router): stop dnsmasq from intercepting host DNS queries' (#79) from worktree-peaceful-stirring-noodle into main
Check NixOS configurations / eval-hosts (push) Successful in 10m33s
Reviewed-on: #79
2026-07-27 23:27:34 +00:00
beatzaplentyandClaude Sonnet 4.6 c7bbf88dce fix(tailscale-router): stop dnsmasq from intercepting host DNS queries
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m30s
NixOS's dnsmasq module defaults resolveLocalQueries to true, which adds
127.0.0.1 to networking.nameservers and binds dnsmasq to listen-address=127.0.0.1.
This made the host route all its own DNS through dnsmasq, which had
no-resolv=true and no upstream for anything outside the tailnet domain —
so every non-tailscale DNS query from the host itself (including SSSD
resolving the IPA server FQDN after the IPA client module was added) failed.

Setting resolveLocalQueries=false limits dnsmasq to its intended role: a
forwarding proxy reachable on the LAN interface for IPA's conditional
forwarder. The host uses domainControllerIp directly (already set in
networking.nameservers in host.nix).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-28 09:26:50 +10:00
beatzaplenty d57145b31e add tailscale-router to domain
Check NixOS configurations / eval-hosts (push) Failing after 9m44s
2026-07-28 09:17:29 +10:00
beatzaplenty 1cbe80c0ca secrets: add IPA keytab for tailscale-router 2026-07-28 09:15:21 +10:00
beatzaplenty 01ee261cf8 Merge pull request 'fix(ipa): stream keytab via sudo cat instead of scp' (#78) from worktree-ipa-client-module into main
Check NixOS configurations / eval-hosts (push) Successful in 10m22s
Reviewed-on: #78
2026-07-27 23:14:41 +00:00
beatzaplentyandClaude Sonnet 4.6 f6f30c675f fix(ipa): stream keytab via sudo cat instead of scp
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m25s
ipa-getkeytab runs as root via sudo so the temp file is root-owned;
scp as wayne gets Permission denied. Pipe through `sudo cat` over SSH
instead, which reads as root but writes locally as the invoking user.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-28 09:08:19 +10:00
beatzaplenty 60b80cbd96 Merge pull request 'fix(ipa): SSH as wayne with sudo instead of root on domain controller' (#77) from worktree-ipa-client-module into main
Check NixOS configurations / eval-hosts (push) Successful in 10m21s
Reviewed-on: #77
2026-07-27 23:06:49 +00:00
beatzaplentyandClaude Sonnet 4.6 27a8c7fad9 fix(ipa): SSH as wayne with sudo instead of root on domain controller
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m23s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-28 09:02:32 +10:00
beatzaplenty d9cee0a674 Merge pull request 'feat(ipa): add create-nixos-ipa-host-account script' (#76) from worktree-ipa-client-module into main
Check NixOS configurations / eval-hosts (push) Successful in 10m22s
Reviewed-on: #76
2026-07-27 22:58:46 +00:00
beatzaplentyandClaude Sonnet 4.6 6c1891812e feat(ipa): add create-nixos-ipa-host-account script
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m24s
Single command to enroll a NixOS host in FreeIPA and produce a
sops-encrypted keytab at secrets/<hostname>.keytab:
  - Adds the .sops.yaml creation rule automatically (with all registered
    platform-variant age keys as recipients)
  - SSHes to the domain controller to run ipa host-add + ipa-getkeytab
  - Refreshes the admin Kerberos ticket via `ssh -t ... kinit admin` if
    missing or expired, so no manual kinit step is needed
  - SCPs the keytab and encrypts it in-place with sops (file must be at
    secrets/<hostname>.keytab before encryption so the path-based creation
    rule matches — the common failure point when doing this manually)

Also adds HOME_DOMAIN and IPA_SERVER to scripts/env.sh, matching
variables.nix's homeDomain/ipaServer (same manual-sync pattern as
NIX_CACHE_HOST/LAN_DOMAIN).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-28 08:57:55 +10:00
beatzaplenty 59316c982e Merge branch 'main' of https://gitea.lan.ddnsgeek.com/beatzaplenty/nixos
Check NixOS configurations / eval-hosts (push) Successful in 10m27s
2026-07-28 07:26:40 +10:00
beatzaplenty 9eca3bd719 Merge pull request 'Worktree ipa client module' (#75) from worktree-ipa-client-module into main
Check NixOS configurations / eval-hosts (push) Failing after 9m45s
Reviewed-on: #75
2026-07-27 21:25:57 +00:00
beatzaplenty f2f0fcf756 certs: add FreeIPA CA certificate
Check NixOS configurations / eval-hosts (pull_request) Failing after 9m45s
2026-07-28 07:25:28 +10:00
beatzaplenty b4fb9c25f2 secrets(nix-cache): add sops-encrypted IPA host keytab 2026-07-28 07:25:25 +10:00
beatzaplentyandClaude Sonnet 4.6 8955840f0a fix(ipa): use in-place sops encryption in module docs
sops matches creation rules against the input file path, so encrypting
/tmp/<host>.keytab directly with stdout redirect fails to find the rule.
Copy to secrets/ first, then use -i to encrypt in-place.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-28 07:19:53 +10:00
beatzaplentyandClaude Sonnet 4.6 a4b49c9909 fix(ipa): correct ipaServer to domain-controller.sweet.home
ipa-getkeytab confirmed ipa.sweet.home doesn't respond to LDAP;
domain-controller.sweet.home is the actual IPA server FQDN.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-28 07:16:53 +10:00
beatzaplenty c0936a10e7 Merge pull request 'feat(ipa): add reusable declarative FreeIPA client module' (#74) from worktree-ipa-client-module into main
Check NixOS configurations / eval-hosts (push) Failing after 9m45s
Reviewed-on: #74
2026-07-27 21:06:28 +00:00
beatzaplentyandClaude Sonnet 4.6 f4bbd6331d feat(ipa): add reusable declarative FreeIPA client module
Check NixOS configurations / eval-hosts (pull_request) Failing after 10m0s
Adds modules/ipa/client.nix — a parameterized module that joins a NixOS host
to the sweet.home FreeIPA domain without ipa-client-install. It configures
security.ipa (SSSD, Kerberos, PAM, NSSwitch) and places a pre-provisioned host
keytab via sops-nix binary secret so enrollment is fully reproducible from the
flake.

- variables.nix: adds ipaServer (FQDN of the FreeIPA KDC; security.ipa.server
  requires a hostname, not an IP, for Kerberos/TLS)
- certs/ipa-ca.crt: placeholder for the IPA CA public certificate (operator
  replaces with: curl http://<ipa-server>/ipa/config/ca.crt)
- secrets/nix-cache.keytab: placeholder binary sops file (operator replaces
  with the encrypted keytab after ipa host-add + ipa-getkeytab)
- .sops.yaml: adds creation rule for secrets/nix-cache.keytab (same recipients
  as secrets/nix-cache.yaml)
- hosts/nix-cache/host.nix: imports the IPA client module; adds
  networking.domain so the host's FQDN resolves correctly

Module header documents the three operator steps needed per host before deploy.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-27 23:31:21 +10:00
beatzaplenty b99ba87cf6 Merge branch 'main' of https://gitea.lan.ddnsgeek.com/beatzaplenty/nixos 2026-07-27 23:05:35 +10:00
beatzaplenty 2128353f9f Merge pull request 'Fix/static ips' (#73) from fix/static-ips into main
Check NixOS configurations / eval-hosts (push) Failing after 42m27s
Reviewed-on: #73
2026-07-27 12:06:33 +00:00
beatzaplentyandClaude Sonnet 4.6 7b4ce0ab3d fix(server): prevent zfs-init-tank from wiping pool on udev race
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m46s
zpool create -f was called if `zpool import -d /dev/disk/by-id` failed,
which could happen due to a race with systemd-udev-settle. The disk
would then be visible by the time zpool create ran, silently destroying
all data on an otherwise-intact pool.

Fix: locate the data disk first, retry the import directly against it
as a fallback, then check zdb -l for existing ZFS label metadata before
concluding the disk is blank. Remove -f so zpool create refuses rather
than overwrites if a pool is present.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ULXzafSDwGhmFGnn3LtDSQ
2026-07-27 21:52:39 +10:00
beatzaplentyandClaude Sonnet 4.6 3123565011 fix(tailscale-router): scope ts.net forwarder to tailnet subdomain
IPA refuses to create a forward zone for ts.net because it's a real
public TLD with DNSimple nameservers. The forward zone must use the
tailnet-specific subdomain (vars.tailnetDomain, e.g. tail13f623.ts.net)
instead. Update dnsmasq server selector and comments to match.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ULXzafSDwGhmFGnn3LtDSQ
2026-07-27 19:23:46 +10:00
beatzaplentyandClaude Sonnet 4.6 36f5ebdf86 feat(tailscale-router): serve ts.net DNS forward zone for LAN hosts
FreeIPA (the new authoritative DNS) cannot reach 100.100.100.100
(Tailscale's internal MagicDNS resolver) directly because the DC is not
a Tailscale node. The tailscale-router IS a Tailscale node and can
reach 100.100.100.100 via tailscale0, so it now runs a dnsmasq
instance on its LAN interface that forwards all ts.net queries to
Tailscale's resolver.

After deploying this host, configure FreeIPA with:
  kinit admin
  ipa dnsforwardzone-add ts.net \
    --forwarder=192.168.2.222 \
    --forward-policy=only

This replaces Pi-hole's conditional forwarder for ts.net and restores
resolution of Tailscale MagicDNS names (e.g. raspberrypi.tail13f623.ts.net)
for all LAN hosts using FreeIPA as their DNS server.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ULXzafSDwGhmFGnn3LtDSQ
2026-07-27 19:20:12 +10:00
beatzaplenty bba054db85 Merge branch 'main' of https://gitea.lan.ddnsgeek.com/beatzaplenty/nixos
# Conflicts:
#	variables.nix
2026-07-27 18:29:07 +10:00
beatzaplenty b63a529a1d Merge branch 'fix/static-ips' 2026-07-27 18:25:37 +10:00
beatzaplentyandClaude Sonnet 4.6 ee96713a50 feat(networking): declare static IPs for all NixOS-managed hosts in flake
Adds static IP configuration to every NixOS host in the flake that
has a fixed LAN address, and centralises all network primitives
(IPs, gateway, prefix length, interface names) in variables.nix so
there is one place to update if any of them change.

variables.nix additions:
- lanGateway / lanPrefixLength — LAN gateway and /24 prefix, replacing
  every hardcoded 192.168.2.254 / 24 across host files
- lxcLanInterface / vmLanInterface / vmStorageInterface — NIC names for
  LXC containers (eth0), Proxmox VMs (ens18), and the HA storage NIC
  (ens19), used as attribute keys so changing the name is a one-line edit
- haStoragePrefixLength — /29 for the storage subnet, mirrors haStorageCidr
- Per-host IP variables: nixCacheIp (.224), tailscaleRouterIp (.222),
  torRelayIp (.221), serverIp (.226), dockerIp (.225)

host.nix changes:
- tailscale-router, tor-relay, nix-cache, pxe-boot: useDHCP = false,
  static address on eth0 (lxcLanInterface), struct-form defaultGateway
  (required when using systemd-networkd which LXC containers use)
- server, docker: useDHCP = false, static address on ens18 (vmLanInterface),
  struct-form defaultGateway (works for both scripted networking and networkd)
- ha-server-1, ha-server-2: replace hardcoded 192.168.2.254 / 24 / 29
  with the new variables; no functional change for these hosts

modules/build-types/pxe-boot.nix:
- Domain-controller kickstart template: replace hardcoded 192.168.2.138
  and 192.168.2.254 with vars.domainControllerIp / vars.lanGateway /
  vars.lanPrefixLength / vars.homeDomain so the template stays correct
  if the DC IP or domain is ever changed again

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ULXzafSDwGhmFGnn3LtDSQ
2026-07-27 18:14:31 +10:00
beatzaplenty 1ece0c75d9 Merge pull request 'chore(pxe-boot): remove log-dhcp debug flag now that PXE boot is confirmed working' (#86) from fix/pxe-dnsmasq-port-conflict into main
Reviewed-on: #86
2026-07-27 05:52:18 +00:00
beatzaplentyandClaude Sonnet 4.6 548c6f5041 chore(pxe-boot): remove log-dhcp debug flag now that PXE boot is confirmed working
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-27 15:50:19 +10:00
beatzaplenty bae4c8171f Merge pull request 'fix(pxe-boot): use current pxe-boot IP (.247) in dnsmasq until Stage 5 renumber' (#85) from fix/pxe-dnsmasq-port-conflict into main
Reviewed-on: #85
2026-07-27 05:41:38 +00:00
beatzaplentyandClaude Sonnet 4.6 3748c86049 fix(pxe-boot): use current pxe-boot IP (.247) in dnsmasq until Stage 5 renumber
vars.pxeServerIp was already set to the post-renumber target (.223) but the
pxe-boot CT is still at .247, so dnsmasq was advertising .223 as the TFTP server
and the VM couldn't reach it.  Update to .247 so PXE boot works now; the comment
reminds us to flip it back to .223 when Stage 5 step 6 renumbers the CT.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-27 15:40:54 +10:00
beatzaplenty 9dd969cf4c Merge pull request 'fix(pxe-boot): open UDP 4011 for PXE boot service discovery' (#84) from fix/pxe-dnsmasq-port-conflict into main
Reviewed-on: #84
2026-07-27 05:22:46 +00:00
beatzaplentyandClaude Sonnet 4.6 7e4b2d33fb fix(pxe-boot): open UDP 4011 for PXE boot service discovery
After the proxy DHCP handshake, the UEFI client queries the PXE boot
server on UDP port 4011 to get the actual boot filename. This port was
not open in the NixOS firewall, so all four PXE discovery attempts from
the test VM silently timed out. dnsmasq was bound to 0.0.0.0:4011 but
never received the packets.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-27 15:12:12 +10:00
beatzaplenty 288d50fd33 Merge pull request 'Fix/pxe dnsmasq port conflict' (#83) from fix/pxe-dnsmasq-port-conflict into main
Reviewed-on: #83
2026-07-27 05:05:42 +00:00
beatzaplentyandClaude Sonnet 4.6 f0e76f8aff fix(pxe-boot): use pxe-service for proxy DHCP initial boot offers
dnsmasq 2.93 does not send proxy DHCP offers from dhcp-boot alone in proxy
mode — pxe-service is required to trigger proxy offers for the initial boot.

Replace the dhcp-boot-based arch detection with pxe-service entries for
arch 0 (BIOS), 7 (EFI BC / OVMF), and 9 (EFI x86-64), then use dhcp-boot
only for the iPXE second-boot HTTP chain (tag:ipxe clients). Add log-dhcp
for post-deploy verification.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-27 15:03:01 +10:00
beatzaplenty e80d195284 Merge branch 'fix/pxe-dnsmasq-port-conflict' 2026-07-27 14:32:00 +10:00
beatzaplentyandClaude Sonnet 4.6 c770feebc9 fix(pxe-boot): disable dnsmasq DNS listener to avoid port 53 conflict
dnsmasq binds port 53 by default even when only proxy DHCP is needed.
systemd-resolved already owns port 53 on the pxe-boot container, so
dnsmasq crashes at startup. Setting port=0 disables the DNS listener
entirely — pxe-boot only needs the proxy DHCP path for iPXE chainloading.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-27 14:29:30 +10:00
beatzaplenty 7fd6d558d5 updated dnsmasq service 2026-07-27 14:07:28 +10:00
beatzaplentyandClaude Sonnet 4.6 58c40292e2 fix(pxe-boot): move dnsmasq out of systemd.services into services.dnsmasq
dnsmasq config was placed inside systemd.services{} block, making Nix
interpret it as systemd.services.dnsmasq.settings (which doesn't exist).
Move to top-level services.dnsmasq and add UDP 67 to firewall for DHCP
proxy mode.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-27 14:02:41 +10:00
beatzaplenty 94842875d0 Merge branch 'main' of https://gitea.lan.ddnsgeek.com/beatzaplenty/nixos into worktree-network-cutover-plan 2026-07-27 14:01:24 +10:00
beatzaplenty cfa36b97fc added dnsmasq to pxe-boot 2026-07-27 13:44:29 +10:00
beatzaplentyandClaude Sonnet 4.6 4444398cac fix(network): wire correct IPs throughout and add cutover plan
variables.nix:
- HA server LAN IPs: .200/.201/.202 → .228/.227/.229 (from ip-addressing.md)
- Add haServer1StorageIp (.228), haServer2StorageIp (.227) for 192.168.4.0/29
- Add haStorageCidr for firewall rules

ha-server host.nix (both nodes):
- Add ens19 interface on storage subnet (/29)
- Fix defaultGateway: 192.168.2.1 → 192.168.2.254
- Fix nameservers: 192.168.2.1/8.8.8.8 → domainControllerIp (.253)

cluster-config.nix:
- DRBD replication addresses: LAN IPs → storage IPs (keep replication off LAN)
- Corosync ring_addrs: LAN IPs → storage IPs
- Firewall: add haStorageCidr to allowed sources

docs/network-cutover.md: step-by-step cutover plan with rollback at every stage

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-27 12:32:54 +10:00
25 changed files with 1142 additions and 60 deletions

No files matched your search

+30
View File
@@ -63,6 +63,16 @@ creation_rules:
- *lxc-nix-cache
- *proxmox-nix-cache
# Host keytab for nix-cache FreeIPA enrollment (binary sops file).
# Generate with: sops -e --input-type binary /tmp/nix-cache.keytab > secrets/nix-cache.keytab
- path_regex: secrets/nix-cache\.keytab$
key_groups:
- age:
- *admin
- *linode-nix-cache
- *lxc-nix-cache
- *proxmox-nix-cache
- path_regex: secrets/server\.yaml$
key_groups:
- age:
@@ -117,6 +127,16 @@ creation_rules:
# scripts/secrets/sync-host-keys.sh yet, so whichever variant is actually
# deployed next needs its recipient added here (and `sops updatekeys` rerun)
# before it can decrypt this.
# Host keytab for tailscale-router FreeIPA enrollment (binary sops file).
# Generated by scripts/ipa/create-nixos-ipa-host-account.sh.
- path_regex: secrets/tailscale-router\.keytab$
key_groups:
- age:
- *admin
- *lxc-tailscale-router
- *proxmox-tailscale-router
- *linode-tailscale-router
- path_regex: secrets/gui\.yaml$
key_groups:
- age:
@@ -125,3 +145,13 @@ creation_rules:
- *baremetal-gui
- *linode-gui
- *proxmox-gui
# Host keytab for tailscale-router FreeIPA enrollment (binary sops file).
# Generated by scripts/ipa/create-nixos-ipa-host-account.sh.
- path_regex: secrets/tailscale-router\.keytab$
key_groups:
- age:
- *admin
- *lxc-tailscale-router
- *proxmox-tailscale-router
- *linode-tailscale-router
+25
View File
@@ -0,0 +1,25 @@
-----BEGIN CERTIFICATE-----
MIIESDCCArCgAwIBAgIBATANBgkqhkiG9w0BAQsFADA1MRMwEQYDVQQKDApTV0VF
VC5IT01FMR4wHAYDVQQDDBVDZXJ0aWZpY2F0ZSBBdXRob3JpdHkwHhcNMjYwNzI2
MjExMzQxWhcNNDYwNzI2MjExMzQxWjA1MRMwEQYDVQQKDApTV0VFVC5IT01FMR4w
HAYDVQQDDBVDZXJ0aWZpY2F0ZSBBdXRob3JpdHkwggGiMA0GCSqGSIb3DQEBAQUA
A4IBjwAwggGKAoIBgQCzljYktbHdMGVJ6Wq0XQJuHLN6dkCSOgtoIzQtriPQkkNI
uo28LwobaiQQ8sX4kGRH/BTKnH8QlId/jug4Uc+sDHnABYu++AiOhPbBX8gCpRQ0
hebBjZiktHSBUEJR31siWOVdBoKBDJEoxehx7XUXvcxIJcaRN+LHYjO86nJN55HB
VwFU2JcYDk98c+144dFJxXdr++MjWe4Z/oVVU8JHIOtNtKhVhvij6oOSWxcYoJO/
S80LRj1vx/o6o/3G6bYug7PjY7JjZk/Oj61whijZkcsoO1MXSYI6UywJZGflv+ZB
7HyufdYAsK3WhE8O2FX3/kq64Ol83HNtoR8Dt68rTg1xpW6K45jS6iDPKueYGkb0
oSx7e++90VAW2PDhj6QQ3JJ4O5VQwrrecekJzUrAean0FOEbmgyi4PsEp1Vk6LDQ
SsIn1x0euyxVivQMlzNX2XrZL3urn1BNPqAdntXQMkR0Wl8sbUiJPe0kxG52CGXs
6yfNEXbPmVGcC0TBdGECAwEAAaNjMGEwHQYDVR0OBBYEFLh5QbI1UWMH0WR4z8bG
lhrOX3X5MB8GA1UdIwQYMBaAFLh5QbI1UWMH0WR4z8bGlhrOX3X5MA8GA1UdEwEB
/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgHGMA0GCSqGSIb3DQEBCwUAA4IBgQCVodVN
owwo53OQe02QhtEbIur2PL7zIfvhvCTRD4J8gwpbMIqT7JQK0tV6Mvsg2L8yTb2O
KjrWeLKHGWaZZlhGSPTbkMFdb/Ls8M9FSnkc2bwcdWW3Z1lOiCjBYYqwLCG6JhvB
5SXVwWNJwXeasL2m7oFTSwhsqPpARJ2t25u2N35o+tqIoCjijKwkmEOT66N9EAbu
2VQjtYZWPkBtP4YCe0Ey6u4oy7sy8ThNAjOylZok+J4JW7QEFjK4Q/emhA4aQq5H
gg9qgMuG+5oi6D1g2Wy+fMTRBaukJtLYZbBpQMQhMYWg44uPp/2bbNPTID/nV1KB
GcPyHaskcVxPdYWxAPMwk3AeJXWyOq7atAPTF5sbk0kQQf2m+vyOqcli5CxRMUgV
rcyi9l6+dZW4U+38Q0ET5M3OuxNI4hA7kVY2cfTakXWNqh97+TIHnstblDhAxECK
6ZLMJQYUy7LqJTX84H27CBWLexEMjXwdr5HCV88Fj6mAK0fRufnIw5FeneA=
-----END CERTIFICATE-----
+366
View File
@@ -0,0 +1,366 @@
# Network Cutover Plan
Moves the LAN from the current flat/Pi-hole-managed state to the new IP scheme
defined in `docs/ip-addressing.md`. Works in five independent stages — each
stage is safe to pause after and resume later. Rollback steps are given at
every point where something can break.
**Before starting anything:** confirm you have
- SSH access to `192.168.2.138` (domain-controller, current IP)
- SSH access to `192.168.2.250` (pve1)
- Browser access to Pi-hole admin at `http://192.168.2.253`
- Browser access to router admin at `http://192.168.2.254`
- The FreeIPA `admin` password to hand
---
## Stage 1 — Prepare FreeIPA DNS (zero downtime)
Everything here is additive. Pi-hole keeps running. Nothing breaks if you stop
mid-stage.
### 1a. Add NextDNS forwarders
```bash
ssh wayne@192.168.2.138
kinit admin # enter FreeIPA admin password when prompted
ipa dnsconfig-mod \
--forwarder=45.90.28.142 \
--forwarder=45.90.30.142 \
--forward-policy=only
```
**Verify external resolution works through FreeIPA before continuing:**
```bash
dig @127.0.0.1 google.com +short # must return an IP, not SERVFAIL
```
### 1b. Add A records for every host at their CURRENT IPs
These represent the live state now. You'll update each record to the new IP
when you renumber that host in Stage 5.
```bash
ipa dnsrecord-add sweet.home pve1 --a-rec 192.168.2.250
ipa dnsrecord-add sweet.home pbs --a-rec 192.168.2.108
ipa dnsrecord-add sweet.home nixos --a-rec 192.168.2.119
ipa dnsrecord-add sweet.home server --a-rec 192.168.2.252
ipa dnsrecord-add sweet.home docker --a-rec 192.168.2.249
ipa dnsrecord-add sweet.home nix-cache --a-rec 192.168.2.120
ipa dnsrecord-add sweet.home pxe-boot --a-rec 192.168.2.247
ipa dnsrecord-add sweet.home tailscale-router --a-rec 192.168.2.121
ipa dnsrecord-add sweet.home tor-relay --a-rec 192.168.2.107
ipa dnsrecord-add sweet.home pdm --a-rec 192.168.2.248
ipa dnsrecord-add sweet.home router --a-rec 192.168.2.254
```
### 1c. Clean up stale reverse-zone PTR records
FreeIPA already has PTR records from an earlier import but some are wrong.
Fix them now so reverse DNS is accurate from day one.
```bash
# Remove stale "win11" entry at .250 (should be pve1)
ipa dnsrecord-del 2.168.192.in-addr.arpa 250 --ptr-rec win11.
ipa dnsrecord-add 2.168.192.in-addr.arpa 250 --ptr-rec pve1.sweet.home.
# Fix unqualified PTR records (missing .sweet.home. suffix)
ipa dnsrecord-mod 2.168.192.in-addr.arpa 108 --ptr-rec pbs.sweet.home.
ipa dnsrecord-mod 2.168.192.in-addr.arpa 248 --ptr-rec pdm.sweet.home.
ipa dnsrecord-mod 2.168.192.in-addr.arpa 249 --ptr-rec docker.sweet.home.
ipa dnsrecord-mod 2.168.192.in-addr.arpa 252 --ptr-rec server.sweet.home.
# Add any missing PTR records
ipa dnsrecord-add 2.168.192.in-addr.arpa 119 --ptr-rec nixos.sweet.home.
ipa dnsrecord-add 2.168.192.in-addr.arpa 120 --ptr-rec nix-cache.sweet.home.
ipa dnsrecord-add 2.168.192.in-addr.arpa 121 --ptr-rec tailscale-router.sweet.home.
ipa dnsrecord-add 2.168.192.in-addr.arpa 247 --ptr-rec pxe-boot.sweet.home.
ipa dnsrecord-add 2.168.192.in-addr.arpa 254 --ptr-rec router.sweet.home.
```
### 1d. Point domain-controller's own DNS at itself
```bash
sudo nmcli connection modify "System eth0" ipv4.dns "127.0.0.1"
sudo nmcli connection up "System eth0"
```
**Verify:**
```bash
dig pve1.sweet.home +short # must return 192.168.2.250
dig google.com +short # must return an IP (NextDNS forwarding)
```
**Rollback 1d:** `sudo nmcli connection modify "System eth0" ipv4.dns "192.168.2.253" && sudo nmcli connection up "System eth0"`
---
## Stage 2 — Move pxe-boot DHCP options off Pi-hole (zero downtime)
Pi-hole's dnsmasq currently serves the iPXE boot options via
`99-ipxe-chainload.conf`. Before Pi-hole is retired, that config must move to
the pxe-boot CT running dnsmasq in proxy mode so PXE boot keeps working.
### 2a. Add dnsmasq proxy config to the pxe-boot NixOS module
In `modules/build-types/pxe-boot.nix`, add:
```nix
services.dnsmasq = {
enable = true;
settings = {
# Proxy mode: respond only to PXE DHCP requests, leave normal leases to router
dhcp-range = [ "192.168.2.0,proxy" ];
# iPXE client detection
dhcp-match = [
"set:ipxe,175"
"set:efi64,option:client-arch,7"
"set:efi64,option:client-arch,9"
];
dhcp-userclass = "set:ipxe,iPXE";
# Boot file selection
dhcp-boot = [
"tag:ipxe,tag:efi64,http://${vars.pxeServerIp}/boot.ipxe"
"tag:ipxe,http://${vars.pxeServerIp}/boot.ipxe"
"tag:efi64,ipxe.efi,,${vars.pxeServerIp}"
"undionly.kpxe,,${vars.pxeServerIp}"
];
};
};
```
### 2b. Rebuild and deploy the pxe-boot CT
```bash
# On pve1 — build the new tarball
nix build .#lxc-pxe-boot.config.system.build.tarball
# Verify dnsmasq starts correctly in the CT after deploy
ssh nixos@192.168.2.247 systemctl status dnsmasq
```
### 2c. Remove the iPXE config from Pi-hole
In the Pi-hole CT, remove `/etc/dnsmasq.d/99-ipxe-chainload.conf` and
restart the FTL service:
```bash
ssh wayne@pve1.sweet.home \
"sudo pct exec 100 -- bash -c 'rm /etc/dnsmasq.d/99-ipxe-chainload.conf && systemctl restart pihole-FTL'"
```
**Verify:** PXE boot a test machine — it should still get an iPXE response and
reach the boot menu.
**Rollback 2c:** restore the file from the Pi-hole config backup at
`/etc/pihole/config_backups/` and restart pihole-FTL.
---
## Stage 3 — DHCP migration: Pi-hole → router (brief maintenance window)
**Do this in the evening.** Existing DHCP leases stay valid during the
switchover so connected devices don't drop — only new lease requests fail
during the gap, which is under 60 seconds if you follow the steps in order.
The key: configure the router's DHCP DNS option to point at `.253` (Pi-hole's
current IP). This way, all new leases issued by the router still get the same
DNS server address — clients never need to change their DNS config. When Pi-hole
is retired and the DC takes `.253` in Stage 4, `.253` just starts answering
differently. No client reconfiguration.
### 3a. Pre-configure router DHCP (do not enable yet)
Log into `http://192.168.2.254`, find the DHCP settings and fill in — but
leave DHCP **disabled** until step 3b:
| Setting | Value |
|---|---|
| Start IP | 192.168.2.10 |
| End IP | 192.168.2.59 |
| Subnet mask | 255.255.255.0 |
| Gateway | 192.168.2.254 |
| Primary DNS | 192.168.2.253 |
| Secondary DNS | *(leave blank)* |
| Lease time | 24h |
Save without enabling.
### 3b. Switchover (do steps in quick succession)
1. **Disable Pi-hole DHCP:** Pi-hole admin UI → Settings → DHCP → uncheck
"DHCP server enabled" → Save
2. **Enable router DHCP** immediately after step 1
### 3c. Verify router DHCP is working
On a phone or laptop, disconnect from WiFi and reconnect (or run
`sudo dhclient -r && sudo dhclient` on a Linux host):
```bash
ip addr show # IP should be in 192.168.2.10–59 range
dig google.com # should resolve (Pi-hole DNS still running at .253)
dig pve1.sweet.home # should resolve via FreeIPA at .138 (relayed via Pi-hole)
```
Wait 10–15 minutes for the most active devices to renew their leases. There's
no need to wait for all leases to expire before proceeding.
**Rollback 3b:** Re-enable Pi-hole DHCP. Disable router DHCP. Done — existing
leases remain valid so most devices are unaffected.
---
## Stage 4 — Move domain-controller from .138 to .253
Pi-hole lives at `.253`. The DC must take `.253` the moment Pi-hole stops so
clients that still have `.253` as their DNS server don't notice the change.
Script these commands in advance and run them in rapid succession.
**Pre-stage: have this SSH command ready before running step 4a:**
```bash
ssh wayne@192.168.2.138 "
sudo nmcli connection modify 'System eth0' \
ipv4.addresses '192.168.2.253/24' \
ipv4.gateway '192.168.2.254' \
ipv4.dns '127.0.0.1' \
ipv4.method manual && \
sudo nmcli connection up 'System eth0'
"
```
**Also update the Proxmox VM config to match (run from pve1):**
```bash
sudo qm set 108 \
--ipconfig0 ip=192.168.2.253/24,gw=192.168.2.254 \
--nameserver 192.168.2.253
```
### 4a. Stop Pi-hole
```bash
ssh wayne@pve1.sweet.home "sudo pct stop 100"
```
### 4b. Immediately: change DC's IP to .253
Run the pre-staged SSH command from above. You have ~30 seconds before any
client notices Pi-hole is gone. If SSH to `.138` refuses (the IP is already
changing), open a Proxmox console to VM 108 and run the `nmcli` commands
there.
### 4c. Update Proxmox VM config
Run the pre-staged `qm set 108` command from above.
### 4d. Verify
```bash
ssh wayne@192.168.2.253 # must connect (new DC IP)
dig @192.168.2.253 pve1.sweet.home +short # must return 192.168.2.250
dig @192.168.2.253 google.com +short # must return an IP
```
From a client device that renewed its DHCP lease in Stage 3:
```bash
cat /etc/resolv.conf # should show 192.168.2.253
dig pve1.sweet.home # should resolve
```
**Rollback 4:** `ssh wayne@pve1.sweet.home "sudo pct start 100"`. Change DC IP
back to .138 via Proxmox console. This restores full Pi-hole DNS/DHCP service.
Leave Pi-hole CT stopped-but-intact for 48 hours before deleting it.
---
## Stage 5 — Host renumbering (one at a time, any order)
For each host:
1. Update FreeIPA DNS A record and PTR record to the new IP
2. Change the static IP on the host itself
3. Verify SSH to new IP
4. Update `variables.nix` if that host has an IP variable (pxe-boot, pbs — already done in this PR)
**FreeIPA record update template** (run as admin on domain-controller):
```bash
ipa dnsrecord-mod sweet.home <hostname> --a-rec <new-ip>
ipa dnsrecord-del 2.168.192.in-addr.arpa <old-last-octet> --ptr-rec <hostname>.sweet.home.
ipa dnsrecord-add 2.168.192.in-addr.arpa <new-last-octet> --ptr-rec <hostname>.sweet.home.
```
### Renumbering order
| # | Host | Old IP | New IP | How to change IP |
|---|---|---|---|---|
| 1 | nixos workstation | .119 | .243 | NetworkManager on guest; or `nmcli connection modify` |
| 2 | nix-cache | .120 | .224 | `pct set 102 --net0 name=eth0,bridge=vmbr0,ip=192.168.2.224/24,gw=192.168.2.254` then `pct reboot 102` |
| 3 | tailscale-router | .121 | .222 | Static config on guest; check Tailscale ACLs if IP is referenced there |
| 4 | tor-relay | .107 | .221 | `pct set 104 --net0 name=eth0,bridge=vmbr0,ip=192.168.2.221/24,gw=192.168.2.254` then `pct reboot 104` |
| 5 | pdm | .248 | .220 | `pct set 106 --net0 name=eth0,bridge=vmbr0,ip=192.168.2.220/24,gw=192.168.2.254` then `pct reboot 106` |
| 6 | pxe-boot | .247 | .223 | `pct set 103 --net0 name=eth0,bridge=vmbr0,ip=192.168.2.223/24,gw=192.168.2.254` then rebuild NixOS (already updated in variables.nix) |
| 7 | server | .252 | .226 | Static config on guest; NFS clients (docker) lose mounts briefly — they remount automatically |
| 8 | docker | .249 | .225 | Static config on guest; do this after server is at .226 |
| 9 | pbs | .108 | .244 | Static config on PBS host itself; update in `pbsIp` already done in variables.nix |
| 10 | pve1 | .250 | .245 | Edit `/etc/network/interfaces` on the Proxmox host — see below |
### pve1 renumber (step 10 — do last)
All guests keep running; only the Proxmox web UI is briefly unreachable.
```bash
ssh wayne@pve1.sweet.home
# Edit /etc/network/interfaces: change address from .250 to .245
sudo nano /etc/network/interfaces
# Change: address 192.168.2.250/24
# To: address 192.168.2.245/24
sudo systemctl restart networking
# SSH will drop here — reconnect to new IP
```
```bash
ssh wayne@192.168.2.245 # verify
```
Update FreeIPA DNS:
```bash
ipa dnsrecord-mod sweet.home pve1 --a-rec 192.168.2.245
ipa dnsrecord-del 2.168.192.in-addr.arpa 250 --ptr-rec pve1.sweet.home.
ipa dnsrecord-add 2.168.192.in-addr.arpa 245 --ptr-rec pve1.sweet.home.
```
**Rollback any step 5 host:** change the IP back on the guest and update the
FreeIPA record back to the old IP. The old IP is unoccupied so you can
temporarily use either.
---
## Stage 6 — Final cleanup
Once all hosts are at their new IPs and verified:
```bash
# Delete the Pi-hole CT (already stopped since Stage 4)
ssh wayne@pve1.sweet.home "sudo pct destroy 100"
# Remove stale FreeIPA records for retired addresses
ipa dnsrecord-del sweet.home pihole --del-all
ipa dnsrecord-del 2.168.192.in-addr.arpa 253 --ptr-rec pihole.sweet.home.
# Rebuild any NixOS hosts that reference pbsIp or pxeServerIp to pick up
# the updated variables.nix values (pxe-boot mandatory; others as convenient)
```
---
## Rollback summary
| What broke | How to roll back |
|---|---|
| FreeIPA DNS not resolving | Check `systemctl status named` on DC; restart if failed |
| FreeIPA DNS unreachable | `pct start 100` on pve1 (restores Pi-hole) |
| Router DHCP not handing out leases | Re-enable Pi-hole DHCP; disable router DHCP |
| DC unreachable after IP change | Proxmox console on VM 108 → `nmcli connection up "System eth0"` with old IP |
| Host unreachable after renumber | Proxmox console → revert IP; or `pct set <id> --net0 ...` old IP and reboot CT |
| pve1 web UI gone after renumber | SSH to .245 and check `/etc/network/interfaces`; if wrong, fix and restart networking |
Generated
+6 -6
View File
@@ -173,11 +173,11 @@
]
},
"locked": {
"lastModified": 1784350909,
"narHash": "sha256-ZWyzLbS1yKUTeFJLmdVuWNnHttL333/ldJbEE+KzCrM=",
"lastModified": 1785119570,
"narHash": "sha256-Rgs2xKnGLFWQscxUaXX07oyZeuMDOHEbqDOsgliLFGM=",
"owner": "nix-community",
"repo": "home-manager",
"rev": "4ce190229c73d44536caa7072f6308fb2d8feeb3",
"rev": "d4fd24667c8cbef124bb70a20380cab75ec8474d",
"type": "github"
},
"original": {
@@ -259,11 +259,11 @@
},
"nixpkgs_2": {
"locked": {
"lastModified": 1784432872,
"narHash": "sha256-n3gKTBIV4ZA5VQpUakffBe3KGu4+mhPoA34rrqS0GkA=",
"lastModified": 1785104993,
"narHash": "sha256-eKbrvPoAOFutbYMdbB3r5EQVmFxKv24iKqHPPUXA0gM=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "fd1462031fdee08f65fd0b4c6b64e22239a77870",
"rev": "8623c4c20aa4ca2f5fb81510d2944066c3fb0d96",
"type": "github"
},
"original": {
+12 -3
View File
@@ -1,8 +1,17 @@
_:
{ vars, ... }:
{
networking.hostName = "docker";
networking.hostId = "007f0200";
networking = {
hostName = "docker";
hostId = "007f0200";
useDHCP = false;
interfaces.${vars.vmLanInterface}.ipv4.addresses = [{
address = vars.dockerIp;
prefixLength = vars.lanPrefixLength;
}];
defaultGateway = { address = vars.lanGateway; interface = vars.vmLanInterface; };
nameservers = [ vars.domainControllerIp ];
};
boot.zfs.forceImportRoot = false;
# Preserved from the pre-refactor `docker` target — stateVersion must never
+8 -4
View File
@@ -11,12 +11,16 @@
hostName = vars.haServer1Host;
hostId = "3a4b5c6d";
useDHCP = false;
interfaces.ens18.ipv4.addresses = [{
interfaces.${vars.vmLanInterface}.ipv4.addresses = [{
address = vars.haServer1Ip;
prefixLength = 24;
prefixLength = vars.lanPrefixLength;
}];
defaultGateway = "192.168.2.1";
nameservers = [ "192.168.2.1" "8.8.8.8" ];
interfaces.${vars.vmStorageInterface}.ipv4.addresses = [{
address = vars.haServer1StorageIp;
prefixLength = vars.haStoragePrefixLength;
}];
defaultGateway = { address = vars.lanGateway; interface = vars.vmLanInterface; };
nameservers = [ vars.domainControllerIp ];
};
# Set KEY after pairing this host with the beszel hub; the token is sops-managed.
+8 -4
View File
@@ -11,12 +11,16 @@
hostName = vars.haServer2Host;
hostId = "7e8f9a0b";
useDHCP = false;
interfaces.ens18.ipv4.addresses = [{
interfaces.${vars.vmLanInterface}.ipv4.addresses = [{
address = vars.haServer2Ip;
prefixLength = 24;
prefixLength = vars.lanPrefixLength;
}];
defaultGateway = "192.168.2.1";
nameservers = [ "192.168.2.1" "8.8.8.8" ];
interfaces.${vars.vmStorageInterface}.ipv4.addresses = [{
address = vars.haServer2StorageIp;
prefixLength = vars.haStoragePrefixLength;
}];
defaultGateway = { address = vars.lanGateway; interface = vars.vmLanInterface; };
nameservers = [ vars.domainControllerIp ];
};
# Set KEY after pairing this host with the beszel hub; the token is sops-managed.
+15 -1
View File
@@ -6,9 +6,23 @@
name = "nix-cache";
sopsFile = ../../secrets/nix-cache.yaml;
})
(import ../../modules/ipa/client.nix {
keytabSopsFile = ../../secrets/nix-cache.keytab;
caCertFile = ../../certs/ipa-ca.crt;
})
];
networking.hostName = vars.nixCacheHost;
networking = {
hostName = vars.nixCacheHost;
domain = vars.homeDomain;
useDHCP = false;
interfaces.${vars.lxcLanInterface}.ipv4.addresses = [{
address = vars.nixCacheIp;
prefixLength = vars.lanPrefixLength;
}];
defaultGateway = { address = vars.lanGateway; interface = vars.lxcLanInterface; };
nameservers = [ vars.domainControllerIp ];
};
services.beszel.agent.environment = {
#DOCKER_HOST = "tcp://docker-socket-proxy:2375";
+11 -2
View File
@@ -1,7 +1,16 @@
_:
{ vars, ... }:
{
networking.hostName = "pxe-boot";
networking = {
hostName = "pxe-boot";
useDHCP = false;
interfaces.${vars.lxcLanInterface}.ipv4.addresses = [{
address = vars.pxeServerIp;
prefixLength = vars.lanPrefixLength;
}];
defaultGateway = { address = vars.lanGateway; interface = vars.lxcLanInterface; };
nameservers = [ vars.domainControllerIp ];
};
# Preserved from the pre-refactor `pxe-boot` target — stateVersion must
# never be bumped on an already-installed machine.
+11 -2
View File
@@ -8,8 +8,17 @@
})
];
networking.hostName = vars.nfsServerHost;
networking.hostId = "6689f93e";
networking = {
hostName = vars.nfsServerHost;
hostId = "6689f93e";
useDHCP = false;
interfaces.${vars.vmLanInterface}.ipv4.addresses = [{
address = vars.serverIp;
prefixLength = vars.lanPrefixLength;
}];
defaultGateway = { address = vars.lanGateway; interface = vars.vmLanInterface; };
nameservers = [ vars.domainControllerIp ];
};
services.beszel.agent.environment = {
#DOCKER_HOST = "tcp://docker-socket-proxy:2375";
+16 -2
View File
@@ -1,4 +1,4 @@
_:
{ vars, ... }:
{
imports = [
@@ -6,9 +6,23 @@ _:
name = "tailscale-router";
sopsFile = ../../secrets/tailscale-router.yaml;
})
(import ../../modules/ipa/client.nix {
keytabSopsFile = ../../secrets/tailscale-router.keytab;
caCertFile = ../../certs/ipa-ca.crt;
})
];
networking.hostName = "tailscale-router";
networking = {
hostName = "tailscale-router";
domain = vars.homeDomain;
useDHCP = false;
interfaces.${vars.lxcLanInterface}.ipv4.addresses = [{
address = vars.tailscaleRouterIp;
prefixLength = vars.lanPrefixLength;
}];
defaultGateway = { address = vars.lanGateway; interface = vars.lxcLanInterface; };
nameservers = [ vars.domainControllerIp ];
};
services.beszel.agent.environment = {
KEY = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFPR9kwtC4TAeTRu46A7+opZsYpxqkRJ+x/ZyB2GWCeG";
+13 -4
View File
@@ -1,4 +1,4 @@
{ ... }:
{ vars, ... }:
{
imports = [
@@ -8,10 +8,19 @@
})
];
networking.hostName = "tor-relay";
networking = {
hostName = "tor-relay";
useDHCP = false;
interfaces.${vars.lxcLanInterface}.ipv4.addresses = [{
address = vars.torRelayIp;
prefixLength = vars.lanPrefixLength;
}];
defaultGateway = { address = vars.lanGateway; interface = vars.lxcLanInterface; };
nameservers = [ vars.domainControllerIp ];
};
# No networking.hostId: only ZFS-touching hosts (server, docker) need one
# for pool-import safety, and this host does neither.
# No networking.hostId: only ZFS-touching hosts need one for pool-import
# safety, and this host does neither.
services.beszel.agent.environment = {
KEY = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFPR9kwtC4TAeTRu46A7+opZsYpxqkRJ+x/ZyB2GWCeG";
+12 -1
View File
@@ -1,6 +1,17 @@
{ config, pkgs, lib, inputs, vars, ... }:
{
imports = [
../docker/enable-service.nix
];
nixpkgs.overlays = [
(final: prev: {
docker = prev.docker_29;
docker_cli = prev.docker_29;
})
];
environment.systemPackages = with pkgs; [
inputs.nixos-conf-editor.packages.${pkgs.stdenv.hostPlatform.system}.nixos-conf-editor
nodejs
@@ -18,7 +29,7 @@
];
boot.loader.grub.useOSProber = true;
programs.direnv.enable = true;
programs.direnv.enable = true;
services = {
xserver = {
enable = true;
+30 -7
View File
@@ -82,7 +82,7 @@ let
rockyFreeIpaKs = pkgs.writeText "rocky-freeipa.ks" ''
#version=RHEL9
# Unattended Rocky Linux 9 + FreeIPA install
# Target: domain-controller.sweet.home 192.168.2.138
# Target: domain-controller.${vars.homeDomain} ${vars.domainControllerIp}
url --url=${rockyMirror}/BaseOS/${rockyArch}/os/
repo --name=appstream --baseurl=${rockyMirror}/AppStream/${rockyArch}/os/
@@ -136,10 +136,10 @@ let
[ipv4]
method=manual
addresses=192.168.2.138/24
gateway=192.168.2.254
dns=192.168.2.253;
dns-search=sweet.home;
addresses=${vars.domainControllerIp}/${toString vars.lanPrefixLength}
gateway=${vars.lanGateway}
dns=${vars.domainControllerIp};
dns-search=${vars.homeDomain};
[ipv6]
method=auto
@@ -148,7 +148,7 @@ let
# -- /etc/hosts: FQDN must resolve to the real IP (not loopback) for IPA --
sed -i '/domain-controller/d' /etc/hosts
echo '192.168.2.138 domain-controller.sweet.home domain-controller' >> /etc/hosts
echo '${vars.domainControllerIp} domain-controller.${vars.homeDomain} domain-controller' >> /etc/hosts
# -- Generate IPA passwords and store securely --
DM_PASS=$(openssl rand -base64 24 | tr -dc 'A-Za-z0-9' | head -c 24)
@@ -420,6 +420,29 @@ in
};
};
services.dnsmasq = {
enable = true;
settings = {
# Disable DNS listener — only proxy DHCP is needed here.
# Without this dnsmasq tries to bind port 53 which systemd-resolved
# already owns, causing startup failure.
port = 0;
dhcp-range = [ "192.168.2.0,proxy" ];
dhcp-match = [
"set:ipxe,175"
"set:efi64,option:client-arch,7"
"set:efi64,option:client-arch,9"
];
dhcp-userclass = "set:ipxe,iPXE";
dhcp-boot = [
"tag:ipxe,tag:efi64,http://${vars.pxeServerIp}/boot.ipxe"
"tag:ipxe,http://${vars.pxeServerIp}/boot.ipxe"
"tag:efi64,ipxe.efi,,${vars.pxeServerIp}"
"undionly.kpxe,,${vars.pxeServerIp}"
];
};
};
networking.firewall.allowedTCPPorts = [ vars.ports.pxeBootHttp ];
networking.firewall.allowedUDPPorts = [ vars.ports.pxeBootTftp ];
networking.firewall.allowedUDPPorts = [ vars.ports.pxeBootTftp 67 ];
}
+26 -9
View File
@@ -44,14 +44,8 @@ in
exit 0
fi
# Pool exists on a device but not yet imported — let the standard
# zfs-import-${poolName}.service handle it normally.
if zpool import -d /dev/disk/by-id -N "${poolName}" 2>/dev/null; then
exit 0
fi
# No pool found at all. Create it on the Proxmox data disk (scsi1),
# which appears as /dev/disk/by-id/scsi-*drive-scsi1 inside the VM.
# Locate the data disk first — used for both the fallback import
# attempt and, only if the disk is genuinely blank, pool creation.
DATA_DISK=""
for candidate in /dev/disk/by-id/scsi-*drive-scsi1; do
[[ "$candidate" == *-part* ]] && continue
@@ -63,8 +57,31 @@ in
exit 1
fi
# Try importing via the by-id symlink directory first (normal path),
# then fall back to scanning the disk directly. The two-step exists
# because of a udev race: systemd-udev-settle.service can clear before
# /dev/disk/by-id/ entries are fully populated, causing the first
# import to fail even when the pool is intact on the disk.
if zpool import -d /dev/disk/by-id -N "${poolName}" 2>/dev/null; then
exit 0
fi
if zpool import -d "$DATA_DISK" -N "${poolName}" 2>/dev/null; then
exit 0
fi
# Both import attempts failed. Before creating a new pool, verify the
# disk is genuinely blank — if ZFS label metadata is present the import
# failed for some other reason and we must not clobber existing data.
if zdb -l "$DATA_DISK" 2>/dev/null | grep -q "name: '${poolName}'"; then
echo "zfs-init-${poolName}: $DATA_DISK has ZFS pool '${poolName}' metadata but import failed — refusing to overwrite existing data. Run 'zpool import -d $DATA_DISK ${poolName}' manually to investigate." >&2
exit 1
fi
# Disk is genuinely blank: create the pool. -f is intentionally
# omitted so that if we somehow reach this point with an existing pool
# on the disk, zpool refuses rather than silently destroying data.
echo "zfs-init-${poolName}: creating pool on $DATA_DISK"
zpool create -f "${poolName}" "$DATA_DISK"
zpool create "${poolName}" "$DATA_DISK"
${lib.concatMapStrings (ds: ''
zfs create "${poolName}/${ds}"
'') poolDatasets}
+1
View File
@@ -3,6 +3,7 @@
{
imports = [
../tailscale/subnet-router.nix
../tailscale/ts-dns-forwarder.nix
../beszel/enable-agent.nix
];
+2 -2
View File
@@ -1,4 +1,4 @@
{ pkgs, ... }:
{ pkgs, vars, ... }:
{
# virtualisation.docker.enable = true;
@@ -15,7 +15,7 @@
# experimental = true;
# };
};
users.users.${vars.primaryUser}.extraGroups = [ "docker" ];
environment.systemPackages = with pkgs; [
docker-compose
docker-buildx
+5 -4
View File
@@ -47,11 +47,11 @@
}
on ${vars.haServer1Host} {
address ${vars.haServer1Ip}:${toString vars.ports.haServerDrbd};
address ${vars.haServer1StorageIp}:${toString vars.ports.haServerDrbd};
}
on ${vars.haServer2Host} {
address ${vars.haServer2Ip}:${toString vars.ports.haServerDrbd};
address ${vars.haServer2StorageIp}:${toString vars.ports.haServerDrbd};
}
}
'';
@@ -75,8 +75,8 @@
services.corosync = {
clusterName = "ha-cluster";
nodelist = [
{ nodeid = 1; name = vars.haServer1Host; ring_addrs = [ vars.haServer1Ip ]; }
{ nodeid = 2; name = vars.haServer2Host; ring_addrs = [ vars.haServer2Ip ]; }
{ nodeid = 1; name = vars.haServer1Host; ring_addrs = [ vars.haServer1StorageIp ]; }
{ nodeid = 2; name = vars.haServer2Host; ring_addrs = [ vars.haServer2StorageIp ]; }
];
};
@@ -101,6 +101,7 @@
extraCommands = ''
iptables -A INPUT -s ${vars.haServer1Ip}/32 -j ACCEPT
iptables -A INPUT -s ${vars.haServer2Ip}/32 -j ACCEPT
iptables -A INPUT -s ${vars.haStorageCidr} -j ACCEPT
'';
};
}
+103
View File
@@ -0,0 +1,103 @@
# Fully declarative FreeIPA domain membership.
#
# Configures security.ipa (SSSD, Kerberos, PAM, NSSwitch) and places a
# pre-provisioned host keytab via sops-nix so no imperative ipa-client-install
# step is needed after deployment.
#
# Usage (in a host.nix imports list):
# (import ../../modules/ipa/client.nix {
# keytabSopsFile = ../../secrets/<hostname>.keytab;
# caCertFile = ../../certs/ipa-ca.crt; # already committed — do not re-fetch
# })
#
# The host.nix networking block must also set:
# networking.domain = vars.homeDomain; # needed for Kerberos FQDN
# networking.nameservers = [ vars.domainControllerIp ]; # IPA DNS
#
# One-time operator setup per host (do this BEFORE deploying):
#
# 0. Generate SSH host keys and the host's age key for sops:
# scripts/secrets/sync-host-keys.sh <flake-target>
# This must run before step 1 so the host age key is in .sops.yaml
# and the keytab can be encrypted for the host to read at boot.
#
# 1. Add the IPA host account and produce the sops-encrypted keytab:
# scripts/ipa/create-nixos-ipa-host-account.sh [--ip <addr>] <hostname>
# The script handles ipa host-add, ipa-getkeytab, .sops.yaml patching,
# and sops encryption in one step. See the script header for details.
#
# 2. Wire up the host (see "Usage" above), then deploy:
# nixos-rebuild switch (or create-proxmox-resource.sh)
# No further manual enrollment steps are required after deployment.
#
# Manual fallback (if the script isn't usable):
# a. On the FreeIPA server: ipa host-add <fqdn> [--ip-address=<ip>] --force
# b. On the FreeIPA server: ipa-getkeytab -s <ipa-server> -p host/<fqdn> -k /tmp/<host>.keytab
# c. From the repo root (path must match for sops creation rule to apply):
# cp /tmp/<host>.keytab secrets/<host>.keytab
# sops -e --input-type binary -i secrets/<host>.keytab
# d. Commit secrets/<host>.keytab and the updated .sops.yaml, then deploy.
#
# vars dependencies: homeDomain, ipaServer, domainControllerIp
{ keytabSopsFile, caCertFile }:
{ config, lib, pkgs, vars, ... }:
let
realm = lib.strings.toUpper vars.homeDomain;
fqdn = "${config.networking.hostName}.${vars.homeDomain}";
# "sweet.home" -> "dc=sweet,dc=home"
basedn = lib.strings.concatMapStringsSep "," (c: "dc=${c}") (lib.strings.splitString "." vars.homeDomain);
# security.ipa.certificate expects a derivation (package), not a raw path.
caCertPkg = pkgs.writeText "ipa-ca.crt" (builtins.readFile caCertFile);
in
{
security.ipa = {
enable = true;
domain = vars.homeDomain;
realm = realm;
server = vars.ipaServer;
certificate = caCertPkg;
basedn = basedn;
ipaHostname = fqdn;
offlinePasswords = true;
cacheCredentials = true;
};
# Fetch SSH public keys from IPA so users can log in with the key stored
# in their IPA profile rather than needing ~/.ssh/authorized_keys on every
# host. sss_ssh_authorizedkeys queries SSSD (which queries IPA LDAP).
#
# /nix/store is 1775 (group-writable by nixbld). OpenSSH 10.0+ rejects
# AuthorizedKeysCommand binaries whose path contains any group-writable
# component, silently skipping the command. Copy to /usr/local/bin (all
# components root-owned, 755) so the path passes sshd's safety check.
systemd.tmpfiles.rules = [
"d /usr/local 0755 root root - -"
"d /usr/local/bin 0755 root root - -"
"C+ /usr/local/bin/sss_ssh_authorizedkeys 0555 root root - ${pkgs.sssd}/bin/sss_ssh_authorizedkeys"
];
services.openssh.extraConfig = ''
AuthorizedKeysCommand /usr/local/bin/sss_ssh_authorizedkeys %u
AuthorizedKeysCommandUser nobody
'';
# Create the home directory on first login if it doesn't exist yet.
# IPA users have no pre-created home on the host; without this sshd
# opens a session to a non-existent directory and resets the connection.
security.pam.services.sshd.makeHomeDir = true;
# Host keytab: pre-provisioned on the IPA server, sops-encrypted binary.
# Placed at /etc/krb5.keytab before SSSD starts so the host authenticates
# to IPA without running ipa-client-install.
sops.secrets."ipa-host-keytab" = {
sopsFile = keytabSopsFile;
format = "binary";
path = "/etc/krb5.keytab";
owner = "root";
group = "root";
mode = "0600";
restartUnits = [ "sssd.service" ];
};
}
+60
View File
@@ -0,0 +1,60 @@
{ vars, ... }:
{
# Run dnsmasq on the LAN interface as a forwarding-only resolver for
# *.ts.net (Tailscale MagicDNS names). FreeIPA's bind-dyndb-ldap
# cannot reach 100.100.100.100 (Tailscale's internal resolver) directly
# because the DC is not a Tailscale node. This host IS a Tailscale node
# and can reach 100.100.100.100 via its tailscale0 interface, so it
# acts as an intermediary: FreeIPA has a conditional forward zone for
# ts.net pointing here (vars.tailscaleRouterIp), and this dnsmasq
# instance forwards those queries onward to Tailscale's resolver.
#
# Configure FreeIPA once after deploying this host:
# kinit admin
# ipa dnsforwardzone-add ${vars.tailnetDomain} \
# --forwarder=${vars.tailscaleRouterIp} \
# --forward-policy=only
# Note: IPA refuses to shadow ts.net (a real public TLD); use the
# tailnet-specific subdomain (vars.tailnetDomain) instead.
services.dnsmasq = {
enable = true;
# NixOS's dnsmasq module defaults resolveLocalQueries to true, which adds
# 127.0.0.1 to networking.nameservers and makes dnsmasq bind to
# listen-address=127.0.0.1. This instance is not the host's local
# resolver — it only serves IPA's conditional forwarder for tailnet names.
# The host uses domainControllerIp directly (networking.nameservers in
# host.nix). Without this, all host DNS goes through dnsmasq, which has
# no upstream for general queries (no-resolv=true), breaking resolution.
resolveLocalQueries = false;
settings = {
# Listen only on the LAN interface — not tailscale0 or loopback.
# bind-interfaces prevents dnsmasq from binding to 0.0.0.0 and
# then filtering by interface later; combined with `interface` this
# ensures it genuinely listens only on eth0.
bind-interfaces = true;
interface = [ vars.lxcLanInterface ];
# Forward-only: no local /etc/hosts or /etc/resolv.conf reading,
# no negative caching of NXDOMAIN for names this instance doesn't
# serve. All ts.net queries come from FreeIPA's conditional forwarder
# and must be answered by Tailscale's resolver.
no-hosts = true;
no-resolv = true;
# Tailscale's internal "Quad100" resolver — reachable from any
# Tailscale node via the tailscale0 interface. Scoped to the
# specific tailnet subdomain (vars.tailnetDomain) rather than
# all of ts.net: FreeIPA refuses to shadow ts.net (a real public
# TLD with DNSimple nameservers) so the conditional forward zone
# in FreeIPA must use the tailnet-specific subdomain instead:
# ipa dnsforwardzone-add ${vars.tailnetDomain} \
# --forwarder=${vars.tailscaleRouterIp} \
# --forward-policy=only
server = [ "/${vars.tailnetDomain}/100.100.100.100" ];
};
};
networking.firewall.allowedUDPPorts = [ 53 ];
networking.firewall.allowedTCPPorts = [ 53 ];
}
+11
View File
@@ -88,6 +88,17 @@ export NIX_CACHE_HOST
: "${LAN_DOMAIN:=gitea.lan.ddnsgeek.com}"
export LAN_DOMAIN
# Matches variables.nix's homeDomain -- the base LAN domain for service
# subdomains, FreeIPA Kerberos realm, and host FQDNs.
: "${HOME_DOMAIN:=sweet.home}"
export HOME_DOMAIN
# Matches variables.nix's ipaServer -- the FreeIPA server hostname.
# scripts/ipa/create-nixos-ipa-host-account.sh SSHes here to run
# ipa host-add and ipa-getkeytab.
: "${IPA_SERVER:=domain-controller.sweet.home}"
export IPA_SERVER
# nix_extra_opts: call as a plain statement (NOT inside $(...)/<(...) --
# that forks a subshell, and the whole point is exporting a decision back
# into *this* shell) to populate the global NIX_OPTS array with whatever
+295
View File
@@ -0,0 +1,295 @@
#!/usr/bin/env bash
# Add a NixOS host to the FreeIPA domain and produce a sops-encrypted keytab
# at secrets/<hostname>.keytab, ready for modules/ipa/client.nix.
#
# One command replaces three error-prone manual steps:
# 1. ipa host-add on the domain controller
# 2. ipa-getkeytab on the domain controller + SCP back
# 3. sops encrypt in-place (must be at secrets/<hostname>.keytab for
# the creation rule to match -- the common mistake that breaks sops)
#
# Usage:
# scripts/ipa/create-nixos-ipa-host-account.sh [options] <hostname>
#
# Arguments:
# <hostname> Short hostname, e.g. "tailscale-router". The FQDN is
# derived as <hostname>.<HOME_DOMAIN>.
#
# Options:
# --ip <addr> Register this IP with the IPA host record (optional).
# --dc <host> SSH to this host to run IPA commands.
# Default: $IPA_SERVER (from env.sh / environment).
# --dc-user <u> SSH user on the domain controller. Default: wayne.
# --dry-run Print what would be done without making any changes.
# -h, --help Show this message.
#
# Prereqs:
# 1. Run from the repo root (so .sops.yaml and secrets/ are found).
# 2. SSH access to the domain controller as --dc-user (default: wayne)
# with passwordless sudo (or sudo cached). IPA commands and kinit run
# as root via sudo so the Kerberos ticket is in root's cache where all
# ipa tools expect it. If there's no valid ticket, the script runs
# `sudo kinit admin` interactively — you'll be prompted for the IPA
# admin password once. The password never touches this script.
# 3. The host's age key(s) must already be in .sops.yaml. Run
# scripts/secrets/sync-host-keys.sh <flake-target> first so the host
# can decrypt its own keytab on boot. This script adds the .sops.yaml
# creation rule for secrets/<hostname>.keytab automatically, but the
# host age key anchor (&lxc-<hostname> etc.) must already exist —
# otherwise only the admin key can decrypt the keytab and the deployed
# host will fail to read it.
# 4. sops in PATH, or Nix available to run it via `nix run`.
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
# shellcheck source=../env.sh
source "${SCRIPT_DIR}/../env.sh"
# --- Argument parsing ---
DC_HOST="${IPA_SERVER}"
DC_USER="wayne"
IP_ADDR=""
DRY_RUN=false
TARGET=""
usage() {
sed -n '/^# Usage:/,/^[^#]/{ /^#/{ s/^# \?//; p } }' "$0"
exit "${1:-0}"
}
while [[ $# -gt 0 ]]; do
case "$1" in
--ip) IP_ADDR="$2"; shift 2 ;;
--dc) DC_HOST="$2"; shift 2 ;;
--dc-user) DC_USER="$2"; shift 2 ;;
--dry-run) DRY_RUN=true; shift ;;
-h|--help) usage 0 ;;
-*) echo "Unknown flag: $1" >&2; usage 1 ;;
*)
if [[ -n "${TARGET}" ]]; then echo "Unexpected argument: $1" >&2; usage 1; fi
TARGET="$1"; shift
;;
esac
done
if [[ -z "${TARGET}" ]]; then
echo "Error: hostname required." >&2
usage 1
fi
FQDN="${TARGET}.${HOME_DOMAIN}"
KEYTAB_SECRET="${REPO_ROOT}/secrets/${TARGET}.keytab"
# Temp path on the domain controller — use a name that won't collide.
DC_TMP="/tmp/nixos-keytab-${TARGET}-$$.keytab"
# --- Helpers ---
log() { echo "==> $*"; }
logn() { echo " $*"; }
run() {
if $DRY_RUN; then
echo "[dry-run] $*"
else
"$@"
fi
}
dc_run() {
# Run a command string on the domain controller via SSH.
if $DRY_RUN; then
echo "[dry-run] ssh ${DC_USER}@${DC_HOST} $*"
else
ssh "${DC_USER}@${DC_HOST}" "$@"
fi
}
# --- Locate sops ---
if command -v sops &>/dev/null; then
SOPS_CMD=(sops)
else
log "sops not in PATH — will use 'nix run nixpkgs#sops'"
SOPS_CMD=(nix run "nixpkgs#sops" --)
fi
# --- Preflight checks ---
cd "${REPO_ROOT}"
[[ -f .sops.yaml ]] || { echo "Error: .sops.yaml not found — run from repo root." >&2; exit 1; }
[[ -d secrets ]] || { echo "Error: secrets/ not found — run from repo root." >&2; exit 1; }
# --- Step 1: Ensure .sops.yaml has a creation rule for this keytab ---
#
# sops matches creation rules against the PATH of the file being encrypted,
# not the output path. To match secrets/<hostname>.keytab, the file must
# already be at that path when sops -e -i is called. The creation rule must
# also exist at that point or sops will refuse with "no matching creation
# rules found."
log "Checking .sops.yaml for creation rule: secrets/${TARGET}.keytab"
RULE_EXISTS=false
# Match "path_regex: secrets/<hostname>...keytab" — using .*keytab rather
# than \.keytab because the file stores the regex verbatim (\.keytab = two
# chars: backslash + dot), which a BRE \. (= escaped literal dot) won't span.
if grep -q "path_regex: secrets/${TARGET}.*keytab" .sops.yaml 2>/dev/null; then
RULE_EXISTS=true
logn "Rule already exists — skipping addition."
fi
if ! $RULE_EXISTS; then
# Collect which platform-variant age anchors exist in .sops.yaml for this
# hostname. The keytab is platform-agnostic (same FQDN regardless of
# whether lxc/proxmox/linode variant is deployed), so all platform anchors
# that have been registered get added as recipients.
RECIPIENTS=("*admin")
for platform in lxc proxmox linode; do
anchor="${platform}-${TARGET}"
if grep -q "^ - &${anchor} " .sops.yaml; then
RECIPIENTS+=("*${anchor}")
fi
done
if [[ ${#RECIPIENTS[@]} -eq 1 ]]; then
echo "Warning: no platform age keys found for '${TARGET}' in .sops.yaml." >&2
echo " Run scripts/secrets/sync-host-keys.sh <flake-target> first," >&2
echo " otherwise only the admin key can decrypt the keytab and the" >&2
echo " deployed host won't be able to read it at boot." >&2
echo " Continuing with admin-only encryption..." >&2
fi
# Build the indented recipient list for the YAML block.
RECIPIENT_YAML=""
for r in "${RECIPIENTS[@]}"; do
RECIPIENT_YAML+=" - ${r}"$'\n'
done
RECIPIENT_YAML="${RECIPIENT_YAML%$'\n'}" # strip trailing newline
NEW_RULE="
# Host keytab for ${TARGET} FreeIPA enrollment (binary sops file).
# Generated by scripts/ipa/create-nixos-ipa-host-account.sh.
- path_regex: secrets/${TARGET}\\.keytab\$
key_groups:
- age:
${RECIPIENT_YAML}"
if $DRY_RUN; then
echo "[dry-run] Would append to .sops.yaml:"
echo "${NEW_RULE}"
else
logn "Adding creation rule (recipients: ${RECIPIENTS[*]})"
printf '%s\n' "${NEW_RULE}" >> .sops.yaml
logn "Added."
fi
fi
# --- Step 2: Add IPA host account (idempotent) ---
log "Adding FreeIPA host account: ${FQDN}"
# Ensure there's a valid admin Kerberos ticket on the DC.
# ipa host-add and ipa-getkeytab both need one. All IPA commands run via
# sudo so the ticket must be in root's cache — check and refresh as root.
# ssh -t allocates a PTY so kinit (and sudo if needed) can prompt normally;
# no password ever touches this script or the shell history.
if ! $DRY_RUN; then
if ! ssh "${DC_USER}@${DC_HOST}" "sudo klist -s" &>/dev/null; then
log "No valid Kerberos ticket on ${DC_HOST} — running sudo kinit admin"
ssh -t "${DC_USER}@${DC_HOST}" "sudo kinit admin"
if ! ssh "${DC_USER}@${DC_HOST}" "sudo klist -s" &>/dev/null; then
echo "Error: kinit admin failed or produced no valid ticket." >&2
exit 1
fi
else
logn "Kerberos ticket on ${DC_HOST} is valid."
fi
fi
IP_FLAG=""
[[ -n "${IP_ADDR}" ]] && IP_FLAG="--ip-address=${IP_ADDR}"
# --force: create the host record even if DNS doesn't resolve it yet.
if $DRY_RUN; then
echo "[dry-run] ssh ${DC_USER}@${DC_HOST} sudo ipa host-add '${FQDN}' ${IP_FLAG} --force"
else
HOST_ADD_OUT=$(ssh "${DC_USER}@${DC_HOST}" "sudo ipa host-add '${FQDN}' ${IP_FLAG} --force 2>&1") \
&& HOST_ADD_RC=0 || HOST_ADD_RC=$?
if [[ $HOST_ADD_RC -eq 0 ]]; then
echo "${HOST_ADD_OUT}"
elif echo "${HOST_ADD_OUT}" | grep -q "already exists"; then
logn "(host already registered)"
else
echo "Error: ipa host-add failed (exit ${HOST_ADD_RC}):" >&2
echo "${HOST_ADD_OUT}" >&2
exit 1
fi
fi
# --- Step 3: Fetch the keytab from the domain controller ---
log "Fetching keytab for host/${FQDN}"
# Remove the plaintext keytab if the script aborts before encryption completes.
# The trap is cleared at the end of step 4 once sops has encrypted it in-place.
trap 'rm -f "${KEYTAB_SECRET}"' EXIT
dc_run "sudo ipa-getkeytab -s '${IPA_SERVER}' -p 'host/${FQDN}' -k '${DC_TMP}'"
if $DRY_RUN; then
echo "[dry-run] Would stream ${DC_USER}@${DC_HOST}:${DC_TMP} → secrets/${TARGET}.keytab"
else
logn "Streaming keytab from ${DC_HOST}:${DC_TMP} → secrets/${TARGET}.keytab"
# scp can't read a root-owned temp file as ${DC_USER}; pipe through sudo cat instead.
ssh "${DC_USER}@${DC_HOST}" "sudo cat '${DC_TMP}'" > "${KEYTAB_SECRET}"
logn "Removing temp file on ${DC_HOST}"
dc_run "sudo rm -f '${DC_TMP}'"
fi
# --- Step 4: Encrypt in-place ---
#
# The file must already be at secrets/<hostname>.keytab (done above) so
# sops matches the creation rule by path. Using -i (in-place) rather than
# stdout redirect keeps the path intact through the encrypt call.
log "Encrypting secrets/${TARGET}.keytab in-place with sops"
run "${SOPS_CMD[@]}" -e --input-type binary -i "${KEYTAB_SECRET}"
# Encryption succeeded — the file is now sops-encrypted; cancel the cleanup trap.
trap - EXIT
# --- Done ---
if ! $DRY_RUN; then
echo ""
echo "Done. secrets/${TARGET}.keytab is sops-encrypted and ready."
echo ""
echo "Next steps:"
echo " 1. Verify: grep '\"data\": \"ENC' secrets/${TARGET}.keytab"
echo " 2. Stage and commit:"
echo " git add secrets/${TARGET}.keytab .sops.yaml"
echo " git commit -m 'secrets: add IPA keytab for ${TARGET}'"
echo " 3. Add to hosts/${TARGET}/host.nix (networking block and imports):"
echo ""
echo " networking = {"
echo " hostName = \"${TARGET}\";"
echo " domain = vars.homeDomain; # required for Kerberos FQDN"
echo " nameservers = [ vars.domainControllerIp ]; # IPA DNS"
echo " ..."
echo " };"
echo ""
echo " imports = ["
echo " (import ../../modules/ipa/client.nix {"
echo " keytabSopsFile = ../../secrets/${TARGET}.keytab;"
echo " caCertFile = ../../certs/ipa-ca.crt;"
echo " })"
echo " ];"
echo ""
echo " 4. Deploy: nixos-rebuild switch (or create-proxmox-resource.sh)"
fi
+26
View File
@@ -0,0 +1,26 @@
{
"data": "ENC[AES256_GCM,data:z2bazlADvtWkGcY4an7NgZgIDSDq8KmYpSq8tcBleW/33HlElMNhdmje8m8hd7WxgrVvCQDdlAfodBLj2mjbgSwBptlPX2zbwcmJdm0g0Ope+YyQZucIio7NnBaYa+OK8meWqmxO2WEeOk+hnNfNPbt8MkzF/FMMbP8xv9wSTKP9CDdmO7s8rdg8G1JxVfn9dyaSDUTuA2JGNtbh+osd4CtNCV/bc84MjXDZTgTfT0W/uv8VH3iPNT7BhQbU9FIY4dsFRYBf758VtJ+3mTBiQkR38IevJBvhFB7Wiqn118LSBgjdTKJnBwmGo6lMdo0LRYHEgUnb2ejhWMNie19x+LEAErDki0RBmjc2BbnOF908kUlW2fUPo5fz/hlfqlhDLS1uOxESKhfnKt7s5qzWiMX/nohpknVfyMbmu9j/M571z9sgFnJAylpW3NLp53lX8N9mINwRGXsyL/ijbNE=,iv:M/MNquAGN+lIIyVVvFRgKR6DKAhLclc/OY1HV5Xwoog=,tag:eEuURMErSqsoh+l9ZaHY9Q==,type:str]",
"sops": {
"age": [
{
"enc": "-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBqZlUxRG0zR3IzMlpDL3Vl\nNmRWQXUvc1NDek5wbWtKVEFTcTlxL3ZoWldNCmJVMkFtbDkxN1FGQ2VLTlhBUW5Q\nZ3VzL0UydUQ2YnlUMmtqa1Z0d0FSVFUKLS0tIGJFelc4Rm84T3ZHSGZDdnNUbUFM\nRXZHV3N3aVlSOXhUZk04UFRMWU5TbFUKhPU80PVYuDFUCxu1CA8+W8bqkr+Ne2fh\n+nBUPJbGxfN9TyD9tUC77AMbcL1R2L5x+SSAh0bEgSWE24/RjsuVKw==\n-----END AGE ENCRYPTED FILE-----\n",
"recipient": "age1njap586hc0q43kr03g6c8eqhdsmk8zcafkl3f83xwlc2gqhlmfgs4tmwad"
},
{
"enc": "-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSB3M1o3S2FibHV6N2tIOFEx\nRytJRDFIUU05azNDbWlpaDdFQXFRWlBUNnlFCnBwNVB0SEVwL3NWNS9aeG4xSUI1\nV3NOcUFxUUhaODYyWXdFMEhWeFpvancKLS0tIHY4NHFQRGJwUURQbWZ2cGlSdm9s\nOFJodGZkekk3UnBRRUwxb2YrQ1ZPcU0K0tCapb1hfVHFSNpmESXexYa5k9OE9Tha\n51QpU4mZHKGrnWPK/kyj7rHiz95TLsmwUdA8Q2hOiYNSxEaVBjYDQw==\n-----END AGE ENCRYPTED FILE-----\n",
"recipient": "age1jcx3yajjhghn8qh8za3yeu8nxykzlg3p4nrv03vnfvzl0mzayg2qmg940e"
},
{
"enc": "-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBkTmNrbUFoVFVGcytzQkk1\nMUxDdmJGL0xDcUNZaDFRQnUwNXoyMU90SEVvCi9EaTlVOGlMeUZtQzRJS1J2TTJY\neWdkMGNFMWZzQXVKOU5KcXVvalk1ekkKLS0tIDN2MlRQcG5aQnN3bUMrSDdvc09X\nM29hQ0pJRzk1amRHaS9mRlhrb2FMZG8KEWkSSP+MqGRU75qo7ctOCL7qHhyCM3l4\nL+ga1XOKxRsQkKQPozCi5Bm+k28W9hIDaC34Rw9difxICM9Z1rAbOA==\n-----END AGE ENCRYPTED FILE-----\n",
"recipient": "age1ufg390ydrmma849t9xfkxxl5xvdkk6mngnlzhmy7mvuaje8sgcmsmnq6l7"
},
{
"enc": "-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSA1TUtxVjJ2STFpU3RNdWpJ\nVjcwUHVORVN6L1pKVC9TeWt4cTNoR2pPWVRBCkR4QVJ1bHFUa2Z5aWw1bG0vQWVy\ncjNGRDl5REZzbzdZWTFMRkU1eHFTc1kKLS0tIHNLVDFiRkRKTHhrSEx2S0J1R3cx\nSkdEQWJya1ptTTRqUjJZT3FFaFhkd0kKgecHrnzW9+Eb+b7c0z1yR+Y0Czsr9Kjh\nx1UUOfleHntJUCs8oYcOogKknEnhBJJoJ5oe/gWruRSwle1fs3cBqQ==\n-----END AGE ENCRYPTED FILE-----\n",
"recipient": "age1jlltcv5jcnm40z5k0q6hv053k2rqpqvemtuecdwn527uw8uqz4es3x7m68"
}
],
"lastmodified": "2026-07-27T21:22:58Z",
"mac": "ENC[AES256_GCM,data:VG2ygV4X6yMxGlJgq3sN4GAzgVqiD06noyMxB2yb/FCmiFVYZAH/9LqF//G7MTInfIjFrTDyoHAny+m/ztUrJkHXI1Fzg0U8R/zYyQ6wj/GguL0gyhA70uriQhvHsRHefHPa7Km61Blo9cME6CBJWejtRvh1IMw9itAioVrSIZE=,iv:zjxNG6IYxkcNqLk08NADoDiIbS6at86y136SBzadW1U=,tag:sj4ZRR7ixU8h7aOUJmTCYg==,type:str]",
"version": "3.13.2"
}
}
+26
View File
@@ -0,0 +1,26 @@
{
"data": "ENC[AES256_GCM,data:+/1AAVha+86abQIX8tebqPJ6BIcCCiTJgPe0OfFOwA2Mcx0NJdiPtQgyeo3G8fAolIQOFkk5reL+GUhlOz4iEbPOCWwFCckdH0tXiCiVcD35qQNMRurY8HmpM5FWFnyzBkKuyNZ8okPuo2+fA3NQh3gs94rpm2kBsfS18xvrlv9b8u1JvAxlH6GRMN1uUgFGmOXlpGAVjDUFiKUHN8tRSZpsxG0aKKPBZ82JdKgYfCiCQifS1366gIFrsjGriUC+P8wkadRnD1JE2ZAGSL7wJLaRmzA4LAMGXFGbitOsFxKxX3q8VWEXaaL+9h4rTe0WCrvmgDM9NUeVHpfRiJrLvQgrFZMUxuZF66vQVsLybjIWpYYJcjC6nfR0U3lMa9gjzOTMPxm8HdGKC53FhLM1HiPoe1a2Nno283fV0uaByMPmo257O6l1CXqb6KoZOGFccm7fKR/VnFAS1AInF+szI7/r+UcaTW6LwYeV3OwLE4a8b4OiqTLgCu6v,iv:kUlVXF4Yl5HGvoLwu9loiuuDVtz0kARRiUU3K+BPL2A=,tag:pVaGzQAX9Etoqc5XMGmNeg==,type:str]",
"sops": {
"age": [
{
"enc": "-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSB6OUVUdkoxM1M5THlYZ1ky\nYWJ5RjhtRGoxTldwRkI2T1p6dHR4Y3gwR1ZrCjRDS1hTM01lZ0FSalBYQ1B0bENs\nUytXNmlrUkV4TmJkMUM1QW1ZaExIcGMKLS0tIHdHKzdEcUxvM0ZYUEp0a2d6SHp5\nYXQydy9uNG15V2FhUUd4NEFTMTNNMEUKkIzKEoYzoVs+nhnpkHFgDkQqrWykatND\ntsNxcr1SXSKeEW1m/QpXZnn/aW3zSQR09PqOHf7PYU47/AdkwrUaAg==\n-----END AGE ENCRYPTED FILE-----\n",
"recipient": "age1njap586hc0q43kr03g6c8eqhdsmk8zcafkl3f83xwlc2gqhlmfgs4tmwad"
},
{
"enc": "-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSByZENrck42dWJjR2hYUVBl\ncERtRjR6dTdING5nWXlBc3o3eXFhaSs5NkFRCk43QXFUeTBJR0U5dG9YTDRmQ0Yx\naEo2blUvUXNZY3dpbEZRTUx1elNzdkkKLS0tIHJwNjRNM3V6WktWZ1BlUUtLRTNI\nMTVEbCtYbllIbTdxTGozWUluS3pIRXMKZCruDIkD/JofdAHWgPuaaKTDsz408ZkY\n77mhO8J+kd03qwt6qhFC5KF1lyjhwEnqrOE195+R/8Yl7hA/DsL2ZQ==\n-----END AGE ENCRYPTED FILE-----\n",
"recipient": "age1k7d2du5mejsmv5rzavm4xwgpthqvcfsehduquv28nzs53zppa3kqngfxq2"
},
{
"enc": "-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBxaC95RjYybzVlU2tYdHdU\naGJ4M2IvNkJwVlNIU0VCZEFUZk5VaksrVnpFCmJXS3luTG9WejR0Rk9DbHA3dngy\nRElFWjU1N2xuc3JaKzQ0L0U2Yktib1kKLS0tIFlrK3F0ZFh5Ync3ejVWMzRKTUx2\nZDhxdEMwa3B4TFZUcWdTdktDeGt1QUEKfgYnK2lW3cZuJGaw+bAKDipLuC4S5vK2\nxK2eJB5TP/xXrp0F3lx9sc2b1FOY9Vt9IQ7zVlBqJFkzJrcw8zYJJw==\n-----END AGE ENCRYPTED FILE-----\n",
"recipient": "age1zhfyuzlq40reuqlr34gf77852nhs3t6mqfzrqmas8z6sxk7tcfhsungrm0"
},
{
"enc": "-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBwOGFkSXNxTUIzLzR1Skk5\nMGZveHFDN3YwVmpwT3VTYkppWEU1aGVpdkhRCjZtRC92bVphd0ZFbzFxVzRUcGk4\nMUd1aWdEbFRaM01FT25JSVRoMTRsNUUKLS0tIDAybmcrYjFVWkFYaGY5TFZjcjFO\nYlZoQjJjN2lFNzd4RlJuSHFlRlNCOFEKgUIPnL2/OJgz9oMYt86/llHa7adTkhs6\n8yGYGV1wtU9aYtUMIR907SfYyZ6M4z8jH2wwzpQLbwQQMLgkejl1jg==\n-----END AGE ENCRYPTED FILE-----\n",
"recipient": "age1f7usptjx9rv4rxauasve200gxtdt9jkqhhdqstlf20wvlm7u75rsjfw50m"
}
],
"lastmodified": "2026-07-27T23:14:51Z",
"mac": "ENC[AES256_GCM,data:CYhzR0Y29GXvWUBbb13s16v9hDQ4k4Xmd4FUpcyNtHe0L/IrDGkx1WHzAohNGZjGMNB9W3BgiEH85OoOo8r1F82El4/8s2prEJi8AARvQU3+2cmRjjhNCCYQWmJkoF/cZNpw6nVyWzZdfUpPvHjbuf6utlI3rtR0qiCpSo/32S0=,iv:XLKTU2Ute4jMkfRAbXiVGxqP9+fjSs6HwRv1JfTTe7w=,tag:oIqS/1HWQZ7mLWaO7GwLFA==,type:str]",
"version": "3.13.2"
}
}
+24 -9
View File
@@ -4,9 +4,20 @@
homeDomain = "sweet.home"; # base LAN domain for service subdomains (pve., docker.)
tailnetDomain = "tail13f623.ts.net"; # Tailscale MagicDNS suffix
lanCidr = "192.168.2.0/24"; # LAN subnet
pxeServerIp = "192.168.2.223"; # pxe-boot host's LAN IP
pbsIp = "192.168.2.244"; # Proxmox Backup Server LAN IP
domainControllerIp = "192.168.2.253"; # FreeIPA domain controller / primary DNS
lanGateway = "192.168.2.254"; # LAN default gateway (router)
lanPrefixLength = 24; # LAN subnet prefix length (/24 = 255.255.255.0)
lxcLanInterface = "eth0"; # LAN NIC name in LXC containers (set by Proxmox --net0 name=eth0)
vmLanInterface = "ens18"; # LAN NIC name in Proxmox VMs (virtio, first NIC)
vmStorageInterface = "ens19"; # storage NIC name in HA server VMs (virtio, second NIC on vmbr1)
pxeServerIp = "192.168.2.223"; # pxe-boot LXC container LAN IP
nixCacheIp = "192.168.2.224"; # nix-cache LXC container LAN IP
tailscaleRouterIp = "192.168.2.222"; # tailscale-router LXC container LAN IP
torRelayIp = "192.168.2.221"; # tor-relay LXC container LAN IP
serverIp = "192.168.2.226"; # server (NFS/ZFS) Proxmox VM LAN IP
dockerIp = "192.168.2.225"; # docker Proxmox VM LAN IP
pbsIp = "192.168.2.244"; # Proxmox Backup Server LAN IP (not NixOS-managed)
domainControllerIp = "192.168.2.253"; # FreeIPA domain controller / primary DNS (not NixOS-managed)
ipaServer = "domain-controller.sweet.home"; # FreeIPA server hostname (used by security.ipa and Kerberos; must be a resolvable FQDN, not an IP)
# Cross-host references (LAN hostnames/users other hosts reach over the network)
nixCacheHost = "nix-cache"; # substituter/remote-builder hostname
@@ -70,16 +81,20 @@
primaryUser = "nixos";
# HA file server cluster
# haServer1Ip / haServer2Ip: static LAN IPs for both HA nodes (must be
# fixed — DRBD and corosync ring addresses are baked into the NixOS config).
# LAN IPs (vmbr0 / ens18) — client-facing: iSCSI initiators, NFS, management.
# Storage IPs (vmbr1 / ens19) — isolated internal bridge, used for DRBD
# replication and Corosync heartbeat only; never leaves pve1.
# haServerVip: floating virtual IP managed by Pacemaker's IPaddr2 resource;
# NFS and iSCSI clients connect here regardless of which node is Active.
# Set all three to real values in variables.nix before deploying.
haServer1Host = "ha-server-1";
haServer2Host = "ha-server-2";
haServer1Ip = "192.168.2.200"; # TODO: confirm production IP
haServer2Ip = "192.168.2.201"; # TODO: confirm production IP
haServerVip = "192.168.2.202"; # TODO: confirm floating VIP
haServer1Ip = "192.168.2.228"; # LAN IP, node 1
haServer2Ip = "192.168.2.227"; # LAN IP, node 2
haServerVip = "192.168.2.229"; # floating VIP (Pacemaker IPaddr2)
haServer1StorageIp = "192.168.4.228"; # storage-net IP, node 1 (vmbr1 / ens19)
haServer2StorageIp = "192.168.4.227"; # storage-net IP, node 2 (vmbr1 / ens19)
haStorageCidr = "192.168.4.0/29"; # storage subnet — internal to pve1 only
haStoragePrefixLength = 29; # storage subnet prefix length (/29)
haStorageRoot = "/srv/ha-data"; # XFS-over-DRBD mount point on the Active node
haIscsiIqn = "iqn.2026-01.home.sweet:ha-storage";