Compare commits

..
Author SHA1 Message Date
beatzaplentyandClaude Sonnet 4.6 e368f68ad7 tor-relay: wire beszel-agent with token secret and fix sops key
Check NixOS configurations / eval-hosts (pull_request) Successful in 11m2s
- Add hosts/tor-relay/host.nix import of host-token.nix so the agent
  gets its TOKEN from a sops-managed environment file
- Add secrets/tor-relay.yaml (encrypted beszel token for this host)
- Add creation_rules entry for secrets/tor-relay.yaml in .sops.yaml
- Update &lxc-tor-relay age key to the host's actual current key
  (old key was from a prior LXC incarnation; new key extracted from
  Switch-nix output: age1gl5ujmhd2pe37...)
- Re-encrypt secrets/common.yaml via sops updatekeys to swap in the
  new key, so the host can decrypt its password hash on next boot

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-24 09:36:03 +10:00
beatzaplenty fa52c2849a Merge pull request 'fix(lxc): prevent SSH host key deletion on every rebuild; add recovery script' (#50) from worktree-rippling-riding-snail into main
Check NixOS configurations / eval-hosts (push) Successful in 10m34s
Reviewed-on: #50
2026-07-23 23:20:30 +00:00
beatzaplentyandClaude Sonnet 4.6 dce3788499 fix(lxc): prevent SSH host key deletion on every rebuild; add recovery script
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m33s
NixOS's etc activation removes files that were in a previous generation's
environment.etc but absent from the current one -- even real copies, not
only symlinks.  LXC tarballs bake the host key into environment.etc (via
NIXOS_HOST_KEYS_DIR), but every subsequent nixos-rebuild switch lacks that
env var, so the key is removed as "obsolete".  sops-nix derives its age
decryption key from /etc/ssh/ssh_host_ed25519_key, so deletion cascades
into "Error getting data key: 0 successful groups required, got 0" for
every sops secret on the host.

Fix: two activation scripts bracket the etc step.
  preserveSshHostKey (no deps, runs before etc): copies the live key to
    /run (tmpfs) before etc can delete it.
  restoreSshHostKey (deps=[etc], runs after etc): reinstalls via `install`
    if etc removed the key.  The resulting file is not tracked in either
    generation's environment.etc, so subsequent rebuilds leave it alone.

scripts/recover-hosts.sh: restore both private and public key files (not
just the private key), use install(1) for atomic mode setting, and add a
post-rebuild sops-nix verification step to confirm success.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014zT1L6hmsq6i1evAEH7dmi
2026-07-24 09:17:58 +10:00
beatzaplenty e00be5d2da added sops
Check NixOS configurations / eval-hosts (push) Failing after 17m21s
2026-07-24 08:55:21 +10:00
beatzaplenty 82eea7f088 synced all keys
Check NixOS configurations / eval-hosts (push) Successful in 10m21s
2026-07-24 08:17:34 +10:00
beatzaplenty 955a443b36 updated github token
Check NixOS configurations / eval-hosts (push) Failing after 15m35s
2026-07-24 07:17:12 +10:00
beatzaplenty 4800aebf43 enable beszel agent
Check NixOS configurations / eval-hosts (push) Successful in 10m32s
2026-07-24 07:03:01 +10:00
beatzaplenty cb737642e5 updated sops keys
Check NixOS configurations / eval-hosts (push) Successful in 10m24s
2026-07-24 06:50:18 +10:00
beatzaplenty 6d5670c8d2 Merge pull request 'server: auto-create tank ZFS pool on first boot if data disk is blank' (#49) from server-boot-fix into main
Check NixOS configurations / eval-hosts (push) Successful in 11m14s
Reviewed-on: #49
2026-07-23 20:26:36 +00:00
beatzaplenty c911a605e9 updated sops
Check NixOS configurations / eval-hosts (push) Successful in 10m22s
2026-07-24 06:23:52 +10:00
beatzaplentyandClaude Sonnet 4.6 6002c5c738 server: auto-create tank ZFS pool on first boot if data disk is blank
Check NixOS configurations / eval-hosts (pull_request) Failing after 11m40s
A fresh proxmox-server deploy has a blank scsi1 disk, so
zfs-import-tank.service spun 60 s then failed with no pool found.
Add zfs-init-tank.service that runs before the import: exits immediately
if the pool already exists, imports it if it exists but isn't imported
yet, or creates it on /dev/disk/by-id/scsi-*drive-scsi1 (Proxmox's
virtio-scsi naming for the second disk) with all required NFS datasets
if the disk is blank.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-24 06:13:58 +10:00
beatzaplenty 92c50df2f1 Merge branch 'main' of https://gitea.lan.ddnsgeek.com/beatzaplenty/nixos
Check NixOS configurations / eval-hosts (push) Failing after 12m12s
2026-07-24 05:39:18 +10:00
beatzaplenty 45e61844d5 added fish 2026-07-24 05:38:30 +10:00
beatzaplenty e72df8fed5 Merge pull request 'feat: add nixos@nixos workstation SSH key to all hosts' (#48) from worktree-zesty-wishing-knuth into main
Check NixOS configurations / eval-hosts (push) Successful in 10m51s
Reviewed-on: #48
2026-07-23 02:27:52 +00:00
beatzaplentyandClaude Sonnet 4.6 5497a5b0ae feat: add nixos@nixos workstation SSH key to all hosts
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m44s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Rcq7iXrN9eYUTUEWrbWqyX
2026-07-23 11:05:51 +10:00
beatzaplenty e10e493ddd Merge pull request 'Worktree harmonic jingling bee' (#47) from worktree-harmonic-jingling-bee into main
Check NixOS configurations / eval-hosts (push) Successful in 10m20s
Reviewed-on: #47
2026-07-23 00:40:36 +00:00
beatzaplentyandClaude Sonnet 4.6 ae9acecbf3 fix: sudo the tarball/image staging into /var/lib/vz
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m20s
The post-build cp/mv into /var/lib/vz/template/cache (LXC) and
/var/lib/vz/import (VM) are Proxmox-owned root directories -- they need
sudo_pfx just like pct/qm/pvesh do. nix build writes to the nix store
as the SSH user, but staging into /var/lib/vz/ requires root.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-23 10:39:09 +10:00
beatzaplentyandClaude Sonnet 4.6 a3be05538b fix: support single-user (non-root) nix in configure-nix-cache-client.sh
The script was root-only and hard-coded /etc/nix/nix.conf and
/etc/ssh/ssh_known_hosts, making it always fail (non-fatally) when
called as a non-root SSH user from create-proxmox-resource.sh.

Add dual-mode detection based on EUID:
- root (multi-user/daemon): existing behavior unchanged -- writes
  /etc/nix/nix.conf, /etc/ssh/ssh_known_hosts, restarts nix-daemon
- non-root (single-user): writes ~/.config/nix/nix.conf and
  ~/.ssh/known_hosts, creates the config file if missing, skips the
  daemon restart (single-user has no daemon), defaults REMOTE_BUILDER_KEY
  to ~/.ssh/id_ed25519 instead of /root/.ssh/id_ed25519

create-proxmox-resource.sh already calls the script without sudo (as the
SSH user), so no change is needed there -- the script now handles both
cases on its own.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-23 10:37:57 +10:00
beatzaplenty 289163c712 Merge pull request 'Worktree harmonic jingling bee' (#46) from worktree-harmonic-jingling-bee into main
Check NixOS configurations / eval-hosts (push) Successful in 10m22s
Reviewed-on: #46
2026-07-23 00:29:56 +00:00
beatzaplentyandClaude Sonnet 4.6 2123e4ad69 fix: reinstall nix as SSH user, not root, on Proxmox nodes
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m27s
nix was installed as root on pve1 (the codex-setup.sh root path, meant
for container/Codex environments), making nix build require sudo there.
After cleaning up the root install and reinstalling as the SSH user
(wayne), nix is owned by that user and runs directly without sudo.

create-proxmox-resource.sh: drop sudo_pfx from nix build in both
remote scripts. The SSH user owns the store after reinstall; nix build
goes through the nix daemon-or-store directly. sudo stays on pct/qm/pvesh
(cluster IPC) and the disko image-writer script (writes to disk).

codex-setup.sh: add build-users-group = (empty) to the user nix.conf
written by the non-root install path. Guards against a stale
/etc/nix/nix.conf from a prior root install (which sets
build-users-group = nixbld) silently breaking single-user builds.

Manual cleanup required once on each Proxmox node that had root's nix:
  sudo rm -rf /nix /etc/nix
  sudo rm -f /etc/profile.d/nix.sh /etc/profile.d/nix-daemon.sh
  for i in $(seq 1 10); do sudo userdel nixbld$i 2>/dev/null||true; done
  sudo groupdel nixbld 2>/dev/null || true
After that, the next create-proxmox-resource.sh run auto-reinstalls
nix as the SSH user via codex-setup.sh.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-23 10:25:43 +10:00
beatzaplentyandClaude Sonnet 4.6 177950dd3d revert: restore sudo for nix build in remote scripts
nix on pve1 was installed as root (single-user), so wayne can't access
/nix/var/nix/db/big-lock without root -- nix build genuinely needs sudo
there. The previous fix to drop sudo_pfx was wrong.

The real fix is node config: add nix to wayne's NOPASSWD rules in
sudoers on pve1 (see below). pct/qm/pvesh already have NOPASSWD and
work fine in non-interactive SSH heredocs; nix was just missing from
that list.

On pve1 as root:
  echo 'wayne ALL=(root) NOPASSWD: ALL' | tee /etc/sudoers.d/wayne-nopasswd
  chmod 440 /etc/sudoers.d/wayne-nopasswd

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-23 10:18:56 +10:00
beatzaplenty cbf1239be4 Merge pull request 'fix: don't sudo nix build in remote scripts' (#45) from worktree-harmonic-jingling-bee into main
Check NixOS configurations / eval-hosts (push) Successful in 10m33s
Reviewed-on: #45
2026-07-23 00:13:51 +00:00
beatzaplentyandClaude Sonnet 4.6 8a282ee32e fix: don't sudo nix build in remote scripts
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m33s
nix build runs through the nix daemon and doesn't need root; the
tooling-check step already confirms the SSH user can run nix directly
(ensure_nix_profile + command -v nix). sudo without a TTY blocks
non-interactive SSH heredoc sessions with "a terminal is required".

Keep sudo on pct/qm/pvesh (cluster IPC) and the disko image-writer
script (writes to block devices) -- those actually require root.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-23 10:10:29 +10:00
beatzaplentyandClaude Sonnet 4.6 25079a7f0a fix: use sudo for nix build on non-root SSH user
Check NixOS configurations / eval-hosts (push) Successful in 10m18s
Single-user Nix installations are owned by root. When PROXMOX_SSH_USER
is not root, prefix the remote nix build command with sudo_prefix, same
as the Proxmox tool invocations. Passes sudo_prefix as an extra arg to
both the LXC tarball and VM disko image build heredocs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-23 09:49:14 +10:00
beatzaplentyandClaude Sonnet 4.6 4952e5224d fix: default PROXMOX_REMOTE_REPO_DIR to SSH user's home dir
Check NixOS configurations / eval-hosts (push) Successful in 10m20s
/root/nixos was only correct when PROXMOX_SSH_USER=root. Now that it
defaults to wayne, use /home/${PROXMOX_SSH_USER}/nixos so git clone
goes somewhere the SSH user can actually write to.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-23 09:44:14 +10:00
beatzaplenty 98409f4502 Merge pull request 'fix: prefix Proxmox commands with sudo for non-root SSH user' (#44) from worktree-reactive-gliding-map into main
Check NixOS configurations / eval-hosts (push) Successful in 10m20s
Reviewed-on: #44
2026-07-22 22:59:35 +00:00
beatzaplentyandClaude Sonnet 4.6 7779f3e137 fix: prefix Proxmox commands with sudo for non-root SSH user
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m18s
PROXMOX_SSH_USER was changed from root to wayne, but all remote pvesh/qm/pct
invocations assumed root. When run as a non-root user these commands fail with
ipcc_send_rec errors because they can't reach the pve-cluster IPC socket.

Adds a global sudo_prefix (empty when PROXMOX_SSH_USER=root, "sudo" otherwise)
and applies it to every remote Proxmox command in the script, including the
duplicate-host heredoc check, pvesh nextid, vmid existence checks, resource
destruction, and all create/start commands. Removes the now-redundant local
sudo_prefix definition that was previously only in the VM image build branch.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-23 08:47:00 +10:00
beatzaplenty 1462829aa6 fix syntax
Check NixOS configurations / eval-hosts (push) Failing after 11m48s
2026-07-23 08:38:36 +10:00
beatzaplenty 48ce2c4097 added sops key path and updated pve1 SSH user
Check NixOS configurations / eval-hosts (push) Failing after 9m45s
2026-07-23 08:37:44 +10:00
beatzaplenty bcae177d8e added jq
Check NixOS configurations / eval-hosts (push) Failing after 9m45s
2026-07-23 06:59:37 +10:00
beatzaplenty d35aca3138 added jq and direnv 2026-07-23 06:59:00 +10:00
beatzaplenty 147cb3803a Update hosts/nixos/home.nix
Check NixOS configurations / eval-hosts (push) Failing after 11m59s
2026-07-22 19:57:14 +00:00
beatzaplenty 98445565d6 Update hosts/nixos/home.nix
Check NixOS configurations / eval-hosts (push) Failing after 11m10s
2026-07-22 19:56:07 +00:00
beatzaplenty eef4b05254 Merge pull request 'Unmount everything under /mnt before zpool export, not just chroot dirs' (#43) from worktree-baremetal-esp-unmount into main
Check NixOS configurations / eval-hosts (push) Successful in 10m20s
Reviewed-on: #43
2026-07-22 04:30:05 +00:00
beatzaplenty 619324589a Unmount everything under /mnt before zpool export, not just the chroot dirs
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m28s
The previous fix (#42) only unmounted /mnt/{dev,proc,sys,run}, but disko
also mounts the ESP at /mnt/boot (modules/disko/baremetal.nix) -- another
nested mount blocking ZFS from unmounting its own root dataset at /mnt
the same way. Confirmed live: zpool export still failed with "cannot
unmount '/mnt': pool or dataset busy" after the chroot-only fix.

Replace the manual dev/proc/sys/run list with a single recursive
`umount -R /mnt`, which clears everything nested under /mnt -- current
and future mountpoints alike -- rather than needing to keep enumerating
whatever nixos-install/disko happen to leave mounted.
2026-07-22 04:23:42 +00:00
beatzaplenty 400af07154 Merge pull request 'Unmount leftover chroot bind mounts before zpool export' (#42) from worktree-baremetal-emergency-access into main
Check NixOS configurations / eval-hosts (push) Successful in 10m26s
Reviewed-on: #42
2026-07-22 04:10:08 +00:00
beatzaplenty 9bb626327f Unmount nixos-install's leftover chroot bind mounts before zpool export
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m20s
nixos-install bind-mounts /dev, /proc, /sys (and usually /run) into
/mnt to run the target's activation script in a chroot, and doesn't
unmount them again afterward. Left in place, those nested mounts made
ZFS refuse to unmount its own root dataset at /mnt: zpool export
failed with "cannot unmount '/mnt': pool or dataset busy", and because
of this script's set -e, that killed the script before it ever reached
reboot -- silently defeating the export-before-reboot fix from #40 on
every real run, which is why the ZFS-import stall kept recurring.
2026-07-22 04:07:32 +00:00
beatzaplenty 1f8bf8c852 Merge pull request 'Allow initrd emergency shell access on baremetal-gui' (#41) from worktree-baremetal-emergency-access into main
Check NixOS configurations / eval-hosts (push) Successful in 10m35s
Reviewed-on: #41
2026-07-22 03:48:48 +00:00
beatzaplenty 5d7a6327b7 Allow initrd emergency shell access on baremetal-gui
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m45s
The systemd-based initrd (default here, since this host has a ZFS
root) locks the root account by default, so sulogin refuses a shell
if something in the initrd fails and it drops to emergency mode --
confirmed live: it just loops re-entering the target instead of
prompting, making an initrd-level ZFS import failure impossible to
diagnose from the console. Only affects the pre-switch-root initrd
shell, not the installed system's own login.
2026-07-22 03:47:54 +00:00
beatzaplenty 0b9f124713 Merge pull request 'Export ZFS root pool before rebooting from the auto-installer' (#40) from worktree-fix-zfs-install-export into main
Check NixOS configurations / eval-hosts (push) Successful in 10m25s
Reviewed-on: #40
2026-07-22 03:22:32 +00:00
beatzaplentyandClaude Sonnet 5 9479d56e11 Export ZFS root pool before rebooting from the auto-installer
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m19s
disko's --mode ...,mount leaves the pool imported (needed for
nixos-install to write into /mnt), and the script rebooted straight
into the newly-installed system without exporting it. That pool is
still stamped with the live installer's own hostid, which never
matches the target host's declared networking.hostId, and since
boot.zfs.forceImportRoot is false (the recommended setting, not a bug),
the first real boot refuses to force-import an unexported pool from a
different hostid -- which is exactly the ZFS-import stall baremetal-gui
was hitting after install. Exporting all pools right before reboot (a
no-op for non-ZFS hosts) clears the in-use state so import succeeds
regardless of hostid.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-22 03:10:25 +00:00
beatzaplenty c53c1940d6 Merge pull request 'Prompt for a host-key path interactively as a third fallback' (#39) from worktree-gui-wifi-module into main
Check NixOS configurations / eval-hosts (push) Successful in 10m18s
Reviewed-on: #39
2026-07-22 02:57:36 +00:00
beatzaplenty 79e8f9f2ce Prompt for a host-key path interactively as a third fallback
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m19s
If neither /etc/host-keys nor /root/host-keys has the target's SSH
host key, auto-install.sh previously went straight to "continue
without one anyway?". Added a third option in between, gated on
[[ -t 0 ]] (only offered when there's an actual operator at stdin, never
in an unattended/non-interactive run): prompt for an arbitrary
directory (USB stick, other mount, etc.), and if the key pair is
there, copy it into /root/host-keys and install it to /mnt same as the
existing pre-seeded-key path. Falls through to the original
warning+confirm if the prompt is skipped, the path doesn't have the
key, or the run isn't interactive at all.

docs/auto-installer.md updated to mention the new fallback. Quick
bash -n + shellcheck pass only, per request.
2026-07-22 02:54:06 +00:00
beatzaplenty 5fe575d362 Merge pull request 'Wrap auto-install.sh in a nix-shell shebang for its required tools' (#38) from worktree-gui-wifi-module into main
Check NixOS configurations / eval-hosts (push) Successful in 10m19s
Reviewed-on: #38
2026-07-22 02:45:34 +00:00
beatzaplenty b46424343f Wrap auto-install.sh in a nix-shell shebang for its required tools
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m20s
Running the script standalone (its whole point per the last commit)
failed with "disko: command not found" -- jq/disko/nixos-install are
only guaranteed present via the built installer image's
environment.systemPackages, not on a plain checkout.

Added a #!/usr/bin/env nix-shell / #!nix-shell -i bash -p jq disko
nixos-install-tools shebang instead of a per-tool fallback: disko's own
generated scripts already hardcode absolute Nix store paths for
everything they shell out to internally (parted/sgdisk/mkfs.*/zfs/...
confirmed by inspecting a generated system.build.formatScript earlier),
so these three are the only genuinely external dependencies the script
itself has. This is a fast no-op on the built installer image (already
has all three) and what makes it also work standalone.

Quick syntax + shellcheck pass only this round (bash -n, shellcheck
with a `shellcheck shell=bash` directive since it doesn't recognize
nix-shell shebangs natively) -- skipping the full codex-maintenance.sh
sweep per request, to get this out for a real hardware test.
2026-07-22 02:44:49 +00:00
beatzaplenty 33b1d5ec79 Merge pull request 'Fix auto-install.sh to work standalone, not just baked into the image' (#37) from worktree-gui-wifi-module into main
Check NixOS configurations / eval-hosts (push) Successful in 11m10s
Reviewed-on: #37
2026-07-22 02:40:03 +00:00
beatzaplenty f565e9c2a1 Fix auto-install.sh to work standalone, not just baked into the image
Check NixOS configurations / eval-hosts (pull_request) Successful in 11m6s
Two real bugs, both hit live:

1. Shebang: #!/run/current-system/sw/bin/bash only resolves on an
   already-activated NixOS system -- running the checked-out script
   directly (e.g. from a stock ISO, cloned repo) failed with "cannot
   execute: required file not found" on a non-NixOS box. Switched to
   #!/usr/bin/env bash, which resolves identically on NixOS
   (environment.usrbinenv's own default) and any normal Linux distro.
   Also fixed the file's missing executable bit.

2. FLAKE_BASE_URL: previously depended on pkgs.replaceVars substituting
   a Nix-templated @lanDomain@ placeholder at build time -- meaning it
   only ever worked when baked into the built installer image, not when
   run straight from a checkout (the literal, unexpanded "@lanDomain@"
   string reached git as a bogus hostname). Replaced with LAN_DOMAIN in
   scripts/env.sh (manually kept in sync with variables.nix's lanDomain,
   same pattern as NIX_CACHE_HOST/nixCacheHost already), sourced by the
   script itself like every other script in scripts/. Dropped
   pkgs.replaceVars from modules/installer/common.nix entirely --
   scripts/env.sh is now baked into the image alongside auto-install.sh
   at a matching relative path (/etc/nixos-installer/env.sh next to
   /etc/nixos-installer/installer/auto-install.sh) so the script's own
   relative `source` line resolves the same way in both contexts.

loginShellInit's invocation path and docs/auto-installer.md updated to
match. Verified: shellcheck clean on both scripts, the baked files are
byte-identical to their checked-in sources (no templating left to
verify), and codex-maintenance.sh (secret grep, fmt, statix, full eval
of every host/package including the installer/pxe artifacts) passes
clean.
2026-07-22 02:38:11 +00:00
beatzaplenty a91634c460 updated permissions on auto-install.sh
Check NixOS configurations / eval-hosts (push) Successful in 10m20s
2026-07-22 02:19:05 +00:00
beatzaplenty 42919ea15c Merge pull request 'Worktree gui wifi module' (#36) from worktree-gui-wifi-module into main
Check NixOS configurations / eval-hosts (push) Successful in 10m31s
Reviewed-on: #36
2026-07-22 02:16:42 +00:00
beatzaplenty 60c155327d Restore guiRootDisk1/guiRootDisk2, lost in a merge conflict on main
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m33s
These were dropped from variables.nix by a stash/merge conflict
resolution on main (commit fb6ee27) that kept the new wifiSsid value
but discarded the two disk-path variables entirely, leaving unresolved
`<<<<<<< Updated upstream` markers in an intermediate commit before
being cleaned up. modules/disko/baremetal.nix references both directly
with no fallback, so baremetal-gui has been failing to evaluate on main
since that commit ("attribute 'guiRootDisk1' missing") -- confirmed by
cloning main fresh and evaluating config.disko.devices.disk.disk1.device
directly.

This commit is rebased onto latest main (through "updated secrets",
which registered baremetal-gui's real sops recipient) rather than the
older base this branch started from.
2026-07-22 02:15:37 +00:00
beatzaplenty 0f78e96b81 Merge pull request 'Run per-host/per-package nix eval and dry-run build concurrently' (#35) from worktree-parallel-host-eval into main
Check NixOS configurations / eval-hosts (push) Successful in 10m21s
Reviewed-on: #35
2026-07-22 02:05:59 +00:00
beatzaplenty 12a2354fad Move auto-install.sh out of Nix config into a real script file
Moves the auto-installer's shell script from an inline Nix string in
modules/installer/common.nix to scripts/installer/auto-install.sh, a
real, version-controlled, directly-editable/shellcheck-able file.
common.nix now wires it in with pkgs.replaceVars, substituting the one
value that actually needs to come from variables.nix (lanDomain) --
every other `${...}` in the script is a literal bash reference, left
untouched. replaceVars fails the build if any @name@-shaped placeholder
is left unsubstituted, so a typo'd or renamed variable is caught at
eval time rather than silently shipping broken.

Verified: built the substituted derivation and diffed it against the
source template -- identical except for the one substituted line, no
leftover unsubstituted placeholders. Full codex-maintenance.sh (secret
grep, fmt, statix, full eval of every host/package including the
installer/pxe artifacts that consume this) passes clean.
2026-07-22 02:05:31 +00:00
beatzaplentyandClaude Sonnet 5 f237a6a3d2 Run per-host/per-package nix eval and dry-run build concurrently
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m20s
codex-maintenance.sh evaluated each affected host/package one at a time,
even though those calls are independent. Added scripts/lib/nix-parallel.sh
(run_nix_parallel) and wired it into the host-eval, package-eval, and
dry-run-build loops.

Concurrency defaults to core count capped by available memory (~1GB/job)
rather than plain nproc: empirically, nproc-many concurrent full-flake
evals OOM-killed each other on a 4GB/6-core box, while 3-4 ran clean and
were still ~2x faster than serial. Override via NIX_PARALLEL_JOBS.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-22 02:04:21 +00:00
beatzaplenty a2ce01d6ce updated secrets
Check NixOS configurations / eval-hosts (push) Successful in 10m18s
2026-07-22 01:36:32 +00:00
beatzaplenty fb6ee27e10 updated variables# Please enter the commit message for your changes. Lines starting
Check NixOS configurations / eval-hosts (push) Successful in 10m31s
2026-07-22 01:35:47 +00:00
beatzaplenty 85ff5e01e8 updated wifi SSID
Check NixOS configurations / eval-hosts (push) Failing after 9m40s
2026-07-22 01:35:02 +00:00
beatzaplenty eb881d4cd8 Merge pull request 'Worktree gui wifi module' (#34) from worktree-gui-wifi-module into main
Check NixOS configurations / eval-hosts (push) Successful in 10m38s
Reviewed-on: #34
2026-07-22 01:33:16 +00:00
beatzaplenty 96cc63671a Add baremetal-gui flake target with ZFS RAID0, AMD GPU, and sops-backed wifi
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m40s
Wires everything staged so far into a real flake target:

- modules/platforms/baremetal.nix (new): the bare-metal platform module,
  composed from a real nixos-generate-config run on the actual gui-host
  hardware (AMD CPU, ahci/xhci/usb storage -- modules/hardware-configuration/baremetal.nix).
  Enables hardware.enableRedistributableFirmware (real wifi/GPU/microcode
  firmware VMs never needed), amdgpu as the Xorg video driver plus
  hardware.graphics for Mesa OpenGL/Vulkan, and imports the ZFS RAID0 disko
  layout + modules/services/zfs/enable-service.nix for root-on-ZFS boot
  support.
- flake.nix: new baremetal-gui target, reusing hosts/nixos/host.nix (same
  identity already shared across linode/proxmox/lxc-gui).
- hosts/nixos/host.nix: added networking.hostId, required now that a ZFS
  root pool is in the picture.
- variables.nix: guiRootDisk1/guiRootDisk2 filled in (/dev/sda, /dev/sdb --
  only used transiently at disko-format time, same as modules/disko/proxmox.nix's
  own plain device path). wifiPassword removed.
- modules/networking/wifi.nix: reworked to pull the wifi password from a
  new sops secret (secrets/gui.yaml, wifi-password) instead of a plaintext
  variable -- NetworkManager's ensureProfiles renders `psk = "$WIFI_PASSWORD"`
  literally (nixpkgs' own documented pattern for this) and envsubst-expands
  it from a sops-rendered EnvironmentFile at activation, so the real value
  never touches the Nix store, only /run.
- .sops.yaml: new secrets/gui\.yaml rule, admin + the currently-registered
  lxc-gui recipient (the only gui variant with a provisioned host key so
  far -- whichever variant is actually deployed next still needs
  scripts/secrets/sync-host-keys.sh run for its own recipient).
- README.md/CLAUDE.md: documented the new platform/target and its module
  layout, per this repo's own drift-prevention note.

Verified end-to-end: nix eval of every existing target (nothing broke),
a temporary real nixosSystem build against the actual disko.nixosModules.disko
confirming the generated zpool create has no mirror/raidz keyword (genuine
stripe), and a temporary test SSID confirming the sops secret/template/
ensureProfiles chain renders correctly before reverting to blank/real values.
Full scripts/codex-maintenance.sh (secret-grep, fmt, statix, full-fallback
eval of every host/package) passes clean.
2026-07-22 01:25:29 +00:00
beatzaplenty 104804dbf6 Stage a ZFS RAID0 disko layout for the bare-metal gui host
Adds modules/disko/baremetal.nix: two disks, each its own top-level
zpool vdev with no mirror/raidz between them (disko's zpool `mode`
defaults to "" for a plain stripe), ESP + systemd-boot on disk1. Device
paths are placeholders in variables.nix (guiRootDisk1/guiRootDisk2)
until the real hardware profile arrives.

Verified structurally by building a throwaway nixosSystem with the
actual disko.nixosModules.disko and reading the generated
system.build.formatScript: it emits `zpool create rpool ... disk1
disk2` with no mirror/raidz keyword, confirming a genuine stripe.

Not yet wired into any flake target -- that happens once the hardware
config lands and a new bare-metal platform module is added, per the
agreed sequencing.
2026-07-21 23:54:00 +00:00
beatzaplenty 0a2298b0e2 update flake.lock
Check NixOS configurations / eval-hosts (push) Successful in 10m21s
2026-07-21 23:48:06 +00:00
beatzaplenty e73ae6044e Merge pull request 'Prestage a declarative wifi connection on the gui host' (#33) from worktree-gui-wifi-module into main
Check NixOS configurations / eval-hosts (push) Failing after 18m53s
Reviewed-on: #33
2026-07-21 23:45:30 +00:00
beatzaplenty 14621e7ad5 Prestage a declarative wifi connection on the gui host
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m32s
Adds modules/networking/wifi.nix using NetworkManager's ensureProfiles
mechanism so the gui host associates to a known SSID on first boot with
no manual nmtui step. Credentials are placeholders in variables.nix
(wifiSsid/wifiPassword, both empty) to be filled in once the bare-metal
hardware profile is wired up — the module is a no-op until then.
2026-07-21 23:42:22 +00:00
beatzaplenty cb141f0a41 Merge pull request 'Rename PXE installer menu entry, add vanilla NixOS minimal netboot entry' (#31) from worktree-pxe-menu-rename-and-minimal into main
Check NixOS configurations / eval-hosts (push) Successful in 10m30s
Reviewed-on: #31
2026-07-21 22:48:49 +00:00
beatzaplenty e92aab617f Rename PXE installer menu entry, add vanilla NixOS minimal netboot entry
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m42s
The iPXE menu's "nixos" entry actually chain-loads this flake's own
custom auto-installer image, not a stock NixOS image — rename it to
"auto-installer" (label "NixOS Auto-Installer") so the menu says what it
boots, and set networking.hostName on netbootSystem to match, so the
generated system name (nixos-system-auto-installer-*) and staged
directory (/srv/pxe/http/auto-installer) agree with the menu entry too.

Add a second, genuinely vanilla NixOS minimal netboot image
(netbootMinimalSystem in flake.nix — nixpkgs' netboot-minimal.nix on its
own, none of modules/installer/common.nix's auto-installer wiring),
built from source the same way as the auto-installer image and exposed
as packages.x86_64-linux.pxe-minimal. Staged and menu-wired the same
way, as "nixos-minimal" (item, hostname, and directory all matching).

modules/pxe-boot/stage-installer-artifacts.nix is generalized to stage
both images via a shared rule-builder instead of one hardcoded set of
paths.

Verified: nix eval confirms both images' config.system.name matches
their menu entry/directory names, the pxe-boot host itself builds
clean with the new menu.ipxe, and the new pxe-minimal image was booted
directly under QEMU (kernel+initrd, no KVM) to a working login shell
with hostname nixos-minimal, no hang.
2026-07-21 22:45:25 +00:00
beatzaplenty b4474cf1e1 Merge pull request 'Fix PXE netboot installer hanging at boot (ISO/netboot module conflict)' (#30) from worktree-fix-pxe-netboot-hang into main
Check NixOS configurations / eval-hosts (push) Successful in 10m33s
Reviewed-on: #30
2026-07-21 22:20:03 +00:00
beatzaplenty 453c7b5513 Fix PXE netboot installer hanging at boot (ISO/netboot module conflict)
Check NixOS configurations / eval-hosts (pull_request) Successful in 11m1s
The netboot build composed ./modules/installer/iso.nix (which pulls in
nixpkgs' installation-cd-minimal.nix) together with nixpkgs'
netboot-minimal.nix. Both installation-cd-base.nix and netboot.nix set
fileSystems."/" via the identical lib.mkImageMediaOverride (mkOverride
60) priority - genuinely conflicting root-filesystem strategies
(ISO-by-label vs. netboot-tmpfs) at the same priority, and the ISO one
was winning. Every netboot boot hung waiting for a device that can
never exist outside a real CD/USB:

  A start job is running for /dev/disk/by-label/nixos-minimal-...

Reproduced live: deployed a scratch lxc-pxe-boot on pve-test, pulled its
built kernel/initrd, and booted them directly with QEMU to confirm the
hang and capture full console output. netboot-minimal.nix's own chain
(netboot-base.nix) already imports profiles/installation-device.nix
independently, so common.nix's initialHashedPassword override still
applies correctly with iso.nix removed from this composition. Rebuilt
and re-booted the same way after the fix - full boot to a working shell
with SSH up, no hang.
2026-07-21 22:15:44 +00:00
beatzaplenty b3463e4b33 Merge pull request 'Add ad hoc pve1 -> pve-test clone script (vzdump + qmrestore/pct restore)' (#29) from worktree-clone-pve1-to-pve-test into main
Check NixOS configurations / eval-hosts (push) Successful in 10m21s
Reviewed-on: #29
2026-07-21 21:55:25 +00:00
beatzaplentyandClaude Sonnet 5 013b2c7009 Add ad hoc pve1 -> pve-test clone script (vzdump + qmrestore/pct restore)
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m21s
Backs up a VM/CT on pve1 (snapshot mode by default, so the source stays
online), relays the archive to pve-test, restores it there with fresh
MAC addresses (--unique), and deletes both the source and relayed
backup copies afterward -- no ad hoc backup files left behind on
either node.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-21 20:22:56 +00:00
beatzaplenty 12f9153957 claude audit report
Check NixOS configurations / eval-hosts (push) Successful in 10m36s
2026-07-21 20:07:30 +00:00
beatzaplenty 1004538f00 updated sops secrets 2026-07-21 20:07:14 +00:00
beatzaplentyandClaude Sonnet 5 87873300e1 Add pve-test.sweet.home as a second Proxmox target
pve1.sweet.home is production; scripts/env.sh now also defines
PVE_TEST_HOST for a separate sandbox node, individually targetable via
--node/PROXMOX_HOST. Tooling defaults are unchanged (still pve1) -- the
new restriction (Claude defaults to pve-test unless explicitly told to
use pve1) is documented as policy in CLAUDE.md, not enforced in the
scripts.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-21 19:57:15 +00:00
beatzaplenty e9e2312163 Merge pull request 'Make lxc-docker a privileged container: unprivileged can't NFS-mount at all' (#28) from worktree-fix-container-dns-search-domain into main
Check NixOS configurations / eval-hosts (push) Successful in 10m24s
Reviewed-on: #28
2026-07-21 08:50:14 +00:00
beatzaplentyandClaude Sonnet 5 6b09a808ed Make lxc-docker a privileged container: unprivileged can't NFS-mount at all
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m33s
The kernel's NFS client filesystem doesn't set FS_USERNS_MOUNT, so mounting
NFS from inside any non-init user namespace -- exactly what an unprivileged
LXC container's UID-mapped root runs in -- is rejected at the VFS layer
with EPERM, regardless of Proxmox's mount=nfs;nfs4 container feature (which
only patches the AppArmor layer). Confirmed live on the redeployed lxc-docker
container: TCP to the NFS server's port 2049 succeeds, the server's export
table matches the container's IP, and mount.nfs: Operation not permitted
still fires immediately with no corresponding denial anywhere in the
server's own logs -- a kernel-level rejection that no amount of DNS/
automount/export tweaking (this branch's earlier commits) could ever fix.

modules/platforms/lxc.nix now keys proxmoxLXC.privileged off hostName
("docker" -> true) rather than a blanket false, since build-types/docker.nix
is also composed for linode-docker/proxmox-docker, which don't import
proxmox-lxc.nix at all -- setting this option there would break their eval.
create-proxmox-resource.sh reads the value back via a new
flake_target_lxc_privileged helper instead of hardcoding --unprivileged 1,
so the two stay in sync automatically for every lxc-* target.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T48qgH3VTvs8wvwj44FEbE
2026-07-21 05:57:06 +00:00
beatzaplenty 9496efdd22 Merge pull request 'Add pve.sweet.home guard rails to CLAUDE.md' (#26) from worktree-claude-md-pve-guardrails into main
Check NixOS configurations / eval-hosts (push) Successful in 10m13s
Reviewed-on: #26
2026-07-21 04:28:14 +00:00
beatzaplenty 33c9506c7d Merge pull request 'Fix NFS mount device strings on lxc-docker: use FQDN, not search domain' (#25) from worktree-fix-container-dns-search-domain into main
Check NixOS configurations / eval-hosts (push) Failing after 7m3s
Reviewed-on: #25
2026-07-21 04:26:32 +00:00
beatzaplentyandClaude Sonnet 5 abe3763cb3 Add pve.sweet.home guard rails to CLAUDE.md
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m24s
Codifies read-only access to existing Proxmox config/VMs/containers,
allows scratch test VMs/containers as long as they're torn down again,
and forbids any change to production on the node.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-21 02:01:15 +00:00
beatzaplenty 744904b19f Merge pull request 'Enable QEMU guest agent on Proxmox VMs; register lxc-gui sops key' (#24) from worktree-flake-e2e-audit into main
Check NixOS configurations / eval-hosts (push) Successful in 16m3s
Reviewed-on: #24
2026-07-21 01:46:06 +00:00
beatzaplentyandClaude Sonnet 5 9cbaf1a070 Enable QEMU guest agent on Proxmox VMs; register lxc-gui sops key
Check NixOS configurations / eval-hosts (pull_request) Successful in 16m19s
create-proxmox-resource.sh's `qm create` never passed --agent, so despite
services.qemuGuest.enable = true being set on every host, Proxmox never
created the virtio-serial channel the guest agent needs -- qm guest exec
and the UI's IP-address display silently never worked for any VM this
script created. Found while live-testing every lxc-*/proxmox-* build type
against pve.sweet.home for an end-to-end flake audit.

Also registers a fresh sops age key for lxc-gui (no prior registration
existed), generated while testing that target live -- needed before
lxc-gui can be deployed with working secrets.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZHwwAM7cacQKkqKfQ5e8a
2026-07-21 01:25:16 +00:00
beatzaplentyandClaude Sonnet 5 42da626397 Fix NFS mount device strings on lxc-docker: use FQDN, not search domain
Check NixOS configurations / eval-hosts (pull_request) Successful in 17m29s
The previous commit's networking.search fix was wrong. Confirmed live
on lxc-docker (vmid 105) after redeploying with it: `resolvectl query
server.sweet.home` started failing again, even though
`resolvectl query --interface=eth0 server.sweet.home` still resolved
correctly to the right IP via the LAN's real DNS server. The debug log
showed why -- adding a *global* search domain via networking.search
gave systemd-resolved a domain-matched but server-less "global" scope,
which it now prioritizes over eth0's correctly-configured scope for
every "*.sweet.home" query, silently sending them to public fallback
DNS (1.1.1.1 et al) instead, which of course returns NXDOMAIN for an
internal-only name. Bare single-label names (e.g. "server") were never
going to work either way -- systemd-resolved only ever tries LLMNR for
those, never DNS search-suffixing, regardless of configuration.

Reverts the networking.search addition and instead has
modules/docker/mount-data.nix build each NFS device string from
"${vars.nfsServerHost}.${vars.homeDomain}" (a plain FQDN, no dependency
on search-domain behavior at all) -- the same pattern
modules/raspi/mount-data.nix already uses for the Raspberry Pi's share
and for the identical reason.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T48qgH3VTvs8wvwj44FEbE
2026-07-21 01:19:42 +00:00
beatzaplenty d7aba8554d Merge pull request 'Add global DNS search domain to fix NFS mounts on lxc-docker' (#23) from worktree-fix-container-dns-search-domain into main
Check NixOS configurations / eval-hosts (push) Successful in 18m29s
Reviewed-on: #23
2026-07-21 00:11:16 +00:00
beatzaplentyandClaude Sonnet 5 e176ff723d Add global DNS search domain to fix NFS mounts on lxc-docker
Check NixOS configurations / eval-hosts (pull_request) Failing after 40m2s
Cross-host references throughout this repo (vars.nfsServerHost,
vars.nixCacheHost, vars.dockerHost) are bare short names, not FQDNs.
Resolving them has always depended on whatever network stack happens
to be in play picking up the DHCP-advertised domain as a search
suffix -- NetworkManager does this by default, which is why it went
unnoticed everywhere else, but LXC containers force-disable
NetworkManager and get their systemd-networkd config written directly
by Proxmox instead, which never sets one.

Confirmed live on lxc-docker (vmid 105) after today's earlier fix for
the automount/mount=nfs bugs: systemd-resolved had no search domain
for eth0, so "server" failed to resolve ("Name or service not known")
while "server.sweet.home" resolved fine via the same DNS server --
every NFS mount in modules/docker/mount-data.nix was still failing.

networking.search sets systemd-resolved's own Domains= globally
rather than depending on the per-link DHCP path, so it isn't at the
mercy of whichever component owns a given host's interface file.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T48qgH3VTvs8wvwj44FEbE
2026-07-20 18:22:12 +00:00
beatzaplenty 61bbe5e6da clean up in ailse 3
Check NixOS configurations / eval-hosts (push) Failing after 8m41s
2026-07-20 17:49:35 +00:00
beatzaplenty ab719cc8eb Merge branch 'worktree-maintenance-script-changed-files-only' into worktree-resolve-maintenance-merge
Check NixOS configurations / eval-hosts (push) Successful in 21m49s
# Conflicts:
#	scripts/codex-maintenance.sh
2026-07-20 17:46:07 +00:00
beatzaplenty 91c977e5e7 Merge pull request 'Fix nix-cache remote-builder trust: stale host key + wrong sshKey path' (#21) from worktree-fix-nix-cache-host-key into main
Check NixOS configurations / eval-hosts (push) Failing after 12m42s
Reviewed-on: #21
2026-07-20 17:32:25 +00:00
beatzaplenty 7e9c0c2a6f Rewrite codex-maintenance.sh to scope CI checks to changed files
Check NixOS configurations / eval-hosts (pull_request) Canceled after 0s
CI was running a full eval of every host + package on every push/PR,
which was slow enough to routinely time out the Gitea runner. Default
mode now diffs against a base ref and scopes nixpkgs-fmt/statix/eval to
the files that changed and the hosts/packages they can affect; a change
to flake.nix/flake.lock/variables.nix/modules/common/* (or any other
modules/*.nix outside platforms//build-types, whose blast radius isn't
inferable from the path) falls back to evaluating everything. The old
full sweep moves behind --full-check, which CI never passes; --dry-run
adds build-planning on top of whichever scope is active.

Also trims codex-setup.sh's redundant full host eval loop -- that's
what codex-maintenance.sh is for; setup should just install tooling.
2026-07-20 17:25:22 +00:00
beatzaplentyandClaude Sonnet 5 fd773b65da Fix nix-cache remote-builder trust: stale host key + wrong sshKey path
Check NixOS configurations / eval-hosts (pull_request) Failing after 11m12s
variables.nix's nixCacheHostKey no longer matched nix-cache's actual SSH
host key (confirmed via ssh-keyscan against the live container), so every
declaratively-configured client's programs.ssh.knownHosts trusted the
wrong key -- distributed builds would fail host-key verification. Also,
modules/nix-cache/remote-builder-client.nix hardcoded sshKey to
/root/.ssh/nixremote, but the `server` host only has its own default
/root/.ssh/id_ed25519 installed (confirmed live via qm guest-agent) --
that file was never even present, so the build machine config pointed at
nothing. Standardize on each client's own default identity, matching the
per-host-key pattern vars.remoteBuilderAuthorizedKeys already uses instead
of a shared/differently-named keypair, and add
scripts/secrets/sync-nix-cache-host-key.sh (wired into
codex-maintenance.sh's --check) so the host-key drift doesn't silently
recur next time nix-cache is rebuilt or recreated.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V7yVH71vGrDVzovh9UaMu8
2026-07-20 17:21:12 +00:00
beatzaplenty d340aca403 Merge pull request 'Authorize this box's SSH key as a nix-cache remote-builder client' (#20) from add-claude-builder-key into main
Check NixOS configurations / eval-hosts (push) Failing after 18m59s
Reviewed-on: #20
2026-07-20 16:31:25 +00:00
beatzaplenty bf8ee3ce48 Merge pull request 'Worktree refactor scripts shared lib' (#19) from worktree-refactor-scripts-shared-lib into main
Check NixOS configurations / eval-hosts (push) Failing after 11m3s
Reviewed-on: #19
2026-07-20 16:29:42 +00:00
beatzaplentyandClaude Sonnet 5 d8687d979c Reorganize scripts/ into secrets/, proxmox/, and lib/ subfolders
Check NixOS configurations / eval-hosts (pull_request) Failing after 30m6s
scripts/ had grown to 10 top-level scripts covering three distinct
concerns (sops/age + SSH host-key management, Proxmox deployment, and
repo-wide bootstrap/CI) with no grouping. Move the key-management scripts
(backup-admin-key.sh, rotate-admin-key.sh, prepare-host-key.sh,
sync-host-keys.sh) into scripts/secrets/, and the Proxmox scripts
(create-proxmox-resource.sh, configure-nix-cache-client.sh) into
scripts/proxmox/; leave env.sh, codex-setup.sh, codex-maintenance.sh, and
bump-nixpkgs-release.sh at the top level (frequently hand-typed or pure
shared config) and scripts/lib/ as-is.

Updates every cross-reference: each moved script's repo_root computation
(now one directory deeper), shellcheck source= directives, inter-script
paths (create-proxmox-resource.sh's call into sync-host-keys.sh and its
remote bootstrap of configure-nix-cache-client.sh on the Proxmox node),
and every doc/module mention (CLAUDE.md's Scripts section reorganized to
match, README.md, docs/auto-installer.md, docs/proxmox-images.md,
modules/installer/common.nix, modules/platforms/lxc.nix). CI workflows
need no change -- they only invoke codex-maintenance.sh, which didn't
move. Verified via bash -n, shellcheck (no new warnings beyond the
pre-existing SC1091/SC2029/SC2095 baseline), and live dry-runs of
sync-host-keys.sh --all and create-proxmox-resource.sh --list from their
new paths.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-20 16:20:45 +00:00
beatzaplentyandClaude Sonnet 5 a2b557c034 Lift duplicated sops/age and confirm-prompt logic into scripts/lib/
scripts/backup-admin-key.sh, rotate-admin-key.sh, and sync-host-keys.sh
each independently resolved sops/age's default key-file path, derived an
age pubkey from an identity file, and (two of them) ran `sops updatekeys`
the same way -- now shared via scripts/lib/sops-age.sh. Also extracted the
"type X to confirm" prompt duplicated across create-proxmox-resource.sh
and sync-host-keys.sh into scripts/lib/confirm.sh. Pure extraction, no
behavior change -- each call site produces identical commands/output to
before.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-20 15:38:32 +00:00
58 changed files with 2959 additions and 581 deletions
Submodule .claude/worktrees/proxmox-remote-build deleted from a5990ccf7d
+9 -1
View File
@@ -13,9 +13,17 @@ jobs:
steps: steps:
- name: Check out repository - name: Check out repository
uses: actions/checkout@v4 uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Install Nix - name: Install Nix
uses: DeterminateSystems/nix-installer-action@v19 uses: DeterminateSystems/nix-installer-action@v19
- name: Run maintenance checks (secrets, fmt, lint, eval) # Scoped to files changed since the PR base / previous push -- see
# scripts/codex-maintenance.sh. CI never passes --full-check: that
# full sweep is for local/manual use, since it's slow enough to time
# out this runner.
- name: Run maintenance checks (secrets, fmt, lint, eval -- changed files only)
env:
MAINT_BASE_SHA: ${{ github.event.pull_request.base.sha || github.event.before }}
run: bash scripts/codex-maintenance.sh run: bash scripts/codex-maintenance.sh
+9 -1
View File
@@ -13,9 +13,17 @@ jobs:
steps: steps:
- name: Check out repository - name: Check out repository
uses: actions/checkout@v4 uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Install Nix - name: Install Nix
uses: DeterminateSystems/nix-installer-action@v19 uses: DeterminateSystems/nix-installer-action@v19
- name: Run maintenance checks (secrets, fmt, lint, eval) # Scoped to files changed since the PR base / previous push -- see
# scripts/codex-maintenance.sh. CI never passes --full-check: that
# full sweep is for local/manual use, since it's slow enough to time
# out this runner.
- name: Run maintenance checks (secrets, fmt, lint, eval -- changed files only)
env:
MAINT_BASE_SHA: ${{ github.event.pull_request.base.sha || github.event.before }}
run: bash scripts/codex-maintenance.sh run: bash scripts/codex-maintenance.sh
+66 -1
View File
@@ -3,11 +3,30 @@ keys:
- &docker age19gfn2yedg76dmztm4hncr7vf3r3c9j0qpt4rap7y7gersjk4m3ks2lhd0e - &docker age19gfn2yedg76dmztm4hncr7vf3r3c9j0qpt4rap7y7gersjk4m3ks2lhd0e
- &server age1ll6hj5ggruetgjwjfnplpn5xtq35uhlcdflksx3xmnjm6s3uad9sz70jkf - &server age1ll6hj5ggruetgjwjfnplpn5xtq35uhlcdflksx3xmnjm6s3uad9sz70jkf
- &nix-cache age120le4a5l8dh3lyfgvmj3d9ksmej6ajs5mer5y7r0vfg3x9fn69dqf8xgzu - &nix-cache age120le4a5l8dh3lyfgvmj3d9ksmej6ajs5mer5y7r0vfg3x9fn69dqf8xgzu
- &lxc-minimal age1qz9d4ka4xgexujyd247s7lp737sulp5fhxl5d65fj2ykvc4j4edqrsdks8
- &nix-minimal age120whqj96g26lsgy4udvgsn8dc9lumh8jeu3a564fx79rjr5lxffqmrljuu - &nix-minimal age120whqj96g26lsgy4udvgsn8dc9lumh8jeu3a564fx79rjr5lxffqmrljuu
- &proxmox-minimal age10at8862478urh0eeuwh8hzln6ck78jgwtztgxatwqlzwagg77y5snm4xzg - &proxmox-minimal age10at8862478urh0eeuwh8hzln6ck78jgwtztgxatwqlzwagg77y5snm4xzg
- &lxc-nix-cache age1xjst4frdh0th6q8m7p7u9g5af7ty5jqeum0p6z8a52a9q7st7ewqw8yl9j - &lxc-nix-cache age1xjst4frdh0th6q8m7p7u9g5af7ty5jqeum0p6z8a52a9q7st7ewqw8yl9j
- &lxc-docker age1ezk9x53zt8kcnscdm80jcyf0xq97vndv7jsn3rl8cc0cwm2jmpmq372dzs - &lxc-docker age1ezk9x53zt8kcnscdm80jcyf0xq97vndv7jsn3rl8cc0cwm2jmpmq372dzs
- &lxc-minimal age1jy444f9d9stygj4p3w9kh54cqcfr654tvr75tdvee5cxsgtdtc9q3v60ep
- &lxc-pxe-boot age1fxxzpnfse8nd9wz78ht3m0plrmraacf4cpga0pe8fm2tdnqcgy8q7qsyvp
- &lxc-gui age190htw7prp4vln076dxjx3gxxaq06h0zl0te7cqgpx79vl3lhkaes8suy05
- &proxmox-server age1ukpqxzl44mnjpy5r96sfuc5sqzm47u4k8ujjh5qdgy6jvl9uqgpspymqfk
- &vm-server age15kh7akxlx7zn00tey79rq2g8lgs4j5y77rcnyfxrxap8ckfu0a9sqvtdhh
- &baremetal-gui age1ehkswwz2pqaz4svzh7ela5tdnssl8kn6d4vwwxd6zwg8exfpd43syyrrjp
- &lxc-tor-relay age1gl5ujmhd2pe37acd0eaar8tmnf2na28n8l28s8xct40vqpxvsdwqf49g4l
- &linode-docker age1eu65wsmez68gegnufl0gqrs0e6w2409mypjlajlq383d7l2e3pjqy5kn6v
- &linode-gui age17pwyghxr6lq06fw46gwqzhc9ut4paz28rpwx5pmv3cxwak6rgyjsw7lk3w
- &linode-minimal age1jukmg69cqxnjd0lp5f534jhqe65rxew2hufcmyjxa3rkw6ayef6s2ylcmy
- &linode-nix-cache age1tzsrtwd3p3lrr9g7nv3z5nvmzsz54t2uc6tfqwutp6usmav83s2sck25cc
- &linode-server age1rf4kj99wuq59k7w8ar326djmgmpl9hcwlnuag07f8gauq8c3y5mqne87s4
- &linode-tailscale-exit-node age1d0zhx7u3mfs3nktl67npey87cze4dwsfvfvgaje4rh3gwv500yssckcf6u
- &lxc-server age1jc6wx33hdhgwhk6nzy5rr8fkgmqxk9um639tk5h632nqfyaw8czskdptj9
- &lxc-tailscale-exit-node age1pgykvq4pmxhhjrqupcp99fyad2uht40pt79dkzg66cfvsjy5apjqls8u68
- &proxmox-docker age19ht95nv8uhz2shjmakeut8mc3l5spvrcxs3thhe85an7u02r6sysyv457n
- &proxmox-gui age1mhaze5tgvc9lwjpml6dnp3xc292337wgm6376yh6tq4fn492ndjq9h23dq
- &proxmox-nix-cache age15me6sx0f8r58xh9v7aqrj6e99n7eu555jkpw422txpkqt4r02v0qpahlyq
- &proxmox-pxe-boot age1q3hu6eh3mt4saey7dc3yu04s7knnk2ygpm9xt6mg6rqtem6l0uyq4wzul5
- &proxmox-tailscale-exit-node age1tczst3x7thwtcz4vce4rg6kmlsszzm6j45kn2nv860z9wa93a92s05ppmc
creation_rules: creation_rules:
# Shared across every currently-deployed host: root/nixos password hash, # Shared across every currently-deployed host: root/nixos password hash,
@@ -26,6 +45,25 @@ creation_rules:
- *lxc-nix-cache - *lxc-nix-cache
- *proxmox-minimal - *proxmox-minimal
- *lxc-docker - *lxc-docker
- *lxc-pxe-boot
- *lxc-gui
- *proxmox-server
- *vm-server
- *baremetal-gui
- *lxc-tor-relay
- *linode-docker
- *linode-gui
- *linode-minimal
- *linode-nix-cache
- *linode-server
- *linode-tailscale-exit-node
- *lxc-server
- *lxc-tailscale-exit-node
- *proxmox-docker
- *proxmox-gui
- *proxmox-nix-cache
- *proxmox-pxe-boot
- *proxmox-tailscale-exit-node
- path_regex: secrets/nix-cache\.yaml$ - path_regex: secrets/nix-cache\.yaml$
key_groups: key_groups:
@@ -33,15 +71,42 @@ creation_rules:
- *admin - *admin
- *nix-cache - *nix-cache
- *lxc-nix-cache - *lxc-nix-cache
- *linode-nix-cache
- *proxmox-nix-cache
- path_regex: secrets/server\.yaml$ - path_regex: secrets/server\.yaml$
key_groups: key_groups:
- age: - age:
- *admin - *admin
- *server - *server
- *proxmox-server
- *vm-server
- *linode-server
- *lxc-server
- path_regex: secrets/docker\.yaml$ - path_regex: secrets/docker\.yaml$
key_groups: key_groups:
- age: - age:
- *admin - *admin
- *docker - *docker
- path_regex: secrets/tor-relay\.yaml$
key_groups:
- age:
- *admin
- *lxc-tor-relay
# gui-host-specific secrets (currently: wifi-password, see
# modules/networking/wifi.nix). Only *lxc-gui has a registered key today
# -- proxmox-gui/linode-gui/baremetal-gui haven't been provisioned via
# scripts/secrets/sync-host-keys.sh yet, so whichever variant is actually
# deployed next needs its recipient added here (and `sops updatekeys` rerun)
# before it can decrypt this.
- path_regex: secrets/gui\.yaml$
key_groups:
- age:
- *admin
- *lxc-gui
- *baremetal-gui
- *linode-gui
- *proxmox-gui
+6 -1
View File
@@ -35,9 +35,14 @@ Use these commands when validating changes:
```bash ```bash
bash scripts/codex-setup.sh bash scripts/codex-setup.sh
bash scripts/codex-maintenance.sh bash scripts/codex-maintenance.sh
bash scripts/codex-maintenance.sh dry-run
``` ```
With no flags, `codex-maintenance.sh` scopes fmt-check/statix/eval to files
changed against a base ref — this is what CI runs on every push/PR. For the
full sweep (every host, every package — slow; CI never runs this), use
`bash scripts/codex-maintenance.sh --full-check` (add `--dry-run` for build
planning on top of whichever scope is active).
Host evaluation is safe when limited to drvPath checks: Host evaluation is safe when limited to drvPath checks:
```bash ```bash
+150
View File
@@ -0,0 +1,150 @@
# Flake End-to-End Audit Report
**Date:** 2026-07-21
**Scope:** Full static lint/eval sweep + live build/deploy/interrogate/destroy testing of every `lxc-*` and `proxmox-*` flake target against `pve.sweet.home`, plus an audit of the operator's ability to manage the flake/secrets tooling.
**Branch:** `worktree-flake-e2e-audit` (this session's isolated worktree)
## Executive Summary
The flake itself is in good shape: `nixpkgs-fmt`, `statix`, and a full eval + dry-run build of every host and package are all clean. Every `lxc-*`/`proxmox-*` target's NixOS configuration builds successfully — no target has a broken derivation graph.
The issues found are **operational, not code-level**:
1. **pve.sweet.home is critically low on disk space** (91-95% full during this session) and cannot currently build the two largest closures (`gui`, `pxe-boot`) to completion — this actively blocks deploying/redeploying those hosts via the documented workflow.
2. **A real, reproducible secrets-decryption failure** was caught live: a stale cached container image (built before a same-day sops-key fix) boots with sshd never starting and every secret failing to decrypt. This is a **general hazard in `create-proxmox-resource.sh`'s "reuse the cached image if present" default**, not a one-off.
3. **sops key/anchor drift**: `proxmox-minimal` has a `.sops.yaml` recipient anchor with no corresponding private key anywhere in this environment; several `lxc-*`/`proxmox-*` targets have no sops registration at all yet.
4. One concrete script bug was found and **fixed in this session**: `create-proxmox-resource.sh` never enabled the QEMU guest agent channel on VMs it creates, despite the guest OS already running it.
5. A management-surface audit (of the operator's ability to run this repo day to day) found 5 process gaps, detailed below.
Nothing here required or received a `nixos-rebuild switch/boot/test`, `nixos-install`, or any disk-formatting command — all validation was `nix build`/`nix eval`, plus disposable `pct`/`qm` create-then-destroy cycles via the repo's own `create-proxmox-resource.sh`.
---
## 1. Static Analysis Results — all clean
`bash scripts/codex-maintenance.sh --full-check --dry-run` (whole-tree sweep, not just changed files):
| Check | Result |
|---|---|
| Secret grep | Clean — only the documented exceptions (installer's own hashed passwords, `access-tokens` comment references) |
| `nixpkgs-fmt --check` | 0/53 files would be reformatted |
| `statix` | No lint warnings |
| nix-cache host key drift check | Up to date |
| Full eval of every host's `system.build.toplevel` | All 19 `nixosConfigurations` targets evaluate cleanly |
| Dry-run build of every host + package | All succeed, no derivation errors |
No drift, no formatting issues, no lint findings anywhere in the tree.
---
## 2. Per-Target Test Results
Legend: **LIVE** = built on pve, `pct`/`qm` create → interrogated → destroyed. **BUILD-ONLY** = `nix build` validated the config (mostly `.config.system.build.toplevel`, occasionally `.tarball`), no resource created on pve.
| Target | Test type | Result | Notes |
|---|---|---|---|
| `lxc-docker` | BUILD-ONLY | ✅ PASS | Live redeploy skipped — CT105 is already running this identity in production; `--allow-duplicate-host` would have destroyed it. |
| `lxc-minimal` | **LIVE** | ✅ PASS (after retry) | First attempt reused a stale cached tarball predating a same-day sops-key commit → activation failed, sshd never started (see Finding #2). Redeployed with `--force-rebuild`: clean boot, `systemctl is-system-running` = `running`, secrets decrypted, sshd listening, users correct. |
| `lxc-nix-cache` | BUILD-ONLY | ✅ PASS (after retry) | Live redeploy skipped — CT101 is already running this identity. First local build attempt appeared to hang on a remote-builder handoff to nix-cache; killed and retried with `--builders ""` (local-only), succeeded. |
| `lxc-gui` | **LIVE (attempted)** | ⚠️ BLOCKED by pve disk space | Registered a fresh sops key (no prior registration existed), built successfully through the full NixOS system closure, then **failed packaging the tarball**: `No space left on device` on pve's root filesystem. Not a flake defect. |
| `lxc-pxe-boot` | **LIVE (attempted)** | ⚠️ BLOCKED by pve disk space | Same failure as `lxc-gui` — this target additionally builds a full nested installer/netboot image (`stage-installer-artifacts.nix`), making it similarly large. Failed with the same `No space left on device` error, immediately after the gui attempt had already consumed pve's remaining headroom. |
| `lxc-server` | BUILD-ONLY | ✅ PASS | No sops key registered yet; live deploy also would have hit `boot.zfs.extraPools` trying to import a real ZFS pool that doesn't exist in an isolated test container — an expected limitation of testing this build type outside its real hardware, not a bug. |
| `lxc-tailscale-exit-node` | BUILD-ONLY | ✅ PASS | No sops key registered yet. |
| `lxc-tor-relay` | BUILD-ONLY | ✅ PASS | Live redeploy skipped — CT106 already holds this identity in production. |
| `proxmox-docker` | BUILD-ONLY | ✅ PASS (after retry) | Live redeploy skipped — both CT105 *and* VM103 already hold `docker` identities. Combined `toplevel` + `diskoImagesScript` build crashed with a **Nix-internal assertion failure** (`worker.cc:360`) under this session's memory pressure (see Finding #6) — not a flake bug. Retried with `toplevel` alone: clean. |
| `proxmox-minimal` | **LIVE (attempted)** | ⚠️ BLOCKED by key drift → BUILD-ONLY | `.sops.yaml` has a registered `&proxmox-minimal` anchor but **no corresponding private key exists anywhere in this environment** — the script correctly refused to generate a mismatched replacement. Fell back to `toplevel` build: ✅ PASS. |
| `proxmox-nix-cache` | BUILD-ONLY | ✅ PASS | No sops key registered yet. |
| `proxmox-gui` | BUILD-ONLY | ⚠️ Killed after ~40min (resource-limited) | This session's local build machine has only 2GB RAM; swap filled completely (2.0/2.0GB) and the build stalled, so it was killed rather than risk destabilizing the session further. **Not a flake defect** — the equivalent `gui` NixOS configuration already proved fully buildable during the `lxc-gui` live attempt above (it built the entire system closure successfully and only failed at the pve-side tarball-packaging step due to disk space, not the config). |
| `proxmox-pxe-boot` | BUILD-ONLY | ⚠️ Killed after ~35min (resource-limited) | Was deep into building the nested installer's kernel initrd (this build type bundles a full netboot installer image via `stage-installer-artifacts.nix`) when killed to keep the audit moving. **Not a flake defect** — this target's own module logic was already effectively validated via the earlier *live* pve deploy attempt (`lxc-pxe-boot` above), which built the complete image and only failed at the final tarball-packaging step due to pve's disk space (Finding 1). |
| `proxmox-server` | BUILD-ONLY | ✅ PASS | No sops key registered yet; same ZFS-pool caveat as `lxc-server` would apply to a live deploy. |
| `proxmox-tailscale-exit-node` | BUILD-ONLY | ✅ PASS | No sops key registered yet. |
**Not tested at all:** `linode-*` targets (not deployable to Proxmox) and `installer` (not a normal host) — both were still covered by the static eval/dry-run-build sweep above.
---
## 3. Findings, Ranked by Severity
### Finding 1 — pve.sweet.home is critically low on disk space (blocks real deployments)
At session start: `/dev/mapper/pve-root` was **95% full, 5.3GB free** (of 94GB). After two failed large builds it recovered slightly to **91% full, 8.2GB free** (nix cleans up its own failed-build scratch space). `/nix/store` alone is 26GB; `nix-store --gc --print-dead` reports **zero** reclaimable garbage — everything currently in the store is a live GC root, so `nix-collect-garbage` won't help without first removing old roots.
**Why it matters:** `create-proxmox-resource.sh` builds every VM/CT image **directly on pve**, not on a build machine and transferred over. With <10GB headroom, any closure approaching a few GB (the `gui` build type: full Cinnamon desktop + Firefox + LibreOffice + GIMP + VS Code + xrdp; the `pxe-boot` build type: nginx/atftpd *plus* an entire nested installer/netboot image) cannot currently be built there at all. Both `lxc-gui` and `lxc-pxe-boot` failed live with `No space left on device` during this audit.
**Recommended action:** Expand `pve-root`'s LV, or free space by pruning old container templates in `/var/lib/vz/template/cache` (1.5GB) / old backups in `/var/lib/vz/dump` (306MB) / auditing what's pinning 26GB of `/nix/store` as live GC roots (likely `result-*` symlinks — see below). This is real production disk state; **not something this session touched or fixed** — it needs the operator's judgment on what's safe to remove.
**Secondary, smaller finding:** every `create-proxmox-resource.sh` run leaves a `result-<target>` symlink in the node's repo checkout as a permanent GC root (`ls /root/nixos/result-*` on pve showed 3 from this session alone: `lxc-docker`, `lxc-minimal`, `lxc-nix-cache`). These accumulate forever and pin their entire closures in the store. Consider having the script clean up its own `result-*` link after staging the built artifact (or use a temp `--out-link` under `/tmp`), so `nix-collect-garbage` can actually reclaim old build outputs.
### Finding 2 — Stale cached images can silently ship broken secrets (reproduced live)
`create-proxmox-resource.sh`'s default behavior is: if the node already has `<target>.tar.xz`/`.raw` staged, **reuse it** — only `--force-rebuild` forces a fresh build. This session hit exactly the failure mode `docs/auto-installer.md` already warns about: `lxc-minimal`'s cached tarball (built 2026-07-20T15:57Z) predated a same-day sops-key fix commit (2026-07-20T17:49Z, "clean up in ailse 3"). The deployed container booted with:
```
sops-install-secrets: failed to decrypt '.../common.yaml': Error getting data key: 0 successful groups required, got 0
Activation script snippet 'setupSecrets' failed (1)
```
— every secret permanently failed to decrypt, `sshd` never started (though the container otherwise looked "running"). This was **not a code bug**: the currently-committed `secrets/common.yaml` decrypts fine for that host's key when checked independently; the *cached artifact on pve* simply reflected an older commit's ciphertext. Redeploying with `--force-rebuild` fixed it immediately.
**Why it matters:** this is silent and easy to trigger by accident — any operator who redeploys a host without remembering `--force-rebuild` after a secrets change gets a container that looks like it started (`pct start` succeeds, `pct status` = running) but is completely inaccessible.
**Recommended action:** Have `create-proxmox-resource.sh` compare the cached image's build timestamp (or embed the source commit hash in the staged filename) against current HEAD, and warn (or refuse without `--force-rebuild`) if they differ — rather than silently trusting presence alone.
### Finding 3 — sops key/anchor drift
Two concrete instances hit live during this session:
- **`proxmox-minimal`**: `.sops.yaml` already has a registered `&proxmox-minimal` age recipient, but this environment's `host-keys/` directory has no corresponding private key file. `sync-host-keys.sh` correctly refused to generate a replacement (it would silently mismatch whatever's already registered/deployed) — but this means **no environment currently has this host's private key**, unless it exists on some other machine that was never backed up here.
- **`lxc-gui`**, and by the same logic `lxc-server`/`lxc-tailscale-exit-node`/most `proxmox-*` targets, have **no sops registration at all yet** — expected for undeployed hosts per `docs/auto-installer.md`, but this session's live-testing needed to register `lxc-gui`'s key on the fly, which immediately hit **Finding 3b**: registering a key locally does nothing for pve's build until it's pushed to `origin/main` (pve builds via `git pull`, not from this uncommitted worktree). This is exactly gap #4 the management-surface audit (below) already flagged in the abstract — this session hit it concretely.
**Recommended action:** for `proxmox-minimal`, decide whether to regenerate its key (destroying old-key decrypt access, if anything still holds it) or track down wherever the original private key lives and back it up here. For the general pattern, see the management-surface audit's recommendation to pre-flight-check key registration before building.
### Finding 4 — QEMU guest agent never wired up (found and fixed this session)
`modules/common/configuration.nix:44` sets `services.qemuGuest.enable = true` on every host — the guest-side agent daemon is correctly enabled everywhere. But `scripts/proxmox/create-proxmox-resource.sh`'s `qm create` call never passed `--agent 1`, so **Proxmox never created the virtio-serial channel** the agent needs. Every `proxmox-*` VM this script ever created was silently missing `qm guest exec`/IP-address reporting in the Proxmox UI, despite the guest daemon actually running.
**Status: fixed in this session's worktree** (`scripts/proxmox/create-proxmox-resource.sh`, `qm create` now includes `--agent enabled=1`) — see the diff, included in the PR from this session.
### Finding 5 — Orphaned container on pve (CT102)
`pve.sweet.home` has a stopped LXC container, **VMID 102**, with an essentially empty config (`lock: create` and nothing else — no hostname, no rootfs, no network) — the leftover of a `pct create` that started and never finished. It predates this session (not created by any of this audit's activity) and wasn't touched. **Recommend the operator confirm it's abandoned and remove it** (`pct destroy 102 --purge 1`) — left as-is it may be someone's genuine in-progress work, so it wasn't assumed safe to delete autonomously.
### Finding 6 — Nix-internal crash under memory pressure (tooling, not flake)
Building `proxmox-docker`'s `toplevel` and `diskoImagesScript` together crashed with a Nix-internal assertion failure (`Assertion '!awake.empty()' failed ... worker.cc:360`, a known class of bug in Nix's multi-goal build scheduler) while this session's 2GB-RAM build container was under heavy swap pressure (1.8-2.0/2GB swap in use) from a separate concurrent build. Retrying the same target alone (no concurrency) succeeded cleanly. **Not a flake defect** — purely an artifact of this session's constrained build environment; noted for completeness since it looked alarming in isolation.
### Finding 7 — Management-surface audit: 5 operability gaps
A focused audit of "can the operator actually run this repo day to day" (flake, home-manager, sops, related scripts) found:
1. **No documented recovery path if the `&admin` sops age key is lost without a backup.** `scripts/secrets/backup-admin-key.sh` exists and works but is referenced nowhere in `README.md`/`docs/` — no forcing function ensures a backup was ever taken. `rotate-admin-key.sh` requires the *old* key to re-key; there's no bootstrap-from-nothing path documented (the real fallback — deriving an age identity from any still-live host's own SSH key — isn't written down anywhere).
2. **home-manager has no standalone iteration path.** It's wired only inside `nixosConfigurations` (`flake.nix`) — no `homeConfigurations` output. The fastest real shortcut (`nix build .#nixosConfigurations.<target>.config.home-manager.users.nixos.home.activationPackage`) isn't documented anywhere, so the practical workflow is a full host rebuild to test one HM tweak.
3. **Gitea's flake-lock-update workflow pushes straight to `main` with no pre-merge validation.** `.gitea/workflows/update-flake-lock.yml` commits and pushes `nix flake update`'s result directly; `codex-maintenance.sh` only runs *after*, on the resulting push — a genuinely broken lockfile bump lands on `main` before anything catches it. (The GitHub-side workflow is safer — PR-based — but has the opposite gap: nothing alerts if the PR sits unmerged.)
4. **No pre-flight check that a build target has a registered sops key before building it.** `docs/auto-installer.md` documents the failure mode (silent, total secrets-decrypt failure) but nothing in `create-proxmox-resource.sh` refuses to proceed when it's about to build a target with no `.sops.yaml` anchor — it's on the operator to remember. This session's `lxc-gui` test hit close to this exact gap (needed the key added on the fly, mid-session).
5. **`vars.remoteBuilderAuthorizedKeys` has the same drift risk as `vars.nixCacheHostKey`, but no checker script.** `sync-nix-cache-host-key.sh --check` guards the latter; the former (and `vars.pxeServerIp`/`vars.pbsIp`) has no equivalent — a rotated/revoked client key just silently stops working with no diagnostic pointing back here.
---
## 4. Action Plan (priority order)
1. **Free up disk space on pve.sweet.home** (or expand `pve-root`). Blocking: `lxc-gui`, `proxmox-gui`, `lxc-pxe-boot`, `proxmox-pxe-boot` cannot currently be built/redeployed on this node at all.
2. **Decide on `proxmox-minimal`'s orphaned sops key**: locate the original private key and back it up here, or accept regenerating it (breaks decrypt access for whoever/whatever currently holds the old one).
3. **Merge this session's PR** (see below) to get the `--agent 1` fix and `lxc-gui`'s new sops registration onto `main` — required before `lxc-gui` can be live-redeployed with working secrets.
4. **Add a staleness guard to `create-proxmox-resource.sh`'s cache-reuse path** (Finding 2) — highest-leverage fix, since it silently produces a broken-but-"running" host.
5. **Add a pre-flight sops-anchor check to `create-proxmox-resource.sh`** (management-surface gap #4) — same root cause class as #4 above, catch it before building instead of at first boot.
6. Investigate/clean up **CT102** on pve (Finding 5) — confirm abandoned, then remove.
7. Document `backup-admin-key.sh` in `README.md`'s Security Notes and add the live-host-key bootstrap-recovery procedure to `docs/` (management-surface gap #1).
8. Add pre-push validation to the Gitea flake-lock-update workflow (management-surface gap #3).
9. Lower-priority: document the home-manager `activationPackage` shortcut (gap #2); extend `sync-nix-cache-host-key.sh`'s drift-check pattern to `remoteBuilderAuthorizedKeys` (gap #5).
10. Follow-up session: finish build-validating `proxmox-gui` and `proxmox-pxe-boot` (both killed here after 35-40min on this session's 2GB-RAM machine — not failures, just unfinished) once pve has headroom (item 1) — ideally from a machine with more RAM. `proxmox-server` and `proxmox-tailscale-exit-node` already passed build-only validation in this session, no follow-up needed.
---
## 5. Uncommitted Changes From This Session
This worktree (`worktree-flake-e2e-audit`) currently has:
- `scripts/proxmox/create-proxmox-resource.sh` — the `--agent enabled=1` fix (Finding 4).
- `.sops.yaml` / `secrets/common.yaml``lxc-gui`'s new age key registered as a recipient (generated live during this session's testing).
Per this session's standard workflow, these will be committed, pushed, and opened as a draft PR rather than pushed to `main` directly — merging it is the operator's call, and is also **prerequisite to live-redeploying `lxc-gui` successfully** (its build will keep hitting the sops-staleness failure from Finding 2 on pve until this registration is on `origin/main`).
+253 -66
View File
@@ -31,17 +31,96 @@ machines when deployed.
the auto-installer (see `docs/auto-installer.md`). Never commit its the auto-installer (see `docs/auto-installer.md`). Never commit its
contents; if `git status` ever shows it as trackable, something is wrong. contents; if `git status` ever shows it as trackable, something is wrong.
### Two Proxmox nodes: `pve1.sweet.home` (production) and `pve-test.sweet.home` (sandbox)
There are two SSH-reachable Proxmox nodes on the LAN, both defined in
`scripts/env.sh` (`PVE1_HOST` / `PVE_TEST_HOST`), individually targetable
via `scripts/proxmox/create-proxmox-resource.sh --node <host>` or by
overriding `PROXMOX_HOST`. `PROXMOX_HOST` itself still defaults to
`PVE1_HOST` (production) — that default, and every other script behavior,
is unchanged from before `pve-test` existed; the only thing new is that
`pve-test` can now be reached at all. They are **not interchangeable**
one is real production infrastructure, the other exists specifically so
there's somewhere safe to test. The restriction below is a policy for
Claude specifically, not a change to the tooling's own default or
anything the operator needs to opt into.
#### `pve1.sweet.home` (production — off-limits to Claude)
A real, live Proxmox node hosting production VMs/containers — not a
sandbox, and not Claude's to touch by default.
- **Off-limits at all times unless the operator has given explicit,
same-session instructions to act on this specific host.** That
authorization is scoped to the task it was given for — don't carry it
forward to unrelated later work in the same conversation, and never
assume it from a previous session.
- **Read-only for existing state is always fine, authorization or not.**
You may SSH in (or use `pvesm`, `qm list`, `pct list`, `qm config`, `pct
config`, the Proxmox API, etc.) to inspect the node's config, storage,
and any existing VM/container — including ones this repo didn't create.
- **Never** modify, stop, restart, delete, reconfigure, or create anything
on this node (`qm set`, `pct set`, `qm destroy`, `pct destroy`, `qm
stop`, `pct stop`, `qm create`, `pct create`, snapshot operations,
storage changes, etc.) — including scratch/test resources — without
that explicit go-ahead. Use `pve-test.sweet.home` for anything
exploratory instead; it exists precisely so `pve1` never has to be the
answer to "where do I test this."
- **This is a Claude-specific policy, not something the scripts enforce.**
`scripts/env.sh`/`create-proxmox-resource.sh` default to `pve1` exactly
as they did before `pve-test` existed, with no extra flag or prompt
required — that's deliberate, so the operator's own existing workflows
don't change. Claude, however, must never rely on that default: every
Proxmox action Claude takes on its own initiative — not explicitly
pointed at `pve1` by the operator this session — targets `pve-test`
instead (e.g. `--node "$PVE_TEST_HOST"`, or `PROXMOX_HOST=$PVE_TEST_HOST`).
Claude's own default is `pve-test`, full stop, regardless of what the
tooling's own unqualified default happens to be.
#### `pve-test.sweet.home` (sandbox — Claude's default target)
A separate Proxmox node set aside for testing. The *tooling's* default is
still production (`PROXMOX_HOST``PVE1_HOST`, see above) — but
**Claude's own default is this node**: absent an explicit, same-session
instruction to use `pve1`, every Proxmox action Claude initiates targets
`pve-test`. Once targeted, it's safe to create, interrogate, and destroy
resources on without asking first.
- **Test VMs/containers are allowed, but must be torn down.** Create a
scratch VM or container here (e.g. via
`scripts/proxmox/create-proxmox-resource.sh` or raw `qm`/`pct create`)
to validate something. Anything created this way must be destroyed
again in the same session, before ending the task — never leave a test
resource running. Use a VMID/name that's obviously scratch (and doesn't
collide with a real flake target) so it's unambiguous what's safe to
remove.
- **Node-level config is still not yours to change.** Creating/destroying
your own scratch guests is fine; Proxmox host config, storage pools, and
networking on `pve-test` itself are still the operator's call to make
manually, same as on `pve1`.
## Commands ## Commands
```bash ```bash
# One-time environment bootstrap (installs Nix if missing, prints hosts) # One-time environment bootstrap (installs Nix if missing, prints hosts)
bash scripts/codex-setup.sh bash scripts/codex-setup.sh
# Full validation: secret grep, nixpkgs-fmt --check, statix lint, eval all hosts # Changed-files-only validation: secret grep (whole repo), nixpkgs-fmt --check
# and statix on changed *.nix files, eval of the hosts/packages those changes
# can affect. This is what CI runs on every push/PR.
bash scripts/codex-maintenance.sh bash scripts/codex-maintenance.sh
# Same, plus a dry-run build (no result symlink) of every host's toplevel # Full sweep: nixpkgs-fmt --check/statix over the whole tree, eval every host
bash scripts/codex-maintenance.sh dry-run # and package. Slow (minutes) -- CI never runs this; use it locally before a
# release or after touching modules/common/*, flake.nix, or variables.nix for
# extra confidence beyond the automatic full-fallback those paths already
# trigger in the default mode (see below).
bash scripts/codex-maintenance.sh --full-check
# Either mode, plus a dry-run build (no result symlink) of every host/package
# in whichever scope is active
bash scripts/codex-maintenance.sh --dry-run
bash scripts/codex-maintenance.sh --full-check --dry-run
# List the hosts the flake currently exposes # List the hosts the flake currently exposes
nix eval --json .#nixosConfigurations --apply builtins.attrNames | jq -r '.[]' nix eval --json .#nixosConfigurations --apply builtins.attrNames | jq -r '.[]'
@@ -58,36 +137,122 @@ maintenance script pulls them via `nix run github:NixOS/nixpkgs/nixos-25.11#<too
There is no test suite — "correctness" here means the flake evaluates and There is no test suite — "correctness" here means the flake evaluates and
`nixpkgs-fmt`/`statix` are clean. `nixpkgs-fmt`/`statix` are clean.
**In an interactive agent session**, prefer targeted checks over full-repo With no flags, `codex-maintenance.sh` diffs against a base ref (env
sweeps: after editing one or two hosts/modules, evaluate just the `MAINT_BASE_SHA`, else the PR base SHA in CI, else `HEAD^` locally) and scopes
`nixosConfigurations.<host>` you touched (plus any `config.system.build.tarball` fmt-check/statix to the changed `*.nix` files and eval to the hosts/packages
/`diskoImagesScript`/package output affected) rather than looping over every those changes can affect — a `hosts/<name>/host.nix` edit only evals that
host`codex-maintenance.sh` evaluates every `nixosConfigurations` host plus host's targets, a `modules/platforms/<platform>.nix` edit only evals that
every package/tarball/image variant and is slow to run after each small platform's hosts, and so on. A change to `flake.nix`, `flake.lock`,
change. Reserve a full `variables.nix`, `modules/common/*`, or any other `modules/*.nix` file outside
`codex-maintenance.sh` run for changes that plausibly affect every host `platforms/`/`build-types/` (whose blast radius isn't safely inferable from
(`modules/common/*`, `flake.nix`, `variables.nix`) or as a final check before the path alone) falls back to evaluating every host and package, same as
committing. This is a session-workflow preference only — it does not apply to `--full-check` would, just without the whole-tree fmt/statix sweep. This
CI, which should keep running the full script on every push/PR regardless of exists because the whole-tree sweep is what was timing out CI; **CI always
diff size; that's the point of it. runs the plain, no-flag form and never passes `--full-check`.**
The default mode's diff is against the working tree (uncommitted and staged
edits included, not just committed ones), so it's already the right tool for
an interactive session too: after editing one or two hosts/modules, plain
`bash scripts/codex-maintenance.sh` naturally scopes to just what you
touched. Reserve `--full-check` for changes that plausibly affect every host
(`modules/common/*`, `flake.nix`, `variables.nix` — though the default mode
already falls back to evaluating everything for those paths, `--full-check`
additionally re-checks fmt/statix over the whole tree) or as a final check
before committing.
## Scripts ## Scripts
Beyond `codex-setup.sh`/`codex-maintenance.sh` above, `scripts/` also has: Beyond `codex-setup.sh`/`codex-maintenance.sh` above, `scripts/` is
organized by purpose: `scripts/secrets/` (sops/age + SSH host-key
management), `scripts/proxmox/` (Proxmox deployment), `scripts/installer/`
(the auto-installer's own shell script, templated into the image — see
below), `scripts/lib/` (shared helpers, sourced by the scripts below — not
run directly), and a handful of repo-wide scripts left at the top level
(`env.sh`, `bump-nixpkgs-release.sh`, plus `codex-setup.sh`/
`codex-maintenance.sh` above). When adding a new script, put it in the
matching subfolder rather than the top level, and if it duplicates logic
another script already has, lift the shared part into `scripts/lib/`
instead of copying it.
- `scripts/sync-host-keys.sh` — generates/registers SSH host keys and their ### `scripts/installer/`
`.sops.yaml`/`secrets/*.yaml` recipients for flake targets, idempotently
(`--all`, `<target>`, `--remove`, `--regenerate-all-keys`, all with - `scripts/installer/auto-install.sh` — the interactive install script
`--dry-run`). The primary tool for provisioning a new host's secrets baked into the auto-installer image (see `docs/auto-installer.md`), kept
access — see "Creating a new machine" in `docs/auto-installer.md`. as a real, version-controlled shell file rather than inline in
- `scripts/prepare-host-key.sh` — narrower predecessor: generates a key by `modules/installer/common.nix`'s Nix. It sources `scripts/env.sh` itself
an arbitrary name without touching `.sops.yaml`. Still useful to for `LAN_DOMAIN` (`export LAN_DOMAIN`/`: "${LAN_DOMAIN:=...}"`, matching
`variables.nix`'s `lanDomain` — manually kept in sync, same pattern as
`NIX_CACHE_HOST` mirroring `nixCacheHost`), rather than Nix-level string
substitution — that's what makes it work identically whether run
straight from a git checkout or from inside the built installer image.
`common.nix` bakes `scripts/env.sh` in alongside it at a matching
relative path (`/etc/nixos-installer/env.sh` next to
`/etc/nixos-installer/installer/auto-install.sh`) so the script's own
`source "$(dirname ...)/../env.sh"` line resolves the same way in both
contexts — this is also why it's invoked from
`/etc/nixos-installer/installer/auto-install.sh` rather than a flat
`/etc/auto-install.sh`. `#!/usr/bin/env bash`, not
`#!/run/current-system/sw/bin/bash`: the latter only resolves on an
already-activated NixOS system, breaking the checked-out-file case
entirely (confirmed live: "cannot execute: required file not found" on
a non-NixOS box); `/usr/bin/env` is reliably present on both NixOS
(`environment.usrbinenv`'s own default) and any normal Linux distro.
### `scripts/secrets/`
- `scripts/secrets/sync-host-keys.sh` — generates/registers SSH host keys
and their `.sops.yaml`/`secrets/*.yaml` recipients for flake targets,
idempotently (`--all`, `<target>`, `--remove`, `--regenerate-all-keys`,
all with `--dry-run`). The primary tool for provisioning a new host's
secrets access — see "Creating a new machine" in `docs/auto-installer.md`.
- `scripts/secrets/prepare-host-key.sh` — narrower predecessor: generates a
key by an arbitrary name without touching `.sops.yaml`. Still useful to
pre-generate a key before its flake target exists yet, since pre-generate a key before its flake target exists yet, since
`sync-host-keys.sh` can only act on targets `nixosConfigurations` already `sync-host-keys.sh` can only act on targets `nixosConfigurations` already
has. has.
- `scripts/create-proxmox-resource.sh` — builds a `lxc-*`/`proxmox-*` - `scripts/secrets/rotate-admin-key.sh <backup-admin-key> [--new-key-file
target's tarball/disk image and creates it on a real Proxmox node <path>] [--dry-run]` — rotates `.sops.yaml`'s `&admin` age key: decrypts
(`pct create` against the tarball as a CT template / `qm create`+ with a backed-up copy of the key currently trusted as `&admin` (verified
by deriving its public key and comparing, not taken on faith), replaces
the `&admin` line with a new key already present in the environment
(defaults to wherever sops/age itself would look), and runs
`sops updatekeys` on every `secrets/*.yaml`. One-way: the old key can no
longer decrypt anything re-encrypted this way. This is the automation
for the manual steps `sync-host-keys.sh`/`create-proxmox-resource.sh`
print when they bootstrap a brand-new, not-yet-trusted key on a machine
with no prior admin access.
- `scripts/secrets/backup-admin-key.sh <dest-path> [--key-file <path>]
[--force] [--dry-run]` — copies the local sops age key (source
resolution matches sops/age itself: `$SOPS_AGE_KEY` inline, then
`--key-file`, then `$SOPS_AGE_KEY_FILE`, then the XDG default) to an
arbitrary destination path with `0600` permissions, validating it's a
real age identity and round-tripping the public key before and after the
write. Refuses to overwrite an existing `<dest-path>` without `--force`.
Purely a local filesystem copy — never touches `.sops.yaml`/
`secrets/*.yaml` or the repo at all. The resulting file is exactly what
`rotate-admin-key.sh` expects as its backup-key argument.
- `scripts/secrets/sync-nix-cache-host-key.sh [--check] [--dry-run]
[--host <name>]` — detects drift between the ed25519 SSH host key
nix-cache is actually serving right now (via `ssh-keyscan`) and
`vars.nixCacheHostKey` (`variables.nix`), the value
`modules/nix-cache/remote-builder-client.nix` bakes into every real
client's declarative `programs.ssh.knownHosts` and
`configure-nix-cache-client.sh` hardcodes as its own default for
non-NixOS clients. That value has no automatic source of truth — it's
set once from whatever nix-cache's host key happened to be at the time,
and silently goes stale if the host is ever rebuilt/recreated with a new
key, breaking every client's distributed-build SSH trust with no error
that points back here. `--check` (used by `codex-maintenance.sh`, which
treats an unreachable nix-cache — e.g. from a non-LAN CI runner — as a
silent skip rather than a failure) only reports drift; the no-flags form
updates both files in place. Declarative clients still need a rebuild to
pick up the fix.
### `scripts/proxmox/`
- `scripts/proxmox/create-proxmox-resource.sh` — builds a `lxc-*`/
`proxmox-*` target's tarball/disk image and creates it on a real Proxmox
node (`pct create` against the tarball as a CT template / `qm create`+
`importdisk`), or reconfigures an existing resource's cores/memory/disk `importdisk`), or reconfigures an existing resource's cores/memory/disk
size (`--modify`, always requires typing the VMID back to confirm). size (`--modify`, always requires typing the VMID back to confirm).
Checks for an already-uploaded image on the node before building Checks for an already-uploaded image on the node before building
@@ -99,15 +264,12 @@ Beyond `codex-setup.sh`/`codex-maintenance.sh` above, `scripts/` also has:
unless `--allow-duplicate-host` is passed. `--dry-run` throughout both unless `--allow-duplicate-host` is passed. `--dry-run` throughout both
modes. The first time it has to bootstrap build tooling on a node (i.e. modes. The first time it has to bootstrap build tooling on a node (i.e.
`nix` wasn't already on its `PATH`), it also runs `nix` wasn't already on its `PATH`), it also runs
`scripts/configure-nix-cache-client.sh` there (non-fatally — a failure `scripts/proxmox/configure-nix-cache-client.sh` there (non-fatally — a
just falls back to building from source / `cache.nixos.org`) so the failure just falls back to building from source / `cache.nixos.org`) so
node substitutes from and can offload builds to nix-cache on every the node substitutes from and can offload builds to nix-cache on every
subsequent run, not just this one. subsequent run, not just this one.
- `scripts/env.sh` — shared config (`PROXMOX_HOST`, storage pool, bridge, - `scripts/proxmox/configure-nix-cache-client.sh [--dry-run]
default cores/memory) sourced by `create-proxmox-resource.sh`. Add new [--no-remote-builder] [--no-restart]` — the non-NixOS equivalent of
cross-script config here instead of duplicating it per-script.
- `scripts/configure-nix-cache-client.sh [--dry-run] [--no-remote-builder]
[--no-restart]` — the non-NixOS equivalent of
`modules/nix-cache/client.nix`/`remote-builder-client.nix`, for a plain `modules/nix-cache/client.nix`/`remote-builder-client.nix`, for a plain
Debian machine with the Nix package manager (not NixOS) already Debian machine with the Nix package manager (not NixOS) already
installed: run as root *on that machine* to add nix-cache as a installed: run as root *on that machine* to add nix-cache as a
@@ -120,32 +282,40 @@ Beyond `codex-setup.sh`/`codex-maintenance.sh` above, `scripts/` also has:
`/etc/ssh/ssh_known_hosts`. Idempotent (re-running replaces its own `/etc/ssh/ssh_known_hosts`. Idempotent (re-running replaces its own
marked block rather than duplicating it); restarts `nix-daemon` by marked block rather than duplicating it); restarts `nix-daemon` by
default so the change takes effect immediately. default so the change takes effect immediately.
### `scripts/lib/`
Sourced by the scripts above, never run directly:
- `nix-bootstrap.sh` — `NIX_CONFIG`/`ensure_nix_profile`, shared by
`codex-setup.sh`/`codex-maintenance.sh` and the remote build commands
`create-proxmox-resource.sh` runs over SSH.
- `nix-eval.sh` — `NIX_EVAL_FLAGS` plus `list_flake_targets`/
`flake_target_hostname` flake-introspection helpers.
- `ssh-host-keys.sh` — `generate_host_ed25519_key`/`ssh_pubkey_to_age`,
shared by `sync-host-keys.sh` and `prepare-host-key.sh`.
- `sops-age.sh` — `age_pubkey_from_identity_file`/`sops_yaml_admin_pubkey`/
`sops_updatekeys` plus the shared sops/age default key-file resolution,
shared by `backup-admin-key.sh`, `rotate-admin-key.sh`, and
`sync-host-keys.sh`.
- `confirm.sh` — `confirm_typed`, the "type X back to confirm" destructive-
action prompt shared by `create-proxmox-resource.sh` and
`sync-host-keys.sh`.
- `sync-host-keys-edit-sops.py` — the `.sops.yaml` anchor/key_groups editor
`sync-host-keys.sh` shells out to (see that script for why: precise,
idempotent YAML edits are impractical in bash).
### Top level
- `scripts/env.sh` — shared config (`PROXMOX_HOST`, storage pool, bridge,
default cores/memory, `NIX_CACHE_HOST`, `LAN_DOMAIN`) sourced by
`create-proxmox-resource.sh` and `scripts/installer/auto-install.sh`. Add
new cross-script config here instead of duplicating it per-script.
- `scripts/bump-nixpkgs-release.sh` — bumps `flake.nix`'s `nixpkgs.url`/ - `scripts/bump-nixpkgs-release.sh` — bumps `flake.nix`'s `nixpkgs.url`/
`home-manager.url` in place. Exists because flake input URLs can't `home-manager.url` in place. Exists because flake input URLs can't
reference `variables.nix` (confirmed empirically — `nix flake metadata` reference `variables.nix` (confirmed empirically — `nix flake metadata`
errors on it), so this is the closest equivalent to a single source of errors on it), so this is the closest equivalent to a single source of
truth for the tracked release. truth for the tracked release.
- `scripts/rotate-admin-key.sh <backup-admin-key> [--new-key-file <path>]
[--dry-run]` — rotates `.sops.yaml`'s `&admin` age key: decrypts with a
backed-up copy of the key currently trusted as `&admin` (verified by
deriving its public key and comparing, not taken on faith), replaces the
`&admin` line with a new key already present in the environment
(defaults to wherever sops/age itself would look), and runs
`sops updatekeys` on every `secrets/*.yaml`. One-way: the old key can no
longer decrypt anything re-encrypted this way. This is the automation
for the manual steps `sync-host-keys.sh`/`create-proxmox-resource.sh`
print when they bootstrap a brand-new, not-yet-trusted key on a machine
with no prior admin access.
- `scripts/backup-admin-key.sh <dest-path> [--key-file <path>] [--force]
[--dry-run]` — copies the local sops age key (source resolution matches
sops/age itself: `$SOPS_AGE_KEY` inline, then `--key-file`, then
`$SOPS_AGE_KEY_FILE`, then the XDG default) to an arbitrary destination
path with `0600` permissions, validating it's a real age identity and
round-tripping the public key before and after the write. Refuses to
overwrite an existing `<dest-path>` without `--force`. Purely a local
filesystem copy — never touches `.sops.yaml`/`secrets/*.yaml` or the
repo at all. The resulting file is exactly what `rotate-admin-key.sh`
expects as its backup-key argument.
`sync-host-keys.sh`, `create-proxmox-resource.sh`, and `sync-host-keys.sh`, `create-proxmox-resource.sh`, and
`rotate-admin-key.sh` genuinely mutate real state when run for real (not `rotate-admin-key.sh` genuinely mutate real state when run for real (not
@@ -178,11 +348,14 @@ nixosSystem {
} }
``` ```
Platforms: `linode`, `proxmox`, `lxc`. Build types: `minimal`, `nix-cache`, Platforms: `linode`, `proxmox`, `lxc`, `baremetal`. Build types: `minimal`,
`server`, `docker`, `gui`, `pxe-boot`, `tailscale-exit-node`, `tor-relay`. Not `nix-cache`, `server`, `docker`, `gui`, `pxe-boot`, `tailscale-exit-node`,
every combination is built — e.g. `pxe-boot` has no `linode` variant `tor-relay`. Not every combination is built — e.g. `pxe-boot` has no `linode`
(PXE/DHCP/TFTP need LAN L2 adjacency a Linode VPS doesn't have), and variant (PXE/DHCP/TFTP need LAN L2 adjacency a Linode VPS doesn't have),
`tor-relay` currently only exists as `lxc-tor-relay`. Treat `flake.nix`'s `tor-relay` currently only exists as `lxc-tor-relay`, and `baremetal`
currently only exists as `baremetal-gui` (the real gui-host hardware —
see `hosts/nixos/host.nix` and `modules/platforms/baremetal.nix`). Treat
`flake.nix`'s
`generatedTargets` as the source `generatedTargets` as the source
of truth for which hosts exist — `README.md`, `AGENTS.md`, of truth for which hosts exist — `README.md`, `AGENTS.md`,
`docs/flake-lock-automation.md`, and the CI eval workflows `docs/flake-lock-automation.md`, and the CI eval workflows
@@ -198,12 +371,16 @@ removing a host.
of their own beyond narrow parameterized helpers (see of their own beyond narrow parameterized helpers (see
`modules/beszel/host-token.nix` below) — all shared behavior comes from the `modules/beszel/host-token.nix` below) — all shared behavior comes from the
platform/build-type modules composed in `flake.nix`, not from the host file. platform/build-type modules composed in `flake.nix`, not from the host file.
- `modules/platforms/{linode,proxmox,lxc}.nix` — platform-specific config: - `modules/platforms/{linode,proxmox,lxc,baremetal}.nix` — platform-specific
boot method, guest tooling, and (for linode/proxmox) the hypervisor-specific config: boot method, guest tooling, and the hardware config, imported
hardware config, imported directly by the platform module itself directly by the platform module itself — **not** wired in from
(`../hardware-configuration/vm/{proxmox,linode}.nix`) — **not** wired in `flake.nix`. VM platforms use `../hardware-configuration/vm/{proxmox,linode}.nix`;
from `flake.nix`. `lxc.nix` has no hardware-configuration counterpart since `baremetal.nix` uses `../hardware-configuration/baremetal.nix` (adapted
containers share the host kernel; instead it imports nixpkgs' own from a real `nixos-generate-config` run on the actual hardware, not a
vm/ file, since it isn't a VM) plus `hardware.enableRedistributableFirmware
= true` for real wifi/GPU/microcode firmware that VMs never needed.
`lxc.nix` has no hardware-configuration counterpart since containers
share the host kernel; instead it imports nixpkgs' own
`virtualisation/proxmox-lxc.nix`, which gives every `lxc-*` host a `virtualisation/proxmox-lxc.nix`, which gives every `lxc-*` host a
`config.system.build.tarball` output — a plain rootfs tarball, used as a `config.system.build.tarball` output — a plain rootfs tarball, used as a
`pct create ... vztmpl` CT template (**not** `pct restore`, which expects `pct create ... vztmpl` CT template (**not** `pct restore`, which expects
@@ -227,6 +404,16 @@ removing a host.
boots, so this declares them with `destroy = false` (disko never wipes boots, so this declares them with `destroy = false` (disko never wipes
them) and a bare `filesystem`/`swap` content type instead of a partition them) and a bare `filesystem`/`swap` content type instead of a partition
table — idempotent against an already-provisioned disk, never destructive. table — idempotent against an already-provisioned disk, never destructive.
- `modules/disko/baremetal.nix` — `baremetal-gui`'s disko config: a ZFS
RAID0 (striped, no redundancy — disko's zpool `mode` defaults to `""`,
which is a plain stripe rather than `"mirror"`/`"raidz"`) root pool
across two disks, ESP + systemd-boot on the first. Device paths
(`vars.guiRootDisk1`/`guiRootDisk2`) are placeholders — fill in stable
`/dev/disk/by-id/...` paths before running disko for real.
`modules/platforms/baremetal.nix` also imports
`modules/services/zfs/enable-service.nix` for this (the `zfs_unstable`
package, autoScrub/autoSnapshot/trim) — the only other importer today is
`server`'s NFS data pool, an unrelated non-root ZFS use.
- `modules/boot/efi.nix` — systemd-boot + EFI vars, paired with the disko module. - `modules/boot/efi.nix` — systemd-boot + EFI vars, paired with the disko module.
- `modules/installer/` — the auto-installer environment (ISO, also served as - `modules/installer/` — the auto-installer environment (ISO, also served as
PXE netboot): `common.nix` (shared config + the generated PXE netboot): `common.nix` (shared config + the generated
+24 -7
View File
@@ -8,13 +8,15 @@ workstation.
Targets are named `<platform>-<buildtype>`, generated from two orthogonal Targets are named `<platform>-<buildtype>`, generated from two orthogonal
pieces composed in `flake.nix`: pieces composed in `flake.nix`:
- **Platforms** (what it runs on): `linode`, `proxmox`, `lxc` - **Platforms** (what it runs on): `linode`, `proxmox`, `lxc`, `baremetal`
- **Build types** (what it's for): `minimal`, `nix-cache`, `server`, `docker`, - **Build types** (what it's for): `minimal`, `nix-cache`, `server`, `docker`,
`gui`, `pxe-boot`, `tailscale-exit-node`, `tor-relay` `gui`, `pxe-boot`, `tailscale-exit-node`, `tor-relay`
Not every combination exists — `pxe-boot` has no `linode` variant, since Not every combination exists — `pxe-boot` has no `linode` variant, since
PXE/DHCP/TFTP need LAN L2 adjacency that a Linode VPS doesn't have, and PXE/DHCP/TFTP need LAN L2 adjacency that a Linode VPS doesn't have,
`tor-relay` currently only exists as `lxc-tor-relay`. The full list: `tor-relay` currently only exists as `lxc-tor-relay`, and `baremetal`
currently only exists as `baremetal-gui` (the real gui-host hardware). The
full list:
| Target | Purpose | | Target | Purpose |
| --- | --- | | --- | --- |
@@ -25,6 +27,7 @@ PXE/DHCP/TFTP need LAN L2 adjacency that a Linode VPS doesn't have, and
| `linode-server` / `proxmox-server` / `lxc-server` | Storage, NFS, backup, and monitoring exporter host — previously the flat `server` target | | `linode-server` / `proxmox-server` / `lxc-server` | Storage, NFS, backup, and monitoring exporter host — previously the flat `server` target |
| `linode-docker` / `proxmox-docker` / `lxc-docker` | Docker host for the main container stack — previously the flat `docker` target | | `linode-docker` / `proxmox-docker` / `lxc-docker` | Docker host for the main container stack — previously the flat `docker` target |
| `linode-gui` / `proxmox-gui` / `lxc-gui` | Cinnamon desktop workstation — previously the flat `nixos` target | | `linode-gui` / `proxmox-gui` / `lxc-gui` | Cinnamon desktop workstation — previously the flat `nixos` target |
| `baremetal-gui` | Same Cinnamon desktop workstation, on the real gui-host hardware — ZFS RAID0 root, systemd-boot |
| `proxmox-pxe-boot` / `lxc-pxe-boot` | HTTP/iPXE boot asset host — previously the flat `pxe-boot` target | | `proxmox-pxe-boot` / `lxc-pxe-boot` | HTTP/iPXE boot asset host — previously the flat `pxe-boot` target |
| `linode-tailscale-exit-node` / `proxmox-tailscale-exit-node` / `lxc-tailscale-exit-node` | Tailscale exit node | | `linode-tailscale-exit-node` / `proxmox-tailscale-exit-node` / `lxc-tailscale-exit-node` | Tailscale exit node |
| `lxc-tor-relay` | Tor middle relay | | `lxc-tor-relay` | Tor middle relay |
@@ -34,8 +37,13 @@ anywhere in this repo — that's live infrastructure state, not something a
committed file can keep accurate, and it changes independently of the code. committed file can keep accurate, and it changes independently of the code.
Check the Proxmox node itself, or `/etc/flake-target` on a running host (see Check the Proxmox node itself, or `/etc/flake-target` on a running host (see
below), if you need to know what's really out there right now. below), if you need to know what's really out there right now.
`scripts/create-proxmox-resource.sh`'s duplicate-host guard works the same `scripts/proxmox/create-proxmox-resource.sh`'s duplicate-host guard works the same
way: it checks the Proxmox node directly rather than any file here. way: it checks the Proxmox node directly rather than any file here.
Real, production deployments live on `pve1.sweet.home`; there's a second
node, `pve-test.sweet.home`, set aside purely for scratch/test resources —
see `scripts/env.sh` (`PVE1_HOST` / `PVE_TEST_HOST`, and the
`--node`/`PROXMOX_HOST` targeting they feed into) and CLAUDE.md's Proxmox
section for which is which.
Each buildtype's `hosts/<name>/host.nix` carries the per-machine identity Each buildtype's `hosts/<name>/host.nix` carries the per-machine identity
(hostname, hostId, per-machine secrets, `system.stateVersion`) that must stay (hostname, hostId, per-machine secrets, `system.stateVersion`) that must stay
@@ -59,7 +67,7 @@ nix eval --json .#nixosConfigurations --apply builtins.attrNames | jq -r '.[]'
| `variables.nix` | Single source of truth for shared values (LAN domain/CIDR, hostnames, timezone, primary username, storage root, NFS share subpaths/mountpoints, service ports, ...) — passed to every module and Home Manager config as the `vars` argument via `specialArgs`/`extraSpecialArgs` | | `variables.nix` | Single source of truth for shared values (LAN domain/CIDR, hostnames, timezone, primary username, storage root, NFS share subpaths/mountpoints, service ports, ...) — passed to every module and Home Manager config as the `vars` argument via `specialArgs`/`extraSpecialArgs` |
| `hosts/<name>/host.nix` | Per-machine identity: hostname, hostId, per-machine secrets, `system.stateVersion` | | `hosts/<name>/host.nix` | Per-machine identity: hostname, hostId, per-machine secrets, `system.stateVersion` |
| `hosts/nixos/home.nix` | Workstation-specific Home Manager config (used by the `gui` build type) | | `hosts/nixos/home.nix` | Workstation-specific Home Manager config (used by the `gui` build type) |
| `modules/platforms/` | Platform-specific config: virtualisation guest tools, boot method, hardware config (`linode.nix`, `proxmox.nix`, `lxc.nix`) | | `modules/platforms/` | Platform-specific config: virtualisation guest tools, boot method, hardware config (`linode.nix`, `proxmox.nix`, `lxc.nix`, `baremetal.nix`) |
| `modules/build-types/` | Build-type-specific config: what makes a system minimal/server/docker/gui/pxe-boot/nix-cache | | `modules/build-types/` | Build-type-specific config: what makes a system minimal/server/docker/gui/pxe-boot/nix-cache |
| `modules/common/` | Shared NixOS config, Home Manager, aliases imported by every host | | `modules/common/` | Shared NixOS config, Home Manager, aliases imported by every host |
| `modules/nix-cache/` | Binary cache and remote builder client/server modules | | `modules/nix-cache/` | Binary cache and remote builder client/server modules |
@@ -74,10 +82,19 @@ Safe validation commands for Codex and local review:
```bash ```bash
bash scripts/codex-setup.sh bash scripts/codex-setup.sh
bash scripts/codex-maintenance.sh dry-run
bash scripts/codex-maintenance.sh bash scripts/codex-maintenance.sh
``` ```
`codex-maintenance.sh` with no flags (what CI runs on every push/PR) scopes
fmt-check/statix/eval to files changed against a base ref — fast, but only
as thorough as the diff. For the full sweep (every host, every package,
fmt-check and statix over the whole tree — slow, CI never runs this):
```bash
bash scripts/codex-maintenance.sh --full-check
bash scripts/codex-maintenance.sh --full-check --dry-run
```
For individual host evaluation: For individual host evaluation:
```bash ```bash
@@ -114,7 +131,7 @@ Three different paths depending on target, none of them involving a manual
disk image and attached to a new VM with no install step — see disk image and attached to a new VM with no install step — see
`docs/proxmox-images.md`. `docs/proxmox-images.md`.
`scripts/create-proxmox-resource.sh --type lxc|vm --host <name>` automates `scripts/proxmox/create-proxmox-resource.sh --type lxc|vm --host <name>` automates
either of the last two end to end (host-key registration, building the either of the last two end to end (host-key registration, building the
image directly on the Proxmox node itself, `pct create`/`qm create`), with image directly on the Proxmox node itself, `pct create`/`qm create`), with
`--dry-run` and a guard against duplicating an already-deployed host's `--dry-run` and a guard against duplicating an already-deployed host's
+37 -12
View File
@@ -8,10 +8,13 @@ lives here.
The installer provides a small NixOS install environment (ISO, or the same The installer provides a small NixOS install environment (ISO, or the same
image netbooted via PXE) with SSH access, Git support, and an interactive image netbooted via PXE) with SSH access, Git support, and an interactive
installation script. installation script.
Logging in as any user (root or `nixos`) runs `/etc/auto-install.sh`, Logging in as any user (root or `nixos`) runs
discovers available hosts from this same flake, lets the operator choose a `/etc/nixos-installer/installer/auto-install.sh` (the same file as
target, applies that host's Disko storage configuration, installs NixOS, and `scripts/installer/auto-install.sh` in this repo — see "Installer process"
reboots. below for why it's baked in at that path rather than a flat
`/etc/auto-install.sh`), discovers available hosts from this same flake,
lets the operator choose a target, applies that host's Disko storage
configuration, installs NixOS, and reboots.
**This applies to every `nixosConfigurations` target except `lxc-*` hosts — **This applies to every `nixosConfigurations` target except `lxc-*` hosts —
see "LXC hosts" immediately below for why those are different.** see "LXC hosts" immediately below for why those are different.**
@@ -73,7 +76,7 @@ booting one:
First boot runs `boot.postBootCommands` (registers the Nix store DB and First boot runs `boot.postBootCommands` (registers the Nix store DB and
system profile) — there's no separate activation step to run yourself. system profile) — there's no separate activation step to run yourself.
`scripts/create-proxmox-resource.sh --type lxc --host <name>` automates all `scripts/proxmox/create-proxmox-resource.sh --type lxc --host <name>` automates all
of this (host-key handling, building the tarball directly on the Proxmox of this (host-key handling, building the tarball directly on the Proxmox
node itself, `pct create` with the flags above) — see its `--help`. node itself, `pct create` with the flags above) — see its `--help`.
@@ -102,7 +105,7 @@ groups required, got 0`, and *every* secret (including this host's own
login) permanently fails to decrypt, silently — no error in the boot log login) permanently fails to decrypt, silently — no error in the boot log
at all, since the activation step that would install secrets only runs on at all, since the activation step that would install secrets only runs on
a from-scratch first activation and skips silently once `/run/current-system` a from-scratch first activation and skips silently once `/run/current-system`
already exists. `scripts/create-proxmox-resource.sh` always builds with already exists. `scripts/proxmox/create-proxmox-resource.sh` always builds with
`NIXOS_HOST_KEYS_DIR` set for this reason. `NIXOS_HOST_KEYS_DIR` set for this reason.
## Layout ## Layout
@@ -116,10 +119,10 @@ already exists. `scripts/create-proxmox-resource.sh` always builds with
`docs/pxe-boot.md`). `docs/pxe-boot.md`).
- `modules/installer/host-keys.nix` — optionally bakes pre-generated SSH - `modules/installer/host-keys.nix` — optionally bakes pre-generated SSH
host keys into the image; see "Host keys" below. host keys into the image; see "Host keys" below.
- `scripts/sync-host-keys.sh` — admin-workstation tool that generates, - `scripts/secrets/sync-host-keys.sh` — admin-workstation tool that generates,
registers, and (via `--remove`/`--regenerate-all-keys`) retires host registers, and (via `--remove`/`--regenerate-all-keys`) retires host
keys; see "Creating a New Machine" below. keys; see "Creating a New Machine" below.
- `scripts/prepare-host-key.sh` — narrower predecessor: generates a single - `scripts/secrets/prepare-host-key.sh` — narrower predecessor: generates a single
key by an arbitrary name without touching `.sops.yaml`. Still useful for key by an arbitrary name without touching `.sops.yaml`. Still useful for
pre-generating a key *before* its flake target exists (`sync-host-keys.sh` pre-generating a key *before* its flake target exists (`sync-host-keys.sh`
can only act on targets `nixosConfigurations` already has); otherwise can only act on targets `nixosConfigurations` already has); otherwise
@@ -150,7 +153,11 @@ use case.
The `pxe` variant is also built automatically as part of the `pxe-boot` host The `pxe` variant is also built automatically as part of the `pxe-boot` host
itself (`modules/pxe-boot/stage-installer-artifacts.nix`) and served over itself (`modules/pxe-boot/stage-installer-artifacts.nix`) and served over
iPXE — see `docs/pxe-boot.md`. iPXE as the menu's "NixOS Auto-Installer" entry — see `docs/pxe-boot.md`.
That same host also builds and serves `packages.x86_64-linux.pxe-minimal`,
a vanilla NixOS minimal netboot image with none of this auto-installer's
wiring, as a separate "NixOS Minimal" menu entry — also documented in
`docs/pxe-boot.md`, not covered further here since it's not this installer.
## Host keys ## Host keys
@@ -188,6 +195,10 @@ default.
`auto-install.sh` still supports the older manual path as a fallback: if a `auto-install.sh` still supports the older manual path as a fallback: if a
host's key isn't baked in (`/etc/host-keys`), it checks `/root/host-keys` host's key isn't baked in (`/etc/host-keys`), it checks `/root/host-keys`
next, where you can `scp` a key in after boot, same as before this migration. next, where you can `scp` a key in after boot, same as before this migration.
If neither has it and the script is running interactively (an actual
operator at the other end of stdin, not an unattended run), it prompts for
an arbitrary directory to check (a mounted USB stick, another filesystem,
etc.) and copies the key pair into `/root/host-keys` from there if found.
## Storage ## Storage
@@ -213,7 +224,21 @@ entirely (see "LXC hosts" above), so it never reaches this code path.
## Installer process ## Installer process
`/etc/auto-install.sh`: `scripts/installer/auto-install.sh` is a real, version-controlled shell
script — not an inline Nix string. It sources `scripts/env.sh` for
`LAN_DOMAIN` itself (same as every other script in `scripts/`), so it
behaves identically whether it's run straight from a git checkout (e.g.
manually, from a stock NixOS ISO that isn't this repo's own installer
image) or from inside the built installer image. That's also why it's
baked in at `/etc/nixos-installer/installer/auto-install.sh` rather than a
flat `/etc/auto-install.sh``modules/installer/common.nix` bakes
`scripts/env.sh` in alongside it at `/etc/nixos-installer/env.sh`,
preserving the same relative layout (`installer/auto-install.sh` ->
`../env.sh`) the checked-out repo has, so the script's own
`source ".../env.sh"` line resolves correctly in both places without any
Nix-level templating.
Once running, it:
1. Queries `nixosConfigurations` from this flake over the network (`git+https://<lanDomain>/beatzaplenty/nixos.git`) — this happens at *install* time, not build time, so a generic installer image always sees whatever hosts are currently committed, without needing a rebuild. 1. Queries `nixosConfigurations` from this flake over the network (`git+https://<lanDomain>/beatzaplenty/nixos.git`) — this happens at *install* time, not build time, so a generic installer image always sees whatever hosts are currently committed, without needing a rebuild.
2. Presents them as a menu; confirms the choice. 2. Presents them as a menu; confirms the choice.
@@ -238,7 +263,7 @@ GitHub token behind sops-nix for all of them).
2. **On your admin workstation, generate and register its host key:** 2. **On your admin workstation, generate and register its host key:**
```sh ```sh
./scripts/sync-host-keys.sh <flake-target> ./scripts/secrets/sync-host-keys.sh <flake-target>
``` ```
This generates `host-keys/<flake-target>_ssh_host_ed25519_key(.pub)`, This generates `host-keys/<flake-target>_ssh_host_ed25519_key(.pub)`,
@@ -250,7 +275,7 @@ GitHub token behind sops-nix for all of them).
Doing this for every host that needs one at once — after adding several Doing this for every host that needs one at once — after adding several
new targets, or just to catch up any that were missed — is new targets, or just to catch up any that were missed — is
`./scripts/sync-host-keys.sh --all`. See `scripts/sync-host-keys.sh --help` `./scripts/secrets/sync-host-keys.sh --all`. See `scripts/secrets/sync-host-keys.sh --help`
for its other modes (`--remove`, `--regenerate-all-keys`). for its other modes (`--remove`, `--regenerate-all-keys`).
3. **Commit and push.** The flake build the installer uses has to see the 3. **Commit and push.** The flake build the installer uses has to see the
+8 -3
View File
@@ -8,9 +8,14 @@ and to verify that declared NixOS hosts still evaluate after dependency updates.
- A scheduled workflow runs `nix flake update` once per week. - A scheduled workflow runs `nix flake update` once per week.
- On GitHub, any resulting `flake.lock` change is proposed through a pull request. - On GitHub, any resulting `flake.lock` change is proposed through a pull request.
- On Gitea, the workflow can commit and push `flake.lock` directly when PR automation is not configured. - On Gitea, the workflow can commit and push `flake.lock` directly when PR automation is not configured.
- A separate CI workflow evaluates every configured host before merge, listed - A separate CI workflow runs `scripts/codex-maintenance.sh` before merge.
dynamically via `nix eval --json .#nixosConfigurations --apply builtins.attrNames` Its default mode scopes eval to the hosts/packages a change can affect,
rather than hand-enumerated, so it can't drift as `<platform>-<buildtype>` determined from a git diff against the PR base — but a `flake.lock` change
is treated as repo-wide and always falls back to evaluating every host, so
a lock-file update PR still gets full coverage. Hosts are still listed
dynamically via
`nix eval --json .#nixosConfigurations --apply builtins.attrNames` rather
than hand-enumerated, so that fallback can't drift as `<platform>-<buildtype>`
targets are added or removed. See `README.md` for the current target list. targets are added or removed. See `README.md` for the current target list.
## Why hosts should stop using `--upgrade-all` ## Why hosts should stop using `--upgrade-all`
+17 -9
View File
@@ -46,18 +46,26 @@ the new key up automatically on next activation — no more manual
## Remote builder SSH keys ## Remote builder SSH keys
On each client, install the private key used to authenticate as `nixremote`: Each client authenticates as `nixremote` using its **own default root SSH
identity** (`/root/.ssh/id_ed25519`) — not a separately-named or shared
keypair. If a client doesn't have one yet:
```bash ```bash
sudo install -d -m 0700 /root/.ssh sudo ssh-keygen -t ed25519 -N '' -f /root/.ssh/id_ed25519
sudo install -m 0600 ./nixremote /root/.ssh/nixremote
sudo ssh -i /root/.ssh/nixremote nixremote@nix-cache nix-store --version
``` ```
On `nix-cache`, install the matching public key used by `nixremote` authorized keys. Then add its `.pub` contents as a new entry in `vars.remoteBuilderAuthorizedKeys`
(`variables.nix`) and rebuild `nix-cache` to pick it up (that list is
declarative — an imperative `ssh-copy-id nixremote@nix-cache` won't stick;
it gets overwritten on every rebuild). Verify with:
The committed `nixremote` authorized keys are public SSH keys only. Keep the ```bash
matching private keys on client hosts and out of the repository. sudo ssh -i /root/.ssh/id_ed25519 nixremote@nix-cache nix-store --version
```
The committed `remoteBuilderAuthorizedKeys` entries are public SSH keys
only. Keep the matching private keys on client hosts and out of the
repository.
nix-cache's own SSH *host* key is trusted declaratively via nix-cache's own SSH *host* key is trusted declaratively via
`programs.ssh.knownHosts` in `modules/nix-cache/remote-builder-client.nix`, `programs.ssh.knownHosts` in `modules/nix-cache/remote-builder-client.nix`,
@@ -76,8 +84,8 @@ After deployment:
curl http://nix-cache/nix-cache-info curl http://nix-cache/nix-cache-info
nix store ping --store http://nix-cache nix store ping --store http://nix-cache
nix show-config | grep -E 'substituters|trusted-public-keys|builders-use-substitutes' nix show-config | grep -E 'substituters|trusted-public-keys|builders-use-substitutes'
sudo ssh -i /root/.ssh/nixremote nixremote@nix-cache nix-store --version sudo ssh -i /root/.ssh/id_ed25519 nixremote@nix-cache nix-store --version
nix build nixpkgs#hello --builders 'ssh://nixremote@nix-cache x86_64-linux /root/.ssh/nixremote 4 2 big-parallel,kvm,nixos-test,benchmark' -L nix build nixpkgs#hello --builders 'ssh://nixremote@nix-cache x86_64-linux /root/.ssh/id_ed25519 4 2 big-parallel,kvm,nixos-test,benchmark' -L
nix path-info -r nixpkgs#hello nix path-info -r nixpkgs#hello
curl -I "http://nix-cache/$(basename "$(nix path-info nixpkgs#hello)").narinfo" curl -I "http://nix-cache/$(basename "$(nix path-info nixpkgs#hello)").narinfo"
``` ```
+2 -2
View File
@@ -7,7 +7,7 @@ config (`modules/disko/proxmox.nix`) already used to format a real disk on
install, so there's nothing host-specific to write; it's available for every install, so there's nothing host-specific to write; it's available for every
`proxmox-*` target automatically. `proxmox-*` target automatically.
`scripts/create-proxmox-resource.sh --type vm --host <name>` automates the `scripts/proxmox/create-proxmox-resource.sh --type vm --host <name>` automates the
whole walkthrough below (and the equivalent LXC one) end to end, including whole walkthrough below (and the equivalent LXC one) end to end, including
host-key handling and building the image directly on the Proxmox node host-key handling and building the image directly on the Proxmox node
itself (no local build, no image transfer) — see its `--help`. The steps itself (no local build, no image transfer) — see its `--help`. The steps
@@ -51,7 +51,7 @@ sudo ./result \
--build-memory 2048 --build-memory 2048
``` ```
Generate the key first with `scripts/sync-host-keys.sh <hostname>`, same Generate the key first with `scripts/secrets/sync-host-keys.sh <hostname>`, same
as any other host — see `docs/auto-installer.md` for the full walkthrough as any other host — see `docs/auto-installer.md` for the full walkthrough
(it registers the new key in `.sops.yaml` and re-encrypts the affected (it registers the new key in `.sops.yaml` and re-encrypts the affected
`secrets/*.yaml` files too, no manual editing needed). `secrets/*.yaml` files too, no manual editing needed).
+35 -16
View File
@@ -1,9 +1,10 @@
# pxe-boot # pxe-boot
The `pxe-boot` host serves HTTP boot assets for iPXE clients — including a The `pxe-boot` host serves HTTP boot assets for iPXE clients — including
self-staged copy of this flake's own auto-installer netboot image, see self-staged copies of both this flake's own auto-installer netboot image
`docs/auto-installer.md` for what that image actually is and does once (see `docs/auto-installer.md` for what that image actually is and does once
booted. booted) and a vanilla, unmodified NixOS minimal netboot image for plain
rescue/inspection use.
## Host Role ## Host Role
@@ -28,7 +29,8 @@ The host creates these directories with systemd tmpfiles:
/srv/pxe /srv/pxe
/srv/pxe/http /srv/pxe/http
/srv/pxe/http/images /srv/pxe/http/images
/srv/pxe/http/nixos /srv/pxe/http/auto-installer
/srv/pxe/http/nixos-minimal
/srv/pxe/http/systemrescue /srv/pxe/http/systemrescue
/srv/pxe/http/ubuntu /srv/pxe/http/ubuntu
/srv/pxe/http/rescue /srv/pxe/http/rescue
@@ -37,7 +39,7 @@ The host creates these directories with systemd tmpfiles:
Mount shared image storage under `/srv/pxe/http`, preferably Mount shared image storage under `/srv/pxe/http`, preferably
`/srv/pxe/http/images` unless a menu entry expects files in a specific `/srv/pxe/http/images` unless a menu entry expects files in a specific
directory such as `/srv/pxe/http/nixos`. directory such as `/srv/pxe/http/auto-installer`.
The HTTP iPXE chain is: The HTTP iPXE chain is:
@@ -50,20 +52,37 @@ undionly.kpxe or ipxe.efi
The generated menu currently exposes entries for: The generated menu currently exposes entries for:
- NixOS installer - NixOS Auto-Installer
- NixOS Minimal
- SystemRescue environment - SystemRescue environment
- iPXE shell - iPXE shell
- Reboot - Reboot
The NixOS installer entry chain-loads `/srv/pxe/http/nixos/netboot.ipxe`, Both NixOS entries chain-load a `netboot.ipxe` staged into their own
which is nixpkgs' own generated netboot iPXE script (correct `init=`/`initrd=` directory (`/srv/pxe/http/auto-installer/netboot.ipxe` and
kernel parameters included) rather than a hand-rolled boot line — that script `/srv/pxe/http/nixos-minimal/netboot.ipxe`), each nixpkgs' own generated
in turn expects its kernel/initrd siblings in the same directory. All three netboot iPXE script (correct `init=`/`initrd=` kernel parameters included)
files (`bzImage`, `initrd`, `netboot.ipxe`) are built from this flake's own rather than a hand-rolled boot line — that script in turn expects its
`modules/installer/iso.nix` netboot image (the same one `nix build .#pxe` kernel/initrd siblings in the same directory. Each directory's three files
produces) and staged automatically by (`bzImage`, `initrd`, `netboot.ipxe`) are built from source and staged
`modules/pxe-boot/stage-installer-artifacts.nix` via `systemd.tmpfiles.rules` automatically by `modules/pxe-boot/stage-installer-artifacts.nix` via
— no manual operator step required. `systemd.tmpfiles.rules` — no manual operator step required:
- `auto-installer` is this flake's own `netbootSystem` (`flake.nix`) — the
same auto-installer image `nix build .#pxe` produces. See
`docs/auto-installer.md`.
- `nixos-minimal` is `netbootMinimalSystem` (`flake.nix`) — nixpkgs'
`netboot-minimal.nix` composed on its own, with none of this flake's
auto-installer wiring (no `common.nix`, no `auto-install.sh`, no baked
host keys or custom users). Same `nix build .#pxe-minimal` mechanism as
the auto-installer image, just a different module composition. Useful
as a plain rescue/inspection shell that doesn't assume anything about
this flake.
Both images set `networking.hostName` to match their menu entry/staged
directory name (`auto-installer` / `nixos-minimal`), so each one's
generated system name (`nixos-system-<name>-*`) is self-describing rather
than the nixpkgs default of `nixos-system-nixos-*` for both.
The SystemRescue entry expects the source ISO at: The SystemRescue entry expects the source ISO at:
Generated
+6 -6
View File
@@ -95,11 +95,11 @@
] ]
}, },
"locked": { "locked": {
"lastModified": 1783740085, "lastModified": 1784350909,
"narHash": "sha256-qajyHfZY29G2oEQk+uHxmsJcRoBUBXP9maTpFlwP/dI=", "narHash": "sha256-ZWyzLbS1yKUTeFJLmdVuWNnHttL333/ldJbEE+KzCrM=",
"owner": "nix-community", "owner": "nix-community",
"repo": "home-manager", "repo": "home-manager",
"rev": "3cd22efe6471dc7365c822bd9ad73a21e55f38fb", "rev": "4ce190229c73d44536caa7072f6308fb2d8feeb3",
"type": "github" "type": "github"
}, },
"original": { "original": {
@@ -147,11 +147,11 @@
}, },
"nixpkgs_2": { "nixpkgs_2": {
"locked": { "locked": {
"lastModified": 1784011430, "lastModified": 1784432872,
"narHash": "sha256-lDebytrYdd47IBLwvNOD+6AGeoqZ78CIKlp70hzW280=", "narHash": "sha256-n3gKTBIV4ZA5VQpUakffBe3KGu4+mhPoA34rrqS0GkA=",
"owner": "NixOS", "owner": "NixOS",
"repo": "nixpkgs", "repo": "nixpkgs",
"rev": "8eeec934ae0dbeca3d7868c059568a65c08b2fc3", "rev": "fd1462031fdee08f65fd0b4c6b64e22239a77870",
"type": "github" "type": "github"
}, },
"original": { "original": {
+54 -2
View File
@@ -65,7 +65,7 @@
# file without a same-option circular dependency (a module # file without a same-option circular dependency (a module
# contributing to environment.etc can't read the merged # contributing to environment.etc can't read the merged
# environment.etc it's itself contributing to). # environment.etc it's itself contributing to).
specialArgs = { inherit inputs vars netbootSystem flakeTarget; }; specialArgs = { inherit inputs vars netbootSystem netbootMinimalSystem flakeTarget; };
}; };
# Generated platform x build-type matrix. pxe-boot has no linode # Generated platform x build-type matrix. pxe-boot has no linode
@@ -91,6 +91,7 @@
linode-gui = mkTarget { platform = "linode"; buildType = "gui"; hostPath = ./hosts/nixos/host.nix; homeFile = ./hosts/nixos/home.nix; }; linode-gui = mkTarget { platform = "linode"; buildType = "gui"; hostPath = ./hosts/nixos/host.nix; homeFile = ./hosts/nixos/home.nix; };
proxmox-gui = mkTarget { platform = "proxmox"; buildType = "gui"; hostPath = ./hosts/nixos/host.nix; homeFile = ./hosts/nixos/home.nix; }; proxmox-gui = mkTarget { platform = "proxmox"; buildType = "gui"; hostPath = ./hosts/nixos/host.nix; homeFile = ./hosts/nixos/home.nix; };
lxc-gui = mkTarget { platform = "lxc"; buildType = "gui"; hostPath = ./hosts/nixos/host.nix; homeFile = ./hosts/nixos/home.nix; }; lxc-gui = mkTarget { platform = "lxc"; buildType = "gui"; hostPath = ./hosts/nixos/host.nix; homeFile = ./hosts/nixos/home.nix; };
baremetal-gui = mkTarget { platform = "baremetal"; buildType = "gui"; hostPath = ./hosts/nixos/host.nix; homeFile = ./hosts/nixos/home.nix; };
proxmox-pxe-boot = mkTarget { platform = "proxmox"; buildType = "pxe-boot"; hostPath = ./hosts/pxe-boot/host.nix; }; proxmox-pxe-boot = mkTarget { platform = "proxmox"; buildType = "pxe-boot"; hostPath = ./hosts/pxe-boot/host.nix; };
lxc-pxe-boot = mkTarget { platform = "lxc"; buildType = "pxe-boot"; hostPath = ./hosts/pxe-boot/host.nix; }; lxc-pxe-boot = mkTarget { platform = "lxc"; buildType = "pxe-boot"; hostPath = ./hosts/pxe-boot/host.nix; };
@@ -119,19 +120,61 @@
# Same installer environment, built as netboot (kernel + initrd + # Same installer environment, built as netboot (kernel + initrd +
# iPXE script) instead of an ISO — this is what packages.pxe bundles. # iPXE script) instead of an ISO — this is what packages.pxe bundles.
#
# Deliberately imports common.nix directly, NOT ./modules/installer/iso.nix
# (which pulls in nixpkgs' installation-cd-minimal.nix) -- confirmed live
# that composing the ISO module together with netboot-minimal.nix hangs
# every boot waiting for a device that can never exist on a netboot
# client ("A start job is running for /dev/disk/by-label/nixos-minimal-...").
# Both installation-cd-base.nix and netboot.nix set fileSystems."/" via
# the identical lib.mkImageMediaOverride (mkOverride 60) priority --
# genuinely conflicting root-filesystem strategies (ISO-by-label vs.
# netboot-tmpfs) at the same priority, and the ISO one was winning.
# netboot-minimal.nix's own chain (netboot-base.nix) already imports
# profiles/installation-device.nix independently, so common.nix's
# initialHashedPassword override (which assumes that profile is
# present) still applies correctly without iso.nix in the mix.
#
# networking.hostName is set explicitly (rather than left at nixpkgs'
# own "nixos" default) so this image's generated system name
# (nixos-system-auto-installer-*) matches its iPXE menu entry —
# see modules/build-types/pxe-boot.nix's :auto-installer item — and
# its staged directory, /srv/pxe/http/auto-installer.
netbootSystem = nixpkgs.lib.nixosSystem { netbootSystem = nixpkgs.lib.nixosSystem {
inherit system; inherit system;
modules = [ modules = [
./modules/installer/iso.nix ./modules/installer/common.nix
({ modulesPath, ... }: { ({ modulesPath, ... }: {
imports = [ imports = [
(modulesPath + "/installer/netboot/netboot-minimal.nix") (modulesPath + "/installer/netboot/netboot-minimal.nix")
]; ];
}) })
{ networking.hostName = "auto-installer"; }
]; ];
specialArgs = { inherit vars; }; specialArgs = { inherit vars; };
}; };
# A genuinely vanilla NixOS minimal netboot image: nixpkgs'
# netboot-minimal.nix on its own, with none of this flake's
# auto-installer wiring (no common.nix — no auto-install.sh, no
# baked host keys, no custom users/passwords). Built from source via
# the same nixosSystem + netboot-minimal.nix path as netbootSystem
# above, so both go through an identical build mechanism; the only
# difference is what's composed in. hostName again matches this
# image's iPXE menu entry (:nixos-minimal) and staged directory
# (/srv/pxe/http/nixos-minimal).
netbootMinimalSystem = nixpkgs.lib.nixosSystem {
inherit system;
modules = [
({ modulesPath, ... }: {
imports = [
(modulesPath + "/installer/netboot/netboot-minimal.nix")
];
})
{ networking.hostName = "nixos-minimal"; }
];
};
in in
{ {
@@ -153,6 +196,15 @@
{ name = "initrd"; path = netbootSystem.config.system.build.netbootRamdisk; } { name = "initrd"; path = netbootSystem.config.system.build.netbootRamdisk; }
{ name = "kernel"; path = netbootSystem.config.system.build.kernel; } { name = "kernel"; path = netbootSystem.config.system.build.kernel; }
]; ];
# Vanilla NixOS minimal netboot bundle — see netbootMinimalSystem
# above. Staged onto the pxe-boot host alongside packages.pxe by
# modules/pxe-boot/stage-installer-artifacts.nix.
pxe-minimal = pkgs.linkFarm "pxe-minimal" [
{ name = "netboot.ipxe"; path = netbootMinimalSystem.config.system.build.netbootIpxeScript; }
{ name = "initrd"; path = netbootMinimalSystem.config.system.build.netbootRamdisk; }
{ name = "kernel"; path = netbootMinimalSystem.config.system.build.kernel; }
];
}; };
}; };
} }
+4
View File
@@ -19,11 +19,15 @@
nextcloud-client nextcloud-client
# vscode # vscode
chromium chromium
claude-code
fish
sops
]; ];
# Optional: set environment vars # Optional: set environment vars
sessionVariables = { sessionVariables = {
EDITOR = "vim"; EDITOR = "vim";
SOPS_AGE_KEY_FILE = "/home/nixos/Nextcloud/Filing Cabinet/keys/nixos-sops-age-key-txt";
}; };
file = { file = {
+9
View File
@@ -1,8 +1,17 @@
_: _:
{ {
imports = [
../../modules/networking/wifi.nix
];
networking.hostName = "nixos"; networking.hostName = "nixos";
# Only needed now that baremetal-gui exists (ZFS root) -- harmless on the
# ext4-rooted linode/proxmox/lxc-gui variants, so set unconditionally
# rather than only on the baremetal platform.
networking.hostId = "de6a9ffc";
# Preserved from the pre-refactor `nixos` target — stateVersion must never # Preserved from the pre-refactor `nixos` target — stateVersion must never
# be bumped on an already-installed machine. # be bumped on an already-installed machine.
system.stateVersion = "25.05"; system.stateVersion = "25.05";
+12 -1
View File
@@ -1,11 +1,22 @@
_: { ... }:
{ {
imports = [
(import ../../modules/beszel/host-token.nix {
name = "tor-relay";
sopsFile = ../../secrets/tor-relay.yaml;
})
];
networking.hostName = "tor-relay"; networking.hostName = "tor-relay";
# No networking.hostId: only ZFS-touching hosts (server, docker) need one # No networking.hostId: only ZFS-touching hosts (server, docker) need one
# for pool-import safety, and this host does neither. # for pool-import safety, and this host does neither.
services.beszel.agent.environment = {
KEY = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFPR9kwtC4TAeTRu46A7+opZsYpxqkRJ+x/ZyB2GWCeG";
};
# A genuinely new host (not a pre-refactor carry-over), so it tracks the # A genuinely new host (not a pre-refactor carry-over), so it tracks the
# flake's current nixpkgs release rather than being pinned to an older one. # flake's current nixpkgs release rather than being pinned to an older one.
system.stateVersion = "26.05"; system.stateVersion = "26.05";
+1 -1
View File
@@ -18,7 +18,7 @@
]; ];
boot.loader.grub.useOSProber = true; boot.loader.grub.useOSProber = true;
programs.direnv.enable = true;
services = { services = {
xserver = { xserver = {
enable = true; enable = true;
+9 -4
View File
@@ -68,15 +68,19 @@ let
set base ${pxeBaseUrl} set base ${pxeBaseUrl}
menu PXE Boot Menu menu PXE Boot Menu
item nixos NixOS Installer item auto-installer NixOS Auto-Installer
item nixos-minimal NixOS Minimal
item rescue Rescue Environment item rescue Rescue Environment
item shell iPXE Shell item shell iPXE Shell
item reboot Reboot item reboot Reboot
choose target && goto ''${target} choose target && goto ''${target}
:nixos :auto-installer
chain ''${base}/nixos/netboot.ipxe chain ''${base}/auto-installer/netboot.ipxe
:nixos-minimal
chain ''${base}/nixos-minimal/netboot.ipxe
:rescue :rescue
chain ''${base}/systemrescue.ipxe chain ''${base}/systemrescue.ipxe
@@ -129,7 +133,8 @@ in
"d ${pxeRoot} 0755 root root -" "d ${pxeRoot} 0755 root root -"
"d ${httpRoot} 0755 root root -" "d ${httpRoot} 0755 root root -"
"d ${httpRoot}/images 0755 root root -" "d ${httpRoot}/images 0755 root root -"
"d ${httpRoot}/nixos 0755 root root -" "d ${httpRoot}/auto-installer 0755 root root -"
"d ${httpRoot}/nixos-minimal 0755 root root -"
"d ${httpRoot}/systemrescue 0755 root root -" "d ${httpRoot}/systemrescue 0755 root root -"
"d ${httpRoot}/ubuntu 0755 root root -" "d ${httpRoot}/ubuntu 0755 root root -"
"d ${httpRoot}/rescue 0755 root root -" "d ${httpRoot}/rescue 0755 root root -"
+64 -2
View File
@@ -1,12 +1,74 @@
{ vars, lib, ... }: { vars, lib, pkgs, ... }:
let
poolName = lib.removePrefix "/" vars.storageRoot;
# For each NFS share subpath, generate every ancestor path so ZFS datasets
# are created parent-first. e.g. "docker/config" → ["docker" "docker/config"]
ancestors = path:
let parts = lib.splitString "/" path;
in lib.imap1 (i: _: lib.concatStringsSep "/" (lib.take i parts)) parts;
poolDatasets = lib.unique (
lib.concatMap (share: ancestors share.subpath) (lib.attrValues vars.nfsShares)
);
in
{ {
imports = [ imports = [
../beszel/enable-agent.nix ../beszel/enable-agent.nix
../services/zfs/enable-service.nix ../services/zfs/enable-service.nix
]; ];
boot.zfs.extraPools = [ (lib.removePrefix "/" vars.storageRoot) ]; boot.zfs.extraPools = [ poolName ];
# On a fresh image deploy the data disk (scsi1) starts blank — no pool
# exists yet, so zfs-import-tank.service would spin for 60 s and fail.
# This service runs first: if the pool is already present it exits instantly;
# otherwise it creates it (with all required datasets) so the standard
# import service finds it ready on the very first boot.
systemd.services."zfs-init-${poolName}" = {
description = "Initialize '${poolName}' ZFS pool on first boot if not present";
wantedBy = [ "zfs-import-${poolName}.service" ];
before = [ "zfs-import-${poolName}.service" ];
after = [ "systemd-udev-settle.service" ];
unitConfig.DefaultDependencies = false;
serviceConfig = {
Type = "oneshot";
RemainAfterExit = true;
};
path = [ pkgs.zfs_unstable ];
script = ''
# Already imported nothing to do.
if zpool list "${poolName}" >/dev/null 2>&1; then
exit 0
fi
# Pool exists on a device but not yet imported let the standard
# zfs-import-${poolName}.service handle it normally.
if zpool import -d /dev/disk/by-id -N "${poolName}" 2>/dev/null; then
exit 0
fi
# No pool found at all. Create it on the Proxmox data disk (scsi1),
# which appears as /dev/disk/by-id/scsi-*drive-scsi1 inside the VM.
DATA_DISK=""
for candidate in /dev/disk/by-id/scsi-*drive-scsi1; do
[[ "$candidate" == *-part* ]] && continue
[ -b "$candidate" ] && DATA_DISK="$candidate" && break
done
if [ -z "$DATA_DISK" ]; then
echo "zfs-init-${poolName}: no data disk found (expected /dev/disk/by-id/scsi-*drive-scsi1)" >&2
exit 1
fi
echo "zfs-init-${poolName}: creating pool on $DATA_DISK"
zpool create -f "${poolName}" "$DATA_DISK"
${lib.concatMapStrings (ds: ''
zfs create "${poolName}/${ds}"
'') poolDatasets}
'';
};
systemd.services.nfs-server = { systemd.services.nfs-server = {
after = [ "zfs-mount.service" ]; after = [ "zfs-mount.service" ];
+1
View File
@@ -3,5 +3,6 @@
{ {
imports = [ imports = [
../tor/enable-relay.nix ../tor/enable-relay.nix
../beszel/enable-agent.nix
]; ];
} }
+2
View File
@@ -31,6 +31,7 @@
btop btop
git git
gcr gcr
jq
]; ];
# Secrets shared by every host, decrypted at activation via each host's # Secrets shared by every host, decrypted at activation via each host's
@@ -76,6 +77,7 @@
openssh.authorizedKeys.keys = [ openssh.authorizedKeys.keys = [
vars.adminSshKey vars.adminSshKey
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICMJhrfFayLBG+gWtO6oAvgambw5nWWgztiTFEaaaVRH debian@surface" "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICMJhrfFayLBG+gWtO6oAvgambw5nWWgztiTFEaaaVRH debian@surface"
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGygkCljN6uKpdJbHTOQtn8ZnH+wKXDLAwrDFbLrE/65 nixos@nixos"
]; ];
}; };
+87
View File
@@ -0,0 +1,87 @@
{ vars, ... }:
{
# ZFS RAID0 (striped, no redundancy) root pool for the bare-metal gui
# host — two disks, each contributing its own top-level vdev. disko's
# zpool `mode` defaults to "" (plain stripe) when left unset, which is
# what gives RAID0 semantics here rather than mirror/raidz.
#
# Device paths are placeholders until the real hardware profile lands —
# fill in vars.guiRootDisk1/guiRootDisk2 (stable /dev/disk/by-id/...
# paths, not /dev/sdX) before running disko against real hardware. Swap
# is deliberately left out for now — sizing that sensibly needs the
# box's actual RAM size, which comes with the hardware profile too.
#
# Not yet imported anywhere: this awaits the new bare-metal platform
# module (alongside modules/boot/efi.nix for systemd-boot, matching
# modules/platforms/proxmox.nix's pattern) once the hardware config is
# in hand.
disko.devices = {
disk = {
disk1 = {
type = "disk";
device = vars.guiRootDisk1;
content = {
type = "gpt";
partitions = {
esp = {
priority = 1;
name = "ESP";
size = "512M";
type = "EF00";
content = {
type = "filesystem";
format = "vfat";
mountpoint = "/boot";
mountOptions = [ "umask=0077" ];
};
};
zfs = {
size = "100%";
content = {
type = "zfs";
pool = "rpool";
};
};
};
};
};
disk2 = {
type = "disk";
device = vars.guiRootDisk2;
content = {
type = "gpt";
partitions = {
zfs = {
size = "100%";
content = {
type = "zfs";
pool = "rpool";
};
};
};
};
};
};
zpool.rpool = {
type = "zpool";
rootFsOptions = {
compression = "zstd";
"com.sun:auto-snapshot" = "false";
};
mountpoint = "/";
options.ashift = "12";
};
};
}
+19 -5
View File
@@ -9,11 +9,25 @@ let
# (the VM platforms rely on automount itself to get that same # (the VM platforms rely on automount itself to get that same
# non-blocking behavior, so they don't need `nofail` too). # non-blocking behavior, so they don't need `nofail` too).
automountOpts = if config.boot.isContainer then [ "nofail" ] else [ "x-systemd.automount" ]; automountOpts = if config.boot.isContainer then [ "nofail" ] else [ "x-systemd.automount" ];
# A bare hostname here never resolves reliably: systemd-resolved only
# ever tries LLMNR for single-label names (never DNS, regardless of any
# configured search domain), and a *global* search domain (the first fix
# attempted here) backfires worse -- confirmed live on lxc-docker, adding
# `networking.search` made systemd-resolved prioritize its domain-matched
# but server-less global scope over eth0's correctly-configured one for
# every "*.sweet.home" query, silently sending them to public fallback
# DNS instead. `resolvectl query --interface=eth0 server.sweet.home`
# resolved fine throughout, proving the LAN DNS server was never the
# problem -- only the ambient, unqualified device string was. Using the
# FQDN directly sidesteps all of that, matching the pattern
# ../raspi/mount-data.nix already uses for the same reason.
nfsServer = "${vars.nfsServerHost}.${vars.homeDomain}";
in in
{ {
fileSystems = { fileSystems = {
${vars.nfsShares.dockerConfig.mountpoint} = { ${vars.nfsShares.dockerConfig.mountpoint} = {
device = "${vars.nfsServerHost}:${vars.storageRoot}/${vars.nfsShares.dockerConfig.subpath}"; device = "${nfsServer}:${vars.storageRoot}/${vars.nfsShares.dockerConfig.subpath}";
fsType = "nfs"; fsType = "nfs";
options = [ options = [
@@ -24,7 +38,7 @@ in
}; };
${vars.nfsShares.dockerDatabases.mountpoint} = { ${vars.nfsShares.dockerDatabases.mountpoint} = {
device = "${vars.nfsServerHost}:${vars.storageRoot}/${vars.nfsShares.dockerDatabases.subpath}"; device = "${nfsServer}:${vars.storageRoot}/${vars.nfsShares.dockerDatabases.subpath}";
fsType = "nfs"; fsType = "nfs";
options = [ options = [
@@ -35,7 +49,7 @@ in
}; };
${vars.nfsShares.dockerVolumes.mountpoint} = { ${vars.nfsShares.dockerVolumes.mountpoint} = {
device = "${vars.nfsServerHost}:${vars.storageRoot}/${vars.nfsShares.dockerVolumes.subpath}"; device = "${nfsServer}:${vars.storageRoot}/${vars.nfsShares.dockerVolumes.subpath}";
fsType = "nfs"; fsType = "nfs";
options = [ options = [
@@ -46,7 +60,7 @@ in
}; };
${vars.nfsShares.nextcloudData.mountpoint} = { ${vars.nfsShares.nextcloudData.mountpoint} = {
device = "${vars.nfsServerHost}:${vars.storageRoot}/${vars.nfsShares.nextcloudData.subpath}"; device = "${nfsServer}:${vars.storageRoot}/${vars.nfsShares.nextcloudData.subpath}";
fsType = "nfs"; fsType = "nfs";
options = [ options = [
@@ -57,7 +71,7 @@ in
}; };
${vars.nfsShares.raspiVolumes.mountpoint} = { ${vars.nfsShares.raspiVolumes.mountpoint} = {
device = "${vars.nfsServerHost}:${vars.storageRoot}/${vars.nfsShares.raspiVolumes.subpath}"; device = "${nfsServer}:${vars.storageRoot}/${vars.nfsShares.raspiVolumes.subpath}";
fsType = "nfs"; fsType = "nfs";
options = [ options = [
@@ -0,0 +1,23 @@
# Adapted from the output of `nixos-generate-config`, run from a live GUI
# ISO boot on the actual gui-host hardware (AMD CPU). fileSystems and
# swapDevices are deliberately omitted -- the live ISO had no formatted
# disks to detect, and disko (modules/disko/baremetal.nix) generates both
# from the declarative zpool layout anyway.
{ config, lib, pkgs, modulesPath, ... }:
{
imports =
[
(modulesPath + "/installer/scan/not-detected.nix")
];
boot = {
initrd.availableKernelModules = [ "xhci_pci" "ahci" "usbhid" "usb_storage" "sd_mod" ];
initrd.kernelModules = [ ];
kernelModules = [ "kvm-amd" ];
extraModulePackages = [ ];
};
nixpkgs.hostPlatform = lib.mkDefault "x86_64-linux";
hardware.cpu.amd.updateMicrocode = lib.mkDefault config.hardware.enableRedistributableFirmware;
}
+15 -134
View File
@@ -45,140 +45,21 @@
disko disko
]; ];
# Write auto-install script to /root # Auto-install script, kept as a real, version-controlled shell file at
etc."auto-install.sh" = { # scripts/installer/auto-install.sh rather than an inline Nix string.
text = '' # It sources scripts/env.sh itself (for LAN_DOMAIN, same as every other
#!/run/current-system/sw/bin/bash # script in this repo) rather than relying on Nix-level templating, so
set -eux # it behaves identically whether it's run straight from a git checkout
# or from here -- baking scripts/env.sh in alongside it at a matching
# relative path (installer/auto-install.sh -> ../env.sh) is what makes
# that resolve correctly in both places.
etc = {
"nixos-installer/env.sh".source = ../../scripts/env.sh;
set -euo pipefail "nixos-installer/installer/auto-install.sh" = {
source = ../../scripts/installer/auto-install.sh;
export FLAKE_BASE_URL="git+https://${vars.lanDomain}/beatzaplenty/nixos.git" mode = "0755";
};
echo "Fetching available NixOS hosts from flake..."
# Two categories deliberately excluded from the menu:
# lxc-* these build a config.system.build.tarball meant for
# `pct restore` on Proxmox directly, not an install.
# Running nixos-install against one here would
# bind-mount / onto /mnt and then refuse to touch the
# filesystem it's currently running on see
# docs/auto-installer.md.
# installer this *is* the installer image's own flake target,
# not a deployable host; "installing" it means
# nixos-install-ing a copy of the installer into
# itself.
mapfile -t options < <(
nix eval --json --no-use-registries --no-accept-flake-config --extra-experimental-features "flakes nix-command" \
"''${FLAKE_BASE_URL}#nixosConfigurations" \
--apply builtins.attrNames \
| jq -r '.[]
| select(startswith("lxc-") | not)
| select(. != "installer")'
)
if [[ ''${#options[@]} -eq 0 ]]; then
echo "ERROR: No NixOS hosts found in ''${FLAKE_BASE_URL}#nixosConfigurations" >&2
exit 1
fi
echo "Note: lxc-* targets aren't installed this way build them with"
echo " nix build .#nixosConfigurations.<name>.config.system.build.tarball"
echo "and 'pct restore' the result on Proxmox directly. See docs/auto-installer.md."
echo "Choose the flake profile to install:"
select choice in "''${options[@]}"; do
if [[ -n "$choice" ]]; then
echo "You selected: $choice"
break
else
echo "Invalid selection. Try again."
fi
done
echo "Starting install with flake: ''${FLAKE_BASE_URL}#''${choice}"
# Optional: confirm before proceeding
read -rp "Proceed with installation? (y/N): " confirm
if [[ ! "$confirm" =~ ^[Yy]$ ]]; then
echo "Aborted."
exit 1
fi
# A nix-cache host is *the* substituter/remote-builder for every other
# host once installed (its own config explicitly excludes itself from
# using either see buildType != "nix-cache" in the nixos flake.nix).
# Installing one shouldn't depend on a nix-cache substituter either,
# for the same reason plus in practice "nix-cache" only resolves over
# Tailscale, which a fresh installer environment was never connected to
# anyway, so it's dead weight even for non-nix-cache installs until
# that's sorted out. Override it away here specifically for nix-cache
# targets to keep install-time behaviour consistent with run-time.
nix_extra_opts=()
if [[ "''${choice}" == *-nix-cache ]]; then
echo "Installing a nix-cache host skipping the nix-cache substituter."
nix_extra_opts+=(--option substituters "https://cache.nixos.org/")
fi
# Every host reachable through this menu has a Disko config (lxc-*
# is filtered out above, and is the only category that doesn't
# see docs/auto-installer.md), so this can run unconditionally: no
# need to probe the flake first and branch on whether Disko applies.
disko --mode destroy,format,mount \
--flake "''${FLAKE_BASE_URL}#''${choice}" "''${nix_extra_opts[@]}" --yes-wipe-all-disks
# sops-nix derives this host's decryption key from its own SSH host key
# at *activation* time, which runs before systemd would otherwise
# generate one on first boot. Without pre-seeding it here, secrets
# (including the login password) fail to decrypt on first boot.
# Generate the key with scripts/prepare-host-key.sh first.
#
# Two places a key can come from, checked in order:
# /etc/host-keys baked into this image at build time (see
# modules/installer/host-keys.nix; only present
# if built with NIXOS_HOST_KEYS_DIR set)
# /root/host-keys scp'd in manually after boot (older fallback,
# still supported for images built without keys)
mkdir -p /root/host-keys
if [[ -f "/etc/host-keys/''${choice}_ssh_host_ed25519_key" ]]; then
echo "Found baked-in SSH host key for ''${choice}, installing to target..."
install -D -m 0600 "/etc/host-keys/''${choice}_ssh_host_ed25519_key" /mnt/etc/ssh/ssh_host_ed25519_key
install -D -m 0644 "/etc/host-keys/''${choice}_ssh_host_ed25519_key.pub" /mnt/etc/ssh/ssh_host_ed25519_key.pub
elif [[ -f "/root/host-keys/''${choice}_ssh_host_ed25519_key" ]]; then
echo "Found pre-seeded SSH host key for ''${choice}, installing to target..."
install -D -m 0600 "/root/host-keys/''${choice}_ssh_host_ed25519_key" /mnt/etc/ssh/ssh_host_ed25519_key
install -D -m 0644 "/root/host-keys/''${choice}_ssh_host_ed25519_key.pub" /mnt/etc/ssh/ssh_host_ed25519_key.pub
else
echo "WARNING: no SSH host key found for ''${choice} (checked /etc/host-keys and /root/host-keys)"
echo "sops-nix secrets (including the login password) will NOT decrypt on first boot."
echo "Run scripts/prepare-host-key.sh for host ''${choice} on your admin workstation first,"
echo "then either rebuild this image with NIXOS_HOST_KEYS_DIR set, or scp the result to"
echo "/root/host-keys/ on this machine."
read -rp "Continue without a pre-seeded key anyway? (y/N): " skip_key
if [[ ! "$skip_key" =~ ^[Yy]$ ]]; then
echo "Aborted."
exit 1
fi
fi
mkdir -p /mnt/install-tmp
export TMPDIR=/mnt/install-tmp
nixos-install \
--flake "''${FLAKE_BASE_URL}#''${choice}" \
"''${nix_extra_opts[@]}" \
--no-root-password
rm -rf /mnt/install-tmp
# Redundant copy of the host's private key the real one is now at
# /etc/ssh/ssh_host_ed25519_key. Nothing NixOS-managed ever cleans this
# up on its own since it was written imperatively, not declaratively.
rm -rf /root/host-keys
sleep 10
reboot
'';
mode = "0755";
}; };
}; };
@@ -192,7 +73,7 @@
# file-copying/chown. # file-copying/chown.
programs.bash.loginShellInit = '' programs.bash.loginShellInit = ''
if [ -n "$PS1" ] && [ ! -e "$HOME/.auto_install_ran" ]; then if [ -n "$PS1" ] && [ ! -e "$HOME/.auto_install_ran" ]; then
sudo /etc/auto-install.sh sudo /etc/nixos-installer/installer/auto-install.sh
touch "$HOME/.auto_install_ran" touch "$HOME/.auto_install_ran"
fi fi
''; '';
+43
View File
@@ -0,0 +1,43 @@
{ config, lib, vars, ... }:
{
# Prestages a NetworkManager connection profile for vars.wifiSsid so the
# host associates on first boot with no manual nmtui/nmcli step. Guarded
# on a non-empty SSID so leaving the placeholder blank in variables.nix
# is a no-op rather than an empty, broken profile — fill it in once the
# network is known.
#
# The password itself lives in secrets/gui.yaml, not variables.nix --
# NetworkManager's ensureProfiles renders `psk = "$WIFI_PASSWORD"`
# literally into the store (see nixpkgs' own ensureProfiles example,
# which does the same for exactly this reason) and its systemd service
# envsubst-expands it from environmentFiles at activation time, so the
# real value only ever touches /run (root-only, UMask 0177), never the
# Nix store.
sops.secrets."wifi-password" = lib.mkIf (vars.wifiSsid != "") {
sopsFile = ../../secrets/gui.yaml;
};
sops.templates."wifi-password.env" = lib.mkIf (vars.wifiSsid != "") {
content = "WIFI_PASSWORD=${config.sops.placeholder."wifi-password"}";
};
networking.networkmanager.ensureProfiles = lib.mkIf (vars.wifiSsid != "") {
environmentFiles = [ config.sops.templates."wifi-password.env".path ];
profiles.${vars.wifiSsid} = {
connection = {
id = vars.wifiSsid;
type = "wifi";
};
wifi = {
mode = "infrastructure";
ssid = vars.wifiSsid;
};
wifi-security = {
key-mgmt = "wpa-psk";
psk = "$WIFI_PASSWORD";
};
};
};
}
+9 -5
View File
@@ -1,10 +1,14 @@
{ pkgs, vars, ... }: { pkgs, vars, ... }:
{ {
# Install the remote builder key on each client host (do not commit private keys): # Authenticate as nixremote using the client host's own default root SSH
# sudo install -d -m 0700 /root/.ssh # identity (/root/.ssh/id_ed25519) rather than a separately-named key --
# sudo install -m 0600 ./nixremote /root/.ssh/nixremote # matches vars.remoteBuilderAuthorizedKeys, which already authorizes
# sudo ssh -i /root/.ssh/nixremote nixremote@nix-cache nix-store --version # each host's own default key (one entry per host, not a shared
# dedicated keypair). If this host doesn't have one yet:
# sudo -u root ssh-keygen -t ed25519 -N '' -f /root/.ssh/id_ed25519
# # then add its .pub to vars.remoteBuilderAuthorizedKeys and rebuild nix-cache
# sudo ssh -i /root/.ssh/id_ed25519 nixremote@nix-cache nix-store --version
# Trust nix-cache's SSH host key declaratively so the nix-daemon (root) # Trust nix-cache's SSH host key declaratively so the nix-daemon (root)
# can connect the first time without a manual ssh-keyscan/known_hosts # can connect the first time without a manual ssh-keyscan/known_hosts
# step on every new client. # step on every new client.
@@ -20,7 +24,7 @@
{ {
hostName = vars.nixCacheHost; hostName = vars.nixCacheHost;
sshUser = vars.remoteBuilderUser; sshUser = vars.remoteBuilderUser;
sshKey = "/root/.ssh/${vars.remoteBuilderUser}"; sshKey = "/root/.ssh/id_ed25519";
inherit (pkgs.stdenv.hostPlatform) system; inherit (pkgs.stdenv.hostPlatform) system;
maxJobs = 4; maxJobs = 4;
speedFactor = 2; speedFactor = 2;
+38
View File
@@ -0,0 +1,38 @@
{ ... }:
{
imports = [
../hardware-configuration/baremetal.nix
../boot/efi.nix
../disko/baremetal.nix
../services/zfs/enable-service.nix
];
# Needed for real wifi/bluetooth/GPU firmware blobs and CPU microcode
# updates (hardware-configuration/baremetal.nix's amd.updateMicrocode
# keys off this) -- irrelevant on the linode/proxmox/lxc platforms,
# which are all VMs with no real hardware to load firmware for.
hardware.enableRedistributableFirmware = true;
# AMD GPU: the amdgpu kernel driver autoloads from the PCI ID with no
# extra boot.kernelModules entry needed; this is the userspace half --
# the dedicated Xorg driver (not just the generic modesetting fallback)
# plus Mesa OpenGL/Vulkan (amdgpu/RADV), same firmware blobs as above.
# 32-bit support is for compatibility with 32-bit apps/games.
services.xserver.videoDrivers = [ "amdgpu" ];
hardware.graphics = {
enable = true;
enable32Bit = true;
};
# The systemd-based initrd (default here since this host has a ZFS root --
# see modules/disko/baremetal.nix) locks the root account by default, so
# sulogin refuses to hand over a shell if something in the initrd (e.g.
# the ZFS pool import) fails and it drops to emergency mode -- confirmed
# live: it just loops re-entering the target instead of prompting. This
# only affects the pre-switch-root initrd shell, not the installed
# system's own login, and is worth the tradeoff on a box already reachable
# at the physical console.
boot.initrd.systemd.emergencyAccess = true;
}
+59 -5
View File
@@ -1,4 +1,4 @@
{ lib, modulesPath, flakeTarget, ... }: { config, lib, modulesPath, flakeTarget, ... }:
let let
# Bakes this exact flake target's pre-generated SSH host key straight # Bakes this exact flake target's pre-generated SSH host key straight
@@ -14,7 +14,7 @@ let
# Without this, config.system.build.tarball's built-in system just # Without this, config.system.build.tarball's built-in system just
# generates a fresh host key at first boot like any other host would -- # generates a fresh host key at first boot like any other host would --
# but sops-nix derives its decryption key from *this* file, and # but sops-nix derives its decryption key from *this* file, and
# .sops.yaml only trusts whatever key scripts/sync-host-keys.sh already # .sops.yaml only trusts whatever key scripts/secrets/sync-host-keys.sh already
# registered for this exact target name. A freshly-generated key can # registered for this exact target name. A freshly-generated key can
# never match that, so every secret (including this host's own login) # never match that, so every secret (including this host's own login)
# permanently fails to decrypt. Confirmed live: sops-install-secrets # permanently fails to decrypt. Confirmed live: sops-install-secrets
@@ -26,7 +26,7 @@ let
hostKeysDir = /. + hostKeysDirStr; hostKeysDir = /. + hostKeysDirStr;
# flakeTarget ("${platform}-${buildType}") comes in via specialArgs from # flakeTarget ("${platform}-${buildType}") comes in via specialArgs from
# flake.nix's mkTarget -- exactly the name scripts/sync-host-keys.sh # flake.nix's mkTarget -- exactly the name scripts/secrets/sync-host-keys.sh
# registers keys under. Deliberately not read back from # registers keys under. Deliberately not read back from
# config.environment.etc."flake-target" (which is set to the same value) # config.environment.etc."flake-target" (which is set to the same value)
# -- this module also *contributes* to environment.etc below, and a # -- this module also *contributes* to environment.etc below, and a
@@ -58,8 +58,28 @@ in
# host.nix declares each host's real hostname (networking.hostName); # host.nix declares each host's real hostname (networking.hostName);
# keep that instead of letting Proxmox's ambient container config win. # keep that instead of letting Proxmox's ambient container config win.
manageHostName = true; manageHostName = true;
# Unprivileged matches how these containers are actually created. # Unprivileged by default -- matches how these containers are actually
privileged = false; # created (scripts/proxmox/create-proxmox-resource.sh reads this value
# back to decide `pct create`'s --unprivileged flag, so the two stay
# in sync).
#
# lxc-docker is the one exception: the kernel's NFS client doesn't set
# FS_USERNS_MOUNT, so mounting NFS from inside *any* non-init user
# namespace -- which is exactly what an unprivileged container's
# UID-mapped root runs in -- is rejected at the VFS layer with EPERM,
# no matter what Proxmox's own `mount=nfs;nfs4` container feature
# allows at the AppArmor layer (confirmed live: TCP to the NFS server
# succeeds, the server's export table matches the container's IP, and
# `mount.nfs: Operation not permitted` still fires immediately with no
# corresponding denial anywhere in the server's logs -- a kernel-level
# rejection, not a network or export-permission one). Keying off
# hostName rather than something docker-build-type-specific because
# modules/build-types/docker.nix is also composed for linode-docker/
# proxmox-docker, which don't import proxmox-lxc.nix at all --setting
# this option there would break their eval with "option does not
# exist" regardless of any mkIf guard, since mkIf only makes a value
# conditional, not whether the option needs to exist somewhere.
privileged = config.networking.hostName == "docker";
}; };
boot.loader = { boot.loader = {
@@ -86,6 +106,40 @@ in
}; };
}; };
# NixOS's etc activation removes any /etc file that was in the previous
# generation's environment.etc but is absent from the current one — even
# real (non-symlink) copies. On every routine nixos-rebuild switch/test that
# lacks NIXOS_HOST_KEYS_DIR the key is absent from environment.etc, so it
# gets removed as "obsolete". sops-nix derives its age decryption key from
# /etc/ssh/ssh_host_ed25519_key; deletion cascades into every sops secret
# failing with "Error getting data key: 0 successful groups required, got 0".
#
# Fix: two activation scripts that bracket the etc step.
# preserveSshHostKey — no deps, runs before etc — saves the live key to
# /run (tmpfs) before etc can delete it.
# restoreSshHostKey — deps=[etc], runs after etc — reinstalls the key via
# `install` (atomic, sets mode) if etc removed it.
# The resulting file is not registered in environment.etc
# for either the previous or current generation, so
# subsequent rebuilds leave it alone permanently.
system.activationScripts.preserveSshHostKey = ''
if [ -f /etc/ssh/ssh_host_ed25519_key ]; then
cp /etc/ssh/ssh_host_ed25519_key /run/sshd-host-key-preserve.tmp
cp /etc/ssh/ssh_host_ed25519_key.pub /run/sshd-host-key-preserve.pub.tmp
fi
'';
system.activationScripts.restoreSshHostKey = {
deps = [ "etc" ];
text = ''
if [ ! -f /etc/ssh/ssh_host_ed25519_key ] && [ -f /run/sshd-host-key-preserve.tmp ]; then
install -m 0600 /run/sshd-host-key-preserve.tmp /etc/ssh/ssh_host_ed25519_key
install -m 0644 /run/sshd-host-key-preserve.pub.tmp /etc/ssh/ssh_host_ed25519_key.pub
fi
rm -f /run/sshd-host-key-preserve.tmp /run/sshd-host-key-preserve.pub.tmp
'';
};
# virtualisation/proxmox-lxc.nix (imported above) registers the Nix # virtualisation/proxmox-lxc.nix (imported above) registers the Nix
# store DB via a systemd service (register-nix-paths) -- it never runs # store DB via a systemd service (register-nix-paths) -- it never runs
# an activation script at all. Confirmed live this means neither # an activation script at all. Confirmed live this means neither
+23 -14
View File
@@ -1,23 +1,32 @@
{ netbootSystem, ... }: { netbootSystem, netbootMinimalSystem, ... }:
let let
# config.system.build.kernel and .netbootRamdisk are directories, not the # config.system.build.kernel and .netbootRamdisk are directories, not the
# files themselves — nixpkgs' own system.build.kexecTree does the same # files themselves — nixpkgs' own system.build.kexecTree does the same
# ${...}/<file> dereference for the same reason. # ${...}/<file> dereference for the same reason.
inherit (netbootSystem.config.system.boot.loader) kernelFile; mkStageRules = { dirName, system }:
let
inherit (system.config.system.boot.loader) kernelFile;
dir = "/srv/pxe/http/${dirName}";
in
[
# Declared here too (not just in build-types/pxe-boot.nix) so this
# module's C+ rules don't depend on cross-module list-merge ordering —
# tmpfiles' C type needs the target directory to already exist.
"d ${dir} 0755 root root -"
"C+ ${dir}/${kernelFile} 0644 root root - ${system.config.system.build.kernel}/${kernelFile}"
"C+ ${dir}/initrd 0644 root root - ${system.config.system.build.netbootRamdisk}/initrd"
"C+ ${dir}/netboot.ipxe 0644 root root - ${system.config.system.build.netbootIpxeScript}/netboot.ipxe"
];
in in
{ {
# Builds this flake's own installer netboot image (the same one # Builds this flake's own installer netboot image (the same one
# `nix build .#pxe` produces) and stages it where menu.ipxe's :nixos # `nix build .#pxe` produces) plus the vanilla NixOS minimal netboot image
# entry expects it, so the pxe-boot host is self-contained — no manual # (`nix build .#pxe-minimal`), and stages both where menu.ipxe's
# operator step to populate /srv/pxe/http/nixos after deploy. # :auto-installer / :nixos-minimal entries expect them, so the pxe-boot
systemd.tmpfiles.rules = [ # host is self-contained — no manual operator step to populate
# Declared here too (not just in build-types/pxe-boot.nix) so this # /srv/pxe/http after deploy.
# module's C+ rules don't depend on cross-module list-merge ordering — systemd.tmpfiles.rules =
# tmpfiles' C type needs the target directory to already exist. mkStageRules { dirName = "auto-installer"; system = netbootSystem; }
"d /srv/pxe/http/nixos 0755 root root -" ++ mkStageRules { dirName = "nixos-minimal"; system = netbootMinimalSystem; };
"C+ /srv/pxe/http/nixos/${kernelFile} 0644 root root - ${netbootSystem.config.system.build.kernel}/${kernelFile}"
"C+ /srv/pxe/http/nixos/initrd 0644 root root - ${netbootSystem.config.system.build.netbootRamdisk}/initrd"
"C+ /srv/pxe/http/nixos/netboot.ipxe 0644 root root - ${netbootSystem.config.system.build.netbootIpxeScript}/netboot.ipxe"
];
} }
Binary file not shown.
+3 -2
View File
@@ -129,5 +129,6 @@ echo "flake.lock still points at the old input revisions until refreshed. Either
echo " nix flake update nixpkgs home-manager # just these two inputs" echo " nix flake update nixpkgs home-manager # just these two inputs"
echo " nix flake update # everything — see docs/flake-lock-automation.md" echo " nix flake update # everything — see docs/flake-lock-automation.md"
echo echo
echo "Then run 'bash scripts/codex-maintenance.sh dry-run' before committing —" echo "Then run 'bash scripts/codex-maintenance.sh --full-check --dry-run' before"
echo "a channel bump can shift option defaults across every host." echo "committing — a channel bump can shift option defaults across every host,"
echo "and only --dry-run actually builds anything to catch that."
+267 -49
View File
@@ -1,4 +1,29 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Validation entry point for CI and local/agent review.
#
# Default mode (what CI runs on every push/PR): fmt-check, statix, and eval
# are scoped to files that actually changed against a base ref, plus
# whichever hosts/packages those changes can affect. This exists because
# the unscoped sweep below is slow enough to time out CI runners -- see
# --full-check.
#
# --full-check: the historical full sweep (every host, every package,
# fmt --check ./statix check . over the whole tree). Slow -- minutes, not
# seconds. CI never passes this; run it locally before a release or after
# touching modules/common/*, flake.nix, or variables.nix if you want extra
# confidence beyond what the changed-files scope already covers for those
# paths (see below).
#
# --dry-run: adds `nix build --dry-run --no-link` for whatever scope is
# active (changed-files scope by default, full scope under --full-check).
#
# Per-host/per-package eval and dry-run build calls run concurrently (see
# scripts/lib/nix-parallel.sh) since they're independent of each other.
# Concurrency defaults to core count capped by available memory (~1GB/job)
# rather than plain core count, since each concurrent `nix eval` evaluates a
# whole NixOS system closure and can OOM a small/memory-constrained CI
# runner otherwise; override via NIX_PARALLEL_JOBS if a runner has more (or
# less) room than that estimate assumes.
set -euo pipefail set -euo pipefail
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
@@ -6,8 +31,42 @@ script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "${script_dir}/lib/nix-bootstrap.sh" source "${script_dir}/lib/nix-bootstrap.sh"
# shellcheck source=lib/nix-eval.sh # shellcheck source=lib/nix-eval.sh
source "${script_dir}/lib/nix-eval.sh" source "${script_dir}/lib/nix-eval.sh"
# shellcheck source=lib/nix-parallel.sh
source "${script_dir}/lib/nix-parallel.sh"
MODE="${1:-validate}" repo_root="$(cd "${script_dir}/.." && pwd)"
cd "$repo_root"
full_check=false
dry_run=false
usage() {
cat <<'EOF'
Usage: scripts/codex-maintenance.sh [--full-check] [--dry-run]
--full-check Run the full sweep: fmt-check and statix over the whole
repo, eval every host and package. Slow. Never run by CI.
--dry-run Additionally run `nix build --dry-run --no-link` for
whatever scope is active.
With neither flag (the CI default), fmt-check/statix/eval are scoped to
files changed against a base ref (env MAINT_BASE_SHA, else the PR base,
else HEAD^), plus the hosts/packages those changes can affect.
EOF
}
for arg in "$@"; do
case "$arg" in
--full-check) full_check=true ;;
--dry-run) dry_run=true ;;
-h|--help) usage; exit 0 ;;
*)
echo "Unknown argument: $arg" >&2
usage >&2
exit 1
;;
esac
done
ensure_nix_profile ensure_nix_profile
@@ -16,12 +75,6 @@ if ! command -v nix >/dev/null 2>&1; then
exit 127 exit 127
fi fi
hosts="$(list_flake_targets .)"
echo "Hosts:"
echo "$hosts"
echo
echo "Checking for obvious committed secrets..." echo "Checking for obvious committed secrets..."
if grep -RInE 'github_pat_|ghp_|access-tokens|hashedPassword[[:space:]]*=' \ if grep -RInE 'github_pat_|ghp_|access-tokens|hashedPassword[[:space:]]*=' \
--exclude-dir=.git \ --exclude-dir=.git \
@@ -33,70 +86,235 @@ else
echo "No obvious token patterns found." echo "No obvious token patterns found."
fi fi
mapfile -t all_hosts < <(list_flake_targets .)
mapfile -t all_packages < <(nix eval --json "${NIX_EVAL_FLAGS[@]}" .#packages.x86_64-linux --apply builtins.attrNames | jq -r '.[]')
# host_targets_for_dir <hosts-subdir-name>
# Prints the nixosConfigurations target names whose hostPath is
# ./hosts/<dir>/host.nix, derived straight from flake.nix's generatedTargets
# (one mkTarget { ... } call per line) rather than a hand-maintained table,
# so it can't drift the way a copied mapping would.
host_targets_for_dir() {
local dir="$1"
grep -oE '^[[:space:]]*[A-Za-z0-9_-]+ = mkTarget \{[^}]*hostPath = \./hosts/'"${dir}"'/host\.nix;[^}]*\};' flake.nix \
| sed -E 's/^[[:space:]]*([A-Za-z0-9_-]+) = mkTarget.*/\1/' \
|| true
}
declare -a changed_files=()
scope_desc="full repo"
if ! $full_check; then
resolve_base_ref() {
if [[ -n "${MAINT_BASE_SHA:-}" ]] && git cat-file -e "${MAINT_BASE_SHA}^{commit}" 2>/dev/null; then
echo "$MAINT_BASE_SHA"
return
fi
if git rev-parse --verify -q HEAD^ >/dev/null 2>&1; then
echo "HEAD^"
return
fi
git hash-object -t tree /dev/null
}
base_ref="$(resolve_base_ref)"
echo
echo "Changed-files scope: diffing against ${base_ref}"
mapfile -t changed_files < <(git diff --name-only --diff-filter=ACMR "$base_ref" -- . | sort -u)
if [[ ${#changed_files[@]} -eq 0 ]]; then
echo "No changed files detected."
else
printf ' %s\n' "${changed_files[@]}"
fi
scope_desc="changed files only (base: ${base_ref})"
fi
# Whole-tree fmt/lint always run under --full-check; otherwise scoped below.
declare -a changed_nix_files=()
for f in "${changed_files[@]:-}"; do
[[ "$f" == *.nix && -f "$f" ]] && changed_nix_files+=("$f")
done
echo echo
echo "Checking Nix formatting with nixpkgs-fmt..." echo "Checking Nix formatting with nixpkgs-fmt..."
nix run "${NIX_EVAL_FLAGS[@]}" github:NixOS/nixpkgs/nixos-25.11#nixpkgs-fmt -- --check . if $full_check; then
nix run "${NIX_EVAL_FLAGS[@]}" github:NixOS/nixpkgs/nixos-25.11#nixpkgs-fmt -- --check .
elif [[ ${#changed_nix_files[@]} -gt 0 ]]; then
nix run "${NIX_EVAL_FLAGS[@]}" github:NixOS/nixpkgs/nixos-25.11#nixpkgs-fmt -- --check "${changed_nix_files[@]}"
else
echo "No changed .nix files; skipping."
fi
echo echo
echo "Running statix lint..." echo "Running statix lint..."
nix run "${NIX_EVAL_FLAGS[@]}" github:NixOS/nixpkgs/nixos-25.11#statix -- check . if $full_check; then
nix run "${NIX_EVAL_FLAGS[@]}" github:NixOS/nixpkgs/nixos-25.11#statix -- check .
elif [[ ${#changed_nix_files[@]} -gt 0 ]]; then
for f in "${changed_nix_files[@]}"; do
nix run "${NIX_EVAL_FLAGS[@]}" github:NixOS/nixpkgs/nixos-25.11#statix -- check "$f"
done
else
echo "No changed .nix files; skipping."
fi
# Figure out which hosts/packages this run needs to eval (and, under
# --dry-run, build). full_check always means "everything"; otherwise a
# change to flake.nix/flake.lock/variables.nix/modules/common/* (repo-wide
# inputs) or to any other modules/*.nix outside platforms//build-types
# (whose blast radius isn't safely inferable from the path alone -- see
# CLAUDE.md's "Grep modules/build-types/*.nix for each build type's imports
# list") also falls back to everything, on the same reasoning CLAUDE.md
# already gives interactive sessions for when to run the full sweep.
# Anything more targeted -- a host.nix, a platform module, a build-type
# module -- narrows to just the hosts it can affect.
declare -A affected_hosts=()
eval_packages=false
if $full_check; then
for h in "${all_hosts[@]}"; do affected_hosts[$h]=1; done
eval_packages=true
else
full_fallback=false
for f in "${changed_files[@]:-}"; do
case "$f" in
flake.nix|flake.lock|variables.nix|modules/common/*)
full_fallback=true
;;
esac
done
if ! $full_fallback; then
for f in "${changed_files[@]:-}"; do
case "$f" in
hosts/*/*)
hostdir="${f#hosts/}"
hostdir="${hostdir%%/*}"
while IFS= read -r t; do
[[ -n "$t" ]] && affected_hosts[$t]=1
done < <(host_targets_for_dir "$hostdir")
;;
modules/platforms/*.nix)
platform="$(basename "$f" .nix)"
for h in "${all_hosts[@]}"; do
[[ "$h" == "${platform}-"* ]] && affected_hosts[$h]=1
done
;;
modules/build-types/*.nix)
buildtype="$(basename "$f" .nix)"
for h in "${all_hosts[@]}"; do
[[ "$h" == *"-${buildtype}" ]] && affected_hosts[$h]=1
done
;;
modules/installer/*)
# iso.nix (imported by both the "installer" nixosConfigurations
# target and netbootSystem, which backs packages.pxe) pulls in
# common.nix, so a common.nix change reaches all three.
affected_hosts[installer]=1
eval_packages=true
;;
modules/pxe-boot/*)
# stage-installer-artifacts.nix is imported by
# modules/build-types/pxe-boot.nix only -- same blast radius as a
# build-types/*.nix change, not a packages one.
for h in "${all_hosts[@]}"; do
[[ "$h" == *"-pxe-boot" ]] && affected_hosts[$h]=1
done
;;
modules/*)
full_fallback=true
;;
esac
done
fi
if $full_fallback; then
echo
echo "Changed files affect shared config; falling back to evaluating every host/package."
for h in "${all_hosts[@]}"; do affected_hosts[$h]=1; done
eval_packages=true
fi
fi
mapfile -t hosts < <(for h in "${!affected_hosts[@]}"; do echo "$h"; done | sort)
echo echo
echo "Evaluating host toplevel derivations..." echo "Checking nix-cache host key for drift..."
for host in $hosts; do if bash "${script_dir}/secrets/sync-nix-cache-host-key.sh" --check; then
echo "==> $host" :
nix eval --raw "${NIX_EVAL_FLAGS[@]}" ".#nixosConfigurations.${host}.config.system.build.toplevel.drvPath" else
drift_status=$?
if [[ "$drift_status" -eq 2 ]]; then
echo "nix-cache unreachable from here -- skipping host-key drift check."
else
echo "WARNING: nix-cache's host key has drifted from variables.nix (see above)." >&2
echo " Run 'bash scripts/secrets/sync-nix-cache-host-key.sh' to fix." >&2
fi
fi
echo
if [[ ${#hosts[@]} -eq 0 ]]; then
echo "No hosts affected by changed files; skipping host eval."
else
echo "Evaluating host toplevel derivations (${scope_desc}, up to ${NIX_PARALLEL_JOBS} at a time)..."
# lxc-* hosts deploy via a directly pct-restore-able tarball instead of # lxc-* hosts deploy via a directly pct-restore-able tarball instead of
# nixos-install (see docs/auto-installer.md); proxmox-* hosts can # nixos-install (see docs/auto-installer.md); proxmox-* hosts can
# alternatively be built as a standalone disk image (see # alternatively be built as a standalone disk image (see
# docs/proxmox-images.md). Both are otherwise-unvalidated buildable # docs/proxmox-images.md). Both are otherwise-unvalidated buildable
# surface, easy to silently break without this. # surface, easy to silently break without this.
case "$host" in declare -a host_eval_jobs=()
lxc-*) for host in "${hosts[@]}"; do
echo "==> $host (tarball)" host_eval_jobs+=("${host}${NIX_PARALLEL_SEP}.#nixosConfigurations.${host}.config.system.build.toplevel.drvPath")
nix eval --raw "${NIX_EVAL_FLAGS[@]}" ".#nixosConfigurations.${host}.config.system.build.tarball.drvPath"
;;
proxmox-*)
echo "==> $host (diskoImagesScript)"
nix eval --raw "${NIX_EVAL_FLAGS[@]}" ".#nixosConfigurations.${host}.config.system.build.diskoImagesScript.drvPath"
;;
esac
done
echo
echo "Evaluating buildable packages..."
packages="$(nix eval --json "${NIX_EVAL_FLAGS[@]}" .#packages.x86_64-linux --apply builtins.attrNames | jq -r '.[]')"
for pkg in $packages; do
echo "==> packages.x86_64-linux.${pkg}"
nix eval --raw "${NIX_EVAL_FLAGS[@]}" ".#packages.x86_64-linux.${pkg}"
done
if [[ "$MODE" == "dry-run" ]]; then
echo
echo "Running dry-run builds for all hosts. This will not create result symlinks."
for host in $hosts; do
echo "==> Dry-run build: $host"
nix build --dry-run --no-link "${NIX_EVAL_FLAGS[@]}" ".#nixosConfigurations.${host}.config.system.build.toplevel"
case "$host" in case "$host" in
lxc-*) lxc-*)
echo "==> Dry-run build: $host (tarball)" host_eval_jobs+=("${host} (tarball)${NIX_PARALLEL_SEP}.#nixosConfigurations.${host}.config.system.build.tarball.drvPath")
nix build --dry-run --no-link "${NIX_EVAL_FLAGS[@]}" ".#nixosConfigurations.${host}.config.system.build.tarball"
;; ;;
proxmox-*) proxmox-*)
echo "==> Dry-run build: $host (diskoImagesScript)" host_eval_jobs+=("${host} (diskoImagesScript)${NIX_PARALLEL_SEP}.#nixosConfigurations.${host}.config.system.build.diskoImagesScript.drvPath")
nix build --dry-run --no-link "${NIX_EVAL_FLAGS[@]}" ".#nixosConfigurations.${host}.config.system.build.diskoImagesScript"
;; ;;
esac esac
done done
run_nix_parallel host_eval_jobs eval --raw "${NIX_EVAL_FLAGS[@]}"
fi
echo echo
echo "Running dry-run builds for all packages." if ! $eval_packages; then
for pkg in $packages; do echo "No packages affected by changed files; skipping package eval."
echo "==> Dry-run build: packages.x86_64-linux.${pkg}" else
nix build --dry-run --no-link "${NIX_EVAL_FLAGS[@]}" ".#packages.x86_64-linux.${pkg}" echo "Evaluating buildable packages (up to ${NIX_PARALLEL_JOBS} at a time)..."
declare -a package_eval_jobs=()
for pkg in "${all_packages[@]}"; do
package_eval_jobs+=("packages.x86_64-linux.${pkg}${NIX_PARALLEL_SEP}.#packages.x86_64-linux.${pkg}")
done done
run_nix_parallel package_eval_jobs eval --raw "${NIX_EVAL_FLAGS[@]}"
fi
if $dry_run; then
echo
echo "Running dry-run builds for the active scope (up to ${NIX_PARALLEL_JOBS} at a time). This will not create result symlinks."
declare -a host_build_jobs=()
for host in "${hosts[@]:-}"; do
host_build_jobs+=("Dry-run build: ${host}${NIX_PARALLEL_SEP}.#nixosConfigurations.${host}.config.system.build.toplevel")
case "$host" in
lxc-*)
host_build_jobs+=("Dry-run build: ${host} (tarball)${NIX_PARALLEL_SEP}.#nixosConfigurations.${host}.config.system.build.tarball")
;;
proxmox-*)
host_build_jobs+=("Dry-run build: ${host} (diskoImagesScript)${NIX_PARALLEL_SEP}.#nixosConfigurations.${host}.config.system.build.diskoImagesScript")
;;
esac
done
run_nix_parallel host_build_jobs build --dry-run --no-link "${NIX_EVAL_FLAGS[@]}"
if $eval_packages; then
echo
echo "Running dry-run builds for packages."
declare -a package_build_jobs=()
for pkg in "${all_packages[@]}"; do
package_build_jobs+=("Dry-run build: packages.x86_64-linux.${pkg}${NIX_PARALLEL_SEP}.#packages.x86_64-linux.${pkg}")
done
run_nix_parallel package_build_jobs build --dry-run --no-link "${NIX_EVAL_FLAGS[@]}"
fi
fi fi
echo echo
+3 -9
View File
@@ -68,6 +68,7 @@ cat > "$HOME/.config/nix/nix.conf" <<'EOF'
experimental-features = nix-command flakes experimental-features = nix-command flakes
accept-flake-config = false accept-flake-config = false
warn-dirty = false warn-dirty = false
build-users-group =
EOF EOF
echo "Nix version:" echo "Nix version:"
@@ -82,13 +83,6 @@ if ! command -v jq >/dev/null 2>&1; then
fi fi
echo "Available NixOS hosts:" echo "Available NixOS hosts:"
hosts="$(list_flake_targets .)" list_flake_targets .
echo "$hosts"
echo "Evaluating all host toplevel derivations..." echo "Codex setup complete. Run bash scripts/codex-maintenance.sh to validate changes."
for host in $hosts; do
echo "==> Evaluating $host"
nix eval --raw "${NIX_EVAL_FLAGS[@]}" ".#nixosConfigurations.${host}.config.system.build.toplevel.drvPath"
done
echo "Codex setup complete."
+31 -15
View File
@@ -3,29 +3,39 @@
# second copy of these values in every script: # second copy of these values in every script:
# source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/env.sh" # source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/env.sh"
# Every variable can still be overridden per-invocation via the # Every variable can still be overridden per-invocation via the
# environment (e.g. PROXMOX_STORAGE=tank-nvme ./scripts/create-proxmox-resource.sh ...) # environment (e.g. PROXMOX_STORAGE=tank-nvme ./scripts/proxmox/create-proxmox-resource.sh ...)
# since each one only sets a default if unset. # since each one only sets a default if unset.
# SSH-reachable Proxmox node that scripts/create-proxmox-resource.sh runs # Two SSH-reachable Proxmox nodes exist on the LAN:
# pct/qm on. Matches the Proxmox web UI hostname already used in # - pve1.sweet.home -- production. Real, live VMs/containers.
# hosts/nixos/home.nix's desktop shortcuts (pve.<homeDomain> from # - pve-test.sweet.home -- sandbox/test node, for scratch VMs/containers
# variables.nix) -- change this if that's not actually reachable over SSH, # that don't belong on production.
# or if you're targeting a different node in a multi-node cluster. #
: "${PROXMOX_HOST:=pve.sweet.home}" # PROXMOX_HOST is what scripts/proxmox/create-proxmox-resource.sh actually
: "${PROXMOX_SSH_USER:=root}" # targets by default -- overridable per-invocation with --node <hostname>,
# or per-variable as usual (e.g. PROXMOX_HOST=$PVE_TEST_HOST). It defaults
# to production, matching this repo's behavior before pve-test existed --
# see CLAUDE.md's "Two Proxmox nodes" section for the policy on which
# situations should target which node (in particular: Claude defaults to
# pve-test, not this variable's own default, unless explicitly told
# otherwise).
: "${PVE1_HOST:=pve1.sweet.home}"
: "${PVE_TEST_HOST:=pve-test.sweet.home}"
: "${PROXMOX_HOST:=$PVE1_HOST}"
: "${PROXMOX_SSH_USER:=wayne}"
# Where this flake repo lives on the Proxmox node itself. # Where this flake repo lives on the Proxmox node itself.
# scripts/create-proxmox-resource.sh builds images directly on the node # scripts/proxmox/create-proxmox-resource.sh builds images directly on the node
# instead of transferring them over the network -- it clones the repo here # instead of transferring them over the network -- it clones the repo here
# (from this checkout's own `origin` remote) the first time it doesn't # (from this checkout's own `origin` remote) the first time it doesn't
# find it, installing build tooling via scripts/codex-setup.sh, then # find it, installing build tooling via scripts/codex-setup.sh, then
# `git pull`s it before every subsequent build. # `git pull`s it before every subsequent build.
: "${PROXMOX_REMOTE_REPO_DIR:=/root/nixos}" : "${PROXMOX_REMOTE_REPO_DIR:=/home/${PROXMOX_SSH_USER}/nixos}"
# Storage pool names -- Proxmox's own stock-install defaults, but this # Storage pool names -- Proxmox's own stock-install defaults, but this
# varies a lot by setup (ZFS pool name, custom LVM-thin volume, etc.). # varies a lot by setup (ZFS pool name, custom LVM-thin volume, etc.).
# Verify with `pvesm status` on the node and correct these if wrong. # Verify with `pvesm status` on the node and correct these if wrong.
: "${PROXMOX_STORAGE:=local-lvm}" # VM disks / CT rootfs : "${PROXMOX_STORAGE:=local-zfs}" # VM disks / CT rootfs
: "${PROXMOX_ISO_STORAGE:=local}" # uploaded images/ISOs/CT templates : "${PROXMOX_ISO_STORAGE:=local}" # uploaded images/ISOs/CT templates
: "${PROXMOX_BRIDGE:=vmbr0}" : "${PROXMOX_BRIDGE:=vmbr0}"
@@ -63,15 +73,21 @@
# nothing here forces a mount to happen. # nothing here forces a mount to happen.
: "${PROXMOX_DEFAULT_LXC_FEATURES:=nesting=1,keyctl=1,mount=nfs;nfs4}" : "${PROXMOX_DEFAULT_LXC_FEATURES:=nesting=1,keyctl=1,mount=nfs;nfs4}"
export PROXMOX_HOST PROXMOX_SSH_USER PROXMOX_STORAGE PROXMOX_ISO_STORAGE \ export PVE1_HOST PVE_TEST_HOST PROXMOX_HOST PROXMOX_SSH_USER PROXMOX_STORAGE \
PROXMOX_BRIDGE PROXMOX_DEFAULT_CORES PROXMOX_DEFAULT_MEMORY_MB \ PROXMOX_ISO_STORAGE PROXMOX_BRIDGE PROXMOX_DEFAULT_CORES \
PROXMOX_DEFAULT_LXC_DISK_GB PROXMOX_DEFAULT_LXC_FEATURES \ PROXMOX_DEFAULT_MEMORY_MB PROXMOX_DEFAULT_LXC_DISK_GB \
PROXMOX_REMOTE_REPO_DIR PROXMOX_DEFAULT_LXC_FEATURES PROXMOX_REMOTE_REPO_DIR
# Matches variables.nix's nixCacheHost -- update both if it ever changes. # Matches variables.nix's nixCacheHost -- update both if it ever changes.
: "${NIX_CACHE_HOST:=nix-cache}" : "${NIX_CACHE_HOST:=nix-cache}"
export NIX_CACHE_HOST export NIX_CACHE_HOST
# Matches variables.nix's lanDomain (the Gitea host this flake's own repo
# is served from -- see scripts/installer/auto-install.sh's FLAKE_BASE_URL)
# -- update both if it ever changes.
: "${LAN_DOMAIN:=gitea.lan.ddnsgeek.com}"
export LAN_DOMAIN
# nix_extra_opts: call as a plain statement (NOT inside $(...)/<(...) -- # nix_extra_opts: call as a plain statement (NOT inside $(...)/<(...) --
# that forks a subshell, and the whole point is exporting a decision back # that forks a subshell, and the whole point is exporting a decision back
# into *this* shell) to populate the global NIX_OPTS array with whatever # into *this* shell) to populate the global NIX_OPTS array with whatever
+206
View File
@@ -0,0 +1,206 @@
#!/usr/bin/env nix-shell
#!nix-shell -i bash -p jq disko nixos-install-tools zfs
# shellcheck shell=bash
# The only genuinely external tools this script calls directly: `jq`
# (parsing the `nix eval` host list), `disko`/`nixos-install` (the
# install itself), and `zpool` (exporting a ZFS root pool before reboot,
# see the comment above that call below). Everything disko shells out to
# internally (parted/sgdisk/mkfs.*/zfs/...) is self-contained -- disko's
# own generated scripts hardcode absolute Nix store paths for those, they
# don't rely on this script's PATH at all (confirmed by inspecting a
# generated system.build.formatScript). The built installer image
# (modules/installer/common.nix, plus the upstream
# installation-cd-minimal.nix it imports via iso.nix) already has all
# four in environment.systemPackages, so this nix-shell wrapper is a
# fast no-op there; it's what makes the script also work standalone
# (e.g. run directly from a checkout on a stock ISO), where they aren't
# guaranteed.
set -eux
set -euo pipefail
# shellcheck source=../env.sh
source "$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)/env.sh"
export FLAKE_BASE_URL="git+https://${LAN_DOMAIN}/beatzaplenty/nixos.git"
echo "Fetching available NixOS hosts from flake..."
# Two categories deliberately excluded from the menu:
# lxc-* — these build a config.system.build.tarball meant for
# `pct restore` on Proxmox directly, not an install.
# Running nixos-install against one here would
# bind-mount / onto /mnt and then refuse to touch the
# filesystem it's currently running on — see
# docs/auto-installer.md.
# installer — this *is* the installer image's own flake target,
# not a deployable host; "installing" it means
# nixos-install-ing a copy of the installer into
# itself.
mapfile -t options < <(
nix eval --json --no-use-registries --no-accept-flake-config --extra-experimental-features "flakes nix-command" \
"${FLAKE_BASE_URL}#nixosConfigurations" \
--apply builtins.attrNames \
| jq -r '.[]
| select(startswith("lxc-") | not)
| select(. != "installer")'
)
if [[ ${#options[@]} -eq 0 ]]; then
echo "ERROR: No NixOS hosts found in ${FLAKE_BASE_URL}#nixosConfigurations" >&2
exit 1
fi
echo "Note: lxc-* targets aren't installed this way — build them with"
echo " nix build .#nixosConfigurations.<name>.config.system.build.tarball"
echo "and 'pct restore' the result on Proxmox directly. See docs/auto-installer.md."
echo "Choose the flake profile to install:"
select choice in "${options[@]}"; do
if [[ -n "$choice" ]]; then
echo "You selected: $choice"
break
else
echo "Invalid selection. Try again."
fi
done
echo "Starting install with flake: ${FLAKE_BASE_URL}#${choice}"
# Optional: confirm before proceeding
read -rp "Proceed with installation? (y/N): " confirm
if [[ ! "$confirm" =~ ^[Yy]$ ]]; then
echo "Aborted."
exit 1
fi
# A nix-cache host is *the* substituter/remote-builder for every other
# host once installed (its own config explicitly excludes itself from
# using either — see buildType != "nix-cache" in the nixos flake.nix).
# Installing one shouldn't depend on a nix-cache substituter either,
# for the same reason — plus in practice "nix-cache" only resolves over
# Tailscale, which a fresh installer environment was never connected to
# anyway, so it's dead weight even for non-nix-cache installs until
# that's sorted out. Override it away here specifically for nix-cache
# targets to keep install-time behaviour consistent with run-time.
nix_extra_opts=()
if [[ "${choice}" == *-nix-cache ]]; then
echo "Installing a nix-cache host — skipping the nix-cache substituter."
nix_extra_opts+=(--option substituters "https://cache.nixos.org/")
fi
# Every host reachable through this menu has a Disko config (lxc-*
# is filtered out above, and is the only category that doesn't —
# see docs/auto-installer.md), so this can run unconditionally: no
# need to probe the flake first and branch on whether Disko applies.
disko --mode destroy,format,mount \
--flake "${FLAKE_BASE_URL}#${choice}" "${nix_extra_opts[@]}" --yes-wipe-all-disks
# sops-nix derives this host's decryption key from its own SSH host key
# at *activation* time, which runs before systemd would otherwise
# generate one on first boot. Without pre-seeding it here, secrets
# (including the login password) fail to decrypt on first boot.
# Generate the key with scripts/secrets/prepare-host-key.sh first.
#
# Two places a key can come from, checked in order:
# /etc/host-keys — baked into this image at build time (see
# modules/installer/host-keys.nix; only present
# if built with NIXOS_HOST_KEYS_DIR set)
# /root/host-keys — scp'd in manually after boot (older fallback,
# still supported for images built without keys)
mkdir -p /root/host-keys
if [[ -f "/etc/host-keys/${choice}_ssh_host_ed25519_key" ]]; then
echo "Found baked-in SSH host key for ${choice}, installing to target..."
install -D -m 0600 "/etc/host-keys/${choice}_ssh_host_ed25519_key" /mnt/etc/ssh/ssh_host_ed25519_key
install -D -m 0644 "/etc/host-keys/${choice}_ssh_host_ed25519_key.pub" /mnt/etc/ssh/ssh_host_ed25519_key.pub
elif [[ -f "/root/host-keys/${choice}_ssh_host_ed25519_key" ]]; then
echo "Found pre-seeded SSH host key for ${choice}, installing to target..."
install -D -m 0600 "/root/host-keys/${choice}_ssh_host_ed25519_key" /mnt/etc/ssh/ssh_host_ed25519_key
install -D -m 0644 "/root/host-keys/${choice}_ssh_host_ed25519_key.pub" /mnt/etc/ssh/ssh_host_ed25519_key.pub
else
# Third place a key can come from: an arbitrary path the operator
# points at interactively (e.g. a USB stick, a mount from another
# machine) -- only offered when there's an actual human at the other
# end of stdin to ask, never in a non-interactive run.
key_copied=0
if [[ -t 0 ]]; then
echo "No SSH host key found for ${choice} (checked /etc/host-keys and /root/host-keys)."
read -rp "Path to a directory containing ${choice}_ssh_host_ed25519_key(.pub) (blank to skip): " key_src_dir
if [[ -n "$key_src_dir" && -f "${key_src_dir}/${choice}_ssh_host_ed25519_key" && -f "${key_src_dir}/${choice}_ssh_host_ed25519_key.pub" ]]; then
cp "${key_src_dir}/${choice}_ssh_host_ed25519_key" "${key_src_dir}/${choice}_ssh_host_ed25519_key.pub" /root/host-keys/
key_copied=1
elif [[ -n "$key_src_dir" ]]; then
echo "WARNING: ${choice}_ssh_host_ed25519_key(.pub) not found in ${key_src_dir}."
fi
fi
if [[ "$key_copied" -eq 1 ]]; then
echo "Copied SSH host key for ${choice} from ${key_src_dir}, installing to target..."
install -D -m 0600 "/root/host-keys/${choice}_ssh_host_ed25519_key" /mnt/etc/ssh/ssh_host_ed25519_key
install -D -m 0644 "/root/host-keys/${choice}_ssh_host_ed25519_key.pub" /mnt/etc/ssh/ssh_host_ed25519_key.pub
else
echo "WARNING: no SSH host key found for ${choice} (checked /etc/host-keys and /root/host-keys)"
echo "sops-nix secrets (including the login password) will NOT decrypt on first boot."
echo "Run scripts/secrets/prepare-host-key.sh for host ${choice} on your admin workstation first,"
echo "then either rebuild this image with NIXOS_HOST_KEYS_DIR set, scp the result to"
echo "/root/host-keys/ on this machine, or point at it when prompted above."
read -rp "Continue without a pre-seeded key anyway? (y/N): " skip_key
if [[ ! "$skip_key" =~ ^[Yy]$ ]]; then
echo "Aborted."
exit 1
fi
fi
fi
mkdir -p /mnt/install-tmp
export TMPDIR=/mnt/install-tmp
nixos-install \
--flake "${FLAKE_BASE_URL}#${choice}" \
"${nix_extra_opts[@]}" \
--no-root-password
rm -rf /mnt/install-tmp
# Redundant copy of the host's private key — the real one is now at
# /etc/ssh/ssh_host_ed25519_key. Nothing NixOS-managed ever cleans this
# up on its own since it was written imperatively, not declaratively.
rm -rf /root/host-keys
# disko's --mode ...,mount left any ZFS root pool imported (that's what
# let nixos-install write into /mnt). If we reboot with it still
# imported, it isn't just "not exported" -- it's stamped with *this*
# live installer environment's hostid, which almost never matches the
# target's own networking.hostId (see hosts/*/host.nix; the installer
# itself sets none). modules/services/zfs/enable-service.nix and
# modules/common/configuration.nix both set boot.zfs.forceImportRoot =
# false deliberately (the safe option per that setting's own docs), so
# the freshly-installed system's first real boot sees a pool "in use by
# another system" and refuses to import it without -f -- which is what
# makes boot stall waiting on the ZFS import. Exporting here (a no-op
# if the chosen host has no ZFS root, e.g. proxmox-*/linode-*) clears
# that in-use state so the next import, from any hostid, succeeds.
#
# Anything still mounted under /mnt -- nixos-install's own leftover
# chroot bind mounts for running the target's activation script
# (/mnt/dev, /mnt/proc, /mnt/sys, /mnt/run), and disko's own /mnt/boot
# ESP mount (modules/disko/baremetal.nix) -- blocks ZFS from unmounting
# its root dataset at /mnt, the same way any nested mount blocks
# unmounting its parent. Confirmed live: zpool export failed with
# "cannot unmount '/mnt': pool or dataset busy" even after handling the
# chroot mounts alone, because /mnt/boot was still mounted too. Because
# of this script's `set -e`, that killed the script before it ever
# reached reboot, silently defeating the whole point of exporting first.
# Unmounting everything under /mnt up front (recursively, so nested
# mounts like /mnt/dev/pts come along for free) sidesteps needing to
# enumerate every mount disko/nixos-install might leave behind.
if mountpoint -q /mnt; then
umount -R /mnt
fi
if [[ -n "$(zpool list -H -o name 2>/dev/null)" ]]; then
echo "Exporting ZFS pool(s) before reboot..."
zpool export -a
fi
sleep 10
reboot
+21
View File
@@ -0,0 +1,21 @@
#!/usr/bin/env bash
# Shared "type X to confirm" prompt for scripts/proxmox/create-proxmox-resource.sh
# (--modify, and replacing an existing --allow-duplicate-host resource) and
# scripts/secrets/sync-host-keys.sh (--regenerate-all-keys) -- three destructive
# confirmations that all work the same way (echo the expected value back
# exactly), kept in one place so the prompt/comparison logic can't drift.
# Source alongside env.sh:
# source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/lib/confirm.sh"
#
# Deliberately does NOT print anything on mismatch or decide exit-vs-return
# -- callers vary on both (a top-level script exits, a subcommand function
# returns; wording differs too), so that stays at the call site.
# confirm_typed <expected> <prompt>
# Prints <prompt> via `read -rp`, then reports (via exit status) whether the
# typed input matched <expected> exactly.
confirm_typed() {
local expected="$1" prompt="$2" input
read -rp "$prompt" input
[[ "$input" == "$expected" ]]
}
+18
View File
@@ -34,3 +34,21 @@ flake_target_hostname() {
nix eval --raw "${NIX_EVAL_FLAGS[@]}" \ nix eval --raw "${NIX_EVAL_FLAGS[@]}" \
"${flake_ref}#nixosConfigurations.${target}.config.networking.hostName" 2>/dev/null "${flake_ref}#nixosConfigurations.${target}.config.networking.hostName" 2>/dev/null
} }
# flake_target_lxc_privileged <flake_ref> <target>
# Prints "true" or "false" for one lxc-* target's config.proxmoxLXC.privileged
# (modules/platforms/lxc.nix is the single source of truth -- e.g.
# lxc-docker sets this true so it can NFS-mount; every other lxc-* host
# stays unprivileged). Only meaningful for lxc-* targets -- the option
# doesn't exist for linode-*/proxmox-* (nixpkgs' proxmox-lxc.nix, which
# declares it, is only ever imported by modules/platforms/lxc.nix). Empty
# (not an error under set -e) if the eval fails.
flake_target_lxc_privileged() {
local flake_ref="$1" target="$2"
# Not --raw: the option is a Nix boolean, and --raw can only coerce
# strings ("cannot coerce a Boolean to a string"). Plain `nix eval`
# prints a bare `true`/`false` for a boolean, which is exactly the
# string this needs.
nix eval "${NIX_EVAL_FLAGS[@]}" \
"${flake_ref}#nixosConfigurations.${target}.config.proxmoxLXC.privileged" 2>/dev/null
}
+95
View File
@@ -0,0 +1,95 @@
#!/usr/bin/env bash
# Shared parallel-nix-invocation helper for scripts/codex-maintenance.sh.
# Source alongside nix-eval.sh:
# source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/lib/nix-parallel.sh"
#
# The per-host/per-package `nix eval`/`nix build --dry-run` calls in
# codex-maintenance.sh are independent of each other, so running them one at
# a time leaves most cores idle for most of the sweep -- run_nix_parallel
# fans a batch of them out across up to NIX_PARALLEL_JOBS processes instead.
# NIX_PARALLEL_JOBS: how many `nix` invocations run_nix_parallel runs at
# once. Defaults to core count capped by available memory (~1GB/job,
# floor 1) rather than plain `nproc` -- each concurrent `nix eval` here
# evaluates a whole NixOS system closure from scratch, and on a small/
# memory-constrained CI runner, `nproc` concurrent evals can OOM-kill each
# other (confirmed empirically: on a 4GB/6-core box, 5-6 concurrent evals
# started getting killed while 3-4 ran clean and were still ~2x faster than
# serial). Override via env if a given machine/CI runner has room to spare
# or needs a tighter cap.
default_nix_parallel_jobs() {
local cores mem_avail_kb mem_cap
cores="$(nproc 2>/dev/null || echo 4)"
mem_avail_kb="$(awk '/^MemAvailable:/ {print $2}' /proc/meminfo 2>/dev/null)"
if [[ -z "$mem_avail_kb" ]]; then
echo "$cores"
return
fi
mem_cap=$((mem_avail_kb / 1024 / 1024))
((mem_cap < 1)) && mem_cap=1
((mem_cap < cores)) && echo "$mem_cap" || echo "$cores"
}
NIX_PARALLEL_JOBS="${NIX_PARALLEL_JOBS:-$(default_nix_parallel_jobs)}"
# Separator between a job's label and its flake attr in the arrays
# run_nix_parallel takes -- a control character so it can't collide with
# anything a label or attr path would plausibly contain.
NIX_PARALLEL_SEP=$'\x1f'
# run_nix_parallel <jobs_array_name> <nix subcommand + flags...>
#
# jobs_array_name: name of an already-populated bash array whose entries are
# "<label>${NIX_PARALLEL_SEP}<attr>" pairs, e.g.
# jobs=("proxmox-docker${NIX_PARALLEL_SEP}.#nixosConfigurations.proxmox-docker...drvPath")
# Remaining args are passed to `nix` before the attr, e.g.:
# run_nix_parallel jobs eval --raw "${NIX_EVAL_FLAGS[@]}"
# run_nix_parallel jobs build --dry-run --no-link "${NIX_EVAL_FLAGS[@]}"
#
# Prints "==> <label>" followed by that job's stdout+stderr for every job,
# in submission order (not completion order) so a run stays readable and
# diffable across invocations even though the work itself doesn't finish in
# that order. Returns non-zero if any job failed, only after every job has
# finished and been printed -- same "surface everything, then fail" contract
# a `set -e` caller gets, just parallelized instead of stopping at the first
# failure.
run_nix_parallel() {
local -n jobs_ref="$1"
shift
local -a nix_args=("$@")
local n=${#jobs_ref[@]}
[[ $n -eq 0 ]] && return 0
local tmp_dir
tmp_dir="$(mktemp -d)"
local i=0 running=0
for job in "${jobs_ref[@]}"; do
local attr="${job#*"${NIX_PARALLEL_SEP}"}"
printf '%s\n' "${job%%"${NIX_PARALLEL_SEP}"*}" >"${tmp_dir}/${i}.label"
(
if nix "${nix_args[@]}" "$attr" >"${tmp_dir}/${i}.out" 2>&1; then
echo 0 >"${tmp_dir}/${i}.status"
else
echo 1 >"${tmp_dir}/${i}.status"
fi
) &
i=$((i + 1))
running=$((running + 1))
if ((running >= NIX_PARALLEL_JOBS)); then
wait -n
running=$((running - 1))
fi
done
wait
local failed=0 j
for ((j = 0; j < n; j++)); do
echo "==> $(cat "${tmp_dir}/${j}.label")"
cat "${tmp_dir}/${j}.out"
[[ "$(cat "${tmp_dir}/${j}.status")" -ne 0 ]] && failed=1
done
rm -rf "$tmp_dir"
return $failed
}
+52
View File
@@ -0,0 +1,52 @@
#!/usr/bin/env bash
# Shared sops/age helpers for scripts/secrets/backup-admin-key.sh,
# scripts/secrets/rotate-admin-key.sh, and scripts/secrets/sync-host-keys.sh -- all three
# derive an age public key from a private identity file the same way, two
# of them resolve the same sops/age default key-file path, and two of them
# run `sops updatekeys` the same way. Kept in one place so they can't drift
# apart. Source alongside env.sh:
# source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/lib/sops-age.sh"
#
# Uses NIX_OPTS (an array of extra `nix-shell` options -- see env.sh's
# nix_extra_opts) if the caller has already set it, same convention as
# lib/ssh-host-keys.sh. Falls back to no extra options if the caller never
# sourced env.sh.
if ! declare -p NIX_OPTS >/dev/null 2>&1; then
declare -a NIX_OPTS=()
fi
# sops/age's own default identity-file resolution order, minus $SOPS_AGE_KEY
# itself (an inline identity, not a path -- callers that accept it check it
# separately, before falling back to this).
: "${DEFAULT_SOPS_AGE_KEY_FILE:=${SOPS_AGE_KEY_FILE:-${XDG_CONFIG_HOME:-$HOME/.config}/sops/age/keys.txt}}"
# age_pubkey_from_identity_file <identity-file>
# Prints the age public key for a private identity file (age-keygen -y).
age_pubkey_from_identity_file() {
local identity_file="$1"
nix-shell "${NIX_OPTS[@]}" -p age --run "age-keygen -y '${identity_file}'"
}
# sops_yaml_admin_pubkey <sops-yaml-path>
# Prints .sops.yaml's current &admin age public key, or empty (not an error
# under set -e) if no such anchor line exists -- callers that need to treat
# "missing" as fatal check for an empty result themselves.
sops_yaml_admin_pubkey() {
local sops_yaml="$1"
grep -E '^ - &admin age1' "$sops_yaml" 2>/dev/null | awk '{print $NF}' || true
}
# sops_updatekeys <secrets-file> [key-file]
# Re-encrypts <secrets-file> for .sops.yaml's current recipient set. If
# <key-file> is given, decrypts with that identity (SOPS_AGE_KEY_FILE)
# instead of whatever's ambient -- needed when the ambient default key
# doesn't match yet (e.g. mid-rotation, decrypting with the outgoing key).
sops_updatekeys() {
local secrets_file="$1" key_file="${2:-}"
if [[ -n "$key_file" ]]; then
SOPS_AGE_KEY_FILE="$key_file" nix-shell "${NIX_OPTS[@]}" -p sops --run \
"sops updatekeys --yes '${secrets_file}'"
else
nix-shell "${NIX_OPTS[@]}" -p sops --run "sops updatekeys --yes '${secrets_file}'"
fi
}
+2 -2
View File
@@ -1,6 +1,6 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Shared SSH-host-key / age-conversion helpers for scripts/sync-host-keys.sh # Shared SSH-host-key / age-conversion helpers for scripts/secrets/sync-host-keys.sh
# and scripts/prepare-host-key.sh -- both generate the same kind of key # and scripts/secrets/prepare-host-key.sh -- both generate the same kind of key
# (ed25519, no passphrase, the sops-nix age-derivation input) and convert it # (ed25519, no passphrase, the sops-nix age-derivation input) and convert it
# to an age recipient the same way; kept in one place so the two can't # to an age recipient the same way; kept in one place so the two can't
# drift apart. # drift apart.
+221
View File
@@ -0,0 +1,221 @@
#!/usr/bin/env bash
# Ad hoc clone of a single VM/CT from pve1 (production) to pve-test
# (sandbox), via vzdump + qmrestore/pct restore -- not a general-purpose
# backup tool, just a quick "give me a disposable copy of this thing on
# pve-test" for testing against real-ish data without touching prod.
#
# Flow:
# 1. vzdump the resource on pve1 into its "local" storage (--mode
# snapshot by default, so the source keeps running throughout --
# see --mode below for when that's not possible).
# 2. Stream the resulting archive straight from pve1 to pve-test
# (ssh pve1 cat ... | ssh pve-test cat > ...) -- this machine is
# just the relay, no separate on-disk staging copy here.
# 3. qmrestore / pct restore it on pve-test under --new-vmid (default:
# same VMID as the source -- pve-test is a separate node/cluster, so
# no collision unless that VMID is already in use there too).
# Always restored with --unique 1 (fresh MAC addresses) since the
# source is typically still running on the same LAN -- restoring
# with the *same* MAC would put two live guests on the wire with
# identical hardware addresses.
# 4. Delete the vzdump archive from pve1's local storage and the
# relayed copy on pve-test, so neither node accumulates ad hoc
# backup files from this script. Only the pve1 original is
# preserved on any failure after step 1, so a failed
# transfer/restore can be retried without re-running the backup.
#
# This script's own defaults are pve1 -> pve-test, unlike
# create-proxmox-resource.sh's --node (which defaults to production) --
# see CLAUDE.md's "Two Proxmox nodes" section. pve1 is only ever touched
# here after typing the source VMID back to confirm; pve-test is treated
# as disposable, matching this repo's usual policy for that node.
#
# See --help for the full option list.
set -euo pipefail
repo_root="$(cd "$(dirname "$0")/../.." && pwd)"
# shellcheck source=../env.sh
source "${repo_root}/scripts/env.sh"
# shellcheck source=../lib/confirm.sh
source "${repo_root}/scripts/lib/confirm.sh"
usage() {
cat <<EOF
Usage: $0 --vmid <n> [options]
--vmid <n> Required: VMID on the source node to clone.
Kind (qemu VM vs LXC CT) is auto-detected.
--new-vmid <n> VMID to restore as on the target node
(default: same as --vmid).
--mode snapshot|suspend|stop
vzdump backup mode (default: snapshot -- the
source resource keeps running throughout;
requires snapshot-capable storage, e.g.
ZFS/LVM-thin/Ceph/qcow2). Fall back to
"suspend" (brief pause) or "stop" (source
goes down for the duration) if the source's
storage doesn't support live snapshots --
vzdump's own error will say so.
--source-node <host> (default: \$PVE1_HOST, ${PVE1_HOST})
--target-node <host> (default: \$PVE_TEST_HOST, ${PVE_TEST_HOST})
--source-storage <pool> Where vzdump writes the backup on the
source node (default: local).
--target-storage <pool> Where the restored disk/rootfs lands on
the target node (default: \$PROXMOX_STORAGE, ${PROXMOX_STORAGE}).
--keep-backup Don't delete the vzdump archive from
either node afterward (debugging aid).
--yes Skip the typed VMID confirmation
before touching the source node.
--dry-run Print the full plan and skip every
mutating step (vzdump, transfer,
restore, delete) and the confirm
prompt. Still makes read-only SSH
calls to look up the source kind
and check the target VMID is free
-- harmless on either node.
-h, --help
EOF
}
vmid=""
new_vmid=""
mode="snapshot"
source_node="$PVE1_HOST"
target_node="$PVE_TEST_HOST"
source_storage="local"
target_storage="$PROXMOX_STORAGE"
keep_backup=0
skip_confirm=0
dry_run=0
while [[ $# -gt 0 ]]; do
case "$1" in
--vmid) vmid="$2"; shift 2 ;;
--new-vmid) new_vmid="$2"; shift 2 ;;
--mode) mode="$2"; shift 2 ;;
--source-node) source_node="$2"; shift 2 ;;
--target-node) target_node="$2"; shift 2 ;;
--source-storage) source_storage="$2"; shift 2 ;;
--target-storage) target_storage="$2"; shift 2 ;;
--keep-backup) keep_backup=1; shift ;;
--yes) skip_confirm=1; shift ;;
--dry-run) dry_run=1; shift ;;
-h | --help) usage; exit 0 ;;
*) echo "Unknown option: $1" >&2; usage >&2; exit 1 ;;
esac
done
if [[ -z "$vmid" ]]; then
echo "ERROR: --vmid is required." >&2
usage >&2
exit 1
fi
if [[ "$mode" != "snapshot" && "$mode" != "suspend" && "$mode" != "stop" ]]; then
echo "ERROR: --mode must be snapshot, suspend, or stop." >&2
exit 1
fi
[[ -z "$new_vmid" ]] && new_vmid="$vmid"
source_target="${PROXMOX_SSH_USER}@${source_node}"
target_target="${PROXMOX_SSH_USER}@${target_node}"
# No dry-run wrapper needed for the calls below: every mutating step
# (vzdump, transfer, restore, delete) is reached only after the --dry-run
# early-exit further down, so a plain `ssh` call is never in the dry-run
# path.
# --- identify the resource kind on the source node -----------------------
echo "==> Looking up VMID ${vmid} on ${source_node}..."
kind=""
if ssh "$source_target" "qm status ${vmid}" >/dev/null 2>&1; then
kind="vm"
elif ssh "$source_target" "pct status ${vmid}" >/dev/null 2>&1; then
kind="lxc"
else
echo "ERROR: VMID ${vmid} doesn't exist on ${source_node} as either a VM or CT." >&2
exit 1
fi
echo "VMID ${vmid} on ${source_node} is a ${kind}."
# --- refuse to clobber an existing resource on the target node -----------
if ssh "$target_target" "qm status ${new_vmid}" >/dev/null 2>&1 \
|| ssh "$target_target" "pct status ${new_vmid}" >/dev/null 2>&1; then
echo "ERROR: VMID ${new_vmid} already exists on ${target_node}. Pass --new-vmid" >&2
echo "with a free ID, or remove the existing resource there first." >&2
exit 1
fi
echo
echo "Plan:"
echo " source: ${kind} VMID ${vmid} on ${source_node} (storage: ${source_storage}, mode: ${mode})"
echo " target: VMID ${new_vmid} on ${target_node} (storage: ${target_storage}, fresh MAC via --unique)"
[[ "$keep_backup" -eq 1 ]] && echo " backup archives are kept on both nodes afterward (--keep-backup)"
if [[ "$dry_run" -eq 1 ]]; then
echo
echo "[dry-run] No backup, transfer, restore, or delete was performed."
exit 0
fi
if [[ "$skip_confirm" -ne 1 ]]; then
echo
if ! confirm_typed "$vmid" "Type the source VMID (${vmid}) to confirm backing it up from ${source_node}: "; then
echo "Cancelled -- input didn't match ${vmid}." >&2
exit 1
fi
fi
# --- vzdump on the source node --------------------------------------------
echo
echo "==> Backing up VMID ${vmid} on ${source_node} (mode=${mode}, storage=${source_storage})..."
vzdump_log="$(ssh "$source_target" \
"vzdump ${vmid} --mode ${mode} --storage ${source_storage} --compress zstd" 2>&1)" \
|| {
echo "$vzdump_log" >&2
echo "ERROR: vzdump failed on ${source_node}." >&2
exit 1
}
echo "$vzdump_log"
archive="$(echo "$vzdump_log" | grep -oP "creating vzdump archive '\K[^']+" | tail -n1)"
if [[ -z "$archive" ]]; then
echo "ERROR: couldn't find the archive path in vzdump's output above." >&2
exit 1
fi
archive_basename="$(basename "$archive")"
target_tmp_archive="/var/tmp/${archive_basename}"
echo "Archive: ${archive}"
# Always clean up the relayed copy on the target node, success or failure
# -- it's only ever a working copy, restored or not.
cleanup_target_tmp() {
if [[ "$keep_backup" -ne 1 ]]; then
ssh "$target_target" "rm -f '${target_tmp_archive}'" >/dev/null 2>&1 || true
fi
}
trap cleanup_target_tmp EXIT
# --- relay the archive from source to target ------------------------------
echo
echo "==> Transferring archive to ${target_node}..."
ssh "$source_target" "cat '${archive}'" | ssh "$target_target" "cat > '${target_tmp_archive}'"
# --- restore on the target node --------------------------------------------
echo
echo "==> Restoring as VMID ${new_vmid} on ${target_node} (storage=${target_storage})..."
if [[ "$kind" == "vm" ]]; then
ssh "$target_target" "qmrestore '${target_tmp_archive}' ${new_vmid} --storage ${target_storage} --unique 1"
else
ssh "$target_target" "pct restore ${new_vmid} '${target_tmp_archive}' --storage ${target_storage} --unique 1"
fi
# --- clean up the source backup now that the restore succeeded -----------
if [[ "$keep_backup" -ne 1 ]]; then
echo
echo "==> Deleting backup archive from ${source_node}'s ${source_storage} storage..."
ssh "$source_target" "rm -f '${archive}' '${archive}.notes' '${archive}.log'" >/dev/null 2>&1 || true
fi
echo
echo "Done. VMID ${new_vmid} (${kind}) is now on ${target_node}, cloned from" \
"VMID ${vmid} on ${source_node}."
@@ -6,17 +6,31 @@
# #
# This is the non-NixOS equivalent of modules/nix-cache/client.nix + # This is the non-NixOS equivalent of modules/nix-cache/client.nix +
# modules/nix-cache/remote-builder-client.nix -- those two only apply to # modules/nix-cache/remote-builder-client.nix -- those two only apply to
# hosts built from this flake. A plain Debian box with Nix installed # hosts built from this flake. A plain Debian box with Nix installed has no
# (single- or multi-user install, nix-daemon running) has no NixOS module # NixOS module system to pick that config up, so this edits nix.conf by hand.
# system to pick that config up, so this edits /etc/nix/nix.conf by hand #
# instead. Run this ON the target Debian machine, as root. # Two modes depending on who runs it:
#
# root (multi-user / daemon install):
# Writes /etc/nix/nix.conf, /etc/ssh/ssh_known_hosts, restarts nix-daemon.
# Requires /etc/nix/nix.conf to already exist (i.e. nix-daemon is set up).
# Run as: sudo ./configure-nix-cache-client.sh [options]
#
# non-root (single-user install):
# Writes ~/.config/nix/nix.conf, ~/.ssh/known_hosts. No daemon to restart.
# Run as: ./configure-nix-cache-client.sh [options]
# #
# The values below mirror variables.nix / modules/nix-cache/client.nix in # The values below mirror variables.nix / modules/nix-cache/client.nix in
# this repo -- update both if nix-cache is ever rebuilt with a new host # this repo -- update both if nix-cache is ever rebuilt with a new host
# key or the cache signing key is rotated (see docs/nix-cache.md). # key or the cache signing key is rotated (see docs/nix-cache.md).
# #
# REMOTE_BUILDER_KEY defaults to the running user's default SSH identity
# (root: /root/.ssh/id_ed25519, other user: ~/.ssh/id_ed25519). That key
# must be listed in vars.remoteBuilderAuthorizedKeys in this repo and
# nix-cache rebuilt before remote building works.
#
# Usage: # Usage:
# sudo ./configure-nix-cache-client.sh [--dry-run] [--no-remote-builder] [--no-restart] # ./configure-nix-cache-client.sh [--dry-run] [--no-remote-builder] [--no-restart]
# #
# Env overrides (defaults match variables.nix): # Env overrides (defaults match variables.nix):
# NIX_CACHE_HOST, NIX_CACHE_HOST_KEY, REMOTE_BUILDER_USER, REMOTE_BUILDER_KEY # NIX_CACHE_HOST, NIX_CACHE_HOST_KEY, REMOTE_BUILDER_USER, REMOTE_BUILDER_KEY
@@ -24,19 +38,30 @@
set -euo pipefail set -euo pipefail
: "${NIX_CACHE_HOST:=nix-cache}" : "${NIX_CACHE_HOST:=nix-cache}"
: "${NIX_CACHE_HOST_KEY:=ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHrMKZlIGUd3pH9G3AqbsruqUGjxIXMAZw52u9MwiBCn lxc-nix-cache}" : "${NIX_CACHE_HOST_KEY:=ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPeWgMsdaiz4axT/deFc1+0B5bN+GX/NOeW9bbQ0c/IT lxc-nix-cache}"
: "${REMOTE_BUILDER_USER:=nixremote}" : "${REMOTE_BUILDER_USER:=nixremote}"
: "${REMOTE_BUILDER_KEY:=/root/.ssh/nixremote}"
CACHE_PUB_KEY="cache.local-1:usoWYanY3Kpq2+kDIS2nhWoLZiRxanmdysdzqCFBHW4=" CACHE_PUB_KEY="cache.local-1:usoWYanY3Kpq2+kDIS2nhWoLZiRxanmdysdzqCFBHW4="
FALLBACK_URL="https://cache.nixos.org/" FALLBACK_URL="https://cache.nixos.org/"
FALLBACK_PUB_KEY="cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=" FALLBACK_PUB_KEY="cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY="
NIX_CONF="/etc/nix/nix.conf"
KNOWN_HOSTS="/etc/ssh/ssh_known_hosts"
MARKER_BEGIN="# BEGIN nix-cache client config (configure-nix-cache-client.sh)" MARKER_BEGIN="# BEGIN nix-cache client config (configure-nix-cache-client.sh)"
MARKER_END="# END nix-cache client config" MARKER_END="# END nix-cache client config"
# Mode: root uses system-wide paths and restarts the daemon; non-root uses
# user-level paths and has no daemon to restart.
if [[ "$EUID" -eq 0 ]]; then
install_mode="multi"
NIX_CONF="/etc/nix/nix.conf"
KNOWN_HOSTS="/etc/ssh/ssh_known_hosts"
: "${REMOTE_BUILDER_KEY:=/root/.ssh/id_ed25519}"
else
install_mode="single"
NIX_CONF="${XDG_CONFIG_HOME:-$HOME/.config}/nix/nix.conf"
KNOWN_HOSTS="$HOME/.ssh/known_hosts"
: "${REMOTE_BUILDER_KEY:=$HOME/.ssh/id_ed25519}"
fi
dry_run=0 dry_run=0
with_remote_builder=1 with_remote_builder=1
restart_daemon=1 restart_daemon=1
@@ -47,7 +72,7 @@ for arg in "$@"; do
--no-remote-builder) with_remote_builder=0 ;; --no-remote-builder) with_remote_builder=0 ;;
--no-restart) restart_daemon=0 ;; --no-restart) restart_daemon=0 ;;
-h|--help) -h|--help)
sed -n '2,20p' "$0" sed -n '2,37p' "$0"
exit 0 exit 0
;; ;;
*) *)
@@ -57,21 +82,22 @@ for arg in "$@"; do
esac esac
done done
if [[ "$dry_run" -eq 0 && "$EUID" -ne 0 ]]; then
echo "ERROR: must run as root (writes $NIX_CONF and, unless --no-remote-builder, $KNOWN_HOSTS)." >&2
exit 1
fi
if ! command -v nix >/dev/null 2>&1; then if ! command -v nix >/dev/null 2>&1; then
echo "ERROR: no 'nix' binary on PATH -- install the Nix package manager first." >&2 echo "ERROR: no 'nix' binary on PATH -- install the Nix package manager first." >&2
exit 1 exit 1
fi fi
if [[ ! -f "$NIX_CONF" ]]; then if [[ "$install_mode" == "multi" && ! -f "$NIX_CONF" ]]; then
echo "ERROR: $NIX_CONF not found -- expected an existing multi-user Nix install." >&2 echo "ERROR: $NIX_CONF not found -- expected an existing multi-user Nix install." >&2
exit 1 exit 1
fi fi
# Single-user: create the config file if it doesn't exist yet.
if [[ "$install_mode" == "single" && "$dry_run" -eq 0 ]]; then
mkdir -p "$(dirname "$NIX_CONF")"
[[ -f "$NIX_CONF" ]] || touch "$NIX_CONF"
fi
builder_line="" builder_line=""
if [[ "$with_remote_builder" -eq 1 ]]; then if [[ "$with_remote_builder" -eq 1 ]]; then
if [[ -f "$REMOTE_BUILDER_KEY" ]]; then if [[ -f "$REMOTE_BUILDER_KEY" ]]; then
@@ -107,7 +133,7 @@ fi
block="${block} block="${block}
$MARKER_END" $MARKER_END"
echo "== nix.conf block to install ==" echo "== nix.conf block to install ($NIX_CONF) =="
echo "$block" echo "$block"
echo "================================" echo "================================"
@@ -149,7 +175,8 @@ if [[ "$with_remote_builder" -eq 1 ]]; then
fi fi
fi fi
if [[ "$dry_run" -eq 0 && "$restart_daemon" -eq 1 ]]; then # Only restart the daemon for multi-user installs -- single-user has no daemon.
if [[ "$dry_run" -eq 0 && "$restart_daemon" -eq 1 && "$install_mode" == "multi" ]]; then
if command -v systemctl >/dev/null 2>&1 && systemctl is-active --quiet nix-daemon 2>/dev/null; then if command -v systemctl >/dev/null 2>&1 && systemctl is-active --quiet nix-daemon 2>/dev/null; then
systemctl restart nix-daemon systemctl restart nix-daemon
echo "Restarted nix-daemon to pick up the new config." echo "Restarted nix-daemon to pick up the new config."
@@ -12,10 +12,21 @@
# copies over the locally-managed host-keys/ (gitignored, so a git pull # copies over the locally-managed host-keys/ (gitignored, so a git pull
# alone wouldn't carry it) before building. # alone wouldn't carry it) before building.
# #
# --node (default: $PROXMOX_HOST, see scripts/env.sh) picks which of the two
# LAN Proxmox nodes this runs against: production, pve1.sweet.home
# ($PVE1_HOST, PROXMOX_HOST's own default), or the sandbox node,
# pve-test.sweet.home ($PVE_TEST_HOST) -- pass --node "$PVE_TEST_HOST" (or
# set PROXMOX_HOST=$PVE_TEST_HOST) to target the sandbox instead. See
# CLAUDE.md's "Two Proxmox nodes" section: an agent session should default
# to pve-test and only touch pve1 when the operator has explicitly said so
# for the current task -- this script itself doesn't enforce that (its own
# default is production, matching this repo's behavior before pve-test
# existed), it's a policy for whoever/whatever is driving it.
#
# Usage: # Usage:
# scripts/create-proxmox-resource.sh --type lxc|vm --host <name> [options] # scripts/proxmox/create-proxmox-resource.sh --type lxc|vm --host <name> [options]
# scripts/create-proxmox-resource.sh --type lxc|vm --list # scripts/proxmox/create-proxmox-resource.sh --type lxc|vm --list
# scripts/create-proxmox-resource.sh --modify --vmid <n> [--cores N] [--memory MB] [--grow-disk GB] # scripts/proxmox/create-proxmox-resource.sh --modify --vmid <n> [--cores N] [--memory MB] [--grow-disk GB]
# #
# SAFETY: # SAFETY:
# - The default (create) mode only ever creates a NEW resource -- it # - The default (create) mode only ever creates a NEW resource -- it
@@ -41,13 +52,15 @@
# See --help for the full option list. # See --help for the full option list.
set -euo pipefail set -euo pipefail
repo_root="$(cd "$(dirname "$0")/.." && pwd)" repo_root="$(cd "$(dirname "$0")/../.." && pwd)"
# shellcheck source=env.sh # shellcheck source=../env.sh
source "${repo_root}/scripts/env.sh" source "${repo_root}/scripts/env.sh"
# shellcheck source=lib/nix-eval.sh # shellcheck source=../lib/nix-eval.sh
source "${repo_root}/scripts/lib/nix-eval.sh" source "${repo_root}/scripts/lib/nix-eval.sh"
# shellcheck source=../lib/confirm.sh
source "${repo_root}/scripts/lib/confirm.sh"
sync_keys="${repo_root}/scripts/sync-host-keys.sh" sync_keys="${repo_root}/scripts/secrets/sync-host-keys.sh"
usage() { usage() {
cat <<EOF cat <<EOF
@@ -118,7 +131,9 @@ Shared:
--iso-storage <pool> (default: \$PROXMOX_ISO_STORAGE, ${PROXMOX_ISO_STORAGE}) --iso-storage <pool> (default: \$PROXMOX_ISO_STORAGE, ${PROXMOX_ISO_STORAGE})
--bridge <bridge> (default: \$PROXMOX_BRIDGE, ${PROXMOX_BRIDGE}) --bridge <bridge> (default: \$PROXMOX_BRIDGE, ${PROXMOX_BRIDGE})
--node <host> Proxmox node to SSH into (default: --node <host> Proxmox node to SSH into (default:
\$PROXMOX_HOST, ${PROXMOX_HOST}) \$PROXMOX_HOST, ${PROXMOX_HOST} --
production; the sandbox node is
\$PVE_TEST_HOST, ${PVE_TEST_HOST}).
--dry-run Print the full plan; touch nothing --dry-run Print the full plan; touch nothing
local or remote, no prompts. local or remote, no prompts.
-h, --help -h, --help
@@ -178,6 +193,16 @@ done
ssh_target="${PROXMOX_SSH_USER}@${node}" ssh_target="${PROXMOX_SSH_USER}@${node}"
# Proxmox tools (pvesh, qm, pct) require root access to the cluster IPC
# socket. When SSH-ing as a non-root user with sudo, prefix every remote
# Proxmox command with sudo.
sudo_prefix=""
sudo_display=""
if [[ "$PROXMOX_SSH_USER" != "root" ]]; then
sudo_prefix="sudo"
sudo_display="sudo "
fi
remote() { remote() {
if [[ "$dry_run" -eq 1 ]]; then if [[ "$dry_run" -eq 1 ]]; then
echo "[dry-run] ssh ${ssh_target} -- $*" echo "[dry-run] ssh ${ssh_target} -- $*"
@@ -199,10 +224,10 @@ cmd_modify() {
echo "Looking up VMID ${vmid} on ${node}..." echo "Looking up VMID ${vmid} on ${node}..."
local kind current_cores current_memory disk_key local kind current_cores current_memory disk_key
if ssh "$ssh_target" "qm status ${vmid}" >/dev/null 2>&1; then if ssh "$ssh_target" "${sudo_prefix} qm status ${vmid}" >/dev/null 2>&1; then
kind="vm" kind="vm"
disk_key="scsi0" disk_key="scsi0"
elif ssh "$ssh_target" "pct status ${vmid}" >/dev/null 2>&1; then elif ssh "$ssh_target" "${sudo_prefix} pct status ${vmid}" >/dev/null 2>&1; then
kind="lxc" kind="lxc"
disk_key="rootfs" disk_key="rootfs"
else else
@@ -210,8 +235,8 @@ cmd_modify() {
exit 1 exit 1
fi fi
local config_cmd="qm config ${vmid}" local config_cmd="${sudo_prefix} qm config ${vmid}"
[[ "$kind" == "lxc" ]] && config_cmd="pct config ${vmid}" [[ "$kind" == "lxc" ]] && config_cmd="${sudo_prefix} pct config ${vmid}"
local current_config local current_config
current_config="$(ssh "$ssh_target" "$config_cmd")" current_config="$(ssh "$ssh_target" "$config_cmd")"
current_cores="$(echo "$current_config" | grep -oP '^cores:\s*\K\S+' || echo '?')" current_cores="$(echo "$current_config" | grep -oP '^cores:\s*\K\S+' || echo '?')"
@@ -230,15 +255,14 @@ cmd_modify() {
fi fi
echo echo
read -rp "Type the VMID (${vmid}) to confirm these changes: " confirm if ! confirm_typed "$vmid" "Type the VMID (${vmid}) to confirm these changes: "; then
if [[ "$confirm" != "$vmid" ]]; then
echo "Cancelled -- input didn't match ${vmid}." echo "Cancelled -- input didn't match ${vmid}."
exit 1 exit 1
fi fi
local set_cmd="qm set" local set_cmd="${sudo_prefix} qm set"
local resize_cmd="qm resize" local resize_cmd="${sudo_prefix} qm resize"
[[ "$kind" == "lxc" ]] && set_cmd="pct set" && resize_cmd="pct resize" [[ "$kind" == "lxc" ]] && set_cmd="${sudo_prefix} pct set" && resize_cmd="${sudo_prefix} pct resize"
if [[ -n "$cores" || -n "$memory" ]]; then if [[ -n "$cores" || -n "$memory" ]]; then
local args="" local args=""
@@ -345,14 +369,15 @@ else
echo echo
echo "==> Checking ${node} for an existing VM/CT identified as '${host}'..." echo "==> Checking ${node} for an existing VM/CT identified as '${host}'..."
ssh_check_status=0 ssh_check_status=0
existing="$(ssh "$ssh_target" bash -s -- "$host" <<'REMOTE_SCRIPT' existing="$(ssh "$ssh_target" bash -s -- "$host" "$sudo_prefix" <<'REMOTE_SCRIPT'
target="$1" target="$1"
for id in $(qm list 2>/dev/null | awk 'NR>1{print $1}'); do sudo_pfx="$2"
n="$(qm config "$id" 2>/dev/null | grep -oP '^name:\s*\K\S+' || true)" for id in $($sudo_pfx qm list 2>/dev/null | awk 'NR>1{print $1}'); do
n="$($sudo_pfx qm config "$id" 2>/dev/null | grep -oP '^name:\s*\K\S+' || true)"
[[ "$n" == "$target" ]] && echo "vm ${id} ${n}" [[ "$n" == "$target" ]] && echo "vm ${id} ${n}"
done done
for id in $(pct list 2>/dev/null | awk 'NR>1{print $1}'); do for id in $($sudo_pfx pct list 2>/dev/null | awk 'NR>1{print $1}'); do
n="$(pct config "$id" 2>/dev/null | grep -oP '^hostname:\s*\K\S+' || true)" n="$($sudo_pfx pct config "$id" 2>/dev/null | grep -oP '^hostname:\s*\K\S+' || true)"
[[ "$n" == "$target" ]] && echo "lxc ${id} ${n}" [[ "$n" == "$target" ]] && echo "lxc ${id} ${n}"
done done
exit 0 exit 0
@@ -429,8 +454,7 @@ REMOTE_SCRIPT
echo " - ${kind} VMID ${id} (${n})" echo " - ${kind} VMID ${id} (${n})"
done done
echo echo
read -rp "Type the hostname (${host}) to confirm destroying the above and replacing it: " confirm if ! confirm_typed "$host" "Type the hostname (${host}) to confirm destroying the above and replacing it: "; then
if [[ "$confirm" != "$host" ]]; then
echo "Cancelled -- input didn't match ${host}." >&2 echo "Cancelled -- input didn't match ${host}." >&2
exit 1 exit 1
fi fi
@@ -440,12 +464,12 @@ REMOTE_SCRIPT
if [[ "$kind" == "vm" ]]; then if [[ "$kind" == "vm" ]]; then
# qm destroy has no --force to stop-then-destroy in one call (pct's # qm destroy has no --force to stop-then-destroy in one call (pct's
# does) -- stop explicitly first if it's running. # does) -- stop explicitly first if it's running.
if ssh "$ssh_target" "qm status ${id}" 2>/dev/null | grep -q running; then if ssh "$ssh_target" "${sudo_prefix} qm status ${id}" 2>/dev/null | grep -q running; then
ssh "$ssh_target" "qm stop ${id}" ssh "$ssh_target" "${sudo_prefix} qm stop ${id}"
fi fi
ssh "$ssh_target" "qm destroy ${id} --purge 1" ssh "$ssh_target" "${sudo_prefix} qm destroy ${id} --purge 1"
else else
ssh "$ssh_target" "pct destroy ${id} --force 1 --purge 1" ssh "$ssh_target" "${sudo_prefix} pct destroy ${id} --force 1 --purge 1"
fi fi
done done
fi fi
@@ -474,7 +498,7 @@ if [[ -z "$vmid" ]]; then
vmid="<next-free-vmid>" vmid="<next-free-vmid>"
echo "[dry-run] would ask ${node} for the next free VMID (pvesh get /cluster/nextid)" echo "[dry-run] would ask ${node} for the next free VMID (pvesh get /cluster/nextid)"
else else
vmid="$(ssh "$ssh_target" "pvesh get /cluster/nextid" | tr -d '[:space:]')" vmid="$(ssh "$ssh_target" "${sudo_prefix} pvesh get /cluster/nextid" | tr -d '[:space:]')"
echo "Auto-assigned VMID: ${vmid}" echo "Auto-assigned VMID: ${vmid}"
fi fi
else else
@@ -488,8 +512,8 @@ if [[ "$dry_run" -eq 0 ]]; then
# both. Any success here means something is already using this ID -- # both. Any success here means something is already using this ID --
# refuse to go anywhere near it. (Reconfiguring an existing resource is # refuse to go anywhere near it. (Reconfiguring an existing resource is
# --modify's job, not this one's.) # --modify's job, not this one's.)
if ssh "$ssh_target" "qm status ${vmid}" >/dev/null 2>&1 \ if ssh "$ssh_target" "${sudo_prefix} qm status ${vmid}" >/dev/null 2>&1 \
|| ssh "$ssh_target" "pct status ${vmid}" >/dev/null 2>&1; then || ssh "$ssh_target" "${sudo_prefix} pct status ${vmid}" >/dev/null 2>&1; then
echo "ERROR: VMID ${vmid} already exists on ${node}. Refusing to touch an" >&2 echo "ERROR: VMID ${vmid} already exists on ${node}. Refusing to touch an" >&2
echo "existing resource here -- use --modify to reconfigure it, pick a" >&2 echo "existing resource here -- use --modify to reconfigure it, pick a" >&2
echo "different --vmid, or omit it to auto-assign." >&2 echo "different --vmid, or omit it to auto-assign." >&2
@@ -524,7 +548,7 @@ ensure_remote_repo() {
echo echo
echo "==> Ensuring ${remote_repo_dir} exists and is current on ${node}..." echo "==> Ensuring ${remote_repo_dir} exists and is current on ${node}..."
if [[ "$dry_run" -eq 1 ]]; then if [[ "$dry_run" -eq 1 ]]; then
echo "[dry-run] would ensure ${remote_repo_dir} exists on ${node} (clone if missing, git pull if present), and would verify/bootstrap build tooling there (scripts/codex-setup.sh) if \`nix\` isn't already on PATH -- and if that bootstrap actually ran, would also configure ${node} as a nix-cache client (scripts/configure-nix-cache-client.sh)" echo "[dry-run] would ensure ${remote_repo_dir} exists on ${node} (clone if missing, git pull if present), and would verify/bootstrap build tooling there (scripts/codex-setup.sh) if \`nix\` isn't already on PATH -- and if that bootstrap actually ran, would also configure ${node} as a nix-cache client (scripts/proxmox/configure-nix-cache-client.sh)"
return return
fi fi
@@ -583,7 +607,7 @@ ensure_remote_repo() {
# above -- ssh's non-interactive command execution won't have picked # above -- ssh's non-interactive command execution won't have picked
# up a freshly single-user-installed `nix` otherwise. # up a freshly single-user-installed `nix` otherwise.
echo "==> Configuring ${node} as a nix-cache substituter/remote-builder client..." echo "==> Configuring ${node} as a nix-cache substituter/remote-builder client..."
if ! ssh "$ssh_target" "cd '${remote_repo_dir}' && . scripts/lib/nix-bootstrap.sh && ensure_nix_profile && bash scripts/configure-nix-cache-client.sh"; then if ! ssh "$ssh_target" "cd '${remote_repo_dir}' && . scripts/lib/nix-bootstrap.sh && ensure_nix_profile && bash scripts/proxmox/configure-nix-cache-client.sh"; then
echo "WARNING: configure-nix-cache-client.sh failed on ${node} -- continuing without it (${node} will build from source / against cache.nixos.org only)." >&2 echo "WARNING: configure-nix-cache-client.sh failed on ${node} -- continuing without it (${node} will build from source / against cache.nixos.org only)." >&2
fi fi
fi fi
@@ -659,11 +683,11 @@ if [[ "$image_already_remote" -eq 0 && -z "$local_image" ]]; then
# hands the result to the remote shell to re-split, which would # hands the result to the remote shell to re-split, which would
# otherwise scatter NIX_EXTRA_OPTS (itself several space-separated, # otherwise scatter NIX_EXTRA_OPTS (itself several space-separated,
# %q-quoted tokens) across the wrong positional parameters below. # %q-quoted tokens) across the wrong positional parameters below.
printf -v remote_cmd 'bash -s -- %q %q %q %q %q' \ printf -v remote_cmd 'bash -s -- %q %q %q %q %q %q' \
"$remote_repo_dir" "$flake_target" "$remote_dir" "$remote_filename" "$NIX_EXTRA_OPTS" "$remote_repo_dir" "$flake_target" "$remote_dir" "$remote_filename" "$NIX_EXTRA_OPTS" "$sudo_prefix"
ssh "$ssh_target" "$remote_cmd" <<'REMOTE_SCRIPT' ssh "$ssh_target" "$remote_cmd" <<'REMOTE_SCRIPT'
set -euo pipefail set -euo pipefail
repo_dir="$1"; target="$2"; dest_dir="$3"; dest_name="$4"; nix_extra_opts_str="$5" repo_dir="$1"; target="$2"; dest_dir="$3"; dest_name="$4"; nix_extra_opts_str="$5"; sudo_pfx="$6"
declare -a NIX_OPTS=() declare -a NIX_OPTS=()
[[ -n "$nix_extra_opts_str" ]] && eval "NIX_OPTS=(${nix_extra_opts_str})" [[ -n "$nix_extra_opts_str" ]] && eval "NIX_OPTS=(${nix_extra_opts_str})"
cd "$repo_dir" cd "$repo_dir"
@@ -681,23 +705,14 @@ if [[ -z "$built" ]]; then
echo "ERROR: no tarball found under result-${target}/tarball after build." >&2 echo "ERROR: no tarball found under result-${target}/tarball after build." >&2
exit 1 exit 1
fi fi
mkdir -p "$dest_dir" $sudo_pfx mkdir -p "$dest_dir"
cp "$built" "${dest_dir}/${dest_name}" $sudo_pfx cp "$built" "${dest_dir}/${dest_name}"
echo "Built and staged: ${dest_dir}/${dest_name}" echo "Built and staged: ${dest_dir}/${dest_name}"
REMOTE_SCRIPT REMOTE_SCRIPT
local_image="$remote_path" local_image="$remote_path"
echo "Built on ${node}: ${remote_path}" echo "Built on ${node}: ${remote_path}"
fi fi
else else
# PROXMOX_SSH_USER defaults to root (env.sh), which needs no sudo and
# can't assume it's even installed on a minimal node -- only shell out
# through sudo when actually running as a non-root SSH user.
sudo_prefix="sudo"
sudo_display="sudo "
if [[ "$PROXMOX_SSH_USER" == "root" ]]; then
sudo_prefix=""
sudo_display=""
fi
if [[ "$dry_run" -eq 1 ]]; then if [[ "$dry_run" -eq 1 ]]; then
echo "[dry-run] would build on ${node}: nix build --no-use-registries --no-accept-flake-config${nix_opts_display} \\" echo "[dry-run] would build on ${node}: nix build --no-use-registries --no-accept-flake-config${nix_opts_display} \\"
echo "[dry-run] .#nixosConfigurations.${flake_target}.config.system.build.diskoImagesScript" echo "[dry-run] .#nixosConfigurations.${flake_target}.config.system.build.diskoImagesScript"
@@ -715,7 +730,7 @@ REMOTE_SCRIPT
"$remote_repo_dir" "$flake_target" "$remote_dir" "$remote_filename" "$NIX_EXTRA_OPTS" "$sudo_prefix" "$remote_repo_dir" "$flake_target" "$remote_dir" "$remote_filename" "$NIX_EXTRA_OPTS" "$sudo_prefix"
ssh "$ssh_target" "$remote_cmd" <<'REMOTE_SCRIPT' ssh "$ssh_target" "$remote_cmd" <<'REMOTE_SCRIPT'
set -euo pipefail set -euo pipefail
repo_dir="$1"; target="$2"; dest_dir="$3"; dest_name="$4"; nix_extra_opts_str="$5"; sudo_prefix="$6" repo_dir="$1"; target="$2"; dest_dir="$3"; dest_name="$4"; nix_extra_opts_str="$5"; sudo_pfx="$6"
declare -a NIX_OPTS=() declare -a NIX_OPTS=()
[[ -n "$nix_extra_opts_str" ]] && eval "NIX_OPTS=(${nix_extra_opts_str})" [[ -n "$nix_extra_opts_str" ]] && eval "NIX_OPTS=(${nix_extra_opts_str})"
cd "$repo_dir" cd "$repo_dir"
@@ -724,7 +739,7 @@ ensure_nix_profile
nix build --no-use-registries --no-accept-flake-config "${NIX_OPTS[@]}" \ nix build --no-use-registries --no-accept-flake-config "${NIX_OPTS[@]}" \
".#nixosConfigurations.${target}.config.system.build.diskoImagesScript" \ ".#nixosConfigurations.${target}.config.system.build.diskoImagesScript" \
--out-link "result-${target}" --out-link "result-${target}"
$sudo_prefix "./result-${target}" \ $sudo_pfx "./result-${target}" \
--pre-format-files "host-keys/${target}_ssh_host_ed25519_key" /etc/ssh/ssh_host_ed25519_key \ --pre-format-files "host-keys/${target}_ssh_host_ed25519_key" /etc/ssh/ssh_host_ed25519_key \
--pre-format-files "host-keys/${target}_ssh_host_ed25519_key.pub" /etc/ssh/ssh_host_ed25519_key.pub \ --pre-format-files "host-keys/${target}_ssh_host_ed25519_key.pub" /etc/ssh/ssh_host_ed25519_key.pub \
--build-memory 2048 --build-memory 2048
@@ -733,8 +748,8 @@ if [[ -z "$built" ]]; then
echo "ERROR: no .raw image found in ${repo_dir} after build." >&2 echo "ERROR: no .raw image found in ${repo_dir} after build." >&2
exit 1 exit 1
fi fi
mkdir -p "$dest_dir" $sudo_pfx mkdir -p "$dest_dir"
mv "$built" "${dest_dir}/${dest_name}" $sudo_pfx mv "$built" "${dest_dir}/${dest_name}"
echo "Built and staged: ${dest_dir}/${dest_name}" echo "Built and staged: ${dest_dir}/${dest_name}"
REMOTE_SCRIPT REMOTE_SCRIPT
local_image="$remote_path" local_image="$remote_path"
@@ -766,13 +781,22 @@ if [[ "$type" == "lxc" ]]; then
# 512M default otherwise (confirmed live: --memory 2048 left swap at # 512M default otherwise (confirmed live: --memory 2048 left swap at
# 512). Default to matching whatever --memory resolved to above. # 512). Default to matching whatever --memory resolved to above.
local_swap="${swap:-$memory}" local_swap="${swap:-$memory}"
# --unprivileged 1: modules/platforms/lxc.nix sets proxmoxLXC.privileged # --unprivileged: read back from modules/platforms/lxc.nix's own
# = false, so the NixOS config inside the image assumes it's running as # proxmoxLXC.privileged (via flake_target_lxc_privileged) rather than
# an unprivileged container (cgroup/capability/mount expectations baked # hardcoded, since that's no longer the same for every lxc-* target --
# in at boot). `pct create`'s own CLI default for this flag is # lxc-docker sets it true so the container's NFS mounts work at all (the
# privileged (unlike the web UI, which defaults its checkbox the other # kernel's NFS client can't mount from inside any unprivileged
# way) -- leaving it unset creates a privileged container running a # container's user namespace, no matter what AppArmor allows -- see that
# NixOS config that assumes unprivileged, a real mismatch. # option's own comment). The NixOS config inside the image bakes in
# cgroup/capability/mount expectations matching whichever value it was
# built with, so this must stay in sync with it -- `pct create`'s own
# CLI default for this flag is privileged (unlike the web UI, which
# defaults its checkbox the other way), so leaving it unset would create
# a privileged container running a NixOS config that assumes
# unprivileged for every target except lxc-docker, a real mismatch.
privileged_eval="$(flake_target_lxc_privileged "$repo_root" "$flake_target")"
unprivileged_flag=1
[[ "$privileged_eval" == "true" ]] && unprivileged_flag=0
# #
# --features nesting=1,keyctl=1: required for a modern (v247+) systemd # --features nesting=1,keyctl=1: required for a modern (v247+) systemd
# guest to actually boot unprivileged -- confirmed live: without this, # guest to actually boot unprivileged -- confirmed live: without this,
@@ -790,36 +814,42 @@ if [[ "$type" == "lxc" ]]; then
# hands the whole string to `ssh` as a single command for the *remote* # hands the whole string to `ssh` as a single command for the *remote*
# shell to parse -- unquoted, that `;` would be read as a remote # shell to parse -- unquoted, that `;` would be read as a remote
# command separator and silently truncate this into two commands. # command separator and silently truncate this into two commands.
create_cmd="pct create ${vmid} ${iso_storage}:vztmpl/${remote_filename} --unprivileged 1 --features '${PROXMOX_DEFAULT_LXC_FEATURES}' --rootfs ${storage}:${local_disk_size} --hostname ${name} --cores ${cores} --memory ${memory} --swap ${local_swap} --net0 name=eth0,bridge=${bridge},ip=dhcp" create_cmd="${sudo_prefix} pct create ${vmid} ${iso_storage}:vztmpl/${remote_filename} --unprivileged ${unprivileged_flag} --features '${PROXMOX_DEFAULT_LXC_FEATURES}' --rootfs ${storage}:${local_disk_size} --hostname ${name} --cores ${cores} --memory ${memory} --swap ${local_swap} --net0 name=eth0,bridge=${bridge},ip=dhcp"
remote "$create_cmd" remote "$create_cmd"
remote "pct start ${vmid}" remote "${sudo_prefix} pct start ${vmid}"
else else
echo "==> Creating VM ${vmid} (${name})..." echo "==> Creating VM ${vmid} (${name})..."
# pre-enrolled-keys=0 disables OVMF's Secure Boot key pre-enrollment -- # pre-enrolled-keys=0 disables OVMF's Secure Boot key pre-enrollment --
# required, or systemd-boot (unsigned) can't be trusted by the firmware. # required, or systemd-boot (unsigned) can't be trusted by the firmware.
remote "qm create ${vmid} --name ${name} --memory ${memory} --cores ${cores} \ # --agent 1: wires up the virtio-serial channel QEMU exposes to the guest.
# modules/common/configuration.nix sets services.qemuGuest.enable = true
# on every host, so the guest-side qemu-ga daemon is already running --
# without this flag Proxmox never creates the channel it listens on, so
# `qm guest exec`/`qm agent` and the UI's IP-address display silently
# never work for any VM this script creates.
remote "${sudo_prefix} qm create ${vmid} --name ${name} --memory ${memory} --cores ${cores} \
--net0 virtio,bridge=${bridge} --bios ovmf --machine q35 --scsihw virtio-scsi-pci \ --net0 virtio,bridge=${bridge} --bios ovmf --machine q35 --scsihw virtio-scsi-pci \
--efidisk0 ${storage}:1,efitype=4m,pre-enrolled-keys=0" --efidisk0 ${storage}:1,efitype=4m,pre-enrolled-keys=0 --agent enabled=1"
if [[ "$dry_run" -eq 1 ]]; then if [[ "$dry_run" -eq 1 ]]; then
echo "[dry-run] ssh ${ssh_target} -- qm importdisk ${vmid} ${remote_path} ${storage}" echo "[dry-run] ssh ${ssh_target} -- ${sudo_display}qm importdisk ${vmid} ${remote_path} ${storage}"
echo "[dry-run] (would parse the resulting disk identifier from that output)" echo "[dry-run] (would parse the resulting disk identifier from that output)"
echo "[dry-run] ssh ${ssh_target} -- qm set ${vmid} --scsi0 ${storage}:<parsed-disk-id>" echo "[dry-run] ssh ${ssh_target} -- ${sudo_display}qm set ${vmid} --scsi0 ${storage}:<parsed-disk-id>"
else else
importdisk_output="$(ssh "$ssh_target" "qm importdisk ${vmid} ${remote_path} ${storage}")" importdisk_output="$(ssh "$ssh_target" "${sudo_prefix} qm importdisk ${vmid} ${remote_path} ${storage}")"
echo "$importdisk_output" echo "$importdisk_output"
disk_id="$(echo "$importdisk_output" | grep -oP "(?<=Successfully imported disk as ')[^']+" | sed 's/^unused[0-9]*://')" disk_id="$(echo "$importdisk_output" | grep -oP "(?<=Successfully imported disk as ')[^']+" | sed 's/^unused[0-9]*://')"
if [[ -z "$disk_id" ]]; then if [[ -z "$disk_id" ]]; then
echo "ERROR: couldn't parse the imported disk identifier from qm importdisk's output above." >&2 echo "ERROR: couldn't parse the imported disk identifier from qm importdisk's output above." >&2
echo "The VM shell (${vmid}) and imported disk both exist -- finish attaching it by hand:" >&2 echo "The VM shell (${vmid}) and imported disk both exist -- finish attaching it by hand:" >&2
echo " ssh ${ssh_target} -- qm set ${vmid} --scsi0 ${storage}:<disk-id-from-output-above>" >&2 echo " ssh ${ssh_target} -- ${sudo_display}qm set ${vmid} --scsi0 ${storage}:<disk-id-from-output-above>" >&2
echo " ssh ${ssh_target} -- qm set ${vmid} --boot order=scsi0" >&2 echo " ssh ${ssh_target} -- ${sudo_display}qm set ${vmid} --boot order=scsi0" >&2
exit 1 exit 1
fi fi
remote "qm set ${vmid} --scsi0 ${disk_id}" remote "${sudo_prefix} qm set ${vmid} --scsi0 ${disk_id}"
fi fi
remote "qm set ${vmid} --boot order=scsi0" remote "${sudo_prefix} qm set ${vmid} --boot order=scsi0"
remote "qm start ${vmid}" remote "${sudo_prefix} qm start ${vmid}"
fi fi
echo echo
+250
View File
@@ -0,0 +1,250 @@
#!/usr/bin/env bash
# recover-hosts.sh — Fix sops/SSH-key/GitHub-token issues on deployed NixOS hosts
# and trigger a Switch-nix rebuild on each.
#
# Run from the repo root on the workstation (nixos@nixos):
# bash scripts/recover-hosts.sh [<hostname> ...]
#
# With no args it discovers and checks every known hostname.
# With args it checks only those hostnames:
# bash scripts/recover-hosts.sh tor-relay
#
# Fixes applied automatically (then prompts before rebuilding):
# 1. SSH host key drift — live key no longer matches host-keys/<target>_ssh_host_ed25519_key
# Fix: scp the registered key back and restore it (needs sudo once per host).
# 2. Stale/invalid GitHub access token — the rendered nix-github-token.conf has
# a token GitHub rejects (401), blocking any rebuild that fetches disko or
# other public GitHub flake inputs.
# Fix: empty the rendered file so nix makes unauthenticated requests instead.
# Public repos (disko, nixpkgs, etc.) work fine without auth. sops-nix
# re-renders the correct new token automatically after the first successful
# rebuild.
#
# Both fixes need one interactive sudo session per host. The script opens a
# single ssh -t per broken host so you enter the password once and all steps
# run in sequence.
set -euo pipefail
cd "$(dirname "$0")/.."
source scripts/env.sh 2>/dev/null || true
SSH_OPTS=(-o StrictHostKeyChecking=no -o BatchMode=yes -o ConnectTimeout=5)
SSH_USER=nixos
# Known flake-target → ssh hostname map for all currently-defined hosts.
# Add new hosts here as they are deployed.
declare -A TARGET_HOST=(
[lxc-docker]=docker
[lxc-nix-cache]=nix-cache
[lxc-pxe-boot]=pxe-boot
[lxc-tor-relay]=tor-relay
[lxc-minimal]=nix-minimal
[proxmox-server]=server
[baremetal-gui]=nixos
)
# ── helpers ───────────────────────────────────────────────────────────────────
info() { echo " [✓] $*"; }
warn() { echo " [!] $*"; }
step() { echo "==> $*"; }
ssh_host_age() {
ssh-keyscan -t ed25519 "$1" 2>/dev/null \
| nix shell nixpkgs#ssh-to-age --command ssh-to-age 2>/dev/null \
| head -1 || true
}
registered_age() {
local keyfile="host-keys/${1}_ssh_host_ed25519_key.pub"
[ -f "$keyfile" ] || return 0
nix shell nixpkgs#ssh-to-age --command ssh-to-age < "$keyfile" 2>/dev/null \
| head -1 || true
}
github_token_valid() {
local host=$1
local raw token code
raw=$(ssh "${SSH_OPTS[@]}" "$SSH_USER@$host" \
"cat /run/secrets/rendered/nix-github-token.conf 2>/dev/null || true")
token=$(echo "$raw" | grep -oP '(?<=github\.com=)\S+' || true)
if [ -z "$token" ]; then
return 0 # no token = unauthenticated, works for public repos
fi
code=$(curl -s -o /dev/null -w "%{http_code}" \
-H "Authorization: token $token" \
"https://api.github.com/repos/nix-community/disko" 2>/dev/null || echo 000)
[ "$code" = "200" ]
}
# ── discover hosts ────────────────────────────────────────────────────────────
if [ $# -gt 0 ]; then
HOSTNAMES=("$@")
else
HOSTNAMES=()
seen=()
for target in "${!TARGET_HOST[@]}"; do
h="${TARGET_HOST[$target]}"
# deduplicate (e.g. proxmox-server and lxc-server both map to "server")
if [[ ! " ${seen[*]:-} " =~ " $h " ]]; then
seen+=("$h")
if ssh "${SSH_OPTS[@]}" "$SSH_USER@$h" "true" 2>/dev/null; then
HOSTNAMES+=("$h")
fi
fi
done
fi
if [ ${#HOSTNAMES[@]} -eq 0 ]; then
echo "No reachable hosts found. Pass hostnames explicitly or check SSH."
exit 1
fi
echo ""
echo "Hosts to check: ${HOSTNAMES[*]}"
echo ""
# ── check phase ───────────────────────────────────────────────────────────────
NEEDS_FIX=()
for host in "${HOSTNAMES[@]}"; do
step "$host"
if ! ssh "${SSH_OPTS[@]}" "$SSH_USER@$host" "true" 2>/dev/null; then
warn "SSH unreachable — clearing stale known_hosts entry"
ssh-keygen -R "$host" 2>/dev/null || true
continue
fi
flake_target=$(ssh "${SSH_OPTS[@]}" "$SSH_USER@$host" \
"cat /etc/flake-target 2>/dev/null || true")
echo " flake-target: ${flake_target:-unknown}"
host_broken=false
# SSH host key
if [ -n "$flake_target" ] && [ -f "host-keys/${flake_target}_ssh_host_ed25519_key.pub" ]; then
live=$(ssh_host_age "$host")
want=$(registered_age "$flake_target")
if [ "$live" = "$want" ]; then
info "SSH host key OK"
else
warn "SSH host key MISMATCH (live ≠ host-keys/)"
echo " live: $live"
echo " registered: $want"
host_broken=true
fi
else
echo " [~] No host-keys/ entry for ${flake_target:-unknown} — skipping key check"
fi
# GitHub token
if github_token_valid "$host"; then
info "GitHub token OK"
else
warn "GitHub token invalid (rebuild will fail with 401)"
host_broken=true
fi
# sops-nix result
sops_result=$(ssh "${SSH_OPTS[@]}" "$SSH_USER@$host" \
"systemctl show sops-nix --property=Result --value 2>/dev/null || echo unknown")
if [ "$sops_result" = "success" ]; then
info "sops-nix: success"
else
warn "sops-nix: $sops_result"
fi
$host_broken && NEEDS_FIX+=("$host")
echo ""
done
# ── fix phase ─────────────────────────────────────────────────────────────────
if [ ${#NEEDS_FIX[@]} -eq 0 ]; then
echo "All hosts healthy — nothing to fix."
exit 0
fi
echo "Hosts needing fixes: ${NEEDS_FIX[*]}"
echo ""
echo "Each fix requires one sudo session per host. You will be prompted for"
echo "the nixos sudo password once per host; all steps run in that session."
echo ""
read -r -p "Proceed with fixes + Switch-nix on each broken host? [y/N] " confirm
[[ "$confirm" =~ ^[Yy]$ ]] || { echo "Aborted."; exit 0; }
echo ""
for host in "${NEEDS_FIX[@]}"; do
step "Fixing $host"
flake_target=$(ssh "${SSH_OPTS[@]}" "$SSH_USER@$host" \
"cat /etc/flake-target 2>/dev/null || true")
fix_script=""
# Fix 1: restore SSH host key
live=$(ssh_host_age "$host")
want=$(registered_age "${flake_target:-}")
if [ -n "$want" ] && [ "$live" != "$want" ]; then
echo " Uploading registered SSH host key (private + public)..."
scp -o StrictHostKeyChecking=no \
"host-keys/${flake_target}_ssh_host_ed25519_key" \
"$SSH_USER@$host:/tmp/recover_ed25519_key"
scp -o StrictHostKeyChecking=no \
"host-keys/${flake_target}_ssh_host_ed25519_key.pub" \
"$SSH_USER@$host:/tmp/recover_ed25519_key.pub"
fix_script+='
echo "[fix] Restoring SSH host key..."
install -m 0600 /tmp/recover_ed25519_key /etc/ssh/ssh_host_ed25519_key
install -m 0644 /tmp/recover_ed25519_key.pub /etc/ssh/ssh_host_ed25519_key.pub
rm -f /tmp/recover_ed25519_key /tmp/recover_ed25519_key.pub
echo " Done."
'
ssh-keygen -R "$host" 2>/dev/null || true
fi
# Fix 2: clear invalid GitHub token
if ! github_token_valid "$host"; then
fix_script+='
echo "[fix] Clearing stale GitHub token (nix will use unauthenticated access)..."
echo "" > /run/secrets/rendered/nix-github-token.conf
systemctl restart nix-daemon 2>/dev/null || true
echo " Done."
'
fi
# Fix 3: rebuild
fix_script+='
echo "[fix] Running nixos-rebuild switch..."
nixos-rebuild switch \
--no-write-lock-file \
--refresh \
--flake "git+https://gitea.lan.ddnsgeek.com/beatzaplenty/nixos.git#$(cat /etc/flake-target)"
echo "[fix] Rebuild complete."
'
echo " Opening SSH session (enter sudo password when prompted)..."
if ssh -t -o StrictHostKeyChecking=no "$SSH_USER@$host" \
"sudo bash -s" <<< "$fix_script"; then
echo ""
info "$host fixed and rebuilt"
else
rc=$?
echo ""
warn "$host: rebuild exited with code $rc (may still have succeeded — check sops-nix below)"
fi
# Verify: re-check sops-nix result post-rebuild
sops_result_after=$(ssh "${SSH_OPTS[@]}" "$SSH_USER@$host" \
"systemctl show sops-nix --property=Result --value 2>/dev/null || echo unknown" 2>/dev/null || echo "ssh-failed")
if [ "$sops_result_after" = "success" ]; then
info "$host sops-nix: success post-rebuild"
else
warn "$host sops-nix: $sops_result_after post-rebuild (may need another pass)"
fi
echo ""
done
echo "Recovery complete."
@@ -6,7 +6,7 @@
# copy is ever lost, or to run either script from a different machine. # copy is ever lost, or to run either script from a different machine.
# #
# Usage: # Usage:
# scripts/backup-admin-key.sh <dest-path> [--key-file <path>] [--force] [--dry-run] # scripts/secrets/backup-admin-key.sh <dest-path> [--key-file <path>] [--force] [--dry-run]
# #
# Source key resolution matches sops/age's own default order: # Source key resolution matches sops/age's own default order:
# $SOPS_AGE_KEY (inline identity text) if set, else # $SOPS_AGE_KEY (inline identity text) if set, else
@@ -15,11 +15,13 @@
# ${XDG_CONFIG_HOME:-$HOME/.config}/sops/age/keys.txt # ${XDG_CONFIG_HOME:-$HOME/.config}/sops/age/keys.txt
set -euo pipefail set -euo pipefail
repo_root="$(cd "$(dirname "$0")/.." && pwd)" repo_root="$(cd "$(dirname "$0")/../.." && pwd)"
sops_yaml="${repo_root}/.sops.yaml" sops_yaml="${repo_root}/.sops.yaml"
# shellcheck source=env.sh # shellcheck source=../env.sh
source "${repo_root}/scripts/env.sh" source "${repo_root}/scripts/env.sh"
# shellcheck source=../lib/sops-age.sh
source "${repo_root}/scripts/lib/sops-age.sh"
# Pin cwd for the same reason rotate-admin-key.sh does: age/sops calls # Pin cwd for the same reason rotate-admin-key.sh does: age/sops calls
# below should never depend on wherever the caller's shell happened to be. # below should never depend on wherever the caller's shell happened to be.
@@ -43,7 +45,7 @@ EOF
dry_run=0 dry_run=0
force=0 force=0
key_file="${SOPS_AGE_KEY_FILE:-${XDG_CONFIG_HOME:-$HOME/.config}/sops/age/keys.txt}" key_file="$DEFAULT_SOPS_AGE_KEY_FILE"
args=() args=()
while [[ $# -gt 0 ]]; do while [[ $# -gt 0 ]]; do
@@ -103,13 +105,13 @@ scratch="$(mktemp)"
trap 'rm -f "$scratch"' EXIT trap 'rm -f "$scratch"' EXIT
( umask 077; printf '%s\n' "$src_content" > "$scratch" ) ( umask 077; printf '%s\n' "$src_content" > "$scratch" )
src_pub="$(nix-shell "${NIX_OPTS[@]}" -p age --run "age-keygen -y '$scratch'")" || { src_pub="$(age_pubkey_from_identity_file "$scratch")" || {
echo "ERROR: source doesn't look like a valid age identity (age-keygen -y failed)." >&2 echo "ERROR: source doesn't look like a valid age identity (age-keygen -y failed)." >&2
exit 1 exit 1
} }
echo " public key: ${src_pub}" echo " public key: ${src_pub}"
current_admin_pub="$(grep -E '^ - &admin age1' "$sops_yaml" 2>/dev/null | awk '{print $NF}' || true)" current_admin_pub="$(sops_yaml_admin_pubkey "$sops_yaml")"
if [[ -n "$current_admin_pub" && "$current_admin_pub" != "$src_pub" ]]; then if [[ -n "$current_admin_pub" && "$current_admin_pub" != "$src_pub" ]]; then
echo "NOTE: this key does not match .sops.yaml's current &admin entry (${current_admin_pub})." echo "NOTE: this key does not match .sops.yaml's current &admin entry (${current_admin_pub})."
echo " Backing it up anyway -- this script doesn't require it to be the admin key." echo " Backing it up anyway -- this script doesn't require it to be the admin key."
@@ -131,7 +133,7 @@ fi
mkdir -p "$(dirname "$dest")" mkdir -p "$(dirname "$dest")"
install -m 600 "$scratch" "$dest" install -m 600 "$scratch" "$dest"
dest_pub="$(nix-shell "${NIX_OPTS[@]}" -p age --run "age-keygen -y '$dest'")" dest_pub="$(age_pubkey_from_identity_file "$dest")"
if [[ "$dest_pub" != "$src_pub" ]]; then if [[ "$dest_pub" != "$src_pub" ]]; then
echo "ERROR: ${dest} was written but its public key doesn't match the source -- investigate before relying on this backup." >&2 echo "ERROR: ${dest} was written but its public key doesn't match the source -- investigate before relying on this backup." >&2
exit 1 exit 1
@@ -144,5 +146,5 @@ Done. Backed up to: ${dest}
This is a private key -- store it somewhere offline/secure, not in this This is a private key -- store it somewhere offline/secure, not in this
repo or anywhere it'd get committed. Restore it with: repo or anywhere it'd get committed. Restore it with:
scripts/rotate-admin-key.sh ${dest} scripts/secrets/rotate-admin-key.sh ${dest}
EOF EOF
@@ -2,7 +2,7 @@
# Generates a new machine's SSH host key by an arbitrary name, before it # Generates a new machine's SSH host key by an arbitrary name, before it
# necessarily has a flake target yet -- prints the .sops.yaml snippet to # necessarily has a flake target yet -- prints the .sops.yaml snippet to
# add by hand. For any host that already has a flake target, # add by hand. For any host that already has a flake target,
# scripts/sync-host-keys.sh <target> does this same job plus the # scripts/secrets/sync-host-keys.sh <target> does this same job plus the
# .sops.yaml/key_groups registration and re-encryption automatically; use # .sops.yaml/key_groups registration and re-encryption automatically; use
# this script only to pre-generate a key ahead of adding the flake target # this script only to pre-generate a key ahead of adding the flake target
# itself. # itself.
@@ -22,13 +22,13 @@
# new machine. # new machine.
set -euo pipefail set -euo pipefail
repo_root="$(cd "$(dirname "$0")/.." && pwd)" repo_root="$(cd "$(dirname "$0")/../.." && pwd)"
# shellcheck source=env.sh # shellcheck source=../env.sh
source "${repo_root}/scripts/env.sh" source "${repo_root}/scripts/env.sh"
# shellcheck source=lib/ssh-host-keys.sh # shellcheck source=../lib/ssh-host-keys.sh
source "${repo_root}/scripts/lib/ssh-host-keys.sh" source "${repo_root}/scripts/lib/ssh-host-keys.sh"
hostname="${1:?usage: scripts/prepare-host-key.sh <hostname>}" hostname="${1:?usage: scripts/secrets/prepare-host-key.sh <hostname>}"
sops_yaml="${repo_root}/.sops.yaml" sops_yaml="${repo_root}/.sops.yaml"
if [[ ! -f "$sops_yaml" ]]; then if [[ ! -f "$sops_yaml" ]]; then
@@ -10,7 +10,7 @@
# sync-host-keys.sh print when they bootstrap a brand-new, not-yet-trusted # sync-host-keys.sh print when they bootstrap a brand-new, not-yet-trusted
# age key on a machine that's never had admin access before: # age key on a machine that's never had admin access before:
# #
# scripts/rotate-admin-key.sh /path/to/backed-up/admin/keys.txt # scripts/secrets/rotate-admin-key.sh /path/to/backed-up/admin/keys.txt
# #
# The backup key's *public* key must match .sops.yaml's current &admin # The backup key's *public* key must match .sops.yaml's current &admin
# entry -- this script verifies that by deriving it, it doesn't just trust # entry -- this script verifies that by deriving it, it doesn't just trust
@@ -19,11 +19,13 @@
# the common case is just pointing this at the restored backup. # the common case is just pointing this at the restored backup.
set -euo pipefail set -euo pipefail
repo_root="$(cd "$(dirname "$0")/.." && pwd)" repo_root="$(cd "$(dirname "$0")/../.." && pwd)"
sops_yaml="${repo_root}/.sops.yaml" sops_yaml="${repo_root}/.sops.yaml"
# shellcheck source=env.sh # shellcheck source=../env.sh
source "${repo_root}/scripts/env.sh" source "${repo_root}/scripts/env.sh"
# shellcheck source=../lib/sops-age.sh
source "${repo_root}/scripts/lib/sops-age.sh"
# sops resolves .sops.yaml by walking up from the process's cwd, not from # sops resolves .sops.yaml by walking up from the process's cwd, not from
# the target file's own path -- if this script were invoked from somewhere # the target file's own path -- if this script were invoked from somewhere
@@ -53,7 +55,7 @@ EOF
} }
dry_run=0 dry_run=0
new_key_file="${SOPS_AGE_KEY_FILE:-${XDG_CONFIG_HOME:-$HOME/.config}/sops/age/keys.txt}" new_key_file="$DEFAULT_SOPS_AGE_KEY_FILE"
args=() args=()
while [[ $# -gt 0 ]]; do while [[ $# -gt 0 ]]; do
@@ -93,13 +95,9 @@ backup_key="${args[0]}"
nix_extra_opts nix_extra_opts
age_pub() {
nix-shell "${NIX_OPTS[@]}" -p age --run "age-keygen -y '$1'"
}
echo "==> Deriving public keys..." echo "==> Deriving public keys..."
old_pub="$(age_pub "$backup_key")" old_pub="$(age_pubkey_from_identity_file "$backup_key")"
new_pub="$(age_pub "$new_key_file")" new_pub="$(age_pubkey_from_identity_file "$new_key_file")"
echo " backup (old admin) key: ${old_pub}" echo " backup (old admin) key: ${old_pub}"
echo " new admin key: ${new_pub}" echo " new admin key: ${new_pub}"
@@ -108,12 +106,11 @@ if [[ "$old_pub" == "$new_pub" ]]; then
exit 1 exit 1
fi fi
current_admin_line="$(grep -E '^ - &admin age1' "$sops_yaml" || true)" current_admin_pub="$(sops_yaml_admin_pubkey "$sops_yaml")"
if [[ -z "$current_admin_line" ]]; then if [[ -z "$current_admin_pub" ]]; then
echo "ERROR: couldn't find a '&admin age1...' line in ${sops_yaml}." >&2 echo "ERROR: couldn't find a '&admin age1...' line in ${sops_yaml}." >&2
exit 1 exit 1
fi fi
current_admin_pub="$(awk '{print $NF}' <<<"$current_admin_line")"
if [[ "$current_admin_pub" != "$old_pub" ]]; then if [[ "$current_admin_pub" != "$old_pub" ]]; then
echo "ERROR: ${backup_key} doesn't match the current &admin key in .sops.yaml." >&2 echo "ERROR: ${backup_key} doesn't match the current &admin key in .sops.yaml." >&2
@@ -129,9 +126,17 @@ if [[ "${#secrets_files[@]}" -eq 0 ]]; then
exit 1 exit 1
fi fi
# sops_can_decrypt <key-file> <secrets-file>: used both to confirm the
# backup key still works before touching anything, and again after
# rotation to confirm the new key does too.
sops_can_decrypt() {
local key_file="$1" secrets_file="$2"
SOPS_AGE_KEY_FILE="$key_file" nix-shell "${NIX_OPTS[@]}" -p sops --run \
"sops -d '${secrets_file}'" >/dev/null
}
echo "==> Confirming the backup key can actually decrypt..." echo "==> Confirming the backup key can actually decrypt..."
if ! SOPS_AGE_KEY_FILE="$backup_key" nix-shell "${NIX_OPTS[@]}" -p sops --run \ if ! sops_can_decrypt "$backup_key" "${secrets_files[0]}"; then
"sops -d '${secrets_files[0]}'" >/dev/null; then
echo "ERROR: backup key failed to decrypt $(basename "${secrets_files[0]}") -- aborting." >&2 echo "ERROR: backup key failed to decrypt $(basename "${secrets_files[0]}") -- aborting." >&2
exit 1 exit 1
fi fi
@@ -162,14 +167,12 @@ echo " Updated."
echo "==> Re-encrypting secrets/*.yaml for the new recipient set..." echo "==> Re-encrypting secrets/*.yaml for the new recipient set..."
for f in "${secrets_files[@]}"; do for f in "${secrets_files[@]}"; do
echo "==> $(basename "$f")" echo "==> $(basename "$f")"
SOPS_AGE_KEY_FILE="$backup_key" nix-shell "${NIX_OPTS[@]}" -p sops --run \ sops_updatekeys "$f" "$backup_key"
"sops updatekeys --yes '${f}'"
done done
echo "==> Verifying the new key can decrypt everything..." echo "==> Verifying the new key can decrypt everything..."
for f in "${secrets_files[@]}"; do for f in "${secrets_files[@]}"; do
if ! SOPS_AGE_KEY_FILE="$new_key_file" nix-shell "${NIX_OPTS[@]}" -p sops --run \ if ! sops_can_decrypt "$new_key_file" "$f"; then
"sops -d '${f}'" >/dev/null; then
echo "ERROR: new key failed to decrypt $(basename "$f") after rotation -- investigate before committing." >&2 echo "ERROR: new key failed to decrypt $(basename "$f") after rotation -- investigate before committing." >&2
exit 1 exit 1
fi fi
@@ -24,17 +24,21 @@
# ever touches keys it itself manages. # ever touches keys it itself manages.
set -euo pipefail set -euo pipefail
repo_root="$(cd "$(dirname "$0")/.." && pwd)" repo_root="$(cd "$(dirname "$0")/../.." && pwd)"
sops_yaml="${repo_root}/.sops.yaml" sops_yaml="${repo_root}/.sops.yaml"
keydir="${repo_root}/host-keys" keydir="${repo_root}/host-keys"
editor="${repo_root}/scripts/lib/sync-host-keys-edit-sops.py" editor="${repo_root}/scripts/lib/sync-host-keys-edit-sops.py"
# shellcheck source=env.sh # shellcheck source=../env.sh
source "${repo_root}/scripts/env.sh" source "${repo_root}/scripts/env.sh"
# shellcheck source=lib/nix-eval.sh # shellcheck source=../lib/nix-eval.sh
source "${repo_root}/scripts/lib/nix-eval.sh" source "${repo_root}/scripts/lib/nix-eval.sh"
# shellcheck source=lib/ssh-host-keys.sh # shellcheck source=../lib/ssh-host-keys.sh
source "${repo_root}/scripts/lib/ssh-host-keys.sh" source "${repo_root}/scripts/lib/ssh-host-keys.sh"
# shellcheck source=../lib/sops-age.sh
source "${repo_root}/scripts/lib/sops-age.sh"
# shellcheck source=../lib/confirm.sh
source "${repo_root}/scripts/lib/confirm.sh"
mkdir -p "$keydir" mkdir -p "$keydir"
@@ -78,7 +82,7 @@ ensure_admin_decrypt_key() {
return return
fi fi
local key_file="${SOPS_AGE_KEY_FILE:-${XDG_CONFIG_HOME:-$HOME/.config}/sops/age/keys.txt}" local key_file="$DEFAULT_SOPS_AGE_KEY_FILE"
if [[ -s "$key_file" ]]; then if [[ -s "$key_file" ]]; then
echo "Found existing sops age key at ${key_file}." echo "Found existing sops age key at ${key_file}."
@@ -97,7 +101,7 @@ ensure_admin_decrypt_key() {
mkdir -p "$(dirname "$key_file")" mkdir -p "$(dirname "$key_file")"
nix-shell "${NIX_OPTS[@]}" -p age --run "age-keygen -o '${key_file}'" 2>&1 | grep -v "^Public key:" || true nix-shell "${NIX_OPTS[@]}" -p age --run "age-keygen -o '${key_file}'" 2>&1 | grep -v "^Public key:" || true
local new_pub local new_pub
new_pub="$(nix-shell "${NIX_OPTS[@]}" -p age --run "age-keygen -y '${key_file}'")" new_pub="$(age_pubkey_from_identity_file "$key_file")"
cat <<EOF cat <<EOF
@@ -239,7 +243,7 @@ apply_edit_plan() {
while IFS= read -r basename; do while IFS= read -r basename; do
[[ -z "$basename" ]] && continue [[ -z "$basename" ]] && continue
echo "==> secrets/${basename}" echo "==> secrets/${basename}"
nix-shell "${NIX_OPTS[@]}" -p sops --run "sops updatekeys --yes '${repo_root}/secrets/${basename}'" sops_updatekeys "${repo_root}/secrets/${basename}"
done <<<"$changed" done <<<"$changed"
fi fi
fi fi
@@ -359,8 +363,7 @@ cmd_regenerate_all() {
echo "image/tarball before it can decrypt secrets again." echo "image/tarball before it can decrypt secrets again."
if [[ "$dry_run" -ne 1 ]]; then if [[ "$dry_run" -ne 1 ]]; then
read -rp "Type REGENERATE to confirm: " confirm if ! confirm_typed "REGENERATE" "Type REGENERATE to confirm: "; then
if [[ "$confirm" != "REGENERATE" ]]; then
echo "Cancelled." echo "Cancelled."
return return
fi fi
+110
View File
@@ -0,0 +1,110 @@
#!/usr/bin/env bash
# Detects and fixes drift between the ed25519 SSH host key nix-cache is
# actually serving right now and vars.nixCacheHostKey (variables.nix) --
# the value modules/nix-cache/remote-builder-client.nix bakes into every
# client's declarative programs.ssh.knownHosts, and
# scripts/proxmox/configure-nix-cache-client.sh hardcodes as its own
# default for non-NixOS clients.
#
# This value has no automatic source of truth: nix-cache's host key is
# generated once (first boot / container recreate) and never touches this
# repo again unless someone remembers to update it by hand afterwards. It
# drifted silently once already -- confirmed live: variables.nix recorded
# a key that no longer matched what nix-cache actually presented, which
# would fail every real client's SSH host-key verification for
# distributed builds without ever producing an obvious error pointing
# back here (a client just sees "Host key verification failed" against
# *some* key, with no hint that the trusted value itself was stale).
#
# codex-maintenance.sh runs this in --check mode on every invocation so
# that drift surfaces as a warning instead of a future debugging session.
#
# Usage:
# scripts/secrets/sync-nix-cache-host-key.sh [--check] [--dry-run] [--host <name>]
#
# --check Only report drift (exit 1 if found, 2 if nix-cache is
# unreachable); never writes. For CI/maintenance use.
# --dry-run Show what would change; never writes.
# --host Override the hostname to scan (default: variables.nix's
# nixCacheHost / env.sh's NIX_CACHE_HOST).
set -euo pipefail
repo_root="$(cd "$(dirname "$0")/../.." && pwd)"
variables_nix="${repo_root}/variables.nix"
client_script="${repo_root}/scripts/proxmox/configure-nix-cache-client.sh"
# shellcheck source=../env.sh
source "${repo_root}/scripts/env.sh"
check_only=0
dry_run=0
host="${NIX_CACHE_HOST}"
while [[ $# -gt 0 ]]; do
case "$1" in
--check) check_only=1; shift ;;
--dry-run) dry_run=1; shift ;;
--host)
host="${2:?--host requires a hostname}"
shift 2
;;
-h|--help)
sed -n '2,23p' "$0"
exit 0
;;
*)
echo "ERROR: unknown argument: $1" >&2
exit 1
;;
esac
done
current_value="$(grep -oE 'nixCacheHostKey = "[^"]+"' "$variables_nix" | sed -E 's/nixCacheHostKey = "(.*)"/\1/')"
if [[ -z "$current_value" ]]; then
echo "ERROR: couldn't find nixCacheHostKey in $variables_nix" >&2
exit 1
fi
current_type_blob="$(awk '{print $1, $2}' <<<"$current_value")"
current_label="$(awk '{print $3}' <<<"$current_value")"
echo "Scanning ${host} for its current ed25519 SSH host key..."
nix_extra_opts
scanned="$(nix-shell "${NIX_OPTS[@]}" -p openssh --run "ssh-keyscan -t ed25519 -T 5 '${host}'" 2>/dev/null | grep -v '^#' | head -1 || true)"
if [[ -z "$scanned" ]]; then
echo "ERROR: couldn't reach ${host} (or got no ed25519 host key back) via ssh-keyscan." >&2
exit 2
fi
scanned_type_blob="$(awk '{print $2, $3}' <<<"$scanned")"
if [[ "$current_type_blob" == "$scanned_type_blob" ]]; then
echo "Up to date: ${host}'s host key matches variables.nix's nixCacheHostKey."
exit 0
fi
echo "DRIFT DETECTED:"
echo " variables.nix has: $current_type_blob"
echo " ${host} is now: $scanned_type_blob"
if [[ "$check_only" -eq 1 ]]; then
echo
echo "Run 'scripts/secrets/sync-nix-cache-host-key.sh' (no flags) to fix." >&2
exit 1
fi
new_value="${scanned_type_blob} ${current_label}"
if [[ "$dry_run" -eq 1 ]]; then
echo "(--dry-run: would update variables.nix and ${client_script##*/} to:)"
echo " $new_value"
exit 0
fi
sed -i "s|nixCacheHostKey = \"[^\"]*\"|nixCacheHostKey = \"${new_value}\"|" "$variables_nix"
sed -i "s|NIX_CACHE_HOST_KEY:=[^}]*}|NIX_CACHE_HOST_KEY:=${new_value}}|" "$client_script"
echo "Updated variables.nix and ${client_script##*/} to:"
echo " $new_value"
echo
echo "This only takes effect on already-deployed NixOS clients after their"
echo "next rebuild (programs.ssh.knownHosts is declarative). Review with"
echo "'git diff', then run 'bash scripts/codex-maintenance.sh' before committing."
+221 -50
View File
@@ -1,90 +1,261 @@
root-hashedPassword: ENC[AES256_GCM,data:Kp0nOZI7vDoLhJHiOJBwJn0rQZ5yhnwapGnAcA+qh8vlDETtFs/iQdetF/2ZxmANf62SviTNd+Ag0q5JIF1996x7onZGXqxgSMCuVzZLBdUlsO5IR0BslWWz47khYGTe4WkUg4NB1itBfQ==,iv:5Sra5vJ79V8hxQT3g9qJ+dOj2W2sumIhqpitqnHjJdk=,tag:3Igu0+8GeUZHqS3fKUVwog==,type:str] root-hashedPassword: ENC[AES256_GCM,data:Kp0nOZI7vDoLhJHiOJBwJn0rQZ5yhnwapGnAcA+qh8vlDETtFs/iQdetF/2ZxmANf62SviTNd+Ag0q5JIF1996x7onZGXqxgSMCuVzZLBdUlsO5IR0BslWWz47khYGTe4WkUg4NB1itBfQ==,iv:5Sra5vJ79V8hxQT3g9qJ+dOj2W2sumIhqpitqnHjJdk=,tag:3Igu0+8GeUZHqS3fKUVwog==,type:str]
nixos-hashedPassword: ENC[AES256_GCM,data:pT7tVRN6X4a+DNUgB7fIUUE3CbnetkjxmoSL1PxSU+ktsFU+fB0mEvJjA1uujsGH5Rcztg7YM815+M0Z67ILmHaXbza5DtFacrqhi4/b277xly0SHRX4yOvBwQh6mJG1jn/0O/wvUUIYdw==,iv:bp2nfhC8nFbk6o5iWDAugvbzu7J/a1xayFnBEtkhNpE=,tag:HqWgkIpSrSM/K9OK2WO+VQ==,type:str] nixos-hashedPassword: ENC[AES256_GCM,data:pT7tVRN6X4a+DNUgB7fIUUE3CbnetkjxmoSL1PxSU+ktsFU+fB0mEvJjA1uujsGH5Rcztg7YM815+M0Z67ILmHaXbza5DtFacrqhi4/b277xly0SHRX4yOvBwQh6mJG1jn/0O/wvUUIYdw==,iv:bp2nfhC8nFbk6o5iWDAugvbzu7J/a1xayFnBEtkhNpE=,tag:HqWgkIpSrSM/K9OK2WO+VQ==,type:str]
nix-github-token: ENC[AES256_GCM,data:OfNRGJg16Ede6EilWUetCs9za+xk5/Lsa3SpVajsqz8PMdA1xQNeCWdX7ZAMdijHClpBhU6ETFGsXvt41O9aORS951uijeGSW7/NH35/bnPISrKdYeBx/+xEiqwH,iv:QGU3v7xOy89uzRTCb1U9ICyJ8XYIpXrUsDt12aL3g2Y=,tag:Bde2wcWNv8H4WLxSEUAodg==,type:str] nix-github-token: ENC[AES256_GCM,data:k1vYz7SqVhzpWa6jTL6NUD8lKOCpHCgTm+HT4IcnbzbSTUZP/bJUYw==,iv:UqAULZnr/4+VcioUDfTwvOSuwM8K9JgGhiApvYQPyoc=,tag:1LKHXhWAO/AHPDIZFBb04A==,type:str]
sops: sops:
age: age:
- enc: | - enc: |
-----BEGIN AGE ENCRYPTED FILE----- -----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBaMUpxYmUzaVY4d1pRY3g3 YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBwU3A1MFJvZFJkL2d3UzNv
empaVUJSN0JaSlNQQlJtZENwc0VMYVhJSkFvCmJQMkpSd3RoYzlKQjBSZWErQzR2 eUF5eFhkUGIyVVpjSEovaUZKdUlVeWxncEdjCkVOQjA0Q1FtbElFdTB2Rkp5aWl6
aG5SVndOQWpSRTBDSWJVQkg0c0hiNFUKLS0tIHhiaEdpY3gwZkpCcHl2TW5CNThn THdmcXVzcnUxeVZ5VUdFMmVKa0tXSzAKLS0tIG1ZcFZwUEoxRGcwakIyRVY3dzNT
aVVUQy9Qd0trb0RNdUpVTXdrdGlrTmcK1uphQAyDV+Gk5+K1YOqw1Z8ynGP5sAPF VjlMTFdXYVVaeUpzTnNXMVhwalQybnMKjEHhkLMCGy3Q5mHPg7WSp7IBncKw6cCV
q5icujja/SGexX18hPYXbkyUtOrBYjW62gCuGJinSBPROoFUJbiP7g== 1WD9H47oGdW0BgBdi4rK4H7fmashw6TxjHZ1JfuCKps4mvvVWb2llg==
-----END AGE ENCRYPTED FILE----- -----END AGE ENCRYPTED FILE-----
recipient: age1njap586hc0q43kr03g6c8eqhdsmk8zcafkl3f83xwlc2gqhlmfgs4tmwad recipient: age1njap586hc0q43kr03g6c8eqhdsmk8zcafkl3f83xwlc2gqhlmfgs4tmwad
- enc: | - enc: |
-----BEGIN AGE ENCRYPTED FILE----- -----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBKOWZxcFlhRHhKelhDS29M YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBRM3A0b0Jub04wcFZWelRo
NGRlTEhJUGUzcFV6QVFCdTNkdEN3MlljWnl3CnI3LzVxSHBwSi9TYlRUcFdyN2py L1RyZmRNS2pTV1FPVXNEbVF1SncxTXFvWnhVClRjSFVtOVIrVVd6VnoxR3NvSEpY
cVZBV0Q3Z3FhRXA0T1NFQzd2R00yeEUKLS0tIHkxL2dHV3RkVFYxZTJNTVZvVFIr Y2lJU01lbWlUcTVraEFiNjJTeUlOdXcKLS0tIDliMElZTElVL0VNNVBseUVhQy9u
TXpJZzdnYlpJaXBmcjdWWUtxNkc4dWMKVsImJiavzUzSFn78pciNJPHaS3KWqJer YnFmM29mUFMxUXhYS1R3ZnpEZmJtSlkKWoXICCvylDiMJ/YWR8oRgIZlSYb9Ubkl
VkxF6kF3tl2HmW46eVXtsQowdu+zVR9HS35i/HvQ1r1TyP6qT5ofcQ== H6BeOPVs1qegS3Hav6vJrd+OMv1rd8rxWw5B5MYhNUEG81cdX5fgkA==
-----END AGE ENCRYPTED FILE----- -----END AGE ENCRYPTED FILE-----
recipient: age19gfn2yedg76dmztm4hncr7vf3r3c9j0qpt4rap7y7gersjk4m3ks2lhd0e recipient: age19gfn2yedg76dmztm4hncr7vf3r3c9j0qpt4rap7y7gersjk4m3ks2lhd0e
- enc: | - enc: |
-----BEGIN AGE ENCRYPTED FILE----- -----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSAyWjdSanRRaEI4VHFZVkdw YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBiZS9ibXZ5VzF4UEcyQmhE
NkxMODRuakFJTFNoTHZIRHptZFNxaGZ3K3lnCjJGR1FJalRsYmtLcHFxZW8yNU5W N1JrOFNsMUgxbC8zRmQ4MWlxbVN1Ny9JTEhrCkM1Zk1IK2JNMnlFVjNPNzBaOVNz
MzBjWmltbzI3MTByUjB6djlEdllHNXMKLS0tIEhEOVhLVjZkRi9vUEtDWUhxT0NX K0lodkVBSnhSQkFvSm5TUmVCaFFRZ3cKLS0tICtaSE5Udnp5WTNIemI2MVJWNVpq
eUZ3bGdEaHdGbXZIYkowYUZuTWJ3d2MKMwgxOqlMH7GfLlseD4J277Dcg0KCD3d5 Nm1XOTRHbVdYczBZTDQ4a3hwRVV3MGsKmOmKG4ixQhLu+TO6bsdtD1Iwnzj45o73
jwJDb82kRWoBMicTJZXoq/5oe5blJNa7dWYoqgkYxMA2O+0igT+I6g== M5cWNcPlKa4cF7JfzHMl7jzvjTzHN9NY1KjBS60EABTzvKGqYVfJng==
-----END AGE ENCRYPTED FILE----- -----END AGE ENCRYPTED FILE-----
recipient: age1ll6hj5ggruetgjwjfnplpn5xtq35uhlcdflksx3xmnjm6s3uad9sz70jkf recipient: age1ll6hj5ggruetgjwjfnplpn5xtq35uhlcdflksx3xmnjm6s3uad9sz70jkf
- enc: | - enc: |
-----BEGIN AGE ENCRYPTED FILE----- -----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBsZzdRMm9zdlJHckFoREVs YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBQQUh2YzJ1MTViYkNRSlFJ
UkVXL3dCeWRuN2NqcnVsOWVNT1FLUHY3andrCmNFY0ZCby9SdnUyYlV6R05YNDB0 OVM5cy8zcldvRHlmY2pIL0pPeWVDQitqcFVNCnhWWjYycGNaRzliZGxQUGdOUlEx
em43SjRCNVhYbmpRZUVWT0cxQlhGdmsKLS0tIDZHazNyck5VNHBuNVM5bmRZUlpR Q1Y3bWplZjc3WHdYUFpOUWQ0NkZOTEEKLS0tIFB5b0pOQVVWM3pRelR0SmlwS1FW
QzBSNy84VDdLVkZZbnNlUFYydXlreEEKYZaR2b7tyRAhPdP+ytpP0veUTi7pY9Nw dnUxTklVa0N3RUl3bUIxRkU2Y3FtNncK7v0Q/fyygDAUr5aSgmillRb0AEE2+JHE
pK0h4hcegLNYJL1AfOYwFQoW7vb256GdmwdcuuBl1YBGXWGraaBnZg== P+ae0gqe2L/i3h01XdiqxAflVXzCF8rhFIG6jf+XCmxQx0eDUSzbLA==
-----END AGE ENCRYPTED FILE----- -----END AGE ENCRYPTED FILE-----
recipient: age120le4a5l8dh3lyfgvmj3d9ksmej6ajs5mer5y7r0vfg3x9fn69dqf8xgzu recipient: age120le4a5l8dh3lyfgvmj3d9ksmej6ajs5mer5y7r0vfg3x9fn69dqf8xgzu
- enc: | - enc: |
-----BEGIN AGE ENCRYPTED FILE----- -----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBFSnZiOGpsL1BESWVEdUxx YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBhRzVrSDQzWGNoL21GU1FC
dXc5LzBHTlNodGJ1S3hOeHd5Wmgxb3BPSHl3CnBEWDNTc3NDdzN3RHQxUlNLOEs5 Z3ByeSsrRU42MWI1WVBjOXNFQ1RyOUluYTN3CkxLQjFBcXFYVS8wTFhkakZDSkZB
RU1SS2tVT29XbDJCWVliWnVkekN4L1kKLS0tIGFqY2pNK2h2S2ZWMndrRVN6eUdN MmZRS1BCNi9Kdy9wa1FRYk9RT0V5TTQKLS0tIGpTV0I4UGZXY3BNYUNZdkxzUFYv
M0NLY2EvTTVIYlZvdk9XY0NwNE14ZlEKcOwgCK3g56kId/4tEt+2iCcylisn+Fca ZEN2L1F0ZXAyb2gwWTRNZDRwbjViMlEKUScRb9D360Ns/ofOn8voUNiJwzJxt7k8
5VWamsXdbwxbxmCEEUbgN3aHrdVz3mV4+8FZqA34yXz8pc45/PUcFg== 9hwsBIiCDvNQtw5tlgEirSD0z5cZJ+6iWiyZbh1FJmFLfmlu8Y43sQ==
-----END AGE ENCRYPTED FILE----- -----END AGE ENCRYPTED FILE-----
recipient: age1qz9d4ka4xgexujyd247s7lp737sulp5fhxl5d65fj2ykvc4j4edqrsdks8 recipient: age1jy444f9d9stygj4p3w9kh54cqcfr654tvr75tdvee5cxsgtdtc9q3v60ep
- enc: | - enc: |
-----BEGIN AGE ENCRYPTED FILE----- -----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBJU1cydFFuRnpCUU5mbjFu YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSA1OTNFVTdHNVlBY1dDVkFH
TTIxREkxZ1hMalVtNjNjcHVaUXpPNWdib2tzCkRScWRTWHhxN3lhckI4UFpMSFRa ZU4xaVlWTkNzcjFKaDBZczU3dTFrSkd1TFJ3CjVxTXpoWXNwemEyU3hhWjZqeVl2
RGxWcldlWnd5SVJFNWxQWjB3R0pMb3MKLS0tIGdjLzlhTHgyT2hjTmRTK1RiZmxS cEZrMWh1V3IxWnFLVE0zR1dkY0o1alEKLS0tIEhwM3ZyNHFIbU5OYzVkYnlkT204
OWRWYklBWkoyUG8raFNzd3JlRUtEUEUKw+NdAp1Mz0dEOUGPbqCV8y7029I3Kye0 RG9ocGd6NWdoRHdVanJPcWFNaStFTFEKFhmCp5Dpxev/ltZaNq4MC5JcTcKJnjUD
keU2T29JGCN1D6x//1NcMUYSaFZKgv0ZVSjVUCl1EnmlJ0nBdbaDXA== Nqby+fqJTcu9WFGqNHHra+I0+ROSAeE9ohhfqOZZg+6HNgzdjqmCEw==
-----END AGE ENCRYPTED FILE----- -----END AGE ENCRYPTED FILE-----
recipient: age120whqj96g26lsgy4udvgsn8dc9lumh8jeu3a564fx79rjr5lxffqmrljuu recipient: age120whqj96g26lsgy4udvgsn8dc9lumh8jeu3a564fx79rjr5lxffqmrljuu
- enc: | - enc: |
-----BEGIN AGE ENCRYPTED FILE----- -----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBnNFlzYVBqaGFDVE9xbmt5 YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBqMUptazlMYjNwekVXRG5C
OVd4ZlZRdEVJd0JFdkRIYVA5ZUEwYjJjWVJjCnZzTExNbDlYbGN6c1dYU2ZlNEVs Rk5YWE9tTm5XVzFOVitjNFEzc1lrRmNqZlVFCmk5R2M3OWZ5SlBoV1YrdkVVbXpQ
ZUo4MXdpc0tzbDI0MHl5eXc3d09VRDQKLS0tIGs4SGMya1RZQ0hpa2NtQk9Wa29C dU1jTTNVWmdYRGtUczBaVmpQMDdpLzAKLS0tIGVMTFRmdUlySTk5WkE2M1VDQXZm
bmxIYWk1VThsaFhxRDlOOGhGQ2tYamsKs+PcOiaeNFujCWwZBr+nq1MzrGohl+ch c0xHUXVCdFJrVFA3M0dHdUROdE50QTgKq5nhd/jlJkMIPCjYsuiMgY4VsW68zEf3
TASI7eNsiHmSVRzSMHv4mX+8yXDHIRgHbFuUty2gdgGFLRjYXe4gDA== qADcbbuYWKfQi/lz92tgMP+d+3GoRjVBIQ2my0giZD8c7ZhibXSLVw==
-----END AGE ENCRYPTED FILE----- -----END AGE ENCRYPTED FILE-----
recipient: age1xjst4frdh0th6q8m7p7u9g5af7ty5jqeum0p6z8a52a9q7st7ewqw8yl9j recipient: age1xjst4frdh0th6q8m7p7u9g5af7ty5jqeum0p6z8a52a9q7st7ewqw8yl9j
- enc: | - enc: |
-----BEGIN AGE ENCRYPTED FILE----- -----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBkcmVQYXV1NzRrQ2IxTElt YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSA4aklLVk5XaUl6dHVsNC9J
QUJRMkhiUUJqRDdwQW03K0lCTEtkSTU5UDJrCjh6SldpQkF1NzIybTJTdnBlOHdQ YUorZGlDTWs4aFIzK0o1dXJSeElLZWpvcWtrCkpOWmFuOUZSUUtNNHZ2dnpENG5v
VlREMy9hM0ZaL21Pa0VPUzcvb0pQQjQKLS0tIFcrK0I3Z3Nhem45eHpZYmdWeit6 Q1MwbVN0bXl1bDA4b2hwNzlBRnFEY1kKLS0tIFZPRHhyYTRRSG15ZU1uSXIyZmVC
eXFtRHVxODhaNTVDdVR1a2tOb1N1U1EKZBdLHq8PRMKX2ndFr3AxFVAZRyvhFa1u eXA5c2FpY1BWamk3dzNXMUtteE91OWsKV+yTdt1MW/7FUxy64abvwjN4OQPaw81l
72R5tordo4IR3HSxG3Z06rokOITd+KKhaQ8NEWEC8qioAAMxEC9QWw== NsOLGSMbZHbQFrqyW1aXTfqYkN1wSuH5rYSZwBL+8gRLBEkFwQtfGw==
-----END AGE ENCRYPTED FILE----- -----END AGE ENCRYPTED FILE-----
recipient: age10at8862478urh0eeuwh8hzln6ck78jgwtztgxatwqlzwagg77y5snm4xzg recipient: age10at8862478urh0eeuwh8hzln6ck78jgwtztgxatwqlzwagg77y5snm4xzg
- enc: | - enc: |
-----BEGIN AGE ENCRYPTED FILE----- -----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSA0WjJPbXhXNTNyQUZzMmxS YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSAwbmpEN0lrNitUcEdyNjFT
Rkd3Yk5Ga3pPRXFtMFpXcmpBZ3RJYnJlaHhjClRDbFBHRVZ0SDdrNFdVWmUwalpT ZUFCUXA2STFTaTFxUXdrWTRuOG51cThTZ2dnCmhadmpNOTJKS0ZvNmFRdkVLWTJ4
WUt5a2RUTnQ4VFBGMlowR3JycDM3aXMKLS0tIERSeFd4cGdSbzRpSmpSQnB1Yng5 ZUpwNGFxMGp2eDdKdjlNTmVCVnc2MncKLS0tIDY3eUZ3SmZwTXhMYkZqOWVTb2Vy
OHBma3lYeWEwQ2RvelpzZHRkN1JPWFEKLXkJfEkH3lo5Z4mj8PZXTfAfZn6B46To WCtxb2toY0UwRmxSemZza2VDOEk0U0EKzk4+Ik7eGUyot4/pCLV7rMY+aNpHFvM1
U0G54tUh0U1EeMnI5ZPRGwDxi5K0eD68HjeE9MBvFBysOChP/ANDPQ== jlShEeqOrPiU1VQ0t4nWSI5E00rpZv+cxs/o1ii9RSUEi7/3VDgHtQ==
-----END AGE ENCRYPTED FILE----- -----END AGE ENCRYPTED FILE-----
recipient: age1ezk9x53zt8kcnscdm80jcyf0xq97vndv7jsn3rl8cc0cwm2jmpmq372dzs recipient: age1ezk9x53zt8kcnscdm80jcyf0xq97vndv7jsn3rl8cc0cwm2jmpmq372dzs
lastmodified: "2026-07-19T02:30:40Z" - enc: |
mac: ENC[AES256_GCM,data:UiL3VMDF6rq4Nr87KspcDx434q3tfNXeb5pwH2O+4ssNQ6xzcYDdzXBnhAY3zLBsqPMKrvHBd4Ot/gEMcq3FMIVe7Q6p9yWKpep66KZ/yWEhAlwIVhD79Oj8VS+1CHKjf25zpRdhZorp04oeFQQd9VfjJB4EE/Q1aVbwTGlpIic=,iv:i/0conaFgFia+wzNTdUL6tlSTw35HTK3Ap1Sr5RGHf8=,tag:ULbz5FllShA/JjlSRdxA0g==,type:str] -----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBJdHBScXYyZEg4Ky9qVUZn
RWpMd3VPVEp0cUc2OUJmWHZwOW1DL3ByVXo0CkNWcldRNVpnVll5eVJTV3FBMTVC
WUJ4a3RZRUFuNFB6b01RVWJLNTZZc3cKLS0tIEF0ckhmWGZ0Y25jcnUyajNxZkMy
akEyWjUrM3RUOGZOZDgxSGlIT2VrejAKXjLTsZ/Qf+77sA2d8kxNublGsCJNiNUa
voxDqyXEYAP9JHk7LZr2iNz2j6lnrJbB80sSerZJAhok8A5cZ/RWsw==
-----END AGE ENCRYPTED FILE-----
recipient: age1fxxzpnfse8nd9wz78ht3m0plrmraacf4cpga0pe8fm2tdnqcgy8q7qsyvp
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBIUjFYQTIrUjFEUytDZDJj
eGU2UlBRV3NobkdyNy9rdjkvWDhVVi84OXlFCjlXSXBmVHljMTJwa25Zc0dIdlFD
RGRkaTI5UmI3ek1oTjZjc2M5UGlHYjAKLS0tIDF2QVpzajdObjhHZVV0dnJXMFZl
VFlJT0M0VjBvSHlrVXoraitkd0IrTjQKU0/plWrjbN5BL782Hls0idImBUHEDg8o
9svRbtapGOqwEVuMoLhPGhJ+FjjHZxYIDVj1o3ZHzs7aVMgiODdAzQ==
-----END AGE ENCRYPTED FILE-----
recipient: age190htw7prp4vln076dxjx3gxxaq06h0zl0te7cqgpx79vl3lhkaes8suy05
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBhRW8xNnNEalNmSmtvaUJM
OUQzQ1M5RFQzNHBVc1NZSUJTekZ3azRwalg4Ck8xUm5HR3UrdUJaOGIyait4ZlFk
RFFXTlY1SlZncy9VQnR1bTVWR3ZZcmsKLS0tIHM2K0oxMWNld3NQSHJhb1ZneWxL
d0J1NEhlMXRnMlZMRkdEMFhwUi9vVkUKgI1e28oU9zTUzGpiZSQUT1k4HVTtPeeV
aqVMxoomQd9Q8Hg9NFmMSqXvbAJm5NC7NVdwUzI5WrfUIGe01a1bFA==
-----END AGE ENCRYPTED FILE-----
recipient: age1ukpqxzl44mnjpy5r96sfuc5sqzm47u4k8ujjh5qdgy6jvl9uqgpspymqfk
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBCSVZkRHVWOWRUSU04cEdL
VG82aXZhTzlaSlZaME9uRDJiUE5ORmlxN244ClVsUlRDVlVhQ0FyaldRQkRLejRY
dHpicE1xektRWXdELzRkZXFwOVJJbUEKLS0tIDRDRXQxbFRJMWFIckIyVnI4ZXRY
OE85QjZOM2R6SnBjRnlYNlZsUFVUZ2cKpdpIA9FO6aX8nSxup6OSYBRGmVAw7uUA
yYwMaRZTd+d/1Xec9WJI03e8oAb0u4WMbok1G3eI1FQJYU22XQ0fDg==
-----END AGE ENCRYPTED FILE-----
recipient: age15kh7akxlx7zn00tey79rq2g8lgs4j5y77rcnyfxrxap8ckfu0a9sqvtdhh
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBKS0hjSStKdW0walhuVDlQ
bVBkL21sVEJ5VGlEcDhsOS9IU2VOR3pmdVFZCmhjSFJac045cWNhV0IwNGRVMGxE
QS9KUHVIZGFQVVZwcFJmTXRSc2VadkUKLS0tIGlXUUZoZ1Q0K1pvRk9HRnM3SC9t
aUc3ME9IMmVteiswUzQ4dWRYTW1ZYVUKogiJ89uSOoeOVklAIox4qzsvNaOhqTIn
DFVVP+nUZuYyqvIhPStAt/O9UpZiBKsTi7rNuTCfzDF5yHMVqQRMtg==
-----END AGE ENCRYPTED FILE-----
recipient: age1ehkswwz2pqaz4svzh7ela5tdnssl8kn6d4vwwxd6zwg8exfpd43syyrrjp
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBLUGxMZnQzN3JJY2ZENy9T
WGlZQ1htZHRjV1E0YVp6ZE9rdHZKS3AwY2p3Ck12eGF5bEkzc2IxQzFjRFVnZTRi
YzZLTlQ2dk9PVjA4ZHRpNTJ4V2U0Q3cKLS0tIFBVVG5VLzdXRjY1THN3QlJMbDNM
bzd5eERLbVVVd1dYOVdVWEFLTWdVbTgK5GAY4iM1+xMBarDrvd/I5VPZxvJJREBd
pB8G7TpkwapsXeLguoSHy8rgc9smMlb0QhpARtyoBJv2bDr0nQRUcg==
-----END AGE ENCRYPTED FILE-----
recipient: age1gl5ujmhd2pe37acd0eaar8tmnf2na28n8l28s8xct40vqpxvsdwqf49g4l
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSA0c0V0bTMzOGdmbnhUenJP
ZXZDVGh5R1pxRDBlQTlqeEpjWFE2Z2FodlFBCklxNTZtc1M2MVNRZ3p4TE5LcU5m
UzlEZkxwTU91VkJ5U2pkbHppSmNuY1kKLS0tIEFvVzl2bHNvVFhKSTJjU0c2YjZ6
UCtjRytHY2J2amdaRHlyYXJDM1JLWTAKX2hb4vJQ43uwCoSOqaofoIUCnWuDeF5t
ir8fkAjpvFFuCTzi5AbajCkncyFXVF7tRTFzWSoj/73FdT3yQ1696w==
-----END AGE ENCRYPTED FILE-----
recipient: age1eu65wsmez68gegnufl0gqrs0e6w2409mypjlajlq383d7l2e3pjqy5kn6v
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSA4VjVuUDZvYThhelBRWXph
QmZZZkw4MFpmNWt4b0pEa1daM2NnQ0ljUXlvCmZDbkp1d1EwNDYrWXI4cDYxME96
OUJpR0VpM1BkMk1TRTIwOEJDaUxieDgKLS0tIHRmSlFBQWl4VEtVNm8wRGZtanFv
RkhyNWppNC9WZHQyajZra0lrYlUzdUEK4GRvBd6bxV5LTRMvoOXrYJJMHKUpX3Bh
oSmmcFkphujOhnZ839jt4XnjdYoTfQ1b/M380Sijn9xhOrHfTWqinA==
-----END AGE ENCRYPTED FILE-----
recipient: age17pwyghxr6lq06fw46gwqzhc9ut4paz28rpwx5pmv3cxwak6rgyjsw7lk3w
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSByT1pPU2RGeEJ2RHFnQ2Jk
WnRISFk4bGRVVzNxOHZKQ3A3dlA1Rno4WTNzCjlsVXRmWk1ZbVkxaTdPTkNjZWVv
QUFLcTlsOW9yZkpic0VGdWtUVHJYWUkKLS0tIFFQTlg5ODBIMG43cTgycGpGQnZx
VEJzaWtvN2NNSHFwSkQyRC8wSGJqcHMKH+UR0NyxjXoWLR6NmiGWphSrl2CgGL3Z
KuIZPGGKFELe79AlH6bOxkcJ7BcYBfVKRFrLzYLfV283DteAfxGMSw==
-----END AGE ENCRYPTED FILE-----
recipient: age1jukmg69cqxnjd0lp5f534jhqe65rxew2hufcmyjxa3rkw6ayef6s2ylcmy
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBsNHdueS9XN20wRzMvQnoz
NUlKeThWSUMxQ2c2ampxbzEvQS9XSWMrdFN3Ckk4YUtVTXdzTmg3QUxjTXhubHA4
azVsQ21yajlteVBjUHBGanYzU2xramsKLS0tIHhBMFNvZnJRc2ROd29vZ2VmdWlI
UStoVitWNjUvYUpkRTc4VFkrV0FOSUkK48TGNM55gQ+MKV8YGIYclv/eMz0nAZ6R
ZiY0XSP/2XE06Y2f3NKBtEauCRGqgwPFt74ITjFOAwitsqIhD0N2Uw==
-----END AGE ENCRYPTED FILE-----
recipient: age1tzsrtwd3p3lrr9g7nv3z5nvmzsz54t2uc6tfqwutp6usmav83s2sck25cc
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBYcUxHbFljWEdBdzBsdklB
VE1IVVNoWFV3YWUzd2RON2ladVVxUThHMXgwClpXVHZUbDV3SHJlSUlTVXhONDQr
aG11YXFWWk11UVRnUG1MOE5FUWRKejQKLS0tIDl2V2M4RXNGTWtIVkgxem56UnZK
RHZXRUZESDFPRzFhUGhJKzlvODJiMjgKN793k0tnuFcfs9PSgdmDXefZ1euAEfC6
DMJ4sXJgwFGX459k0sG8f64eaZrQasg//tcz07J/gRx2D3VyaC6a5g==
-----END AGE ENCRYPTED FILE-----
recipient: age1rf4kj99wuq59k7w8ar326djmgmpl9hcwlnuag07f8gauq8c3y5mqne87s4
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBrejFCSG5VcFgzb1VwNXNL
VnFmVHdaNk1UcnlkRFNscVJ5RFJSbXQrL2tzCllmZ0VVZUpSLyt5UEd6aFNQcnhl
QnNqT1o0Y0pIQWtyVC9OMmtOOXFQK0kKLS0tIGdvV0M0dHc0Rnd6RkJ6KzlOSEg1
YkM3Y1lpNjdBdUxLWlMzOHFwTGJ3bVUK92lLtBdwmLeRD/I+v2h7+InJA71fr6vE
5+BDHs8LAAIgu37QsQKUBKVsEp/elgm73+PQkWgW0/lbHnhevIE3NQ==
-----END AGE ENCRYPTED FILE-----
recipient: age1d0zhx7u3mfs3nktl67npey87cze4dwsfvfvgaje4rh3gwv500yssckcf6u
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBaR01RUDJhOHRtMldiMWxB
U2tSdTExWndFUU1KUmc0M0Ivb0lFRm1NckE0CmZTWERSMVl1SGtENC9keHZGQWMz
bWJnSEdCRXFiMkxESEcwMmtaaXZ5UVEKLS0tIDdsdTVseEFmVE0vbTJtVWVnUkQy
a0FuOUdvaUc5ZCtkZHBmRGFiNXEySjAK6W065nWaOSAs5HFfKRh8Dcbs4VEClXXK
TyKkVcBMxsB3axT7wE7uLi3soHWqaI/tKZceqXfrDuh3m+jNHqglYQ==
-----END AGE ENCRYPTED FILE-----
recipient: age1jc6wx33hdhgwhk6nzy5rr8fkgmqxk9um639tk5h632nqfyaw8czskdptj9
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBKQUQ4UW1GcjNDQ3BDUjVr
QzZ6L2FjN3QwZzlNMTRab1prQmZHRmNZLzM0ClZER3FwcmJwc1dhbzRWYUdONFRr
YlZRbE9CRk41UWR5emVtY204eFRxRlEKLS0tIGFoblJ1RWtuN0d1Q0pqWVZyRStZ
eGg0T0tHN0llQnl0cUw2blB4OENGRm8KHvfSTxceDjVizRzXZ/5GH5NRYzK7AH1K
AP6/JxOP2m+wXqI2xa6kna4k0xaUOZTEo3al59UM5yTcZGlBj87Y/A==
-----END AGE ENCRYPTED FILE-----
recipient: age1pgykvq4pmxhhjrqupcp99fyad2uht40pt79dkzg66cfvsjy5apjqls8u68
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSAwMUIrTWx2QWtuOEFiQjhs
SFlLZVlJM2VTTzJhOWVWbmNkS21MbFZyWWp3Ckp4Rms3djdMWnhKVHh2aEJJN1Zw
N0ZhamZMYlV2djQ0ZGovNGFmVVJNb2cKLS0tIEQwcUlqM1NweWxuWHZBZE9sc21V
ajlqSVVEcUpvUkR0ZFQ3NmNYSjJ1b2MKc6Hj9JU+jNYNzGzzGyIRZjRVGolowl0g
2pXB3VEVTfuAzDTH0hXiKSw8AgJdmNzR0QziXRwgoRIlZqSFKiykow==
-----END AGE ENCRYPTED FILE-----
recipient: age19ht95nv8uhz2shjmakeut8mc3l5spvrcxs3thhe85an7u02r6sysyv457n
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBRZkREQ3RTbDEvSm83S0tV
UXcydk5kaDhKdHN3UzZJZ3FlUDRLTkQxSEFNCkcyblo5N0U2Zm4rSW1KUloxUHFY
VlFNbDJKc2dzTEdjYU0zYXo4NW5OWGMKLS0tIFo1MDBDSlJzUVg3TzdqTU1HeVQ3
VVNhMnZ3bC9jQXpRR09wOGdjLzMyK2sKwBokd7Jvo2Vohzipi8vJPsbxoVWoI/MQ
p2K/TpeWO44A33llWVGevwo8QvGu6zR1ANgUo5b3loj553hR6l6qJw==
-----END AGE ENCRYPTED FILE-----
recipient: age1mhaze5tgvc9lwjpml6dnp3xc292337wgm6376yh6tq4fn492ndjq9h23dq
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSA3QzBuMWE0VGdTMCtIYjZZ
SEo4OWN3ZldnenNFWCs2a0VXN21CK21XWlE4ClpzWFM4dllROGJsYXRTSGhFQjFi
Z3JpYlBCMmh5bjJQekUycWxSanFoaTQKLS0tIGdTWDdRdU1jS0VablNSYWVjdCtm
K3E0Szc3Rko2aFM2b2hzR3hCcDUrS3MKk09b4/pWkp2FqQRbB/Ggf//DN2mfWHxt
ofR0g7o9hg19a9/t34BD1eBfoTtIWfrWTAsZsJ/4+9u3YhFfubeGDA==
-----END AGE ENCRYPTED FILE-----
recipient: age15me6sx0f8r58xh9v7aqrj6e99n7eu555jkpw422txpkqt4r02v0qpahlyq
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBQNU5Va3VHazNyNElWUVU2
Q3NCbzVucDVsNkVSMDIyZFhuYjluR0xNV25zClBTMkdlaDN1c2pOa04wVWI1NDNk
RHZBK0dNWG40TkhxM1NQOHBnektJOGcKLS0tIEp1WStuSDMxejVIeTVFYzFNZ0gx
Y2tUdHdEb3MySm5MT3k1YjNvQnpkaDQKroAPWf5iUKvdK3dHdwBtJT/jQjgi0kCe
6Yuk1ZKlS+6Scf0UkIG/NR0wvebM6O+SvNIcngz/3EVAJyU5mAgwIw==
-----END AGE ENCRYPTED FILE-----
recipient: age1q3hu6eh3mt4saey7dc3yu04s7knnk2ygpm9xt6mg6rqtem6l0uyq4wzul5
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBJdFNBS2Z2SE9pQUliczFB
NExlWFp4ZXAxL0FabWQycVJ1eG1XRTB5RUR3CllFRFhVQjgycGxxdCtmQ21oRzly
T2FNSEd4R3R3MFRkYzYxSndWcGxNVmMKLS0tIGpvMExFeVFHZ2FMckoxYUhSWGVp
ME4xK2VSb0ZSUDBvUFA4Qm5ZeXF6bVEKLdktQF5NQUipJ7X0+d8KJiIC17i60FST
GYg9NR4NKk9m1fxs4UpZD26tzTtPF0MzCJjrzu2/bt2sx6sbOm/2cg==
-----END AGE ENCRYPTED FILE-----
recipient: age1tczst3x7thwtcz4vce4rg6kmlsszzm6j45kn2nv860z9wa93a92s05ppmc
lastmodified: "2026-07-23T21:15:41Z"
mac: ENC[AES256_GCM,data:qFhnPra6IE3wyKQ4WKweON0S0YtD5I0adGZVfA0m6BVilN6bX5oC/1j5NK2oHrsz920hSl0SOF8LrpqOrUyGjSRkPsN4kq8qr9bJcrX4URiktP0oRden5LLt6hf+ZRP7WmRXFqixPkPHJnZIoAvkNnTFce7cDq5NEAHkKUEKG7k=,iv:nyblUDGeu3TUfFivYylOn3C/HITj99qiPI2+mh8AGh4=,tag:FrtRzSCylC4wlIoqZdfx7w==,type:str]
unencrypted_suffix: _unencrypted unencrypted_suffix: _unencrypted
version: 3.13.1 version: 3.13.2
+52
View File
@@ -0,0 +1,52 @@
wifi-password: ENC[AES256_GCM,data:SZQPtU6PYHbf9o83wq3KTupx,iv:FxO68Pn/+N58r/OPLfkAMYPFpP8TYxszMniFd/01E38=,tag:jwxaY6zDEcO5r9OWSfvUyw==,type:str]
sops:
age:
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSAvMGJzZjVoandxc1FCaDR6
Z0hKTVFYRU9JMzA5djk1V3VBZjBETDJDS2xBCnZaRTkzTGg5OVBwckw0cXA4bzFi
OHJpdHFwRFBGbXZCbzUrKytGTGx2S2sKLS0tIGNSdW1rQmdOL2lpek91N051L3gz
K0duYnF5RlE3cmNXUVN3V0hLM0lUWlUKRYmIfAAfGCJOFwlimYk6bAvIKLokKufR
OYnDcUZuPlj9cx/FJNC+7/K7WWmfrpEEeNOVzv7JdTDLjWBslTCWGQ==
-----END AGE ENCRYPTED FILE-----
recipient: age1njap586hc0q43kr03g6c8eqhdsmk8zcafkl3f83xwlc2gqhlmfgs4tmwad
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBGNHJPODlpaHpIRFNVUlhF
WTJsdjB6Rmd6RlEvdDVXckduUkwxUGhVOFF3CkVJMXYzUVRkVjV0VVkzbGc1UlUr
VVVXU0wwYkpzU2pZdmlpaVB3enZ1djAKLS0tIGdkeFVrYkdzM1ZNdElJamhpajFp
YVZuVCtqUW95R3VHTlBadVoyZUZvNU0K/AtAp1bZ3QXKcVChHfMZVI9AHzTktXIB
JNAPKJAxQycMOYPNFklyslOj+oKmvKm5AdXx4XPYW5qOK4k2HMDIQw==
-----END AGE ENCRYPTED FILE-----
recipient: age190htw7prp4vln076dxjx3gxxaq06h0zl0te7cqgpx79vl3lhkaes8suy05
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSA0a0dPSmdFbFRjQk1OWWNo
Q09VSHhmNlUrVjVXQzcrdi9hVklnRjliWG5vCjJ3bVRUNWdPV3d6STEzcjFxZVQr
MXZvdEVOVTlkWEVVNFV5UkUzWThSc2MKLS0tIGIwR1dNMnI3dlIxQ1d6NjhxZ3Vk
a1N5aHQvUU5PSEgwVlprcUNEeVorVGsKmmdhYnpOqk3x+5P0h4AfGJco1PGLmAtS
67hvtBQo1KSTwFyaYQUByfiTwIeMSu50SxvLi56ogT24CNADW6FjUQ==
-----END AGE ENCRYPTED FILE-----
recipient: age1ehkswwz2pqaz4svzh7ela5tdnssl8kn6d4vwwxd6zwg8exfpd43syyrrjp
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBTTXYwS0dXUEI2WjRhZHNp
Qzd5STdlWUNFTG04UEdySUhSODF2NklvNVhrCkxFd1pVSjRmb0hPaFpyWCtoQi9K
eHVBcUU0amt3K1BONnV5Vnc2Z0JoazQKLS0tIGFrL0I5SjJYYThUS0Y0VVlraW9x
Z1VLT1hpNWswd3FsWWVGRTkrNkJOZG8KY74xa+6Pfc9lpupTodTBa1hqIrotE2Gu
H5+0g5D9838zxf/p7pzxh7rE7eo2gd9JT9bW2I4Pwn257Omblsyv1A==
-----END AGE ENCRYPTED FILE-----
recipient: age17pwyghxr6lq06fw46gwqzhc9ut4paz28rpwx5pmv3cxwak6rgyjsw7lk3w
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBpUExNcmw2S3hSKytvMVQr
ZGJkVStyVjgvTURPSTRyMXZkdjdpRnhQZFV3CkVtQzBsUjFlRjNsc2dwU0R4UnlL
YkhtZWVNSW5zWkhJM0NwOUpkVlJGQVUKLS0tIEJiTWxQT0dPeUI0TTIvRys0Sk9s
RDFmcjZ6UGkrbWlFb1h0bmdRUlQwT28KK5vH6Svg8lFUKqXSzweb9405B0Z91TaW
65G51im23qSOeD+NLBHP+nqIwYrjDlODrgOeW9Up/9rE5rLgFfjoOw==
-----END AGE ENCRYPTED FILE-----
recipient: age1mhaze5tgvc9lwjpml6dnp3xc292337wgm6376yh6tq4fn492ndjq9h23dq
lastmodified: "2026-07-22T01:15:21Z"
mac: ENC[AES256_GCM,data:dC/oIqMUHkOh3AocOwP7Gc6XGH3L+nTqJfhFNts1DNbRXsopNIxVBtIz2pEhwnWSQrqPisDLmPHFBwRpGVn01u8w8IU1FKbAKC0J2nJXF8ozpInbjzDOmehqPWZG7yaKoq8cwAnp5XOk+IVO4l6tPxLxkExU5fT2ALuMq+sgOko=,iv:jaVyArpf6zMCFa6J9X1aQMGrmFq+W2CPZdWO6vVW68c=,tag:S+qF8/FkgHc4uW0e4ICmSQ==,type:str]
unencrypted_suffix: _unencrypted
version: 3.13.2
+33 -15
View File
@@ -4,31 +4,49 @@ sops:
age: age:
- enc: | - enc: |
-----BEGIN AGE ENCRYPTED FILE----- -----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBPWE1HTUhiSUp5ZEUwWEpI YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBYcFFSVFVxOWp2dCtEVzg4
bGpkZlBIMUo5ZlYrQ09SN3Q1a0ZkQ0ZnOEhVCnJPNEZQenVWWGZiODlzQzNEc1Zq VFF3Mk94Q0xVSDZNL1lBSXBKNzVXaU05TnlzCnhueTd1ek1BZ1JpZUMwM3ZBUHFh
c3l4OWZJTElJc2Y2UE15OGtEUzhyY1EKLS0tIHNJUStyWnlQWjZBbEZjQ3UwdUpz UVFKZjduZUpUejJ0enBXK0pwMjZKSDQKLS0tIHJ1Mi9OQThrWndiWFE3WTRJREJN
ZndoUDR6bisrNGJCUHk3TGI4bTZaMFUK87fFsm9ne9s+PK2pcwtrDjqyGBss2r2E Z2E5WTI5dzB3WDFtaFRoUERZa3N6TDQKy2FK57wDNLSPl2/L/FUBUP8rJZuAOqAR
8lhqoeiKZ2j96z8kP/7ChzovwTCmqdcmAQuyNQD+ZAFijseipSvfbQ== I0hBe4epAKL28PlJ4M9P77qqg893beZK3UIJQ5O6d5fGPMNTJiIzAg==
-----END AGE ENCRYPTED FILE----- -----END AGE ENCRYPTED FILE-----
recipient: age1njap586hc0q43kr03g6c8eqhdsmk8zcafkl3f83xwlc2gqhlmfgs4tmwad recipient: age1njap586hc0q43kr03g6c8eqhdsmk8zcafkl3f83xwlc2gqhlmfgs4tmwad
- enc: | - enc: |
-----BEGIN AGE ENCRYPTED FILE----- -----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBUYi9SRFFGV3Z6cFd2Znk5 YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBoaEZ5clpxV0ZQWmhSckxY
b2FLbWtzTllJMDBUaGk0NTViOTNBa2hQclVZClZHKzNhbGVjQUJhWkFWdTFBMG5a a3I5djhISUphMmV1RU9DREJsZFpDdktyaUJzClBQSUNvN054aXp4SzNkMmNqNVhP
cUFJdUdyVG5HQXJRRnJId3hqRTN2cXMKLS0tIEFMRjh3WE1ON0U2TTNTZ3hxMTR4 QUdnUEppMmdENzUraHAyRVJLWm95V00KLS0tIHZuVXh4SkdhVU05YzV6VlI2c2RS
ZGRlemlIbDZKeExmVHROc3Eyak5DdzQKaLwIVDi6BN4cxpVxJoqTYvJETPOp4thc SWttRW9nS29MM0lJT2x0QS9rVU4yb2cKYZ1gts1qRFdod8PFWl6Tg0Ry0sXhKMrP
l9uVMvIGuEsEZgDsvShw1dYLljd+uGy/A+dXbcxIUCP/mmPkwmd1Pw== VwttGQj0iDg6UudB+qa2ngFgAc0C++OJebuLMFnK1K+PvUpfw2nRQg==
-----END AGE ENCRYPTED FILE----- -----END AGE ENCRYPTED FILE-----
recipient: age120le4a5l8dh3lyfgvmj3d9ksmej6ajs5mer5y7r0vfg3x9fn69dqf8xgzu recipient: age120le4a5l8dh3lyfgvmj3d9ksmej6ajs5mer5y7r0vfg3x9fn69dqf8xgzu
- enc: | - enc: |
-----BEGIN AGE ENCRYPTED FILE----- -----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSArOWovSW9DeFpxL0VDUDQ3 YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBKNXBOeS9od1lBd0VGSDRX
SHUwTzJVZUtPV01ZRkdCUXZGL2lTRCtCNFNnCjBSNExqRW5mTEN5SFVucHJHSzZt RE5BM3g0Zk53ZzBhdW9XUFhscmpHdEV1aG1vCnU0OGZZdENBdHd0Tzd5b05BR3RR
cDlNc3BjY3M1c1k1Z2tkVEg4R1pacGsKLS0tIEFWbHNKZW0vbVh1Y2VhQW93OUwx ZGNZRlVSVk41YzgyRnlHK1NUaXhqZVUKLS0tIGFKcWF3VGdibldnMndmSis5Skc5
MWV0eW9sOXdQd0l2ZjlWOEVVc1dwcTgK2s4p9xoNkawH2OkGsl80bNIo3ad5vn4W aEZJVGh1UDFVUldrRndVZURGbVJMUXMKia/z+zwpfcaILCt4fpy7wsR0QkFKf07d
Z2w+jwppSoUmbQnD3WFbLmSSxmuobmU8HILwElv6SZu+KE3aspF6XA== QwkV+gEfM3EydhcA7invDieIKbcOUoKKgOmf2sXMgW07CsjG1QXk6w==
-----END AGE ENCRYPTED FILE----- -----END AGE ENCRYPTED FILE-----
recipient: age1xjst4frdh0th6q8m7p7u9g5af7ty5jqeum0p6z8a52a9q7st7ewqw8yl9j recipient: age1xjst4frdh0th6q8m7p7u9g5af7ty5jqeum0p6z8a52a9q7st7ewqw8yl9j
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBpcTJHRy9CZlVWcERMRzlL
MVFyVHlRT1hnQTBsSlBYOWtoSzVyejJIM0RzCmFKM3J5cE1BejhHYVl0VzNTSXdj
dENsTCtJT0JFT3I0eGJtOFMvdjFmTm8KLS0tIE1zSmhBMjNnelQrdkJ3eVR6SFo1
QXJtMVRuS1daOU53dmxGdFIwN0ZOVW8KS0XIDkbp+1W+wUDHkaKXZ84QpcO0zK54
FrMzkChowtm/a+RSLVadBGDQv2R4gV89NArP/YAEQa7GK9aHvDlH/g==
-----END AGE ENCRYPTED FILE-----
recipient: age1tzsrtwd3p3lrr9g7nv3z5nvmzsz54t2uc6tfqwutp6usmav83s2sck25cc
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSA2bnJZUFVVRTBrZExoSWJV
NDI2WUluaWJHVVdiR1JBUjhJT0owaTNoL1JBCjh2Y3ZvZXYrSE4raGtrcXVZai9L
cUFERVRvMEZPaC9oa01NSHR1ejBIdk0KLS0tIG5ZNmZqZWZxKzd4c2pYOEhwa0li
NDRQV1hDZ3ZwZksyTUtjcEVGWVpFMTgKmpLn5Ov03MKITQ7QihjK4GsYlHQiAFAc
pI7z78qQv34e7DLLJotxbeWYyX/IOe+DvUm9T8NGY4ujI2y3HjXPlQ==
-----END AGE ENCRYPTED FILE-----
recipient: age15me6sx0f8r58xh9v7aqrj6e99n7eu555jkpw422txpkqt4r02v0qpahlyq
lastmodified: "2026-07-19T23:30:21Z" lastmodified: "2026-07-19T23:30:21Z"
mac: ENC[AES256_GCM,data:kLGE2xawQT7mx+sfw68hmGk5nCEGiEjZrqTEl9B1dtQmTrMwmoVr/1RISi4LfJrwxy31mDgff4lcIL4wIJuM373uk3X8j4RNyYQNTfKEkORT6r8NHeepNs267O77pKGd7OmcM4MT/BqOnB8ELS7Wlf2ect7CAlvUUVyc8icxgZE=,iv:EYLDsHYHZ1XOQXafOTqHHWpk/OBNq/R6IJnOBYV33E4=,tag:thxrCPC5oGvDjhK7Dz87YA==,type:str] mac: ENC[AES256_GCM,data:kLGE2xawQT7mx+sfw68hmGk5nCEGiEjZrqTEl9B1dtQmTrMwmoVr/1RISi4LfJrwxy31mDgff4lcIL4wIJuM373uk3X8j4RNyYQNTfKEkORT6r8NHeepNs267O77pKGd7OmcM4MT/BqOnB8ELS7Wlf2ect7CAlvUUVyc8icxgZE=,iv:EYLDsHYHZ1XOQXafOTqHHWpk/OBNq/R6IJnOBYV33E4=,tag:thxrCPC5oGvDjhK7Dz87YA==,type:str]
unencrypted_suffix: _unencrypted unencrypted_suffix: _unencrypted
+46 -10
View File
@@ -3,22 +3,58 @@ sops:
age: age:
- enc: | - enc: |
-----BEGIN AGE ENCRYPTED FILE----- -----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBURmMzN3hrSlNrUkkvVWNl YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBwbWFOSlB0ckJFc2hiQTlh
L3M1dEhWeW14N0RFNVRPci9QK1YyTFdqRVVJCk5WaWswT2NicldkYzZjbVhYU2xu eEo1NzZSTWovRXNnTGJDWTVZRUsvbjQ2NGx3CithSDVSbFpCYW9KQjJndzR3aVlL
MGFsNmUzeTN2TS9wOEdvRURpVUVYZXMKLS0tIDZ6MEdPTVhCaTQ2UXFWTUFtc0pm MHBDRmQxSEpTZ1FsRXFWQlE2aFc1VGsKLS0tIDc5bmI4WWtvcStZVk5WNDhva1dk
MFlJb0c2WXJtMGRLZEZYY0pZWWpFWm8K/mlYZIe8UC0QU+1mq3NtrtTF5b2m5hCK cEVKM2ZCTGdLM2NEbmRudk1SdDNzMTAKB6QCrxxNzazSbif28j6RNIfdfUTKxklM
+K0QiZLTKmmDcr4bRhZ32VE7R7GRwtMNnOP/mElZvPAyWyHHhRiOHg== Xip5xoLjTRNOjq561OS34gaMA1GE27YQqsqtgiU15i7SZ7IGefWpZQ==
-----END AGE ENCRYPTED FILE----- -----END AGE ENCRYPTED FILE-----
recipient: age1njap586hc0q43kr03g6c8eqhdsmk8zcafkl3f83xwlc2gqhlmfgs4tmwad recipient: age1njap586hc0q43kr03g6c8eqhdsmk8zcafkl3f83xwlc2gqhlmfgs4tmwad
- enc: | - enc: |
-----BEGIN AGE ENCRYPTED FILE----- -----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBURHlUL0RNMUtNallEcG5p YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSB2MDVmZGp0bEpwZWRLZFFM
eG4xby8yVzdQUTBaNkl4ano0YjBMcDd0Wm5NCkZvaXNPZm9wemJkMmNSdGdOaTI4 ckY1eERsSHFuMnhPK3J0eFRLY3lHYnlEK0NnCmNhQjlnUTNFOVF0UVBLendHYmdx
Z1RwUnhiRUpCMWZaeWtlSVBmNW5KOXMKLS0tIDk0R0k3ZHczTFNCWUZxSWF0M0FJ aXVIa2hmNENUNGdZL2FmbTlNekd5NUkKLS0tIDdwbmFGZVFlaG94UEgvOUNxdVlz
MGlZMmtuSFYrcG1meDNMWDNqSjFxcE0KDu2dAc0gqmmPkpbpBe4YohM7rYmUwEkI VHJZenplYVVPa2NUSUFWeEN5bGQ1MkUKT5CqEiCN61aOpvMlBp0effS3bh345E7u
V2FUQwjlvh50svtjCVdYbx2xuq4sQLnKelk/q1onLw60FwsVfzD8sQ== wfxrMwMhrEzfnmpkvcafLCn9PMicqLuz48J9ipF5IHLf7MIvbT31ow==
-----END AGE ENCRYPTED FILE----- -----END AGE ENCRYPTED FILE-----
recipient: age1ll6hj5ggruetgjwjfnplpn5xtq35uhlcdflksx3xmnjm6s3uad9sz70jkf recipient: age1ll6hj5ggruetgjwjfnplpn5xtq35uhlcdflksx3xmnjm6s3uad9sz70jkf
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBPS05tWG41Z3BwREhxUVRY
ZVBMQjhxRDNwUDBjVE1jNnlVOWZtcE81Smk0CnlHaEl2OTd2UnJhWmlHNmY2V3NQ
SUJKakZOZkdXeFhnZTF6dmxzTGFzZjAKLS0tIFhwKy84Z0l3cEhSQ0NBbVJIKytM
QzhxbTY0SkdkOWN5VDU2d0dTMzlGN00KtgobQ2hsilZnvoq/BesaM/RcYbVNHxSh
vEA80nUhyEdRim8WP/yu5KrgHsEVQ9ryclU4B49F1pINdAVaGRo9LA==
-----END AGE ENCRYPTED FILE-----
recipient: age1ukpqxzl44mnjpy5r96sfuc5sqzm47u4k8ujjh5qdgy6jvl9uqgpspymqfk
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSB5NUxUMHdrbzl0OEl2RXoy
S1ovL05lTXhSNXBXZEtWU2hsdXhBZHlqc21rCnY0a1Z4QXZnTnJyTDR0WVgyYk5m
cHJJZkhoM2lNaG51TmY3K0hZeXZKM1kKLS0tIGU2MmV1TTFTY3QyQVNEb2ZxYmlU
c1doTkVLZEhKeDJTWmtLWjIrclRUenMKRem1nKPK/ghuipzOLTf6V3Ke5LK5S6LW
5ExYf7y++FC90IqkAqeIA67l2KihcPE0DqnBmwrDlNgL5tXyhYV39g==
-----END AGE ENCRYPTED FILE-----
recipient: age15kh7akxlx7zn00tey79rq2g8lgs4j5y77rcnyfxrxap8ckfu0a9sqvtdhh
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBvZkxwdUhOU3diRVFjSkhH
SmFDbjYzblFUZ2NLRmpiVGxuUGVrWC9BWERZCkl6MVJ0M3dvNGU0UmQ5Nm1VWmRm
ckVjSENWb05qMWtLQzNJd0tUWmgydWsKLS0tICtydnQyUXluWmxwWVpQWVRqOW9L
VjhoelJCZjM4aHh4UG0wcUJMSVFGbncK6B1Rou2axRoRQNXudD6BVq5buz9MJapB
vYMEhQPVCU2SinltqIrgd00qjgE7+oPnGr5pTFSQisJLlN3X1LUbhQ==
-----END AGE ENCRYPTED FILE-----
recipient: age1rf4kj99wuq59k7w8ar326djmgmpl9hcwlnuag07f8gauq8c3y5mqne87s4
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBvV0E2MnpDZzNkbC9DNGZ4
U1JLQ3J3ZHo2cEpraFlSdTA0V01wdlNaMkE0CjQ0SDBXT2tYU2VmZjNja0M0VG5K
Z1FkVWhYODVDNkhBR0o1NW43SjZRZW8KLS0tIEtrMFI0YjZIQzd5bHVMSmRSWi83
U20zMkg2UWpEclMxalNhMThlWXdhaDQKX6qsiRZqlFVhq5lP5B/XVqStq8Impvx9
MTmVRuMzFQ5qTs4BjhIqO25Ez+CI2GAq2K10prdSEin69gSTxntkIg==
-----END AGE ENCRYPTED FILE-----
recipient: age1jc6wx33hdhgwhk6nzy5rr8fkgmqxk9um639tk5h632nqfyaw8czskdptj9
lastmodified: "2026-07-19T02:30:40Z" lastmodified: "2026-07-19T02:30:40Z"
mac: ENC[AES256_GCM,data:rKHZjU/MH08ASTlu32HZO9uWmsBYuMCEC6M8gwVhzuWvmablnP05tS2z13XfaWaCEUXk6kmGJKuU0zu5+IKVZgamCF6DAMtxQb6bVCaLsoAm/GSqWQ5VI9eHqgnSSdN/o3ul/33Rf8iBQo4aw8FFAmDVuNz8bfAn0QefFTj0ByI=,iv:JD2gtqRinOY77etg6PUmZNovkYl1Q3F6ZvRi4x7RznQ=,tag:/5IMpWKRVt+l1luCTQE0BA==,type:str] mac: ENC[AES256_GCM,data:rKHZjU/MH08ASTlu32HZO9uWmsBYuMCEC6M8gwVhzuWvmablnP05tS2z13XfaWaCEUXk6kmGJKuU0zu5+IKVZgamCF6DAMtxQb6bVCaLsoAm/GSqWQ5VI9eHqgnSSdN/o3ul/33Rf8iBQo4aw8FFAmDVuNz8bfAn0QefFTj0ByI=,iv:JD2gtqRinOY77etg6PUmZNovkYl1Q3F6ZvRi4x7RznQ=,tag:/5IMpWKRVt+l1luCTQE0BA==,type:str]
unencrypted_suffix: _unencrypted unencrypted_suffix: _unencrypted
+25
View File
@@ -0,0 +1,25 @@
beszel-token: ENC[AES256_GCM,data:gjbT3uROiVKQOJaUeafTxjVknQO1Tvbyx/Pl2bTad7DezByX,iv:3ikf7OaT2omO8yd6G6UwYbaRBSzyvbn+NghxAe5bcgI=,tag:Zuc2EP8rUtdDhr5CzSW2Pw==,type:str]
sops:
age:
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSB5QVJmWlBlNmFTQ2RUaWdI
amdKLzdRVVdxdW5ROU9CRldKanU5QXkzc3hZClBGRWp5ajNFWWcrdlJaM2hEK1B0
emllQ2JLSkZFMG03WjNvQkc3RThrUlkKLS0tIElnR0tZVTlvTk00OGhrZUorUGJo
RGYraGU1dUg4SGJGcnVqdklINTdNbHcKeAotrwsYxsE4ktOzUgEaIibRprGf7pXQ
5GttilLcmfQNBeaA8z870impTdj6EZfUVItwaIzhN0dG9TcK7oJBoA==
-----END AGE ENCRYPTED FILE-----
recipient: age1njap586hc0q43kr03g6c8eqhdsmk8zcafkl3f83xwlc2gqhlmfgs4tmwad
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBHbFY4WVp6OVNydnJ4eFc4
OEFDTzBTSzlVQkh5Vko4Rk80VURHSjlqbzBBCnRJNTJZRUNPaGNVUm9FV0ptdjVD
d3RUV3l2VERUMXhZVGRXRmsyR05lQk0KLS0tIC9RM052SHRrVjdyZkRhMk04RWI4
UHVxYXp5N0NKaCtmZ291MjNNakFwb1kKB3RlWa0adUCgaIUxGaN8Uzwkak7EtmX2
rBnOkujIVWMy9KHPltZtfRAQrY7nS2njhK1hFfTSomqdoJMFgH/iCQ==
-----END AGE ENCRYPTED FILE-----
recipient: age1gl5ujmhd2pe37acd0eaar8tmnf2na28n8l28s8xct40vqpxvsdwqf49g4l
lastmodified: "2026-07-23T23:32:57Z"
mac: ENC[AES256_GCM,data:l9a/yNRoxY1hvSkLuR4N7deeKue/1JPlSvZvJfCSNbQ21p1qR433BbSDYvfW+kXQXS8GVcfgXSd9ywNzgVvkA5lR1++uYsZBLbYxJ+s3TKWs6/yECAZ0eM1KBA0BEm7cLSsHTOwd+2WspvmCYir++FDO9XRuS3guiMnQBglDf/E=,iv:eE6GVqexQNSiLYfmTTUdUx5AO//wyjSIsr96xAX1pcI=,tag:CNsoyDZYLUt5Seu7W5wJrw==,type:str]
unencrypted_suffix: _unencrypted
version: 3.13.2
+15 -1
View File
@@ -26,7 +26,7 @@
# fresh client that has never manually ssh'd to nix-cache before. Update # fresh client that has never manually ssh'd to nix-cache before. Update
# this if nix-cache's host key is ever rotated or the host is rebuilt # this if nix-cache's host key is ever rotated or the host is rebuilt
# from scratch. # from scratch.
nixCacheHostKey = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHrMKZlIGUd3pH9G3AqbsruqUGjxIXMAZw52u9MwiBCn lxc-nix-cache"; nixCacheHostKey = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPeWgMsdaiz4axT/deFc1+0B5bN+GX/NOeW9bbQ0c/IT lxc-nix-cache";
# Public keys authorized to SSH in as remoteBuilderUser on the nix-cache # Public keys authorized to SSH in as remoteBuilderUser on the nix-cache
# host (modules/nix-cache/server.nix) — one per client host that's allowed # host (modules/nix-cache/server.nix) — one per client host that's allowed
@@ -45,6 +45,20 @@
# the installer image's nixos/root users. # the installer image's nixos/root users.
adminSshKey = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCq/Q5LvIXlZwO2kdeAN5nLGZ59nZB7JHYMEszHxmNtGMzv1lM31jiPNsr0z2EKVZhE7OOfa2IF9rhWYD7JUA9G0yzdZ4WTXFNGVVOJoOVH6vAF3XCxoVilOEwTc7h2Wiy+rzd0B28/3spffzQQWJhY6GRQVa8j+6xAGF60Fcvl1vLosYT9Bn2ZbK4TCWOwAn2jqXIieGpZdn/UNZbGOeKRiCvhktDfMAzuQzN/9jMu/oF4pkPn2X1UrsQdNlvp0Ci8md612MozIpncQJyAF1ADhunr3sMx0isUXiqD29R5DS4TftpekqLNLak+zcxFa8N7DcRNp3DcKfJvyTkwQrR4r+b7lFLYOLHLagSso9CzeW/paAS2q9I5SBm/2DtE1diLLg2jZikYcstsu/G5RgvbzbKqjiaMwTdXC3AMvDxQrs7U5pDRZFzoofG3cpODbTm+uy3m0kP70z0M1K45UbDG0p+itnTu9x40JbQEgefbx38AItNvAIx1A8HO4I1VX28= wayne@stream"; adminSshKey = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCq/Q5LvIXlZwO2kdeAN5nLGZ59nZB7JHYMEszHxmNtGMzv1lM31jiPNsr0z2EKVZhE7OOfa2IF9rhWYD7JUA9G0yzdZ4WTXFNGVVOJoOVH6vAF3XCxoVilOEwTc7h2Wiy+rzd0B28/3spffzQQWJhY6GRQVa8j+6xAGF60Fcvl1vLosYT9Bn2ZbK4TCWOwAn2jqXIieGpZdn/UNZbGOeKRiCvhktDfMAzuQzN/9jMu/oF4pkPn2X1UrsQdNlvp0Ci8md612MozIpncQJyAF1ADhunr3sMx0isUXiqD29R5DS4TftpekqLNLak+zcxFa8N7DcRNp3DcKfJvyTkwQrR4r+b7lFLYOLHLagSso9CzeW/paAS2q9I5SBm/2DtE1diLLg2jZikYcstsu/G5RgvbzbKqjiaMwTdXC3AMvDxQrs7U5pDRZFzoofG3cpODbTm+uy3m0kP70z0M1K45UbDG0p+itnTu9x40JbQEgefbx38AItNvAIx1A8HO4I1VX28= wayne@stream";
# Prestaged wifi SSID for the gui host's NetworkManager profile
# (modules/networking/wifi.nix). The password is not here -- it's
# sops-encrypted in secrets/gui.yaml (wifi-password) instead, since this
# file isn't a secret store.
wifiSsid = "nbn-fttp-net-5G";
# Bare-metal gui host's two disks for a ZFS RAID0 (striped) root pool
# (modules/disko/baremetal.nix). Only used transiently at disko-format
# time (partitioning); the resulting fileSystems/zpool import reference
# by-partlabel/by-id paths afterward regardless, same as
# modules/disko/proxmox.nix's own plain "/dev/sda".
guiRootDisk1 = "/dev/sda";
guiRootDisk2 = "/dev/sdb";
# System # System
timeZone = "Australia/Brisbane"; timeZone = "Australia/Brisbane";