Make lxc-docker a privileged container: unprivileged can't NFS-mount at all

The kernel's NFS client filesystem doesn't set FS_USERNS_MOUNT, so mounting NFS from inside any non-init user namespace -- exactly what an unprivileged LXC container's UID-mapped root runs in -- is rejected at the VFS layer with EPERM, regardless of Proxmox's mount=nfs;nfs4 container feature (which only patches the AppArmor layer). Confirmed live on the redeployed lxc-docker container: TCP to the NFS server's port 2049 succeeds, the server's export table matches the container's IP, and mount.nfs: Operation not permitted still fires immediately with no corresponding denial anywhere in the server's own logs -- a kernel-level rejection that no amount of DNS/ automount/export tweaking (this branch's earlier commits) could ever fix. modules/platforms/lxc.nix now keys proxmoxLXC.privileged off hostName ("docker" -> true) rather than a blanket false, since build-types/docker.nix is also composed for linode-docker/proxmox-docker, which don't import proxmox-lxc.nix at all -- setting this option there would break their eval. create-proxmox-resource.sh reads the value back via a new flake_target_lxc_privileged helper instead of hardcoding --unprivileged 1, so the two stay in sync automatically for every lxc-* target. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T48qgH3VTvs8wvwj44FEbE
Fix NFS mount device strings on lxc-docker: use FQDN, not search domain
2026-07-21 05:57:06 +00:00 · 2026-07-21 01:19:42 +00:00 · 2026-07-20 18:22:12 +00:00 · 2026-07-20 17:49:35 +00:00 · 2026-07-20 17:46:07 +00:00 · 2026-07-20 17:32:25 +00:00
32 changed files with 871 additions and 272 deletions
@@ -13,9 +13,17 @@ jobs:
    steps:
      - name: Check out repository
        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0

      - name: Install Nix
        uses: DeterminateSystems/nix-installer-action@v19

-      - name: Run maintenance checks (secrets, fmt, lint, eval)
+      # Scoped to files changed since the PR base / previous push -- see
+      # scripts/codex-maintenance.sh. CI never passes --full-check: that
+      # full sweep is for local/manual use, since it's slow enough to time
+      # out this runner.
+      - name: Run maintenance checks (secrets, fmt, lint, eval -- changed files only)
+        env:
+          MAINT_BASE_SHA: ${{ github.event.pull_request.base.sha || github.event.before }}
        run: bash scripts/codex-maintenance.sh
@@ -13,9 +13,17 @@ jobs:
    steps:
      - name: Check out repository
        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0

      - name: Install Nix
        uses: DeterminateSystems/nix-installer-action@v19

-      - name: Run maintenance checks (secrets, fmt, lint, eval)
+      # Scoped to files changed since the PR base / previous push -- see
+      # scripts/codex-maintenance.sh. CI never passes --full-check: that
+      # full sweep is for local/manual use, since it's slow enough to time
+      # out this runner.
+      - name: Run maintenance checks (secrets, fmt, lint, eval -- changed files only)
+        env:
+          MAINT_BASE_SHA: ${{ github.event.pull_request.base.sha || github.event.before }}
        run: bash scripts/codex-maintenance.sh
@@ -3,11 +3,12 @@ keys:
  - &docker age19gfn2yedg76dmztm4hncr7vf3r3c9j0qpt4rap7y7gersjk4m3ks2lhd0e
  - &server age1ll6hj5ggruetgjwjfnplpn5xtq35uhlcdflksx3xmnjm6s3uad9sz70jkf
  - &nix-cache age120le4a5l8dh3lyfgvmj3d9ksmej6ajs5mer5y7r0vfg3x9fn69dqf8xgzu
-  - &lxc-minimal age1qz9d4ka4xgexujyd247s7lp737sulp5fhxl5d65fj2ykvc4j4edqrsdks8
  - &nix-minimal age120whqj96g26lsgy4udvgsn8dc9lumh8jeu3a564fx79rjr5lxffqmrljuu
  - &proxmox-minimal age10at8862478urh0eeuwh8hzln6ck78jgwtztgxatwqlzwagg77y5snm4xzg
  - &lxc-nix-cache age1xjst4frdh0th6q8m7p7u9g5af7ty5jqeum0p6z8a52a9q7st7ewqw8yl9j
  - &lxc-docker age1ezk9x53zt8kcnscdm80jcyf0xq97vndv7jsn3rl8cc0cwm2jmpmq372dzs
+  - &lxc-minimal age1jy444f9d9stygj4p3w9kh54cqcfr654tvr75tdvee5cxsgtdtc9q3v60ep
+  - &lxc-pxe-boot age1fxxzpnfse8nd9wz78ht3m0plrmraacf4cpga0pe8fm2tdnqcgy8q7qsyvp

 creation_rules:
  # Shared across every currently-deployed host: root/nixos password hash,
@@ -26,6 +27,7 @@ creation_rules:
          - *lxc-nix-cache
          - *proxmox-minimal
          - *lxc-docker
+          - *lxc-pxe-boot

  - path_regex: secrets/nix-cache\.yaml$
    key_groups:
@@ -35,9 +35,14 @@ Use these commands when validating changes:
 ```bash
 bash scripts/codex-setup.sh
 bash scripts/codex-maintenance.sh
-bash scripts/codex-maintenance.sh dry-run
 ```

+With no flags, `codex-maintenance.sh` scopes fmt-check/statix/eval to files
+changed against a base ref — this is what CI runs on every push/PR. For the
+full sweep (every host, every package — slow; CI never runs this), use
+`bash scripts/codex-maintenance.sh --full-check` (add `--dry-run` for build
+planning on top of whichever scope is active).
+
 Host evaluation is safe when limited to drvPath checks:

 ```bash
@@ -37,11 +37,22 @@ machines when deployed.
 # One-time environment bootstrap (installs Nix if missing, prints hosts)
 bash scripts/codex-setup.sh

-# Full validation: secret grep, nixpkgs-fmt --check, statix lint, eval all hosts
+# Changed-files-only validation: secret grep (whole repo), nixpkgs-fmt --check
+# and statix on changed *.nix files, eval of the hosts/packages those changes
+# can affect. This is what CI runs on every push/PR.
 bash scripts/codex-maintenance.sh

-# Same, plus a dry-run build (no result symlink) of every host's toplevel
-bash scripts/codex-maintenance.sh dry-run
+# Full sweep: nixpkgs-fmt --check/statix over the whole tree, eval every host
+# and package. Slow (minutes) -- CI never runs this; use it locally before a
+# release or after touching modules/common/*, flake.nix, or variables.nix for
+# extra confidence beyond the automatic full-fallback those paths already
+# trigger in the default mode (see below).
+bash scripts/codex-maintenance.sh --full-check
+
+# Either mode, plus a dry-run build (no result symlink) of every host/package
+# in whichever scope is active
+bash scripts/codex-maintenance.sh --dry-run
+bash scripts/codex-maintenance.sh --full-check --dry-run

 # List the hosts the flake currently exposes
 nix eval --json .#nixosConfigurations --apply builtins.attrNames | jq -r '.[]'
@@ -58,36 +69,96 @@ maintenance script pulls them via `nix run github:NixOS/nixpkgs/nixos-25.11#<too
 There is no test suite — "correctness" here means the flake evaluates and
 `nixpkgs-fmt`/`statix` are clean.

-**In an interactive agent session**, prefer targeted checks over full-repo
-sweeps: after editing one or two hosts/modules, evaluate just the
-`nixosConfigurations.<host>` you touched (plus any `config.system.build.tarball`
-/`diskoImagesScript`/package output affected) rather than looping over every
-host — `codex-maintenance.sh` evaluates every `nixosConfigurations` host plus
-every package/tarball/image variant and is slow to run after each small
-change. Reserve a full
-`codex-maintenance.sh` run for changes that plausibly affect every host
-(`modules/common/*`, `flake.nix`, `variables.nix`) or as a final check before
-committing. This is a session-workflow preference only — it does not apply to
-CI, which should keep running the full script on every push/PR regardless of
-diff size; that's the point of it.
+With no flags, `codex-maintenance.sh` diffs against a base ref (env
+`MAINT_BASE_SHA`, else the PR base SHA in CI, else `HEAD^` locally) and scopes
+fmt-check/statix to the changed `*.nix` files and eval to the hosts/packages
+those changes can affect — a `hosts/<name>/host.nix` edit only evals that
+host's targets, a `modules/platforms/<platform>.nix` edit only evals that
+platform's hosts, and so on. A change to `flake.nix`, `flake.lock`,
+`variables.nix`, `modules/common/*`, or any other `modules/*.nix` file outside
+`platforms/`/`build-types/` (whose blast radius isn't safely inferable from
+the path alone) falls back to evaluating every host and package, same as
+`--full-check` would, just without the whole-tree fmt/statix sweep. This
+exists because the whole-tree sweep is what was timing out CI; **CI always
+runs the plain, no-flag form and never passes `--full-check`.**
+
+The default mode's diff is against the working tree (uncommitted and staged
+edits included, not just committed ones), so it's already the right tool for
+an interactive session too: after editing one or two hosts/modules, plain
+`bash scripts/codex-maintenance.sh` naturally scopes to just what you
+touched. Reserve `--full-check` for changes that plausibly affect every host
+(`modules/common/*`, `flake.nix`, `variables.nix` — though the default mode
+already falls back to evaluating everything for those paths, `--full-check`
+additionally re-checks fmt/statix over the whole tree) or as a final check
+before committing.

 ## Scripts

-Beyond `codex-setup.sh`/`codex-maintenance.sh` above, `scripts/` also has:
+Beyond `codex-setup.sh`/`codex-maintenance.sh` above, `scripts/` is
+organized by purpose: `scripts/secrets/` (sops/age + SSH host-key
+management), `scripts/proxmox/` (Proxmox deployment), `scripts/lib/`
+(shared helpers, sourced by the scripts below — not run directly), and a
+handful of repo-wide scripts left at the top level (`env.sh`,
+`bump-nixpkgs-release.sh`, plus `codex-setup.sh`/`codex-maintenance.sh`
+above). When adding a new script, put it in the matching subfolder rather
+than the top level, and if it duplicates logic another script already has,
+lift the shared part into `scripts/lib/` instead of copying it.

- `scripts/sync-host-keys.sh` — generates/registers SSH host keys and their
-  `.sops.yaml`/`secrets/*.yaml` recipients for flake targets, idempotently
-  (`--all`, `<target>`, `--remove`, `--regenerate-all-keys`, all with
-  `--dry-run`). The primary tool for provisioning a new host's secrets
-  access — see "Creating a new machine" in `docs/auto-installer.md`.
- `scripts/prepare-host-key.sh` — narrower predecessor: generates a key by
-  an arbitrary name without touching `.sops.yaml`. Still useful to
+### `scripts/secrets/`
+
+- `scripts/secrets/sync-host-keys.sh` — generates/registers SSH host keys
+  and their `.sops.yaml`/`secrets/*.yaml` recipients for flake targets,
+  idempotently (`--all`, `<target>`, `--remove`, `--regenerate-all-keys`,
+  all with `--dry-run`). The primary tool for provisioning a new host's
+  secrets access — see "Creating a new machine" in `docs/auto-installer.md`.
+- `scripts/secrets/prepare-host-key.sh` — narrower predecessor: generates a
+  key by an arbitrary name without touching `.sops.yaml`. Still useful to
  pre-generate a key before its flake target exists yet, since
  `sync-host-keys.sh` can only act on targets `nixosConfigurations` already
  has.
- `scripts/create-proxmox-resource.sh` — builds a `lxc-*`/`proxmox-*`
-  target's tarball/disk image and creates it on a real Proxmox node
-  (`pct create` against the tarball as a CT template / `qm create`+
+- `scripts/secrets/rotate-admin-key.sh <backup-admin-key> [--new-key-file
+  <path>] [--dry-run]` — rotates `.sops.yaml`'s `&admin` age key: decrypts
+  with a backed-up copy of the key currently trusted as `&admin` (verified
+  by deriving its public key and comparing, not taken on faith), replaces
+  the `&admin` line with a new key already present in the environment
+  (defaults to wherever sops/age itself would look), and runs
+  `sops updatekeys` on every `secrets/*.yaml`. One-way: the old key can no
+  longer decrypt anything re-encrypted this way. This is the automation
+  for the manual steps `sync-host-keys.sh`/`create-proxmox-resource.sh`
+  print when they bootstrap a brand-new, not-yet-trusted key on a machine
+  with no prior admin access.
+- `scripts/secrets/backup-admin-key.sh <dest-path> [--key-file <path>]
+  [--force] [--dry-run]` — copies the local sops age key (source
+  resolution matches sops/age itself: `$SOPS_AGE_KEY` inline, then
+  `--key-file`, then `$SOPS_AGE_KEY_FILE`, then the XDG default) to an
+  arbitrary destination path with `0600` permissions, validating it's a
+  real age identity and round-tripping the public key before and after the
+  write. Refuses to overwrite an existing `<dest-path>` without `--force`.
+  Purely a local filesystem copy — never touches `.sops.yaml`/
+  `secrets/*.yaml` or the repo at all. The resulting file is exactly what
+  `rotate-admin-key.sh` expects as its backup-key argument.
+- `scripts/secrets/sync-nix-cache-host-key.sh [--check] [--dry-run]
+  [--host <name>]` — detects drift between the ed25519 SSH host key
+  nix-cache is actually serving right now (via `ssh-keyscan`) and
+  `vars.nixCacheHostKey` (`variables.nix`), the value
+  `modules/nix-cache/remote-builder-client.nix` bakes into every real
+  client's declarative `programs.ssh.knownHosts` and
+  `configure-nix-cache-client.sh` hardcodes as its own default for
+  non-NixOS clients. That value has no automatic source of truth — it's
+  set once from whatever nix-cache's host key happened to be at the time,
+  and silently goes stale if the host is ever rebuilt/recreated with a new
+  key, breaking every client's distributed-build SSH trust with no error
+  that points back here. `--check` (used by `codex-maintenance.sh`, which
+  treats an unreachable nix-cache — e.g. from a non-LAN CI runner — as a
+  silent skip rather than a failure) only reports drift; the no-flags form
+  updates both files in place. Declarative clients still need a rebuild to
+  pick up the fix.
+
+### `scripts/proxmox/`
+
+- `scripts/proxmox/create-proxmox-resource.sh` — builds a `lxc-*`/
+  `proxmox-*` target's tarball/disk image and creates it on a real Proxmox
+  node (`pct create` against the tarball as a CT template / `qm create`+
  `importdisk`), or reconfigures an existing resource's cores/memory/disk
  size (`--modify`, always requires typing the VMID back to confirm).
  Checks for an already-uploaded image on the node before building
@@ -99,15 +170,12 @@ Beyond `codex-setup.sh`/`codex-maintenance.sh` above, `scripts/` also has:
  unless `--allow-duplicate-host` is passed. `--dry-run` throughout both
  modes. The first time it has to bootstrap build tooling on a node (i.e.
  `nix` wasn't already on its `PATH`), it also runs
-  `scripts/configure-nix-cache-client.sh` there (non-fatally — a failure
-  just falls back to building from source / `cache.nixos.org`) so the
-  node substitutes from and can offload builds to nix-cache on every
+  `scripts/proxmox/configure-nix-cache-client.sh` there (non-fatally — a
+  failure just falls back to building from source / `cache.nixos.org`) so
+  the node substitutes from and can offload builds to nix-cache on every
  subsequent run, not just this one.
- `scripts/env.sh` — shared config (`PROXMOX_HOST`, storage pool, bridge,
-  default cores/memory) sourced by `create-proxmox-resource.sh`. Add new
-  cross-script config here instead of duplicating it per-script.
- `scripts/configure-nix-cache-client.sh [--dry-run] [--no-remote-builder]
-  [--no-restart]` — the non-NixOS equivalent of
+- `scripts/proxmox/configure-nix-cache-client.sh [--dry-run]
+  [--no-remote-builder] [--no-restart]` — the non-NixOS equivalent of
  `modules/nix-cache/client.nix`/`remote-builder-client.nix`, for a plain
  Debian machine with the Nix package manager (not NixOS) already
  installed: run as root *on that machine* to add nix-cache as a
@@ -120,32 +188,39 @@ Beyond `codex-setup.sh`/`codex-maintenance.sh` above, `scripts/` also has:
  `/etc/ssh/ssh_known_hosts`. Idempotent (re-running replaces its own
  marked block rather than duplicating it); restarts `nix-daemon` by
  default so the change takes effect immediately.
+
+### `scripts/lib/`
+
+Sourced by the scripts above, never run directly:
+
+- `nix-bootstrap.sh` — `NIX_CONFIG`/`ensure_nix_profile`, shared by
+  `codex-setup.sh`/`codex-maintenance.sh` and the remote build commands
+  `create-proxmox-resource.sh` runs over SSH.
+- `nix-eval.sh` — `NIX_EVAL_FLAGS` plus `list_flake_targets`/
+  `flake_target_hostname` flake-introspection helpers.
+- `ssh-host-keys.sh` — `generate_host_ed25519_key`/`ssh_pubkey_to_age`,
+  shared by `sync-host-keys.sh` and `prepare-host-key.sh`.
+- `sops-age.sh` — `age_pubkey_from_identity_file`/`sops_yaml_admin_pubkey`/
+  `sops_updatekeys` plus the shared sops/age default key-file resolution,
+  shared by `backup-admin-key.sh`, `rotate-admin-key.sh`, and
+  `sync-host-keys.sh`.
+- `confirm.sh` — `confirm_typed`, the "type X back to confirm" destructive-
+  action prompt shared by `create-proxmox-resource.sh` and
+  `sync-host-keys.sh`.
+- `sync-host-keys-edit-sops.py` — the `.sops.yaml` anchor/key_groups editor
+  `sync-host-keys.sh` shells out to (see that script for why: precise,
+  idempotent YAML edits are impractical in bash).
+
+### Top level
+
+- `scripts/env.sh` — shared config (`PROXMOX_HOST`, storage pool, bridge,
+  default cores/memory) sourced by `create-proxmox-resource.sh`. Add new
+  cross-script config here instead of duplicating it per-script.
 - `scripts/bump-nixpkgs-release.sh` — bumps `flake.nix`'s `nixpkgs.url`/
  `home-manager.url` in place. Exists because flake input URLs can't
  reference `variables.nix` (confirmed empirically — `nix flake metadata`
  errors on it), so this is the closest equivalent to a single source of
  truth for the tracked release.
- `scripts/rotate-admin-key.sh <backup-admin-key> [--new-key-file <path>]
-  [--dry-run]` — rotates `.sops.yaml`'s `&admin` age key: decrypts with a
-  backed-up copy of the key currently trusted as `&admin` (verified by
-  deriving its public key and comparing, not taken on faith), replaces the
-  `&admin` line with a new key already present in the environment
-  (defaults to wherever sops/age itself would look), and runs
-  `sops updatekeys` on every `secrets/*.yaml`. One-way: the old key can no
-  longer decrypt anything re-encrypted this way. This is the automation
-  for the manual steps `sync-host-keys.sh`/`create-proxmox-resource.sh`
-  print when they bootstrap a brand-new, not-yet-trusted key on a machine
-  with no prior admin access.
- `scripts/backup-admin-key.sh <dest-path> [--key-file <path>] [--force]
-  [--dry-run]` — copies the local sops age key (source resolution matches
-  sops/age itself: `$SOPS_AGE_KEY` inline, then `--key-file`, then
-  `$SOPS_AGE_KEY_FILE`, then the XDG default) to an arbitrary destination
-  path with `0600` permissions, validating it's a real age identity and
-  round-tripping the public key before and after the write. Refuses to
-  overwrite an existing `<dest-path>` without `--force`. Purely a local
-  filesystem copy — never touches `.sops.yaml`/`secrets/*.yaml` or the
-  repo at all. The resulting file is exactly what `rotate-admin-key.sh`
-  expects as its backup-key argument.

 `sync-host-keys.sh`, `create-proxmox-resource.sh`, and
 `rotate-admin-key.sh` genuinely mutate real state when run for real (not
@@ -34,7 +34,7 @@ anywhere in this repo — that's live infrastructure state, not something a
 committed file can keep accurate, and it changes independently of the code.
 Check the Proxmox node itself, or `/etc/flake-target` on a running host (see
 below), if you need to know what's really out there right now.
-`scripts/create-proxmox-resource.sh`'s duplicate-host guard works the same
+`scripts/proxmox/create-proxmox-resource.sh`'s duplicate-host guard works the same
 way: it checks the Proxmox node directly rather than any file here.

 Each buildtype's `hosts/<name>/host.nix` carries the per-machine identity
@@ -74,10 +74,19 @@ Safe validation commands for Codex and local review:

 ```bash
 bash scripts/codex-setup.sh
-bash scripts/codex-maintenance.sh dry-run
 bash scripts/codex-maintenance.sh
 ```

+`codex-maintenance.sh` with no flags (what CI runs on every push/PR) scopes
+fmt-check/statix/eval to files changed against a base ref — fast, but only
+as thorough as the diff. For the full sweep (every host, every package,
+fmt-check and statix over the whole tree — slow, CI never runs this):
+
+```bash
+bash scripts/codex-maintenance.sh --full-check
+bash scripts/codex-maintenance.sh --full-check --dry-run
+```
+
 For individual host evaluation:

 ```bash
@@ -114,7 +123,7 @@ Three different paths depending on target, none of them involving a manual
  disk image and attached to a new VM with no install step — see
  `docs/proxmox-images.md`.

-`scripts/create-proxmox-resource.sh --type lxc|vm --host <name>` automates
+`scripts/proxmox/create-proxmox-resource.sh --type lxc|vm --host <name>` automates
 either of the last two end to end (host-key registration, building the
 image directly on the Proxmox node itself, `pct create`/`qm create`), with
 `--dry-run` and a guard against duplicating an already-deployed host's
@@ -73,7 +73,7 @@ booting one:

 First boot runs `boot.postBootCommands` (registers the Nix store DB and
 system profile) — there's no separate activation step to run yourself.
-`scripts/create-proxmox-resource.sh --type lxc --host <name>` automates all
+`scripts/proxmox/create-proxmox-resource.sh --type lxc --host <name>` automates all
 of this (host-key handling, building the tarball directly on the Proxmox
 node itself, `pct create` with the flags above) — see its `--help`.

@@ -102,7 +102,7 @@ groups required, got 0`, and *every* secret (including this host's own
 login) permanently fails to decrypt, silently — no error in the boot log
 at all, since the activation step that would install secrets only runs on
 a from-scratch first activation and skips silently once `/run/current-system`
-already exists. `scripts/create-proxmox-resource.sh` always builds with
+already exists. `scripts/proxmox/create-proxmox-resource.sh` always builds with
 `NIXOS_HOST_KEYS_DIR` set for this reason.

 ## Layout
@@ -116,10 +116,10 @@ already exists. `scripts/create-proxmox-resource.sh` always builds with
  `docs/pxe-boot.md`).
 - `modules/installer/host-keys.nix` — optionally bakes pre-generated SSH
  host keys into the image; see "Host keys" below.
- `scripts/sync-host-keys.sh` — admin-workstation tool that generates,
+- `scripts/secrets/sync-host-keys.sh` — admin-workstation tool that generates,
  registers, and (via `--remove`/`--regenerate-all-keys`) retires host
  keys; see "Creating a New Machine" below.
- `scripts/prepare-host-key.sh` — narrower predecessor: generates a single
+- `scripts/secrets/prepare-host-key.sh` — narrower predecessor: generates a single
  key by an arbitrary name without touching `.sops.yaml`. Still useful for
  pre-generating a key *before* its flake target exists (`sync-host-keys.sh`
  can only act on targets `nixosConfigurations` already has); otherwise
@@ -238,7 +238,7 @@ GitHub token behind sops-nix for all of them).
 2. **On your admin workstation, generate and register its host key:**

   ```sh
-   ./scripts/sync-host-keys.sh <flake-target>
+   ./scripts/secrets/sync-host-keys.sh <flake-target>
   ```

   This generates `host-keys/<flake-target>_ssh_host_ed25519_key(.pub)`,
@@ -250,7 +250,7 @@ GitHub token behind sops-nix for all of them).

   Doing this for every host that needs one at once — after adding several
   new targets, or just to catch up any that were missed — is
-   `./scripts/sync-host-keys.sh --all`. See `scripts/sync-host-keys.sh --help`
+   `./scripts/secrets/sync-host-keys.sh --all`. See `scripts/secrets/sync-host-keys.sh --help`
   for its other modes (`--remove`, `--regenerate-all-keys`).

 3. **Commit and push.** The flake build the installer uses has to see the
@@ -8,9 +8,14 @@ and to verify that declared NixOS hosts still evaluate after dependency updates.
 - A scheduled workflow runs `nix flake update` once per week.
 - On GitHub, any resulting `flake.lock` change is proposed through a pull request.
 - On Gitea, the workflow can commit and push `flake.lock` directly when PR automation is not configured.
- A separate CI workflow evaluates every configured host before merge, listed
-  dynamically via `nix eval --json .#nixosConfigurations --apply builtins.attrNames`
-  rather than hand-enumerated, so it can't drift as `<platform>-<buildtype>`
+- A separate CI workflow runs `scripts/codex-maintenance.sh` before merge.
+  Its default mode scopes eval to the hosts/packages a change can affect,
+  determined from a git diff against the PR base — but a `flake.lock` change
+  is treated as repo-wide and always falls back to evaluating every host, so
+  a lock-file update PR still gets full coverage. Hosts are still listed
+  dynamically via
+  `nix eval --json .#nixosConfigurations --apply builtins.attrNames` rather
+  than hand-enumerated, so that fallback can't drift as `<platform>-<buildtype>`
  targets are added or removed. See `README.md` for the current target list.

 ## Why hosts should stop using `--upgrade-all`
@@ -46,18 +46,26 @@ the new key up automatically on next activation — no more manual

 ## Remote builder SSH keys

-On each client, install the private key used to authenticate as `nixremote`:
+Each client authenticates as `nixremote` using its **own default root SSH
+identity** (`/root/.ssh/id_ed25519`) — not a separately-named or shared
+keypair. If a client doesn't have one yet:

 ```bash
-sudo install -d -m 0700 /root/.ssh
-sudo install -m 0600 ./nixremote /root/.ssh/nixremote
-sudo ssh -i /root/.ssh/nixremote nixremote@nix-cache nix-store --version
+sudo ssh-keygen -t ed25519 -N '' -f /root/.ssh/id_ed25519
 ```

-On `nix-cache`, install the matching public key used by `nixremote` authorized keys.
+Then add its `.pub` contents as a new entry in `vars.remoteBuilderAuthorizedKeys`
+(`variables.nix`) and rebuild `nix-cache` to pick it up (that list is
+declarative — an imperative `ssh-copy-id nixremote@nix-cache` won't stick;
+it gets overwritten on every rebuild). Verify with:

-The committed `nixremote` authorized keys are public SSH keys only. Keep the
-matching private keys on client hosts and out of the repository.
+```bash
+sudo ssh -i /root/.ssh/id_ed25519 nixremote@nix-cache nix-store --version
+```
+
+The committed `remoteBuilderAuthorizedKeys` entries are public SSH keys
+only. Keep the matching private keys on client hosts and out of the
+repository.

 nix-cache's own SSH *host* key is trusted declaratively via
 `programs.ssh.knownHosts` in `modules/nix-cache/remote-builder-client.nix`,
@@ -76,8 +84,8 @@ After deployment:
 curl http://nix-cache/nix-cache-info
 nix store ping --store http://nix-cache
 nix show-config | grep -E 'substituters|trusted-public-keys|builders-use-substitutes'
-sudo ssh -i /root/.ssh/nixremote nixremote@nix-cache nix-store --version
-nix build nixpkgs#hello --builders 'ssh://nixremote@nix-cache x86_64-linux /root/.ssh/nixremote 4 2 big-parallel,kvm,nixos-test,benchmark' -L
+sudo ssh -i /root/.ssh/id_ed25519 nixremote@nix-cache nix-store --version
+nix build nixpkgs#hello --builders 'ssh://nixremote@nix-cache x86_64-linux /root/.ssh/id_ed25519 4 2 big-parallel,kvm,nixos-test,benchmark' -L
 nix path-info -r nixpkgs#hello
 curl -I "http://nix-cache/$(basename "$(nix path-info nixpkgs#hello)").narinfo"
 ```
@@ -7,7 +7,7 @@ config (`modules/disko/proxmox.nix`) already used to format a real disk on
 install, so there's nothing host-specific to write; it's available for every
 `proxmox-*` target automatically.

-`scripts/create-proxmox-resource.sh --type vm --host <name>` automates the
+`scripts/proxmox/create-proxmox-resource.sh --type vm --host <name>` automates the
 whole walkthrough below (and the equivalent LXC one) end to end, including
 host-key handling and building the image directly on the Proxmox node
 itself (no local build, no image transfer) — see its `--help`. The steps
@@ -51,7 +51,7 @@ sudo ./result \
  --build-memory 2048
 ```

-Generate the key first with `scripts/sync-host-keys.sh <hostname>`, same
+Generate the key first with `scripts/secrets/sync-host-keys.sh <hostname>`, same
 as any other host — see `docs/auto-installer.md` for the full walkthrough
 (it registers the new key in `.sops.yaml` and re-encrypts the affected
 `secrets/*.yaml` files too, no manual editing needed).
@@ -9,11 +9,25 @@ let
  # (the VM platforms rely on automount itself to get that same
  # non-blocking behavior, so they don't need `nofail` too).
  automountOpts = if config.boot.isContainer then [ "nofail" ] else [ "x-systemd.automount" ];
+
+  # A bare hostname here never resolves reliably: systemd-resolved only
+  # ever tries LLMNR for single-label names (never DNS, regardless of any
+  # configured search domain), and a *global* search domain (the first fix
+  # attempted here) backfires worse -- confirmed live on lxc-docker, adding
+  # `networking.search` made systemd-resolved prioritize its domain-matched
+  # but server-less global scope over eth0's correctly-configured one for
+  # every "*.sweet.home" query, silently sending them to public fallback
+  # DNS instead. `resolvectl query --interface=eth0 server.sweet.home`
+  # resolved fine throughout, proving the LAN DNS server was never the
+  # problem -- only the ambient, unqualified device string was. Using the
+  # FQDN directly sidesteps all of that, matching the pattern
+  # ../raspi/mount-data.nix already uses for the same reason.
+  nfsServer = "${vars.nfsServerHost}.${vars.homeDomain}";
 in
 {
  fileSystems = {
    ${vars.nfsShares.dockerConfig.mountpoint} = {
-      device = "${vars.nfsServerHost}:${vars.storageRoot}/${vars.nfsShares.dockerConfig.subpath}";
+      device = "${nfsServer}:${vars.storageRoot}/${vars.nfsShares.dockerConfig.subpath}";
      fsType = "nfs";

      options = [
@@ -24,7 +38,7 @@ in
    };

    ${vars.nfsShares.dockerDatabases.mountpoint} = {
-      device = "${vars.nfsServerHost}:${vars.storageRoot}/${vars.nfsShares.dockerDatabases.subpath}";
+      device = "${nfsServer}:${vars.storageRoot}/${vars.nfsShares.dockerDatabases.subpath}";
      fsType = "nfs";

      options = [
@@ -35,7 +49,7 @@ in
    };

    ${vars.nfsShares.dockerVolumes.mountpoint} = {
-      device = "${vars.nfsServerHost}:${vars.storageRoot}/${vars.nfsShares.dockerVolumes.subpath}";
+      device = "${nfsServer}:${vars.storageRoot}/${vars.nfsShares.dockerVolumes.subpath}";
      fsType = "nfs";

      options = [
@@ -46,7 +60,7 @@ in
    };

    ${vars.nfsShares.nextcloudData.mountpoint} = {
-      device = "${vars.nfsServerHost}:${vars.storageRoot}/${vars.nfsShares.nextcloudData.subpath}";
+      device = "${nfsServer}:${vars.storageRoot}/${vars.nfsShares.nextcloudData.subpath}";
      fsType = "nfs";

      options = [
@@ -57,7 +71,7 @@ in
    };

    ${vars.nfsShares.raspiVolumes.mountpoint} = {
-      device = "${vars.nfsServerHost}:${vars.storageRoot}/${vars.nfsShares.raspiVolumes.subpath}";
+      device = "${nfsServer}:${vars.storageRoot}/${vars.nfsShares.raspiVolumes.subpath}";
      fsType = "nfs";

      options = [
@@ -130,7 +130,7 @@
        # at *activation* time, which runs before systemd would otherwise
        # generate one on first boot. Without pre-seeding it here, secrets
        # (including the login password) fail to decrypt on first boot.
-        # Generate the key with scripts/prepare-host-key.sh first.
+        # Generate the key with scripts/secrets/prepare-host-key.sh first.
        #
        # Two places a key can come from, checked in order:
        #   /etc/host-keys  — baked into this image at build time (see
@@ -150,7 +150,7 @@
        else
            echo "WARNING: no SSH host key found for ''${choice} (checked /etc/host-keys and /root/host-keys)"
            echo "sops-nix secrets (including the login password) will NOT decrypt on first boot."
-            echo "Run scripts/prepare-host-key.sh for host ''${choice} on your admin workstation first,"
+            echo "Run scripts/secrets/prepare-host-key.sh for host ''${choice} on your admin workstation first,"
            echo "then either rebuild this image with NIXOS_HOST_KEYS_DIR set, or scp the result to"
            echo "/root/host-keys/ on this machine."
            read -rp "Continue without a pre-seeded key anyway? (y/N): " skip_key
@@ -1,10 +1,14 @@
 { pkgs, vars, ... }:

 {
-  # Install the remote builder key on each client host (do not commit private keys):
-  #   sudo install -d -m 0700 /root/.ssh
-  #   sudo install -m 0600 ./nixremote /root/.ssh/nixremote
-  #   sudo ssh -i /root/.ssh/nixremote nixremote@nix-cache nix-store --version
+  # Authenticate as nixremote using the client host's own default root SSH
+  # identity (/root/.ssh/id_ed25519) rather than a separately-named key --
+  # matches vars.remoteBuilderAuthorizedKeys, which already authorizes
+  # each host's own default key (one entry per host, not a shared
+  # dedicated keypair). If this host doesn't have one yet:
+  #   sudo -u root ssh-keygen -t ed25519 -N '' -f /root/.ssh/id_ed25519
+  #   # then add its .pub to vars.remoteBuilderAuthorizedKeys and rebuild nix-cache
+  #   sudo ssh -i /root/.ssh/id_ed25519 nixremote@nix-cache nix-store --version
  # Trust nix-cache's SSH host key declaratively so the nix-daemon (root)
  # can connect the first time without a manual ssh-keyscan/known_hosts
  # step on every new client.
@@ -20,7 +24,7 @@
      {
        hostName = vars.nixCacheHost;
        sshUser = vars.remoteBuilderUser;
-        sshKey = "/root/.ssh/${vars.remoteBuilderUser}";
+        sshKey = "/root/.ssh/id_ed25519";
        inherit (pkgs.stdenv.hostPlatform) system;
        maxJobs = 4;
        speedFactor = 2;
@@ -1,4 +1,4 @@
-{ lib, modulesPath, flakeTarget, ... }:
+{ config, lib, modulesPath, flakeTarget, ... }:

 let
  # Bakes this exact flake target's pre-generated SSH host key straight
@@ -14,7 +14,7 @@ let
  # Without this, config.system.build.tarball's built-in system just
  # generates a fresh host key at first boot like any other host would --
  # but sops-nix derives its decryption key from *this* file, and
-  # .sops.yaml only trusts whatever key scripts/sync-host-keys.sh already
+  # .sops.yaml only trusts whatever key scripts/secrets/sync-host-keys.sh already
  # registered for this exact target name. A freshly-generated key can
  # never match that, so every secret (including this host's own login)
  # permanently fails to decrypt. Confirmed live: sops-install-secrets
@@ -26,7 +26,7 @@ let
  hostKeysDir = /. + hostKeysDirStr;

  # flakeTarget ("${platform}-${buildType}") comes in via specialArgs from
-  # flake.nix's mkTarget -- exactly the name scripts/sync-host-keys.sh
+  # flake.nix's mkTarget -- exactly the name scripts/secrets/sync-host-keys.sh
  # registers keys under. Deliberately not read back from
  # config.environment.etc."flake-target" (which is set to the same value)
  # -- this module also *contributes* to environment.etc below, and a
@@ -58,8 +58,28 @@ in
    # host.nix declares each host's real hostname (networking.hostName);
    # keep that instead of letting Proxmox's ambient container config win.
    manageHostName = true;
-    # Unprivileged matches how these containers are actually created.
-    privileged = false;
+    # Unprivileged by default -- matches how these containers are actually
+    # created (scripts/proxmox/create-proxmox-resource.sh reads this value
+    # back to decide `pct create`'s --unprivileged flag, so the two stay
+    # in sync).
+    #
+    # lxc-docker is the one exception: the kernel's NFS client doesn't set
+    # FS_USERNS_MOUNT, so mounting NFS from inside *any* non-init user
+    # namespace -- which is exactly what an unprivileged container's
+    # UID-mapped root runs in -- is rejected at the VFS layer with EPERM,
+    # no matter what Proxmox's own `mount=nfs;nfs4` container feature
+    # allows at the AppArmor layer (confirmed live: TCP to the NFS server
+    # succeeds, the server's export table matches the container's IP, and
+    # `mount.nfs: Operation not permitted` still fires immediately with no
+    # corresponding denial anywhere in the server's logs -- a kernel-level
+    # rejection, not a network or export-permission one). Keying off
+    # hostName rather than something docker-build-type-specific because
+    # modules/build-types/docker.nix is also composed for linode-docker/
+    # proxmox-docker, which don't import proxmox-lxc.nix at all --setting
+    # this option there would break their eval with "option does not
+    # exist" regardless of any mkIf guard, since mkIf only makes a value
+    # conditional, not whether the option needs to exist somewhere.
+    privileged = config.networking.hostName == "docker";
  };

  boot.loader = {
@@ -129,5 +129,6 @@ echo "flake.lock still points at the old input revisions until refreshed. Either
 echo "  nix flake update nixpkgs home-manager   # just these two inputs"
 echo "  nix flake update                        # everything — see docs/flake-lock-automation.md"
 echo
-echo "Then run 'bash scripts/codex-maintenance.sh dry-run' before committing —"
-echo "a channel bump can shift option defaults across every host."
+echo "Then run 'bash scripts/codex-maintenance.sh --full-check --dry-run' before"
+echo "committing — a channel bump can shift option defaults across every host,"
+echo "and only --dry-run actually builds anything to catch that."
@@ -1,4 +1,21 @@
 #!/usr/bin/env bash
+# Validation entry point for CI and local/agent review.
+#
+# Default mode (what CI runs on every push/PR): fmt-check, statix, and eval
+# are scoped to files that actually changed against a base ref, plus
+# whichever hosts/packages those changes can affect. This exists because
+# the unscoped sweep below is slow enough to time out CI runners -- see
+# --full-check.
+#
+# --full-check: the historical full sweep (every host, every package,
+# fmt --check ./statix check . over the whole tree). Slow -- minutes, not
+# seconds. CI never passes this; run it locally before a release or after
+# touching modules/common/*, flake.nix, or variables.nix if you want extra
+# confidence beyond what the changed-files scope already covers for those
+# paths (see below).
+#
+# --dry-run: adds `nix build --dry-run --no-link` for whatever scope is
+# active (changed-files scope by default, full scope under --full-check).
 set -euo pipefail

 script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
@@ -7,7 +24,39 @@ source "${script_dir}/lib/nix-bootstrap.sh"
 # shellcheck source=lib/nix-eval.sh
 source "${script_dir}/lib/nix-eval.sh"

-MODE="${1:-validate}"
+repo_root="$(cd "${script_dir}/.." && pwd)"
+cd "$repo_root"
+
+full_check=false
+dry_run=false
+
+usage() {
+  cat <<'EOF'
+Usage: scripts/codex-maintenance.sh [--full-check] [--dry-run]
+
+  --full-check  Run the full sweep: fmt-check and statix over the whole
+                repo, eval every host and package. Slow. Never run by CI.
+  --dry-run     Additionally run `nix build --dry-run --no-link` for
+                whatever scope is active.
+
+With neither flag (the CI default), fmt-check/statix/eval are scoped to
+files changed against a base ref (env MAINT_BASE_SHA, else the PR base,
+else HEAD^), plus the hosts/packages those changes can affect.
+EOF
+}
+
+for arg in "$@"; do
+  case "$arg" in
+    --full-check) full_check=true ;;
+    --dry-run) dry_run=true ;;
+    -h|--help) usage; exit 0 ;;
+    *)
+      echo "Unknown argument: $arg" >&2
+      usage >&2
+      exit 1
+      ;;
+  esac
+done

 ensure_nix_profile

@@ -16,12 +65,6 @@ if ! command -v nix >/dev/null 2>&1; then
  exit 127
 fi

-hosts="$(list_flake_targets .)"
-
-echo "Hosts:"
-echo "$hosts"
-
-echo
 echo "Checking for obvious committed secrets..."
 if grep -RInE 'github_pat_|ghp_|access-tokens|hashedPassword[[:space:]]*=' \
  --exclude-dir=.git \
@@ -33,17 +76,178 @@ else
  echo "No obvious token patterns found."
 fi

+mapfile -t all_hosts < <(list_flake_targets .)
+mapfile -t all_packages < <(nix eval --json "${NIX_EVAL_FLAGS[@]}" .#packages.x86_64-linux --apply builtins.attrNames | jq -r '.[]')
+
+# host_targets_for_dir <hosts-subdir-name>
+# Prints the nixosConfigurations target names whose hostPath is
+# ./hosts/<dir>/host.nix, derived straight from flake.nix's generatedTargets
+# (one mkTarget { ... } call per line) rather than a hand-maintained table,
+# so it can't drift the way a copied mapping would.
+host_targets_for_dir() {
+  local dir="$1"
+  grep -oE '^[[:space:]]*[A-Za-z0-9_-]+ = mkTarget \{[^}]*hostPath = \./hosts/'"${dir}"'/host\.nix;[^}]*\};' flake.nix \
+    | sed -E 's/^[[:space:]]*([A-Za-z0-9_-]+) = mkTarget.*/\1/' \
+    || true
+}
+
+declare -a changed_files=()
+scope_desc="full repo"
+
+if ! $full_check; then
+  resolve_base_ref() {
+    if [[ -n "${MAINT_BASE_SHA:-}" ]] && git cat-file -e "${MAINT_BASE_SHA}^{commit}" 2>/dev/null; then
+      echo "$MAINT_BASE_SHA"
+      return
+    fi
+    if git rev-parse --verify -q HEAD^ >/dev/null 2>&1; then
+      echo "HEAD^"
+      return
+    fi
+    git hash-object -t tree /dev/null
+  }
+
+  base_ref="$(resolve_base_ref)"
+  echo
+  echo "Changed-files scope: diffing against ${base_ref}"
+  mapfile -t changed_files < <(git diff --name-only --diff-filter=ACMR "$base_ref" -- . | sort -u)
+
+  if [[ ${#changed_files[@]} -eq 0 ]]; then
+    echo "No changed files detected."
+  else
+    printf '  %s\n' "${changed_files[@]}"
+  fi
+  scope_desc="changed files only (base: ${base_ref})"
+fi
+
+# Whole-tree fmt/lint always run under --full-check; otherwise scoped below.
+declare -a changed_nix_files=()
+for f in "${changed_files[@]:-}"; do
+  [[ "$f" == *.nix && -f "$f" ]] && changed_nix_files+=("$f")
+done
+
 echo
 echo "Checking Nix formatting with nixpkgs-fmt..."
+if $full_check; then
  nix run "${NIX_EVAL_FLAGS[@]}" github:NixOS/nixpkgs/nixos-25.11#nixpkgs-fmt -- --check .
+elif [[ ${#changed_nix_files[@]} -gt 0 ]]; then
+  nix run "${NIX_EVAL_FLAGS[@]}" github:NixOS/nixpkgs/nixos-25.11#nixpkgs-fmt -- --check "${changed_nix_files[@]}"
+else
+  echo "No changed .nix files; skipping."
+fi

 echo
 echo "Running statix lint..."
+if $full_check; then
  nix run "${NIX_EVAL_FLAGS[@]}" github:NixOS/nixpkgs/nixos-25.11#statix -- check .
+elif [[ ${#changed_nix_files[@]} -gt 0 ]]; then
+  for f in "${changed_nix_files[@]}"; do
+    nix run "${NIX_EVAL_FLAGS[@]}" github:NixOS/nixpkgs/nixos-25.11#statix -- check "$f"
+  done
+else
+  echo "No changed .nix files; skipping."
+fi
+
+# Figure out which hosts/packages this run needs to eval (and, under
+# --dry-run, build). full_check always means "everything"; otherwise a
+# change to flake.nix/flake.lock/variables.nix/modules/common/* (repo-wide
+# inputs) or to any other modules/*.nix outside platforms//build-types
+# (whose blast radius isn't safely inferable from the path alone -- see
+# CLAUDE.md's "Grep modules/build-types/*.nix for each build type's imports
+# list") also falls back to everything, on the same reasoning CLAUDE.md
+# already gives interactive sessions for when to run the full sweep.
+# Anything more targeted -- a host.nix, a platform module, a build-type
+# module -- narrows to just the hosts it can affect.
+declare -A affected_hosts=()
+eval_packages=false
+
+if $full_check; then
+  for h in "${all_hosts[@]}"; do affected_hosts[$h]=1; done
+  eval_packages=true
+else
+  full_fallback=false
+  for f in "${changed_files[@]:-}"; do
+    case "$f" in
+      flake.nix|flake.lock|variables.nix|modules/common/*)
+        full_fallback=true
+        ;;
+    esac
+  done
+
+  if ! $full_fallback; then
+    for f in "${changed_files[@]:-}"; do
+      case "$f" in
+        hosts/*/*)
+          hostdir="${f#hosts/}"
+          hostdir="${hostdir%%/*}"
+          while IFS= read -r t; do
+            [[ -n "$t" ]] && affected_hosts[$t]=1
+          done < <(host_targets_for_dir "$hostdir")
+          ;;
+        modules/platforms/*.nix)
+          platform="$(basename "$f" .nix)"
+          for h in "${all_hosts[@]}"; do
+            [[ "$h" == "${platform}-"* ]] && affected_hosts[$h]=1
+          done
+          ;;
+        modules/build-types/*.nix)
+          buildtype="$(basename "$f" .nix)"
+          for h in "${all_hosts[@]}"; do
+            [[ "$h" == *"-${buildtype}" ]] && affected_hosts[$h]=1
+          done
+          ;;
+        modules/installer/*)
+          # iso.nix (imported by both the "installer" nixosConfigurations
+          # target and netbootSystem, which backs packages.pxe) pulls in
+          # common.nix, so a common.nix change reaches all three.
+          affected_hosts[installer]=1
+          eval_packages=true
+          ;;
+        modules/pxe-boot/*)
+          # stage-installer-artifacts.nix is imported by
+          # modules/build-types/pxe-boot.nix only -- same blast radius as a
+          # build-types/*.nix change, not a packages one.
+          for h in "${all_hosts[@]}"; do
+            [[ "$h" == *"-pxe-boot" ]] && affected_hosts[$h]=1
+          done
+          ;;
+        modules/*)
+          full_fallback=true
+          ;;
+      esac
+    done
+  fi
+
+  if $full_fallback; then
+    echo
+    echo "Changed files affect shared config; falling back to evaluating every host/package."
+    for h in "${all_hosts[@]}"; do affected_hosts[$h]=1; done
+    eval_packages=true
+  fi
+fi
+
+mapfile -t hosts < <(for h in "${!affected_hosts[@]}"; do echo "$h"; done | sort)

 echo
-echo "Evaluating host toplevel derivations..."
-for host in $hosts; do
+echo "Checking nix-cache host key for drift..."
+if bash "${script_dir}/secrets/sync-nix-cache-host-key.sh" --check; then
+  :
+else
+  drift_status=$?
+  if [[ "$drift_status" -eq 2 ]]; then
+    echo "nix-cache unreachable from here -- skipping host-key drift check."
+  else
+    echo "WARNING: nix-cache's host key has drifted from variables.nix (see above)." >&2
+    echo "  Run 'bash scripts/secrets/sync-nix-cache-host-key.sh' to fix." >&2
+  fi
+fi
+
+echo
+if [[ ${#hosts[@]} -eq 0 ]]; then
+  echo "No hosts affected by changed files; skipping host eval."
+else
+  echo "Evaluating host toplevel derivations (${scope_desc})..."
+  for host in "${hosts[@]}"; do
    echo "==> $host"
    nix eval --raw "${NIX_EVAL_FLAGS[@]}" ".#nixosConfigurations.${host}.config.system.build.toplevel.drvPath"

@@ -63,19 +267,23 @@ for host in $hosts; do
        ;;
    esac
  done
+fi

 echo
+if ! $eval_packages; then
+  echo "No packages affected by changed files; skipping package eval."
+else
  echo "Evaluating buildable packages..."
-packages="$(nix eval --json "${NIX_EVAL_FLAGS[@]}" .#packages.x86_64-linux --apply builtins.attrNames | jq -r '.[]')"
-for pkg in $packages; do
+  for pkg in "${all_packages[@]}"; do
    echo "==> packages.x86_64-linux.${pkg}"
    nix eval --raw "${NIX_EVAL_FLAGS[@]}" ".#packages.x86_64-linux.${pkg}"
  done
+fi

-if [[ "$MODE" == "dry-run" ]]; then
+if $dry_run; then
  echo
-  echo "Running dry-run builds for all hosts. This will not create result symlinks."
-  for host in $hosts; do
+  echo "Running dry-run builds for the active scope. This will not create result symlinks."
+  for host in "${hosts[@]:-}"; do
    echo "==> Dry-run build: $host"
    nix build --dry-run --no-link "${NIX_EVAL_FLAGS[@]}" ".#nixosConfigurations.${host}.config.system.build.toplevel"

@@ -91,13 +299,15 @@ if [[ "$MODE" == "dry-run" ]]; then
    esac
  done

+  if $eval_packages; then
    echo
-  echo "Running dry-run builds for all packages."
-  for pkg in $packages; do
+    echo "Running dry-run builds for packages."
+    for pkg in "${all_packages[@]}"; do
      echo "==> Dry-run build: packages.x86_64-linux.${pkg}"
      nix build --dry-run --no-link "${NIX_EVAL_FLAGS[@]}" ".#packages.x86_64-linux.${pkg}"
    done
  fi
+fi

 echo
 echo "Maintenance checks complete."
@@ -82,13 +82,6 @@ if ! command -v jq >/dev/null 2>&1; then
 fi

 echo "Available NixOS hosts:"
-hosts="$(list_flake_targets .)"
-echo "$hosts"
+list_flake_targets .

-echo "Evaluating all host toplevel derivations..."
-for host in $hosts; do
-  echo "==> Evaluating $host"
-  nix eval --raw "${NIX_EVAL_FLAGS[@]}" ".#nixosConfigurations.${host}.config.system.build.toplevel.drvPath"
-done
-
-echo "Codex setup complete."
+echo "Codex setup complete. Run bash scripts/codex-maintenance.sh to validate changes."
@@ -3,10 +3,10 @@
 # second copy of these values in every script:
 #   source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/env.sh"
 # Every variable can still be overridden per-invocation via the
-# environment (e.g. PROXMOX_STORAGE=tank-nvme ./scripts/create-proxmox-resource.sh ...)
+# environment (e.g. PROXMOX_STORAGE=tank-nvme ./scripts/proxmox/create-proxmox-resource.sh ...)
 # since each one only sets a default if unset.

-# SSH-reachable Proxmox node that scripts/create-proxmox-resource.sh runs
+# SSH-reachable Proxmox node that scripts/proxmox/create-proxmox-resource.sh runs
 # pct/qm on. Matches the Proxmox web UI hostname already used in
 # hosts/nixos/home.nix's desktop shortcuts (pve.<homeDomain> from
 # variables.nix) -- change this if that's not actually reachable over SSH,
@@ -15,7 +15,7 @@
 : "${PROXMOX_SSH_USER:=root}"

 # Where this flake repo lives on the Proxmox node itself.
-# scripts/create-proxmox-resource.sh builds images directly on the node
+# scripts/proxmox/create-proxmox-resource.sh builds images directly on the node
 # instead of transferring them over the network -- it clones the repo here
 # (from this checkout's own `origin` remote) the first time it doesn't
 # find it, installing build tooling via scripts/codex-setup.sh, then
@@ -0,0 +1,21 @@
+#!/usr/bin/env bash
+# Shared "type X to confirm" prompt for scripts/proxmox/create-proxmox-resource.sh
+# (--modify, and replacing an existing --allow-duplicate-host resource) and
+# scripts/secrets/sync-host-keys.sh (--regenerate-all-keys) -- three destructive
+# confirmations that all work the same way (echo the expected value back
+# exactly), kept in one place so the prompt/comparison logic can't drift.
+# Source alongside env.sh:
+#   source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/lib/confirm.sh"
+#
+# Deliberately does NOT print anything on mismatch or decide exit-vs-return
+# -- callers vary on both (a top-level script exits, a subcommand function
+# returns; wording differs too), so that stays at the call site.
+
+# confirm_typed <expected> <prompt>
+# Prints <prompt> via `read -rp`, then reports (via exit status) whether the
+# typed input matched <expected> exactly.
+confirm_typed() {
+  local expected="$1" prompt="$2" input
+  read -rp "$prompt" input
+  [[ "$input" == "$expected" ]]
+}
@@ -34,3 +34,21 @@ flake_target_hostname() {
  nix eval --raw "${NIX_EVAL_FLAGS[@]}" \
    "${flake_ref}#nixosConfigurations.${target}.config.networking.hostName" 2>/dev/null
 }
+
+# flake_target_lxc_privileged <flake_ref> <target>
+# Prints "true" or "false" for one lxc-* target's config.proxmoxLXC.privileged
+# (modules/platforms/lxc.nix is the single source of truth -- e.g.
+# lxc-docker sets this true so it can NFS-mount; every other lxc-* host
+# stays unprivileged). Only meaningful for lxc-* targets -- the option
+# doesn't exist for linode-*/proxmox-* (nixpkgs' proxmox-lxc.nix, which
+# declares it, is only ever imported by modules/platforms/lxc.nix). Empty
+# (not an error under set -e) if the eval fails.
+flake_target_lxc_privileged() {
+  local flake_ref="$1" target="$2"
+  # Not --raw: the option is a Nix boolean, and --raw can only coerce
+  # strings ("cannot coerce a Boolean to a string"). Plain `nix eval`
+  # prints a bare `true`/`false` for a boolean, which is exactly the
+  # string this needs.
+  nix eval "${NIX_EVAL_FLAGS[@]}" \
+    "${flake_ref}#nixosConfigurations.${target}.config.proxmoxLXC.privileged" 2>/dev/null
+}
@@ -0,0 +1,52 @@
+#!/usr/bin/env bash
+# Shared sops/age helpers for scripts/secrets/backup-admin-key.sh,
+# scripts/secrets/rotate-admin-key.sh, and scripts/secrets/sync-host-keys.sh -- all three
+# derive an age public key from a private identity file the same way, two
+# of them resolve the same sops/age default key-file path, and two of them
+# run `sops updatekeys` the same way. Kept in one place so they can't drift
+# apart. Source alongside env.sh:
+#   source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/lib/sops-age.sh"
+#
+# Uses NIX_OPTS (an array of extra `nix-shell` options -- see env.sh's
+# nix_extra_opts) if the caller has already set it, same convention as
+# lib/ssh-host-keys.sh. Falls back to no extra options if the caller never
+# sourced env.sh.
+if ! declare -p NIX_OPTS >/dev/null 2>&1; then
+  declare -a NIX_OPTS=()
+fi
+
+# sops/age's own default identity-file resolution order, minus $SOPS_AGE_KEY
+# itself (an inline identity, not a path -- callers that accept it check it
+# separately, before falling back to this).
+: "${DEFAULT_SOPS_AGE_KEY_FILE:=${SOPS_AGE_KEY_FILE:-${XDG_CONFIG_HOME:-$HOME/.config}/sops/age/keys.txt}}"
+
+# age_pubkey_from_identity_file <identity-file>
+# Prints the age public key for a private identity file (age-keygen -y).
+age_pubkey_from_identity_file() {
+  local identity_file="$1"
+  nix-shell "${NIX_OPTS[@]}" -p age --run "age-keygen -y '${identity_file}'"
+}
+
+# sops_yaml_admin_pubkey <sops-yaml-path>
+# Prints .sops.yaml's current &admin age public key, or empty (not an error
+# under set -e) if no such anchor line exists -- callers that need to treat
+# "missing" as fatal check for an empty result themselves.
+sops_yaml_admin_pubkey() {
+  local sops_yaml="$1"
+  grep -E '^  - &admin age1' "$sops_yaml" 2>/dev/null | awk '{print $NF}' || true
+}
+
+# sops_updatekeys <secrets-file> [key-file]
+# Re-encrypts <secrets-file> for .sops.yaml's current recipient set. If
+# <key-file> is given, decrypts with that identity (SOPS_AGE_KEY_FILE)
+# instead of whatever's ambient -- needed when the ambient default key
+# doesn't match yet (e.g. mid-rotation, decrypting with the outgoing key).
+sops_updatekeys() {
+  local secrets_file="$1" key_file="${2:-}"
+  if [[ -n "$key_file" ]]; then
+    SOPS_AGE_KEY_FILE="$key_file" nix-shell "${NIX_OPTS[@]}" -p sops --run \
+      "sops updatekeys --yes '${secrets_file}'"
+  else
+    nix-shell "${NIX_OPTS[@]}" -p sops --run "sops updatekeys --yes '${secrets_file}'"
+  fi
+}
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
-# Shared SSH-host-key / age-conversion helpers for scripts/sync-host-keys.sh
-# and scripts/prepare-host-key.sh -- both generate the same kind of key
+# Shared SSH-host-key / age-conversion helpers for scripts/secrets/sync-host-keys.sh
+# and scripts/secrets/prepare-host-key.sh -- both generate the same kind of key
 # (ed25519, no passphrase, the sops-nix age-derivation input) and convert it
 # to an age recipient the same way; kept in one place so the two can't
 # drift apart.
@@ -15,6 +15,16 @@
 # this repo -- update both if nix-cache is ever rebuilt with a new host
 # key or the cache signing key is rotated (see docs/nix-cache.md).
 #
+# REMOTE_BUILDER_KEY defaults to this machine's own default root SSH
+# identity (matches modules/nix-cache/remote-builder-client.nix's
+# convention for real NixOS clients: authenticate as nixremote with the
+# host's own default key, added individually to
+# vars.remoteBuilderAuthorizedKeys, rather than a separately-named or
+# shared keypair) -- generate one with
+# `ssh-keygen -t ed25519 -N '' -f /root/.ssh/id_ed25519` if this machine
+# doesn't have one yet, then add its .pub to vars.remoteBuilderAuthorizedKeys
+# and rebuild nix-cache.
+#
 # Usage:
 #   sudo ./configure-nix-cache-client.sh [--dry-run] [--no-remote-builder] [--no-restart]
 #
@@ -24,9 +34,9 @@
 set -euo pipefail

 : "${NIX_CACHE_HOST:=nix-cache}"
-: "${NIX_CACHE_HOST_KEY:=ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHrMKZlIGUd3pH9G3AqbsruqUGjxIXMAZw52u9MwiBCn lxc-nix-cache}"
+: "${NIX_CACHE_HOST_KEY:=ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPeWgMsdaiz4axT/deFc1+0B5bN+GX/NOeW9bbQ0c/IT lxc-nix-cache}"
 : "${REMOTE_BUILDER_USER:=nixremote}"
-: "${REMOTE_BUILDER_KEY:=/root/.ssh/nixremote}"
+: "${REMOTE_BUILDER_KEY:=/root/.ssh/id_ed25519}"

 CACHE_PUB_KEY="cache.local-1:usoWYanY3Kpq2+kDIS2nhWoLZiRxanmdysdzqCFBHW4="
 FALLBACK_URL="https://cache.nixos.org/"
@@ -13,9 +13,9 @@
 # alone wouldn't carry it) before building.
 #
 # Usage:
-#   scripts/create-proxmox-resource.sh --type lxc|vm --host <name> [options]
-#   scripts/create-proxmox-resource.sh --type lxc|vm --list
-#   scripts/create-proxmox-resource.sh --modify --vmid <n> [--cores N] [--memory MB] [--grow-disk GB]
+#   scripts/proxmox/create-proxmox-resource.sh --type lxc|vm --host <name> [options]
+#   scripts/proxmox/create-proxmox-resource.sh --type lxc|vm --list
+#   scripts/proxmox/create-proxmox-resource.sh --modify --vmid <n> [--cores N] [--memory MB] [--grow-disk GB]
 #
 # SAFETY:
 #   - The default (create) mode only ever creates a NEW resource -- it
@@ -41,13 +41,15 @@
 # See --help for the full option list.
 set -euo pipefail

-repo_root="$(cd "$(dirname "$0")/.." && pwd)"
-# shellcheck source=env.sh
+repo_root="$(cd "$(dirname "$0")/../.." && pwd)"
+# shellcheck source=../env.sh
 source "${repo_root}/scripts/env.sh"
-# shellcheck source=lib/nix-eval.sh
+# shellcheck source=../lib/nix-eval.sh
 source "${repo_root}/scripts/lib/nix-eval.sh"
+# shellcheck source=../lib/confirm.sh
+source "${repo_root}/scripts/lib/confirm.sh"

-sync_keys="${repo_root}/scripts/sync-host-keys.sh"
+sync_keys="${repo_root}/scripts/secrets/sync-host-keys.sh"

 usage() {
  cat <<EOF
@@ -230,8 +232,7 @@ cmd_modify() {
  fi

  echo
-  read -rp "Type the VMID (${vmid}) to confirm these changes: " confirm
-  if [[ "$confirm" != "$vmid" ]]; then
+  if ! confirm_typed "$vmid" "Type the VMID (${vmid}) to confirm these changes: "; then
    echo "Cancelled -- input didn't match ${vmid}."
    exit 1
  fi
@@ -429,8 +430,7 @@ REMOTE_SCRIPT
      echo "  - ${kind} VMID ${id} (${n})"
    done
    echo
-    read -rp "Type the hostname (${host}) to confirm destroying the above and replacing it: " confirm
-    if [[ "$confirm" != "$host" ]]; then
+    if ! confirm_typed "$host" "Type the hostname (${host}) to confirm destroying the above and replacing it: "; then
      echo "Cancelled -- input didn't match ${host}." >&2
      exit 1
    fi
@@ -524,7 +524,7 @@ ensure_remote_repo() {
  echo
  echo "==> Ensuring ${remote_repo_dir} exists and is current on ${node}..."
  if [[ "$dry_run" -eq 1 ]]; then
-    echo "[dry-run] would ensure ${remote_repo_dir} exists on ${node} (clone if missing, git pull if present), and would verify/bootstrap build tooling there (scripts/codex-setup.sh) if \`nix\` isn't already on PATH -- and if that bootstrap actually ran, would also configure ${node} as a nix-cache client (scripts/configure-nix-cache-client.sh)"
+    echo "[dry-run] would ensure ${remote_repo_dir} exists on ${node} (clone if missing, git pull if present), and would verify/bootstrap build tooling there (scripts/codex-setup.sh) if \`nix\` isn't already on PATH -- and if that bootstrap actually ran, would also configure ${node} as a nix-cache client (scripts/proxmox/configure-nix-cache-client.sh)"
    return
  fi

@@ -583,7 +583,7 @@ ensure_remote_repo() {
    # above -- ssh's non-interactive command execution won't have picked
    # up a freshly single-user-installed `nix` otherwise.
    echo "==> Configuring ${node} as a nix-cache substituter/remote-builder client..."
-    if ! ssh "$ssh_target" "cd '${remote_repo_dir}' && . scripts/lib/nix-bootstrap.sh && ensure_nix_profile && bash scripts/configure-nix-cache-client.sh"; then
+    if ! ssh "$ssh_target" "cd '${remote_repo_dir}' && . scripts/lib/nix-bootstrap.sh && ensure_nix_profile && bash scripts/proxmox/configure-nix-cache-client.sh"; then
      echo "WARNING: configure-nix-cache-client.sh failed on ${node} -- continuing without it (${node} will build from source / against cache.nixos.org only)." >&2
    fi
  fi
@@ -766,13 +766,22 @@ if [[ "$type" == "lxc" ]]; then
  # 512M default otherwise (confirmed live: --memory 2048 left swap at
  # 512). Default to matching whatever --memory resolved to above.
  local_swap="${swap:-$memory}"
-  # --unprivileged 1: modules/platforms/lxc.nix sets proxmoxLXC.privileged
-  # = false, so the NixOS config inside the image assumes it's running as
-  # an unprivileged container (cgroup/capability/mount expectations baked
-  # in at boot). `pct create`'s own CLI default for this flag is
-  # privileged (unlike the web UI, which defaults its checkbox the other
-  # way) -- leaving it unset creates a privileged container running a
-  # NixOS config that assumes unprivileged, a real mismatch.
+  # --unprivileged: read back from modules/platforms/lxc.nix's own
+  # proxmoxLXC.privileged (via flake_target_lxc_privileged) rather than
+  # hardcoded, since that's no longer the same for every lxc-* target --
+  # lxc-docker sets it true so the container's NFS mounts work at all (the
+  # kernel's NFS client can't mount from inside any unprivileged
+  # container's user namespace, no matter what AppArmor allows -- see that
+  # option's own comment). The NixOS config inside the image bakes in
+  # cgroup/capability/mount expectations matching whichever value it was
+  # built with, so this must stay in sync with it -- `pct create`'s own
+  # CLI default for this flag is privileged (unlike the web UI, which
+  # defaults its checkbox the other way), so leaving it unset would create
+  # a privileged container running a NixOS config that assumes
+  # unprivileged for every target except lxc-docker, a real mismatch.
+  privileged_eval="$(flake_target_lxc_privileged "$repo_root" "$flake_target")"
+  unprivileged_flag=1
+  [[ "$privileged_eval" == "true" ]] && unprivileged_flag=0
  #
  # --features nesting=1,keyctl=1: required for a modern (v247+) systemd
  # guest to actually boot unprivileged -- confirmed live: without this,
@@ -790,7 +799,7 @@ if [[ "$type" == "lxc" ]]; then
  # hands the whole string to `ssh` as a single command for the *remote*
  # shell to parse -- unquoted, that `;` would be read as a remote
  # command separator and silently truncate this into two commands.
-  create_cmd="pct create ${vmid} ${iso_storage}:vztmpl/${remote_filename} --unprivileged 1 --features '${PROXMOX_DEFAULT_LXC_FEATURES}' --rootfs ${storage}:${local_disk_size} --hostname ${name} --cores ${cores} --memory ${memory} --swap ${local_swap} --net0 name=eth0,bridge=${bridge},ip=dhcp"
+  create_cmd="pct create ${vmid} ${iso_storage}:vztmpl/${remote_filename} --unprivileged ${unprivileged_flag} --features '${PROXMOX_DEFAULT_LXC_FEATURES}' --rootfs ${storage}:${local_disk_size} --hostname ${name} --cores ${cores} --memory ${memory} --swap ${local_swap} --net0 name=eth0,bridge=${bridge},ip=dhcp"
  remote "$create_cmd"
  remote "pct start ${vmid}"
 else
@@ -6,7 +6,7 @@
 # copy is ever lost, or to run either script from a different machine.
 #
 # Usage:
-#   scripts/backup-admin-key.sh <dest-path> [--key-file <path>] [--force] [--dry-run]
+#   scripts/secrets/backup-admin-key.sh <dest-path> [--key-file <path>] [--force] [--dry-run]
 #
 # Source key resolution matches sops/age's own default order:
 #   $SOPS_AGE_KEY (inline identity text) if set, else
@@ -15,11 +15,13 @@
 #   ${XDG_CONFIG_HOME:-$HOME/.config}/sops/age/keys.txt
 set -euo pipefail

-repo_root="$(cd "$(dirname "$0")/.." && pwd)"
+repo_root="$(cd "$(dirname "$0")/../.." && pwd)"
 sops_yaml="${repo_root}/.sops.yaml"

-# shellcheck source=env.sh
+# shellcheck source=../env.sh
 source "${repo_root}/scripts/env.sh"
+# shellcheck source=../lib/sops-age.sh
+source "${repo_root}/scripts/lib/sops-age.sh"

 # Pin cwd for the same reason rotate-admin-key.sh does: age/sops calls
 # below should never depend on wherever the caller's shell happened to be.
@@ -43,7 +45,7 @@ EOF

 dry_run=0
 force=0
-key_file="${SOPS_AGE_KEY_FILE:-${XDG_CONFIG_HOME:-$HOME/.config}/sops/age/keys.txt}"
+key_file="$DEFAULT_SOPS_AGE_KEY_FILE"
 args=()

 while [[ $# -gt 0 ]]; do
@@ -103,13 +105,13 @@ scratch="$(mktemp)"
 trap 'rm -f "$scratch"' EXIT
 ( umask 077; printf '%s\n' "$src_content" > "$scratch" )

-src_pub="$(nix-shell "${NIX_OPTS[@]}" -p age --run "age-keygen -y '$scratch'")" || {
+src_pub="$(age_pubkey_from_identity_file "$scratch")" || {
  echo "ERROR: source doesn't look like a valid age identity (age-keygen -y failed)." >&2
  exit 1
 }
 echo "    public key: ${src_pub}"

-current_admin_pub="$(grep -E '^  - &admin age1' "$sops_yaml" 2>/dev/null | awk '{print $NF}' || true)"
+current_admin_pub="$(sops_yaml_admin_pubkey "$sops_yaml")"
 if [[ -n "$current_admin_pub" && "$current_admin_pub" != "$src_pub" ]]; then
  echo "NOTE: this key does not match .sops.yaml's current &admin entry (${current_admin_pub})."
  echo "      Backing it up anyway -- this script doesn't require it to be the admin key."
@@ -131,7 +133,7 @@ fi
 mkdir -p "$(dirname "$dest")"
 install -m 600 "$scratch" "$dest"

-dest_pub="$(nix-shell "${NIX_OPTS[@]}" -p age --run "age-keygen -y '$dest'")"
+dest_pub="$(age_pubkey_from_identity_file "$dest")"
 if [[ "$dest_pub" != "$src_pub" ]]; then
  echo "ERROR: ${dest} was written but its public key doesn't match the source -- investigate before relying on this backup." >&2
  exit 1
@@ -144,5 +146,5 @@ Done. Backed up to: ${dest}

 This is a private key -- store it somewhere offline/secure, not in this
 repo or anywhere it'd get committed. Restore it with:
-  scripts/rotate-admin-key.sh ${dest}
+  scripts/secrets/rotate-admin-key.sh ${dest}
 EOF
@@ -2,7 +2,7 @@
 # Generates a new machine's SSH host key by an arbitrary name, before it
 # necessarily has a flake target yet -- prints the .sops.yaml snippet to
 # add by hand. For any host that already has a flake target,
-# scripts/sync-host-keys.sh <target> does this same job plus the
+# scripts/secrets/sync-host-keys.sh <target> does this same job plus the
 # .sops.yaml/key_groups registration and re-encryption automatically; use
 # this script only to pre-generate a key ahead of adding the flake target
 # itself.
@@ -22,13 +22,13 @@
 # new machine.
 set -euo pipefail

-repo_root="$(cd "$(dirname "$0")/.." && pwd)"
-# shellcheck source=env.sh
+repo_root="$(cd "$(dirname "$0")/../.." && pwd)"
+# shellcheck source=../env.sh
 source "${repo_root}/scripts/env.sh"
-# shellcheck source=lib/ssh-host-keys.sh
+# shellcheck source=../lib/ssh-host-keys.sh
 source "${repo_root}/scripts/lib/ssh-host-keys.sh"

-hostname="${1:?usage: scripts/prepare-host-key.sh <hostname>}"
+hostname="${1:?usage: scripts/secrets/prepare-host-key.sh <hostname>}"
 sops_yaml="${repo_root}/.sops.yaml"

 if [[ ! -f "$sops_yaml" ]]; then
@@ -10,7 +10,7 @@
 # sync-host-keys.sh print when they bootstrap a brand-new, not-yet-trusted
 # age key on a machine that's never had admin access before:
 #
-#   scripts/rotate-admin-key.sh /path/to/backed-up/admin/keys.txt
+#   scripts/secrets/rotate-admin-key.sh /path/to/backed-up/admin/keys.txt
 #
 # The backup key's *public* key must match .sops.yaml's current &admin
 # entry -- this script verifies that by deriving it, it doesn't just trust
@@ -19,11 +19,13 @@
 # the common case is just pointing this at the restored backup.
 set -euo pipefail

-repo_root="$(cd "$(dirname "$0")/.." && pwd)"
+repo_root="$(cd "$(dirname "$0")/../.." && pwd)"
 sops_yaml="${repo_root}/.sops.yaml"

-# shellcheck source=env.sh
+# shellcheck source=../env.sh
 source "${repo_root}/scripts/env.sh"
+# shellcheck source=../lib/sops-age.sh
+source "${repo_root}/scripts/lib/sops-age.sh"

 # sops resolves .sops.yaml by walking up from the process's cwd, not from
 # the target file's own path -- if this script were invoked from somewhere
@@ -53,7 +55,7 @@ EOF
 }

 dry_run=0
-new_key_file="${SOPS_AGE_KEY_FILE:-${XDG_CONFIG_HOME:-$HOME/.config}/sops/age/keys.txt}"
+new_key_file="$DEFAULT_SOPS_AGE_KEY_FILE"
 args=()

 while [[ $# -gt 0 ]]; do
@@ -93,13 +95,9 @@ backup_key="${args[0]}"

 nix_extra_opts

-age_pub() {
-  nix-shell "${NIX_OPTS[@]}" -p age --run "age-keygen -y '$1'"
-}
-
 echo "==> Deriving public keys..."
-old_pub="$(age_pub "$backup_key")"
-new_pub="$(age_pub "$new_key_file")"
+old_pub="$(age_pubkey_from_identity_file "$backup_key")"
+new_pub="$(age_pubkey_from_identity_file "$new_key_file")"
 echo "    backup (old admin) key: ${old_pub}"
 echo "    new admin key:          ${new_pub}"

@@ -108,12 +106,11 @@ if [[ "$old_pub" == "$new_pub" ]]; then
  exit 1
 fi

-current_admin_line="$(grep -E '^  - &admin age1' "$sops_yaml" || true)"
-if [[ -z "$current_admin_line" ]]; then
+current_admin_pub="$(sops_yaml_admin_pubkey "$sops_yaml")"
+if [[ -z "$current_admin_pub" ]]; then
  echo "ERROR: couldn't find a '&admin age1...' line in ${sops_yaml}." >&2
  exit 1
 fi
-current_admin_pub="$(awk '{print $NF}' <<<"$current_admin_line")"

 if [[ "$current_admin_pub" != "$old_pub" ]]; then
  echo "ERROR: ${backup_key} doesn't match the current &admin key in .sops.yaml." >&2
@@ -129,9 +126,17 @@ if [[ "${#secrets_files[@]}" -eq 0 ]]; then
  exit 1
 fi

+# sops_can_decrypt <key-file> <secrets-file>: used both to confirm the
+# backup key still works before touching anything, and again after
+# rotation to confirm the new key does too.
+sops_can_decrypt() {
+  local key_file="$1" secrets_file="$2"
+  SOPS_AGE_KEY_FILE="$key_file" nix-shell "${NIX_OPTS[@]}" -p sops --run \
+    "sops -d '${secrets_file}'" >/dev/null
+}
+
 echo "==> Confirming the backup key can actually decrypt..."
-if ! SOPS_AGE_KEY_FILE="$backup_key" nix-shell "${NIX_OPTS[@]}" -p sops --run \
-  "sops -d '${secrets_files[0]}'" >/dev/null; then
+if ! sops_can_decrypt "$backup_key" "${secrets_files[0]}"; then
  echo "ERROR: backup key failed to decrypt $(basename "${secrets_files[0]}") -- aborting." >&2
  exit 1
 fi
@@ -162,14 +167,12 @@ echo "    Updated."
 echo "==> Re-encrypting secrets/*.yaml for the new recipient set..."
 for f in "${secrets_files[@]}"; do
  echo "==> $(basename "$f")"
-  SOPS_AGE_KEY_FILE="$backup_key" nix-shell "${NIX_OPTS[@]}" -p sops --run \
-    "sops updatekeys --yes '${f}'"
+  sops_updatekeys "$f" "$backup_key"
 done

 echo "==> Verifying the new key can decrypt everything..."
 for f in "${secrets_files[@]}"; do
-  if ! SOPS_AGE_KEY_FILE="$new_key_file" nix-shell "${NIX_OPTS[@]}" -p sops --run \
-    "sops -d '${f}'" >/dev/null; then
+  if ! sops_can_decrypt "$new_key_file" "$f"; then
    echo "ERROR: new key failed to decrypt $(basename "$f") after rotation -- investigate before committing." >&2
    exit 1
  fi
@@ -24,17 +24,21 @@
 # ever touches keys it itself manages.
 set -euo pipefail

-repo_root="$(cd "$(dirname "$0")/.." && pwd)"
+repo_root="$(cd "$(dirname "$0")/../.." && pwd)"
 sops_yaml="${repo_root}/.sops.yaml"
 keydir="${repo_root}/host-keys"
 editor="${repo_root}/scripts/lib/sync-host-keys-edit-sops.py"

-# shellcheck source=env.sh
+# shellcheck source=../env.sh
 source "${repo_root}/scripts/env.sh"
-# shellcheck source=lib/nix-eval.sh
+# shellcheck source=../lib/nix-eval.sh
 source "${repo_root}/scripts/lib/nix-eval.sh"
-# shellcheck source=lib/ssh-host-keys.sh
+# shellcheck source=../lib/ssh-host-keys.sh
 source "${repo_root}/scripts/lib/ssh-host-keys.sh"
+# shellcheck source=../lib/sops-age.sh
+source "${repo_root}/scripts/lib/sops-age.sh"
+# shellcheck source=../lib/confirm.sh
+source "${repo_root}/scripts/lib/confirm.sh"

 mkdir -p "$keydir"

@@ -78,7 +82,7 @@ ensure_admin_decrypt_key() {
    return
  fi

-  local key_file="${SOPS_AGE_KEY_FILE:-${XDG_CONFIG_HOME:-$HOME/.config}/sops/age/keys.txt}"
+  local key_file="$DEFAULT_SOPS_AGE_KEY_FILE"

  if [[ -s "$key_file" ]]; then
    echo "Found existing sops age key at ${key_file}."
@@ -97,7 +101,7 @@ ensure_admin_decrypt_key() {
  mkdir -p "$(dirname "$key_file")"
  nix-shell "${NIX_OPTS[@]}" -p age --run "age-keygen -o '${key_file}'" 2>&1 | grep -v "^Public key:" || true
  local new_pub
-  new_pub="$(nix-shell "${NIX_OPTS[@]}" -p age --run "age-keygen -y '${key_file}'")"
+  new_pub="$(age_pubkey_from_identity_file "$key_file")"

  cat <<EOF

@@ -239,7 +243,7 @@ apply_edit_plan() {
      while IFS= read -r basename; do
        [[ -z "$basename" ]] && continue
        echo "==> secrets/${basename}"
-        nix-shell "${NIX_OPTS[@]}" -p sops --run "sops updatekeys --yes '${repo_root}/secrets/${basename}'"
+        sops_updatekeys "${repo_root}/secrets/${basename}"
      done <<<"$changed"
    fi
  fi
@@ -359,8 +363,7 @@ cmd_regenerate_all() {
  echo "image/tarball before it can decrypt secrets again."

  if [[ "$dry_run" -ne 1 ]]; then
-    read -rp "Type REGENERATE to confirm: " confirm
-    if [[ "$confirm" != "REGENERATE" ]]; then
+    if ! confirm_typed "REGENERATE" "Type REGENERATE to confirm: "; then
      echo "Cancelled."
      return
    fi
@@ -0,0 +1,110 @@
+#!/usr/bin/env bash
+# Detects and fixes drift between the ed25519 SSH host key nix-cache is
+# actually serving right now and vars.nixCacheHostKey (variables.nix) --
+# the value modules/nix-cache/remote-builder-client.nix bakes into every
+# client's declarative programs.ssh.knownHosts, and
+# scripts/proxmox/configure-nix-cache-client.sh hardcodes as its own
+# default for non-NixOS clients.
+#
+# This value has no automatic source of truth: nix-cache's host key is
+# generated once (first boot / container recreate) and never touches this
+# repo again unless someone remembers to update it by hand afterwards. It
+# drifted silently once already -- confirmed live: variables.nix recorded
+# a key that no longer matched what nix-cache actually presented, which
+# would fail every real client's SSH host-key verification for
+# distributed builds without ever producing an obvious error pointing
+# back here (a client just sees "Host key verification failed" against
+# *some* key, with no hint that the trusted value itself was stale).
+#
+# codex-maintenance.sh runs this in --check mode on every invocation so
+# that drift surfaces as a warning instead of a future debugging session.
+#
+# Usage:
+#   scripts/secrets/sync-nix-cache-host-key.sh [--check] [--dry-run] [--host <name>]
+#
+#   --check     Only report drift (exit 1 if found, 2 if nix-cache is
+#               unreachable); never writes. For CI/maintenance use.
+#   --dry-run   Show what would change; never writes.
+#   --host      Override the hostname to scan (default: variables.nix's
+#               nixCacheHost / env.sh's NIX_CACHE_HOST).
+set -euo pipefail
+
+repo_root="$(cd "$(dirname "$0")/../.." && pwd)"
+variables_nix="${repo_root}/variables.nix"
+client_script="${repo_root}/scripts/proxmox/configure-nix-cache-client.sh"
+
+# shellcheck source=../env.sh
+source "${repo_root}/scripts/env.sh"
+
+check_only=0
+dry_run=0
+host="${NIX_CACHE_HOST}"
+
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    --check) check_only=1; shift ;;
+    --dry-run) dry_run=1; shift ;;
+    --host)
+      host="${2:?--host requires a hostname}"
+      shift 2
+      ;;
+    -h|--help)
+      sed -n '2,23p' "$0"
+      exit 0
+      ;;
+    *)
+      echo "ERROR: unknown argument: $1" >&2
+      exit 1
+      ;;
+  esac
+done
+
+current_value="$(grep -oE 'nixCacheHostKey = "[^"]+"' "$variables_nix" | sed -E 's/nixCacheHostKey = "(.*)"/\1/')"
+if [[ -z "$current_value" ]]; then
+  echo "ERROR: couldn't find nixCacheHostKey in $variables_nix" >&2
+  exit 1
+fi
+current_type_blob="$(awk '{print $1, $2}' <<<"$current_value")"
+current_label="$(awk '{print $3}' <<<"$current_value")"
+
+echo "Scanning ${host} for its current ed25519 SSH host key..."
+nix_extra_opts
+scanned="$(nix-shell "${NIX_OPTS[@]}" -p openssh --run "ssh-keyscan -t ed25519 -T 5 '${host}'" 2>/dev/null | grep -v '^#' | head -1 || true)"
+if [[ -z "$scanned" ]]; then
+  echo "ERROR: couldn't reach ${host} (or got no ed25519 host key back) via ssh-keyscan." >&2
+  exit 2
+fi
+scanned_type_blob="$(awk '{print $2, $3}' <<<"$scanned")"
+
+if [[ "$current_type_blob" == "$scanned_type_blob" ]]; then
+  echo "Up to date: ${host}'s host key matches variables.nix's nixCacheHostKey."
+  exit 0
+fi
+
+echo "DRIFT DETECTED:"
+echo "  variables.nix has: $current_type_blob"
+echo "  ${host} is now:    $scanned_type_blob"
+
+if [[ "$check_only" -eq 1 ]]; then
+  echo
+  echo "Run 'scripts/secrets/sync-nix-cache-host-key.sh' (no flags) to fix." >&2
+  exit 1
+fi
+
+new_value="${scanned_type_blob} ${current_label}"
+
+if [[ "$dry_run" -eq 1 ]]; then
+  echo "(--dry-run: would update variables.nix and ${client_script##*/} to:)"
+  echo "  $new_value"
+  exit 0
+fi
+
+sed -i "s|nixCacheHostKey = \"[^\"]*\"|nixCacheHostKey = \"${new_value}\"|" "$variables_nix"
+sed -i "s|NIX_CACHE_HOST_KEY:=[^}]*}|NIX_CACHE_HOST_KEY:=${new_value}}|" "$client_script"
+
+echo "Updated variables.nix and ${client_script##*/} to:"
+echo "  $new_value"
+echo
+echo "This only takes effect on already-deployed NixOS clients after their"
+echo "next rebuild (programs.ssh.knownHosts is declarative). Review with"
+echo "'git diff', then run 'bash scripts/codex-maintenance.sh' before committing."
@@ -5,85 +5,94 @@ sops:
    age:
        - enc: |
            -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBaMUpxYmUzaVY4d1pRY3g3
-            empaVUJSN0JaSlNQQlJtZENwc0VMYVhJSkFvCmJQMkpSd3RoYzlKQjBSZWErQzR2
-            aG5SVndOQWpSRTBDSWJVQkg0c0hiNFUKLS0tIHhiaEdpY3gwZkpCcHl2TW5CNThn
-            aVVUQy9Qd0trb0RNdUpVTXdrdGlrTmcK1uphQAyDV+Gk5+K1YOqw1Z8ynGP5sAPF
-            q5icujja/SGexX18hPYXbkyUtOrBYjW62gCuGJinSBPROoFUJbiP7g==
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBWcWtCck9HajZXTW8xTGFR
+            UnplbDZ1SWcxNnpDYjRyVG5QeGw0YXRTUVJFCjV3anBuVWNnNHFJa2tXeEdZR0ly
+            eXVtRXM3T3BHbXRSa2lwWVBDS0pYTTgKLS0tIEhDY280OWVwNEphU1FJRGRSbzFH
+            LzdQVVlmRDl0em82ZVVWNkdFSFZ5VXMKhKnOOigvGhy+GzcGjJBzfHuoiRKbgVVA
+            S/zM+fDQdS1qlg3Bp/GofCpeKpmERe/SLMlib5LgCUCWyxB+2IPPKw==
            -----END AGE ENCRYPTED FILE-----
          recipient: age1njap586hc0q43kr03g6c8eqhdsmk8zcafkl3f83xwlc2gqhlmfgs4tmwad
        - enc: |
            -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBKOWZxcFlhRHhKelhDS29M
-            NGRlTEhJUGUzcFV6QVFCdTNkdEN3MlljWnl3CnI3LzVxSHBwSi9TYlRUcFdyN2py
-            cVZBV0Q3Z3FhRXA0T1NFQzd2R00yeEUKLS0tIHkxL2dHV3RkVFYxZTJNTVZvVFIr
-            TXpJZzdnYlpJaXBmcjdWWUtxNkc4dWMKVsImJiavzUzSFn78pciNJPHaS3KWqJer
-            VkxF6kF3tl2HmW46eVXtsQowdu+zVR9HS35i/HvQ1r1TyP6qT5ofcQ==
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBBbDhMaUNNMk9MY2FPUm51
+            MVYzUEdoKytEVDRSU2RESGFCamkrVXpITmhJCkl4VVJmM1NBbUE0cCtDWEgyWTNo
+            STJ2cEZZRStJZkY3YTM1TCtiMnpXemcKLS0tIEpkODdLTlAxaFkxbk4ycW1lanVh
+            TnVPR2xwVG9aTHhPYm01SG44ajJpeUEKNObIucwjlj3Y7YqhJwQGdeV1wFAcANmn
+            564nMYDfh/KvN86r5yhwwYzkJHV2ejlh/30lxAPG5Z9gdtgnFRJrVw==
            -----END AGE ENCRYPTED FILE-----
          recipient: age19gfn2yedg76dmztm4hncr7vf3r3c9j0qpt4rap7y7gersjk4m3ks2lhd0e
        - enc: |
            -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSAyWjdSanRRaEI4VHFZVkdw
-            NkxMODRuakFJTFNoTHZIRHptZFNxaGZ3K3lnCjJGR1FJalRsYmtLcHFxZW8yNU5W
-            MzBjWmltbzI3MTByUjB6djlEdllHNXMKLS0tIEhEOVhLVjZkRi9vUEtDWUhxT0NX
-            eUZ3bGdEaHdGbXZIYkowYUZuTWJ3d2MKMwgxOqlMH7GfLlseD4J277Dcg0KCD3d5
-            jwJDb82kRWoBMicTJZXoq/5oe5blJNa7dWYoqgkYxMA2O+0igT+I6g==
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBSTzNpdzZ3U05zKzNnSXkz
+            Z1NmNURPbXlnelNIQTVUYXBHc002VmlLU2xrCm93TUwxaHgyWWdGVVdOQlFjTFAw
+            aWNHeEFFRjVMRnk2TUpxY1lUQlFzVGMKLS0tIHBTbGhmcVhxa2h0M21qcElnRGp1
+            RDBNSnRhUWExcnh0TFlaNzZ4MXBQYkEKrpCjW26PsgwM8AqS3d+EjBkpNlUcUxlQ
+            a4GlRW+FDTKrVzvfnZRhILKU/ekJQP5nGyjLS9QdWloDxiANbUjddA==
            -----END AGE ENCRYPTED FILE-----
          recipient: age1ll6hj5ggruetgjwjfnplpn5xtq35uhlcdflksx3xmnjm6s3uad9sz70jkf
        - enc: |
            -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBsZzdRMm9zdlJHckFoREVs
-            UkVXL3dCeWRuN2NqcnVsOWVNT1FLUHY3andrCmNFY0ZCby9SdnUyYlV6R05YNDB0
-            em43SjRCNVhYbmpRZUVWT0cxQlhGdmsKLS0tIDZHazNyck5VNHBuNVM5bmRZUlpR
-            QzBSNy84VDdLVkZZbnNlUFYydXlreEEKYZaR2b7tyRAhPdP+ytpP0veUTi7pY9Nw
-            pK0h4hcegLNYJL1AfOYwFQoW7vb256GdmwdcuuBl1YBGXWGraaBnZg==
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBGc0UwZjFJdUZXaGpjS09Q
+            Nk9jNElBYlVjWVFCeHFObm9ScVRrMVQrNUg0CnpGWkE3U0xSS0hHcVlnOERRY0Rx
+            Y3pKeU51QmovdUt2VkcyajZpWDk2ZDAKLS0tIHA0VmZqc0xQS0RvbU4zNnRhL2VK
+            VWFhWExzb2M3ZjgrOERzVlN6S3c5SUUKa2QQz0S9i9IPCZnwDGqLsnCZhdsC3Dwa
+            gND6ExDrXuvDK5HgZjixCzpYOFC/JloJPOONH8gkc3hCany6N9rCpw==
            -----END AGE ENCRYPTED FILE-----
          recipient: age120le4a5l8dh3lyfgvmj3d9ksmej6ajs5mer5y7r0vfg3x9fn69dqf8xgzu
        - enc: |
            -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBFSnZiOGpsL1BESWVEdUxx
-            dXc5LzBHTlNodGJ1S3hOeHd5Wmgxb3BPSHl3CnBEWDNTc3NDdzN3RHQxUlNLOEs5
-            RU1SS2tVT29XbDJCWVliWnVkekN4L1kKLS0tIGFqY2pNK2h2S2ZWMndrRVN6eUdN
-            M0NLY2EvTTVIYlZvdk9XY0NwNE14ZlEKcOwgCK3g56kId/4tEt+2iCcylisn+Fca
-            5VWamsXdbwxbxmCEEUbgN3aHrdVz3mV4+8FZqA34yXz8pc45/PUcFg==
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSB0Y3VjL1dYVDU0eXdCKzJH
+            VG00eFZoK2ZnYUZIaWhnLzVlN01XTFBMRGlRCmNBSXJoVHVxd1o3bitjSzNsMjZZ
+            ZU9pSFN0UG00ZUdDY0ZWZ2t0YlVLbTgKLS0tIHA1bUNBT0V0bTBWcmY2UVhkYW1J
+            dmExckdTUVlCWWdpaG0vUDl1aG94d00KdbkEFjCfdkdRABYXLfK/kwwuQubOK51q
+            w5SIKfOkyGX3WATarPWNQAxmNknozQECub1tCs8G0l7eCUjfpLuXdw==
            -----END AGE ENCRYPTED FILE-----
-          recipient: age1qz9d4ka4xgexujyd247s7lp737sulp5fhxl5d65fj2ykvc4j4edqrsdks8
+          recipient: age1jy444f9d9stygj4p3w9kh54cqcfr654tvr75tdvee5cxsgtdtc9q3v60ep
        - enc: |
            -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBJU1cydFFuRnpCUU5mbjFu
-            TTIxREkxZ1hMalVtNjNjcHVaUXpPNWdib2tzCkRScWRTWHhxN3lhckI4UFpMSFRa
-            RGxWcldlWnd5SVJFNWxQWjB3R0pMb3MKLS0tIGdjLzlhTHgyT2hjTmRTK1RiZmxS
-            OWRWYklBWkoyUG8raFNzd3JlRUtEUEUKw+NdAp1Mz0dEOUGPbqCV8y7029I3Kye0
-            keU2T29JGCN1D6x//1NcMUYSaFZKgv0ZVSjVUCl1EnmlJ0nBdbaDXA==
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSA0L0tmV0dldkczS0lSRnR2
+            cE5YaTR3eTc0QTAwSmNZKzVVeC93a1dGT3pBCkFYOWZSekxvMi9JclVKeU5CRk9N
+            SVVXQittb3AvU1FEUFYzQ1VVVEgrWFEKLS0tIGRxVFMxQjBKTWZlakxmWXBXRCt1
+            RTI2amIxYXdsMGdEVmliUXpaUU44dzgK9l6o/NozRvk8cViqgbDetifjZuoRXf5f
+            qQUVc6Qg97RZN2TIczN3YEKYXhzk7A1ihD4yds7FBgzMuvYQWH983g==
            -----END AGE ENCRYPTED FILE-----
          recipient: age120whqj96g26lsgy4udvgsn8dc9lumh8jeu3a564fx79rjr5lxffqmrljuu
        - enc: |
            -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBnNFlzYVBqaGFDVE9xbmt5
-            OVd4ZlZRdEVJd0JFdkRIYVA5ZUEwYjJjWVJjCnZzTExNbDlYbGN6c1dYU2ZlNEVs
-            ZUo4MXdpc0tzbDI0MHl5eXc3d09VRDQKLS0tIGs4SGMya1RZQ0hpa2NtQk9Wa29C
-            bmxIYWk1VThsaFhxRDlOOGhGQ2tYamsKs+PcOiaeNFujCWwZBr+nq1MzrGohl+ch
-            TASI7eNsiHmSVRzSMHv4mX+8yXDHIRgHbFuUty2gdgGFLRjYXe4gDA==
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBkdkpXZnJNN1ZuU0MybWhG
+            WWE3TVNFOURzR25wcTZ6MjIvNG4wSmUrbUhzCnM2aXRHV1h2ZmRVV2xpNVU0WkRy
+            bmFMcHhtRW9FTlRlbklNODErQkZHRWMKLS0tIGwvcXVOVlE4cHNUeUVCV0NLd1FC
+            QUgrWmVXRHE1d05KaEtuUHdISytvam8KgjQaJeQvOUhcPyrfSPhniRBDahaINlkj
+            d9cXhwSVvsB8L9SN6U4mPK/6s8FhWgkwFBjUV4rCMGVr2ngjMZ8BWg==
            -----END AGE ENCRYPTED FILE-----
          recipient: age1xjst4frdh0th6q8m7p7u9g5af7ty5jqeum0p6z8a52a9q7st7ewqw8yl9j
        - enc: |
            -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBkcmVQYXV1NzRrQ2IxTElt
-            QUJRMkhiUUJqRDdwQW03K0lCTEtkSTU5UDJrCjh6SldpQkF1NzIybTJTdnBlOHdQ
-            VlREMy9hM0ZaL21Pa0VPUzcvb0pQQjQKLS0tIFcrK0I3Z3Nhem45eHpZYmdWeit6
-            eXFtRHVxODhaNTVDdVR1a2tOb1N1U1EKZBdLHq8PRMKX2ndFr3AxFVAZRyvhFa1u
-            72R5tordo4IR3HSxG3Z06rokOITd+KKhaQ8NEWEC8qioAAMxEC9QWw==
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBNdGUyZnBxemhEQ2MySkhB
+            RDNtM01pVWxEb2hpVllYOTFWTHdNVWxhUzFNCkZTNlZCME8vYnowSVpKN0dvOWk1
+            QjE1K25QWkNCd2RKenRMR2l5empMclEKLS0tIEtCb3FnREozbWV0RXNQcEtqV21r
+            Y3YxQlJsVml6dm9LMlZ5VDIzNGVTZFEK5Usyy2tYoExD9X9Zw1tof56qA8bPVJAQ
+            zFvHVFQsmou/qH63UPbuJ4xkNf+j8payFLb1iZq0UhgWv0gVlu1q1w==
            -----END AGE ENCRYPTED FILE-----
          recipient: age10at8862478urh0eeuwh8hzln6ck78jgwtztgxatwqlzwagg77y5snm4xzg
        - enc: |
            -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSA0WjJPbXhXNTNyQUZzMmxS
-            Rkd3Yk5Ga3pPRXFtMFpXcmpBZ3RJYnJlaHhjClRDbFBHRVZ0SDdrNFdVWmUwalpT
-            WUt5a2RUTnQ4VFBGMlowR3JycDM3aXMKLS0tIERSeFd4cGdSbzRpSmpSQnB1Yng5
-            OHBma3lYeWEwQ2RvelpzZHRkN1JPWFEKLXkJfEkH3lo5Z4mj8PZXTfAfZn6B46To
-            U0G54tUh0U1EeMnI5ZPRGwDxi5K0eD68HjeE9MBvFBysOChP/ANDPQ==
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBXb2FWU3JhN293V3lYWTh1
+            ZGY4bFRjc2srODdVUk5BK0NqVlRVRXZrcVU0Cm9vYlJkblZ0KzVMVmN1MmVkUlQ0
+            WnhKeXUwT3B6eUt1L0NLQTl3aFJ4WGMKLS0tIFY4M1k3VkR6ZUtUOEI1N1VhMStv
+            dXBoTVdGeXdIT3YvUmo1MXhld1VXWkkKEZkOaVkAxRL85ubMWOQuq2xreU8R1EGx
+            28y71UsnqCSOTRj+fqHgZMo6pqsSHC5I7e+/BsSpshmbJbe86glhuw==
            -----END AGE ENCRYPTED FILE-----
          recipient: age1ezk9x53zt8kcnscdm80jcyf0xq97vndv7jsn3rl8cc0cwm2jmpmq372dzs
+        - enc: |
+            -----BEGIN AGE ENCRYPTED FILE-----
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBhVE4raWVUTmVvOE1lV0Vz
+            d2hvVWlBcWM0b0VOVFEyN0F1WVZBWkVUcGlVCmRBdWV2WmRDMHFtVGwvbk5uZXBi
+            cFB4UjJpQkNlckhXaUdsd3B6UEV0SjAKLS0tICtLUDVNWEREbVVxNEYrMzlXdHFs
+            RDdENE9FbDJXQkNOM1BhSGhGRHgvL0kKovOmMfELUnHFeAL+kqnmD8quZdwLhuUN
+            fDetQQWrz0lQFH3CQ5fGLw9PelKcywifJV2a0zQB+U7aSIfuhfVThQ==
+            -----END AGE ENCRYPTED FILE-----
+          recipient: age1fxxzpnfse8nd9wz78ht3m0plrmraacf4cpga0pe8fm2tdnqcgy8q7qsyvp
    lastmodified: "2026-07-19T02:30:40Z"
    mac: ENC[AES256_GCM,data:UiL3VMDF6rq4Nr87KspcDx434q3tfNXeb5pwH2O+4ssNQ6xzcYDdzXBnhAY3zLBsqPMKrvHBd4Ot/gEMcq3FMIVe7Q6p9yWKpep66KZ/yWEhAlwIVhD79Oj8VS+1CHKjf25zpRdhZorp04oeFQQd9VfjJB4EE/Q1aVbwTGlpIic=,iv:i/0conaFgFia+wzNTdUL6tlSTw35HTK3Ap1Sr5RGHf8=,tag:ULbz5FllShA/JjlSRdxA0g==,type:str]
    unencrypted_suffix: _unencrypted
@@ -26,7 +26,7 @@
  # fresh client that has never manually ssh'd to nix-cache before. Update
  # this if nix-cache's host key is ever rotated or the host is rebuilt
  # from scratch.
-  nixCacheHostKey = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHrMKZlIGUd3pH9G3AqbsruqUGjxIXMAZw52u9MwiBCn lxc-nix-cache";
+  nixCacheHostKey = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPeWgMsdaiz4axT/deFc1+0B5bN+GX/NOeW9bbQ0c/IT lxc-nix-cache";

  # Public keys authorized to SSH in as remoteBuilderUser on the nix-cache
  # host (modules/nix-cache/server.nix) — one per client host that's allowed
@@ -38,6 +38,7 @@
    "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIGtOWOCS+ImHc7NehguoyD7PbonGosKMZqc9+QR3v/h root@nixos"
    "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHxXTQxFnArK5HXG7czeoybZebCGfxpUdusJkPn+BCSp root@server"
    "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICMJhrfFayLBG+gWtO6oAvgambw5nWWgztiTFEaaaVRH debian@surface"
+    "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJEj26SL/emsVjW2YhRucJVp2kTz8WgcEQgjBEBLRikk root@claude"
  ];

  # Admin SSH public key, authorized on the primary user of every host and
Author	SHA1	Message	Date
beatzaplentyandClaude Sonnet 5	6b09a808ed	Make lxc-docker a privileged container: unprivileged can't NFS-mount at all Check NixOS configurations / eval-hosts (pull_request) Successful in 10m33s Details The kernel's NFS client filesystem doesn't set FS_USERNS_MOUNT, so mounting NFS from inside any non-init user namespace -- exactly what an unprivileged LXC container's UID-mapped root runs in -- is rejected at the VFS layer with EPERM, regardless of Proxmox's mount=nfs;nfs4 container feature (which only patches the AppArmor layer). Confirmed live on the redeployed lxc-docker container: TCP to the NFS server's port 2049 succeeds, the server's export table matches the container's IP, and mount.nfs: Operation not permitted still fires immediately with no corresponding denial anywhere in the server's own logs -- a kernel-level rejection that no amount of DNS/ automount/export tweaking (this branch's earlier commits) could ever fix. modules/platforms/lxc.nix now keys proxmoxLXC.privileged off hostName ("docker" -> true) rather than a blanket false, since build-types/docker.nix is also composed for linode-docker/proxmox-docker, which don't import proxmox-lxc.nix at all -- setting this option there would break their eval. create-proxmox-resource.sh reads the value back via a new flake_target_lxc_privileged helper instead of hardcoding --unprivileged 1, so the two stay in sync automatically for every lxc-* target. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T48qgH3VTvs8wvwj44FEbE	2026-07-21 05:57:06 +00:00
beatzaplentyandClaude Sonnet 5	42da626397	Fix NFS mount device strings on lxc-docker: use FQDN, not search domain Check NixOS configurations / eval-hosts (pull_request) Successful in 17m29s Details The previous commit's networking.search fix was wrong. Confirmed live on lxc-docker (vmid 105) after redeploying with it: `resolvectl query server.sweet.home` started failing again, even though `resolvectl query --interface=eth0 server.sweet.home` still resolved correctly to the right IP via the LAN's real DNS server. The debug log showed why -- adding a global search domain via networking.search gave systemd-resolved a domain-matched but server-less "global" scope, which it now prioritizes over eth0's correctly-configured scope for every "*.sweet.home" query, silently sending them to public fallback DNS (1.1.1.1 et al) instead, which of course returns NXDOMAIN for an internal-only name. Bare single-label names (e.g. "server") were never going to work either way -- systemd-resolved only ever tries LLMNR for those, never DNS search-suffixing, regardless of configuration. Reverts the networking.search addition and instead has modules/docker/mount-data.nix build each NFS device string from "${vars.nfsServerHost}.${vars.homeDomain}" (a plain FQDN, no dependency on search-domain behavior at all) -- the same pattern modules/raspi/mount-data.nix already uses for the Raspberry Pi's share and for the identical reason. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T48qgH3VTvs8wvwj44FEbE	2026-07-21 01:19:42 +00:00
beatzaplentyandClaude Sonnet 5	e176ff723d	Add global DNS search domain to fix NFS mounts on lxc-docker Check NixOS configurations / eval-hosts (pull_request) Failing after 40m2s Details Cross-host references throughout this repo (vars.nfsServerHost, vars.nixCacheHost, vars.dockerHost) are bare short names, not FQDNs. Resolving them has always depended on whatever network stack happens to be in play picking up the DHCP-advertised domain as a search suffix -- NetworkManager does this by default, which is why it went unnoticed everywhere else, but LXC containers force-disable NetworkManager and get their systemd-networkd config written directly by Proxmox instead, which never sets one. Confirmed live on lxc-docker (vmid 105) after today's earlier fix for the automount/mount=nfs bugs: systemd-resolved had no search domain for eth0, so "server" failed to resolve ("Name or service not known") while "server.sweet.home" resolved fine via the same DNS server -- every NFS mount in modules/docker/mount-data.nix was still failing. networking.search sets systemd-resolved's own Domains= globally rather than depending on the per-link DHCP path, so it isn't at the mercy of whichever component owns a given host's interface file. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T48qgH3VTvs8wvwj44FEbE	2026-07-20 18:22:12 +00:00
beatzaplenty	61bbe5e6da	clean up in ailse 3 Check NixOS configurations / eval-hosts (push) Failing after 8m41s Details	2026-07-20 17:49:35 +00:00
beatzaplenty	ab719cc8eb	Merge branch 'worktree-maintenance-script-changed-files-only' into worktree-resolve-maintenance-merge Check NixOS configurations / eval-hosts (push) Successful in 21m49s Details # Conflicts: # scripts/codex-maintenance.sh	2026-07-20 17:46:07 +00:00
beatzaplenty	91c977e5e7	Merge pull request 'Fix nix-cache remote-builder trust: stale host key + wrong sshKey path' (#21 ) from worktree-fix-nix-cache-host-key into main Check NixOS configurations / eval-hosts (push) Failing after 12m42s Details Reviewed-on: #21	2026-07-20 17:32:25 +00:00
beatzaplenty	7e9c0c2a6f	Rewrite codex-maintenance.sh to scope CI checks to changed files Check NixOS configurations / eval-hosts (pull_request) Canceled after 0s Details CI was running a full eval of every host + package on every push/PR, which was slow enough to routinely time out the Gitea runner. Default mode now diffs against a base ref and scopes nixpkgs-fmt/statix/eval to the files that changed and the hosts/packages they can affect; a change to flake.nix/flake.lock/variables.nix/modules/common/* (or any other modules/*.nix outside platforms//build-types, whose blast radius isn't inferable from the path) falls back to evaluating everything. The old full sweep moves behind --full-check, which CI never passes; --dry-run adds build-planning on top of whichever scope is active. Also trims codex-setup.sh's redundant full host eval loop -- that's what codex-maintenance.sh is for; setup should just install tooling.	2026-07-20 17:25:22 +00:00
beatzaplentyandClaude Sonnet 5	fd773b65da	Fix nix-cache remote-builder trust: stale host key + wrong sshKey path Check NixOS configurations / eval-hosts (pull_request) Failing after 11m12s Details variables.nix's nixCacheHostKey no longer matched nix-cache's actual SSH host key (confirmed via ssh-keyscan against the live container), so every declaratively-configured client's programs.ssh.knownHosts trusted the wrong key -- distributed builds would fail host-key verification. Also, modules/nix-cache/remote-builder-client.nix hardcoded sshKey to /root/.ssh/nixremote, but the `server` host only has its own default /root/.ssh/id_ed25519 installed (confirmed live via qm guest-agent) -- that file was never even present, so the build machine config pointed at nothing. Standardize on each client's own default identity, matching the per-host-key pattern vars.remoteBuilderAuthorizedKeys already uses instead of a shared/differently-named keypair, and add scripts/secrets/sync-nix-cache-host-key.sh (wired into codex-maintenance.sh's --check) so the host-key drift doesn't silently recur next time nix-cache is rebuilt or recreated. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V7yVH71vGrDVzovh9UaMu8	2026-07-20 17:21:12 +00:00
beatzaplenty	d340aca403	Merge pull request 'Authorize this box's SSH key as a nix-cache remote-builder client' (#20 ) from add-claude-builder-key into main Check NixOS configurations / eval-hosts (push) Failing after 18m59s Details Reviewed-on: #20	2026-07-20 16:31:25 +00:00
beatzaplenty	bf8ee3ce48	Merge pull request 'Worktree refactor scripts shared lib' (#19 ) from worktree-refactor-scripts-shared-lib into main Check NixOS configurations / eval-hosts (push) Failing after 11m3s Details Reviewed-on: #19	2026-07-20 16:29:42 +00:00
beatzaplentyandClaude Sonnet 5	98d4545e8f	Authorize this box's SSH key as a nix-cache remote-builder client Check NixOS configurations / eval-hosts (pull_request) Failing after 11m4s Details Following the existing per-client-key pattern in vars.remoteBuilderAuthorizedKeys (one entry per host's own default root key, not a shared dedicated keypair -- docs/nix-cache.md's "./nixremote" filename is just a generic placeholder in the instructions). This box had no entry yet, which is why its earlier `ssh-copy-id nixremote@nix-cache` never stuck: nix-cache's nixremote authorized_keys is NixOS-declarative and gets overwritten on every rebuild. Public key only; requires a nix-cache rebuild to take effect. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-20 16:29:42 +00:00
beatzaplentyandClaude Sonnet 5	d8687d979c	Reorganize scripts/ into secrets/, proxmox/, and lib/ subfolders Check NixOS configurations / eval-hosts (pull_request) Failing after 30m6s Details scripts/ had grown to 10 top-level scripts covering three distinct concerns (sops/age + SSH host-key management, Proxmox deployment, and repo-wide bootstrap/CI) with no grouping. Move the key-management scripts (backup-admin-key.sh, rotate-admin-key.sh, prepare-host-key.sh, sync-host-keys.sh) into scripts/secrets/, and the Proxmox scripts (create-proxmox-resource.sh, configure-nix-cache-client.sh) into scripts/proxmox/; leave env.sh, codex-setup.sh, codex-maintenance.sh, and bump-nixpkgs-release.sh at the top level (frequently hand-typed or pure shared config) and scripts/lib/ as-is. Updates every cross-reference: each moved script's repo_root computation (now one directory deeper), shellcheck source= directives, inter-script paths (create-proxmox-resource.sh's call into sync-host-keys.sh and its remote bootstrap of configure-nix-cache-client.sh on the Proxmox node), and every doc/module mention (CLAUDE.md's Scripts section reorganized to match, README.md, docs/auto-installer.md, docs/proxmox-images.md, modules/installer/common.nix, modules/platforms/lxc.nix). CI workflows need no change -- they only invoke codex-maintenance.sh, which didn't move. Verified via bash -n, shellcheck (no new warnings beyond the pre-existing SC1091/SC2029/SC2095 baseline), and live dry-runs of sync-host-keys.sh --all and create-proxmox-resource.sh --list from their new paths. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-20 16:20:45 +00:00
beatzaplentyandClaude Sonnet 5	a2b557c034	Lift duplicated sops/age and confirm-prompt logic into scripts/lib/ scripts/backup-admin-key.sh, rotate-admin-key.sh, and sync-host-keys.sh each independently resolved sops/age's default key-file path, derived an age pubkey from an identity file, and (two of them) ran `sops updatekeys` the same way -- now shared via scripts/lib/sops-age.sh. Also extracted the "type X to confirm" prompt duplicated across create-proxmox-resource.sh and sync-host-keys.sh into scripts/lib/confirm.sh. Pure extraction, no behavior change -- each call site produces identical commands/output to before. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-20 15:38:32 +00:00
beatzaplenty	1d44523181	Merge pull request 'Re-encrypt secrets/nix-cache.yaml for rotated lxc-nix-cache key' (#18 ) from worktree-nix-cache-502-fix into main Check NixOS configurations / eval-hosts (push) Failing after 11m15s Details Reviewed-on: #18	2026-07-20 15:33:31 +00:00