Rewrite codex-maintenance.sh to scope CI checks to changed files

CI was running a full eval of every host + package on every push/PR, which was slow enough to routinely time out the Gitea runner. Default mode now diffs against a base ref and scopes nixpkgs-fmt/statix/eval to the files that changed and the hosts/packages they can affect; a change to flake.nix/flake.lock/variables.nix/modules/common/* (or any other modules/*.nix outside platforms//build-types, whose blast radius isn't inferable from the path) falls back to evaluating everything. The old full sweep moves behind --full-check, which CI never passes; --dry-run adds build-planning on top of whichever scope is active. Also trims codex-setup.sh's redundant full host eval loop -- that's what codex-maintenance.sh is for; setup should just install tooling.
Merge pull request 'Authorize this box's SSH key as a nix-cache remote-builder client' (#20 ) from add-claude-builder-key into main
2026-07-20 17:25:22 +00:00 · 2026-07-20 16:31:25 +00:00 · 2026-07-20 16:29:42 +00:00 · 2026-07-20 16:29:42 +00:00 · 2026-07-20 16:20:45 +00:00 · 2026-07-20 15:38:32 +00:00
34 changed files with 1035 additions and 529 deletions
@@ -13,9 +13,17 @@ jobs:
    steps:
      - name: Check out repository
        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0

      - name: Install Nix
        uses: DeterminateSystems/nix-installer-action@v19

-      - name: Run maintenance checks (secrets, fmt, lint, eval)
+      # Scoped to files changed since the PR base / previous push -- see
+      # scripts/codex-maintenance.sh. CI never passes --full-check: that
+      # full sweep is for local/manual use, since it's slow enough to time
+      # out this runner.
+      - name: Run maintenance checks (secrets, fmt, lint, eval -- changed files only)
+        env:
+          MAINT_BASE_SHA: ${{ github.event.pull_request.base.sha || github.event.before }}
        run: bash scripts/codex-maintenance.sh
@@ -13,9 +13,17 @@ jobs:
    steps:
      - name: Check out repository
        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0

      - name: Install Nix
        uses: DeterminateSystems/nix-installer-action@v19

-      - name: Run maintenance checks (secrets, fmt, lint, eval)
+      # Scoped to files changed since the PR base / previous push -- see
+      # scripts/codex-maintenance.sh. CI never passes --full-check: that
+      # full sweep is for local/manual use, since it's slow enough to time
+      # out this runner.
+      - name: Run maintenance checks (secrets, fmt, lint, eval -- changed files only)
+        env:
+          MAINT_BASE_SHA: ${{ github.event.pull_request.base.sha || github.event.before }}
        run: bash scripts/codex-maintenance.sh
@@ -7,7 +7,7 @@ servers and workstation.

 The flake exposes NixOS configurations named `<platform>-<buildtype>`
 (platforms: `linode`, `proxmox`, `lxc`; build types: `minimal`, `nix-cache`,
-`server`, `docker`, `gui`, `pxe-boot`, `tailscale-exit-node`), generated from `modules/platforms/*`
+`server`, `docker`, `gui`, `pxe-boot`, `tailscale-exit-node`, `tor-relay`), generated from `modules/platforms/*`
 and `modules/build-types/*` by the `mkTarget` function in `flake.nix`. Not
 every combination is built — `pxe-boot` has no `linode` variant. See
 `README.md` for the full current target list; treat `flake.nix` as the
@@ -35,9 +35,14 @@ Use these commands when validating changes:
 ```bash
 bash scripts/codex-setup.sh
 bash scripts/codex-maintenance.sh
-bash scripts/codex-maintenance.sh dry-run
 ```

+With no flags, `codex-maintenance.sh` scopes fmt-check/statix/eval to files
+changed against a base ref — this is what CI runs on every push/PR. For the
+full sweep (every host, every package — slow; CI never runs this), use
+`bash scripts/codex-maintenance.sh --full-check` (add `--dry-run` for build
+planning on top of whichever scope is active).
+
 Host evaluation is safe when limited to drvPath checks:

 ```bash
@@ -37,11 +37,22 @@ machines when deployed.
 # One-time environment bootstrap (installs Nix if missing, prints hosts)
 bash scripts/codex-setup.sh

-# Full validation: secret grep, nixpkgs-fmt --check, statix lint, eval all hosts
+# Changed-files-only validation: secret grep (whole repo), nixpkgs-fmt --check
+# and statix on changed *.nix files, eval of the hosts/packages those changes
+# can affect. This is what CI runs on every push/PR.
 bash scripts/codex-maintenance.sh

-# Same, plus a dry-run build (no result symlink) of every host's toplevel
-bash scripts/codex-maintenance.sh dry-run
+# Full sweep: nixpkgs-fmt --check/statix over the whole tree, eval every host
+# and package. Slow (minutes) -- CI never runs this; use it locally before a
+# release or after touching modules/common/*, flake.nix, or variables.nix for
+# extra confidence beyond the automatic full-fallback those paths already
+# trigger in the default mode (see below).
+bash scripts/codex-maintenance.sh --full-check
+
+# Either mode, plus a dry-run build (no result symlink) of every host/package
+# in whichever scope is active
+bash scripts/codex-maintenance.sh --dry-run
+bash scripts/codex-maintenance.sh --full-check --dry-run

 # List the hosts the flake currently exposes
 nix eval --json .#nixosConfigurations --apply builtins.attrNames | jq -r '.[]'
@@ -58,36 +69,80 @@ maintenance script pulls them via `nix run github:NixOS/nixpkgs/nixos-25.11#<too
 There is no test suite — "correctness" here means the flake evaluates and
 `nixpkgs-fmt`/`statix` are clean.

-**In an interactive agent session**, prefer targeted checks over full-repo
-sweeps: after editing one or two hosts/modules, evaluate just the
-`nixosConfigurations.<host>` you touched (plus any `config.system.build.tarball`
-/`diskoImagesScript`/package output affected) rather than looping over every
-host — `codex-maintenance.sh` evaluates every `nixosConfigurations` host plus
-every package/tarball/image variant and is slow to run after each small
-change. Reserve a full
-`codex-maintenance.sh` run for changes that plausibly affect every host
-(`modules/common/*`, `flake.nix`, `variables.nix`) or as a final check before
-committing. This is a session-workflow preference only — it does not apply to
-CI, which should keep running the full script on every push/PR regardless of
-diff size; that's the point of it.
+With no flags, `codex-maintenance.sh` diffs against a base ref (env
+`MAINT_BASE_SHA`, else the PR base SHA in CI, else `HEAD^` locally) and scopes
+fmt-check/statix to the changed `*.nix` files and eval to the hosts/packages
+those changes can affect — a `hosts/<name>/host.nix` edit only evals that
+host's targets, a `modules/platforms/<platform>.nix` edit only evals that
+platform's hosts, and so on. A change to `flake.nix`, `flake.lock`,
+`variables.nix`, `modules/common/*`, or any other `modules/*.nix` file outside
+`platforms/`/`build-types/` (whose blast radius isn't safely inferable from
+the path alone) falls back to evaluating every host and package, same as
+`--full-check` would, just without the whole-tree fmt/statix sweep. This
+exists because the whole-tree sweep is what was timing out CI; **CI always
+runs the plain, no-flag form and never passes `--full-check`.**
+
+The default mode's diff is against the working tree (uncommitted and staged
+edits included, not just committed ones), so it's already the right tool for
+an interactive session too: after editing one or two hosts/modules, plain
+`bash scripts/codex-maintenance.sh` naturally scopes to just what you
+touched. Reserve `--full-check` for changes that plausibly affect every host
+(`modules/common/*`, `flake.nix`, `variables.nix` — though the default mode
+already falls back to evaluating everything for those paths, `--full-check`
+additionally re-checks fmt/statix over the whole tree) or as a final check
+before committing.

 ## Scripts

-Beyond `codex-setup.sh`/`codex-maintenance.sh` above, `scripts/` also has:
+Beyond `codex-setup.sh`/`codex-maintenance.sh` above, `scripts/` is
+organized by purpose: `scripts/secrets/` (sops/age + SSH host-key
+management), `scripts/proxmox/` (Proxmox deployment), `scripts/lib/`
+(shared helpers, sourced by the scripts below — not run directly), and a
+handful of repo-wide scripts left at the top level (`env.sh`,
+`bump-nixpkgs-release.sh`, plus `codex-setup.sh`/`codex-maintenance.sh`
+above). When adding a new script, put it in the matching subfolder rather
+than the top level, and if it duplicates logic another script already has,
+lift the shared part into `scripts/lib/` instead of copying it.

- `scripts/sync-host-keys.sh` — generates/registers SSH host keys and their
-  `.sops.yaml`/`secrets/*.yaml` recipients for flake targets, idempotently
-  (`--all`, `<target>`, `--remove`, `--regenerate-all-keys`, all with
-  `--dry-run`). The primary tool for provisioning a new host's secrets
-  access — see "Creating a new machine" in `docs/auto-installer.md`.
- `scripts/prepare-host-key.sh` — narrower predecessor: generates a key by
-  an arbitrary name without touching `.sops.yaml`. Still useful to
+### `scripts/secrets/`
+
+- `scripts/secrets/sync-host-keys.sh` — generates/registers SSH host keys
+  and their `.sops.yaml`/`secrets/*.yaml` recipients for flake targets,
+  idempotently (`--all`, `<target>`, `--remove`, `--regenerate-all-keys`,
+  all with `--dry-run`). The primary tool for provisioning a new host's
+  secrets access — see "Creating a new machine" in `docs/auto-installer.md`.
+- `scripts/secrets/prepare-host-key.sh` — narrower predecessor: generates a
+  key by an arbitrary name without touching `.sops.yaml`. Still useful to
  pre-generate a key before its flake target exists yet, since
  `sync-host-keys.sh` can only act on targets `nixosConfigurations` already
  has.
- `scripts/create-proxmox-resource.sh` — builds a `lxc-*`/`proxmox-*`
-  target's tarball/disk image and creates it on a real Proxmox node
-  (`pct create` against the tarball as a CT template / `qm create`+
+- `scripts/secrets/rotate-admin-key.sh <backup-admin-key> [--new-key-file
+  <path>] [--dry-run]` — rotates `.sops.yaml`'s `&admin` age key: decrypts
+  with a backed-up copy of the key currently trusted as `&admin` (verified
+  by deriving its public key and comparing, not taken on faith), replaces
+  the `&admin` line with a new key already present in the environment
+  (defaults to wherever sops/age itself would look), and runs
+  `sops updatekeys` on every `secrets/*.yaml`. One-way: the old key can no
+  longer decrypt anything re-encrypted this way. This is the automation
+  for the manual steps `sync-host-keys.sh`/`create-proxmox-resource.sh`
+  print when they bootstrap a brand-new, not-yet-trusted key on a machine
+  with no prior admin access.
+- `scripts/secrets/backup-admin-key.sh <dest-path> [--key-file <path>]
+  [--force] [--dry-run]` — copies the local sops age key (source
+  resolution matches sops/age itself: `$SOPS_AGE_KEY` inline, then
+  `--key-file`, then `$SOPS_AGE_KEY_FILE`, then the XDG default) to an
+  arbitrary destination path with `0600` permissions, validating it's a
+  real age identity and round-tripping the public key before and after the
+  write. Refuses to overwrite an existing `<dest-path>` without `--force`.
+  Purely a local filesystem copy — never touches `.sops.yaml`/
+  `secrets/*.yaml` or the repo at all. The resulting file is exactly what
+  `rotate-admin-key.sh` expects as its backup-key argument.
+
+### `scripts/proxmox/`
+
+- `scripts/proxmox/create-proxmox-resource.sh` — builds a `lxc-*`/
+  `proxmox-*` target's tarball/disk image and creates it on a real Proxmox
+  node (`pct create` against the tarball as a CT template / `qm create`+
  `importdisk`), or reconfigures an existing resource's cores/memory/disk
  size (`--modify`, always requires typing the VMID back to confirm).
  Checks for an already-uploaded image on the node before building
@@ -97,7 +152,51 @@ Beyond `codex-setup.sh`/`codex-maintenance.sh` above, `scripts/` also has:
  Refuses to create a target whose host identity already exists live on
  the node (checked directly via `qm`/`pct`, not any file in this repo)
  unless `--allow-duplicate-host` is passed. `--dry-run` throughout both
-  modes.
+  modes. The first time it has to bootstrap build tooling on a node (i.e.
+  `nix` wasn't already on its `PATH`), it also runs
+  `scripts/proxmox/configure-nix-cache-client.sh` there (non-fatally — a
+  failure just falls back to building from source / `cache.nixos.org`) so
+  the node substitutes from and can offload builds to nix-cache on every
+  subsequent run, not just this one.
+- `scripts/proxmox/configure-nix-cache-client.sh [--dry-run]
+  [--no-remote-builder] [--no-restart]` — the non-NixOS equivalent of
+  `modules/nix-cache/client.nix`/`remote-builder-client.nix`, for a plain
+  Debian machine with the Nix package manager (not NixOS) already
+  installed: run as root *on that machine* to add nix-cache as a
+  substituter in `/etc/nix/nix.conf` (`https://cache.nixos.org/` kept as
+  fallback) via `extra-substituters`/`extra-trusted-public-keys` so it
+  layers on top of whatever's already there instead of clobbering it, and,
+  if `/root/.ssh/nixremote` is already present (see docs/nix-cache.md
+  "Remote builder SSH keys"), configures it as a distributed-build
+  machine too and trusts nix-cache's SSH host key in
+  `/etc/ssh/ssh_known_hosts`. Idempotent (re-running replaces its own
+  marked block rather than duplicating it); restarts `nix-daemon` by
+  default so the change takes effect immediately.
+
+### `scripts/lib/`
+
+Sourced by the scripts above, never run directly:
+
+- `nix-bootstrap.sh` — `NIX_CONFIG`/`ensure_nix_profile`, shared by
+  `codex-setup.sh`/`codex-maintenance.sh` and the remote build commands
+  `create-proxmox-resource.sh` runs over SSH.
+- `nix-eval.sh` — `NIX_EVAL_FLAGS` plus `list_flake_targets`/
+  `flake_target_hostname` flake-introspection helpers.
+- `ssh-host-keys.sh` — `generate_host_ed25519_key`/`ssh_pubkey_to_age`,
+  shared by `sync-host-keys.sh` and `prepare-host-key.sh`.
+- `sops-age.sh` — `age_pubkey_from_identity_file`/`sops_yaml_admin_pubkey`/
+  `sops_updatekeys` plus the shared sops/age default key-file resolution,
+  shared by `backup-admin-key.sh`, `rotate-admin-key.sh`, and
+  `sync-host-keys.sh`.
+- `confirm.sh` — `confirm_typed`, the "type X back to confirm" destructive-
+  action prompt shared by `create-proxmox-resource.sh` and
+  `sync-host-keys.sh`.
+- `sync-host-keys-edit-sops.py` — the `.sops.yaml` anchor/key_groups editor
+  `sync-host-keys.sh` shells out to (see that script for why: precise,
+  idempotent YAML edits are impractical in bash).
+
+### Top level
+
 - `scripts/env.sh` — shared config (`PROXMOX_HOST`, storage pool, bridge,
  default cores/memory) sourced by `create-proxmox-resource.sh`. Add new
  cross-script config here instead of duplicating it per-script.
@@ -106,27 +205,6 @@ Beyond `codex-setup.sh`/`codex-maintenance.sh` above, `scripts/` also has:
  reference `variables.nix` (confirmed empirically — `nix flake metadata`
  errors on it), so this is the closest equivalent to a single source of
  truth for the tracked release.
- `scripts/rotate-admin-key.sh <backup-admin-key> [--new-key-file <path>]
-  [--dry-run]` — rotates `.sops.yaml`'s `&admin` age key: decrypts with a
-  backed-up copy of the key currently trusted as `&admin` (verified by
-  deriving its public key and comparing, not taken on faith), replaces the
-  `&admin` line with a new key already present in the environment
-  (defaults to wherever sops/age itself would look), and runs
-  `sops updatekeys` on every `secrets/*.yaml`. One-way: the old key can no
-  longer decrypt anything re-encrypted this way. This is the automation
-  for the manual steps `sync-host-keys.sh`/`create-proxmox-resource.sh`
-  print when they bootstrap a brand-new, not-yet-trusted key on a machine
-  with no prior admin access.
- `scripts/backup-admin-key.sh <dest-path> [--key-file <path>] [--force]
-  [--dry-run]` — copies the local sops age key (source resolution matches
-  sops/age itself: `$SOPS_AGE_KEY` inline, then `--key-file`, then
-  `$SOPS_AGE_KEY_FILE`, then the XDG default) to an arbitrary destination
-  path with `0600` permissions, validating it's a real age identity and
-  round-tripping the public key before and after the write. Refuses to
-  overwrite an existing `<dest-path>` without `--force`. Purely a local
-  filesystem copy — never touches `.sops.yaml`/`secrets/*.yaml` or the
-  repo at all. The resulting file is exactly what `rotate-admin-key.sh`
-  expects as its backup-key argument.

 `sync-host-keys.sh`, `create-proxmox-resource.sh`, and
 `rotate-admin-key.sh` genuinely mutate real state when run for real (not
@@ -160,9 +238,10 @@ nixosSystem {
 ```

 Platforms: `linode`, `proxmox`, `lxc`. Build types: `minimal`, `nix-cache`,
-`server`, `docker`, `gui`, `pxe-boot`, `tailscale-exit-node`. Not every
-combination is built — e.g. `pxe-boot` has no `linode` variant (PXE/DHCP/TFTP
-need LAN L2 adjacency a Linode VPS doesn't have). Treat `flake.nix`'s
+`server`, `docker`, `gui`, `pxe-boot`, `tailscale-exit-node`, `tor-relay`. Not
+every combination is built — e.g. `pxe-boot` has no `linode` variant
+(PXE/DHCP/TFTP need LAN L2 adjacency a Linode VPS doesn't have), and
+`tor-relay` currently only exists as `lxc-tor-relay`. Treat `flake.nix`'s
 `generatedTargets` as the source
 of truth for which hosts exist — `README.md`, `AGENTS.md`,
 `docs/flake-lock-automation.md`, and the CI eval workflows
@@ -190,7 +269,7 @@ removing a host.
  `vzdump` backup-archive metadata this doesn't have), no install step —
  see `docs/auto-installer.md`.
 - `modules/build-types/*.nix` — what a system is for:
-  minimal/server/docker/gui/pxe-boot/nix-cache/tailscale-exit-node.
+  minimal/server/docker/gui/pxe-boot/nix-cache/tailscale-exit-node/tor-relay.
 - `modules/common/configuration.nix` — base NixOS config imported by every
  host: locale, users, nix settings, git.
 - `modules/common/home.nix` / `hosts/nixos/home.nix` — Home Manager config for
@@ -225,7 +304,8 @@ removing a host.
  and `environmentFile`; used by `hosts/server/host.nix` and
  `hosts/nix-cache/host.nix` to avoid duplicating that boilerplate.
 - `modules/tailscale/`, `modules/docker/`, `modules/networking/`,
-  `modules/traefik/`, `modules/services/*` — single-purpose, single-host
+  `modules/traefik/`, `modules/tor/`, `modules/services/*` — single-purpose,
+  single-host
  feature modules (e.g. `docker/enable-service.nix`,
  `services/zfs/enable-service.nix`). Grep `modules/build-types/*.nix` for
  each build type's `imports` list to see which modules apply where.
@@ -10,11 +10,11 @@ pieces composed in `flake.nix`:

 - **Platforms** (what it runs on): `linode`, `proxmox`, `lxc`
 - **Build types** (what it's for): `minimal`, `nix-cache`, `server`, `docker`,
-  `gui`, `pxe-boot`, `tailscale-exit-node`
+  `gui`, `pxe-boot`, `tailscale-exit-node`, `tor-relay`

 Not every combination exists — `pxe-boot` has no `linode` variant, since
-PXE/DHCP/TFTP need LAN L2 adjacency that a Linode VPS doesn't have. The full
-list:
+PXE/DHCP/TFTP need LAN L2 adjacency that a Linode VPS doesn't have, and
+`tor-relay` currently only exists as `lxc-tor-relay`. The full list:

 | Target | Purpose |
 | --- | --- |
@@ -27,13 +27,14 @@ list:
 | `linode-gui` / `proxmox-gui` / `lxc-gui` | Cinnamon desktop workstation — previously the flat `nixos` target |
 | `proxmox-pxe-boot` / `lxc-pxe-boot` | HTTP/iPXE boot asset host — previously the flat `pxe-boot` target |
 | `linode-tailscale-exit-node` / `proxmox-tailscale-exit-node` / `lxc-tailscale-exit-node` | Tailscale exit node |
+| `lxc-tor-relay` | Tor middle relay |

 Which variant of a given buildtype is actually deployed isn't tracked
 anywhere in this repo — that's live infrastructure state, not something a
 committed file can keep accurate, and it changes independently of the code.
 Check the Proxmox node itself, or `/etc/flake-target` on a running host (see
 below), if you need to know what's really out there right now.
-`scripts/create-proxmox-resource.sh`'s duplicate-host guard works the same
+`scripts/proxmox/create-proxmox-resource.sh`'s duplicate-host guard works the same
 way: it checks the Proxmox node directly rather than any file here.

 Each buildtype's `hosts/<name>/host.nix` carries the per-machine identity
@@ -73,10 +74,19 @@ Safe validation commands for Codex and local review:

 ```bash
 bash scripts/codex-setup.sh
-bash scripts/codex-maintenance.sh dry-run
 bash scripts/codex-maintenance.sh
 ```

+`codex-maintenance.sh` with no flags (what CI runs on every push/PR) scopes
+fmt-check/statix/eval to files changed against a base ref — fast, but only
+as thorough as the diff. For the full sweep (every host, every package,
+fmt-check and statix over the whole tree — slow, CI never runs this):
+
+```bash
+bash scripts/codex-maintenance.sh --full-check
+bash scripts/codex-maintenance.sh --full-check --dry-run
+```
+
 For individual host evaluation:

 ```bash
@@ -113,7 +123,7 @@ Three different paths depending on target, none of them involving a manual
  disk image and attached to a new VM with no install step — see
  `docs/proxmox-images.md`.

-`scripts/create-proxmox-resource.sh --type lxc|vm --host <name>` automates
+`scripts/proxmox/create-proxmox-resource.sh --type lxc|vm --host <name>` automates
 either of the last two end to end (host-key registration, building the
 image directly on the Proxmox node itself, `pct create`/`qm create`), with
 `--dry-run` and a guard against duplicating an already-deployed host's
@@ -19,7 +19,7 @@ see "LXC hosts" immediately below for why those are different.**
 ## LXC hosts

 `lxc-*` targets (`lxc-minimal`, `lxc-nix-cache`, `lxc-server`, `lxc-docker`,
-`lxc-gui`, `lxc-pxe-boot`, `lxc-tailscale-exit-node`) are **not** installed via `auto-install.sh` — the
+`lxc-gui`, `lxc-pxe-boot`, `lxc-tailscale-exit-node`, `lxc-tor-relay`) are **not** installed via `auto-install.sh` — the
 interactive menu deliberately excludes them. Don't try to select one there;
 `nixos-install` would bind-mount `/` onto `/mnt` (LXC containers have no raw
 disk to partition) and then refuse to touch the filesystem it's currently
@@ -73,7 +73,7 @@ booting one:

 First boot runs `boot.postBootCommands` (registers the Nix store DB and
 system profile) — there's no separate activation step to run yourself.
-`scripts/create-proxmox-resource.sh --type lxc --host <name>` automates all
+`scripts/proxmox/create-proxmox-resource.sh --type lxc --host <name>` automates all
 of this (host-key handling, building the tarball directly on the Proxmox
 node itself, `pct create` with the flags above) — see its `--help`.

@@ -102,7 +102,7 @@ groups required, got 0`, and *every* secret (including this host's own
 login) permanently fails to decrypt, silently — no error in the boot log
 at all, since the activation step that would install secrets only runs on
 a from-scratch first activation and skips silently once `/run/current-system`
-already exists. `scripts/create-proxmox-resource.sh` always builds with
+already exists. `scripts/proxmox/create-proxmox-resource.sh` always builds with
 `NIXOS_HOST_KEYS_DIR` set for this reason.

 ## Layout
@@ -116,10 +116,10 @@ already exists. `scripts/create-proxmox-resource.sh` always builds with
  `docs/pxe-boot.md`).
 - `modules/installer/host-keys.nix` — optionally bakes pre-generated SSH
  host keys into the image; see "Host keys" below.
- `scripts/sync-host-keys.sh` — admin-workstation tool that generates,
+- `scripts/secrets/sync-host-keys.sh` — admin-workstation tool that generates,
  registers, and (via `--remove`/`--regenerate-all-keys`) retires host
  keys; see "Creating a New Machine" below.
- `scripts/prepare-host-key.sh` — narrower predecessor: generates a single
+- `scripts/secrets/prepare-host-key.sh` — narrower predecessor: generates a single
  key by an arbitrary name without touching `.sops.yaml`. Still useful for
  pre-generating a key *before* its flake target exists (`sync-host-keys.sh`
  can only act on targets `nixosConfigurations` already has); otherwise
@@ -238,7 +238,7 @@ GitHub token behind sops-nix for all of them).
 2. **On your admin workstation, generate and register its host key:**

   ```sh
-   ./scripts/sync-host-keys.sh <flake-target>
+   ./scripts/secrets/sync-host-keys.sh <flake-target>
   ```

   This generates `host-keys/<flake-target>_ssh_host_ed25519_key(.pub)`,
@@ -250,7 +250,7 @@ GitHub token behind sops-nix for all of them).

   Doing this for every host that needs one at once — after adding several
   new targets, or just to catch up any that were missed — is
-   `./scripts/sync-host-keys.sh --all`. See `scripts/sync-host-keys.sh --help`
+   `./scripts/secrets/sync-host-keys.sh --all`. See `scripts/secrets/sync-host-keys.sh --help`
   for its other modes (`--remove`, `--regenerate-all-keys`).

 3. **Commit and push.** The flake build the installer uses has to see the
@@ -8,9 +8,14 @@ and to verify that declared NixOS hosts still evaluate after dependency updates.
 - A scheduled workflow runs `nix flake update` once per week.
 - On GitHub, any resulting `flake.lock` change is proposed through a pull request.
 - On Gitea, the workflow can commit and push `flake.lock` directly when PR automation is not configured.
- A separate CI workflow evaluates every configured host before merge, listed
-  dynamically via `nix eval --json .#nixosConfigurations --apply builtins.attrNames`
-  rather than hand-enumerated, so it can't drift as `<platform>-<buildtype>`
+- A separate CI workflow runs `scripts/codex-maintenance.sh` before merge.
+  Its default mode scopes eval to the hosts/packages a change can affect,
+  determined from a git diff against the PR base — but a `flake.lock` change
+  is treated as repo-wide and always falls back to evaluating every host, so
+  a lock-file update PR still gets full coverage. Hosts are still listed
+  dynamically via
+  `nix eval --json .#nixosConfigurations --apply builtins.attrNames` rather
+  than hand-enumerated, so that fallback can't drift as `<platform>-<buildtype>`
  targets are added or removed. See `README.md` for the current target list.

 ## Why hosts should stop using `--upgrade-all`
@@ -7,7 +7,7 @@ config (`modules/disko/proxmox.nix`) already used to format a real disk on
 install, so there's nothing host-specific to write; it's available for every
 `proxmox-*` target automatically.

-`scripts/create-proxmox-resource.sh --type vm --host <name>` automates the
+`scripts/proxmox/create-proxmox-resource.sh --type vm --host <name>` automates the
 whole walkthrough below (and the equivalent LXC one) end to end, including
 host-key handling and building the image directly on the Proxmox node
 itself (no local build, no image transfer) — see its `--help`. The steps
@@ -51,7 +51,7 @@ sudo ./result \
  --build-memory 2048
 ```

-Generate the key first with `scripts/sync-host-keys.sh <hostname>`, same
+Generate the key first with `scripts/secrets/sync-host-keys.sh <hostname>`, same
 as any other host — see `docs/auto-installer.md` for the full walkthrough
 (it registers the new key in `.sops.yaml` and re-encrypts the affected
 `secrets/*.yaml` files too, no manual editing needed).
@@ -1,151 +0,0 @@
-# Spec: Refactor Flake Targets into Platform × Build-Type Matrix
-
-**Status: implemented.** `flake.nix`'s `generatedTargets`/`mkTarget` and
-`modules/platforms/*`/`modules/build-types/*` are the result of this spec —
-kept here for historical rationale only (referenced from `CLAUDE.md`'s
-"Composition pattern" section), not as an active or open plan. The "Open
-Questions" below were resolved during implementation; don't treat them as
-outstanding. A `tailscale-exit-node` build type was added later, beyond this
-spec's original scope.
-
-## Context
-
-The flake at `~/nixos` currently defines these output targets (flat, ad-hoc naming):
-
- `docker`
- `linode-minimal`
- `nix-cache`
- `nix-minimal`
- `nixos`
- `server`
- `pxe-boot`
-
-Some already follow a `platform-buildtype` convention (`linode-minimal`), most don't.
-`~/nix-auto-installer` is a related repo and should be checked for any coupling to
-these target names (scripts, docs, CI, or install automation that reference them by
-name) before renaming anything.
-
-## Goal
-
-Restructure the flake so targets are generated from two orthogonal concepts:
-
-**Build types** (what the system is for):
- `minimal`
- `nix-cache`
- `server`
- `docker`
- `pxe-boot`
- `gui`
-
-**Platforms** (what it's deployed on):
- `linode` (Linode VM)
- `proxmox` (Proxmox VM)
- `lxc` (Proxmox LXC container)
-
-Final targets should be named consistently as `<platform>-<buildtype>`, e.g.:
-
-```
-linode-minimal    proxmox-minimal    lxc-minimal
-linode-nix-cache  proxmox-nix-cache  lxc-nix-cache
-linode-server     proxmox-server     lxc-server
-linode-docker     proxmox-docker     lxc-docker
-linode-pxe-boot   proxmox-pxe-boot   lxc-pxe-boot
-linode-gui        proxmox-gui        lxc-gui
-```
-
-That's the full matrix (18 targets) if every build type applies to every platform.
-See **Open Questions** below — some combinations may not make sense and should be
-confirmed with me before being built out, not silently included or dropped.
-
-## Migration mapping (old → new)
-
-| Old target        | New target                                          | Notes |
-|--------------------|------------------------------------------------------|-------|
-| `linode-minimal`   | `linode-minimal`                                      | Already correct, keep as-is |
-| `nix-minimal`      | likely `proxmox-minimal` or a platform-less base module | Ambiguous — see Open Questions |
-| `nix-cache`        | base module consumed by `linode-nix-cache`, `proxmox-nix-cache`, `lxc-nix-cache` | Currently platform-less; needs to become a build-type module, not a standalone target |
-| `server`           | base module consumed by `linode-server`, `proxmox-server`, `lxc-server` | Same as above |
-| `docker`           | base module consumed by `linode-docker`, `proxmox-docker`, `lxc-docker` | Confirm docker actually makes sense as an LXC/VM guest build vs. a standalone container image — see Open Questions |
-| `pxe-boot`         | TBD — may stay a single target rather than a per-platform one | See Open Questions |
-| `nixos`            | TBD — unclear what this maps to in the new scheme | See Open Questions |
-
-## Open Questions (Claude Code: raise these with me before implementing, don't guess)
-
-1. **`nixos` target** — what is this currently used for (bare metal install, dev
-   shell, template)? It doesn't obviously map to any of the six build types.
-2. **`nix-minimal` vs `linode-minimal`** — are these two different things, or is
-   `nix-minimal` a leftover/duplicate?
-3. **`pxe-boot` and `gui` across all three platforms** — does PXE boot make sense
-   for an LXC container or a cloud VM (Linode), or is it inherently bare-metal/
-   network-boot only and should remain a single non-platform target? Does `gui`
-   make sense inside an LXC container?
-4. **`docker` as a build type** — is this "a NixOS host configured to run Docker"
-   (which would sensibly have linode/proxmox/lxc variants), or "a Docker container
-   image built by the flake" (which wouldn't take a platform prefix at all, since
-   it doesn't run on Linode/Proxmox/LXC as a guest OS)? These are structurally
-   different and change how it should be wired in.
-5. Confirm whether all 18 combinations should actually exist, or whether this is
-   meant to produce only the combinations that are genuinely useful (e.g. maybe no
-   one needs `lxc-pxe-boot`).
-
-## Implementation approach
-
-1. **Inventory first.** Read the current `flake.nix` and any `nixosConfigurations`/
-   `modules` structure. Map every existing target to what module(s) it actually
-   pulls in. Don't assume — confirm against the real file contents.
-2. **Separate build-type and platform into their own module directories**, e.g.:
-   ```
-   modules/build-types/minimal.nix
-   modules/build-types/nix-cache.nix
-   modules/build-types/server.nix
-   modules/build-types/docker.nix
-   modules/build-types/pxe-boot.nix
-   modules/build-types/gui.nix
-
-   modules/platforms/linode.nix
-   modules/platforms/proxmox.nix
-   modules/platforms/lxc.nix
-   ```
-   Build-type modules should contain only what makes a system "minimal" vs
-   "server" vs "gui", etc. Platform modules should contain only what's specific
-   to running as a Linode VM vs Proxmox VM vs LXC container (virtualisation
-   guest tools, boot method, filesystem/image format, LXC-specific constraints
-   like no kernel modules, etc).
-3. **Generate the target matrix programmatically** in `flake.nix` rather than
-   hand-writing 18 near-identical `nixosConfigurations` entries — e.g. a small
-   function that takes a platform name and build-type name, composes the two
-   modules plus any shared base module, and produces the named output. This
-   keeps future build types/platforms a one-line addition rather than a copy-paste
-   job.
-4. **Only build combinations we've confirmed make sense** (see Open Questions) —
-   don't emit all 18 by default if some are structurally invalid.
-5. **Preserve existing working configs during the transition.** Don't delete the
-   old target names until their replacements build successfully — rename/alias
-   at the end, not the start, so there's no window where the flake is broken.
-
-## Verification
-
-For every new target produced:
-```bash
-nix flake check
-nix build .#nixosConfigurations.<target>.config.system.build.toplevel
-```
-Confirm each builds without evaluation errors before considering it done. If a
-target fails to build, report which one and why rather than silently skipping it.
-
-## Deliverables
-
- Refactored `flake.nix` using the composed module + generated-matrix approach.
- New `modules/build-types/*.nix` and `modules/platforms/*.nix` files.
- Old flat target names removed only after their replacements are verified.
- A short `README.md` (or section in existing docs) listing the final target
-  names and what each one is for.
- A summary at the end of what changed, what was removed, and any of the Open
-  Questions above that got resolved differently than expected.
-
-## Out of scope
-
- Don't touch `~/nix-auto-installer` contents beyond checking it for references
-  to the old target names — if changes there are needed, flag them, don't make
-  them without confirming.
- Don't add new build types or platforms beyond the ones listed here.
@@ -98,6 +98,8 @@
        linode-tailscale-exit-node = mkTarget { platform = "linode"; buildType = "tailscale-exit-node"; hostPath = ./hosts/tailscale-exit-node/host.nix; };
        proxmox-tailscale-exit-node = mkTarget { platform = "proxmox"; buildType = "tailscale-exit-node"; hostPath = ./hosts/tailscale-exit-node/host.nix; };
        lxc-tailscale-exit-node = mkTarget { platform = "lxc"; buildType = "tailscale-exit-node"; hostPath = ./hosts/tailscale-exit-node/host.nix; };
+
+        lxc-tor-relay = mkTarget { platform = "lxc"; buildType = "tor-relay"; hostPath = ./hosts/tor-relay/host.nix; };
      };

      # Auto-install environments (migrated from the former nix-auto-installer
@@ -0,0 +1,12 @@
+_:
+
+{
+  networking.hostName = "tor-relay";
+
+  # No networking.hostId: only ZFS-touching hosts (server, docker) need one
+  # for pool-import safety, and this host does neither.
+
+  # A genuinely new host (not a pre-refactor carry-over), so it tracks the
+  # flake's current nixpkgs release rather than being pinned to an older one.
+  system.stateVersion = "26.05";
+}
@@ -0,0 +1,7 @@
+{ ... }:
+
+{
+  imports = [
+    ../tor/enable-relay.nix
+  ];
+}
@@ -1,5 +1,15 @@
 { config, lib, pkgs, vars, ... }:

+let
+  # `x-systemd.automount` never works inside a Linux container (LXC
+  # included, regardless of privilege) -- confirmed live on lxc-docker:
+  # systemd logs "Starting of <unit>.automount unsupported" for every
+  # share and never mounts them. Mount eagerly there instead, with
+  # `nofail` so a boot with the NFS server unreachable doesn't hang
+  # (the VM platforms rely on automount itself to get that same
+  # non-blocking behavior, so they don't need `nofail` too).
+  automountOpts = if config.boot.isContainer then [ "nofail" ] else [ "x-systemd.automount" ];
+in
 {
  fileSystems = {
    ${vars.nfsShares.dockerConfig.mountpoint} = {
@@ -9,9 +19,8 @@
      options = [
        "nfsvers=4.2"
        "_netdev"
-        "x-systemd.automount"
        "noatime"
-      ];
+      ] ++ automountOpts;
    };

    ${vars.nfsShares.dockerDatabases.mountpoint} = {
@@ -21,9 +30,8 @@
      options = [
        "nfsvers=4.2"
        "_netdev"
-        "x-systemd.automount"
        "noatime"
-      ];
+      ] ++ automountOpts;
    };

    ${vars.nfsShares.dockerVolumes.mountpoint} = {
@@ -33,9 +41,8 @@
      options = [
        "nfsvers=4.2"
        "_netdev"
-        "x-systemd.automount"
        "noatime"
-      ];
+      ] ++ automountOpts;
    };

    ${vars.nfsShares.nextcloudData.mountpoint} = {
@@ -45,9 +52,8 @@
      options = [
        "nfsvers=4.2"
        "_netdev"
-        "x-systemd.automount"
        "noatime"
-      ];
+      ] ++ automountOpts;
    };

    ${vars.nfsShares.raspiVolumes.mountpoint} = {
@@ -57,9 +63,8 @@
      options = [
        "nfsvers=4.2"
        "_netdev"
-        "x-systemd.automount"
        "noatime"
-      ];
+      ] ++ automountOpts;
    };
  };
 }
@@ -130,7 +130,7 @@
        # at *activation* time, which runs before systemd would otherwise
        # generate one on first boot. Without pre-seeding it here, secrets
        # (including the login password) fail to decrypt on first boot.
-        # Generate the key with scripts/prepare-host-key.sh first.
+        # Generate the key with scripts/secrets/prepare-host-key.sh first.
        #
        # Two places a key can come from, checked in order:
        #   /etc/host-keys  — baked into this image at build time (see
@@ -150,7 +150,7 @@
        else
            echo "WARNING: no SSH host key found for ''${choice} (checked /etc/host-keys and /root/host-keys)"
            echo "sops-nix secrets (including the login password) will NOT decrypt on first boot."
-            echo "Run scripts/prepare-host-key.sh for host ''${choice} on your admin workstation first,"
+            echo "Run scripts/secrets/prepare-host-key.sh for host ''${choice} on your admin workstation first,"
            echo "then either rebuild this image with NIXOS_HOST_KEYS_DIR set, or scp the result to"
            echo "/root/host-keys/ on this machine."
            read -rp "Continue without a pre-seeded key anyway? (y/N): " skip_key
@@ -14,7 +14,7 @@ let
  # Without this, config.system.build.tarball's built-in system just
  # generates a fresh host key at first boot like any other host would --
  # but sops-nix derives its decryption key from *this* file, and
-  # .sops.yaml only trusts whatever key scripts/sync-host-keys.sh already
+  # .sops.yaml only trusts whatever key scripts/secrets/sync-host-keys.sh already
  # registered for this exact target name. A freshly-generated key can
  # never match that, so every secret (including this host's own login)
  # permanently fails to decrypt. Confirmed live: sops-install-secrets
@@ -26,7 +26,7 @@ let
  hostKeysDir = /. + hostKeysDirStr;

  # flakeTarget ("${platform}-${buildType}") comes in via specialArgs from
-  # flake.nix's mkTarget -- exactly the name scripts/sync-host-keys.sh
+  # flake.nix's mkTarget -- exactly the name scripts/secrets/sync-host-keys.sh
  # registers keys under. Deliberately not read back from
  # config.environment.etc."flake-target" (which is set to the same value)
  # -- this module also *contributes* to environment.etc below, and a
@@ -1,4 +1,4 @@
-{ vars, ... }:
+{ config, lib, vars, ... }:

 {
  fileSystems.${vars.raspiMountpoint} = {
@@ -9,6 +9,15 @@
      "_netdev"
      "noatime"

+      # Explicitly use NFSv4.2 if supported
+      "nfsvers=4.2"
+    ] ++ lib.optionals (!config.boot.isContainer) [
+      # `x-systemd.automount` never works inside a Linux container (LXC
+      # included) -- confirmed live on lxc-docker: systemd logs "Starting
+      # of <unit>.automount unsupported" and never mounts it. `nofail`
+      # above already keeps boot non-blocking there, so plain eager
+      # mounting is fine.
+
      # Don't mount until first access
      "x-systemd.automount"

@@ -17,9 +26,6 @@

      # Give the Pi/Tailscale a little time to appear
      "x-systemd.device-timeout=10s"
-
-      # Explicitly use NFSv4.2 if supported
-      "nfsvers=4.2"
    ];
  };

@@ -0,0 +1,35 @@
+{ pkgs, vars, ... }:
+
+{
+  services.tor = {
+    enable = true;
+
+    # Opens settings.ORPort (and DirPort, unset here) in the firewall —
+    # see the nixpkgs tor module's own networking.firewall.mkIf block.
+    openFirewall = true;
+
+    relay = {
+      enable = true;
+      # Plain middle/guard relay, not "exit" — relays onion traffic between
+      # other Tor nodes without ever making requests to the public internet
+      # on a user's behalf, avoiding the abuse complaints and legal exposure
+      # an exit node invites.
+      role = "relay";
+    };
+
+    settings.ORPort = vars.ports.torRelayOrPort;
+
+    # Unix control socket at /run/tor/control (GroupWritable, group "tor")
+    # -- what nyx below actually monitors the relay through. Nyx's own
+    # default control-socket path (/var/run/tor/control) resolves to the
+    # same place, so no extra nyx config is needed.
+    controlSocket.enable = true;
+  };
+
+  # Lets the primary user's shell session read/write the control socket
+  # above without being root -- otherwise nyx fails to authenticate against
+  # it at all.
+  users.users.${vars.primaryUser}.extraGroups = [ "tor" ];
+
+  environment.systemPackages = [ pkgs.nyx ];
+}
@@ -1,146 +0,0 @@
-# Spec: Remove Sensitive Information from NixOS Flake
-
-## Goal
-
-Every secret currently readable in plaintext anywhere in this repo (working tree *and* git history) gets removed, replaced with `sops-nix`-managed encrypted references, and rotated. When this is done, the repo should be safe to make public without exposing anything about the systems it configures.
-
-Treat this as three sequential milestones. Do not start git history rewriting (Milestone 3) until Milestones 1 and 2 are fully verified and the flake still builds. This should be its own branch (`refactor/secrets`) until fully verified, then merged.
-
---
-
-## Milestone 1 — Audit
-
-Before touching anything, produce a complete inventory. Do not guess at scope — grep the whole tree and the whole history.
-
-1. Run a secret scanner across the working tree and full history. Use both, since they catch different things:
-   - `gitleaks detect --source . -v --log-opts="--all"` (scans history too)
-   - `trufflehog git file://. --since-commit=$(git rev-list --max-parents=0 HEAD) --only-verified=false`
-   If neither is installed, add them via a temporary `nix-shell -p gitleaks trufflehog` — don't install anything globally on the host.
-
-2. Manually grep for the categories below, since scanners miss config-specific patterns:
-   - `hashedPassword`, `password`, `initialPassword`, `initialHashedPassword` in any `users.users.*` block
-   - `age.secrets`, `sops.secrets` (if any partial secrets work already exists — check for it)
-   - PSK / `preSharedKey`, `privateKeyFile` inline values (vs. file references) for WireGuard
-   - `authKey`, `apiToken`, `api_key`, `token =`, `secret =` in service modules (Tailscale, Cloudflare, backup tools, etc.)
-   - SSH private key material: search for `BEGIN OPENSSH PRIVATE KEY` / `BEGIN RSA PRIVATE KEY` literals
-   - TLS cert/key pairs committed under e.g. `secrets/`, `certs/`, `pki/`
-   - Real name, personal email, home address, or anything in comments/hostnames that maps a machine to your physical identity or network layout (e.g. hostnames like `wayne-desktop`, static LAN IPs, ISP-identifying info)
-   - `.env` files, `secrets.nix`, `secrets.yaml`, or any file that looks like it was meant to be gitignored but wasn't
-
-3. Produce `secrets-inventory.md` (temporary, delete before finishing) listing: file path, line, secret type, and which host/service it belongs to. This becomes the checklist for Milestone 2 — every row must be either migrated to sops or deleted, with nothing left unaccounted for.
-
---
-
-## Milestone 2 — Migrate to sops-nix
-
-### 2.1 Set up sops-nix
-
-1. Add the flake input:
-   ```nix
-   sops-nix.url = "github:Mic92/sops-nix";
-   sops-nix.inputs.nixpkgs.follows = "nixpkgs";
-   ```
-2. Import `sops-nix.nixosModules.sops` into each host's module list (or into a shared `common.nix` if all hosts use it).
-3. Generate an age keypair **per host** (not one shared key for everything — a compromised host shouldn't decrypt every other host's secrets):
-   ```
-   nix-shell -p age --run "age-keygen -o /var/lib/sops-nix/key.txt"
-   ```
-   Print the public key (`age-keygen -y`) for each host — you'll need it for `.sops.yaml`.
-4. Also generate one age key for yourself (your admin workstation) so you can edit secrets without needing to SSH into a host: store it at `~/.config/sops/age/keys.txt`, back it up somewhere outside this repo (password manager, offline). **If this key is lost, every secret encrypted with it is unrecoverable — losing the age key is equivalent to losing the secrets.**
-5. Create `.sops.yaml` at the repo root defining creation rules: which age public keys can decrypt which secrets files, keyed by path regex, so e.g. `secrets/hostA.yaml` is decryptable by your admin key + hostA's key, `secrets/hostB.yaml` by your admin key + hostB's key.
-
-### 2.2 Migrate each secret category from the inventory
-
-For each row in `secrets-inventory.md`:
-
- **Password hashes**: generate hash with `mkpasswd -m sha-512` (or `bcrypt` if your setup wants that), store under `sops.secrets."<name>/hashedPassword"`, reference via `users.users.<name>.hashedPasswordFile = config.sops.secrets."<name>/hashedPassword".path;`. Do not put the *plaintext* password anywhere, only the hash, and only the hash goes into the encrypted sops file.
- **API tokens / auth keys**: move the raw value into the per-host sops YAML, reference in the module via `config.sops.secrets."<service>/token".path` — most NixOS service modules that take a token also accept a `*File` variant (e.g. `environmentFile`, `tokenFile`); use that instead of passing the value directly.
- **Private keys / certs**: move the PEM/key content wholesale into a sops secret, output as a file with appropriate `sops.secrets.<name>.path`, `owner`, `mode`, `restartUnits` so the depending service (sshd, wireguard, nginx) reloads when the secret changes.
- **Personal/identifying info**: this doesn't belong in sops (it's not "secret," it's just information you don't want public). Replace real names/emails with placeholders or move to a small untracked `local.nix` that's `.gitignore`'d and imported conditionally, with a documented template (`local.nix.example`) committed instead.
-
-### 2.3 Verify before moving on
-
- `nixos-rebuild dry-build --flake .#<host>` succeeds for every host.
- `sudo nixos-rebuild switch --flake .#<host>` on at least one real machine (or a VM) confirms secrets decrypt and services start.
- Confirm decrypted secrets land under `/run/secrets/` (not the Nix store — anything placed in `/nix/store` is world-readable by design, so sops-nix's runtime-only placement is the whole point; double check no module accidentally pulls a secret path into a store-built config file).
- Re-run the grep/scanner sweep from Milestone 1 against the *working tree only* (not history yet) — it should now come back clean.
-
---
-
-## Milestone 3 — Scrub git history
-
-Do this only after Milestone 2 is merged to your main branch and confirmed working, since it rewrites every commit SHA from the point of the earliest offending commit onward.
-
-**This is destructive and irreversible on your local clone. Back up first:**
-```
-cp -r /path/to/nixos-repo /path/to/nixos-repo-backup-$(date +%F)
-```
-
-1. Install `git-filter-repo` (not the older `git filter-branch` / BFG — filter-repo is the currently maintained, faster, safer tool):
-   ```
-   nix-shell -p git-filter-repo
-   ```
-2. Use the `secrets-inventory.md` list to build a list of literal strings/paths to strip. Two approaches, use both:
-   - Path-based: if whole files were secret (e.g. `secrets.nix`, a `.env`, a private key file), remove them entirely from history:
-     ```
-     git filter-repo --path secrets.nix --path .env --invert-paths
-     ```
-   - Value-based: for secrets embedded inline in files you're keeping (not deleting the whole file), use `--replace-text` with a file listing each literal secret string to replace with `***REMOVED***`:
-     ```
-     git filter-repo --replace-text expressions.txt
-     ```
-3. After filtering, verify: run the Milestone 1 scanners again against full history (`--log-opts="--all"`). They must come back clean.
-4. Force-push the rewritten history:
-   ```
-   git push origin --force --all
-   git push origin --force --tags
-   ```
-5. **Every other clone of this repo (other machines, WSL instances, CI) must be deleted and re-cloned fresh** — a `git pull` against rewritten history will not work cleanly and risks resurrecting the old commits. Don't try to reconcile old clones; throw them away and re-clone.
-6. If this repo has ever been pushed to a public host (GitHub, etc.) or a fork/mirror exists, treat every secret that was ever in history as **permanently compromised regardless of the rewrite** — caches, forks, and Wayback-style archives can retain old commits indefinitely. History scrubbing prevents *future* exposure via `git clone`; it does not undo past exposure.
-
---
-
-## Milestone 4 — Rotate everything
-
-Because the secrets were exposed in history (even briefly, even in a private repo), the migration is not complete until every credential in the inventory has been **rotated**, not just re-encrypted. Re-encrypting an already-leaked value protects it going forward but doesn't undo the leak.
-
-For each row in the original inventory:
- Password hashes → change the actual account password, regenerate the hash, update the sops file.
- API tokens/auth keys → revoke the old token in the issuing service's dashboard (Cloudflare, Tailscale, backup provider, etc.) and generate a new one.
- SSH/WireGuard private keys → generate new keypairs, update the corresponding public key wherever it's trusted (authorized_keys, peer configs, etc.), retire the old ones.
- TLS certs → reissue if the private key was exposed.
-
-Keep `secrets-inventory.md` open during this step and check off each row as rotated. Delete the file only once every row is checked off — it should not be committed.
-
---
-
-## Ongoing prevention
-
-Add a pre-commit hook (or a `nix flake check` step) running `gitleaks protect --staged` so a secret can't be committed again by accident. Document in the repo README (briefly) that new secrets go through `sops <file>` to edit, never as plaintext in a tracked file.
-
---
-
-## Definition of done
-
-**Status as of 2026-07-20:** Milestones 1–3 are done — sops-nix is fully
-wired (`.sops.yaml`, `secrets/*.yaml`, referenced via `hashedPasswordFile`/
-`*File`/`sops.secrets.*.path` throughout), and history has been scrubbed
-with `git-filter-repo` + force-push (this removed a GitHub fine-grained PAT
-that had been committed in plaintext in `flake.nix`/`common/home.nix`
-between 2025-07-16 and 2026-02-09, later migrated to sops but never scrubbed
-from history until now). **Milestone 4 is not confirmed** — whether that PAT
-(or any other historically-plaintext credential) was actually rotated, not
-just re-encrypted, isn't something this repo can attest to; that's an
-operator action against the issuing service (GitHub, etc.), not a repo
-change. Do that before considering this fully closed.
-
- [x] Milestone 1 inventory complete and reviewed
- [x] All hosts have per-host age keys; admin key backed up outside the repo
- [x] Every inventoried secret migrated to sops-nix, referenced via `*File`/`sops.secrets.*.path`, nothing plaintext in the working tree
- [x] `nixos-rebuild dry-build` and at least one real `switch` verified per host
- [x] Working-tree scanner sweep clean
- [x] History rewritten with `git-filter-repo`, force-pushed, full-history scanner sweep clean
- [ ] All other clones deleted and re-cloned from the rewritten history — every clone that existed before 2026-07-20's rewrite (any other machine, WSL instance, or CI checkout) needs this
- [ ] Every credential in the original inventory rotated (not just re-encrypted) — **the GitHub PAT found in history specifically still needs this**
- [x] Pre-commit secret scanning hook added (`.githooks/pre-commit`, `gitleaks protect --staged`)
- [x] `secrets-inventory.md` deleted from the working directory (never committed)
@@ -129,5 +129,6 @@ echo "flake.lock still points at the old input revisions until refreshed. Either
 echo "  nix flake update nixpkgs home-manager   # just these two inputs"
 echo "  nix flake update                        # everything — see docs/flake-lock-automation.md"
 echo
-echo "Then run 'bash scripts/codex-maintenance.sh dry-run' before committing —"
-echo "a channel bump can shift option defaults across every host."
+echo "Then run 'bash scripts/codex-maintenance.sh --full-check --dry-run' before"
+echo "committing — a channel bump can shift option defaults across every host,"
+echo "and only --dry-run actually builds anything to catch that."
@@ -1,4 +1,21 @@
 #!/usr/bin/env bash
+# Validation entry point for CI and local/agent review.
+#
+# Default mode (what CI runs on every push/PR): fmt-check, statix, and eval
+# are scoped to files that actually changed against a base ref, plus
+# whichever hosts/packages those changes can affect. This exists because
+# the unscoped sweep below is slow enough to time out CI runners -- see
+# --full-check.
+#
+# --full-check: the historical full sweep (every host, every package,
+# fmt --check ./statix check . over the whole tree). Slow -- minutes, not
+# seconds. CI never passes this; run it locally before a release or after
+# touching modules/common/*, flake.nix, or variables.nix if you want extra
+# confidence beyond what the changed-files scope already covers for those
+# paths (see below).
+#
+# --dry-run: adds `nix build --dry-run --no-link` for whatever scope is
+# active (changed-files scope by default, full scope under --full-check).
 set -euo pipefail

 script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
@@ -7,7 +24,39 @@ source "${script_dir}/lib/nix-bootstrap.sh"
 # shellcheck source=lib/nix-eval.sh
 source "${script_dir}/lib/nix-eval.sh"

-MODE="${1:-validate}"
+repo_root="$(cd "${script_dir}/.." && pwd)"
+cd "$repo_root"
+
+full_check=false
+dry_run=false
+
+usage() {
+  cat <<'EOF'
+Usage: scripts/codex-maintenance.sh [--full-check] [--dry-run]
+
+  --full-check  Run the full sweep: fmt-check and statix over the whole
+                repo, eval every host and package. Slow. Never run by CI.
+  --dry-run     Additionally run `nix build --dry-run --no-link` for
+                whatever scope is active.
+
+With neither flag (the CI default), fmt-check/statix/eval are scoped to
+files changed against a base ref (env MAINT_BASE_SHA, else the PR base,
+else HEAD^), plus the hosts/packages those changes can affect.
+EOF
+}
+
+for arg in "$@"; do
+  case "$arg" in
+    --full-check) full_check=true ;;
+    --dry-run) dry_run=true ;;
+    -h|--help) usage; exit 0 ;;
+    *)
+      echo "Unknown argument: $arg" >&2
+      usage >&2
+      exit 1
+      ;;
+  esac
+done

 ensure_nix_profile

@@ -16,12 +65,6 @@ if ! command -v nix >/dev/null 2>&1; then
  exit 127
 fi

-hosts="$(list_flake_targets .)"
-
-echo "Hosts:"
-echo "$hosts"
-
-echo
 echo "Checking for obvious committed secrets..."
 if grep -RInE 'github_pat_|ghp_|access-tokens|hashedPassword[[:space:]]*=' \
  --exclude-dir=.git \
@@ -33,17 +76,164 @@ else
  echo "No obvious token patterns found."
 fi

+mapfile -t all_hosts < <(list_flake_targets .)
+mapfile -t all_packages < <(nix eval --json "${NIX_EVAL_FLAGS[@]}" .#packages.x86_64-linux --apply builtins.attrNames | jq -r '.[]')
+
+# host_targets_for_dir <hosts-subdir-name>
+# Prints the nixosConfigurations target names whose hostPath is
+# ./hosts/<dir>/host.nix, derived straight from flake.nix's generatedTargets
+# (one mkTarget { ... } call per line) rather than a hand-maintained table,
+# so it can't drift the way a copied mapping would.
+host_targets_for_dir() {
+  local dir="$1"
+  grep -oE '^[[:space:]]*[A-Za-z0-9_-]+ = mkTarget \{[^}]*hostPath = \./hosts/'"${dir}"'/host\.nix;[^}]*\};' flake.nix \
+    | sed -E 's/^[[:space:]]*([A-Za-z0-9_-]+) = mkTarget.*/\1/' \
+    || true
+}
+
+declare -a changed_files=()
+scope_desc="full repo"
+
+if ! $full_check; then
+  resolve_base_ref() {
+    if [[ -n "${MAINT_BASE_SHA:-}" ]] && git cat-file -e "${MAINT_BASE_SHA}^{commit}" 2>/dev/null; then
+      echo "$MAINT_BASE_SHA"
+      return
+    fi
+    if git rev-parse --verify -q HEAD^ >/dev/null 2>&1; then
+      echo "HEAD^"
+      return
+    fi
+    git hash-object -t tree /dev/null
+  }
+
+  base_ref="$(resolve_base_ref)"
+  echo
+  echo "Changed-files scope: diffing against ${base_ref}"
+  mapfile -t changed_files < <(git diff --name-only --diff-filter=ACMR "$base_ref" -- . | sort -u)
+
+  if [[ ${#changed_files[@]} -eq 0 ]]; then
+    echo "No changed files detected."
+  else
+    printf '  %s\n' "${changed_files[@]}"
+  fi
+  scope_desc="changed files only (base: ${base_ref})"
+fi
+
+# Whole-tree fmt/lint always run under --full-check; otherwise scoped below.
+declare -a changed_nix_files=()
+for f in "${changed_files[@]:-}"; do
+  [[ "$f" == *.nix && -f "$f" ]] && changed_nix_files+=("$f")
+done
+
 echo
 echo "Checking Nix formatting with nixpkgs-fmt..."
+if $full_check; then
  nix run "${NIX_EVAL_FLAGS[@]}" github:NixOS/nixpkgs/nixos-25.11#nixpkgs-fmt -- --check .
+elif [[ ${#changed_nix_files[@]} -gt 0 ]]; then
+  nix run "${NIX_EVAL_FLAGS[@]}" github:NixOS/nixpkgs/nixos-25.11#nixpkgs-fmt -- --check "${changed_nix_files[@]}"
+else
+  echo "No changed .nix files; skipping."
+fi

 echo
 echo "Running statix lint..."
+if $full_check; then
  nix run "${NIX_EVAL_FLAGS[@]}" github:NixOS/nixpkgs/nixos-25.11#statix -- check .
+elif [[ ${#changed_nix_files[@]} -gt 0 ]]; then
+  for f in "${changed_nix_files[@]}"; do
+    nix run "${NIX_EVAL_FLAGS[@]}" github:NixOS/nixpkgs/nixos-25.11#statix -- check "$f"
+  done
+else
+  echo "No changed .nix files; skipping."
+fi
+
+# Figure out which hosts/packages this run needs to eval (and, under
+# --dry-run, build). full_check always means "everything"; otherwise a
+# change to flake.nix/flake.lock/variables.nix/modules/common/* (repo-wide
+# inputs) or to any other modules/*.nix outside platforms//build-types
+# (whose blast radius isn't safely inferable from the path alone -- see
+# CLAUDE.md's "Grep modules/build-types/*.nix for each build type's imports
+# list") also falls back to everything, on the same reasoning CLAUDE.md
+# already gives interactive sessions for when to run the full sweep.
+# Anything more targeted -- a host.nix, a platform module, a build-type
+# module -- narrows to just the hosts it can affect.
+declare -A affected_hosts=()
+eval_packages=false
+
+if $full_check; then
+  for h in "${all_hosts[@]}"; do affected_hosts[$h]=1; done
+  eval_packages=true
+else
+  full_fallback=false
+  for f in "${changed_files[@]:-}"; do
+    case "$f" in
+      flake.nix|flake.lock|variables.nix|modules/common/*)
+        full_fallback=true
+        ;;
+    esac
+  done
+
+  if ! $full_fallback; then
+    for f in "${changed_files[@]:-}"; do
+      case "$f" in
+        hosts/*/*)
+          hostdir="${f#hosts/}"
+          hostdir="${hostdir%%/*}"
+          while IFS= read -r t; do
+            [[ -n "$t" ]] && affected_hosts[$t]=1
+          done < <(host_targets_for_dir "$hostdir")
+          ;;
+        modules/platforms/*.nix)
+          platform="$(basename "$f" .nix)"
+          for h in "${all_hosts[@]}"; do
+            [[ "$h" == "${platform}-"* ]] && affected_hosts[$h]=1
+          done
+          ;;
+        modules/build-types/*.nix)
+          buildtype="$(basename "$f" .nix)"
+          for h in "${all_hosts[@]}"; do
+            [[ "$h" == *"-${buildtype}" ]] && affected_hosts[$h]=1
+          done
+          ;;
+        modules/installer/*)
+          # iso.nix (imported by both the "installer" nixosConfigurations
+          # target and netbootSystem, which backs packages.pxe) pulls in
+          # common.nix, so a common.nix change reaches all three.
+          affected_hosts[installer]=1
+          eval_packages=true
+          ;;
+        modules/pxe-boot/*)
+          # stage-installer-artifacts.nix is imported by
+          # modules/build-types/pxe-boot.nix only -- same blast radius as a
+          # build-types/*.nix change, not a packages one.
+          for h in "${all_hosts[@]}"; do
+            [[ "$h" == *"-pxe-boot" ]] && affected_hosts[$h]=1
+          done
+          ;;
+        modules/*)
+          full_fallback=true
+          ;;
+      esac
+    done
+  fi
+
+  if $full_fallback; then
+    echo
+    echo "Changed files affect shared config; falling back to evaluating every host/package."
+    for h in "${all_hosts[@]}"; do affected_hosts[$h]=1; done
+    eval_packages=true
+  fi
+fi
+
+mapfile -t hosts < <(for h in "${!affected_hosts[@]}"; do echo "$h"; done | sort)

 echo
-echo "Evaluating host toplevel derivations..."
-for host in $hosts; do
+if [[ ${#hosts[@]} -eq 0 ]]; then
+  echo "No hosts affected by changed files; skipping host eval."
+else
+  echo "Evaluating host toplevel derivations (${scope_desc})..."
+  for host in "${hosts[@]}"; do
    echo "==> $host"
    nix eval --raw "${NIX_EVAL_FLAGS[@]}" ".#nixosConfigurations.${host}.config.system.build.toplevel.drvPath"

@@ -63,19 +253,23 @@ for host in $hosts; do
        ;;
    esac
  done
+fi

 echo
+if ! $eval_packages; then
+  echo "No packages affected by changed files; skipping package eval."
+else
  echo "Evaluating buildable packages..."
-packages="$(nix eval --json "${NIX_EVAL_FLAGS[@]}" .#packages.x86_64-linux --apply builtins.attrNames | jq -r '.[]')"
-for pkg in $packages; do
+  for pkg in "${all_packages[@]}"; do
    echo "==> packages.x86_64-linux.${pkg}"
    nix eval --raw "${NIX_EVAL_FLAGS[@]}" ".#packages.x86_64-linux.${pkg}"
  done
+fi

-if [[ "$MODE" == "dry-run" ]]; then
+if $dry_run; then
  echo
-  echo "Running dry-run builds for all hosts. This will not create result symlinks."
-  for host in $hosts; do
+  echo "Running dry-run builds for the active scope. This will not create result symlinks."
+  for host in "${hosts[@]:-}"; do
    echo "==> Dry-run build: $host"
    nix build --dry-run --no-link "${NIX_EVAL_FLAGS[@]}" ".#nixosConfigurations.${host}.config.system.build.toplevel"

@@ -91,13 +285,15 @@ if [[ "$MODE" == "dry-run" ]]; then
    esac
  done

+  if $eval_packages; then
    echo
-  echo "Running dry-run builds for all packages."
-  for pkg in $packages; do
+    echo "Running dry-run builds for packages."
+    for pkg in "${all_packages[@]}"; do
      echo "==> Dry-run build: packages.x86_64-linux.${pkg}"
      nix build --dry-run --no-link "${NIX_EVAL_FLAGS[@]}" ".#packages.x86_64-linux.${pkg}"
    done
  fi
+fi

 echo
 echo "Maintenance checks complete."
@@ -41,6 +41,17 @@ warn-dirty = false
 build-users-group = nixbld
 EOF

+    # The official installer's single-user root path still shells out to
+    # `sudo` to create /nix even though it already knows it's running as
+    # root -- confirmed live against a sudo-less minimal Debian/Proxmox
+    # node, where it fails with "sudo: not found" and prints this exact
+    # mkdir/chown as the manual fix. Pre-create it so that branch of the
+    # installer is skipped entirely.
+    if [ ! -d /nix ]; then
+      mkdir -m 0755 /nix
+      chown root /nix
+    fi
+
    sh <(curl -L https://nixos.org/nix/install) --no-daemon
  else
    sh <(curl -L https://nixos.org/nix/install) --no-daemon
@@ -71,13 +82,6 @@ if ! command -v jq >/dev/null 2>&1; then
 fi

 echo "Available NixOS hosts:"
-hosts="$(list_flake_targets .)"
-echo "$hosts"
+list_flake_targets .

-echo "Evaluating all host toplevel derivations..."
-for host in $hosts; do
-  echo "==> Evaluating $host"
-  nix eval --raw "${NIX_EVAL_FLAGS[@]}" ".#nixosConfigurations.${host}.config.system.build.toplevel.drvPath"
-done
-
-echo "Codex setup complete."
+echo "Codex setup complete. Run bash scripts/codex-maintenance.sh to validate changes."
@@ -3,10 +3,10 @@
 # second copy of these values in every script:
 #   source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/env.sh"
 # Every variable can still be overridden per-invocation via the
-# environment (e.g. PROXMOX_STORAGE=tank-nvme ./scripts/create-proxmox-resource.sh ...)
+# environment (e.g. PROXMOX_STORAGE=tank-nvme ./scripts/proxmox/create-proxmox-resource.sh ...)
 # since each one only sets a default if unset.

-# SSH-reachable Proxmox node that scripts/create-proxmox-resource.sh runs
+# SSH-reachable Proxmox node that scripts/proxmox/create-proxmox-resource.sh runs
 # pct/qm on. Matches the Proxmox web UI hostname already used in
 # hosts/nixos/home.nix's desktop shortcuts (pve.<homeDomain> from
 # variables.nix) -- change this if that's not actually reachable over SSH,
@@ -15,7 +15,7 @@
 : "${PROXMOX_SSH_USER:=root}"

 # Where this flake repo lives on the Proxmox node itself.
-# scripts/create-proxmox-resource.sh builds images directly on the node
+# scripts/proxmox/create-proxmox-resource.sh builds images directly on the node
 # instead of transferring them over the network -- it clones the repo here
 # (from this checkout's own `origin` remote) the first time it doesn't
 # find it, installing build tooling via scripts/codex-setup.sh, then
@@ -53,7 +53,15 @@
 # crash-loops on a denied `/run/credentials/*` mount every ~3s (visible
 # as garbage on the console) and core services like nsncd fail the same
 # way on userns_create; system.build.tarball never finishes activating.
-: "${PROXMOX_DEFAULT_LXC_FEATURES:=nesting=1,keyctl=1}"
+#
+# mount=nfs;nfs4: without it, AppArmor blanket-denies the `nfs`/
+# `rpc_pipefs` mount syscalls any NFS client share needs -- confirmed
+# live on lxc-docker (which mounts several, see modules/docker/mount-data.nix
+# and modules/raspi/mount-data.nix): `mount: /var/lib/nfs/rpc_pipefs:
+# permission denied`. Harmless to grant on lxc targets that don't mount
+# NFS at all -- it only widens what the container is *allowed* to mount,
+# nothing here forces a mount to happen.
+: "${PROXMOX_DEFAULT_LXC_FEATURES:=nesting=1,keyctl=1,mount=nfs;nfs4}"

 export PROXMOX_HOST PROXMOX_SSH_USER PROXMOX_STORAGE PROXMOX_ISO_STORAGE \
  PROXMOX_BRIDGE PROXMOX_DEFAULT_CORES PROXMOX_DEFAULT_MEMORY_MB \
@@ -0,0 +1,21 @@
+#!/usr/bin/env bash
+# Shared "type X to confirm" prompt for scripts/proxmox/create-proxmox-resource.sh
+# (--modify, and replacing an existing --allow-duplicate-host resource) and
+# scripts/secrets/sync-host-keys.sh (--regenerate-all-keys) -- three destructive
+# confirmations that all work the same way (echo the expected value back
+# exactly), kept in one place so the prompt/comparison logic can't drift.
+# Source alongside env.sh:
+#   source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/lib/confirm.sh"
+#
+# Deliberately does NOT print anything on mismatch or decide exit-vs-return
+# -- callers vary on both (a top-level script exits, a subcommand function
+# returns; wording differs too), so that stays at the call site.
+
+# confirm_typed <expected> <prompt>
+# Prints <prompt> via `read -rp`, then reports (via exit status) whether the
+# typed input matched <expected> exactly.
+confirm_typed() {
+  local expected="$1" prompt="$2" input
+  read -rp "$prompt" input
+  [[ "$input" == "$expected" ]]
+}
@@ -0,0 +1,52 @@
+#!/usr/bin/env bash
+# Shared sops/age helpers for scripts/secrets/backup-admin-key.sh,
+# scripts/secrets/rotate-admin-key.sh, and scripts/secrets/sync-host-keys.sh -- all three
+# derive an age public key from a private identity file the same way, two
+# of them resolve the same sops/age default key-file path, and two of them
+# run `sops updatekeys` the same way. Kept in one place so they can't drift
+# apart. Source alongside env.sh:
+#   source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/lib/sops-age.sh"
+#
+# Uses NIX_OPTS (an array of extra `nix-shell` options -- see env.sh's
+# nix_extra_opts) if the caller has already set it, same convention as
+# lib/ssh-host-keys.sh. Falls back to no extra options if the caller never
+# sourced env.sh.
+if ! declare -p NIX_OPTS >/dev/null 2>&1; then
+  declare -a NIX_OPTS=()
+fi
+
+# sops/age's own default identity-file resolution order, minus $SOPS_AGE_KEY
+# itself (an inline identity, not a path -- callers that accept it check it
+# separately, before falling back to this).
+: "${DEFAULT_SOPS_AGE_KEY_FILE:=${SOPS_AGE_KEY_FILE:-${XDG_CONFIG_HOME:-$HOME/.config}/sops/age/keys.txt}}"
+
+# age_pubkey_from_identity_file <identity-file>
+# Prints the age public key for a private identity file (age-keygen -y).
+age_pubkey_from_identity_file() {
+  local identity_file="$1"
+  nix-shell "${NIX_OPTS[@]}" -p age --run "age-keygen -y '${identity_file}'"
+}
+
+# sops_yaml_admin_pubkey <sops-yaml-path>
+# Prints .sops.yaml's current &admin age public key, or empty (not an error
+# under set -e) if no such anchor line exists -- callers that need to treat
+# "missing" as fatal check for an empty result themselves.
+sops_yaml_admin_pubkey() {
+  local sops_yaml="$1"
+  grep -E '^  - &admin age1' "$sops_yaml" 2>/dev/null | awk '{print $NF}' || true
+}
+
+# sops_updatekeys <secrets-file> [key-file]
+# Re-encrypts <secrets-file> for .sops.yaml's current recipient set. If
+# <key-file> is given, decrypts with that identity (SOPS_AGE_KEY_FILE)
+# instead of whatever's ambient -- needed when the ambient default key
+# doesn't match yet (e.g. mid-rotation, decrypting with the outgoing key).
+sops_updatekeys() {
+  local secrets_file="$1" key_file="${2:-}"
+  if [[ -n "$key_file" ]]; then
+    SOPS_AGE_KEY_FILE="$key_file" nix-shell "${NIX_OPTS[@]}" -p sops --run \
+      "sops updatekeys --yes '${secrets_file}'"
+  else
+    nix-shell "${NIX_OPTS[@]}" -p sops --run "sops updatekeys --yes '${secrets_file}'"
+  fi
+}
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
-# Shared SSH-host-key / age-conversion helpers for scripts/sync-host-keys.sh
-# and scripts/prepare-host-key.sh -- both generate the same kind of key
+# Shared SSH-host-key / age-conversion helpers for scripts/secrets/sync-host-keys.sh
+# and scripts/secrets/prepare-host-key.sh -- both generate the same kind of key
 # (ed25519, no passphrase, the sops-nix age-derivation input) and convert it
 # to an age recipient the same way; kept in one place so the two can't
 # drift apart.
@@ -0,0 +1,172 @@
+#!/usr/bin/env bash
+# Points a non-NixOS Debian machine's Nix install at nix-cache: adds it as
+# a substituter (with cache.nixos.org kept as fallback) and, once the
+# remote-builder private key is installed, as a distributed-build machine
+# too.
+#
+# This is the non-NixOS equivalent of modules/nix-cache/client.nix +
+# modules/nix-cache/remote-builder-client.nix -- those two only apply to
+# hosts built from this flake. A plain Debian box with Nix installed
+# (single- or multi-user install, nix-daemon running) has no NixOS module
+# system to pick that config up, so this edits /etc/nix/nix.conf by hand
+# instead. Run this ON the target Debian machine, as root.
+#
+# The values below mirror variables.nix / modules/nix-cache/client.nix in
+# this repo -- update both if nix-cache is ever rebuilt with a new host
+# key or the cache signing key is rotated (see docs/nix-cache.md).
+#
+# Usage:
+#   sudo ./configure-nix-cache-client.sh [--dry-run] [--no-remote-builder] [--no-restart]
+#
+# Env overrides (defaults match variables.nix):
+#   NIX_CACHE_HOST, NIX_CACHE_HOST_KEY, REMOTE_BUILDER_USER, REMOTE_BUILDER_KEY
+
+set -euo pipefail
+
+: "${NIX_CACHE_HOST:=nix-cache}"
+: "${NIX_CACHE_HOST_KEY:=ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHrMKZlIGUd3pH9G3AqbsruqUGjxIXMAZw52u9MwiBCn lxc-nix-cache}"
+: "${REMOTE_BUILDER_USER:=nixremote}"
+: "${REMOTE_BUILDER_KEY:=/root/.ssh/nixremote}"
+
+CACHE_PUB_KEY="cache.local-1:usoWYanY3Kpq2+kDIS2nhWoLZiRxanmdysdzqCFBHW4="
+FALLBACK_URL="https://cache.nixos.org/"
+FALLBACK_PUB_KEY="cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY="
+
+NIX_CONF="/etc/nix/nix.conf"
+KNOWN_HOSTS="/etc/ssh/ssh_known_hosts"
+MARKER_BEGIN="# BEGIN nix-cache client config (configure-nix-cache-client.sh)"
+MARKER_END="# END nix-cache client config"
+
+dry_run=0
+with_remote_builder=1
+restart_daemon=1
+
+for arg in "$@"; do
+  case "$arg" in
+    --dry-run) dry_run=1 ;;
+    --no-remote-builder) with_remote_builder=0 ;;
+    --no-restart) restart_daemon=0 ;;
+    -h|--help)
+      sed -n '2,20p' "$0"
+      exit 0
+      ;;
+    *)
+      echo "ERROR: unknown argument: $arg" >&2
+      exit 1
+      ;;
+  esac
+done
+
+if [[ "$dry_run" -eq 0 && "$EUID" -ne 0 ]]; then
+  echo "ERROR: must run as root (writes $NIX_CONF and, unless --no-remote-builder, $KNOWN_HOSTS)." >&2
+  exit 1
+fi
+
+if ! command -v nix >/dev/null 2>&1; then
+  echo "ERROR: no 'nix' binary on PATH -- install the Nix package manager first." >&2
+  exit 1
+fi
+
+if [[ ! -f "$NIX_CONF" ]]; then
+  echo "ERROR: $NIX_CONF not found -- expected an existing multi-user Nix install." >&2
+  exit 1
+fi
+
+builder_line=""
+if [[ "$with_remote_builder" -eq 1 ]]; then
+  if [[ -f "$REMOTE_BUILDER_KEY" ]]; then
+    case "$(uname -m)" in
+      x86_64) nix_system="x86_64-linux" ;;
+      aarch64) nix_system="aarch64-linux" ;;
+      *)
+        echo "WARNING: unrecognized architecture '$(uname -m)' -- skipping remote builder, keeping substituter config." >&2
+        with_remote_builder=0
+        ;;
+    esac
+    if [[ "$with_remote_builder" -eq 1 ]]; then
+      builder_line="builders = ssh://${REMOTE_BUILDER_USER}@${NIX_CACHE_HOST} ${nix_system} ${REMOTE_BUILDER_KEY} 4 2 big-parallel,kvm,nixos-test,benchmark"
+    fi
+  else
+    echo "WARNING: $REMOTE_BUILDER_KEY not found -- skipping remote builder config (substituter still configured)." >&2
+    echo "  See docs/nix-cache.md 'Remote builder SSH keys' for how to install it, then re-run this script." >&2
+    with_remote_builder=0
+  fi
+fi
+
+block="$(cat <<EOF
+$MARKER_BEGIN
+extra-substituters = http://${NIX_CACHE_HOST} ${FALLBACK_URL}
+extra-trusted-public-keys = ${CACHE_PUB_KEY} ${FALLBACK_PUB_KEY}
+EOF
+)"
+if [[ "$with_remote_builder" -eq 1 ]]; then
+  block="${block}
+builders-use-substitutes = true
+${builder_line}"
+fi
+block="${block}
+$MARKER_END"
+
+echo "== nix.conf block to install =="
+echo "$block"
+echo "================================"
+
+if [[ "$dry_run" -eq 1 ]]; then
+  echo "(--dry-run: not writing $NIX_CONF)"
+else
+  tmp_conf="$(mktemp)"
+  trap 'rm -f "$tmp_conf"' EXIT
+
+  if grep -qF "$MARKER_BEGIN" "$NIX_CONF"; then
+    awk -v begin="$MARKER_BEGIN" -v end="$MARKER_END" -v block="$block" '
+      $0 == begin { print block; skip = 1; next }
+      $0 == end   { skip = 0; next }
+      skip        { next }
+      { print }
+    ' "$NIX_CONF" > "$tmp_conf"
+  else
+    cp "$NIX_CONF" "$tmp_conf"
+    printf '\n%s\n' "$block" >> "$tmp_conf"
+  fi
+
+  cp "$NIX_CONF" "${NIX_CONF}.bak.$(date +%Y%m%d%H%M%S)"
+  install -m 0644 "$tmp_conf" "$NIX_CONF"
+  echo "Updated $NIX_CONF (backup saved alongside it)."
+fi
+
+if [[ "$with_remote_builder" -eq 1 ]]; then
+  known_hosts_line="${NIX_CACHE_HOST} ${NIX_CACHE_HOST_KEY}"
+  if [[ "$dry_run" -eq 1 ]]; then
+    echo "(--dry-run: would ensure this line is present in $KNOWN_HOSTS)"
+    echo "  $known_hosts_line"
+  else
+    mkdir -p "$(dirname "$KNOWN_HOSTS")"
+    touch "$KNOWN_HOSTS"
+    if ! grep -qF "$known_hosts_line" "$KNOWN_HOSTS" 2>/dev/null; then
+      echo "$known_hosts_line" >> "$KNOWN_HOSTS"
+      echo "Added nix-cache's SSH host key to $KNOWN_HOSTS."
+    fi
+  fi
+fi
+
+if [[ "$dry_run" -eq 0 && "$restart_daemon" -eq 1 ]]; then
+  if command -v systemctl >/dev/null 2>&1 && systemctl is-active --quiet nix-daemon 2>/dev/null; then
+    systemctl restart nix-daemon
+    echo "Restarted nix-daemon to pick up the new config."
+  else
+    echo "nix-daemon not managed by systemd (or not running) -- restart it manually to pick up the new config."
+  fi
+fi
+
+cat <<EOF
+
+Done. Verify with:
+  curl http://${NIX_CACHE_HOST}/nix-cache-info
+  nix show-config | grep -E 'substituters|trusted-public-keys|builders'
+EOF
+if [[ "$with_remote_builder" -eq 1 ]]; then
+  cat <<EOF
+  ssh -i ${REMOTE_BUILDER_KEY} ${REMOTE_BUILDER_USER}@${NIX_CACHE_HOST} nix-store --version
+  nix build nixpkgs#hello -L
+EOF
+fi
@@ -13,32 +13,43 @@
 # alone wouldn't carry it) before building.
 #
 # Usage:
-#   scripts/create-proxmox-resource.sh --type lxc|vm --host <name> [options]
-#   scripts/create-proxmox-resource.sh --type lxc|vm --list
-#   scripts/create-proxmox-resource.sh --modify --vmid <n> [--cores N] [--memory MB] [--grow-disk GB]
+#   scripts/proxmox/create-proxmox-resource.sh --type lxc|vm --host <name> [options]
+#   scripts/proxmox/create-proxmox-resource.sh --type lxc|vm --list
+#   scripts/proxmox/create-proxmox-resource.sh --modify --vmid <n> [--cores N] [--memory MB] [--grow-disk GB]
 #
 # SAFETY:
 #   - The default (create) mode only ever creates a NEW resource -- it
 #     refuses to run if the target VMID already exists on the node, or if
 #     a VM/CT identified as --host already exists under any other VMID
 #     (checked live against the node; --allow-duplicate-host overrides).
+#   - --allow-duplicate-host distinguishes an exact match (same --type
+#     *and* --host, e.g. re-running --type lxc --host docker while an
+#     lxc-docker container already exists -- almost always a redeploy of
+#     the same target to pick up a rebuilt image) from a cross-type match
+#     (a different platform sharing the same host identity, e.g. a
+#     proxmox-docker VM coexisting with lxc-docker). Only the exact match
+#     is destroyed and replaced, after typing the hostname back to
+#     confirm (outside --dry-run) -- a cross-type match is always left
+#     untouched, matching-or-not.
 #   - --modify only ever touches a resource you name explicitly via
 #     --vmid, shows exactly what will change first, and (outside
 #     --dry-run) always requires typing that VMID back to confirm before
 #     anything is sent to the node. There is no bulk/implicit modify.
-#   - Neither mode can start/stop/delete a resource. Not implemented on
-#     purpose -- ask before adding it.
+#   - Outside of --allow-duplicate-host's exact-match replace above,
+#     neither mode can start/stop/delete a resource.
 #
 # See --help for the full option list.
 set -euo pipefail

-repo_root="$(cd "$(dirname "$0")/.." && pwd)"
-# shellcheck source=env.sh
+repo_root="$(cd "$(dirname "$0")/../.." && pwd)"
+# shellcheck source=../env.sh
 source "${repo_root}/scripts/env.sh"
-# shellcheck source=lib/nix-eval.sh
+# shellcheck source=../lib/nix-eval.sh
 source "${repo_root}/scripts/lib/nix-eval.sh"
+# shellcheck source=../lib/confirm.sh
+source "${repo_root}/scripts/lib/confirm.sh"

-sync_keys="${repo_root}/scripts/sync-host-keys.sh"
+sync_keys="${repo_root}/scripts/secrets/sync-host-keys.sh"

 usage() {
  cat <<EOF
@@ -77,7 +88,13 @@ Create mode (default):
                                already exists on the node (checked live via
                                qm/pct, not any file in this repo) --
                                otherwise refused, since it'd share that
-                                host's hostName/hostId.
+                                host's hostName/hostId. An existing resource
+                                of this *same* --type (e.g. re-running --type
+                                lxc --host docker over an existing lxc-docker)
+                                is destroyed and replaced, after confirming --
+                                a different --type sharing the same --host
+                                (e.g. a proxmox-docker VM) is always left
+                                untouched.

 Modify mode (reconfigure an EXISTING resource -- requires --modify):
  --modify                Switch to modify mode.
@@ -215,8 +232,7 @@ cmd_modify() {
  fi

  echo
-  read -rp "Type the VMID (${vmid}) to confirm these changes: " confirm
-  if [[ "$confirm" != "$vmid" ]]; then
+  if ! confirm_typed "$vmid" "Type the VMID (${vmid}) to confirm these changes: "; then
    echo "Cancelled -- input didn't match ${vmid}."
    exit 1
  fi
@@ -319,12 +335,13 @@ fi
 # this script itself always uses unless --name is overridden) -- a guest
 # manually renamed on the node afterwards wouldn't match, but nothing here
 # creates guests that way.
-if [[ "$allow_duplicate_host" -eq 1 ]]; then
-  echo
-  echo "--allow-duplicate-host: skipping the check for an existing '${host}' on ${node}."
-elif [[ "$dry_run" -eq 1 ]]; then
+if [[ "$dry_run" -eq 1 ]]; then
  echo
  echo "[dry-run] would check ${node} for an existing VM/CT identified as '${host}'"
+  if [[ "$allow_duplicate_host" -eq 1 ]]; then
+    echo "[dry-run] --allow-duplicate-host: an existing ${type} named '${host}' would be" \
+      "destroyed and replaced; a different-type match would be left untouched"
+  fi
 else
  echo
  echo "==> Checking ${node} for an existing VM/CT identified as '${host}'..."
@@ -349,17 +366,89 @@ REMOTE_SCRIPT
    echo "check entirely)." >&2
    exit 1
  fi
+
+  # Split into "exact" (same resource kind as --type -- i.e. literally this
+  # same host+platform combo already exists, almost always a redeploy of
+  # the same target to test a rebuilt image) vs "cross-type" (a different
+  # platform sharing this host identity, e.g. a stopped proxmox-docker VM
+  # coexisting with an lxc-docker container -- a deliberate, valid setup
+  # this script has never managed and still won't). Read via a herestring
+  # (not a pipe) so the appends below survive outside the loop.
+  this_kind="$type"
+  exact_matches=""
+  cross_matches=""
  if [[ -n "$existing" ]]; then
-    echo "ERROR: '${host}' already exists on ${node}:" >&2
-    echo "$existing" | while read -r kind id n; do
+    while read -r kind id n; do
+      [[ -z "$kind" ]] && continue
+      if [[ "$kind" == "$this_kind" ]]; then
+        exact_matches+="${kind} ${id} ${n}"$'\n'
+      else
+        cross_matches+="${kind} ${id} ${n}"$'\n'
+      fi
+    done <<<"$existing"
+  fi
+
+  if [[ -n "$exact_matches" && "$allow_duplicate_host" -ne 1 ]]; then
+    echo "ERROR: '${host}' already exists on ${node} as this same resource type:" >&2
+    echo "$exact_matches" | while read -r kind id n; do
+      [[ -z "$kind" ]] && continue
+      echo "  - ${kind} VMID ${id} (${n})" >&2
+    done
+    echo "Refusing to create a second ${this_kind} sharing this identity. Pass" >&2
+    echo "--allow-duplicate-host to destroy it and create a fresh one in its place" >&2
+    echo "(after confirming), or use --modify to reconfigure the existing one instead." >&2
+    exit 1
+  fi
+
+  if [[ -n "$cross_matches" && "$allow_duplicate_host" -ne 1 ]]; then
+    echo "ERROR: '${host}' already exists on ${node} as a different resource type:" >&2
+    echo "$cross_matches" | while read -r kind id n; do
+      [[ -z "$kind" ]] && continue
      echo "  - ${kind} VMID ${id} (${n})" >&2
    done
    echo "Refusing to create a second resource sharing this identity. Pass" >&2
    echo "--allow-duplicate-host to create one anyway (it gets its own distinct" >&2
-    echo "sops key and VMID -- the existing resource(s) above are left untouched)," >&2
+    echo "sops key and VMID -- the existing resource above is left untouched)," >&2
    echo "or use --modify to reconfigure the existing one instead." >&2
    exit 1
  fi
+
+  if [[ -n "$cross_matches" ]]; then
+    echo "--allow-duplicate-host: '${host}' also exists on ${node} as a different resource" \
+      "type -- leaving it untouched:"
+    echo "$cross_matches" | while read -r kind id n; do
+      [[ -z "$kind" ]] && continue
+      echo "  - ${kind} VMID ${id} (${n})"
+    done
+  fi
+
+  if [[ -n "$exact_matches" ]]; then
+    echo "--allow-duplicate-host: '${host}' already exists on ${node} as this same resource" \
+      "type -- it will be destroyed and replaced:"
+    echo "$exact_matches" | while read -r kind id n; do
+      [[ -z "$kind" ]] && continue
+      echo "  - ${kind} VMID ${id} (${n})"
+    done
+    echo
+    if ! confirm_typed "$host" "Type the hostname (${host}) to confirm destroying the above and replacing it: "; then
+      echo "Cancelled -- input didn't match ${host}." >&2
+      exit 1
+    fi
+    echo "$exact_matches" | while read -r kind id n; do
+      [[ -z "$kind" ]] && continue
+      echo "==> Destroying ${kind} VMID ${id} (${n})..."
+      if [[ "$kind" == "vm" ]]; then
+        # qm destroy has no --force to stop-then-destroy in one call (pct's
+        # does) -- stop explicitly first if it's running.
+        if ssh "$ssh_target" "qm status ${id}" 2>/dev/null | grep -q running; then
+          ssh "$ssh_target" "qm stop ${id}"
+        fi
+        ssh "$ssh_target" "qm destroy ${id} --purge 1"
+      else
+        ssh "$ssh_target" "pct destroy ${id} --force 1 --purge 1"
+      fi
+    done
+  fi
 fi

 echo "Target: ${flake_target} (host=${host}, type=${type}) -> Proxmox resource '${name}'"
@@ -435,7 +524,7 @@ ensure_remote_repo() {
  echo
  echo "==> Ensuring ${remote_repo_dir} exists and is current on ${node}..."
  if [[ "$dry_run" -eq 1 ]]; then
-    echo "[dry-run] would ensure ${remote_repo_dir} exists on ${node} (clone + scripts/codex-setup.sh if missing, git pull if present)"
+    echo "[dry-run] would ensure ${remote_repo_dir} exists on ${node} (clone if missing, git pull if present), and would verify/bootstrap build tooling there (scripts/codex-setup.sh) if \`nix\` isn't already on PATH -- and if that bootstrap actually ran, would also configure ${node} as a nix-cache client (scripts/proxmox/configure-nix-cache-client.sh)"
    return
  fi

@@ -453,8 +542,50 @@ ensure_remote_repo() {
    fi
    echo "Not present -- cloning from ${origin_url}..."
    ssh "$ssh_target" "git clone '${origin_url}' '${remote_repo_dir}'"
+  fi
+
+  # Trivial check, run every time (not just right after a fresh clone) --
+  # confirmed live: a first bootstrap can clone the repo successfully and
+  # still leave the node without a working `nix` (e.g. the node had no
+  # `sudo`, which the Nix installer's root path depends on -- see the fix
+  # in scripts/codex-setup.sh), and a later run with the repo already
+  # present would otherwise never retry it. Sources
+  # scripts/lib/nix-bootstrap.sh's ensure_nix_profile first -- a
+  # single-user Nix install typically only gets sourced into login shells,
+  # and ssh's non-interactive command execution is neither, so a
+  # freshly-installed `nix` still wouldn't be on PATH here without it.
+  #
+  # Just `nix` today -- the only thing the remote build commands below
+  # actually invoke -- but a list (not a single hardcoded check) so a
+  # future remote step needing another tool can add itself here instead of
+  # growing a parallel check.
+  local remote_required_cmds=(nix)
+  local tooling_check_cmd="cd '${remote_repo_dir}' && . scripts/lib/nix-bootstrap.sh && ensure_nix_profile"
+  local cmd
+  for cmd in "${remote_required_cmds[@]}"; do
+    tooling_check_cmd="${tooling_check_cmd} && command -v ${cmd}"
+  done
+
+  if ssh "$ssh_target" "$tooling_check_cmd" >/dev/null 2>&1; then
+    echo "Build tooling already present on ${node}."
+  else
    echo "==> Bootstrapping build tooling on ${node} (scripts/codex-setup.sh)..."
    ssh "$ssh_target" "cd '${remote_repo_dir}' && bash scripts/codex-setup.sh"
+
+    # Only on this first-time bootstrap, not every run -- a node that
+    # already has tooling either already went through this once, or had
+    # it configured some other way, and re-running is harmless but
+    # pointless. Non-fatal: this only makes the node's own builds faster
+    # (substitute from nix-cache instead of building from source) and
+    # offloadable to it as a remote builder -- worth trying, not worth
+    # aborting the image build over if nix-cache happens to be down right
+    # now. Needs ensure_nix_profile first, same as the tooling_check_cmd
+    # above -- ssh's non-interactive command execution won't have picked
+    # up a freshly single-user-installed `nix` otherwise.
+    echo "==> Configuring ${node} as a nix-cache substituter/remote-builder client..."
+    if ! ssh "$ssh_target" "cd '${remote_repo_dir}' && . scripts/lib/nix-bootstrap.sh && ensure_nix_profile && bash scripts/proxmox/configure-nix-cache-client.sh"; then
+      echo "WARNING: configure-nix-cache-client.sh failed on ${node} -- continuing without it (${node} will build from source / against cache.nixos.org only)." >&2
+    fi
  fi
 }

@@ -536,6 +667,11 @@ repo_dir="$1"; target="$2"; dest_dir="$3"; dest_name="$4"; nix_extra_opts_str="$
 declare -a NIX_OPTS=()
 [[ -n "$nix_extra_opts_str" ]] && eval "NIX_OPTS=(${nix_extra_opts_str})"
 cd "$repo_dir"
+# A single-user Nix install only gets sourced into login shells; this ssh
+# session is neither, so `nix` wouldn't otherwise be on PATH here even
+# right after a successful install.
+. scripts/lib/nix-bootstrap.sh
+ensure_nix_profile
 NIXOS_HOST_KEYS_DIR="$(pwd)/host-keys" nix build --impure \
  --no-use-registries --no-accept-flake-config "${NIX_OPTS[@]}" \
  ".#nixosConfigurations.${target}.config.system.build.tarball" \
@@ -583,6 +719,8 @@ repo_dir="$1"; target="$2"; dest_dir="$3"; dest_name="$4"; nix_extra_opts_str="$
 declare -a NIX_OPTS=()
 [[ -n "$nix_extra_opts_str" ]] && eval "NIX_OPTS=(${nix_extra_opts_str})"
 cd "$repo_dir"
+. scripts/lib/nix-bootstrap.sh
+ensure_nix_profile
 nix build --no-use-registries --no-accept-flake-config "${NIX_OPTS[@]}" \
  ".#nixosConfigurations.${target}.config.system.build.diskoImagesScript" \
  --out-link "result-${target}"
@@ -642,7 +780,17 @@ if [[ "$type" == "lxc" ]]; then
  # systemd routinely uses (even plain getty units), and every getty
  # crash-loops on a denied mount every ~3s (visible as garbage on the
  # console) while core services like nsncd fail the same way.
-  create_cmd="pct create ${vmid} ${iso_storage}:vztmpl/${remote_filename} --unprivileged 1 --features ${PROXMOX_DEFAULT_LXC_FEATURES} --rootfs ${storage}:${local_disk_size} --hostname ${name} --cores ${cores} --memory ${memory} --swap ${local_swap} --net0 name=eth0,bridge=${bridge},ip=dhcp"
+  #
+  # ...,mount=nfs;nfs4: without it AppArmor blanket-denies the `nfs`/
+  # `rpc_pipefs` mount syscalls any NFS client share needs -- confirmed
+  # live on lxc-docker: `mount: /var/lib/nfs/rpc_pipefs: permission
+  # denied`. The value's `;` (Proxmox's own multi-fstype separator for
+  # this one feature, per PVE::LXC's use of PVE::ParseUtils::split_list)
+  # must stay single-quoted here: create_cmd is sent to `remote()`, which
+  # hands the whole string to `ssh` as a single command for the *remote*
+  # shell to parse -- unquoted, that `;` would be read as a remote
+  # command separator and silently truncate this into two commands.
+  create_cmd="pct create ${vmid} ${iso_storage}:vztmpl/${remote_filename} --unprivileged 1 --features '${PROXMOX_DEFAULT_LXC_FEATURES}' --rootfs ${storage}:${local_disk_size} --hostname ${name} --cores ${cores} --memory ${memory} --swap ${local_swap} --net0 name=eth0,bridge=${bridge},ip=dhcp"
  remote "$create_cmd"
  remote "pct start ${vmid}"
 else
@@ -6,7 +6,7 @@
 # copy is ever lost, or to run either script from a different machine.
 #
 # Usage:
-#   scripts/backup-admin-key.sh <dest-path> [--key-file <path>] [--force] [--dry-run]
+#   scripts/secrets/backup-admin-key.sh <dest-path> [--key-file <path>] [--force] [--dry-run]
 #
 # Source key resolution matches sops/age's own default order:
 #   $SOPS_AGE_KEY (inline identity text) if set, else
@@ -15,11 +15,13 @@
 #   ${XDG_CONFIG_HOME:-$HOME/.config}/sops/age/keys.txt
 set -euo pipefail

-repo_root="$(cd "$(dirname "$0")/.." && pwd)"
+repo_root="$(cd "$(dirname "$0")/../.." && pwd)"
 sops_yaml="${repo_root}/.sops.yaml"

-# shellcheck source=env.sh
+# shellcheck source=../env.sh
 source "${repo_root}/scripts/env.sh"
+# shellcheck source=../lib/sops-age.sh
+source "${repo_root}/scripts/lib/sops-age.sh"

 # Pin cwd for the same reason rotate-admin-key.sh does: age/sops calls
 # below should never depend on wherever the caller's shell happened to be.
@@ -43,7 +45,7 @@ EOF

 dry_run=0
 force=0
-key_file="${SOPS_AGE_KEY_FILE:-${XDG_CONFIG_HOME:-$HOME/.config}/sops/age/keys.txt}"
+key_file="$DEFAULT_SOPS_AGE_KEY_FILE"
 args=()

 while [[ $# -gt 0 ]]; do
@@ -103,13 +105,13 @@ scratch="$(mktemp)"
 trap 'rm -f "$scratch"' EXIT
 ( umask 077; printf '%s\n' "$src_content" > "$scratch" )

-src_pub="$(nix-shell "${NIX_OPTS[@]}" -p age --run "age-keygen -y '$scratch'")" || {
+src_pub="$(age_pubkey_from_identity_file "$scratch")" || {
  echo "ERROR: source doesn't look like a valid age identity (age-keygen -y failed)." >&2
  exit 1
 }
 echo "    public key: ${src_pub}"

-current_admin_pub="$(grep -E '^  - &admin age1' "$sops_yaml" 2>/dev/null | awk '{print $NF}' || true)"
+current_admin_pub="$(sops_yaml_admin_pubkey "$sops_yaml")"
 if [[ -n "$current_admin_pub" && "$current_admin_pub" != "$src_pub" ]]; then
  echo "NOTE: this key does not match .sops.yaml's current &admin entry (${current_admin_pub})."
  echo "      Backing it up anyway -- this script doesn't require it to be the admin key."
@@ -131,7 +133,7 @@ fi
 mkdir -p "$(dirname "$dest")"
 install -m 600 "$scratch" "$dest"

-dest_pub="$(nix-shell "${NIX_OPTS[@]}" -p age --run "age-keygen -y '$dest'")"
+dest_pub="$(age_pubkey_from_identity_file "$dest")"
 if [[ "$dest_pub" != "$src_pub" ]]; then
  echo "ERROR: ${dest} was written but its public key doesn't match the source -- investigate before relying on this backup." >&2
  exit 1
@@ -144,5 +146,5 @@ Done. Backed up to: ${dest}

 This is a private key -- store it somewhere offline/secure, not in this
 repo or anywhere it'd get committed. Restore it with:
-  scripts/rotate-admin-key.sh ${dest}
+  scripts/secrets/rotate-admin-key.sh ${dest}
 EOF
@@ -2,7 +2,7 @@
 # Generates a new machine's SSH host key by an arbitrary name, before it
 # necessarily has a flake target yet -- prints the .sops.yaml snippet to
 # add by hand. For any host that already has a flake target,
-# scripts/sync-host-keys.sh <target> does this same job plus the
+# scripts/secrets/sync-host-keys.sh <target> does this same job plus the
 # .sops.yaml/key_groups registration and re-encryption automatically; use
 # this script only to pre-generate a key ahead of adding the flake target
 # itself.
@@ -22,13 +22,13 @@
 # new machine.
 set -euo pipefail

-repo_root="$(cd "$(dirname "$0")/.." && pwd)"
-# shellcheck source=env.sh
+repo_root="$(cd "$(dirname "$0")/../.." && pwd)"
+# shellcheck source=../env.sh
 source "${repo_root}/scripts/env.sh"
-# shellcheck source=lib/ssh-host-keys.sh
+# shellcheck source=../lib/ssh-host-keys.sh
 source "${repo_root}/scripts/lib/ssh-host-keys.sh"

-hostname="${1:?usage: scripts/prepare-host-key.sh <hostname>}"
+hostname="${1:?usage: scripts/secrets/prepare-host-key.sh <hostname>}"
 sops_yaml="${repo_root}/.sops.yaml"

 if [[ ! -f "$sops_yaml" ]]; then
@@ -10,7 +10,7 @@
 # sync-host-keys.sh print when they bootstrap a brand-new, not-yet-trusted
 # age key on a machine that's never had admin access before:
 #
-#   scripts/rotate-admin-key.sh /path/to/backed-up/admin/keys.txt
+#   scripts/secrets/rotate-admin-key.sh /path/to/backed-up/admin/keys.txt
 #
 # The backup key's *public* key must match .sops.yaml's current &admin
 # entry -- this script verifies that by deriving it, it doesn't just trust
@@ -19,11 +19,13 @@
 # the common case is just pointing this at the restored backup.
 set -euo pipefail

-repo_root="$(cd "$(dirname "$0")/.." && pwd)"
+repo_root="$(cd "$(dirname "$0")/../.." && pwd)"
 sops_yaml="${repo_root}/.sops.yaml"

-# shellcheck source=env.sh
+# shellcheck source=../env.sh
 source "${repo_root}/scripts/env.sh"
+# shellcheck source=../lib/sops-age.sh
+source "${repo_root}/scripts/lib/sops-age.sh"

 # sops resolves .sops.yaml by walking up from the process's cwd, not from
 # the target file's own path -- if this script were invoked from somewhere
@@ -53,7 +55,7 @@ EOF
 }

 dry_run=0
-new_key_file="${SOPS_AGE_KEY_FILE:-${XDG_CONFIG_HOME:-$HOME/.config}/sops/age/keys.txt}"
+new_key_file="$DEFAULT_SOPS_AGE_KEY_FILE"
 args=()

 while [[ $# -gt 0 ]]; do
@@ -93,13 +95,9 @@ backup_key="${args[0]}"

 nix_extra_opts

-age_pub() {
-  nix-shell "${NIX_OPTS[@]}" -p age --run "age-keygen -y '$1'"
-}
-
 echo "==> Deriving public keys..."
-old_pub="$(age_pub "$backup_key")"
-new_pub="$(age_pub "$new_key_file")"
+old_pub="$(age_pubkey_from_identity_file "$backup_key")"
+new_pub="$(age_pubkey_from_identity_file "$new_key_file")"
 echo "    backup (old admin) key: ${old_pub}"
 echo "    new admin key:          ${new_pub}"

@@ -108,12 +106,11 @@ if [[ "$old_pub" == "$new_pub" ]]; then
  exit 1
 fi

-current_admin_line="$(grep -E '^  - &admin age1' "$sops_yaml" || true)"
-if [[ -z "$current_admin_line" ]]; then
+current_admin_pub="$(sops_yaml_admin_pubkey "$sops_yaml")"
+if [[ -z "$current_admin_pub" ]]; then
  echo "ERROR: couldn't find a '&admin age1...' line in ${sops_yaml}." >&2
  exit 1
 fi
-current_admin_pub="$(awk '{print $NF}' <<<"$current_admin_line")"

 if [[ "$current_admin_pub" != "$old_pub" ]]; then
  echo "ERROR: ${backup_key} doesn't match the current &admin key in .sops.yaml." >&2
@@ -129,9 +126,17 @@ if [[ "${#secrets_files[@]}" -eq 0 ]]; then
  exit 1
 fi

+# sops_can_decrypt <key-file> <secrets-file>: used both to confirm the
+# backup key still works before touching anything, and again after
+# rotation to confirm the new key does too.
+sops_can_decrypt() {
+  local key_file="$1" secrets_file="$2"
+  SOPS_AGE_KEY_FILE="$key_file" nix-shell "${NIX_OPTS[@]}" -p sops --run \
+    "sops -d '${secrets_file}'" >/dev/null
+}
+
 echo "==> Confirming the backup key can actually decrypt..."
-if ! SOPS_AGE_KEY_FILE="$backup_key" nix-shell "${NIX_OPTS[@]}" -p sops --run \
-  "sops -d '${secrets_files[0]}'" >/dev/null; then
+if ! sops_can_decrypt "$backup_key" "${secrets_files[0]}"; then
  echo "ERROR: backup key failed to decrypt $(basename "${secrets_files[0]}") -- aborting." >&2
  exit 1
 fi
@@ -162,14 +167,12 @@ echo "    Updated."
 echo "==> Re-encrypting secrets/*.yaml for the new recipient set..."
 for f in "${secrets_files[@]}"; do
  echo "==> $(basename "$f")"
-  SOPS_AGE_KEY_FILE="$backup_key" nix-shell "${NIX_OPTS[@]}" -p sops --run \
-    "sops updatekeys --yes '${f}'"
+  sops_updatekeys "$f" "$backup_key"
 done

 echo "==> Verifying the new key can decrypt everything..."
 for f in "${secrets_files[@]}"; do
-  if ! SOPS_AGE_KEY_FILE="$new_key_file" nix-shell "${NIX_OPTS[@]}" -p sops --run \
-    "sops -d '${f}'" >/dev/null; then
+  if ! sops_can_decrypt "$new_key_file" "$f"; then
    echo "ERROR: new key failed to decrypt $(basename "$f") after rotation -- investigate before committing." >&2
    exit 1
  fi
@@ -24,17 +24,21 @@
 # ever touches keys it itself manages.
 set -euo pipefail

-repo_root="$(cd "$(dirname "$0")/.." && pwd)"
+repo_root="$(cd "$(dirname "$0")/../.." && pwd)"
 sops_yaml="${repo_root}/.sops.yaml"
 keydir="${repo_root}/host-keys"
 editor="${repo_root}/scripts/lib/sync-host-keys-edit-sops.py"

-# shellcheck source=env.sh
+# shellcheck source=../env.sh
 source "${repo_root}/scripts/env.sh"
-# shellcheck source=lib/nix-eval.sh
+# shellcheck source=../lib/nix-eval.sh
 source "${repo_root}/scripts/lib/nix-eval.sh"
-# shellcheck source=lib/ssh-host-keys.sh
+# shellcheck source=../lib/ssh-host-keys.sh
 source "${repo_root}/scripts/lib/ssh-host-keys.sh"
+# shellcheck source=../lib/sops-age.sh
+source "${repo_root}/scripts/lib/sops-age.sh"
+# shellcheck source=../lib/confirm.sh
+source "${repo_root}/scripts/lib/confirm.sh"

 mkdir -p "$keydir"

@@ -78,7 +82,7 @@ ensure_admin_decrypt_key() {
    return
  fi

-  local key_file="${SOPS_AGE_KEY_FILE:-${XDG_CONFIG_HOME:-$HOME/.config}/sops/age/keys.txt}"
+  local key_file="$DEFAULT_SOPS_AGE_KEY_FILE"

  if [[ -s "$key_file" ]]; then
    echo "Found existing sops age key at ${key_file}."
@@ -97,7 +101,7 @@ ensure_admin_decrypt_key() {
  mkdir -p "$(dirname "$key_file")"
  nix-shell "${NIX_OPTS[@]}" -p age --run "age-keygen -o '${key_file}'" 2>&1 | grep -v "^Public key:" || true
  local new_pub
-  new_pub="$(nix-shell "${NIX_OPTS[@]}" -p age --run "age-keygen -y '${key_file}'")"
+  new_pub="$(age_pubkey_from_identity_file "$key_file")"

  cat <<EOF

@@ -239,7 +243,7 @@ apply_edit_plan() {
      while IFS= read -r basename; do
        [[ -z "$basename" ]] && continue
        echo "==> secrets/${basename}"
-        nix-shell "${NIX_OPTS[@]}" -p sops --run "sops updatekeys --yes '${repo_root}/secrets/${basename}'"
+        sops_updatekeys "${repo_root}/secrets/${basename}"
      done <<<"$changed"
    fi
  fi
@@ -359,8 +363,7 @@ cmd_regenerate_all() {
  echo "image/tarball before it can decrypt secrets again."

  if [[ "$dry_run" -ne 1 ]]; then
-    read -rp "Type REGENERATE to confirm: " confirm
-    if [[ "$confirm" != "REGENERATE" ]]; then
+    if ! confirm_typed "REGENERATE" "Type REGENERATE to confirm: "; then
      echo "Cancelled."
      return
    fi
@@ -4,31 +4,31 @@ sops:
    age:
        - enc: |
            -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBmOHlPcGZvN3o5aEZCcFdz
-            WlYydFdoQ0ZHbElMdTRkZWljMmoxTHZqNjJZCnE3cmgzKzZya3FROEczbHVveDND
-            VnEzRW12cnRKdzhkZm5uTXpkTEtrNUEKLS0tIHFDbkc4Mk4wVlM2R29zZXYwR2Ri
-            ODBML1p4eUZiZldYUERQTUhTU1llV0UKxjvH6zbW6wKghzR1o34CyKPEa2FqZmo0
-            PxgqyuXkIwas9soXVAkScx7ElaV09Fjaj+mDrKwi4a+DwdoSP7czyA==
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBPWE1HTUhiSUp5ZEUwWEpI
+            bGpkZlBIMUo5ZlYrQ09SN3Q1a0ZkQ0ZnOEhVCnJPNEZQenVWWGZiODlzQzNEc1Zq
+            c3l4OWZJTElJc2Y2UE15OGtEUzhyY1EKLS0tIHNJUStyWnlQWjZBbEZjQ3UwdUpz
+            ZndoUDR6bisrNGJCUHk3TGI4bTZaMFUK87fFsm9ne9s+PK2pcwtrDjqyGBss2r2E
+            8lhqoeiKZ2j96z8kP/7ChzovwTCmqdcmAQuyNQD+ZAFijseipSvfbQ==
            -----END AGE ENCRYPTED FILE-----
          recipient: age1njap586hc0q43kr03g6c8eqhdsmk8zcafkl3f83xwlc2gqhlmfgs4tmwad
        - enc: |
            -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSB0VDBXYUZNbnpEZHlqV05C
-            WWxQZjVYcWlZeXlGc0RiaUpERzR0T21CWWtBCkM0dW5kYlFNK1RGSzRudFZ2Z0t3
-            aVBreElGM1BIT0RzQkxTVTA3S2NhQncKLS0tIERyTU8zWWRTRm95SmRZQ1BhalVV
-            SVdBajN1V1BuM2s4K216S3c0VHczNlkKM5jvsSEfCBA5uZRjBJNbM91lLRQkj+jK
-            rM5uSfGLTvSjPgXIMIq03OXxH1CE7GoKxAPwFrJdAFMMQcutIethhw==
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBUYi9SRFFGV3Z6cFd2Znk5
+            b2FLbWtzTllJMDBUaGk0NTViOTNBa2hQclVZClZHKzNhbGVjQUJhWkFWdTFBMG5a
+            cUFJdUdyVG5HQXJRRnJId3hqRTN2cXMKLS0tIEFMRjh3WE1ON0U2TTNTZ3hxMTR4
+            ZGRlemlIbDZKeExmVHROc3Eyak5DdzQKaLwIVDi6BN4cxpVxJoqTYvJETPOp4thc
+            l9uVMvIGuEsEZgDsvShw1dYLljd+uGy/A+dXbcxIUCP/mmPkwmd1Pw==
            -----END AGE ENCRYPTED FILE-----
          recipient: age120le4a5l8dh3lyfgvmj3d9ksmej6ajs5mer5y7r0vfg3x9fn69dqf8xgzu
        - enc: |
            -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSAvbk9tRE9jYkpxNThQeElH
-            dWpKYThDb1ZkUWtMWlRxc2tYbjhBbEFEcWtVCkNiK0NBRko2TkJQdUVtdGtHNGcv
-            andSaUlKSFA4THRyZXNTYmI0Yk5WbmsKLS0tIHlQTUtpWDdPeTVZL0M1RElRdFNk
-            QWlGdFo5NkZWSmY0YXdpNzNUQnlsK3MKtzC0bM7Ek+K73nMranOA1Mc98RUnYnq1
-            hAt0QEFKWK4QVKubaN/rG3AzE0U7qPKWHTzoxgnAiL3WyV9teLW+iA==
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSArOWovSW9DeFpxL0VDUDQ3
+            SHUwTzJVZUtPV01ZRkdCUXZGL2lTRCtCNFNnCjBSNExqRW5mTEN5SFVucHJHSzZt
+            cDlNc3BjY3M1c1k1Z2tkVEg4R1pacGsKLS0tIEFWbHNKZW0vbVh1Y2VhQW93OUwx
+            MWV0eW9sOXdQd0l2ZjlWOEVVc1dwcTgK2s4p9xoNkawH2OkGsl80bNIo3ad5vn4W
+            Z2w+jwppSoUmbQnD3WFbLmSSxmuobmU8HILwElv6SZu+KE3aspF6XA==
            -----END AGE ENCRYPTED FILE-----
-          recipient: age164px2a8e48ptsf9ngtan38aa6jls4jdl26mzrgzf6sn3vcvt49hqjrgr8w
+          recipient: age1xjst4frdh0th6q8m7p7u9g5af7ty5jqeum0p6z8a52a9q7st7ewqw8yl9j
    lastmodified: "2026-07-19T23:30:21Z"
    mac: ENC[AES256_GCM,data:kLGE2xawQT7mx+sfw68hmGk5nCEGiEjZrqTEl9B1dtQmTrMwmoVr/1RISi4LfJrwxy31mDgff4lcIL4wIJuM373uk3X8j4RNyYQNTfKEkORT6r8NHeepNs267O77pKGd7OmcM4MT/BqOnB8ELS7Wlf2ect7CAlvUUVyc8icxgZE=,iv:EYLDsHYHZ1XOQXafOTqHHWpk/OBNq/R6IJnOBYV33E4=,tag:thxrCPC5oGvDjhK7Dz87YA==,type:str]
    unencrypted_suffix: _unencrypted
@@ -38,6 +38,7 @@
    "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIGtOWOCS+ImHc7NehguoyD7PbonGosKMZqc9+QR3v/h root@nixos"
    "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHxXTQxFnArK5HXG7czeoybZebCGfxpUdusJkPn+BCSp root@server"
    "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICMJhrfFayLBG+gWtO6oAvgambw5nWWgztiTFEaaaVRH debian@surface"
+    "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJEj26SL/emsVjW2YhRucJVp2kTz8WgcEQgjBEBLRikk root@claude"
  ];

  # Admin SSH public key, authorized on the primary user of every host and
@@ -137,6 +138,14 @@
    # shortcuts on the gui build type (hosts/nixos/home.nix).
    pveWeb = 8006;
    pbsWeb = 8007;
+
+    # Tor relay's ORPort — the port other Tor relays connect to for onion
+    # routing traffic (modules/tor/enable-relay.nix). Tor's own conventional
+    # default; opened via services.tor.openFirewall rather than
+    # networking.firewall.allowedTCPPorts directly, but kept here anyway so
+    # it's not a bare literal duplicated between the relay's settings and
+    # anything else that ever needs to reference it.
+    torRelayOrPort = 9001;
  };

  # .raw disk image size for every proxmox-* host's standalone Disko image
Author	SHA1	Message	Date
beatzaplenty	7e9c0c2a6f	Rewrite codex-maintenance.sh to scope CI checks to changed files Check NixOS configurations / eval-hosts (pull_request) Canceled after 0s Details CI was running a full eval of every host + package on every push/PR, which was slow enough to routinely time out the Gitea runner. Default mode now diffs against a base ref and scopes nixpkgs-fmt/statix/eval to the files that changed and the hosts/packages they can affect; a change to flake.nix/flake.lock/variables.nix/modules/common/* (or any other modules/*.nix outside platforms//build-types, whose blast radius isn't inferable from the path) falls back to evaluating everything. The old full sweep moves behind --full-check, which CI never passes; --dry-run adds build-planning on top of whichever scope is active. Also trims codex-setup.sh's redundant full host eval loop -- that's what codex-maintenance.sh is for; setup should just install tooling.	2026-07-20 17:25:22 +00:00
beatzaplenty	d340aca403	Merge pull request 'Authorize this box's SSH key as a nix-cache remote-builder client' (#20 ) from add-claude-builder-key into main Check NixOS configurations / eval-hosts (push) Failing after 18m59s Details Reviewed-on: #20	2026-07-20 16:31:25 +00:00
beatzaplenty	bf8ee3ce48	Merge pull request 'Worktree refactor scripts shared lib' (#19 ) from worktree-refactor-scripts-shared-lib into main Check NixOS configurations / eval-hosts (push) Failing after 11m3s Details Reviewed-on: #19	2026-07-20 16:29:42 +00:00
beatzaplentyandClaude Sonnet 5	98d4545e8f	Authorize this box's SSH key as a nix-cache remote-builder client Check NixOS configurations / eval-hosts (pull_request) Failing after 11m4s Details Following the existing per-client-key pattern in vars.remoteBuilderAuthorizedKeys (one entry per host's own default root key, not a shared dedicated keypair -- docs/nix-cache.md's "./nixremote" filename is just a generic placeholder in the instructions). This box had no entry yet, which is why its earlier `ssh-copy-id nixremote@nix-cache` never stuck: nix-cache's nixremote authorized_keys is NixOS-declarative and gets overwritten on every rebuild. Public key only; requires a nix-cache rebuild to take effect. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-20 16:29:42 +00:00
beatzaplentyandClaude Sonnet 5	d8687d979c	Reorganize scripts/ into secrets/, proxmox/, and lib/ subfolders Check NixOS configurations / eval-hosts (pull_request) Failing after 30m6s Details scripts/ had grown to 10 top-level scripts covering three distinct concerns (sops/age + SSH host-key management, Proxmox deployment, and repo-wide bootstrap/CI) with no grouping. Move the key-management scripts (backup-admin-key.sh, rotate-admin-key.sh, prepare-host-key.sh, sync-host-keys.sh) into scripts/secrets/, and the Proxmox scripts (create-proxmox-resource.sh, configure-nix-cache-client.sh) into scripts/proxmox/; leave env.sh, codex-setup.sh, codex-maintenance.sh, and bump-nixpkgs-release.sh at the top level (frequently hand-typed or pure shared config) and scripts/lib/ as-is. Updates every cross-reference: each moved script's repo_root computation (now one directory deeper), shellcheck source= directives, inter-script paths (create-proxmox-resource.sh's call into sync-host-keys.sh and its remote bootstrap of configure-nix-cache-client.sh on the Proxmox node), and every doc/module mention (CLAUDE.md's Scripts section reorganized to match, README.md, docs/auto-installer.md, docs/proxmox-images.md, modules/installer/common.nix, modules/platforms/lxc.nix). CI workflows need no change -- they only invoke codex-maintenance.sh, which didn't move. Verified via bash -n, shellcheck (no new warnings beyond the pre-existing SC1091/SC2029/SC2095 baseline), and live dry-runs of sync-host-keys.sh --all and create-proxmox-resource.sh --list from their new paths. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-20 16:20:45 +00:00
beatzaplentyandClaude Sonnet 5	a2b557c034	Lift duplicated sops/age and confirm-prompt logic into scripts/lib/ scripts/backup-admin-key.sh, rotate-admin-key.sh, and sync-host-keys.sh each independently resolved sops/age's default key-file path, derived an age pubkey from an identity file, and (two of them) ran `sops updatekeys` the same way -- now shared via scripts/lib/sops-age.sh. Also extracted the "type X to confirm" prompt duplicated across create-proxmox-resource.sh and sync-host-keys.sh into scripts/lib/confirm.sh. Pure extraction, no behavior change -- each call site produces identical commands/output to before. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-20 15:38:32 +00:00
beatzaplenty	1d44523181	Merge pull request 'Re-encrypt secrets/nix-cache.yaml for rotated lxc-nix-cache key' (#18 ) from worktree-nix-cache-502-fix into main Check NixOS configurations / eval-hosts (push) Failing after 11m15s Details Reviewed-on: #18	2026-07-20 15:33:31 +00:00
beatzaplentyandClaude Sonnet 5	222a3ced69	Re-encrypt secrets/nix-cache.yaml for current lxc-nix-cache key Check NixOS configurations / eval-hosts (pull_request) Failing after 11m53s Details .sops.yaml's &lxc-nix-cache anchor was rotated (commit `b3c8145`) after today's LXC rebuild, but secrets/nix-cache.yaml was never re-run through sops updatekeys -- it stayed encrypted for the pre-rotation key. Per docs/nix-cache.md and docs/auto-installer.md, sops-nix decrypt failures are silent: cache-priv-key never materialized, nix-serve had nothing to bind its secretKeyFile to and failed to start, and nginx proxy_pass had no upstream -- the 502 Bad Gateway seen from clients. Confirmed the new key matches the currently-deployed host by deriving the age identity from host-keys/lxc-nix-cache_ssh_host_ed25519_key.pub via ssh-to-age and comparing to .sops.yaml. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-20 15:29:34 +00:00
beatzaplenty	03137eef9a	Merge pull request 'Add nix-cache client config script for non-NixOS Debian machines' (#17 ) from worktree-nix-cache-client-script into main Check NixOS configurations / eval-hosts (push) Failing after 37m6s Details Reviewed-on: #17	2026-07-20 15:14:52 +00:00
beatzaplentyandClaude Sonnet 5	af0fe5bdfd	Wire configure-nix-cache-client.sh into create-proxmox-resource.sh's tooling bootstrap Check NixOS configurations / eval-hosts (pull_request) Failing after 11m15s Details Run it once, right after a node's first-time Nix bootstrap (not on every invocation, and not inside codex-setup.sh/codex-maintenance.sh themselves), so a freshly-bootstrapped Proxmox node substitutes from and can offload builds to nix-cache on every subsequent run. Non-fatal on failure -- the build still proceeds, just without nix-cache. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-20 15:09:57 +00:00
beatzaplentyandClaude Sonnet 5	23b910a011	Add script to configure nix-cache as substituter/remote builder on Debian clients Non-NixOS machines with just the Nix package manager installed have no module system to pick up modules/nix-cache/client.nix, so this edits /etc/nix/nix.conf directly (extra-substituters/extra-trusted-public-keys, plus the SSH remote-builder config once the nixremote key is installed), falling back to cache.nixos.org when nix-cache is unreachable. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-20 15:07:04 +00:00
beatzaplenty	19f076bba1	Merge pull request 'Fix proxmox remote build bootstrap' (#16 ) from fix-proxmox-remote-build-bootstrap into main Check NixOS configurations / eval-hosts (push) Failing after 12m2s Details Reviewed-on: #16	2026-07-20 14:43:31 +00:00
beatzaplentyandClaude Sonnet 5	9a1d6842d7	Generalize the remote tooling check to a list of required commands Check NixOS configurations / eval-hosts (pull_request) Failing after 12m13s Details Per-run tooling verification (added in the previous commit) was hardcoded to checking just `nix`. Turn it into a small array instead, so a future remote build step needing another tool extends that list rather than growing a second parallel check. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-20 14:42:00 +00:00
beatzaplentyandClaude Sonnet 5	f5ef3194d4	Fix remote-build bootstrap: retry on partial failure, fix sudo-less root Nix install Two bugs found running scripts/create-proxmox-resource.sh against a real Proxmox node for the first time: - The Nix installer's single-user root path still shells out to `sudo` to create /nix even though it already detected it's running as root, which fails outright on a minimal, sudo-less Debian/Proxmox node. codex-setup.sh now pre-creates /nix itself so that branch of the installer is skipped. - ensure_remote_repo() only ran scripts/codex-setup.sh right after a fresh git clone, so a bootstrap that cloned the repo but then failed installing Nix (exactly the failure above) left every subsequent run silently building with a `nix` that was never actually installed, since the repo already existing skipped tooling setup entirely. It now checks `command -v nix` (via the same ensure_nix_profile used elsewhere, since a non-interactive ssh session won't otherwise have a single-user install on PATH) on every run and re-bootstraps if it's missing. Both remote build heredocs also now source scripts/lib/nix-bootstrap.sh themselves for the same PATH reason. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-20 14:39:43 +00:00
beatzaplenty	90e3397b42	worktree deleted Check NixOS configurations / eval-hosts (push) Failing after 12m2s Details	2026-07-20 14:34:32 +00:00
beatzaplenty	be5812d5bb	Merge pull request 'Build Proxmox images directly on the node instead of transferring them' (#15 ) from worktree-proxmox-remote-build into main Check NixOS configurations / eval-hosts (push) Failing after 11m29s Details Reviewed-on: #15	2026-07-20 14:17:56 +00:00
beatzaplenty	84f7e038cb	removed old work specs Check NixOS configurations / eval-hosts (push) Failing after 11m42s Details	2026-07-20 13:36:03 +00:00
beatzaplentyandClaude Sonnet 5	91d8f8fab1	Add lxc-tor-relay build type with nyx monitoring New tor-relay build type (currently lxc-only) running a plain Tor middle relay via modules/tor/enable-relay.nix, plus nyx for interactive monitoring over the relay's control socket. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-20 13:36:03 +00:00
beatzaplenty	723212a81f	Merge pull request 'Worktree fix lxc docker nfs mounts' (#14 ) from worktree-fix-lxc-docker-nfs-mounts into main Check NixOS configurations / eval-hosts (push) Failing after 11m25s Details Reviewed-on: #14	2026-07-20 13:31:28 +00:00
beatzaplenty	75d09d57e3	Fix --allow-duplicate-host creating real duplicates in create-proxmox-resource.sh Check NixOS configurations / eval-hosts (pull_request) Failing after 11m24s Details --allow-duplicate-host previously just skipped the existing-resource check entirely, so re-running e.g. --type lxc --host docker while an lxc-docker container already existed created a second container sharing the same hostname/identity instead of replacing it -- both then fight over DNS/DHCP for that hostname, and it's easy to end up testing the stale one without realizing. Now splits matches into "exact" (same --type as the one being created, e.g. another lxc-docker) and "cross-type" (a different platform sharing this host identity, e.g. a proxmox-docker VM alongside an lxc-docker container -- a deliberate, valid coexistence this script has never managed and still won't). Only an exact match is destroyed and replaced, after typing the hostname back to confirm; a cross-type match is always left untouched. Without --allow-duplicate-host, both cases still refuse to run exactly as before. Verified live against pve.sweet.home: correctly split VMID 103 (a stopped proxmox-docker VM, cross-type -- left untouched) from VMID 105 (the running lxc-docker container, exact-type -- flagged for destroy+replace), and confirmed the destroy prompt safely aborts on a non-matching confirmation, leaving both resources untouched.	2026-07-20 13:22:08 +00:00
beatzaplenty	6847a7a6f4	Fix NFS shares never mounting on lxc-docker Two compounding bugs, confirmed live on the running lxc-docker container (vmid 102 on pve.sweet.home): 1. x-systemd.automount never works inside any Linux container -- systemd logs "Starting of <unit>.automount unsupported" for every share and never mounts them. modules/docker/mount-data.nix and modules/raspi/mount-data.nix now key off config.boot.isContainer (set true by nixpkgs' proxmox-lxc.nix) to mount eagerly with `nofail` there instead, while VM-based docker targets keep automount unchanged. 2. The container's Proxmox `features` never included `mount=nfs`, so AppArmor blanket-denies the nfs/rpc_pipefs mount syscalls NFS needs ("permission denied"). scripts/env.sh's PROXMOX_DEFAULT_LXC_FEATURES now includes mount=nfs;nfs4 for future lxc-* containers -- the semicolon required quoting the --features value in create-proxmox-resource.sh's remote pct-create command, since it's sent as a raw string for the remote shell to parse and an unquoted `;` would be read as a command separator. The already-running container needs a matching `pct set --features` plus a restart to pick this up -- that's an operator step outside this repo.	2026-07-20 12:56:42 +00:00