Merge pull request 'Add scripts/rotate-admin-key.sh to automate sops admin key rotation' (#10) from worktree-rotate-admin-key-script into main

Reviewed-on: #10

AGENTS.md

@@ -7,7 +7,7 @@ servers and workstation.
 
 The flake exposes NixOS configurations named `<platform>-<buildtype>`
 (platforms: `linode`, `proxmox`, `lxc`; build types: `minimal`, `nix-cache`,
-`server`, `docker`, `gui`, `pxe-boot`), generated from `modules/platforms/*`
+`server`, `docker`, `gui`, `pxe-boot`, `tailscale-exit-node`), generated from `modules/platforms/*`
 and `modules/build-types/*` by the `mkTarget` function in `flake.nix`. Not
 every combination is built — `pxe-boot` has no `linode` variant. See
 `README.md` for the full current target list; treat `flake.nix` as the

CLAUDE.md

@@ -62,8 +62,9 @@ There is no test suite — "correctness" here means the flake evaluates and
 sweeps: after editing one or two hosts/modules, evaluate just the
 `nixosConfigurations.<host>` you touched (plus any `config.system.build.tarball`
 /`diskoImagesScript`/package output affected) rather than looping over every
-host — `codex-maintenance.sh` evaluates 18 hosts plus every package/tarball/
+host — `codex-maintenance.sh` evaluates every `nixosConfigurations` host plus
-image variant now and is slow to run after each small change. Reserve a full
+every package/tarball/image variant and is slow to run after each small
+change. Reserve a full
 `codex-maintenance.sh` run for changes that plausibly affect every host
 (`modules/common/*`, `flake.nix`, `variables.nix`) or as a final check before
 committing. This is a session-workflow preference only — it does not apply to
@@ -105,13 +106,25 @@ Beyond `codex-setup.sh`/`codex-maintenance.sh` above, `scripts/` also has:
   reference `variables.nix` (confirmed empirically — `nix flake metadata`
   errors on it), so this is the closest equivalent to a single source of
   truth for the tracked release.
+- `scripts/rotate-admin-key.sh <backup-admin-key> [--new-key-file <path>]
+  [--dry-run]` — rotates `.sops.yaml`'s `&admin` age key: decrypts with a
+  backed-up copy of the key currently trusted as `&admin` (verified by
+  deriving its public key and comparing, not taken on faith), replaces the
+  `&admin` line with a new key already present in the environment
+  (defaults to wherever sops/age itself would look), and runs
+  `sops updatekeys` on every `secrets/*.yaml`. One-way: the old key can no
+  longer decrypt anything re-encrypted this way. This is the automation
+  for the manual steps `sync-host-keys.sh`/`create-proxmox-resource.sh`
+  print when they bootstrap a brand-new, not-yet-trusted key on a machine
+  with no prior admin access.
 
-`sync-host-keys.sh` and `create-proxmox-resource.sh` genuinely mutate real
+`sync-host-keys.sh`, `create-proxmox-resource.sh`, and
-state when run for real (not `--dry-run`): real `secrets/*.yaml`
+`rotate-admin-key.sh` genuinely mutate real state when run for real (not
-recipients, real Proxmox VMs/containers. They require the operator's own
+`--dry-run`): real `secrets/*.yaml` recipients, real Proxmox VMs/
-SSH/sops access, which an agent session doesn't have — but don't suggest
+containers, real revocation of decrypt access. They require the
-running either non-dry-run without the operator's explicit go-ahead even
+operator's own SSH/sops access, which an agent session doesn't have — but
-if it becomes technically reachable.
+don't suggest running any of them non-dry-run without the operator's
+explicit go-ahead even if it becomes technically reachable.
 
 ## Architecture
 
@@ -164,7 +177,7 @@ removing a host.
   `vzdump` backup-archive metadata this doesn't have), no install step —
   see `docs/auto-installer.md`.
 - `modules/build-types/*.nix` — what a system is for:
-  minimal/server/docker/gui/pxe-boot/nix-cache.
+  minimal/server/docker/gui/pxe-boot/nix-cache/tailscale-exit-node.
 - `modules/common/configuration.nix` — base NixOS config imported by every
   host: locale, users, nix settings, git.
 - `modules/common/home.nix` / `hosts/nixos/home.nix` — Home Manager config for

README.md

@@ -18,22 +18,23 @@ list:
 
 | Target | Purpose |
 | --- | --- |
-| `linode-minimal` | Minimal NixOS host profile on a Linode VPS (real, deployed) |
+| `linode-minimal` | Minimal NixOS host profile on a Linode VPS |
-| `proxmox-minimal` | Minimal NixOS host profile on Proxmox (real, deployed — previously the flat `nix-minimal` target) |
+| `proxmox-minimal` | Minimal NixOS host profile on Proxmox — previously the flat `nix-minimal` target |
 | `lxc-minimal` | Minimal NixOS host profile in a Proxmox LXC container |
-| `linode-nix-cache` / `proxmox-nix-cache` / `lxc-nix-cache` | Local Nix binary cache and remote builder (`lxc-nix-cache` is the real, deployed one — previously `proxmox-nix-cache`, itself previously the flat `nix-cache` target) |
+| `linode-nix-cache` / `proxmox-nix-cache` / `lxc-nix-cache` | Local Nix binary cache and remote builder — previously the flat `nix-cache` target |
-| `linode-server` / `proxmox-server` / `lxc-server` | Storage, NFS, backup, and monitoring exporter host (`proxmox-server` is the real, deployed one — previously the flat `server` target) |
+| `linode-server` / `proxmox-server` / `lxc-server` | Storage, NFS, backup, and monitoring exporter host — previously the flat `server` target |
-| `linode-docker` / `proxmox-docker` / `lxc-docker` | Docker host for the main container stack (`proxmox-docker` is the real, deployed one — previously the flat `docker` target) |
+| `linode-docker` / `proxmox-docker` / `lxc-docker` | Docker host for the main container stack — previously the flat `docker` target |
-| `linode-gui` / `proxmox-gui` / `lxc-gui` | Cinnamon desktop workstation (`proxmox-gui` is the real, deployed one — previously the flat `nixos` target) |
+| `linode-gui` / `proxmox-gui` / `lxc-gui` | Cinnamon desktop workstation — previously the flat `nixos` target |
-| `proxmox-pxe-boot` / `lxc-pxe-boot` | HTTP/iPXE boot asset host (`proxmox-pxe-boot` is the real, deployed one — previously the flat `pxe-boot` target) |
+| `proxmox-pxe-boot` / `lxc-pxe-boot` | HTTP/iPXE boot asset host — previously the flat `pxe-boot` target |
-| `linode-tailscale-exit-node` / `proxmox-tailscale-exit-node` / `lxc-tailscale-exit-node` | Tailscale exit node (no deployed target yet; `lxc-tailscale-exit-node` is the one planned for actual use) |
+| `linode-tailscale-exit-node` / `proxmox-tailscale-exit-node` / `lxc-tailscale-exit-node` | Tailscale exit node |
 
-This table is the only place "(real, deployed)" status is tracked — there's
+Which variant of a given buildtype is actually deployed isn't tracked
-no separate machine-readable copy to keep in sync. `scripts/create-proxmox-resource.sh`
+anywhere in this repo — that's live infrastructure state, not something a
-guards against creating a same-identity duplicate of an already-deployed host
+committed file can keep accurate, and it changes independently of the code.
-by checking the Proxmox node itself (live `qm`/`pct` state) rather than any
+Check the Proxmox node itself, or `/etc/flake-target` on a running host (see
-file in this repo, since a static list can't track whether a resource still
+below), if you need to know what's really out there right now.
-actually exists.
+`scripts/create-proxmox-resource.sh`'s duplicate-host guard works the same
+way: it checks the Proxmox node directly rather than any file here.
 
 Each buildtype's `hosts/<name>/host.nix` carries the per-machine identity
 (hostname, hostId, per-machine secrets, `system.stateVersion`) that must stay

docs/auto-installer.md

@@ -19,7 +19,7 @@ see "LXC hosts" immediately below for why those are different.**
 ## LXC hosts
 
 `lxc-*` targets (`lxc-minimal`, `lxc-nix-cache`, `lxc-server`, `lxc-docker`,
-`lxc-gui`, `lxc-pxe-boot`) are **not** installed via `auto-install.sh` — the
+`lxc-gui`, `lxc-pxe-boot`, `lxc-tailscale-exit-node`) are **not** installed via `auto-install.sh` — the
 interactive menu deliberately excludes them. Don't try to select one there;
 `nixos-install` would bind-mount `/` onto `/mnt` (LXC containers have no raw
 disk to partition) and then refuse to touch the filesystem it's currently

flake-target-refactor-spec.md

@@ -1,5 +1,13 @@
 # Spec: Refactor Flake Targets into Platform × Build-Type Matrix
 
+**Status: implemented.** `flake.nix`'s `generatedTargets`/`mkTarget` and
+`modules/platforms/*`/`modules/build-types/*` are the result of this spec —
+kept here for historical rationale only (referenced from `CLAUDE.md`'s
+"Composition pattern" section), not as an active or open plan. The "Open
+Questions" below were resolved during implementation; don't treat them as
+outstanding. A `tailscale-exit-node` build type was added later, beyond this
+spec's original scope.
+
 ## Context
 
 The flake at `~/nixos` currently defines these output targets (flat, ad-hoc naming):

remove-sensetive-info-refactor.md

@@ -122,13 +122,25 @@ Add a pre-commit hook (or a `nix flake check` step) running `gitleaks protect --
 
 ## Definition of done
 
-- [ ] Milestone 1 inventory complete and reviewed
+**Status as of 2026-07-20:** Milestones 1–3 are done — sops-nix is fully
-- [ ] All hosts have per-host age keys; admin key backed up outside the repo
+wired (`.sops.yaml`, `secrets/*.yaml`, referenced via `hashedPasswordFile`/
-- [ ] Every inventoried secret migrated to sops-nix, referenced via `*File`/`sops.secrets.*.path`, nothing plaintext in the working tree
+`*File`/`sops.secrets.*.path` throughout), and history has been scrubbed
-- [ ] `nixos-rebuild dry-build` and at least one real `switch` verified per host
+with `git-filter-repo` + force-push (this removed a GitHub fine-grained PAT
-- [ ] Working-tree scanner sweep clean
+that had been committed in plaintext in `flake.nix`/`common/home.nix`
-- [ ] History rewritten with `git-filter-repo`, force-pushed, full-history scanner sweep clean
+between 2025-07-16 and 2026-02-09, later migrated to sops but never scrubbed
-- [ ] All other clones deleted and re-cloned from the rewritten history
+from history until now). **Milestone 4 is not confirmed** — whether that PAT
-- [ ] Every credential in the original inventory rotated (not just re-encrypted)
+(or any other historically-plaintext credential) was actually rotated, not
-- [ ] Pre-commit secret scanning hook added
+just re-encrypted, isn't something this repo can attest to; that's an
-- [ ] `secrets-inventory.md` deleted from the working directory (never committed)
+operator action against the issuing service (GitHub, etc.), not a repo
+change. Do that before considering this fully closed.
+
+- [x] Milestone 1 inventory complete and reviewed
+- [x] All hosts have per-host age keys; admin key backed up outside the repo
+- [x] Every inventoried secret migrated to sops-nix, referenced via `*File`/`sops.secrets.*.path`, nothing plaintext in the working tree
+- [x] `nixos-rebuild dry-build` and at least one real `switch` verified per host
+- [x] Working-tree scanner sweep clean
+- [x] History rewritten with `git-filter-repo`, force-pushed, full-history scanner sweep clean
+- [ ] All other clones deleted and re-cloned from the rewritten history — every clone that existed before 2026-07-20's rewrite (any other machine, WSL instance, or CI checkout) needs this
+- [ ] Every credential in the original inventory rotated (not just re-encrypted) — **the GitHub PAT found in history specifically still needs this**
+- [x] Pre-commit secret scanning hook added (`.githooks/pre-commit`, `gitleaks protect --staged`)
+- [x] `secrets-inventory.md` deleted from the working directory (never committed)

scripts/create-proxmox-resource.sh

@@ -320,6 +320,7 @@ for id in $(pct list 2>/dev/null | awk 'NR>1{print $1}'); do
   n="$(pct config "$id" 2>/dev/null | grep -oP '^hostname:\s*\K\S+' || true)"
   [[ "$n" == "$target" ]] && echo "lxc ${id} ${n}"
 done
+exit 0
 REMOTE_SCRIPT
   )" || ssh_check_status=$?
   if [[ "$ssh_check_status" -ne 0 ]]; then

scripts/rotate-admin-key.sh

@@ -0,0 +1,190 @@
+#!/usr/bin/env bash
+# Rotates the &admin sops age key: decrypts with a backed-up copy of the
+# key CURRENTLY trusted as &admin, replaces .sops.yaml's &admin entry with
+# a new key already present in this environment, and re-encrypts every
+# secrets/*.yaml for the new recipient set. After this runs, the old key
+# can no longer decrypt anything -- this is a real, one-way handoff of
+# trust, not a preview.
+#
+# This is the automation for the manual steps create-proxmox-resource.sh /
+# sync-host-keys.sh print when they bootstrap a brand-new, not-yet-trusted
+# age key on a machine that's never had admin access before:
+#
+#   scripts/rotate-admin-key.sh /path/to/backed-up/admin/keys.txt
+#
+# The backup key's *public* key must match .sops.yaml's current &admin
+# entry -- this script verifies that by deriving it, it doesn't just trust
+# the filename or take it on faith. The new key defaults to wherever sops
+# itself would already look ($SOPS_AGE_KEY_FILE, then the XDG default), so
+# the common case is just pointing this at the restored backup.
+set -euo pipefail
+
+repo_root="$(cd "$(dirname "$0")/.." && pwd)"
+sops_yaml="${repo_root}/.sops.yaml"
+
+# shellcheck source=env.sh
+source "${repo_root}/scripts/env.sh"
+
+# sops resolves .sops.yaml by walking up from the process's cwd, not from
+# the target file's own path -- if this script were invoked from somewhere
+# other than the repo root (or from inside another checkout/worktree that
+# happens to have its own .sops.yaml), `sops updatekeys` would silently
+# re-encrypt against the WRONG config's recipient list instead of this
+# repo's. Pin cwd here so every sops/age call below is unambiguous
+# regardless of where the caller's shell started out.
+cd "$repo_root"
+
+usage() {
+  cat <<EOF
+Usage: $0 <path-to-backed-up-admin-key> [--new-key-file <path>] [--dry-run]
+
+  <path-to-backed-up-admin-key>  age identity file for the key CURRENTLY
+                                 trusted as &admin. Only ever read -- never
+                                 copied or modified.
+  --new-key-file <path>          age identity file for the key to promote
+                                 to &admin. Defaults to \$SOPS_AGE_KEY_FILE,
+                                 then
+                                 \${XDG_CONFIG_HOME:-\$HOME/.config}/sops/age/keys.txt
+                                 (sops/age's own default resolution order).
+  --dry-run                      Print what would change; touches nothing
+                                  (.sops.yaml untouched, no sops updatekeys
+                                  calls).
+EOF
+}
+
+dry_run=0
+new_key_file="${SOPS_AGE_KEY_FILE:-${XDG_CONFIG_HOME:-$HOME/.config}/sops/age/keys.txt}"
+args=()
+
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    --dry-run)
+      dry_run=1
+      shift
+      ;;
+    --new-key-file)
+      new_key_file="${2:?--new-key-file requires a path}"
+      shift 2
+      ;;
+    -h | --help)
+      usage
+      exit 0
+      ;;
+    --*)
+      echo "Unknown option: $1" >&2
+      usage >&2
+      exit 1
+      ;;
+    *)
+      args+=("$1")
+      shift
+      ;;
+  esac
+done
+
+if [[ "${#args[@]}" -ne 1 ]]; then
+  usage >&2
+  exit 1
+fi
+backup_key="${args[0]}"
+
+[[ -s "$backup_key" ]] || { echo "ERROR: backup key file not found or empty: ${backup_key}" >&2; exit 1; }
+[[ -s "$new_key_file" ]] || { echo "ERROR: new key file not found or empty: ${new_key_file}" >&2; exit 1; }
+
+nix_extra_opts
+
+age_pub() {
+  nix-shell "${NIX_OPTS[@]}" -p age --run "age-keygen -y '$1'"
+}
+
+echo "==> Deriving public keys..."
+old_pub="$(age_pub "$backup_key")"
+new_pub="$(age_pub "$new_key_file")"
+echo "    backup (old admin) key: ${old_pub}"
+echo "    new admin key:          ${new_pub}"
+
+if [[ "$old_pub" == "$new_pub" ]]; then
+  echo "ERROR: backup key and new key are identical -- nothing to rotate." >&2
+  exit 1
+fi
+
+current_admin_line="$(grep -E '^  - &admin age1' "$sops_yaml" || true)"
+if [[ -z "$current_admin_line" ]]; then
+  echo "ERROR: couldn't find a '&admin age1...' line in ${sops_yaml}." >&2
+  exit 1
+fi
+current_admin_pub="$(awk '{print $NF}' <<<"$current_admin_line")"
+
+if [[ "$current_admin_pub" != "$old_pub" ]]; then
+  echo "ERROR: ${backup_key} doesn't match the current &admin key in .sops.yaml." >&2
+  echo "  .sops.yaml &admin: ${current_admin_pub}" >&2
+  echo "  backup key pubkey: ${old_pub}" >&2
+  echo "Wrong backup file, or .sops.yaml has already moved on -- not touching anything." >&2
+  exit 1
+fi
+
+mapfile -t secrets_files < <(find "${repo_root}/secrets" -maxdepth 1 -name '*.yaml' | sort)
+if [[ "${#secrets_files[@]}" -eq 0 ]]; then
+  echo "ERROR: no secrets/*.yaml files found under ${repo_root}/secrets." >&2
+  exit 1
+fi
+
+echo "==> Confirming the backup key can actually decrypt..."
+if ! SOPS_AGE_KEY_FILE="$backup_key" nix-shell "${NIX_OPTS[@]}" -p sops --run \
+  "sops -d '${secrets_files[0]}'" >/dev/null; then
+  echo "ERROR: backup key failed to decrypt $(basename "${secrets_files[0]}") -- aborting." >&2
+  exit 1
+fi
+echo "    OK: decrypted $(basename "${secrets_files[0]}")"
+
+if [[ "$dry_run" -eq 1 ]]; then
+  echo
+  echo "[dry-run] would replace .sops.yaml's &admin line:"
+  echo "[dry-run]   - ${current_admin_pub}"
+  echo "[dry-run]   + ${new_pub}"
+  echo "[dry-run] would then re-encrypt (sops updatekeys --yes) for the new recipient set:"
+  for f in "${secrets_files[@]}"; do
+    echo "[dry-run]   secrets/$(basename "$f")"
+  done
+  echo
+  echo "[dry-run] Nothing was changed. Re-run without --dry-run to apply this."
+  exit 0
+fi
+
+echo "==> Rotating .sops.yaml's &admin key..."
+sed -i "s|^  - &admin age1[a-z0-9]*|  - \&admin ${new_pub}|" "$sops_yaml"
+grep -qF "$new_pub" "$sops_yaml" || {
+  echo "ERROR: sed edit didn't take -- .sops.yaml left unchanged, check it by hand." >&2
+  exit 1
+}
+echo "    Updated."
+
+echo "==> Re-encrypting secrets/*.yaml for the new recipient set..."
+for f in "${secrets_files[@]}"; do
+  echo "==> $(basename "$f")"
+  SOPS_AGE_KEY_FILE="$backup_key" nix-shell "${NIX_OPTS[@]}" -p sops --run \
+    "sops updatekeys --yes '${f}'"
+done
+
+echo "==> Verifying the new key can decrypt everything..."
+for f in "${secrets_files[@]}"; do
+  if ! SOPS_AGE_KEY_FILE="$new_key_file" nix-shell "${NIX_OPTS[@]}" -p sops --run \
+    "sops -d '${f}'" >/dev/null; then
+    echo "ERROR: new key failed to decrypt $(basename "$f") after rotation -- investigate before committing." >&2
+    exit 1
+  fi
+  echo "    OK: $(basename "$f")"
+done
+
+cat <<EOF
+
+Done. .sops.yaml's &admin key is now:
+  ${new_pub}
+
+The old key (${old_pub}) can no longer decrypt any secrets/*.yaml
+re-encrypted above.
+
+Review the diff, then commit:
+  git add .sops.yaml secrets/*.yaml
+  git commit -m "Rotate sops admin age key"
+EOF