Merge branch 'main' into worktree-migrate-vm-script

Reuses the clone script's vzdump/relay/restore flow but as a move rather
than a copy: preserves the original MAC by default, adds a stop-source/
start-target cutover step, and an opt-in --remove-source to destroy the
source guest afterward. Not a true qm migrate (nodes aren't clustered),
so the default --mode snapshot has a documented write gap between
snapshot and cutover; --mode stop avoids it at the cost of downtime.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Lkv8k24Wuua3GRxrfMBLz

.sops.yaml

@@ -10,6 +10,8 @@ keys:
   - &lxc-minimal age1jy444f9d9stygj4p3w9kh54cqcfr654tvr75tdvee5cxsgtdtc9q3v60ep
   - &lxc-pxe-boot age1fxxzpnfse8nd9wz78ht3m0plrmraacf4cpga0pe8fm2tdnqcgy8q7qsyvp
   - &lxc-gui age190htw7prp4vln076dxjx3gxxaq06h0zl0te7cqgpx79vl3lhkaes8suy05
+  - &proxmox-server age1ukpqxzl44mnjpy5r96sfuc5sqzm47u4k8ujjh5qdgy6jvl9uqgpspymqfk
+  - &vm-server age15kh7akxlx7zn00tey79rq2g8lgs4j5y77rcnyfxrxap8ckfu0a9sqvtdhh
 
 creation_rules:
   # Shared across every currently-deployed host: root/nixos password hash,
@@ -30,6 +32,8 @@ creation_rules:
           - *lxc-docker
           - *lxc-pxe-boot
           - *lxc-gui
+          - *proxmox-server
+          - *vm-server
 
   - path_regex: secrets/nix-cache\.yaml$
     key_groups:
@@ -43,6 +47,8 @@ creation_rules:
       - age:
           - *admin
           - *server
+          - *proxmox-server
+          - *vm-server
 
   - path_regex: secrets/docker\.yaml$
     key_groups:

AUDIT_REPORT.md

@@ -0,0 +1,150 @@
+# Flake End-to-End Audit Report
+
+**Date:** 2026-07-21
+**Scope:** Full static lint/eval sweep + live build/deploy/interrogate/destroy testing of every `lxc-*` and `proxmox-*` flake target against `pve.sweet.home`, plus an audit of the operator's ability to manage the flake/secrets tooling.
+**Branch:** `worktree-flake-e2e-audit` (this session's isolated worktree)
+
+## Executive Summary
+
+The flake itself is in good shape: `nixpkgs-fmt`, `statix`, and a full eval + dry-run build of every host and package are all clean. Every `lxc-*`/`proxmox-*` target's NixOS configuration builds successfully — no target has a broken derivation graph.
+
+The issues found are **operational, not code-level**:
+
+1. **pve.sweet.home is critically low on disk space** (91-95% full during this session) and cannot currently build the two largest closures (`gui`, `pxe-boot`) to completion — this actively blocks deploying/redeploying those hosts via the documented workflow.
+2. **A real, reproducible secrets-decryption failure** was caught live: a stale cached container image (built before a same-day sops-key fix) boots with sshd never starting and every secret failing to decrypt. This is a **general hazard in `create-proxmox-resource.sh`'s "reuse the cached image if present" default**, not a one-off.
+3. **sops key/anchor drift**: `proxmox-minimal` has a `.sops.yaml` recipient anchor with no corresponding private key anywhere in this environment; several `lxc-*`/`proxmox-*` targets have no sops registration at all yet.
+4. One concrete script bug was found and **fixed in this session**: `create-proxmox-resource.sh` never enabled the QEMU guest agent channel on VMs it creates, despite the guest OS already running it.
+5. A management-surface audit (of the operator's ability to run this repo day to day) found 5 process gaps, detailed below.
+
+Nothing here required or received a `nixos-rebuild switch/boot/test`, `nixos-install`, or any disk-formatting command — all validation was `nix build`/`nix eval`, plus disposable `pct`/`qm` create-then-destroy cycles via the repo's own `create-proxmox-resource.sh`.
+
+---
+
+## 1. Static Analysis Results — all clean
+
+`bash scripts/codex-maintenance.sh --full-check --dry-run` (whole-tree sweep, not just changed files):
+
+| Check | Result |
+|---|---|
+| Secret grep | Clean — only the documented exceptions (installer's own hashed passwords, `access-tokens` comment references) |
+| `nixpkgs-fmt --check` | 0/53 files would be reformatted |
+| `statix` | No lint warnings |
+| nix-cache host key drift check | Up to date |
+| Full eval of every host's `system.build.toplevel` | All 19 `nixosConfigurations` targets evaluate cleanly |
+| Dry-run build of every host + package | All succeed, no derivation errors |
+
+No drift, no formatting issues, no lint findings anywhere in the tree.
+
+---
+
+## 2. Per-Target Test Results
+
+Legend: **LIVE** = built on pve, `pct`/`qm` create → interrogated → destroyed. **BUILD-ONLY** = `nix build` validated the config (mostly `.config.system.build.toplevel`, occasionally `.tarball`), no resource created on pve.
+
+| Target | Test type | Result | Notes |
+|---|---|---|---|
+| `lxc-docker` | BUILD-ONLY | ✅ PASS | Live redeploy skipped — CT105 is already running this identity in production; `--allow-duplicate-host` would have destroyed it. |
+| `lxc-minimal` | **LIVE** | ✅ PASS (after retry) | First attempt reused a stale cached tarball predating a same-day sops-key commit → activation failed, sshd never started (see Finding #2). Redeployed with `--force-rebuild`: clean boot, `systemctl is-system-running` = `running`, secrets decrypted, sshd listening, users correct. |
+| `lxc-nix-cache` | BUILD-ONLY | ✅ PASS (after retry) | Live redeploy skipped — CT101 is already running this identity. First local build attempt appeared to hang on a remote-builder handoff to nix-cache; killed and retried with `--builders ""` (local-only), succeeded. |
+| `lxc-gui` | **LIVE (attempted)** | ⚠️ BLOCKED by pve disk space | Registered a fresh sops key (no prior registration existed), built successfully through the full NixOS system closure, then **failed packaging the tarball**: `No space left on device` on pve's root filesystem. Not a flake defect. |
+| `lxc-pxe-boot` | **LIVE (attempted)** | ⚠️ BLOCKED by pve disk space | Same failure as `lxc-gui` — this target additionally builds a full nested installer/netboot image (`stage-installer-artifacts.nix`), making it similarly large. Failed with the same `No space left on device` error, immediately after the gui attempt had already consumed pve's remaining headroom. |
+| `lxc-server` | BUILD-ONLY | ✅ PASS | No sops key registered yet; live deploy also would have hit `boot.zfs.extraPools` trying to import a real ZFS pool that doesn't exist in an isolated test container — an expected limitation of testing this build type outside its real hardware, not a bug. |
+| `lxc-tailscale-exit-node` | BUILD-ONLY | ✅ PASS | No sops key registered yet. |
+| `lxc-tor-relay` | BUILD-ONLY | ✅ PASS | Live redeploy skipped — CT106 already holds this identity in production. |
+| `proxmox-docker` | BUILD-ONLY | ✅ PASS (after retry) | Live redeploy skipped — both CT105 *and* VM103 already hold `docker` identities. Combined `toplevel` + `diskoImagesScript` build crashed with a **Nix-internal assertion failure** (`worker.cc:360`) under this session's memory pressure (see Finding #6) — not a flake bug. Retried with `toplevel` alone: clean. |
+| `proxmox-minimal` | **LIVE (attempted)** | ⚠️ BLOCKED by key drift → BUILD-ONLY | `.sops.yaml` has a registered `&proxmox-minimal` anchor but **no corresponding private key exists anywhere in this environment** — the script correctly refused to generate a mismatched replacement. Fell back to `toplevel` build: ✅ PASS. |
+| `proxmox-nix-cache` | BUILD-ONLY | ✅ PASS | No sops key registered yet. |
+| `proxmox-gui` | BUILD-ONLY | ⚠️ Killed after ~40min (resource-limited) | This session's local build machine has only 2GB RAM; swap filled completely (2.0/2.0GB) and the build stalled, so it was killed rather than risk destabilizing the session further. **Not a flake defect** — the equivalent `gui` NixOS configuration already proved fully buildable during the `lxc-gui` live attempt above (it built the entire system closure successfully and only failed at the pve-side tarball-packaging step due to disk space, not the config). |
+| `proxmox-pxe-boot` | BUILD-ONLY | ⚠️ Killed after ~35min (resource-limited) | Was deep into building the nested installer's kernel initrd (this build type bundles a full netboot installer image via `stage-installer-artifacts.nix`) when killed to keep the audit moving. **Not a flake defect** — this target's own module logic was already effectively validated via the earlier *live* pve deploy attempt (`lxc-pxe-boot` above), which built the complete image and only failed at the final tarball-packaging step due to pve's disk space (Finding 1). |
+| `proxmox-server` | BUILD-ONLY | ✅ PASS | No sops key registered yet; same ZFS-pool caveat as `lxc-server` would apply to a live deploy. |
+| `proxmox-tailscale-exit-node` | BUILD-ONLY | ✅ PASS | No sops key registered yet. |
+
+**Not tested at all:** `linode-*` targets (not deployable to Proxmox) and `installer` (not a normal host) — both were still covered by the static eval/dry-run-build sweep above.
+
+---
+
+## 3. Findings, Ranked by Severity
+
+### Finding 1 — pve.sweet.home is critically low on disk space (blocks real deployments)
+
+At session start: `/dev/mapper/pve-root` was **95% full, 5.3GB free** (of 94GB). After two failed large builds it recovered slightly to **91% full, 8.2GB free** (nix cleans up its own failed-build scratch space). `/nix/store` alone is 26GB; `nix-store --gc --print-dead` reports **zero** reclaimable garbage — everything currently in the store is a live GC root, so `nix-collect-garbage` won't help without first removing old roots.
+
+**Why it matters:** `create-proxmox-resource.sh` builds every VM/CT image **directly on pve**, not on a build machine and transferred over. With <10GB headroom, any closure approaching a few GB (the `gui` build type: full Cinnamon desktop + Firefox + LibreOffice + GIMP + VS Code + xrdp; the `pxe-boot` build type: nginx/atftpd *plus* an entire nested installer/netboot image) cannot currently be built there at all. Both `lxc-gui` and `lxc-pxe-boot` failed live with `No space left on device` during this audit.
+
+**Recommended action:** Expand `pve-root`'s LV, or free space by pruning old container templates in `/var/lib/vz/template/cache` (1.5GB) / old backups in `/var/lib/vz/dump` (306MB) / auditing what's pinning 26GB of `/nix/store` as live GC roots (likely `result-*` symlinks — see below). This is real production disk state; **not something this session touched or fixed** — it needs the operator's judgment on what's safe to remove.
+
+**Secondary, smaller finding:** every `create-proxmox-resource.sh` run leaves a `result-<target>` symlink in the node's repo checkout as a permanent GC root (`ls /root/nixos/result-*` on pve showed 3 from this session alone: `lxc-docker`, `lxc-minimal`, `lxc-nix-cache`). These accumulate forever and pin their entire closures in the store. Consider having the script clean up its own `result-*` link after staging the built artifact (or use a temp `--out-link` under `/tmp`), so `nix-collect-garbage` can actually reclaim old build outputs.
+
+### Finding 2 — Stale cached images can silently ship broken secrets (reproduced live)
+
+`create-proxmox-resource.sh`'s default behavior is: if the node already has `<target>.tar.xz`/`.raw` staged, **reuse it** — only `--force-rebuild` forces a fresh build. This session hit exactly the failure mode `docs/auto-installer.md` already warns about: `lxc-minimal`'s cached tarball (built 2026-07-20T15:57Z) predated a same-day sops-key fix commit (2026-07-20T17:49Z, "clean up in ailse 3"). The deployed container booted with:
+
+```
+sops-install-secrets: failed to decrypt '.../common.yaml': Error getting data key: 0 successful groups required, got 0
+Activation script snippet 'setupSecrets' failed (1)
+```
+
+— every secret permanently failed to decrypt, `sshd` never started (though the container otherwise looked "running"). This was **not a code bug**: the currently-committed `secrets/common.yaml` decrypts fine for that host's key when checked independently; the *cached artifact on pve* simply reflected an older commit's ciphertext. Redeploying with `--force-rebuild` fixed it immediately.
+
+**Why it matters:** this is silent and easy to trigger by accident — any operator who redeploys a host without remembering `--force-rebuild` after a secrets change gets a container that looks like it started (`pct start` succeeds, `pct status` = running) but is completely inaccessible.
+
+**Recommended action:** Have `create-proxmox-resource.sh` compare the cached image's build timestamp (or embed the source commit hash in the staged filename) against current HEAD, and warn (or refuse without `--force-rebuild`) if they differ — rather than silently trusting presence alone.
+
+### Finding 3 — sops key/anchor drift
+
+Two concrete instances hit live during this session:
+
+- **`proxmox-minimal`**: `.sops.yaml` already has a registered `&proxmox-minimal` age recipient, but this environment's `host-keys/` directory has no corresponding private key file. `sync-host-keys.sh` correctly refused to generate a replacement (it would silently mismatch whatever's already registered/deployed) — but this means **no environment currently has this host's private key**, unless it exists on some other machine that was never backed up here.
+- **`lxc-gui`**, and by the same logic `lxc-server`/`lxc-tailscale-exit-node`/most `proxmox-*` targets, have **no sops registration at all yet** — expected for undeployed hosts per `docs/auto-installer.md`, but this session's live-testing needed to register `lxc-gui`'s key on the fly, which immediately hit **Finding 3b**: registering a key locally does nothing for pve's build until it's pushed to `origin/main` (pve builds via `git pull`, not from this uncommitted worktree). This is exactly gap #4 the management-surface audit (below) already flagged in the abstract — this session hit it concretely.
+
+**Recommended action:** for `proxmox-minimal`, decide whether to regenerate its key (destroying old-key decrypt access, if anything still holds it) or track down wherever the original private key lives and back it up here. For the general pattern, see the management-surface audit's recommendation to pre-flight-check key registration before building.
+
+### Finding 4 — QEMU guest agent never wired up (found and fixed this session)
+
+`modules/common/configuration.nix:44` sets `services.qemuGuest.enable = true` on every host — the guest-side agent daemon is correctly enabled everywhere. But `scripts/proxmox/create-proxmox-resource.sh`'s `qm create` call never passed `--agent 1`, so **Proxmox never created the virtio-serial channel** the agent needs. Every `proxmox-*` VM this script ever created was silently missing `qm guest exec`/IP-address reporting in the Proxmox UI, despite the guest daemon actually running.
+
+**Status: fixed in this session's worktree** (`scripts/proxmox/create-proxmox-resource.sh`, `qm create` now includes `--agent enabled=1`) — see the diff, included in the PR from this session.
+
+### Finding 5 — Orphaned container on pve (CT102)
+
+`pve.sweet.home` has a stopped LXC container, **VMID 102**, with an essentially empty config (`lock: create` and nothing else — no hostname, no rootfs, no network) — the leftover of a `pct create` that started and never finished. It predates this session (not created by any of this audit's activity) and wasn't touched. **Recommend the operator confirm it's abandoned and remove it** (`pct destroy 102 --purge 1`) — left as-is it may be someone's genuine in-progress work, so it wasn't assumed safe to delete autonomously.
+
+### Finding 6 — Nix-internal crash under memory pressure (tooling, not flake)
+
+Building `proxmox-docker`'s `toplevel` and `diskoImagesScript` together crashed with a Nix-internal assertion failure (`Assertion '!awake.empty()' failed ... worker.cc:360`, a known class of bug in Nix's multi-goal build scheduler) while this session's 2GB-RAM build container was under heavy swap pressure (1.8-2.0/2GB swap in use) from a separate concurrent build. Retrying the same target alone (no concurrency) succeeded cleanly. **Not a flake defect** — purely an artifact of this session's constrained build environment; noted for completeness since it looked alarming in isolation.
+
+### Finding 7 — Management-surface audit: 5 operability gaps
+
+A focused audit of "can the operator actually run this repo day to day" (flake, home-manager, sops, related scripts) found:
+
+1. **No documented recovery path if the `&admin` sops age key is lost without a backup.** `scripts/secrets/backup-admin-key.sh` exists and works but is referenced nowhere in `README.md`/`docs/` — no forcing function ensures a backup was ever taken. `rotate-admin-key.sh` requires the *old* key to re-key; there's no bootstrap-from-nothing path documented (the real fallback — deriving an age identity from any still-live host's own SSH key — isn't written down anywhere).
+2. **home-manager has no standalone iteration path.** It's wired only inside `nixosConfigurations` (`flake.nix`) — no `homeConfigurations` output. The fastest real shortcut (`nix build .#nixosConfigurations.<target>.config.home-manager.users.nixos.home.activationPackage`) isn't documented anywhere, so the practical workflow is a full host rebuild to test one HM tweak.
+3. **Gitea's flake-lock-update workflow pushes straight to `main` with no pre-merge validation.** `.gitea/workflows/update-flake-lock.yml` commits and pushes `nix flake update`'s result directly; `codex-maintenance.sh` only runs *after*, on the resulting push — a genuinely broken lockfile bump lands on `main` before anything catches it. (The GitHub-side workflow is safer — PR-based — but has the opposite gap: nothing alerts if the PR sits unmerged.)
+4. **No pre-flight check that a build target has a registered sops key before building it.** `docs/auto-installer.md` documents the failure mode (silent, total secrets-decrypt failure) but nothing in `create-proxmox-resource.sh` refuses to proceed when it's about to build a target with no `.sops.yaml` anchor — it's on the operator to remember. This session's `lxc-gui` test hit close to this exact gap (needed the key added on the fly, mid-session).
+5. **`vars.remoteBuilderAuthorizedKeys` has the same drift risk as `vars.nixCacheHostKey`, but no checker script.** `sync-nix-cache-host-key.sh --check` guards the latter; the former (and `vars.pxeServerIp`/`vars.pbsIp`) has no equivalent — a rotated/revoked client key just silently stops working with no diagnostic pointing back here.
+
+---
+
+## 4. Action Plan (priority order)
+
+1. **Free up disk space on pve.sweet.home** (or expand `pve-root`). Blocking: `lxc-gui`, `proxmox-gui`, `lxc-pxe-boot`, `proxmox-pxe-boot` cannot currently be built/redeployed on this node at all.
+2. **Decide on `proxmox-minimal`'s orphaned sops key**: locate the original private key and back it up here, or accept regenerating it (breaks decrypt access for whoever/whatever currently holds the old one).
+3. **Merge this session's PR** (see below) to get the `--agent 1` fix and `lxc-gui`'s new sops registration onto `main` — required before `lxc-gui` can be live-redeployed with working secrets.
+4. **Add a staleness guard to `create-proxmox-resource.sh`'s cache-reuse path** (Finding 2) — highest-leverage fix, since it silently produces a broken-but-"running" host.
+5. **Add a pre-flight sops-anchor check to `create-proxmox-resource.sh`** (management-surface gap #4) — same root cause class as #4 above, catch it before building instead of at first boot.
+6. Investigate/clean up **CT102** on pve (Finding 5) — confirm abandoned, then remove.
+7. Document `backup-admin-key.sh` in `README.md`'s Security Notes and add the live-host-key bootstrap-recovery procedure to `docs/` (management-surface gap #1).
+8. Add pre-push validation to the Gitea flake-lock-update workflow (management-surface gap #3).
+9. Lower-priority: document the home-manager `activationPackage` shortcut (gap #2); extend `sync-nix-cache-host-key.sh`'s drift-check pattern to `remoteBuilderAuthorizedKeys` (gap #5).
+10. Follow-up session: finish build-validating `proxmox-gui` and `proxmox-pxe-boot` (both killed here after 35-40min on this session's 2GB-RAM machine — not failures, just unfinished) once pve has headroom (item 1) — ideally from a machine with more RAM. `proxmox-server` and `proxmox-tailscale-exit-node` already passed build-only validation in this session, no follow-up needed.
+
+---
+
+## 5. Uncommitted Changes From This Session
+
+This worktree (`worktree-flake-e2e-audit`) currently has:
+
+- `scripts/proxmox/create-proxmox-resource.sh` — the `--agent enabled=1` fix (Finding 4).
+- `.sops.yaml` / `secrets/common.yaml` — `lxc-gui`'s new age key registered as a recipient (generated live during this session's testing).
+
+Per this session's standard workflow, these will be committed, pushed, and opened as a draft PR rather than pushed to `main` directly — merging it is the operator's call, and is also **prerequisite to live-redeploying `lxc-gui` successfully** (its build will keep hitting the sops-staleness failure from Finding 2 on pve until this registration is on `origin/main`).

CLAUDE.md

@@ -31,6 +31,74 @@ machines when deployed.
   the auto-installer (see `docs/auto-installer.md`). Never commit its
   contents; if `git status` ever shows it as trackable, something is wrong.
 
+### Two Proxmox nodes: `pve1.sweet.home` (production) and `pve-test.sweet.home` (sandbox)
+
+There are two SSH-reachable Proxmox nodes on the LAN, both defined in
+`scripts/env.sh` (`PVE1_HOST` / `PVE_TEST_HOST`), individually targetable
+via `scripts/proxmox/create-proxmox-resource.sh --node <host>` or by
+overriding `PROXMOX_HOST`. `PROXMOX_HOST` itself still defaults to
+`PVE1_HOST` (production) — that default, and every other script behavior,
+is unchanged from before `pve-test` existed; the only thing new is that
+`pve-test` can now be reached at all. They are **not interchangeable** —
+one is real production infrastructure, the other exists specifically so
+there's somewhere safe to test. The restriction below is a policy for
+Claude specifically, not a change to the tooling's own default or
+anything the operator needs to opt into.
+
+#### `pve1.sweet.home` (production — off-limits to Claude)
+
+A real, live Proxmox node hosting production VMs/containers — not a
+sandbox, and not Claude's to touch by default.
+
+- **Off-limits at all times unless the operator has given explicit,
+  same-session instructions to act on this specific host.** That
+  authorization is scoped to the task it was given for — don't carry it
+  forward to unrelated later work in the same conversation, and never
+  assume it from a previous session.
+- **Read-only for existing state is always fine, authorization or not.**
+  You may SSH in (or use `pvesm`, `qm list`, `pct list`, `qm config`, `pct
+  config`, the Proxmox API, etc.) to inspect the node's config, storage,
+  and any existing VM/container — including ones this repo didn't create.
+- **Never** modify, stop, restart, delete, reconfigure, or create anything
+  on this node (`qm set`, `pct set`, `qm destroy`, `pct destroy`, `qm
+  stop`, `pct stop`, `qm create`, `pct create`, snapshot operations,
+  storage changes, etc.) — including scratch/test resources — without
+  that explicit go-ahead. Use `pve-test.sweet.home` for anything
+  exploratory instead; it exists precisely so `pve1` never has to be the
+  answer to "where do I test this."
+- **This is a Claude-specific policy, not something the scripts enforce.**
+  `scripts/env.sh`/`create-proxmox-resource.sh` default to `pve1` exactly
+  as they did before `pve-test` existed, with no extra flag or prompt
+  required — that's deliberate, so the operator's own existing workflows
+  don't change. Claude, however, must never rely on that default: every
+  Proxmox action Claude takes on its own initiative — not explicitly
+  pointed at `pve1` by the operator this session — targets `pve-test`
+  instead (e.g. `--node "$PVE_TEST_HOST"`, or `PROXMOX_HOST=$PVE_TEST_HOST`).
+  Claude's own default is `pve-test`, full stop, regardless of what the
+  tooling's own unqualified default happens to be.
+
+#### `pve-test.sweet.home` (sandbox — Claude's default target)
+
+A separate Proxmox node set aside for testing. The *tooling's* default is
+still production (`PROXMOX_HOST` → `PVE1_HOST`, see above) — but
+**Claude's own default is this node**: absent an explicit, same-session
+instruction to use `pve1`, every Proxmox action Claude initiates targets
+`pve-test`. Once targeted, it's safe to create, interrogate, and destroy
+resources on without asking first.
+
+- **Test VMs/containers are allowed, but must be torn down.** Create a
+  scratch VM or container here (e.g. via
+  `scripts/proxmox/create-proxmox-resource.sh` or raw `qm`/`pct create`)
+  to validate something. Anything created this way must be destroyed
+  again in the same session, before ending the task — never leave a test
+  resource running. Use a VMID/name that's obviously scratch (and doesn't
+  collide with a real flake target) so it's unambiguous what's safe to
+  remove.
+- **Node-level config is still not yours to change.** Creating/destroying
+  your own scratch guests is fine; Proxmox host config, storage pools, and
+  networking on `pve-test` itself are still the operator's call to make
+  manually, same as on `pve1`.
+
 ## Commands
 
 ```bash
@@ -188,6 +256,33 @@ lift the shared part into `scripts/lib/` instead of copying it.
   `/etc/ssh/ssh_known_hosts`. Idempotent (re-running replaces its own
   marked block rather than duplicating it); restarts `nix-daemon` by
   default so the change takes effect immediately.
+- `scripts/proxmox/clone-pve1-to-pve-test.sh --vmid <n> [options]` — ad hoc
+  copy of a single VM/CT from one node to another via `vzdump` +
+  `qmrestore`/`pct restore`, relayed through this machine
+  (`ssh src cat ... | ssh dst cat > ...`, no on-disk staging copy).
+  Defaults `pve1` → `pve-test` (see "Two Proxmox nodes" above); source is
+  left running throughout (`--mode snapshot` default) and untouched
+  beyond the vzdump + a typed-VMID confirm, restored on the target with a
+  fresh MAC (`--unique 1`, always) since the source is expected to still
+  be live on the same LAN. Backup archives are deleted from both nodes
+  afterward unless `--keep-backup`. `--dry-run` throughout.
+- `scripts/proxmox/migrate-vm.sh --vmid <n> [options]` — moves (rather
+  than clones) a single VM/CT between nodes, built on the same
+  vzdump/relay/restore pattern as `clone-pve1-to-pve-test.sh` above, plus
+  a cutover step (stop the guest on the source node, start it on the
+  target) and, optionally, `--remove-source` to destroy it on the source
+  node afterward (its own separate typed confirmation, since that step is
+  irreversible) — the default instead leaves the stopped source guest
+  intact as a safety net. Restores with the *original* MAC (no
+  `--unique`) unless `--unique` is passed, since the source is expected to
+  go away rather than keep running alongside the target. **Not** a true
+  `qm migrate`-style live migration — pve1/pve-test aren't a clustered
+  pair with shared storage, so this is vzdump-based and, with the default
+  `--mode snapshot`, any writes on the source between the snapshot and
+  cutover are lost (use `--mode stop` if that's not acceptable). Generic
+  node-to-node (not pve1/pve-test-specific) via
+  `--source-node`/`--target-node`, defaulting to pve1 → pve-test like the
+  clone script. `--dry-run` throughout.
 
 ### `scripts/lib/`
 

README.md

@@ -36,6 +36,11 @@ Check the Proxmox node itself, or `/etc/flake-target` on a running host (see
 below), if you need to know what's really out there right now.
 `scripts/proxmox/create-proxmox-resource.sh`'s duplicate-host guard works the same
 way: it checks the Proxmox node directly rather than any file here.
+Real, production deployments live on `pve1.sweet.home`; there's a second
+node, `pve-test.sweet.home`, set aside purely for scratch/test resources —
+see `scripts/env.sh` (`PVE1_HOST` / `PVE_TEST_HOST`, and the
+`--node`/`PROXMOX_HOST` targeting they feed into) and CLAUDE.md's Proxmox
+section for which is which.
 
 Each buildtype's `hosts/<name>/host.nix` carries the per-machine identity
 (hostname, hostId, per-machine secrets, `system.stateVersion`) that must stay

docs/auto-installer.md

@@ -150,7 +150,11 @@ use case.
 
 The `pxe` variant is also built automatically as part of the `pxe-boot` host
 itself (`modules/pxe-boot/stage-installer-artifacts.nix`) and served over
-iPXE — see `docs/pxe-boot.md`.
+iPXE as the menu's "NixOS Auto-Installer" entry — see `docs/pxe-boot.md`.
+That same host also builds and serves `packages.x86_64-linux.pxe-minimal`,
+a vanilla NixOS minimal netboot image with none of this auto-installer's
+wiring, as a separate "NixOS Minimal" menu entry — also documented in
+`docs/pxe-boot.md`, not covered further here since it's not this installer.
 
 ## Host keys
 

docs/pxe-boot.md

@@ -1,9 +1,10 @@
 # pxe-boot
 
-The `pxe-boot` host serves HTTP boot assets for iPXE clients — including a
-self-staged copy of this flake's own auto-installer netboot image, see
-`docs/auto-installer.md` for what that image actually is and does once
-booted.
+The `pxe-boot` host serves HTTP boot assets for iPXE clients — including
+self-staged copies of both this flake's own auto-installer netboot image
+(see `docs/auto-installer.md` for what that image actually is and does once
+booted) and a vanilla, unmodified NixOS minimal netboot image for plain
+rescue/inspection use.
 
 ## Host Role
 
@@ -28,7 +29,8 @@ The host creates these directories with systemd tmpfiles:
 /srv/pxe
 /srv/pxe/http
 /srv/pxe/http/images
-/srv/pxe/http/nixos
+/srv/pxe/http/auto-installer
+/srv/pxe/http/nixos-minimal
 /srv/pxe/http/systemrescue
 /srv/pxe/http/ubuntu
 /srv/pxe/http/rescue
@@ -37,7 +39,7 @@ The host creates these directories with systemd tmpfiles:
 
 Mount shared image storage under `/srv/pxe/http`, preferably
 `/srv/pxe/http/images` unless a menu entry expects files in a specific
-directory such as `/srv/pxe/http/nixos`.
+directory such as `/srv/pxe/http/auto-installer`.
 
 The HTTP iPXE chain is:
 
@@ -50,20 +52,37 @@ undionly.kpxe or ipxe.efi
 
 The generated menu currently exposes entries for:
 
-- NixOS installer
+- NixOS Auto-Installer
+- NixOS Minimal
 - SystemRescue environment
 - iPXE shell
 - Reboot
 
-The NixOS installer entry chain-loads `/srv/pxe/http/nixos/netboot.ipxe`,
-which is nixpkgs' own generated netboot iPXE script (correct `init=`/`initrd=`
-kernel parameters included) rather than a hand-rolled boot line — that script
-in turn expects its kernel/initrd siblings in the same directory. All three
-files (`bzImage`, `initrd`, `netboot.ipxe`) are built from this flake's own
-`modules/installer/iso.nix` netboot image (the same one `nix build .#pxe`
-produces) and staged automatically by
-`modules/pxe-boot/stage-installer-artifacts.nix` via `systemd.tmpfiles.rules`
-— no manual operator step required.
+Both NixOS entries chain-load a `netboot.ipxe` staged into their own
+directory (`/srv/pxe/http/auto-installer/netboot.ipxe` and
+`/srv/pxe/http/nixos-minimal/netboot.ipxe`), each nixpkgs' own generated
+netboot iPXE script (correct `init=`/`initrd=` kernel parameters included)
+rather than a hand-rolled boot line — that script in turn expects its
+kernel/initrd siblings in the same directory. Each directory's three files
+(`bzImage`, `initrd`, `netboot.ipxe`) are built from source and staged
+automatically by `modules/pxe-boot/stage-installer-artifacts.nix` via
+`systemd.tmpfiles.rules` — no manual operator step required:
+
+- `auto-installer` is this flake's own `netbootSystem` (`flake.nix`) — the
+  same auto-installer image `nix build .#pxe` produces. See
+  `docs/auto-installer.md`.
+- `nixos-minimal` is `netbootMinimalSystem` (`flake.nix`) — nixpkgs'
+  `netboot-minimal.nix` composed on its own, with none of this flake's
+  auto-installer wiring (no `common.nix`, no `auto-install.sh`, no baked
+  host keys or custom users). Same `nix build .#pxe-minimal` mechanism as
+  the auto-installer image, just a different module composition. Useful
+  as a plain rescue/inspection shell that doesn't assume anything about
+  this flake.
+
+Both images set `networking.hostName` to match their menu entry/staged
+directory name (`auto-installer` / `nixos-minimal`), so each one's
+generated system name (`nixos-system-<name>-*`) is self-describing rather
+than the nixpkgs default of `nixos-system-nixos-*` for both.
 
 The SystemRescue entry expects the source ISO at:
 

flake.nix

@@ -65,7 +65,7 @@
           # file without a same-option circular dependency (a module
           # contributing to environment.etc can't read the merged
           # environment.etc it's itself contributing to).
-          specialArgs = { inherit inputs vars netbootSystem flakeTarget; };
+          specialArgs = { inherit inputs vars netbootSystem netbootMinimalSystem flakeTarget; };
         };
 
       # Generated platform x build-type matrix. pxe-boot has no linode
@@ -119,19 +119,61 @@
 
       # Same installer environment, built as netboot (kernel + initrd +
       # iPXE script) instead of an ISO — this is what packages.pxe bundles.
+      #
+      # Deliberately imports common.nix directly, NOT ./modules/installer/iso.nix
+      # (which pulls in nixpkgs' installation-cd-minimal.nix) -- confirmed live
+      # that composing the ISO module together with netboot-minimal.nix hangs
+      # every boot waiting for a device that can never exist on a netboot
+      # client ("A start job is running for /dev/disk/by-label/nixos-minimal-...").
+      # Both installation-cd-base.nix and netboot.nix set fileSystems."/" via
+      # the identical lib.mkImageMediaOverride (mkOverride 60) priority --
+      # genuinely conflicting root-filesystem strategies (ISO-by-label vs.
+      # netboot-tmpfs) at the same priority, and the ISO one was winning.
+      # netboot-minimal.nix's own chain (netboot-base.nix) already imports
+      # profiles/installation-device.nix independently, so common.nix's
+      # initialHashedPassword override (which assumes that profile is
+      # present) still applies correctly without iso.nix in the mix.
+      #
+      # networking.hostName is set explicitly (rather than left at nixpkgs'
+      # own "nixos" default) so this image's generated system name
+      # (nixos-system-auto-installer-*) matches its iPXE menu entry —
+      # see modules/build-types/pxe-boot.nix's :auto-installer item — and
+      # its staged directory, /srv/pxe/http/auto-installer.
       netbootSystem = nixpkgs.lib.nixosSystem {
         inherit system;
         modules = [
-          ./modules/installer/iso.nix
+          ./modules/installer/common.nix
           ({ modulesPath, ... }: {
             imports = [
               (modulesPath + "/installer/netboot/netboot-minimal.nix")
             ];
           })
+          { networking.hostName = "auto-installer"; }
         ];
         specialArgs = { inherit vars; };
       };
 
+      # A genuinely vanilla NixOS minimal netboot image: nixpkgs'
+      # netboot-minimal.nix on its own, with none of this flake's
+      # auto-installer wiring (no common.nix — no auto-install.sh, no
+      # baked host keys, no custom users/passwords). Built from source via
+      # the same nixosSystem + netboot-minimal.nix path as netbootSystem
+      # above, so both go through an identical build mechanism; the only
+      # difference is what's composed in. hostName again matches this
+      # image's iPXE menu entry (:nixos-minimal) and staged directory
+      # (/srv/pxe/http/nixos-minimal).
+      netbootMinimalSystem = nixpkgs.lib.nixosSystem {
+        inherit system;
+        modules = [
+          ({ modulesPath, ... }: {
+            imports = [
+              (modulesPath + "/installer/netboot/netboot-minimal.nix")
+            ];
+          })
+          { networking.hostName = "nixos-minimal"; }
+        ];
+      };
+
     in
     {
 
@@ -153,6 +195,15 @@
           { name = "initrd"; path = netbootSystem.config.system.build.netbootRamdisk; }
           { name = "kernel"; path = netbootSystem.config.system.build.kernel; }
         ];
+
+        # Vanilla NixOS minimal netboot bundle — see netbootMinimalSystem
+        # above. Staged onto the pxe-boot host alongside packages.pxe by
+        # modules/pxe-boot/stage-installer-artifacts.nix.
+        pxe-minimal = pkgs.linkFarm "pxe-minimal" [
+          { name = "netboot.ipxe"; path = netbootMinimalSystem.config.system.build.netbootIpxeScript; }
+          { name = "initrd"; path = netbootMinimalSystem.config.system.build.netbootRamdisk; }
+          { name = "kernel"; path = netbootMinimalSystem.config.system.build.kernel; }
+        ];
       };
     };
 }

modules/build-types/pxe-boot.nix

@@ -68,15 +68,19 @@ let
     set base ${pxeBaseUrl}
 
     menu PXE Boot Menu
-    item nixos NixOS Installer
+    item auto-installer NixOS Auto-Installer
+    item nixos-minimal NixOS Minimal
     item rescue Rescue Environment
     item shell iPXE Shell
     item reboot Reboot
 
     choose target && goto ''${target}
 
-    :nixos
-    chain ''${base}/nixos/netboot.ipxe
+    :auto-installer
+    chain ''${base}/auto-installer/netboot.ipxe
+
+    :nixos-minimal
+    chain ''${base}/nixos-minimal/netboot.ipxe
 
     :rescue
     chain ''${base}/systemrescue.ipxe
@@ -129,7 +133,8 @@ in
     "d ${pxeRoot} 0755 root root -"
     "d ${httpRoot} 0755 root root -"
     "d ${httpRoot}/images 0755 root root -"
-    "d ${httpRoot}/nixos 0755 root root -"
+    "d ${httpRoot}/auto-installer 0755 root root -"
+    "d ${httpRoot}/nixos-minimal 0755 root root -"
     "d ${httpRoot}/systemrescue 0755 root root -"
     "d ${httpRoot}/ubuntu 0755 root root -"
     "d ${httpRoot}/rescue 0755 root root -"

modules/common/configuration.nix

@@ -13,24 +13,6 @@
 
   networking.networkmanager.enable = true; # Easiest to use and most distros use this by default.
 
-  # No host declares a DNS search domain anywhere else, and cross-host
-  # references throughout this repo (vars.nfsServerHost, vars.nixCacheHost,
-  # vars.dockerHost, ...) are bare short names, not FQDNs -- resolving them
-  # depends entirely on whatever network stack happens to be in play
-  # picking up the DHCP-advertised domain as a search suffix. NetworkManager
-  # does that by default, which is why this went unnoticed on
-  # NetworkManager-managed hosts, but LXC containers (modules/platforms/lxc.nix
-  # force-disables NetworkManager and Proxmox writes their systemd-networkd
-  # config itself) never get one. Confirmed live on lxc-docker: systemd-resolved
-  # had no search domain for eth0, "server" failed to resolve
-  # ("Name or service not known") while "server.sweet.home" resolved fine via
-  # the same DNS server, so every NFS mount in modules/docker/mount-data.nix
-  # failed even after fixing the automount/mount=nfs bugs. This applies the
-  # search domain globally via systemd-resolved's own config rather than the
-  # per-link DHCP path, so it isn't at the mercy of whichever component owns
-  # a given host's interface file.
-  networking.search = [ vars.homeDomain ];
-
   # Recommended over the true default (bypasses ZFS's own import safeguards)
   # per the option's own docs; matches hosts/docker/host.nix and
   # modules/services/zfs/enable-service.nix, which already set this

modules/docker/mount-data.nix

@@ -9,11 +9,25 @@ let
   # (the VM platforms rely on automount itself to get that same
   # non-blocking behavior, so they don't need `nofail` too).
   automountOpts = if config.boot.isContainer then [ "nofail" ] else [ "x-systemd.automount" ];
+
+  # A bare hostname here never resolves reliably: systemd-resolved only
+  # ever tries LLMNR for single-label names (never DNS, regardless of any
+  # configured search domain), and a *global* search domain (the first fix
+  # attempted here) backfires worse -- confirmed live on lxc-docker, adding
+  # `networking.search` made systemd-resolved prioritize its domain-matched
+  # but server-less global scope over eth0's correctly-configured one for
+  # every "*.sweet.home" query, silently sending them to public fallback
+  # DNS instead. `resolvectl query --interface=eth0 server.sweet.home`
+  # resolved fine throughout, proving the LAN DNS server was never the
+  # problem -- only the ambient, unqualified device string was. Using the
+  # FQDN directly sidesteps all of that, matching the pattern
+  # ../raspi/mount-data.nix already uses for the same reason.
+  nfsServer = "${vars.nfsServerHost}.${vars.homeDomain}";
 in
 {
   fileSystems = {
     ${vars.nfsShares.dockerConfig.mountpoint} = {
-      device = "${vars.nfsServerHost}:${vars.storageRoot}/${vars.nfsShares.dockerConfig.subpath}";
+      device = "${nfsServer}:${vars.storageRoot}/${vars.nfsShares.dockerConfig.subpath}";
       fsType = "nfs";
 
       options = [
@@ -24,7 +38,7 @@ in
     };
 
     ${vars.nfsShares.dockerDatabases.mountpoint} = {
-      device = "${vars.nfsServerHost}:${vars.storageRoot}/${vars.nfsShares.dockerDatabases.subpath}";
+      device = "${nfsServer}:${vars.storageRoot}/${vars.nfsShares.dockerDatabases.subpath}";
       fsType = "nfs";
 
       options = [
@@ -35,7 +49,7 @@ in
     };
 
     ${vars.nfsShares.dockerVolumes.mountpoint} = {
-      device = "${vars.nfsServerHost}:${vars.storageRoot}/${vars.nfsShares.dockerVolumes.subpath}";
+      device = "${nfsServer}:${vars.storageRoot}/${vars.nfsShares.dockerVolumes.subpath}";
       fsType = "nfs";
 
       options = [
@@ -46,7 +60,7 @@ in
     };
 
     ${vars.nfsShares.nextcloudData.mountpoint} = {
-      device = "${vars.nfsServerHost}:${vars.storageRoot}/${vars.nfsShares.nextcloudData.subpath}";
+      device = "${nfsServer}:${vars.storageRoot}/${vars.nfsShares.nextcloudData.subpath}";
       fsType = "nfs";
 
       options = [
@@ -57,7 +71,7 @@ in
     };
 
     ${vars.nfsShares.raspiVolumes.mountpoint} = {
-      device = "${vars.nfsServerHost}:${vars.storageRoot}/${vars.nfsShares.raspiVolumes.subpath}";
+      device = "${nfsServer}:${vars.storageRoot}/${vars.nfsShares.raspiVolumes.subpath}";
       fsType = "nfs";
 
       options = [

modules/platforms/lxc.nix

@@ -1,4 +1,4 @@
-{ lib, modulesPath, flakeTarget, ... }:
+{ config, lib, modulesPath, flakeTarget, ... }:
 
 let
   # Bakes this exact flake target's pre-generated SSH host key straight
@@ -58,8 +58,28 @@ in
     # host.nix declares each host's real hostname (networking.hostName);
     # keep that instead of letting Proxmox's ambient container config win.
     manageHostName = true;
-    # Unprivileged matches how these containers are actually created.
-    privileged = false;
+    # Unprivileged by default -- matches how these containers are actually
+    # created (scripts/proxmox/create-proxmox-resource.sh reads this value
+    # back to decide `pct create`'s --unprivileged flag, so the two stay
+    # in sync).
+    #
+    # lxc-docker is the one exception: the kernel's NFS client doesn't set
+    # FS_USERNS_MOUNT, so mounting NFS from inside *any* non-init user
+    # namespace -- which is exactly what an unprivileged container's
+    # UID-mapped root runs in -- is rejected at the VFS layer with EPERM,
+    # no matter what Proxmox's own `mount=nfs;nfs4` container feature
+    # allows at the AppArmor layer (confirmed live: TCP to the NFS server
+    # succeeds, the server's export table matches the container's IP, and
+    # `mount.nfs: Operation not permitted` still fires immediately with no
+    # corresponding denial anywhere in the server's logs -- a kernel-level
+    # rejection, not a network or export-permission one). Keying off
+    # hostName rather than something docker-build-type-specific because
+    # modules/build-types/docker.nix is also composed for linode-docker/
+    # proxmox-docker, which don't import proxmox-lxc.nix at all --setting
+    # this option there would break their eval with "option does not
+    # exist" regardless of any mkIf guard, since mkIf only makes a value
+    # conditional, not whether the option needs to exist somewhere.
+    privileged = config.networking.hostName == "docker";
   };
 
   boot.loader = {

modules/pxe-boot/stage-installer-artifacts.nix

@@ -1,23 +1,32 @@
-{ netbootSystem, ... }:
+{ netbootSystem, netbootMinimalSystem, ... }:
 
 let
   # config.system.build.kernel and .netbootRamdisk are directories, not the
   # files themselves — nixpkgs' own system.build.kexecTree does the same
   # ${...}/<file> dereference for the same reason.
-  inherit (netbootSystem.config.system.boot.loader) kernelFile;
+  mkStageRules = { dirName, system }:
+    let
+      inherit (system.config.system.boot.loader) kernelFile;
+      dir = "/srv/pxe/http/${dirName}";
     in
-{
-  # Builds this flake's own installer netboot image (the same one
-  # `nix build .#pxe` produces) and stages it where menu.ipxe's :nixos
-  # entry expects it, so the pxe-boot host is self-contained — no manual
-  # operator step to populate /srv/pxe/http/nixos after deploy.
-  systemd.tmpfiles.rules = [
+    [
       # Declared here too (not just in build-types/pxe-boot.nix) so this
       # module's C+ rules don't depend on cross-module list-merge ordering —
       # tmpfiles' C type needs the target directory to already exist.
-    "d /srv/pxe/http/nixos 0755 root root -"
-    "C+ /srv/pxe/http/nixos/${kernelFile} 0644 root root - ${netbootSystem.config.system.build.kernel}/${kernelFile}"
-    "C+ /srv/pxe/http/nixos/initrd 0644 root root - ${netbootSystem.config.system.build.netbootRamdisk}/initrd"
-    "C+ /srv/pxe/http/nixos/netboot.ipxe 0644 root root - ${netbootSystem.config.system.build.netbootIpxeScript}/netboot.ipxe"
+      "d ${dir} 0755 root root -"
+      "C+ ${dir}/${kernelFile} 0644 root root - ${system.config.system.build.kernel}/${kernelFile}"
+      "C+ ${dir}/initrd 0644 root root - ${system.config.system.build.netbootRamdisk}/initrd"
+      "C+ ${dir}/netboot.ipxe 0644 root root - ${system.config.system.build.netbootIpxeScript}/netboot.ipxe"
     ];
+in
+{
+  # Builds this flake's own installer netboot image (the same one
+  # `nix build .#pxe` produces) plus the vanilla NixOS minimal netboot image
+  # (`nix build .#pxe-minimal`), and stages both where menu.ipxe's
+  # :auto-installer / :nixos-minimal entries expect them, so the pxe-boot
+  # host is self-contained — no manual operator step to populate
+  # /srv/pxe/http after deploy.
+  systemd.tmpfiles.rules =
+    mkStageRules { dirName = "auto-installer"; system = netbootSystem; }
+    ++ mkStageRules { dirName = "nixos-minimal"; system = netbootMinimalSystem; };
 }

scripts/env.sh

@@ -6,12 +6,22 @@
 # environment (e.g. PROXMOX_STORAGE=tank-nvme ./scripts/proxmox/create-proxmox-resource.sh ...)
 # since each one only sets a default if unset.
 
-# SSH-reachable Proxmox node that scripts/proxmox/create-proxmox-resource.sh runs
-# pct/qm on. Matches the Proxmox web UI hostname already used in
-# hosts/nixos/home.nix's desktop shortcuts (pve.<homeDomain> from
-# variables.nix) -- change this if that's not actually reachable over SSH,
-# or if you're targeting a different node in a multi-node cluster.
-: "${PROXMOX_HOST:=pve.sweet.home}"
+# Two SSH-reachable Proxmox nodes exist on the LAN:
+#   - pve1.sweet.home      -- production. Real, live VMs/containers.
+#   - pve-test.sweet.home  -- sandbox/test node, for scratch VMs/containers
+#     that don't belong on production.
+#
+# PROXMOX_HOST is what scripts/proxmox/create-proxmox-resource.sh actually
+# targets by default -- overridable per-invocation with --node <hostname>,
+# or per-variable as usual (e.g. PROXMOX_HOST=$PVE_TEST_HOST). It defaults
+# to production, matching this repo's behavior before pve-test existed --
+# see CLAUDE.md's "Two Proxmox nodes" section for the policy on which
+# situations should target which node (in particular: Claude defaults to
+# pve-test, not this variable's own default, unless explicitly told
+# otherwise).
+: "${PVE1_HOST:=pve1.sweet.home}"
+: "${PVE_TEST_HOST:=pve-test.sweet.home}"
+: "${PROXMOX_HOST:=$PVE1_HOST}"
 : "${PROXMOX_SSH_USER:=root}"
 
 # Where this flake repo lives on the Proxmox node itself.
@@ -25,7 +35,7 @@
 # Storage pool names -- Proxmox's own stock-install defaults, but this
 # varies a lot by setup (ZFS pool name, custom LVM-thin volume, etc.).
 # Verify with `pvesm status` on the node and correct these if wrong.
-: "${PROXMOX_STORAGE:=local-lvm}"     # VM disks / CT rootfs
+: "${PROXMOX_STORAGE:=local-zfs}"     # VM disks / CT rootfs
 : "${PROXMOX_ISO_STORAGE:=local}"     # uploaded images/ISOs/CT templates
 
 : "${PROXMOX_BRIDGE:=vmbr0}"
@@ -63,10 +73,10 @@
 # nothing here forces a mount to happen.
 : "${PROXMOX_DEFAULT_LXC_FEATURES:=nesting=1,keyctl=1,mount=nfs;nfs4}"
 
-export PROXMOX_HOST PROXMOX_SSH_USER PROXMOX_STORAGE PROXMOX_ISO_STORAGE \
-  PROXMOX_BRIDGE PROXMOX_DEFAULT_CORES PROXMOX_DEFAULT_MEMORY_MB \
-  PROXMOX_DEFAULT_LXC_DISK_GB PROXMOX_DEFAULT_LXC_FEATURES \
-  PROXMOX_REMOTE_REPO_DIR
+export PVE1_HOST PVE_TEST_HOST PROXMOX_HOST PROXMOX_SSH_USER PROXMOX_STORAGE \
+  PROXMOX_ISO_STORAGE PROXMOX_BRIDGE PROXMOX_DEFAULT_CORES \
+  PROXMOX_DEFAULT_MEMORY_MB PROXMOX_DEFAULT_LXC_DISK_GB \
+  PROXMOX_DEFAULT_LXC_FEATURES PROXMOX_REMOTE_REPO_DIR
 
 # Matches variables.nix's nixCacheHost -- update both if it ever changes.
 : "${NIX_CACHE_HOST:=nix-cache}"

scripts/lib/nix-eval.sh

@@ -34,3 +34,21 @@ flake_target_hostname() {
   nix eval --raw "${NIX_EVAL_FLAGS[@]}" \
     "${flake_ref}#nixosConfigurations.${target}.config.networking.hostName" 2>/dev/null
 }
+
+# flake_target_lxc_privileged <flake_ref> <target>
+# Prints "true" or "false" for one lxc-* target's config.proxmoxLXC.privileged
+# (modules/platforms/lxc.nix is the single source of truth -- e.g.
+# lxc-docker sets this true so it can NFS-mount; every other lxc-* host
+# stays unprivileged). Only meaningful for lxc-* targets -- the option
+# doesn't exist for linode-*/proxmox-* (nixpkgs' proxmox-lxc.nix, which
+# declares it, is only ever imported by modules/platforms/lxc.nix). Empty
+# (not an error under set -e) if the eval fails.
+flake_target_lxc_privileged() {
+  local flake_ref="$1" target="$2"
+  # Not --raw: the option is a Nix boolean, and --raw can only coerce
+  # strings ("cannot coerce a Boolean to a string"). Plain `nix eval`
+  # prints a bare `true`/`false` for a boolean, which is exactly the
+  # string this needs.
+  nix eval "${NIX_EVAL_FLAGS[@]}" \
+    "${flake_ref}#nixosConfigurations.${target}.config.proxmoxLXC.privileged" 2>/dev/null
+}

scripts/proxmox/clone-pve1-to-pve-test.sh

@@ -0,0 +1,221 @@
+#!/usr/bin/env bash
+# Ad hoc clone of a single VM/CT from pve1 (production) to pve-test
+# (sandbox), via vzdump + qmrestore/pct restore -- not a general-purpose
+# backup tool, just a quick "give me a disposable copy of this thing on
+# pve-test" for testing against real-ish data without touching prod.
+#
+# Flow:
+#   1. vzdump the resource on pve1 into its "local" storage (--mode
+#      snapshot by default, so the source keeps running throughout --
+#      see --mode below for when that's not possible).
+#   2. Stream the resulting archive straight from pve1 to pve-test
+#      (ssh pve1 cat ... | ssh pve-test cat > ...) -- this machine is
+#      just the relay, no separate on-disk staging copy here.
+#   3. qmrestore / pct restore it on pve-test under --new-vmid (default:
+#      same VMID as the source -- pve-test is a separate node/cluster, so
+#      no collision unless that VMID is already in use there too).
+#      Always restored with --unique 1 (fresh MAC addresses) since the
+#      source is typically still running on the same LAN -- restoring
+#      with the *same* MAC would put two live guests on the wire with
+#      identical hardware addresses.
+#   4. Delete the vzdump archive from pve1's local storage and the
+#      relayed copy on pve-test, so neither node accumulates ad hoc
+#      backup files from this script. Only the pve1 original is
+#      preserved on any failure after step 1, so a failed
+#      transfer/restore can be retried without re-running the backup.
+#
+# This script's own defaults are pve1 -> pve-test, unlike
+# create-proxmox-resource.sh's --node (which defaults to production) --
+# see CLAUDE.md's "Two Proxmox nodes" section. pve1 is only ever touched
+# here after typing the source VMID back to confirm; pve-test is treated
+# as disposable, matching this repo's usual policy for that node.
+#
+# See --help for the full option list.
+set -euo pipefail
+
+repo_root="$(cd "$(dirname "$0")/../.." && pwd)"
+# shellcheck source=../env.sh
+source "${repo_root}/scripts/env.sh"
+# shellcheck source=../lib/confirm.sh
+source "${repo_root}/scripts/lib/confirm.sh"
+
+usage() {
+  cat <<EOF
+Usage: $0 --vmid <n> [options]
+
+  --vmid <n>              Required: VMID on the source node to clone.
+                           Kind (qemu VM vs LXC CT) is auto-detected.
+  --new-vmid <n>           VMID to restore as on the target node
+                           (default: same as --vmid).
+  --mode snapshot|suspend|stop
+                           vzdump backup mode (default: snapshot -- the
+                           source resource keeps running throughout;
+                           requires snapshot-capable storage, e.g.
+                           ZFS/LVM-thin/Ceph/qcow2). Fall back to
+                           "suspend" (brief pause) or "stop" (source
+                           goes down for the duration) if the source's
+                           storage doesn't support live snapshots --
+                           vzdump's own error will say so.
+  --source-node <host>      (default: \$PVE1_HOST, ${PVE1_HOST})
+  --target-node <host>       (default: \$PVE_TEST_HOST, ${PVE_TEST_HOST})
+  --source-storage <pool>      Where vzdump writes the backup on the
+                               source node (default: local).
+  --target-storage <pool>       Where the restored disk/rootfs lands on
+                               the target node (default: \$PROXMOX_STORAGE, ${PROXMOX_STORAGE}).
+  --keep-backup                  Don't delete the vzdump archive from
+                                 either node afterward (debugging aid).
+  --yes                            Skip the typed VMID confirmation
+                                   before touching the source node.
+  --dry-run                          Print the full plan and skip every
+                                     mutating step (vzdump, transfer,
+                                     restore, delete) and the confirm
+                                     prompt. Still makes read-only SSH
+                                     calls to look up the source kind
+                                     and check the target VMID is free
+                                     -- harmless on either node.
+  -h, --help
+EOF
+}
+
+vmid=""
+new_vmid=""
+mode="snapshot"
+source_node="$PVE1_HOST"
+target_node="$PVE_TEST_HOST"
+source_storage="local"
+target_storage="$PROXMOX_STORAGE"
+keep_backup=0
+skip_confirm=0
+dry_run=0
+
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+  --vmid) vmid="$2"; shift 2 ;;
+  --new-vmid) new_vmid="$2"; shift 2 ;;
+  --mode) mode="$2"; shift 2 ;;
+  --source-node) source_node="$2"; shift 2 ;;
+  --target-node) target_node="$2"; shift 2 ;;
+  --source-storage) source_storage="$2"; shift 2 ;;
+  --target-storage) target_storage="$2"; shift 2 ;;
+  --keep-backup) keep_backup=1; shift ;;
+  --yes) skip_confirm=1; shift ;;
+  --dry-run) dry_run=1; shift ;;
+  -h | --help) usage; exit 0 ;;
+  *) echo "Unknown option: $1" >&2; usage >&2; exit 1 ;;
+  esac
+done
+
+if [[ -z "$vmid" ]]; then
+  echo "ERROR: --vmid is required." >&2
+  usage >&2
+  exit 1
+fi
+if [[ "$mode" != "snapshot" && "$mode" != "suspend" && "$mode" != "stop" ]]; then
+  echo "ERROR: --mode must be snapshot, suspend, or stop." >&2
+  exit 1
+fi
+[[ -z "$new_vmid" ]] && new_vmid="$vmid"
+
+source_target="${PROXMOX_SSH_USER}@${source_node}"
+target_target="${PROXMOX_SSH_USER}@${target_node}"
+
+# No dry-run wrapper needed for the calls below: every mutating step
+# (vzdump, transfer, restore, delete) is reached only after the --dry-run
+# early-exit further down, so a plain `ssh` call is never in the dry-run
+# path.
+
+# --- identify the resource kind on the source node -----------------------
+echo "==> Looking up VMID ${vmid} on ${source_node}..."
+kind=""
+if ssh "$source_target" "qm status ${vmid}" >/dev/null 2>&1; then
+  kind="vm"
+elif ssh "$source_target" "pct status ${vmid}" >/dev/null 2>&1; then
+  kind="lxc"
+else
+  echo "ERROR: VMID ${vmid} doesn't exist on ${source_node} as either a VM or CT." >&2
+  exit 1
+fi
+echo "VMID ${vmid} on ${source_node} is a ${kind}."
+
+# --- refuse to clobber an existing resource on the target node -----------
+if ssh "$target_target" "qm status ${new_vmid}" >/dev/null 2>&1 \
+  || ssh "$target_target" "pct status ${new_vmid}" >/dev/null 2>&1; then
+  echo "ERROR: VMID ${new_vmid} already exists on ${target_node}. Pass --new-vmid" >&2
+  echo "with a free ID, or remove the existing resource there first." >&2
+  exit 1
+fi
+
+echo
+echo "Plan:"
+echo "  source: ${kind} VMID ${vmid} on ${source_node} (storage: ${source_storage}, mode: ${mode})"
+echo "  target: VMID ${new_vmid} on ${target_node} (storage: ${target_storage}, fresh MAC via --unique)"
+[[ "$keep_backup" -eq 1 ]] && echo "  backup archives are kept on both nodes afterward (--keep-backup)"
+
+if [[ "$dry_run" -eq 1 ]]; then
+  echo
+  echo "[dry-run] No backup, transfer, restore, or delete was performed."
+  exit 0
+fi
+
+if [[ "$skip_confirm" -ne 1 ]]; then
+  echo
+  if ! confirm_typed "$vmid" "Type the source VMID (${vmid}) to confirm backing it up from ${source_node}: "; then
+    echo "Cancelled -- input didn't match ${vmid}." >&2
+    exit 1
+  fi
+fi
+
+# --- vzdump on the source node --------------------------------------------
+echo
+echo "==> Backing up VMID ${vmid} on ${source_node} (mode=${mode}, storage=${source_storage})..."
+vzdump_log="$(ssh "$source_target" \
+  "vzdump ${vmid} --mode ${mode} --storage ${source_storage} --compress zstd" 2>&1)" \
+  || {
+    echo "$vzdump_log" >&2
+    echo "ERROR: vzdump failed on ${source_node}." >&2
+    exit 1
+  }
+echo "$vzdump_log"
+
+archive="$(echo "$vzdump_log" | grep -oP "creating vzdump archive '\K[^']+" | tail -n1)"
+if [[ -z "$archive" ]]; then
+  echo "ERROR: couldn't find the archive path in vzdump's output above." >&2
+  exit 1
+fi
+archive_basename="$(basename "$archive")"
+target_tmp_archive="/var/tmp/${archive_basename}"
+echo "Archive: ${archive}"
+
+# Always clean up the relayed copy on the target node, success or failure
+# -- it's only ever a working copy, restored or not.
+cleanup_target_tmp() {
+  if [[ "$keep_backup" -ne 1 ]]; then
+    ssh "$target_target" "rm -f '${target_tmp_archive}'" >/dev/null 2>&1 || true
+  fi
+}
+trap cleanup_target_tmp EXIT
+
+# --- relay the archive from source to target ------------------------------
+echo
+echo "==> Transferring archive to ${target_node}..."
+ssh "$source_target" "cat '${archive}'" | ssh "$target_target" "cat > '${target_tmp_archive}'"
+
+# --- restore on the target node --------------------------------------------
+echo
+echo "==> Restoring as VMID ${new_vmid} on ${target_node} (storage=${target_storage})..."
+if [[ "$kind" == "vm" ]]; then
+  ssh "$target_target" "qmrestore '${target_tmp_archive}' ${new_vmid} --storage ${target_storage} --unique 1"
+else
+  ssh "$target_target" "pct restore ${new_vmid} '${target_tmp_archive}' --storage ${target_storage} --unique 1"
+fi
+
+# --- clean up the source backup now that the restore succeeded -----------
+if [[ "$keep_backup" -ne 1 ]]; then
+  echo
+  echo "==> Deleting backup archive from ${source_node}'s ${source_storage} storage..."
+  ssh "$source_target" "rm -f '${archive}' '${archive}.notes' '${archive}.log'" >/dev/null 2>&1 || true
+fi
+
+echo
+echo "Done. VMID ${new_vmid} (${kind}) is now on ${target_node}, cloned from" \
+  "VMID ${vmid} on ${source_node}."

scripts/proxmox/create-proxmox-resource.sh

@@ -12,6 +12,17 @@
 # copies over the locally-managed host-keys/ (gitignored, so a git pull
 # alone wouldn't carry it) before building.
 #
+# --node (default: $PROXMOX_HOST, see scripts/env.sh) picks which of the two
+# LAN Proxmox nodes this runs against: production, pve1.sweet.home
+# ($PVE1_HOST, PROXMOX_HOST's own default), or the sandbox node,
+# pve-test.sweet.home ($PVE_TEST_HOST) -- pass --node "$PVE_TEST_HOST" (or
+# set PROXMOX_HOST=$PVE_TEST_HOST) to target the sandbox instead. See
+# CLAUDE.md's "Two Proxmox nodes" section: an agent session should default
+# to pve-test and only touch pve1 when the operator has explicitly said so
+# for the current task -- this script itself doesn't enforce that (its own
+# default is production, matching this repo's behavior before pve-test
+# existed), it's a policy for whoever/whatever is driving it.
+#
 # Usage:
 #   scripts/proxmox/create-proxmox-resource.sh --type lxc|vm --host <name> [options]
 #   scripts/proxmox/create-proxmox-resource.sh --type lxc|vm --list
@@ -120,7 +131,9 @@ Shared:
   --iso-storage <pool>            (default: \$PROXMOX_ISO_STORAGE, ${PROXMOX_ISO_STORAGE})
   --bridge <bridge>                (default: \$PROXMOX_BRIDGE, ${PROXMOX_BRIDGE})
   --node <host>                     Proxmox node to SSH into (default:
-                                    \$PROXMOX_HOST, ${PROXMOX_HOST})
+                                    \$PROXMOX_HOST, ${PROXMOX_HOST} --
+                                    production; the sandbox node is
+                                    \$PVE_TEST_HOST, ${PVE_TEST_HOST}).
   --dry-run                           Print the full plan; touch nothing
                                       local or remote, no prompts.
   -h, --help
@@ -766,13 +779,22 @@ if [[ "$type" == "lxc" ]]; then
   # 512M default otherwise (confirmed live: --memory 2048 left swap at
   # 512). Default to matching whatever --memory resolved to above.
   local_swap="${swap:-$memory}"
-  # --unprivileged 1: modules/platforms/lxc.nix sets proxmoxLXC.privileged
-  # = false, so the NixOS config inside the image assumes it's running as
-  # an unprivileged container (cgroup/capability/mount expectations baked
-  # in at boot). `pct create`'s own CLI default for this flag is
-  # privileged (unlike the web UI, which defaults its checkbox the other
-  # way) -- leaving it unset creates a privileged container running a
-  # NixOS config that assumes unprivileged, a real mismatch.
+  # --unprivileged: read back from modules/platforms/lxc.nix's own
+  # proxmoxLXC.privileged (via flake_target_lxc_privileged) rather than
+  # hardcoded, since that's no longer the same for every lxc-* target --
+  # lxc-docker sets it true so the container's NFS mounts work at all (the
+  # kernel's NFS client can't mount from inside any unprivileged
+  # container's user namespace, no matter what AppArmor allows -- see that
+  # option's own comment). The NixOS config inside the image bakes in
+  # cgroup/capability/mount expectations matching whichever value it was
+  # built with, so this must stay in sync with it -- `pct create`'s own
+  # CLI default for this flag is privileged (unlike the web UI, which
+  # defaults its checkbox the other way), so leaving it unset would create
+  # a privileged container running a NixOS config that assumes
+  # unprivileged for every target except lxc-docker, a real mismatch.
+  privileged_eval="$(flake_target_lxc_privileged "$repo_root" "$flake_target")"
+  unprivileged_flag=1
+  [[ "$privileged_eval" == "true" ]] && unprivileged_flag=0
   #
   # --features nesting=1,keyctl=1: required for a modern (v247+) systemd
   # guest to actually boot unprivileged -- confirmed live: without this,
@@ -790,7 +812,7 @@ if [[ "$type" == "lxc" ]]; then
   # hands the whole string to `ssh` as a single command for the *remote*
   # shell to parse -- unquoted, that `;` would be read as a remote
   # command separator and silently truncate this into two commands.
-  create_cmd="pct create ${vmid} ${iso_storage}:vztmpl/${remote_filename} --unprivileged 1 --features '${PROXMOX_DEFAULT_LXC_FEATURES}' --rootfs ${storage}:${local_disk_size} --hostname ${name} --cores ${cores} --memory ${memory} --swap ${local_swap} --net0 name=eth0,bridge=${bridge},ip=dhcp"
+  create_cmd="pct create ${vmid} ${iso_storage}:vztmpl/${remote_filename} --unprivileged ${unprivileged_flag} --features '${PROXMOX_DEFAULT_LXC_FEATURES}' --rootfs ${storage}:${local_disk_size} --hostname ${name} --cores ${cores} --memory ${memory} --swap ${local_swap} --net0 name=eth0,bridge=${bridge},ip=dhcp"
   remote "$create_cmd"
   remote "pct start ${vmid}"
 else

scripts/proxmox/migrate-vm.sh

@@ -0,0 +1,347 @@
+#!/usr/bin/env bash
+# Move a single VM/CT from one Proxmox node to another via vzdump +
+# qmrestore/pct restore -- the same relay pattern as
+# clone-pve1-to-pve-test.sh, but for a *migration* (the guest ends up
+# living on the target node only) rather than a *clone* (a disposable
+# copy, source untouched, fresh MAC).
+#
+# There is no real `qm migrate` here: that command only works between
+# nodes in the same Proxmox cluster with shared/replicated storage, which
+# pve1 and pve-test are not (see CLAUDE.md's "Two Proxmox nodes" section --
+# they're deliberately separate, unclustered nodes). This script is the
+# closest equivalent across two independent nodes, built out of the same
+# primitives as the clone script. It is NOT a true live migration -- read
+# the caveat below before using it for anything where losing a few
+# seconds/minutes of writes is unacceptable.
+#
+# Flow:
+#   1. vzdump the resource on the source node (--mode snapshot by
+#      default, so the source keeps running throughout the bulk copy --
+#      this is the part that's "live": the guest stays reachable while
+#      its disk data is captured and shipped to the target).
+#   2. Stream the archive straight from the source node to the target
+#      node (ssh source cat ... | ssh target cat > ...), same relay this
+#      machine does in the clone script -- no separate on-disk staging
+#      copy here either.
+#   3. qmrestore / pct restore it on the target node under the same VMID
+#      by default (--new-vmid to pick a different one). Restored WITHOUT
+#      --unique (i.e. original MAC preserved) unless --unique is passed
+#      explicitly -- unlike the clone script, the source is being retired,
+#      not left running alongside a copy, so there's no address collision
+#      to avoid. qmrestore/pct restore don't start the guest, so nothing
+#      is live on the target node yet.
+#   4. Cutover: stop the guest on the source node, then (unless --no-start)
+#      start it on the target node. This is the only real downtime window
+#      -- everything before this point runs with the source guest still
+#      up.
+#   5. Unless --remove-source is passed, the now-stopped guest is LEFT ON
+#      the source node as a safety net (config + disk intact, just
+#      powered off) -- easy to start back up if the target copy turns out
+#      to be broken. Pass --remove-source to actually destroy it there
+#      once you've verified the target is good; this step gets its own
+#      typed confirmation since qm destroy/pct destroy is irreversible.
+#   6. Delete the vzdump archive from both nodes' storage (unless
+#      --keep-backup), same as the clone script -- neither node
+#      accumulates ad hoc backup files from this script.
+#
+# --- IMPORTANT CAVEAT: this is not byte-perfect live migration ---
+# vzdump is not incremental. With the default --mode snapshot, the source
+# guest keeps running (and can keep writing to disk) for the entire time
+# between when the snapshot is taken and when this script stops it at
+# cutover. Any writes in that window are NOT captured in the archive and
+# will NOT exist on the target. For a large disk this window can be
+# minutes. If the workload can't tolerate that:
+#   - use --mode stop (or --mode suspend) instead, which makes the source
+#     guest go down *before* vzdump reads its disk, so the archive is
+#     exactly the state being migrated and cutover has nothing left to
+#     lose -- at the cost of the guest being down for the whole backup
+#     duration instead of just the final cutover.
+#   - or don't use this script -- put both nodes in an actual Proxmox
+#     cluster with shared storage and use `qm migrate --online` instead,
+#     which is the real thing this script is only approximating.
+#
+# This script's own defaults are pve1 -> pve-test, matching
+# clone-pve1-to-pve-test.sh and CLAUDE.md's "Two Proxmox nodes" section --
+# but unlike that script, this one is a generic node-to-node mover: pass
+# --source-node/--target-node for any other pair. Regardless of node
+# names, the source resource is only ever touched here after typing the
+# source VMID back to confirm, and destroying it on the source node
+# (--remove-source) requires a second, separate typed confirmation.
+#
+# See --help for the full option list.
+set -euo pipefail
+
+repo_root="$(cd "$(dirname "$0")/../.." && pwd)"
+# shellcheck source=../env.sh
+source "${repo_root}/scripts/env.sh"
+# shellcheck source=../lib/confirm.sh
+source "${repo_root}/scripts/lib/confirm.sh"
+
+usage() {
+  cat <<EOF
+Usage: $0 --vmid <n> [options]
+
+  --vmid <n>              Required: VMID on the source node to migrate.
+                           Kind (qemu VM vs LXC CT) is auto-detected.
+  --new-vmid <n>           VMID to restore as on the target node
+                           (default: same as --vmid).
+  --mode snapshot|suspend|stop
+                           vzdump backup mode (default: snapshot -- the
+                           source resource keeps running until cutover;
+                           requires snapshot-capable storage, e.g.
+                           ZFS/LVM-thin/Ceph/qcow2). Use "stop" for a
+                           byte-perfect migration with no post-snapshot
+                           write gap (source goes down for the whole
+                           backup duration instead of just cutover) --
+                           see the caveat at the top of this script.
+  --source-node <host>      (default: \$PVE1_HOST, ${PVE1_HOST})
+  --target-node <host>       (default: \$PVE_TEST_HOST, ${PVE_TEST_HOST})
+  --source-storage <pool>      Where vzdump writes the backup on the
+                               source node (default: local).
+  --target-storage <pool>       Where the restored disk/rootfs lands on
+                               the target node (default: \$PROXMOX_STORAGE, ${PROXMOX_STORAGE}).
+  --unique                        Restore with a fresh MAC address
+                                  (--unique 1), as the clone script always
+                                  does. Off by default here since the
+                                  source is being retired, not left
+                                  running alongside the target.
+  --no-start                        Don't start the guest on the target
+                                    node after cutover (default: start
+                                    it).
+  --remove-source                    Destroy the guest on the source
+                                     node (qm destroy/pct destroy) after
+                                     a successful cutover, instead of
+                                     just leaving it stopped there.
+                                     Irreversible -- prompts for its own
+                                     typed confirmation unless --yes.
+  --keep-backup                  Don't delete the vzdump archive from
+                                 either node afterward (debugging aid).
+  --yes                            Skip all typed confirmations (initial
+                                   backup, cutover, and --remove-source
+                                   if passed).
+  --dry-run                          Print the full plan and skip every
+                                     mutating step (vzdump, transfer,
+                                     restore, cutover, destroy, delete)
+                                     and every confirm prompt. Still makes
+                                     read-only SSH calls to look up the
+                                     source kind and check the target
+                                     VMID is free -- harmless on either
+                                     node.
+  -h, --help
+EOF
+}
+
+vmid=""
+new_vmid=""
+mode="snapshot"
+source_node="$PVE1_HOST"
+target_node="$PVE_TEST_HOST"
+source_storage="local"
+target_storage="$PROXMOX_STORAGE"
+unique=0
+start_target=1
+remove_source=0
+keep_backup=0
+skip_confirm=0
+dry_run=0
+
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+  --vmid) vmid="$2"; shift 2 ;;
+  --new-vmid) new_vmid="$2"; shift 2 ;;
+  --mode) mode="$2"; shift 2 ;;
+  --source-node) source_node="$2"; shift 2 ;;
+  --target-node) target_node="$2"; shift 2 ;;
+  --source-storage) source_storage="$2"; shift 2 ;;
+  --target-storage) target_storage="$2"; shift 2 ;;
+  --unique) unique=1; shift ;;
+  --no-start) start_target=0; shift ;;
+  --remove-source) remove_source=1; shift ;;
+  --keep-backup) keep_backup=1; shift ;;
+  --yes) skip_confirm=1; shift ;;
+  --dry-run) dry_run=1; shift ;;
+  -h | --help) usage; exit 0 ;;
+  *) echo "Unknown option: $1" >&2; usage >&2; exit 1 ;;
+  esac
+done
+
+if [[ -z "$vmid" ]]; then
+  echo "ERROR: --vmid is required." >&2
+  usage >&2
+  exit 1
+fi
+if [[ "$mode" != "snapshot" && "$mode" != "suspend" && "$mode" != "stop" ]]; then
+  echo "ERROR: --mode must be snapshot, suspend, or stop." >&2
+  exit 1
+fi
+[[ -z "$new_vmid" ]] && new_vmid="$vmid"
+
+source_target="${PROXMOX_SSH_USER}@${source_node}"
+target_target="${PROXMOX_SSH_USER}@${target_node}"
+
+# No dry-run wrapper needed for the calls below: every mutating step
+# (vzdump, transfer, restore, cutover, destroy, delete) is reached only
+# after the --dry-run early-exit further down, so a plain `ssh` call is
+# never in the dry-run path.
+
+# --- identify the resource kind on the source node -----------------------
+echo "==> Looking up VMID ${vmid} on ${source_node}..."
+kind=""
+if ssh "$source_target" "qm status ${vmid}" >/dev/null 2>&1; then
+  kind="vm"
+elif ssh "$source_target" "pct status ${vmid}" >/dev/null 2>&1; then
+  kind="lxc"
+else
+  echo "ERROR: VMID ${vmid} doesn't exist on ${source_node} as either a VM or CT." >&2
+  exit 1
+fi
+echo "VMID ${vmid} on ${source_node} is a ${kind}."
+
+# --- refuse to clobber an existing resource on the target node -----------
+if ssh "$target_target" "qm status ${new_vmid}" >/dev/null 2>&1 \
+  || ssh "$target_target" "pct status ${new_vmid}" >/dev/null 2>&1; then
+  echo "ERROR: VMID ${new_vmid} already exists on ${target_node}. Pass --new-vmid" >&2
+  echo "with a free ID, or remove the existing resource there first." >&2
+  exit 1
+fi
+
+echo
+echo "Plan:"
+echo "  source: ${kind} VMID ${vmid} on ${source_node} (storage: ${source_storage}, mode: ${mode})"
+echo "  target: VMID ${new_vmid} on ${target_node} (storage: ${target_storage}," \
+  "$([[ "$unique" -eq 1 ]] && echo "fresh MAC via --unique" || echo "original MAC preserved"))"
+echo "  cutover: stop VMID ${vmid} on ${source_node}," \
+  "$([[ "$start_target" -eq 1 ]] && echo "then start VMID ${new_vmid} on ${target_node}" || echo "target left stopped (--no-start)")"
+if [[ "$remove_source" -eq 1 ]]; then
+  echo "  after cutover: DESTROY VMID ${vmid} on ${source_node} (--remove-source, irreversible)"
+else
+  echo "  after cutover: source VMID ${vmid} left stopped (but intact) on ${source_node}"
+fi
+[[ "$keep_backup" -eq 1 ]] && echo "  backup archives are kept on both nodes afterward (--keep-backup)"
+
+if [[ "$dry_run" -eq 1 ]]; then
+  echo
+  echo "[dry-run] No backup, transfer, restore, cutover, destroy, or delete was performed."
+  exit 0
+fi
+
+if [[ "$skip_confirm" -ne 1 ]]; then
+  echo
+  if ! confirm_typed "$vmid" "Type the source VMID (${vmid}) to confirm backing it up from ${source_node} for migration: "; then
+    echo "Cancelled -- input didn't match ${vmid}." >&2
+    exit 1
+  fi
+fi
+
+# --- vzdump on the source node --------------------------------------------
+echo
+echo "==> Backing up VMID ${vmid} on ${source_node} (mode=${mode}, storage=${source_storage})..."
+vzdump_log="$(ssh "$source_target" \
+  "vzdump ${vmid} --mode ${mode} --storage ${source_storage} --compress zstd" 2>&1)" \
+  || {
+    echo "$vzdump_log" >&2
+    echo "ERROR: vzdump failed on ${source_node}." >&2
+    exit 1
+  }
+echo "$vzdump_log"
+
+archive="$(echo "$vzdump_log" | grep -oP "creating vzdump archive '\K[^']+" | tail -n1)"
+if [[ -z "$archive" ]]; then
+  echo "ERROR: couldn't find the archive path in vzdump's output above." >&2
+  exit 1
+fi
+archive_basename="$(basename "$archive")"
+target_tmp_archive="/var/tmp/${archive_basename}"
+echo "Archive: ${archive}"
+
+# Always clean up the relayed copy on the target node, success or failure
+# -- it's only ever a working copy, restored or not.
+cleanup_target_tmp() {
+  if [[ "$keep_backup" -ne 1 ]]; then
+    ssh "$target_target" "rm -f '${target_tmp_archive}'" >/dev/null 2>&1 || true
+  fi
+}
+trap cleanup_target_tmp EXIT
+
+# --- relay the archive from source to target ------------------------------
+echo
+echo "==> Transferring archive to ${target_node}..."
+ssh "$source_target" "cat '${archive}'" | ssh "$target_target" "cat > '${target_tmp_archive}'"
+
+# --- restore on the target node --------------------------------------------
+echo
+echo "==> Restoring as VMID ${new_vmid} on ${target_node} (storage=${target_storage})..."
+restore_unique_flag=0
+[[ "$unique" -eq 1 ]] && restore_unique_flag=1
+if [[ "$kind" == "vm" ]]; then
+  ssh "$target_target" "qmrestore '${target_tmp_archive}' ${new_vmid} --storage ${target_storage} --unique ${restore_unique_flag}"
+else
+  ssh "$target_target" "pct restore ${new_vmid} '${target_tmp_archive}' --storage ${target_storage} --unique ${restore_unique_flag}"
+fi
+echo "Restored on ${target_node}. Source VMID ${vmid} on ${source_node} is still up --" \
+  "the guest hasn't moved yet."
+
+# --- cutover: stop source, start target -----------------------------------
+if [[ "$skip_confirm" -ne 1 ]]; then
+  echo
+  if ! confirm_typed "$vmid" "Type the source VMID (${vmid}) again to confirm CUTOVER (stop it on ${source_node}$([[ "$start_target" -eq 1 ]] && echo ", start VMID ${new_vmid} on ${target_node}")): "; then
+    echo "Cancelled before cutover -- input didn't match ${vmid}." >&2
+    echo "The restored (but not started) copy remains on ${target_node} as VMID ${new_vmid};" >&2
+    echo "the source on ${source_node} is untouched. Re-run cutover manually, or clean up" >&2
+    echo "the target copy if you no longer want it." >&2
+    exit 1
+  fi
+fi
+
+echo
+echo "==> Stopping VMID ${vmid} on ${source_node}..."
+if [[ "$kind" == "vm" ]]; then
+  ssh "$source_target" "qm stop ${vmid}"
+else
+  ssh "$source_target" "pct stop ${vmid}"
+fi
+
+if [[ "$start_target" -eq 1 ]]; then
+  echo
+  echo "==> Starting VMID ${new_vmid} on ${target_node}..."
+  if [[ "$kind" == "vm" ]]; then
+    ssh "$target_target" "qm start ${new_vmid}"
+  else
+    ssh "$target_target" "pct start ${new_vmid}"
+  fi
+fi
+
+# --- optionally destroy the now-stopped source resource -------------------
+if [[ "$remove_source" -eq 1 ]]; then
+  if [[ "$skip_confirm" -ne 1 ]]; then
+    echo
+    if ! confirm_typed "$vmid" "Type the source VMID (${vmid}) one more time to permanently DESTROY it on ${source_node}: "; then
+      echo "Cancelled -- input didn't match ${vmid}." >&2
+      echo "VMID ${vmid} on ${source_node} is left stopped (not destroyed). VMID ${new_vmid}" >&2
+      echo "on ${target_node} is up and running." >&2
+      exit 1
+    fi
+  fi
+  echo
+  echo "==> Destroying VMID ${vmid} on ${source_node}..."
+  if [[ "$kind" == "vm" ]]; then
+    ssh "$source_target" "qm destroy ${vmid} --purge"
+  else
+    ssh "$source_target" "pct destroy ${vmid} --purge"
+  fi
+fi
+
+# --- clean up backup archives now that the migration succeeded -----------
+if [[ "$keep_backup" -ne 1 ]]; then
+  echo
+  echo "==> Deleting backup archive from ${source_node}'s ${source_storage} storage..."
+  ssh "$source_target" "rm -f '${archive}' '${archive}.notes' '${archive}.log'" >/dev/null 2>&1 || true
+fi
+
+echo
+echo "Done. VMID ${new_vmid} (${kind}) is now on ${target_node}, migrated from" \
+  "VMID ${vmid} on ${source_node}."
+if [[ "$remove_source" -ne 1 ]]; then
+  echo "The source copy is stopped but still present on ${source_node} -- re-run with" \
+    "--remove-source once you've verified the target, or remove it manually."
+fi

secrets/common.yaml

@@ -5,103 +5,121 @@ sops:
     age:
         - enc: |
             -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSAzdUlybVpDamVxYk94MDEv
-            VjNiVjJkWjZvYjN1SUhMSG5ZOS91a3FrNDJRCi9IQWh4RVdwdkhwQjd6TTBhcW9J
-            alhLMlArM1VvVkE5VUxOQ1ZWNmw2SmcKLS0tIFRvWklUN0xxSFdKSk1vakExQ3Rv
-            OWpmSGRTUHlvVysvR2N4UHRYWHgvMHMKqapmFB4ct1FTPa1hMWyylvLycUvOEFop
-            enZI5SV1F86HOTEhK0QbnEW3jMl4ZXOtAlwcys2oE9a783MSgVPxeg==
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBQUFhWVFVlVlBnNE5FTnMz
+            VkxkTmxpRXlzZ3pSNTVZWFUrSllsYWo0alRnCkJSc25TYktSTFFJdkQydHcxOUlj
+            ajhQU1ZIb1lodEpHTnVhQjJ6WEthaDQKLS0tIDJCY1E2UVBaU3BoMzhXUXlIdnMv
+            djZTcE1rcWNTOXFPMmFDYTVoRGo4ZTQKYy8g6pqP3VpTKDIBPbnC8NzCdDvOCKnL
+            14kSrKmKlzefTrbkVyriz2Jdl2s0F374yfQQFreZ3m4AffSACCxziQ==
             -----END AGE ENCRYPTED FILE-----
           recipient: age1njap586hc0q43kr03g6c8eqhdsmk8zcafkl3f83xwlc2gqhlmfgs4tmwad
         - enc: |
             -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBEc0lHazMrVUR2UUlaQ21O
-            T2Jha0EvUmJzVC96Wksxay9GVlEwU2lZWVRRClBQdWVqTjNTYnBrQldMeWR0dWNz
-            QTN6T0VtQzhpM3UwKzVjUjl0Rkp4VzQKLS0tIEg5VzhUT1VTamQzZjNYT2orMnY0
-            Q1BpNzI2cVk3by9Ic3lrQ25sM05YMlkKXVofmCu7iI/my1o47p2eUzhXuP/V0NS8
-            mmIerz2v2uUwoS252qSU4a2vmxPOgWM0Os1vWsYamsapofQc6TaU2Q==
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBVQnpVWC9wcnIreG9GeE9U
+            UWhuRytkc2Flc0hyQm5yMjZnelNwaXhlWWc4CjBkWnd4cHNRRXQ0UXFkZGp4QlR1
+            eW5NNnE1WFhnb054M1pac2ZidFg4Y3MKLS0tIFdTNmk2V1l2WC9rUk8yd0ZnOEJS
+            VkNnejVGVUZPZkorQkltVEplN2FmdTAKRY7DPP5HeFQntn2f/fXLjU6M1V6iug86
+            BD09PI+T2DbIBQPotRZisw8IzHu9gY/O3+h0TccyIsXjI9wy/XPCAQ==
             -----END AGE ENCRYPTED FILE-----
           recipient: age19gfn2yedg76dmztm4hncr7vf3r3c9j0qpt4rap7y7gersjk4m3ks2lhd0e
         - enc: |
             -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBYZHVnTEk2SjUwdTdKdWNV
-            dVFPNDFvdzY2VUlzMU5nK0t0UlUxU3Ewa0g0CjJCd0ZrMjhRZldaUDNIS1pGZTZn
-            Z2h2MEJ6TUpna1NNdXl4R3d1S1djVDQKLS0tIFJqMlZ3b0Zzalp2N0hIc0NHMzU4
-            RGRoVjdvMEkxam9DRzdYaW16enAyQ3cKUZTDqvWnmEMKHhI430coKHw3raIiD/o1
-            1BtDWOmKFFcuaF5mRx/qTUjEwU5OZWOujqLSPoVbmsbhyxsG3XQSHg==
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBkczFSTVhxWHlIWjhRcWlV
+            V2JPQXd5Wnk5R3NwWC81T3Z0MW4vYnd5S1ZZClV1NlU1Tzd6UkxPQ2M4MmhLV01G
+            d3VIb0RhR1RiNTZqNjlQcmg2YjdPeGsKLS0tIDQ1RTFTWGN4MnEvWkRUR3VnN204
+            WVdFOXdmNC9FVFhBSGNEUUgyYWpYYzAKfdpeaFL/RrIbqpD9hNj8L7UxpmiBjE2I
+            go/dR2E1LLXsDPtnSuJb2EZYoFvSsjsIQQQDt+YwRv0fplRtssKdxQ==
             -----END AGE ENCRYPTED FILE-----
           recipient: age1ll6hj5ggruetgjwjfnplpn5xtq35uhlcdflksx3xmnjm6s3uad9sz70jkf
         - enc: |
             -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBzbXdrajlVQU5RditMckh6
-            ZGhvcU01UEdMbURlcG9mZzg0RTJhcHJ4QlQwCjgyRGVXYytxUFB0WXhadHQ1YVpF
-            VUJpVzE2SzFPc3FHV25FQmVSQzNJTm8KLS0tIG9DazRkUkwyNHNQMnp5R3RySDBo
-            aGFLUFBjZjVxeWFzTXFldHVRNkNyYTAKd6eS5lks4+3SV1bFBQWyPi7OZcvRMDIc
-            kYD5C/vMkKlvBYyODCWJyim2xgM/nyQNcf/q4BUS3HF/RidRdJh9NQ==
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSB6TjhmOTJ0bUpMQWowb0hB
+            T3Era1loU1pMdmkxdnAvRkViekpqZWZjaWgwCkFRQXhFUy9PRVBma2JMUDhqY2F1
+            VVFDRFNVbWpNaEczY1JVQUMyck9XdEkKLS0tIGpxc0tGdVFKK3FteVJKM1Fxa2ky
+            a21WLy9qV05hUURCTVBvcVh3cE45Z3cKXCYfXSjhApBoLbHDu2OOd57Y1zN54yy+
+            WDQvz8PpMxhc1nU5Kw/cI+WmL1KvN0qQZfOx/7D4W+dy/ZDWX27TpA==
             -----END AGE ENCRYPTED FILE-----
           recipient: age120le4a5l8dh3lyfgvmj3d9ksmej6ajs5mer5y7r0vfg3x9fn69dqf8xgzu
         - enc: |
             -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBFZEhIT1REUTFtdkIzaDFY
-            dGU4VjU3QlpPU1Rod1dTUWhBbjc0WTR5bUF3CkNGY3F2Q251aXlGNCt5ZjFWMTE1
-            RjY4bHZ1T29aK3N0SktjdUpZbSs4QVEKLS0tIDZrMWd3aFVRMEZMU1N1dlZxYTZS
-            U3gxWU5EVVRXUGxJVlZQZmpZRmdhbW8KJ8yD9laK2T1qn0z2uYNeI80rtMlOVi2M
-            qIPn/FSoWnKQ7NotSTGUtEhC+f/nXfUQrBd5fsfmja++hrevTmkqYw==
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBZK0FYQnBHMHZ6dlpMYTlC
+            WFdOWDFkRVBuY1pmdTFiUndLV3JXcndZa3pNCnJsd0tHN0FveWV6UUNQSEdpdWw5
+            dWZITkxWelNIRlpKS1pnN0ZmVlQvZjAKLS0tIEUwMXdtNFdkUWdIRjlxc0owdTRr
+            c1o1TmptWWd1ZGxzcWJJNzJ0K25PTTAKoos5rnkyQBCm+ZuhCCaMJwqJBo1fpnsl
+            G74wu5vbTBG4VjVhI5KqyiuiTRU4jPcGxysECqe7AyZUBGp7ndewgw==
             -----END AGE ENCRYPTED FILE-----
           recipient: age1jy444f9d9stygj4p3w9kh54cqcfr654tvr75tdvee5cxsgtdtc9q3v60ep
         - enc: |
             -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSA0N1owUGpuWWxWb2t2Mk5Z
-            QlZ6Nm5lcnNBY0d1NldBQXY3OThLM2UxbkY4CkZyRUl4S1JRaDhnemRIYWR1OHZx
-            aWliVWh3V1RBL0R3Q0RqOE1SM0VBblUKLS0tIGpaUjJxdnV0UW55T01HcFBsdHR2
-            aEJ3UzJ6RklZVFRrNGMraUtGYXVZd0kKaQmzjnI6xgV7H1YcoI+gRn2IVfOLTJBY
-            rO43fvVtZ79c1as6CL+GoEIJY/uVA/3M4gQ0vYbDDn57vm5OOJW8jg==
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBTVkVXaCtGTXVkRXRZMFZ1
+            ZUdXM1hRcDhaSURqcGh2eksxTUNCckk5SVJjCnhkSDRKdDFSckxUWXd4SmVxWG5p
+            bU5OV0hzaWR6VDFwcDY2WlY4WnN3cFEKLS0tIElOVzRCcXR4U0dhajJySUhaZGps
+            MW9rQk1JVDFWRnFxVzhCUkRIS09EamsK1rVidD48PqwlEWQyjF7iQWU7aBdPqQHy
+            z5LaSi3LvJX3rNE/+q0E8/gbZyjGpbEn3AUI5mBF64GY3IZkRxZSXQ==
             -----END AGE ENCRYPTED FILE-----
           recipient: age120whqj96g26lsgy4udvgsn8dc9lumh8jeu3a564fx79rjr5lxffqmrljuu
         - enc: |
             -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBMWTcwd0JLS0lybWREaU5h
-            NHczcEQxeUx4U09oV2k5ZXpKTGxxR3ZYd2tnCkVXR3kyTHZSSWNaUCszL3Rvakhm
-            bVl5bnFRVkhJRFVaY1NQYTNrd2JKMTAKLS0tIDJTT0YvUE5HUWVKRnhXY2tvMjI1
-            VXNlcnJqcDM5d3poVU1SKzQyNWdpRzQKTht/ko7cy7OY0wGfza4eierYS7q/nCFH
-            YsG74ez3piUQ1rdJRi0e29QWz4xL3JeU9oE+tr0rMB6WvgTmuX5gwQ==
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSAvbnJCWU9UMXJXMG92b3hm
+            WWNBTFpQamVWQThITmt5QUVwR3h2OHI2SlVRCjcxOEJTVkFjN0NhamFZQ1plK29w
+            dk5XYkYxOXQ0YkVzcVc3VnhCQWlsV0UKLS0tIHZ5cWtFZUhDKzZkOE1BK2Y5TStR
+            SFlDRjE4ZHpiVEJOQk5TUGNEN1B4amMKUCJ8CL8QpmRpFs83HD9TUn7NrPguuP8S
+            JQH/bzPorXTXJuyOKuKAZq1hK8BmiMUFksaZ03yN6YaFVIOeelEEMg==
             -----END AGE ENCRYPTED FILE-----
           recipient: age1xjst4frdh0th6q8m7p7u9g5af7ty5jqeum0p6z8a52a9q7st7ewqw8yl9j
         - enc: |
             -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBLWmtNOVVvRW9QbU50OUxv
-            TUYxWXJ4cFVaMjVnVHJVK1VTZzhDd2tCMWpnCllPWnl2bjcxemF6cFp1blZnMEMy
-            clhqdlVzdEZTYnY3b0ZhR2hsMXRlQTgKLS0tIHEzNU9yNmhackt6R2dLT2dZSEFJ
-            UnJXeWVyQld6NFhXUWowUlVXaEtKVUkK2Saa8w9/SOWw1VXWkUg62ay31bSIqHWB
-            8QmNafddV0WQ8oYcD8ZMf8Bm8jP9yLMUDOiVzVZiiTGWEx7EbUQV3Q==
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBxWHV0S2dxeHZEMDNRSkhO
+            ZFJHaiszd0tzYzlzd3gvbXNSQmlMVlJUNGcwCnVrM29MdFZCR1NBYnpkQ1k5VFZQ
+            b3Z2Q3ZGekVQZkZKWGlka3NDOHJ0R1EKLS0tIDlXTmNzUk0wVXo0UWhkd0ZvK3FI
+            UzJxU3RkdWs4aTZYVVkrS056bTN1ek0KgKJNz8GvynX5pK33aW9x3v6yr2Ox0LCT
+            GGrt+ddbKLcwpBpYjfWkFhffO330EKui73S+c/qMf8N9j6wzalOTpQ==
             -----END AGE ENCRYPTED FILE-----
           recipient: age10at8862478urh0eeuwh8hzln6ck78jgwtztgxatwqlzwagg77y5snm4xzg
         - enc: |
             -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSA2OEs4dWJ5UTBoY2hHanZY
-            N3VNMWdGVkJHMlVublBmN2NiMUNtRnVnaXdrCnYzV3EvRzV1OXNyOFVKc2lXMFov
-            Mkgvc2dFdyt0MGJIMlFPckpBeDZ5c0UKLS0tIERrOGc4aWtFSTJWN1M2V0hyQ3hS
-            TC9HYVMyVWVBc1lHbUlqbnpsOWJobU0KkZZph2dnKwi4w+RXf7RXoHCYkXxmuLNj
-            rinQQlQeOwQptBn7+kRJubNjyHOlSHXhpsbV3/IkJRNMhw69knzasQ==
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSA3ZWJlbDl1TEkvWEpBMC9Y
+            ejY1MnFMUExBeDhITndxNy9YUi9hU0tXM1JJCjcwNEppcDdxYzlCSVMrMExWa3A4
+            cmFPSjV3LzkyMXZCUDU2QmtHRmpHRmsKLS0tIFFTdXBOaDJJTERseXlGbmdrQzhD
+            TWtnRFdIRXpsNkY0U1BiczNsdUk1V1kKGpndKmT8kj/oIxQuxQALfzscw+CsVmnj
+            cyPC3bF+tG6LcqqoKLjPSJfcIgzhnX7cAr/wwESavemLn8L/zQMe4w==
             -----END AGE ENCRYPTED FILE-----
           recipient: age1ezk9x53zt8kcnscdm80jcyf0xq97vndv7jsn3rl8cc0cwm2jmpmq372dzs
         - enc: |
             -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBoWkljeFFBS3hhOG0relVz
-            V0JUK2hWaklxTGVhMWR4bytPczhjcldheno4CmZUTWszREVUdVVQazNOdXAvY0FN
-            bEtQSDJveHplNTE0a0FvWEFxVlEreDgKLS0tIGNKUkNVQ3BDQ3d0V1lyY0VsaHda
-            cDNHa3lYTkxBeVFlSGtIMVFScmxsazAK5ZyJ35/jjFRhQdG1PMOcEJ3MJf96i8DZ
-            AvCw9jc7Hsbs6+LoZ45K1QKoWoZmBkmAatbksXo3jnAzEHmcQqODKQ==
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBpQVVyY3UrSHFWSmpDdmRE
+            dlh5akFqVFdDVFNMSHE1eVJnZzR6YzFHSVFRCko4UE9EdXNxZzF2MW5PTTN6dEdU
+            ZHM1MGowcVB2Y1ZlOTVHdnNtY3diM2cKLS0tIDBYSmh5dVVPaTM3d0ErcC8wMDNB
+            eUpHWnZlYnJsbHZuS3pwbG15UGtwN2MKVPQA1MpjIfYAsNacoAbpvZNuAIkvx7ER
+            CvWBKEHUVm6m8905BXzv8MdGTAk0EyCIP3aMmYqTIYfv2k9pP0T08A==
             -----END AGE ENCRYPTED FILE-----
           recipient: age1fxxzpnfse8nd9wz78ht3m0plrmraacf4cpga0pe8fm2tdnqcgy8q7qsyvp
         - enc: |
             -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBtWGRLTCt5dXZMR1d6a3lD
-            Z0t1SDAwY3V3MXI1MGt5RGkrc1JHSVRUUzNZCi9UOTBTcVY3NkhFb1FFOS9vbTY2
-            S1BqNmd2eWVXbnliOFJWRDY0UmhYV3MKLS0tIExYNlJxcTBidnpObHgweHlqNHhz
-            OXpwRHhyOGhNRXNTWDBIUVJCR0VPTkkKQZh9e7lOINL0khHS6tBehCrm+SX5Q6XE
-            XL4FpBsJ5+MnxTx1O/bu6/f7OvZqfnxhQv0lkKOMJnkZfPZx5D+meg==
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSA0MTQ4MzN0bDJNL1l1bmpX
+            ekxDUXRNa0JHWWltZFNGTVltTFdSSE82SVZvCmdUWUdja3JIajMzY09IMUE5elox
+            MDdEakFJTmtkRWF2R1BGNkQ4U3grNWsKLS0tIE9hZUhkVGI1ZEpzdDhRU21EZm91
+            VnJNb1kyQ05MM0RJa1lLUEtjWWxkSTAKHVAKcGcWl6LncJALRBU9RKP7ot6C6GSE
+            1iZtj1SNX6wzEWrhOEnV37aQ8bKZj6u+Y/q6/vJ4qiBs78y/drdIzA==
             -----END AGE ENCRYPTED FILE-----
           recipient: age190htw7prp4vln076dxjx3gxxaq06h0zl0te7cqgpx79vl3lhkaes8suy05
+        - enc: |
+            -----BEGIN AGE ENCRYPTED FILE-----
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBNUEVWY2NpVEU4OTZPZTZR
+            UzdKcktpUGJOcnJ0aEhkQlhvUWdUYkV2SkdzCm51eGJVeHJMcVRRRld0dFRCYUxr
+            TTN2WEhOVjRqV0FtQXowZWNTbkJneEUKLS0tIFpUazZpTUNWZUZBSFE0VDZZbkJu
+            SFVlUVhySnNqUENYOG9qUm5ZMDc1ZW8Kv0lY5dhnCEheM0sttfr4p7IL+EVog16T
+            OapUdbuXL2l7t7URzHnvfG/nbOtJIjH8a0XFsWyJChtNXpF2d/vf2g==
+            -----END AGE ENCRYPTED FILE-----
+          recipient: age1ukpqxzl44mnjpy5r96sfuc5sqzm47u4k8ujjh5qdgy6jvl9uqgpspymqfk
+        - enc: |
+            -----BEGIN AGE ENCRYPTED FILE-----
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBvTzc2aExGTHFrc0ZtL3pq
+            YlZLbzd3MVZHZTB0VUQvSXZ4NGVsZk42c2hBCk1vbTk0Tnl3b01vbVZaMkJ5aldE
+            WmZBMGFjb2pjQXpYcnBxWmp0UUsrdXcKLS0tIG16SG9JbFdkbmVidCsxUnpBR3V3
+            RzNOY3hIRWk4UXh6N3NrcjNSU3ZwWTgKaExY4U2s8E6ojljJ+4TU+YJhcLXyuVA1
+            ROB70jQCjFvQOeo6thjQohSSUoPKhxSl1/nr4ZiGBO3/VskzihckKg==
+            -----END AGE ENCRYPTED FILE-----
+          recipient: age15kh7akxlx7zn00tey79rq2g8lgs4j5y77rcnyfxrxap8ckfu0a9sqvtdhh
     lastmodified: "2026-07-19T02:30:40Z"
     mac: ENC[AES256_GCM,data:UiL3VMDF6rq4Nr87KspcDx434q3tfNXeb5pwH2O+4ssNQ6xzcYDdzXBnhAY3zLBsqPMKrvHBd4Ot/gEMcq3FMIVe7Q6p9yWKpep66KZ/yWEhAlwIVhD79Oj8VS+1CHKjf25zpRdhZorp04oeFQQd9VfjJB4EE/Q1aVbwTGlpIic=,iv:i/0conaFgFia+wzNTdUL6tlSTw35HTK3Ap1Sr5RGHf8=,tag:ULbz5FllShA/JjlSRdxA0g==,type:str]
     unencrypted_suffix: _unencrypted

secrets/server.yaml

@@ -3,22 +3,40 @@ sops:
     age:
         - enc: |
             -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBURmMzN3hrSlNrUkkvVWNl
-            L3M1dEhWeW14N0RFNVRPci9QK1YyTFdqRVVJCk5WaWswT2NicldkYzZjbVhYU2xu
-            MGFsNmUzeTN2TS9wOEdvRURpVUVYZXMKLS0tIDZ6MEdPTVhCaTQ2UXFWTUFtc0pm
-            MFlJb0c2WXJtMGRLZEZYY0pZWWpFWm8K/mlYZIe8UC0QU+1mq3NtrtTF5b2m5hCK
-            +K0QiZLTKmmDcr4bRhZ32VE7R7GRwtMNnOP/mElZvPAyWyHHhRiOHg==
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBaYU9HR0lETDhkYXR5NWlj
+            L0diSE5JQU9KWjRoU3hrVjdqZ2tPNUtOdFZvCjdUVGNFbDVYa3pVSC9ZWVNORER4
+            QWozUlhoSEtjTk9IRlM3VkZoYlc1RTAKLS0tIDlHS01WVWlOMWFEQU5GTVRLZVhV
+            VTcwekhrRHB3SlVYT2MzOW5GbE52dkkKKCWehPhpdGapdyzpll20NJUcZwvW/7X8
+            KQ1EqAgI2fewnbwuIDYCleN0b0SLJNUeSV/tFKDDoTMnHWCdeD4ECg==
             -----END AGE ENCRYPTED FILE-----
           recipient: age1njap586hc0q43kr03g6c8eqhdsmk8zcafkl3f83xwlc2gqhlmfgs4tmwad
         - enc: |
             -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBURHlUL0RNMUtNallEcG5p
-            eG4xby8yVzdQUTBaNkl4ano0YjBMcDd0Wm5NCkZvaXNPZm9wemJkMmNSdGdOaTI4
-            Z1RwUnhiRUpCMWZaeWtlSVBmNW5KOXMKLS0tIDk0R0k3ZHczTFNCWUZxSWF0M0FJ
-            MGlZMmtuSFYrcG1meDNMWDNqSjFxcE0KDu2dAc0gqmmPkpbpBe4YohM7rYmUwEkI
-            V2FUQwjlvh50svtjCVdYbx2xuq4sQLnKelk/q1onLw60FwsVfzD8sQ==
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSB4YndybFBTQ2p4SGZ4SDJs
+            a2p0eFRQOVVWcGd2a25ESW9ESGx1RWw0Zno4CnhpVVh3cGI5UjY0YmFINFFPMTh4
+            b1B5SjJ3NTNvUE1QUmJjVFozY1dYS2MKLS0tIC9HZGNpOFZhZGNFZGt5blJuZXVV
+            SXpkRzV4d2ppV3ZQZSt1dmxYNGVFMUEKmSe9dkrmkND81Hw2/ATAmFvcmhk1tUC1
+            LxxBw54IVHUqwYKgRYUYRNu+pykDT5OnFDPiskd49Xso99LY87PyiA==
             -----END AGE ENCRYPTED FILE-----
           recipient: age1ll6hj5ggruetgjwjfnplpn5xtq35uhlcdflksx3xmnjm6s3uad9sz70jkf
+        - enc: |
+            -----BEGIN AGE ENCRYPTED FILE-----
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBLd1FSSHVTSGdHWm8wMVQz
+            d0dlOEsxeGwxdHU0eUlFSUxka0ZmcVpnOTM0CndYNUVjVy92QVhNY2orQmpSQjYy
+            VS9KM0NUTXhuM0lCSDBZMWtISEdtWTgKLS0tIExTL2wvS3FEdVViUmRYZEFsR3R0
+            YTJFM200RjF6MjNxOFA4eHRpWVhtRkUKOqBIT445HnPXrrH/qV6FIuAhAuJmSL6V
+            +PQopM/m3PAnK5m5Mu3cfjYfDiB8+GWTABhljfT+GbcoK7CqWLehrw==
+            -----END AGE ENCRYPTED FILE-----
+          recipient: age1ukpqxzl44mnjpy5r96sfuc5sqzm47u4k8ujjh5qdgy6jvl9uqgpspymqfk
+        - enc: |
+            -----BEGIN AGE ENCRYPTED FILE-----
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSAvVXF4a2NmaW15R2VXS3FP
+            SDVPUGpUWTlIWjl2N2N2SXR1UlRkdllWNVRJClZDbVo4dUhrZytqZkMrYWtpOEZx
+            c0dSZGViN04zQ1B1WEZEWm1QM1lsejQKLS0tIE0wa0k5Rm1xZmw3OHFESkVXc25j
+            NlFpYTJSckQ4MVlZQ01reDlkaWY3TkEKsmQlreRhRAjVZ/q5x52FNATDF2sLhbHo
+            djOZigZx2rs6shqQ6It/XRJ8CiPaXZBPOt529Gwmplu+hWlAU1+l2Q==
+            -----END AGE ENCRYPTED FILE-----
+          recipient: age15kh7akxlx7zn00tey79rq2g8lgs4j5y77rcnyfxrxap8ckfu0a9sqvtdhh
     lastmodified: "2026-07-19T02:30:40Z"
     mac: ENC[AES256_GCM,data:rKHZjU/MH08ASTlu32HZO9uWmsBYuMCEC6M8gwVhzuWvmablnP05tS2z13XfaWaCEUXk6kmGJKuU0zu5+IKVZgamCF6DAMtxQb6bVCaLsoAm/GSqWQ5VI9eHqgnSSdN/o3ul/33Rf8iBQo4aw8FFAmDVuNz8bfAn0QefFTj0ByI=,iv:JD2gtqRinOY77etg6PUmZNovkYl1Q3F6ZvRi4x7RznQ=,tag:/5IMpWKRVt+l1luCTQE0BA==,type:str]
     unencrypted_suffix: _unencrypted