Archived
beatzaplentyandClaude Sonnet 4.6
684351b89b
Check NixOS configurations / eval-hosts (push) Successful in 10m26s
Root cause of recurring sops failures on new VM boots: disko builds raw disk images, and create-proxmox-resource.sh only syncs the clan-var SSH host key to the Proxmox node (for proxmox.nix to bake into the image) when it actually builds — reusing a cached image skips sync_remote_host_keys, so destroy+recreate reuses a stale image with the wrong or randomly-generated key baked in. On first boot the VM gets a different key than what .sops.yaml was encrypted for, and sops fails permanently. Fix 1 — deploy.sh Phase 3: always pass --force-rebuild so every VM creation rebuilds the disko image fresh with the current clan-var key baked in via NIXOS_HOST_KEYS_DIR (proxmox.nix already reads this under --impure). Fix 2 — deploy.sh Phase 5.5: after VMs boot, scan their actual ed25519 host keys and, if they drift from clan vars, update the clan var pub-key files, rewrite the .sops.yaml age anchors, and re-encrypt all affected sops files. Defence-in-depth: normally a no-op after Fix 1, but catches any residual mismatch (e.g. --skip-create-vms reuse of an older image). Fix 3 — cluster-init.sh: add crm_standby -v on for both nodes before DRBD metadata init. Without this, Pacemaker's OCF DRBD agent races: it sees drbdadm down as a failure and immediately calls drbdadm up again, leaving /dev/sdb busy when create-md / write-dev-uuid runs (drbdmeta exits 20 with "stdin not a TTY, not waiting for confirmation"). Standby suppresses resource scheduling during init; crm_standby -v off restores it after DRBD is up on both nodes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>