fix(lxc): prevent SSH host key deletion on every rebuild; add recovery script #50

2026-07-23T23:20:26Z

beatzaplenty commented

2026-07-23 23:20:26 +00:00

NixOS's etc activation removes files that were in a previous generation's
environment.etc but absent from the current one -- even real copies, not
only symlinks. LXC tarballs bake the host key into environment.etc (via
NIXOS_HOST_KEYS_DIR), but every subsequent nixos-rebuild switch lacks that
env var, so the key is removed as "obsolete". sops-nix derives its age
decryption key from /etc/ssh/ssh_host_ed25519_key, so deletion cascades
into "Error getting data key: 0 successful groups required, got 0" for
every sops secret on the host.

Fix: two activation scripts bracket the etc step.
preserveSshHostKey (no deps, runs before etc): copies the live key to
/run (tmpfs) before etc can delete it.
restoreSshHostKey (deps=[etc], runs after etc): reinstalls via install
if etc removed the key. The resulting file is not tracked in either
generation's environment.etc, so subsequent rebuilds leave it alone.

scripts/recover-hosts.sh: restore both private and public key files (not
just the private key), use install(1) for atomic mode setting, and add a
post-rebuild sops-nix verification step to confirm success.

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_014zT1L6hmsq6i1evAEH7dmi

NixOS's etc activation removes files that were in a previous generation's environment.etc but absent from the current one -- even real copies, not only symlinks. LXC tarballs bake the host key into environment.etc (via NIXOS_HOST_KEYS_DIR), but every subsequent nixos-rebuild switch lacks that env var, so the key is removed as "obsolete". sops-nix derives its age decryption key from /etc/ssh/ssh_host_ed25519_key, so deletion cascades into "Error getting data key: 0 successful groups required, got 0" for every sops secret on the host. Fix: two activation scripts bracket the etc step. preserveSshHostKey (no deps, runs before etc): copies the live key to /run (tmpfs) before etc can delete it. restoreSshHostKey (deps=[etc], runs after etc): reinstalls via `install` if etc removed the key. The resulting file is not tracked in either generation's environment.etc, so subsequent rebuilds leave it alone. scripts/recover-hosts.sh: restore both private and public key files (not just the private key), use install(1) for atomic mode setting, and add a post-rebuild sops-nix verification step to confirm success. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014zT1L6hmsq6i1evAEH7dmi

beatzaplenty added 1 commit 2026-07-23 23:20:26 +00:00

fix(lxc): prevent SSH host key deletion on every rebuild; add recovery script

Check NixOS configurations / eval-hosts (pull_request) Successful in 10m33s

Details

dce3788499

NixOS's etc activation removes files that were in a previous generation's
environment.etc but absent from the current one -- even real copies, not
only symlinks.  LXC tarballs bake the host key into environment.etc (via
NIXOS_HOST_KEYS_DIR), but every subsequent nixos-rebuild switch lacks that
env var, so the key is removed as "obsolete".  sops-nix derives its age
decryption key from /etc/ssh/ssh_host_ed25519_key, so deletion cascades
into "Error getting data key: 0 successful groups required, got 0" for
every sops secret on the host.

Fix: two activation scripts bracket the etc step.
  preserveSshHostKey (no deps, runs before etc): copies the live key to
    /run (tmpfs) before etc can delete it.
  restoreSshHostKey (deps=[etc], runs after etc): reinstalls via `install`
    if etc removed the key.  The resulting file is not tracked in either
    generation's environment.etc, so subsequent rebuilds leave it alone.

scripts/recover-hosts.sh: restore both private and public key files (not
just the private key), use install(1) for atomic mode setting, and add a
post-rebuild sops-nix verification step to confirm success.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014zT1L6hmsq6i1evAEH7dmi

beatzaplenty merged commit fa52c2849a into main

2026-07-23 23:20:30 +00:00

This repo is archived. You cannot comment on pull requests.

1 Participants

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: beatzaplenty/nixos#50