Re-encrypt secrets/nix-cache.yaml for rotated lxc-nix-cache key #18

2026-07-20T15:31:24Z

beatzaplenty commented

2026-07-20 15:31:24 +00:00

Summary

Fixes the 502 Bad Gateway on http://nix-cache after today's lxc-nix-cache rebuild.
.sops.yaml's &lxc-nix-cache anchor was rotated in b3c8145 (after the rebuild), but secrets/nix-cache.yaml was never re-run through sops updatekeys -- it stayed encrypted for the pre-rotation key.
Per docs/nix-cache.md/docs/auto-installer.md, sops-nix decrypt failures are silent: cache-priv-key never materialized on the box, nix-serve had no secretKeyFile to bind to and failed to start, and nginx's proxy_pass had a dead upstream -- hence 502 from nginx (as opposed to the earlier "unreachable" state, which was network-level, not this).
Verified the new key is correct by deriving the age identity from host-keys/lxc-nix-cache_ssh_host_ed25519_key.pub via ssh-to-age and comparing to .sops.yaml -- matches.
Checked secrets/common.yaml and secrets/server.yaml against their .sops.yaml creation_rules too -- both already consistent, only nix-cache.yaml had drifted.

Test plan

Merge and pull onto whatever redeploys/rebuilds lxc-nix-cache (or nixos-rebuild switch on the box itself, done manually by the operator per repo safety rules).
curl http://nix-cache/nix-cache-info returns 200, not 502.
sudo systemctl status nix-serve on nix-cache is active.

## Summary - Fixes the 502 Bad Gateway on `http://nix-cache` after today's `lxc-nix-cache` rebuild. - `.sops.yaml`'s `&lxc-nix-cache` anchor was rotated in `b3c8145` (after the rebuild), but `secrets/nix-cache.yaml` was never re-run through `sops updatekeys` -- it stayed encrypted for the pre-rotation key. - Per `docs/nix-cache.md`/`docs/auto-installer.md`, sops-nix decrypt failures are silent: `cache-priv-key` never materialized on the box, `nix-serve` had no `secretKeyFile` to bind to and failed to start, and nginx's `proxy_pass` had a dead upstream -- hence 502 from nginx (as opposed to the earlier "unreachable" state, which was network-level, not this). - Verified the new key is correct by deriving the age identity from `host-keys/lxc-nix-cache_ssh_host_ed25519_key.pub` via `ssh-to-age` and comparing to `.sops.yaml` -- matches. - Checked `secrets/common.yaml` and `secrets/server.yaml` against their `.sops.yaml` creation_rules too -- both already consistent, only `nix-cache.yaml` had drifted. ## Test plan - [ ] Merge and pull onto whatever redeploys/rebuilds `lxc-nix-cache` (or `nixos-rebuild switch` on the box itself, done manually by the operator per repo safety rules). - [ ] `curl http://nix-cache/nix-cache-info` returns 200, not 502. - [ ] `sudo systemctl status nix-serve` on `nix-cache` is active.

beatzaplenty added 1 commit 2026-07-20 15:31:32 +00:00

Re-encrypt secrets/nix-cache.yaml for current lxc-nix-cache key

Check NixOS configurations / eval-hosts (pull_request) Failing after 11m53s

Details

222a3ced69

.sops.yaml's &lxc-nix-cache anchor was rotated (commit b3c8145) after
today's LXC rebuild, but secrets/nix-cache.yaml was never re-run through
sops updatekeys -- it stayed encrypted for the pre-rotation key. Per
docs/nix-cache.md and docs/auto-installer.md, sops-nix decrypt failures
are silent: cache-priv-key never materialized, nix-serve had nothing to
bind its secretKeyFile to and failed to start, and nginx proxy_pass had
no upstream -- the 502 Bad Gateway seen from clients.

Confirmed the new key matches the currently-deployed host by deriving
the age identity from host-keys/lxc-nix-cache_ssh_host_ed25519_key.pub
via ssh-to-age and comparing to .sops.yaml.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

beatzaplenty merged commit 1d44523181 into main

2026-07-20 15:33:38 +00:00

This repo is archived. You cannot comment on pull requests.

1 Participants

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: beatzaplenty/nixos#18