Fix nix-cache remote-builder trust: stale host key + wrong sshKey path #21

2026-07-20T17:22:11Z

beatzaplenty commented

2026-07-20 17:22:11 +00:00

Summary

variables.nix's nixCacheHostKey had drifted from nix-cache's actual current SSH host key (confirmed live via ssh-keyscan against the running container) -- every declaratively-configured client trusted the wrong key, breaking distributed-build SSH verification for everyone.
modules/nix-cache/remote-builder-client.nix hardcoded sshKey to /root/.ssh/nixremote, but server only has its own default /root/.ssh/id_ed25519 installed (confirmed live via qm guest-agent) -- that file never existed there, so server's distributed-build config pointed at nothing. Standardized on each client's own default identity (matches docker, server, and this session's sandbox already, and matches the per-host-key pattern vars.remoteBuilderAuthorizedKeys already uses).
Added scripts/secrets/sync-nix-cache-host-key.sh (--check/--dry-run/apply), wired into codex-maintenance.sh as a non-fatal drift check, so the host-key value doesn't silently go stale again next time nix-cache is rebuilt or recreated.
Updated docs/nix-cache.md and scripts/proxmox/configure-nix-cache-client.sh's default to match the corrected convention.

Follow-up needed from the operator (not done here -- requires manual/deploy access this session doesn't have)

Rebuild nix-cache so it picks up the corrected nixCacheHostKey (no functional change there, just documentation-of-reality) and this sandbox's newly-authorized key (root@claude, added in #20).
On docker: cp /root/.ssh/nixremote /root/.ssh/id_ed25519 && cp /root/.ssh/nixremote.pub /root/.ssh/id_ed25519.pub so it keeps working under the corrected sshKey path (same key content, already trusted -- no nix-cache change needed).
Rebuild any other real client host that imports remote-builder-client.nix to pick up the corrected sshKey path.

Test plan

bash scripts/codex-maintenance.sh (full eval, fmt, lint, secret scan) run twice from this branch -- clean, exit 0
nixpkgs-fmt --check / statix check on the changed .nix files -- clean
New sync-nix-cache-host-key.sh exercised in --check, --dry-run, and apply mode against both the stale and corrected variables.nix values -- behaves as expected in all three
Re-ran configure-nix-cache-client.sh on this session's own sandbox with the corrected default -- now correctly configures the builders line (previously silently skipped since /root/.ssh/nixremote never existed here)
Operator: verify a real distributed build against nix-cache succeeds after the above follow-up steps

https://claude.ai/code/session_01V7yVH71vGrDVzovh9UaMu8

## Summary - `variables.nix`'s `nixCacheHostKey` had drifted from nix-cache's actual current SSH host key (confirmed live via `ssh-keyscan` against the running container) -- every declaratively-configured client trusted the wrong key, breaking distributed-build SSH verification for everyone. - `modules/nix-cache/remote-builder-client.nix` hardcoded `sshKey` to `/root/.ssh/nixremote`, but `server` only has its own default `/root/.ssh/id_ed25519` installed (confirmed live via `qm guest-agent`) -- that file never existed there, so `server`'s distributed-build config pointed at nothing. Standardized on each client's own default identity (matches `docker`, `server`, and this session's sandbox already, and matches the per-host-key pattern `vars.remoteBuilderAuthorizedKeys` already uses). - Added `scripts/secrets/sync-nix-cache-host-key.sh` (`--check`/`--dry-run`/apply), wired into `codex-maintenance.sh` as a non-fatal drift check, so the host-key value doesn't silently go stale again next time nix-cache is rebuilt or recreated. - Updated `docs/nix-cache.md` and `scripts/proxmox/configure-nix-cache-client.sh`'s default to match the corrected convention. ## Follow-up needed from the operator (not done here -- requires manual/deploy access this session doesn't have) - Rebuild `nix-cache` so it picks up the corrected `nixCacheHostKey` (no functional change there, just documentation-of-reality) and this sandbox's newly-authorized key (`root@claude`, added in #20). - On `docker`: `cp /root/.ssh/nixremote /root/.ssh/id_ed25519 && cp /root/.ssh/nixremote.pub /root/.ssh/id_ed25519.pub` so it keeps working under the corrected `sshKey` path (same key content, already trusted -- no nix-cache change needed). - Rebuild any other real client host that imports `remote-builder-client.nix` to pick up the corrected `sshKey` path. ## Test plan - [x] `bash scripts/codex-maintenance.sh` (full eval, fmt, lint, secret scan) run twice from this branch -- clean, exit 0 - [x] `nixpkgs-fmt --check` / `statix check` on the changed `.nix` files -- clean - [x] New `sync-nix-cache-host-key.sh` exercised in `--check`, `--dry-run`, and apply mode against both the stale and corrected `variables.nix` values -- behaves as expected in all three - [x] Re-ran `configure-nix-cache-client.sh` on this session's own sandbox with the corrected default -- now correctly configures the `builders` line (previously silently skipped since `/root/.ssh/nixremote` never existed here) - [ ] Operator: verify a real distributed build against `nix-cache` succeeds after the above follow-up steps https://claude.ai/code/session_01V7yVH71vGrDVzovh9UaMu8

beatzaplenty added 1 commit 2026-07-20 17:22:19 +00:00

Fix nix-cache remote-builder trust: stale host key + wrong sshKey path

Check NixOS configurations / eval-hosts (pull_request) Failing after 11m12s

Details

fd773b65da

variables.nix's nixCacheHostKey no longer matched nix-cache's actual SSH
host key (confirmed via ssh-keyscan against the live container), so every
declaratively-configured client's programs.ssh.knownHosts trusted the
wrong key -- distributed builds would fail host-key verification. Also,
modules/nix-cache/remote-builder-client.nix hardcoded sshKey to
/root/.ssh/nixremote, but the `server` host only has its own default
/root/.ssh/id_ed25519 installed (confirmed live via qm guest-agent) --
that file was never even present, so the build machine config pointed at
nothing. Standardize on each client's own default identity, matching the
per-host-key pattern vars.remoteBuilderAuthorizedKeys already uses instead
of a shared/differently-named keypair, and add
scripts/secrets/sync-nix-cache-host-key.sh (wired into
codex-maintenance.sh's --check) so the host-key drift doesn't silently
recur next time nix-cache is rebuilt or recreated.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V7yVH71vGrDVzovh9UaMu8

beatzaplenty marked the pull request as work in progress 2026-07-20 17:22:58 +00:00

beatzaplenty marked the pull request as ready for review 2026-07-20 17:32:16 +00:00

beatzaplenty merged commit 91c977e5e7 into main

2026-07-20 17:32:33 +00:00

This repo is archived. You cannot comment on pull requests.

1 Participants

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: beatzaplenty/nixos#21