Archived
Fix nix-cache remote-builder trust: stale host key + wrong sshKey path
Check NixOS configurations / eval-hosts (pull_request) Failing after 11m12s
Check NixOS configurations / eval-hosts (pull_request) Failing after 11m12s
variables.nix's nixCacheHostKey no longer matched nix-cache's actual SSH host key (confirmed via ssh-keyscan against the live container), so every declaratively-configured client's programs.ssh.knownHosts trusted the wrong key -- distributed builds would fail host-key verification. Also, modules/nix-cache/remote-builder-client.nix hardcoded sshKey to /root/.ssh/nixremote, but the `server` host only has its own default /root/.ssh/id_ed25519 installed (confirmed live via qm guest-agent) -- that file was never even present, so the build machine config pointed at nothing. Standardize on each client's own default identity, matching the per-host-key pattern vars.remoteBuilderAuthorizedKeys already uses instead of a shared/differently-named keypair, and add scripts/secrets/sync-nix-cache-host-key.sh (wired into codex-maintenance.sh's --check) so the host-key drift doesn't silently recur next time nix-cache is rebuilt or recreated. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V7yVH71vGrDVzovh9UaMu8
This commit is contained in:
+17
-9
@@ -46,18 +46,26 @@ the new key up automatically on next activation — no more manual
|
||||
|
||||
## Remote builder SSH keys
|
||||
|
||||
On each client, install the private key used to authenticate as `nixremote`:
|
||||
Each client authenticates as `nixremote` using its **own default root SSH
|
||||
identity** (`/root/.ssh/id_ed25519`) — not a separately-named or shared
|
||||
keypair. If a client doesn't have one yet:
|
||||
|
||||
```bash
|
||||
sudo install -d -m 0700 /root/.ssh
|
||||
sudo install -m 0600 ./nixremote /root/.ssh/nixremote
|
||||
sudo ssh -i /root/.ssh/nixremote nixremote@nix-cache nix-store --version
|
||||
sudo ssh-keygen -t ed25519 -N '' -f /root/.ssh/id_ed25519
|
||||
```
|
||||
|
||||
On `nix-cache`, install the matching public key used by `nixremote` authorized keys.
|
||||
Then add its `.pub` contents as a new entry in `vars.remoteBuilderAuthorizedKeys`
|
||||
(`variables.nix`) and rebuild `nix-cache` to pick it up (that list is
|
||||
declarative — an imperative `ssh-copy-id nixremote@nix-cache` won't stick;
|
||||
it gets overwritten on every rebuild). Verify with:
|
||||
|
||||
The committed `nixremote` authorized keys are public SSH keys only. Keep the
|
||||
matching private keys on client hosts and out of the repository.
|
||||
```bash
|
||||
sudo ssh -i /root/.ssh/id_ed25519 nixremote@nix-cache nix-store --version
|
||||
```
|
||||
|
||||
The committed `remoteBuilderAuthorizedKeys` entries are public SSH keys
|
||||
only. Keep the matching private keys on client hosts and out of the
|
||||
repository.
|
||||
|
||||
nix-cache's own SSH *host* key is trusted declaratively via
|
||||
`programs.ssh.knownHosts` in `modules/nix-cache/remote-builder-client.nix`,
|
||||
@@ -76,8 +84,8 @@ After deployment:
|
||||
curl http://nix-cache/nix-cache-info
|
||||
nix store ping --store http://nix-cache
|
||||
nix show-config | grep -E 'substituters|trusted-public-keys|builders-use-substitutes'
|
||||
sudo ssh -i /root/.ssh/nixremote nixremote@nix-cache nix-store --version
|
||||
nix build nixpkgs#hello --builders 'ssh://nixremote@nix-cache x86_64-linux /root/.ssh/nixremote 4 2 big-parallel,kvm,nixos-test,benchmark' -L
|
||||
sudo ssh -i /root/.ssh/id_ed25519 nixremote@nix-cache nix-store --version
|
||||
nix build nixpkgs#hello --builders 'ssh://nixremote@nix-cache x86_64-linux /root/.ssh/id_ed25519 4 2 big-parallel,kvm,nixos-test,benchmark' -L
|
||||
nix path-info -r nixpkgs#hello
|
||||
curl -I "http://nix-cache/$(basename "$(nix path-info nixpkgs#hello)").narinfo"
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user