diff --git a/README.md b/README.md index 1d35c71..ac2e577 100644 --- a/README.md +++ b/README.md @@ -4,10 +4,10 @@ This repository builds a custom NixOS installer environment that can automatical install any host exposed by a separate NixOS flake. The installer provides a small NixOS netboot/install environment with SSH access, -Git support, and an interactive installation script. When the `nixos` user logs -in, it runs `/etc/auto-install.sh`, discovers available hosts from the target -flake, allows the operator to choose a system profile, applies the matching -Disko storage configuration, installs NixOS, and reboots. +Git support, and an interactive installation script. Logging in as any user +(root or `nixos`) runs `/etc/auto-install.sh`, discovers available hosts from +the target flake, allows the operator to choose a system profile, applies the +matching Disko storage configuration, installs NixOS, and reboots. The installer is designed for a mixed environment where different machines may have different storage layouts, including Proxmox VMs, Linode instances, and @@ -18,17 +18,35 @@ defined by each host's NixOS configuration using Disko. * `flake.nix` - * Defines the installer build outputs. - * Builds an `install-iso` image using `nixos-generators`. + * Defines the installer build outputs directly via `nixosSystem` (no + external `nixos-generators` dependency). + * `nixosConfigurations.installer` — ISO/netboot installer image. + * `nixosConfigurations.proxmox-lxc` — Proxmox LXC-based installer image. * Builds netboot kernel, initrd, and iPXE scripts. +* `common.nix` + + * Shared by every installer target (imported by both `installer.nix` + and `proxmox-lxc.nix`). + * Enables SSH access, configures Git credentials for the target flake + repository, creates the `nixos` and `root` users. + * Provides the generated `/etc/auto-install.sh` installation script and + the `programs.bash.loginShellInit` hook that runs it on login. + * `installer.nix` - * Defines the installer environment. - * Enables SSH access. - * Configures Git access to the target flake repository. - * Creates the `nixos` and `root` users. - * Provides the generated `/etc/auto-install.sh` installation script. + * ISO/netboot-specific: imports the stock `installation-cd-minimal.nix` + module plus `common.nix`. + +* `proxmox-lxc.nix` + + * Proxmox LXC-specific: disables bootloader options (containers don't + need one) plus `common.nix`. + +* `scripts/prepare-host-key.sh` + + * Admin-workstation pre-flight tool — see "Creating a New Machine" + below. The installed system configuration lives in a separate NixOS flake. Each target host is exposed through: @@ -87,7 +105,7 @@ These can be used to PXE boot the installer environment. 1. Boot the generated ISO or netboot environment on the target machine. -2. Log in as the `nixos` user. +2. Log in as `root` or `nixos` (over SSH or local console). 3. The login shell launches: @@ -200,9 +218,9 @@ The generated `/etc/auto-install.sh` performs the following steps: 6. Prepare `/mnt` using the Disko-generated mount configuration. 7. Check for a pre-seeded SSH host key at `/root/host-keys/_ssh_host_ed25519_key` - and install it to `/mnt/etc/ssh/` if present (see "Pre-Seeding SSH Host - Keys for sops-nix" below) — prompts for confirmation before continuing - without one. + and install it to `/mnt/etc/ssh/` if present (see "Creating a New + Machine" below) — prompts for confirmation before continuing without + one. 8. Install NixOS: @@ -216,42 +234,98 @@ The generated `/etc/auto-install.sh` performs the following steps: 10. Reboot. -## Pre-Seeding SSH Host Keys for sops-nix +## Creating a New Machine + +Full walkthrough for installing a new host, including the sops-nix +pre-seeding step. Do this instead of jumping straight to "Use The +Installer" whenever the target host consumes any secret managed by the +`nixos` flake's sops-nix setup (as of this writing, that's every host — +`modules/common/configuration.nix` puts the root/nixos password hash and +the GitHub token behind sops-nix for all of them). + +### Why the extra step exists The `nixos` flake manages secrets with sops-nix, using an age key derived from each host's own `/etc/ssh/ssh_host_ed25519_key`. That key is normally generated by a systemd service (`sshd-keygen`) the first time a host boots -— but sops-nix decrypts secrets (including the login password) earlier -than that, during system *activation*, which runs before systemd starts -pursuing the target that `sshd-keygen` is gated behind. On a genuinely -fresh install, this means secrets fail to decrypt on the very first boot -unless the host key already exists beforehand. +— but sops-nix decrypts secrets during system *activation*, which runs +**before** systemd starts pursuing the target `sshd-keygen` is gated +behind. On a genuinely fresh install, that means secrets — including the +login password — fail to decrypt on the very first boot unless the host +key already exists beforehand. Pre-seeding it is what fixes that. -To avoid this, generate the key ahead of time and register it with the -`nixos` flake's sops-nix setup **before** installing: +Machines that don't consume any sops-nix secret don't strictly need this, +but since the shared password hash currently applies to every host, treat +it as required unless you've specifically confirmed otherwise. -```sh -./scripts/prepare-host-key.sh [path-to-nixos-repo] -``` +### Steps -This generates `host-keys/_ssh_host_ed25519_key(.pub)` locally -and prints the exact steps to add its derived age key to `nixos/.sops.yaml`, -re-encrypt the relevant `secrets/*.yaml` files with `sops updatekeys`, and -commit + push the `nixos` repo. +**1. On your admin workstation** (the machine with `~/.config/sops/age/keys.txt` + from the `nixos` repo's sops-nix setup), decide the new machine's flake + target name (`-`, e.g. `proxmox-server`) and + generate + register its host key: -Once that's done and the target machine is booted into the installer, -copy the generated key onto it before running (or continuing) -`/etc/auto-install.sh`: + ```sh + ./scripts/prepare-host-key.sh [path-to-nixos-repo] + ``` -```sh -scp host-keys/_ssh_host_ed25519_key{,.pub} root@:/root/host-keys/ -``` + This generates `host-keys/_ssh_host_ed25519_key(.pub)` + locally and prints: -`auto-install.sh` checks for this file automatically and installs it to -the target's `/mnt/etc/ssh/` before running `nixos-install`. If it's -missing, the script warns and asks for confirmation before continuing — -useful for hosts that don't consume any sops-nix secrets, but skipping it -for a host that does will lock secrets out of decrypting on first boot. + - the line to add under `keys:` in `nixos/.sops.yaml` + - which `creation_rules` key_groups to add it to (`secrets/common.yaml` + always; a per-host `secrets/.yaml` too if this build type + has its own secrets, same pattern as `nix-cache`/`server`) + +**2. Edit `nixos/.sops.yaml`** by hand with the printed snippet, then + re-encrypt every secrets file you added the new host to: + + ```sh + nix-shell -p sops --run 'sops updatekeys nixos/secrets/common.yaml' + ``` + +**3. Commit and push the `nixos` repo.** The flake build the installer + uses has to see the new recipient before you install, or decryption + will fail on first boot regardless of step 4 below. + +**4. Build and boot the installer image** on the target machine (ISO, + netboot, or Proxmox LXC — see "Build The Installer ISO" / "Build + Netboot Components" above). + +**5. Copy the generated private key onto the live installer environment** + once it's reachable over SSH: + + ```sh + scp host-keys/_ssh_host_ed25519_key{,.pub} root@:/root/host-keys/ + ``` + +**6. Log in** (as `root` or `nixos`) to trigger `/etc/auto-install.sh` + automatically, or run it manually if it doesn't (see Troubleshooting). + Select ``'s flake target from the menu and confirm. The + script will: + + - run Disko to format and mount `/mnt` + - find the key you copied in at `/root/host-keys/_ssh_host_ed25519_key` + and install it to `/mnt/etc/ssh/` — if it's missing, the script + warns and asks for confirmation before continuing without it (fine + for a host with no sops-nix secrets; for any other host, continuing + anyway means the machine will boot with no working login password) + - run `nixos-install` and reboot + +**7. Verify after reboot.** SSH into the new machine and confirm secrets + actually decrypted: + + ```sh + ls /run/secrets/ + ``` + + If that's empty or login fails, the most likely cause is the host's + age key not being in `.sops.yaml` (or not re-encrypted into the + secrets file it needs) at the time `nixos-install` ran — re-check + steps 1–3, fix `nixos/.sops.yaml`/`secrets/*.yaml`, push, then + re-run `nixos-install --flake # --no-root-password` + from a rescue/live environment against the existing `/mnt` (or just + redo the install). `host-keys/` is gitignored — never commit private key material. @@ -285,11 +359,11 @@ The installer currently assumes: * The target host is responsible for defining its own storage layout. -Change these values in `installer.nix` if your environment differs. +Change these values in `common.nix` if your environment differs. ## Security Notes -`installer.nix` currently contains: +`common.nix` currently contains: * Git credentials * Password hashes @@ -369,8 +443,18 @@ Verify: ### Installer does not start automatically -Run manually: +It's triggered by `programs.bash.loginShellInit` (rendered into +`/etc/profile`), which only fires for genuine login shells. If you're not +seeing it — e.g. you attached to an existing shell rather than logging in +fresh — run it manually: ```sh sudo /etc/auto-install.sh ``` + +### Login succeeds but the machine has no working password after install + +The new host's sops-nix secrets didn't decrypt on first boot — almost +always because its age key wasn't registered in `nixos/.sops.yaml` and +re-encrypted into the secrets file(s) it needs before `nixos-install` ran. +See "Creating a New Machine" above, specifically step 7.