nix-auto-installer/README.md

# Nix Auto Installer

This repository builds a custom NixOS installer environment that can automatically
install any host exposed by a separate NixOS flake.

The installer provides a small NixOS netboot/install environment with SSH access,
Git support, and an interactive installation script. Logging in as any user
(root or `nixos`) runs `/etc/auto-install.sh`, discovers available hosts from
the target flake, allows the operator to choose a system profile, applies the
matching Disko storage configuration, installs NixOS, and reboots.

The installer is designed for a mixed environment where different machines may
have different storage layouts, including Proxmox VMs, Linode instances, and
physical hardware. Disk layout is no longer hardcoded in the installer; it is
defined by each host's NixOS configuration using Disko.

## Repository Layout

* `flake.nix`

  * Defines the installer build outputs directly via `nixosSystem` (no
    external `nixos-generators` dependency).
  * `nixosConfigurations.installer` — ISO/netboot installer image.
  * `nixosConfigurations.proxmox-lxc` — Proxmox LXC-based installer image.
  * Builds netboot kernel, initrd, and iPXE scripts.

* `common.nix`

  * Shared by every installer target (imported by both `installer.nix`
    and `proxmox-lxc.nix`).
  * Enables SSH access, configures Git credentials for the target flake
    repository, creates the `nixos` and `root` users.
  * Provides the generated `/etc/auto-install.sh` installation script and
    the `programs.bash.loginShellInit` hook that runs it on login.

* `installer.nix`

  * ISO/netboot-specific: imports the stock `installation-cd-minimal.nix`
    module plus `common.nix`.

* `proxmox-lxc.nix`

  * Proxmox LXC-specific: disables bootloader options (containers don't
    need one) plus `common.nix`.

* `scripts/prepare-host-key.sh`

  * Admin-workstation pre-flight tool — see "Creating a New Machine"
    below.

The installed system configuration lives in a separate NixOS flake. Each target
host is exposed through:

```nix
nixosConfigurations.<hostname>
```

and may optionally include a Disko storage configuration.

## Requirements

The build machine requires:

* Nix installed with flakes enabled.
* Network access to fetch flake inputs.
* Ability to build NixOS images.

The target machine requires:

* Network connectivity during installation.
* A disk defined by its Disko configuration.
* SSH access or local console access.

## Build The Installer ISO

From the repository root:

```sh
nix build
```

The generated installer ISO will be available through the `result` symlink.

The ISO is generated using:

```nix
nixos-generators.nixosGenerate {
  format = "install-iso";
}
```

## Build Netboot Components

The flake also exposes netboot components:

```sh
nix build .#netboot-ipxe
nix build .#netboot-initrd
nix build .#netboot-kernel
```

These can be used to PXE boot the installer environment.

## Use The Installer

1. Boot the generated ISO or netboot environment on the target machine.

2. Log in as `root` or `nixos` (over SSH or local console).

3. The login shell launches:

   ```sh
   /etc/auto-install.sh
   ```

4. The installer queries the target flake:

   ```text
   git+https://gitea.lan.ddnsgeek.com/beatzaplenty/nixos.git
   ```

5. Available installation profiles are discovered from:

   ```nix
   nixosConfigurations
   ```

6. Select the host profile to install.

7. Confirm the installation.

8. The installer applies the selected host's Disko storage layout.

9. NixOS is installed using the selected flake configuration.

10. The system reboots into the installed OS.

## Storage Management

Disk partitioning is handled by Disko.

The installer no longer contains hardcoded commands such as:

```sh
parted
mkfs.ext4
mkswap
mount
```

Instead, each NixOS host defines its own storage layout.

Example:

```nix
{
  disko.devices = {
    disk.main = {
      type = "disk";
      device = "/dev/sda";

      content = {
        type = "gpt";

        partitions = {
          root = {
            size = "-8G";

            content = {
              type = "filesystem";
              format = "ext4";
              mountpoint = "/";
            };
          };

          swap = {
            size = "100%";

            content = {
              type = "swap";
            };
          };
        };
      };
    };
  };
}
```

This allows different hosts to define different layouts:

* Proxmox VMs
* Linode instances
* Physical servers
* ZFS systems
* Future hardware-specific layouts

A host without a Disko configuration will not be automatically partitioned.

## Installer Process

The generated `/etc/auto-install.sh` performs the following steps:

1. Fetch available hosts from the configured NixOS flake.

2. Present available `nixosConfigurations` as a menu.

3. Confirm the selected installation target.

4. Check whether the selected host has a Disko configuration.

5. Run:

   ```sh
   disko --mode destroy,format,mount --flake <flake>#<host>
   ```

6. Prepare `/mnt` using the Disko-generated mount configuration.

7. Check for a pre-seeded SSH host key at `/root/host-keys/<host>_ssh_host_ed25519_key`
   and install it to `/mnt/etc/ssh/` if present (see "Creating a New
   Machine" below) — prompts for confirmation before continuing without
   one.

8. Install NixOS:

   ```sh
   nixos-install \
     --flake <flake>#<host> \
     --no-root-password
   ```

9. Remove temporary installation files.

10. Reboot.

## Creating a New Machine

Full walkthrough for installing a new host, including the sops-nix
pre-seeding step. Do this instead of jumping straight to "Use The
Installer" whenever the target host consumes any secret managed by the
`nixos` flake's sops-nix setup (as of this writing, that's every host —
`modules/common/configuration.nix` puts the root/nixos password hash and
the GitHub token behind sops-nix for all of them).

### Why the extra step exists

The `nixos` flake manages secrets with sops-nix, using an age key derived
from each host's own `/etc/ssh/ssh_host_ed25519_key`. That key is normally
generated by a systemd service (`sshd-keygen`) the first time a host boots
— but sops-nix decrypts secrets during system *activation*, which runs
**before** systemd starts pursuing the target `sshd-keygen` is gated
behind. On a genuinely fresh install, that means secrets — including the
login password — fail to decrypt on the very first boot unless the host
key already exists beforehand. Pre-seeding it is what fixes that.

Machines that don't consume any sops-nix secret don't strictly need this,
but since the shared password hash currently applies to every host, treat
it as required unless you've specifically confirmed otherwise.

### Steps

**1. On your admin workstation** (the machine with `~/.config/sops/age/keys.txt`
   from the `nixos` repo's sops-nix setup), decide the new machine's flake
   target name (`<platform>-<buildtype>`, e.g. `proxmox-server`) and
   generate + register its host key:

   ```sh
   ./scripts/prepare-host-key.sh <hostname> [path-to-nixos-repo]
   ```

   This generates `host-keys/<hostname>_ssh_host_ed25519_key(.pub)`
   locally and prints:

   - the line to add under `keys:` in `nixos/.sops.yaml`
   - which `creation_rules` key_groups to add it to (`secrets/common.yaml`
     always; a per-host `secrets/<hostname>.yaml` too if this build type
     has its own secrets, same pattern as `nix-cache`/`server`)

**2. Edit `nixos/.sops.yaml`** by hand with the printed snippet, then
   re-encrypt every secrets file you added the new host to:

   ```sh
   nix-shell -p sops --run 'sops updatekeys nixos/secrets/common.yaml'
   ```

**3. Commit and push the `nixos` repo.** The flake build the installer
   uses has to see the new recipient before you install, or decryption
   will fail on first boot regardless of step 4 below.

**4. Build and boot the installer image** on the target machine (ISO,
   netboot, or Proxmox LXC — see "Build The Installer ISO" / "Build
   Netboot Components" above).

**5. Copy the generated private key onto the live installer environment**
   once it's reachable over SSH:

   ```sh
   scp host-keys/<hostname>_ssh_host_ed25519_key{,.pub} root@<target-ip>:/root/host-keys/
   ```

**6. Log in** (as `root` or `nixos`) to trigger `/etc/auto-install.sh`
   automatically, or run it manually if it doesn't (see Troubleshooting).
   Select `<hostname>`'s flake target from the menu and confirm. The
   script will:

   - run Disko to format and mount `/mnt`
   - find the key you copied in at `/root/host-keys/<hostname>_ssh_host_ed25519_key`
     and install it to `/mnt/etc/ssh/` — if it's missing, the script
     warns and asks for confirmation before continuing without it (fine
     for a host with no sops-nix secrets; for any other host, continuing
     anyway means the machine will boot with no working login password)
   - run `nixos-install` and reboot

**7. Verify after reboot.** SSH into the new machine and confirm secrets
   actually decrypted:

   ```sh
   ls /run/secrets/
   ```

   If that's empty or login fails, the most likely cause is the host's
   age key not being in `.sops.yaml` (or not re-encrypted into the
   secrets file it needs) at the time `nixos-install` ran — re-check
   steps 1–3, fix `nixos/.sops.yaml`/`secrets/*.yaml`, push, then
   re-run `nixos-install --flake <flake>#<hostname> --no-root-password`
   from a rescue/live environment against the existing `/mnt` (or just
   redo the install).

`host-keys/` is gitignored — never commit private key material.

## Configuration Notes

The installer currently assumes:

* Target flake:

  ```text
  git+https://gitea.lan.ddnsgeek.com/beatzaplenty/nixos.git
  ```

* Host definitions are under:

  ```nix
  nixosConfigurations
  ```

* The installer has network access before installation.

* SSH is enabled.

* Root SSH login is permitted.

* The timezone is:

  ```text
  Australia/Brisbane
  ```

* The target host is responsible for defining its own storage layout.

Change these values in `common.nix` if your environment differs.

## Security Notes

`common.nix` currently contains:

* Git credentials
* Password hashes
* SSH public keys

Treat the repository and generated installer images as sensitive.

For production use, consider replacing embedded credentials with:

* Short-lived tokens
* SSH deploy keys
* External secrets management
* Runtime credential injection

## Safety Warnings

This installer is destructive.

The command:

```sh
disko --mode destroy,format,mount
```

will erase any disks defined by the selected host's Disko configuration.

Always verify:

* The selected host profile.
* The Disko device paths.
* The target machine.

Never boot this installer on a machine containing important data unless the
storage configuration has been reviewed.

## Troubleshooting

### No hosts appear in the menu

Check that the installer can reach the flake repository:

```sh
nix eval --json \
  "git+https://gitea.lan.ddnsgeek.com/beatzaplenty/nixos.git#nixosConfigurations" \
  --apply builtins.attrNames
```

### Selected host has no Disko configuration

The installer checks for:

```nix
config.disko.devices
```

Hosts without Disko storage definitions must either:

* Add a Disko module, or
* Be installed using another storage method.

### Disko evaluation fails

Check the selected host locally:

```sh
nix eval \
  ".#nixosConfigurations.<host>.config.disko.devices"
```

### Installation fails fetching the flake

Verify:

* Network connectivity.
* Git credentials.
* Flake input availability.

### Installer does not start automatically

It's triggered by `programs.bash.loginShellInit` (rendered into
`/etc/profile`), which only fires for genuine login shells. If you're not
seeing it — e.g. you attached to an existing shell rather than logging in
fresh — run it manually:

```sh
sudo /etc/auto-install.sh
```

### Login succeeds but the machine has no working password after install

The new host's sops-nix secrets didn't decrypt on first boot — almost
always because its age key wasn't registered in `nixos/.sops.yaml` and
re-encrypted into the secrets file(s) it needs before `nixos-install` ran.
See "Creating a New Machine" above, specifically step 7.