beatzaplenty/nixos
Archived
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
This repository has been archived on 2026-07-30. You can view files and clone it. You cannot open issues or pull requests or push a commit.
Files
40856b2e5eddeb753a8f7955b0f756590389f422
nixos/docs/pxe-boot.md
T
beatzaplentyandClaude Sonnet 4.6 40856b2e5e feat(pxe-boot): add FreeIPA Server (Rocky Linux 9) iPXE menu entry 
Adds an unattended install option to the PXE boot menu that installs
Rocky Linux 9 and configures FreeIPA on the domain-controller.sweet.home
host without any operator interaction after selecting the menu entry.

How it works:
- fetch-rocky-pxeboot.service downloads the Rocky 9 Anaconda pxeboot
  kernel and initrd from the Rocky mirror on first pxe-boot deploy
  (idempotent, same pattern as fetch-debian-netboot)
- rocky-freeipa.ipxe boots Anaconda with inst.ks pointing at the
  hosted Kickstart and net.ifnames=0 biosdevname=0 for stable eth0
- rocky-freeipa.ks (generated, includes vars.adminSshKey) performs:
  - Minimal Rocky 9 install with ipa-server + ipa-server-dns
  - Static IP 192.168.2.138 via NM connection file written in %post
  - /etc/hosts fixed for FreeIPA FQDN requirement
  - Random DM + admin passwords generated and saved to
    /root/ipa-credentials.txt (chmod 600, never hardcoded)
  - freeipa-first-boot.service oneshot enabled to run
    ipa-server-install on the first real boot (~20 min)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Jbvxx4xbHVcx1NkK3vtmK
2026-07-27 07:35:13 +10:00

8.3 KiB
Raw Blame History

pxe-boot

The pxe-boot host serves HTTP boot assets for iPXE clients — including self-staged copies of both this flake's own auto-installer netboot image (see docs/auto-installer.md for what that image actually is and does once booted) and a vanilla, unmodified NixOS minimal netboot image for plain rescue/inspection use.

Host Role

  • Hostname: pxe-boot
  • Web service: nginx on TCP port 80
  • PXE root: /srv/pxe
  • HTTP root for scripts and images: /srv/pxe/http
  • TFTP root for first-stage bootloaders: /srv/pxe/tftp
  • iPXE entry script: /srv/pxe/http/boot.ipxe
  • Generated iPXE menu: /srv/pxe/http/menu.ipxe
  • Debian Minimal iPXE script: /srv/pxe/http/debian.ipxe
  • SystemRescue iPXE script: /srv/pxe/http/systemrescue.ipxe
  • TFTP fallback script: /srv/pxe/tftp/autoexec.ipxe
  • Boot binaries copied from the Nix ipxe package:
    • /srv/pxe/tftp/ipxe.efi
    • /srv/pxe/tftp/undionly.kpxe

Directory Layout

The host creates these directories with systemd tmpfiles:

/srv/pxe
/srv/pxe/http
/srv/pxe/http/images  -> /mnt/pxe-images  (symlink to NFS share)
/srv/pxe/http/auto-installer
/srv/pxe/http/nixos-minimal
/srv/pxe/http/debian
/srv/pxe/http/systemrescue
/srv/pxe/http/ubuntu
/srv/pxe/http/rescue
/srv/pxe/tftp

/srv/pxe/http/images is a symlink to /mnt/pxe-images, which is an NFS mount of server.sweet.home:/tank/pxe-boot/images (modules/pxe-boot/mount-pxe-images.nix). Place large images there (ISOs, disk images) rather than on the pxe-boot host's own root disk. For an LXC pxe-boot container the mount uses NFSv3+nolock with nofail (eager, non-blocking on server unavailability); for a Proxmox VM it uses NFSv4.2 with x-systemd.automount (lazy, triggered on first access).

When running as lxc-pxe-boot, the Proxmox container must have features: nesting=1,mount=nfs (at minimum) in its Proxmox config. nesting=1 is required by systemd 260+ for credential isolation (user namespace creation and internal move-mounts); without it, AppArmor denies both, and every systemd service that uses PrivateUsers, PrivateDevices, or credential passing fails on boot. mount=nfs allows the NFSv3 mount. Both are set automatically by scripts/proxmox/create-proxmox-resource.sh (via PROXMOX_DEFAULT_LXC_FEATURES in scripts/env.sh which defaults to nesting=1,keyctl=1,mount=nfs;nfs4). If you ever change these features manually via pct set, be sure to include both — pct set replaces the entire features string, it does not append to it.

The HTTP iPXE chain is:

undionly.kpxe or ipxe.efi
  -> autoexec.ipxe from the TFTP root, when iPXE requests it
  -> http://192.168.2.247/boot.ipxe
  -> http://192.168.2.247/menu.ipxe

The generated menu currently exposes entries for:

  • NixOS Auto-Installer
  • NixOS Minimal
  • Debian Minimal
  • FreeIPA Server (Rocky Linux 9)
  • SystemRescue environment
  • iPXE shell
  • Reboot

Both NixOS entries chain-load a netboot.ipxe staged into their own directory (/srv/pxe/http/auto-installer/netboot.ipxe and /srv/pxe/http/nixos-minimal/netboot.ipxe), each nixpkgs' own generated netboot iPXE script (correct init=/initrd= kernel parameters included) rather than a hand-rolled boot line — that script in turn expects its kernel/initrd siblings in the same directory. Each directory's three files (bzImage, initrd, netboot.ipxe) are built from source and staged automatically by modules/pxe-boot/stage-installer-artifacts.nix via systemd.tmpfiles.rules — no manual operator step required:

  • auto-installer is this flake's own netbootSystem (flake.nix) — the same auto-installer image nix build .#pxe produces. See docs/auto-installer.md.
  • nixos-minimal is netbootMinimalSystem (flake.nix) — nixpkgs' netboot-minimal.nix composed on its own, with none of this flake's auto-installer wiring (no common.nix, no auto-install.sh, no baked host keys or custom users). Same nix build .#pxe-minimal mechanism as the auto-installer image, just a different module composition. Useful as a plain rescue/inspection shell that doesn't assume anything about this flake.

Both images set networking.hostName to match their menu entry/staged directory name (auto-installer / nixos-minimal), so each one's generated system name (nixos-system-<name>-*) is self-describing rather than the nixpkgs default of nixos-system-nixos-* for both.

The Debian Minimal entry chains http://<pxeServerIp>/debian.ipxe, which loads the Debian bookworm netboot kernel and initrd from /srv/pxe/http/debian/. The fetch-debian-netboot.service oneshot downloads these files from deb.debian.org on first boot (idempotent — skips if files are already present):

/srv/pxe/http/debian/linux      (Debian bookworm netboot kernel)
/srv/pxe/http/debian/initrd.gz  (Debian bookworm netboot initrd)

The service requires outbound internet access on the pxe-boot host. To re-download (e.g. after a Debian point release), delete the files and restart the service:

rm /srv/pxe/http/debian/linux /srv/pxe/http/debian/initrd.gz
systemctl restart fetch-debian-netboot.service

To update to a different Debian release, change debianRelease in modules/build-types/pxe-boot.nix and redeploy.

The FreeIPA Server (Rocky Linux 9) entry chains http://<pxeServerIp>/rocky-freeipa.ipxe, which boots the Rocky Linux 9 Anaconda installer with a Kickstart file (rocky-freeipa.ks) hosted on the same server. The fetch-rocky-pxeboot.service oneshot downloads the pxeboot kernel and initrd from the Rocky Linux mirror on first boot (idempotent):

/srv/pxe/http/rocky/vmlinuz     (Rocky Linux 9 Anaconda pxeboot kernel)
/srv/pxe/http/rocky/initrd.img  (Rocky Linux 9 Anaconda pxeboot initrd)

The Kickstart file is generated from the NixOS module and staged at /srv/pxe/http/rocky-freeipa.ks. It performs a fully unattended install:

  1. Installs Rocky Linux 9 with ipa-server + ipa-server-dns packages
  2. Configures static IP 192.168.2.138, hostname domain-controller.sweet.home
  3. Creates user wayne with the adminSshKey from variables.nix
  4. Generates random IPA passwords and writes them to /root/ipa-credentials.txt
  5. Creates a freeipa-first-boot.service oneshot that runs ipa-server-install on first reboot (~20 minutes)

After the install completes:

  • SSH in as wayne@domain-controller using the admin key
  • Monitor FreeIPA install progress: sudo tail -f /root/freeipa-install.log
  • Retrieve credentials: sudo cat /root/ipa-credentials.txt (save to password manager)
  • Configure Pi-hole: server=/sweet.home/192.168.2.138 in dnsmasq

To refresh the pxeboot files (e.g. after a Rocky point release):

rm /srv/pxe/http/rocky/vmlinuz /srv/pxe/http/rocky/initrd.img
systemctl restart fetch-rocky-pxeboot.service

To update to a different Rocky release, change rockyRelease in modules/build-types/pxe-boot.nix and redeploy.

The SystemRescue entry expects the source ISO at:

/srv/pxe/http/images/systemrescue.iso

Since /srv/pxe/http/images is the NFS-backed symlink, place the ISO on the NFS share at server.sweet.home:/tank/pxe-boot/images/systemrescue.iso.

The stage-systemrescue.service oneshot extracts that ISO into:

/srv/pxe/http/systemrescue

The rescue menu entry then chains http://192.168.2.247/systemrescue.ipxe, which loads the SystemRescue kernel and initramfs from the extracted tree and uses archiso_http_srv to fetch the squashfs payload over HTTP.

Validation

Safe evaluation check:

nix eval .#nixosConfigurations.proxmox-pxe-boot.config.system.build.toplevel.drvPath --raw

After deployment by an operator, basic service checks are:

curl http://pxe-boot/boot.ipxe
curl http://pxe-boot/menu.ipxe
curl http://pxe-boot/debian.ipxe
curl -I http://pxe-boot/debian/linux
curl -I http://pxe-boot/debian/initrd.gz
curl http://pxe-boot/rocky-freeipa.ipxe
curl http://pxe-boot/rocky-freeipa.ks
curl -I http://pxe-boot/rocky/vmlinuz
curl -I http://pxe-boot/rocky/initrd.img
curl http://pxe-boot/systemrescue.ipxe
curl -I http://pxe-boot/systemrescue/sysresccd/boot/x86_64/vmlinuz
curl -I http://pxe-boot/systemrescue/sysresccd/boot/x86_64/sysresccd.img

During a successful BIOS chainload, TFTP should deliver undionly.kpxe once, then nginx should log requests for /boot.ipxe and /menu.ipxe. Repeated TFTP downloads of undionly.kpxe indicate the iPXE stage is still not reaching the HTTP chain.

Reference in New Issue View Git Blame Copy Permalink