From ff021ca6f70349a50a54b1e4884e6434e92b9a68 Mon Sep 17 00:00:00 2001 From: beatzaplenty Date: Mon, 20 Jul 2026 07:02:23 +1000 Subject: [PATCH] Consolidate CI on codex-maintenance.sh; clean up dead scripts and docs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both check-nixos.yml workflows (GitHub + Gitea) now call scripts/codex-maintenance.sh instead of a hand-rolled eval-only loop, closing a real gap: CI previously enforced none of the secret grep, nixpkgs-fmt, or statix checks that codex-maintenance.sh already runs locally — nothing was stopping that from regressing. One script now backs both, instead of two copies that can drift from each other. codex-maintenance.sh itself is extended to cover buildable surface that wasn't validated anywhere before: packages.x86_64-linux.*, plus config.system.build.tarball (lxc-* hosts) and config.system.build.diskoImagesScript (proxmox-*, excluding the installer's own proxmox-lxc target, which has no disko config). Also: - scripts/prepare-host-key.sh: dropped the redundant [path-to-nixos-repo] parameter — it always defaults to the repo the script itself lives in now, so a second argument never made sense after the nix-auto-installer migration. - Removed prepare.sh (dead pre-disko manual parted/mkfs/mkswap partitioning, fully superseded) and scripts/create-linode-installer-disk.sh (incomplete draft for an abandoned dd-via-rescue-mode approach; Linode hosts already deploy fine through the normal auto-installer flow). - docs/pxe-boot.md: fixed a stale `nixosConfigurations.pxe-boot` eval command (pre-refactor flat name, not a real flake attribute anymore) and added a cross-reference to docs/auto-installer.md. - CLAUDE.md/README.md: full documentation pass reconciling this session's changes — modules/installer/, modules/pxe-boot/, the LXC/Proxmox image-building deployment paths, corrected the password-hash/SSH-key locations in the safety-rules section (both had drifted to reference files/paths that no longer exist), and added session-workflow guidance to prefer targeted host evals over full-repo sweeps for incremental changes (explicitly scoped to interactive sessions, not CI). Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01La55Nsss8jZ7ZuzUV9mfot --- .gitea/workflows/check-nixos.yml | 12 +----- .github/workflows/check-nixos.yml | 12 +----- CLAUDE.md | 54 +++++++++++++++++++++---- README.md | 23 ++++++++++- docs/pxe-boot.md | 7 +++- prepare.sh | 30 -------------- scripts/codex-maintenance.sh | 46 +++++++++++++++++++++ scripts/create-linode-installer-disk.sh | 9 ----- scripts/prepare-host-key.sh | 9 ++--- 9 files changed, 128 insertions(+), 74 deletions(-) delete mode 100755 prepare.sh delete mode 100644 scripts/create-linode-installer-disk.sh diff --git a/.gitea/workflows/check-nixos.yml b/.gitea/workflows/check-nixos.yml index 0645d9f..2e8db3c 100644 --- a/.gitea/workflows/check-nixos.yml +++ b/.gitea/workflows/check-nixos.yml @@ -17,13 +17,5 @@ jobs: - name: Install Nix uses: DeterminateSystems/nix-installer-action@v19 - - name: Evaluate all NixOS hosts - run: | - set -euo pipefail - hosts="$(nix --extra-experimental-features 'nix-command flakes' eval --json \ - .#nixosConfigurations --apply builtins.attrNames | jq -r '.[]')" - for host in $hosts; do - echo "Evaluating ${host}" - nix --extra-experimental-features 'nix-command flakes' eval \ - ".#nixosConfigurations.${host}.config.system.build.toplevel.drvPath" --raw - done + - name: Run maintenance checks (secrets, fmt, lint, eval) + run: bash scripts/codex-maintenance.sh diff --git a/.github/workflows/check-nixos.yml b/.github/workflows/check-nixos.yml index 0645d9f..2e8db3c 100644 --- a/.github/workflows/check-nixos.yml +++ b/.github/workflows/check-nixos.yml @@ -17,13 +17,5 @@ jobs: - name: Install Nix uses: DeterminateSystems/nix-installer-action@v19 - - name: Evaluate all NixOS hosts - run: | - set -euo pipefail - hosts="$(nix --extra-experimental-features 'nix-command flakes' eval --json \ - .#nixosConfigurations --apply builtins.attrNames | jq -r '.[]')" - for host in $hosts; do - echo "Evaluating ${host}" - nix --extra-experimental-features 'nix-command flakes' eval \ - ".#nixosConfigurations.${host}.config.system.build.toplevel.drvPath" --raw - done + - name: Run maintenance checks (secrets, fmt, lint, eval) + run: bash scripts/codex-maintenance.sh diff --git a/CLAUDE.md b/CLAUDE.md index 7f7f9bd..f75beba 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -17,11 +17,19 @@ machines when deployed. - Validation is limited to evaluation, linting, formatting checks, and `nix build --dry-run --no-link`. - Do not add secrets, tokens, private keys, or new password hashes to the repo. -- This repo currently contains **committed password hashes** (e.g. - `prepare.sh`, `hosts/nixos/configuration.nix`) and SSH public keys (e.g. - `modules/nix-cache/server.nix`). The hashes are known tech debt — do not use - them as a template for new hosts, and flag any *new* secret-like string you - encounter instead of committing it. +- This repo currently contains **committed password hashes** in + `modules/installer/common.nix` (the auto-installer's own root/nixos login — + a deliberate, documented choice, see `docs/auto-installer.md`, not + accidental tech debt) and **SSH public keys** in `variables.nix` + (`vars.adminSshKey`, `vars.remoteBuilderAuthorizedKeys`) plus a couple of + per-host `KEY` values for beszel-agent auth (`hosts/server/host.nix`, + `hosts/nix-cache/host.nix`). Don't use the installer's hardcoded hash as a + template for a *real* host — every other host uses sops-nix + (`hashedPasswordFile`, see "Security Notes" in `README.md`). Flag any *new* + secret-like string you encounter instead of committing it. +- `host-keys/` is gitignored — locally-generated *private* SSH host keys for + the auto-installer (see `docs/auto-installer.md`). Never commit its + contents; if `git status` ever shows it as trackable, something is wrong. ## Commands @@ -50,6 +58,18 @@ maintenance script pulls them via `nix run github:NixOS/nixpkgs/nixos-25.11#` you touched (plus any `config.system.build.tarball` +/`diskoImagesScript`/package output affected) rather than looping over every +host — `codex-maintenance.sh` evaluates 19 hosts plus every package/tarball/ +image variant now and is slow to run after each small change. Reserve a full +`codex-maintenance.sh` run for changes that plausibly affect every host +(`modules/common/*`, `flake.nix`, `variables.nix`) or as a final check before +committing. This is a session-workflow preference only — it does not apply to +CI, which should keep running the full script on every push/PR regardless of +diff size; that's the point of it. + ## Architecture `flake.nix` is the single entry point. It generates one @@ -93,7 +113,10 @@ removing a host. hardware config, imported directly by the platform module itself (`../hardware-configuration/vm/{proxmox,linode}.nix`) — **not** wired in from `flake.nix`. `lxc.nix` has no hardware-configuration counterpart since - containers share the host kernel. + containers share the host kernel; instead it imports nixpkgs' own + `virtualisation/proxmox-lxc.nix`, which gives every `lxc-*` host a + `config.system.build.tarball` output (`pct restore`-ready, no install step + — see `docs/auto-installer.md`). - `modules/build-types/*.nix` — what a system is for: minimal/server/docker/gui/pxe-boot/nix-cache. - `modules/common/configuration.nix` — base NixOS config imported by every @@ -102,8 +125,19 @@ removing a host. the `nixos` user; the `nixos` workstation (`gui` build type) has its own, other hosts share `modules/common/home.nix`. - `modules/disko/proxmox.nix` — declarative disk layout (GPT: ESP + swap + - ext4 root) via disko, used by all Proxmox-VM hosts. + ext4 root) via disko, used by all Proxmox-VM hosts (`proxmox-*`, not + `lxc-*`). Also carries `imageSize`/`imageName`, letting every `proxmox-*` + host be built as a standalone, `qm importdisk`-ready `.raw` image with no + install step — see `docs/proxmox-images.md`. - `modules/boot/efi.nix` — systemd-boot + EFI vars, paired with the disko module. +- `modules/installer/` — the auto-installer environment (ISO/netboot/Proxmox + LXC): `common.nix` (shared config + the generated `auto-install.sh`), + `iso.nix`/`proxmox-lxc.nix` (per-variant), `host-keys.nix` (optionally + bakes `host-keys/` into the image under `--impure`). See + `docs/auto-installer.md`. +- `modules/pxe-boot/stage-installer-artifacts.nix` — builds the installer's + netboot image and stages it on the `pxe-boot` host so its iPXE menu can + chain straight to it. See `docs/pxe-boot.md`. - `modules/nix-cache/{client,server,remote-builder-client}.nix` — binary cache substituter + SSH remote-builder wiring; see `docs/nix-cache.md` for the full design (per-host local stores, no shared `/nix/store`, and how the @@ -129,6 +163,12 @@ duplicating config. handling. - `docs/pxe-boot.md` — the `pxe-boot` host's iPXE/TFTP/HTTP boot chain and directory layout under `/srv/pxe`. +- `docs/auto-installer.md` — the installer environment (ISO/netboot/Proxmox + LXC), `host-keys/` and the sops-nix pre-seeding problem it solves, and why + `lxc-*` hosts are deliberately excluded from its menu. +- `docs/proxmox-images.md` — building `proxmox-*` hosts as standalone `.raw` + disk images (disko's image builder) instead of installing, and deploying + the result to Proxmox. - `docs/flake-lock-automation.md` — how `flake.lock` updates flow through CI (scheduled `nix flake update` PR + host-eval-on-PR workflow) and why hosts should track the committed lock file rather than `nixos-rebuild --upgrade-all`. diff --git a/README.md b/README.md index a6bb5c5..6265fd9 100644 --- a/README.md +++ b/README.md @@ -55,7 +55,7 @@ nix eval --json .#nixosConfigurations --apply builtins.attrNames | jq -r '.[]' | `modules/nix-cache/` | Binary cache and remote builder client/server modules | | `modules/installer/` | Auto-installer environment (ISO/netboot/Proxmox LXC) — see `docs/auto-installer.md` | | `host-keys/` | Gitignored, locally-generated SSH host keys for the auto-installer — see `docs/auto-installer.md` | -| `docs/` | Operational notes for cache, builders, lock updates, boot services, and the auto-installer | +| `docs/` | Operational notes for cache, builders, lock updates, boot services, the auto-installer, and Proxmox image builds | | `scripts/` | Codex setup, validation, and host-key-prep helpers | ## Validation @@ -87,6 +87,20 @@ review sessions. client hosts. - `pxe-boot` serves iPXE boot files over HTTP from `/srv/pxe`. +### Deploying a new host + +Three different paths depending on target, none of them involving a manual +`nixos-rebuild switch` from this repo: + +- Most hosts: boot the auto-installer, pick the target from its menu — see + `docs/auto-installer.md`. +- `lxc-*` targets: not installed at all — build a ready-to-run container + tarball and `pct restore` it directly. `docs/auto-installer.md` covers why + (and the installer's menu excludes them for the same reason). +- `proxmox-*` targets: can alternatively be built as a standalone `.raw` + disk image and attached to a new VM with no install step — see + `docs/proxmox-images.md`. + ## Security Notes Do not commit tokens, private keys, live credentials, or new password hashes @@ -107,6 +121,13 @@ enabled via `git config core.hooksPath .githooks`, done automatically by `scripts/codex-setup.sh`) runs `gitleaks protect --staged` to catch mistakes before they're committed. +The auto-installer environment is the one deliberate exception to +sops-nix-everywhere: it has a hardcoded login password instead (no stable +per-boot host key for sops-nix to derive from on ephemeral media) — see +"Host keys" in `docs/auto-installer.md` for why, and how the private keys it +*does* pre-seed for target hosts stay out of git via the gitignored +`host-keys/` directory. + This repository's git *history* still contains secrets committed before this migration (see `remove-sensetive-info-refactor.md`) — those are being scrubbed and rotated separately; don't treat the repo as safe to make public diff --git a/docs/pxe-boot.md b/docs/pxe-boot.md index b852522..7529768 100644 --- a/docs/pxe-boot.md +++ b/docs/pxe-boot.md @@ -1,6 +1,9 @@ # pxe-boot -The `pxe-boot` host serves HTTP boot assets for iPXE clients. +The `pxe-boot` host serves HTTP boot assets for iPXE clients — including a +self-staged copy of this flake's own auto-installer netboot image, see +`docs/auto-installer.md` for what that image actually is and does once +booted. ## Host Role @@ -83,7 +86,7 @@ uses `archiso_http_srv` to fetch the squashfs payload over HTTP. Safe evaluation check: ```bash -nix eval .#nixosConfigurations.pxe-boot.config.system.build.toplevel.drvPath --raw +nix eval .#nixosConfigurations.proxmox-pxe-boot.config.system.build.toplevel.drvPath --raw ``` After deployment by an operator, basic service checks are: diff --git a/prepare.sh b/prepare.sh deleted file mode 100755 index 9e38eb1..0000000 --- a/prepare.sh +++ /dev/null @@ -1,30 +0,0 @@ -#create MBR table -parted /dev/sda -- mklabel msdos -#create nixos partition -parted /dev/sda -- mkpart primary 1MB -8GB -#set nixos partition to bootable -parted /dev/sda -- set 1 boot on -# create swap partition -parted /dev/sda -- mkpart primary linux-swap -8GB 100% - -#format OS partition -mkfs.ext4 -L nixos /dev/sda1 -#format swap -mkswap -L swap /dev/sda2 - -#activate swap -swapon /dev/sda2 - -#mount nixos partition -mount /dev/disk/by-label/nixos /mnt -export TMPDIR=/mnt/install-tmp -mkdir -p /mnt/install-tmp -#Generate config -#nixos-generate-config --root /mnt/ - -#copy customised configuration over -#cp configuration.nix /mnt/etc/nixos/configuration.nix - -#nixos-install --no-root-passwd - -#reboot \ No newline at end of file diff --git a/scripts/codex-maintenance.sh b/scripts/codex-maintenance.sh index 5c4c2bf..f8e556d 100755 --- a/scripts/codex-maintenance.sh +++ b/scripts/codex-maintenance.sh @@ -55,6 +55,33 @@ echo "Evaluating host toplevel derivations..." for host in $hosts; do echo "==> $host" nix eval --raw --no-use-registries --no-accept-flake-config ".#nixosConfigurations.${host}.config.system.build.toplevel.drvPath" + + # lxc-* hosts deploy via a directly pct-restore-able tarball instead of + # nixos-install (see docs/auto-installer.md); proxmox-* hosts (excluding + # proxmox-lxc, the installer's own LXC target, which has no disko config) + # can alternatively be built as a standalone disk image (see + # docs/proxmox-images.md). Both are otherwise-unvalidated buildable + # surface, easy to silently break without this. + case "$host" in + lxc-*) + echo "==> $host (tarball)" + nix eval --raw --no-use-registries --no-accept-flake-config ".#nixosConfigurations.${host}.config.system.build.tarball.drvPath" + ;; + proxmox-lxc) ;; + proxmox-*) + echo "==> $host (diskoImagesScript)" + nix eval --raw --no-use-registries --no-accept-flake-config ".#nixosConfigurations.${host}.config.system.build.diskoImagesScript.drvPath" + ;; + esac +done + +echo +echo "Evaluating buildable packages..." +packages_json="$(nix eval --json --no-use-registries --no-accept-flake-config .#packages.x86_64-linux --apply builtins.attrNames)" +packages="$(echo "$packages_json" | jq -r '.[]')" +for pkg in $packages; do + echo "==> packages.x86_64-linux.${pkg}" + nix eval --raw --no-use-registries --no-accept-flake-config ".#packages.x86_64-linux.${pkg}" done if [[ "$MODE" == "dry-run" ]]; then @@ -63,6 +90,25 @@ if [[ "$MODE" == "dry-run" ]]; then for host in $hosts; do echo "==> Dry-run build: $host" nix build --dry-run --no-link --no-use-registries --no-accept-flake-config ".#nixosConfigurations.${host}.config.system.build.toplevel" + + case "$host" in + lxc-*) + echo "==> Dry-run build: $host (tarball)" + nix build --dry-run --no-link --no-use-registries --no-accept-flake-config ".#nixosConfigurations.${host}.config.system.build.tarball" + ;; + proxmox-lxc) ;; + proxmox-*) + echo "==> Dry-run build: $host (diskoImagesScript)" + nix build --dry-run --no-link --no-use-registries --no-accept-flake-config ".#nixosConfigurations.${host}.config.system.build.diskoImagesScript" + ;; + esac + done + + echo + echo "Running dry-run builds for all packages." + for pkg in $packages; do + echo "==> Dry-run build: packages.x86_64-linux.${pkg}" + nix build --dry-run --no-link --no-use-registries --no-accept-flake-config ".#packages.x86_64-linux.${pkg}" done fi diff --git a/scripts/create-linode-installer-disk.sh b/scripts/create-linode-installer-disk.sh deleted file mode 100644 index 43a60f5..0000000 --- a/scripts/create-linode-installer-disk.sh +++ /dev/null @@ -1,9 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -#boot to rescue mode -# set root password -scp $RESULT_ISO root@$LINODE_IP:/tmp/nixos-auto.iso - -#in LISH or ssh to rescue mode -dd if=/tmp/nixos.iso of=/dev/sda bs=4M status=progress conv=fsync \ No newline at end of file diff --git a/scripts/prepare-host-key.sh b/scripts/prepare-host-key.sh index 5ee83f0..07910ac 100755 --- a/scripts/prepare-host-key.sh +++ b/scripts/prepare-host-key.sh @@ -20,12 +20,11 @@ set -euo pipefail repo_root="$(cd "$(dirname "$0")/.." && pwd)" -hostname="${1:?usage: scripts/prepare-host-key.sh [path-to-nixos-repo]}" -nixos_repo="${2:-$repo_root}" -sops_yaml="${nixos_repo}/.sops.yaml" +hostname="${1:?usage: scripts/prepare-host-key.sh }" +sops_yaml="${repo_root}/.sops.yaml" if [[ ! -f "$sops_yaml" ]]; then - echo "ERROR: $sops_yaml not found. Pass the nixos repo path as the 2nd argument." >&2 + echo "ERROR: $sops_yaml not found — is this script still under nixos/scripts/?" >&2 exit 1 fi @@ -55,7 +54,7 @@ Generated: ${keyfile}(.pub) nix-cache/server.) === 3. Re-encrypt every secrets file you just added it to === - nix-shell -p sops --run 'sops updatekeys ${nixos_repo}/secrets/common.yaml' + nix-shell -p sops --run 'sops updatekeys ${repo_root}/secrets/common.yaml' === 4. Commit + push this repo so the flake build picks up the new recipient ===