Archived
Consolidate CI on codex-maintenance.sh; clean up dead scripts and docs
Check NixOS configurations / eval-hosts (push) Failing after 53m34s
Check NixOS configurations / eval-hosts (push) Failing after 53m34s
Both check-nixos.yml workflows (GitHub + Gitea) now call scripts/codex-maintenance.sh instead of a hand-rolled eval-only loop, closing a real gap: CI previously enforced none of the secret grep, nixpkgs-fmt, or statix checks that codex-maintenance.sh already runs locally — nothing was stopping that from regressing. One script now backs both, instead of two copies that can drift from each other. codex-maintenance.sh itself is extended to cover buildable surface that wasn't validated anywhere before: packages.x86_64-linux.*, plus config.system.build.tarball (lxc-* hosts) and config.system.build.diskoImagesScript (proxmox-*, excluding the installer's own proxmox-lxc target, which has no disko config). Also: - scripts/prepare-host-key.sh: dropped the redundant [path-to-nixos-repo] parameter — it always defaults to the repo the script itself lives in now, so a second argument never made sense after the nix-auto-installer migration. - Removed prepare.sh (dead pre-disko manual parted/mkfs/mkswap partitioning, fully superseded) and scripts/create-linode-installer-disk.sh (incomplete draft for an abandoned dd-via-rescue-mode approach; Linode hosts already deploy fine through the normal auto-installer flow). - docs/pxe-boot.md: fixed a stale `nixosConfigurations.pxe-boot` eval command (pre-refactor flat name, not a real flake attribute anymore) and added a cross-reference to docs/auto-installer.md. - CLAUDE.md/README.md: full documentation pass reconciling this session's changes — modules/installer/, modules/pxe-boot/, the LXC/Proxmox image-building deployment paths, corrected the password-hash/SSH-key locations in the safety-rules section (both had drifted to reference files/paths that no longer exist), and added session-workflow guidance to prefer targeted host evals over full-repo sweeps for incremental changes (explicitly scoped to interactive sessions, not CI). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01La55Nsss8jZ7ZuzUV9mfot
This commit is contained in:
@@ -17,13 +17,5 @@ jobs:
|
||||
- name: Install Nix
|
||||
uses: DeterminateSystems/nix-installer-action@v19
|
||||
|
||||
- name: Evaluate all NixOS hosts
|
||||
run: |
|
||||
set -euo pipefail
|
||||
hosts="$(nix --extra-experimental-features 'nix-command flakes' eval --json \
|
||||
.#nixosConfigurations --apply builtins.attrNames | jq -r '.[]')"
|
||||
for host in $hosts; do
|
||||
echo "Evaluating ${host}"
|
||||
nix --extra-experimental-features 'nix-command flakes' eval \
|
||||
".#nixosConfigurations.${host}.config.system.build.toplevel.drvPath" --raw
|
||||
done
|
||||
- name: Run maintenance checks (secrets, fmt, lint, eval)
|
||||
run: bash scripts/codex-maintenance.sh
|
||||
|
||||
@@ -17,13 +17,5 @@ jobs:
|
||||
- name: Install Nix
|
||||
uses: DeterminateSystems/nix-installer-action@v19
|
||||
|
||||
- name: Evaluate all NixOS hosts
|
||||
run: |
|
||||
set -euo pipefail
|
||||
hosts="$(nix --extra-experimental-features 'nix-command flakes' eval --json \
|
||||
.#nixosConfigurations --apply builtins.attrNames | jq -r '.[]')"
|
||||
for host in $hosts; do
|
||||
echo "Evaluating ${host}"
|
||||
nix --extra-experimental-features 'nix-command flakes' eval \
|
||||
".#nixosConfigurations.${host}.config.system.build.toplevel.drvPath" --raw
|
||||
done
|
||||
- name: Run maintenance checks (secrets, fmt, lint, eval)
|
||||
run: bash scripts/codex-maintenance.sh
|
||||
|
||||
@@ -17,11 +17,19 @@ machines when deployed.
|
||||
- Validation is limited to evaluation, linting, formatting checks, and
|
||||
`nix build --dry-run --no-link`.
|
||||
- Do not add secrets, tokens, private keys, or new password hashes to the repo.
|
||||
- This repo currently contains **committed password hashes** (e.g.
|
||||
`prepare.sh`, `hosts/nixos/configuration.nix`) and SSH public keys (e.g.
|
||||
`modules/nix-cache/server.nix`). The hashes are known tech debt — do not use
|
||||
them as a template for new hosts, and flag any *new* secret-like string you
|
||||
encounter instead of committing it.
|
||||
- This repo currently contains **committed password hashes** in
|
||||
`modules/installer/common.nix` (the auto-installer's own root/nixos login —
|
||||
a deliberate, documented choice, see `docs/auto-installer.md`, not
|
||||
accidental tech debt) and **SSH public keys** in `variables.nix`
|
||||
(`vars.adminSshKey`, `vars.remoteBuilderAuthorizedKeys`) plus a couple of
|
||||
per-host `KEY` values for beszel-agent auth (`hosts/server/host.nix`,
|
||||
`hosts/nix-cache/host.nix`). Don't use the installer's hardcoded hash as a
|
||||
template for a *real* host — every other host uses sops-nix
|
||||
(`hashedPasswordFile`, see "Security Notes" in `README.md`). Flag any *new*
|
||||
secret-like string you encounter instead of committing it.
|
||||
- `host-keys/` is gitignored — locally-generated *private* SSH host keys for
|
||||
the auto-installer (see `docs/auto-installer.md`). Never commit its
|
||||
contents; if `git status` ever shows it as trackable, something is wrong.
|
||||
|
||||
## Commands
|
||||
|
||||
@@ -50,6 +58,18 @@ maintenance script pulls them via `nix run github:NixOS/nixpkgs/nixos-25.11#<too
|
||||
There is no test suite — "correctness" here means the flake evaluates and
|
||||
`nixpkgs-fmt`/`statix` are clean.
|
||||
|
||||
**In an interactive agent session**, prefer targeted checks over full-repo
|
||||
sweeps: after editing one or two hosts/modules, evaluate just the
|
||||
`nixosConfigurations.<host>` you touched (plus any `config.system.build.tarball`
|
||||
/`diskoImagesScript`/package output affected) rather than looping over every
|
||||
host — `codex-maintenance.sh` evaluates 19 hosts plus every package/tarball/
|
||||
image variant now and is slow to run after each small change. Reserve a full
|
||||
`codex-maintenance.sh` run for changes that plausibly affect every host
|
||||
(`modules/common/*`, `flake.nix`, `variables.nix`) or as a final check before
|
||||
committing. This is a session-workflow preference only — it does not apply to
|
||||
CI, which should keep running the full script on every push/PR regardless of
|
||||
diff size; that's the point of it.
|
||||
|
||||
## Architecture
|
||||
|
||||
`flake.nix` is the single entry point. It generates one
|
||||
@@ -93,7 +113,10 @@ removing a host.
|
||||
hardware config, imported directly by the platform module itself
|
||||
(`../hardware-configuration/vm/{proxmox,linode}.nix`) — **not** wired in
|
||||
from `flake.nix`. `lxc.nix` has no hardware-configuration counterpart since
|
||||
containers share the host kernel.
|
||||
containers share the host kernel; instead it imports nixpkgs' own
|
||||
`virtualisation/proxmox-lxc.nix`, which gives every `lxc-*` host a
|
||||
`config.system.build.tarball` output (`pct restore`-ready, no install step
|
||||
— see `docs/auto-installer.md`).
|
||||
- `modules/build-types/*.nix` — what a system is for:
|
||||
minimal/server/docker/gui/pxe-boot/nix-cache.
|
||||
- `modules/common/configuration.nix` — base NixOS config imported by every
|
||||
@@ -102,8 +125,19 @@ removing a host.
|
||||
the `nixos` user; the `nixos` workstation (`gui` build type) has its own,
|
||||
other hosts share `modules/common/home.nix`.
|
||||
- `modules/disko/proxmox.nix` — declarative disk layout (GPT: ESP + swap +
|
||||
ext4 root) via disko, used by all Proxmox-VM hosts.
|
||||
ext4 root) via disko, used by all Proxmox-VM hosts (`proxmox-*`, not
|
||||
`lxc-*`). Also carries `imageSize`/`imageName`, letting every `proxmox-*`
|
||||
host be built as a standalone, `qm importdisk`-ready `.raw` image with no
|
||||
install step — see `docs/proxmox-images.md`.
|
||||
- `modules/boot/efi.nix` — systemd-boot + EFI vars, paired with the disko module.
|
||||
- `modules/installer/` — the auto-installer environment (ISO/netboot/Proxmox
|
||||
LXC): `common.nix` (shared config + the generated `auto-install.sh`),
|
||||
`iso.nix`/`proxmox-lxc.nix` (per-variant), `host-keys.nix` (optionally
|
||||
bakes `host-keys/` into the image under `--impure`). See
|
||||
`docs/auto-installer.md`.
|
||||
- `modules/pxe-boot/stage-installer-artifacts.nix` — builds the installer's
|
||||
netboot image and stages it on the `pxe-boot` host so its iPXE menu can
|
||||
chain straight to it. See `docs/pxe-boot.md`.
|
||||
- `modules/nix-cache/{client,server,remote-builder-client}.nix` — binary cache
|
||||
substituter + SSH remote-builder wiring; see `docs/nix-cache.md` for the
|
||||
full design (per-host local stores, no shared `/nix/store`, and how the
|
||||
@@ -129,6 +163,12 @@ duplicating config.
|
||||
handling.
|
||||
- `docs/pxe-boot.md` — the `pxe-boot` host's iPXE/TFTP/HTTP boot chain and
|
||||
directory layout under `/srv/pxe`.
|
||||
- `docs/auto-installer.md` — the installer environment (ISO/netboot/Proxmox
|
||||
LXC), `host-keys/` and the sops-nix pre-seeding problem it solves, and why
|
||||
`lxc-*` hosts are deliberately excluded from its menu.
|
||||
- `docs/proxmox-images.md` — building `proxmox-*` hosts as standalone `.raw`
|
||||
disk images (disko's image builder) instead of installing, and deploying
|
||||
the result to Proxmox.
|
||||
- `docs/flake-lock-automation.md` — how `flake.lock` updates flow through CI
|
||||
(scheduled `nix flake update` PR + host-eval-on-PR workflow) and why hosts
|
||||
should track the committed lock file rather than `nixos-rebuild --upgrade-all`.
|
||||
|
||||
@@ -55,7 +55,7 @@ nix eval --json .#nixosConfigurations --apply builtins.attrNames | jq -r '.[]'
|
||||
| `modules/nix-cache/` | Binary cache and remote builder client/server modules |
|
||||
| `modules/installer/` | Auto-installer environment (ISO/netboot/Proxmox LXC) — see `docs/auto-installer.md` |
|
||||
| `host-keys/` | Gitignored, locally-generated SSH host keys for the auto-installer — see `docs/auto-installer.md` |
|
||||
| `docs/` | Operational notes for cache, builders, lock updates, boot services, and the auto-installer |
|
||||
| `docs/` | Operational notes for cache, builders, lock updates, boot services, the auto-installer, and Proxmox image builds |
|
||||
| `scripts/` | Codex setup, validation, and host-key-prep helpers |
|
||||
|
||||
## Validation
|
||||
@@ -87,6 +87,20 @@ review sessions.
|
||||
client hosts.
|
||||
- `pxe-boot` serves iPXE boot files over HTTP from `/srv/pxe`.
|
||||
|
||||
### Deploying a new host
|
||||
|
||||
Three different paths depending on target, none of them involving a manual
|
||||
`nixos-rebuild switch` from this repo:
|
||||
|
||||
- Most hosts: boot the auto-installer, pick the target from its menu — see
|
||||
`docs/auto-installer.md`.
|
||||
- `lxc-*` targets: not installed at all — build a ready-to-run container
|
||||
tarball and `pct restore` it directly. `docs/auto-installer.md` covers why
|
||||
(and the installer's menu excludes them for the same reason).
|
||||
- `proxmox-*` targets: can alternatively be built as a standalone `.raw`
|
||||
disk image and attached to a new VM with no install step — see
|
||||
`docs/proxmox-images.md`.
|
||||
|
||||
## Security Notes
|
||||
|
||||
Do not commit tokens, private keys, live credentials, or new password hashes
|
||||
@@ -107,6 +121,13 @@ enabled via `git config core.hooksPath .githooks`, done automatically by
|
||||
`scripts/codex-setup.sh`) runs `gitleaks protect --staged` to catch mistakes
|
||||
before they're committed.
|
||||
|
||||
The auto-installer environment is the one deliberate exception to
|
||||
sops-nix-everywhere: it has a hardcoded login password instead (no stable
|
||||
per-boot host key for sops-nix to derive from on ephemeral media) — see
|
||||
"Host keys" in `docs/auto-installer.md` for why, and how the private keys it
|
||||
*does* pre-seed for target hosts stay out of git via the gitignored
|
||||
`host-keys/` directory.
|
||||
|
||||
This repository's git *history* still contains secrets committed before this
|
||||
migration (see `remove-sensetive-info-refactor.md`) — those are being
|
||||
scrubbed and rotated separately; don't treat the repo as safe to make public
|
||||
|
||||
+5
-2
@@ -1,6 +1,9 @@
|
||||
# pxe-boot
|
||||
|
||||
The `pxe-boot` host serves HTTP boot assets for iPXE clients.
|
||||
The `pxe-boot` host serves HTTP boot assets for iPXE clients — including a
|
||||
self-staged copy of this flake's own auto-installer netboot image, see
|
||||
`docs/auto-installer.md` for what that image actually is and does once
|
||||
booted.
|
||||
|
||||
## Host Role
|
||||
|
||||
@@ -83,7 +86,7 @@ uses `archiso_http_srv` to fetch the squashfs payload over HTTP.
|
||||
Safe evaluation check:
|
||||
|
||||
```bash
|
||||
nix eval .#nixosConfigurations.pxe-boot.config.system.build.toplevel.drvPath --raw
|
||||
nix eval .#nixosConfigurations.proxmox-pxe-boot.config.system.build.toplevel.drvPath --raw
|
||||
```
|
||||
|
||||
After deployment by an operator, basic service checks are:
|
||||
|
||||
-30
@@ -1,30 +0,0 @@
|
||||
#create MBR table
|
||||
parted /dev/sda -- mklabel msdos
|
||||
#create nixos partition
|
||||
parted /dev/sda -- mkpart primary 1MB -8GB
|
||||
#set nixos partition to bootable
|
||||
parted /dev/sda -- set 1 boot on
|
||||
# create swap partition
|
||||
parted /dev/sda -- mkpart primary linux-swap -8GB 100%
|
||||
|
||||
#format OS partition
|
||||
mkfs.ext4 -L nixos /dev/sda1
|
||||
#format swap
|
||||
mkswap -L swap /dev/sda2
|
||||
|
||||
#activate swap
|
||||
swapon /dev/sda2
|
||||
|
||||
#mount nixos partition
|
||||
mount /dev/disk/by-label/nixos /mnt
|
||||
export TMPDIR=/mnt/install-tmp
|
||||
mkdir -p /mnt/install-tmp
|
||||
#Generate config
|
||||
#nixos-generate-config --root /mnt/
|
||||
|
||||
#copy customised configuration over
|
||||
#cp configuration.nix /mnt/etc/nixos/configuration.nix
|
||||
|
||||
#nixos-install --no-root-passwd
|
||||
|
||||
#reboot
|
||||
@@ -55,6 +55,33 @@ echo "Evaluating host toplevel derivations..."
|
||||
for host in $hosts; do
|
||||
echo "==> $host"
|
||||
nix eval --raw --no-use-registries --no-accept-flake-config ".#nixosConfigurations.${host}.config.system.build.toplevel.drvPath"
|
||||
|
||||
# lxc-* hosts deploy via a directly pct-restore-able tarball instead of
|
||||
# nixos-install (see docs/auto-installer.md); proxmox-* hosts (excluding
|
||||
# proxmox-lxc, the installer's own LXC target, which has no disko config)
|
||||
# can alternatively be built as a standalone disk image (see
|
||||
# docs/proxmox-images.md). Both are otherwise-unvalidated buildable
|
||||
# surface, easy to silently break without this.
|
||||
case "$host" in
|
||||
lxc-*)
|
||||
echo "==> $host (tarball)"
|
||||
nix eval --raw --no-use-registries --no-accept-flake-config ".#nixosConfigurations.${host}.config.system.build.tarball.drvPath"
|
||||
;;
|
||||
proxmox-lxc) ;;
|
||||
proxmox-*)
|
||||
echo "==> $host (diskoImagesScript)"
|
||||
nix eval --raw --no-use-registries --no-accept-flake-config ".#nixosConfigurations.${host}.config.system.build.diskoImagesScript.drvPath"
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
echo
|
||||
echo "Evaluating buildable packages..."
|
||||
packages_json="$(nix eval --json --no-use-registries --no-accept-flake-config .#packages.x86_64-linux --apply builtins.attrNames)"
|
||||
packages="$(echo "$packages_json" | jq -r '.[]')"
|
||||
for pkg in $packages; do
|
||||
echo "==> packages.x86_64-linux.${pkg}"
|
||||
nix eval --raw --no-use-registries --no-accept-flake-config ".#packages.x86_64-linux.${pkg}"
|
||||
done
|
||||
|
||||
if [[ "$MODE" == "dry-run" ]]; then
|
||||
@@ -63,6 +90,25 @@ if [[ "$MODE" == "dry-run" ]]; then
|
||||
for host in $hosts; do
|
||||
echo "==> Dry-run build: $host"
|
||||
nix build --dry-run --no-link --no-use-registries --no-accept-flake-config ".#nixosConfigurations.${host}.config.system.build.toplevel"
|
||||
|
||||
case "$host" in
|
||||
lxc-*)
|
||||
echo "==> Dry-run build: $host (tarball)"
|
||||
nix build --dry-run --no-link --no-use-registries --no-accept-flake-config ".#nixosConfigurations.${host}.config.system.build.tarball"
|
||||
;;
|
||||
proxmox-lxc) ;;
|
||||
proxmox-*)
|
||||
echo "==> Dry-run build: $host (diskoImagesScript)"
|
||||
nix build --dry-run --no-link --no-use-registries --no-accept-flake-config ".#nixosConfigurations.${host}.config.system.build.diskoImagesScript"
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
echo
|
||||
echo "Running dry-run builds for all packages."
|
||||
for pkg in $packages; do
|
||||
echo "==> Dry-run build: packages.x86_64-linux.${pkg}"
|
||||
nix build --dry-run --no-link --no-use-registries --no-accept-flake-config ".#packages.x86_64-linux.${pkg}"
|
||||
done
|
||||
fi
|
||||
|
||||
|
||||
@@ -1,9 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
#boot to rescue mode
|
||||
# set root password
|
||||
scp $RESULT_ISO root@$LINODE_IP:/tmp/nixos-auto.iso
|
||||
|
||||
#in LISH or ssh to rescue mode
|
||||
dd if=/tmp/nixos.iso of=/dev/sda bs=4M status=progress conv=fsync
|
||||
@@ -20,12 +20,11 @@ set -euo pipefail
|
||||
|
||||
repo_root="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
|
||||
hostname="${1:?usage: scripts/prepare-host-key.sh <hostname> [path-to-nixos-repo]}"
|
||||
nixos_repo="${2:-$repo_root}"
|
||||
sops_yaml="${nixos_repo}/.sops.yaml"
|
||||
hostname="${1:?usage: scripts/prepare-host-key.sh <hostname>}"
|
||||
sops_yaml="${repo_root}/.sops.yaml"
|
||||
|
||||
if [[ ! -f "$sops_yaml" ]]; then
|
||||
echo "ERROR: $sops_yaml not found. Pass the nixos repo path as the 2nd argument." >&2
|
||||
echo "ERROR: $sops_yaml not found — is this script still under nixos/scripts/?" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
@@ -55,7 +54,7 @@ Generated: ${keyfile}(.pub)
|
||||
nix-cache/server.)
|
||||
|
||||
=== 3. Re-encrypt every secrets file you just added it to ===
|
||||
nix-shell -p sops --run 'sops updatekeys ${nixos_repo}/secrets/common.yaml'
|
||||
nix-shell -p sops --run 'sops updatekeys ${repo_root}/secrets/common.yaml'
|
||||
|
||||
=== 4. Commit + push this repo so the flake build picks up the new recipient ===
|
||||
|
||||
|
||||
Reference in New Issue
Block a user