# CLAUDE.md This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. ## Repo purpose Flake-based NixOS configuration for Wayne's LAN servers and workstation. There is no application code here — changes are Nix module edits that affect real machines when deployed. ## Safety rules (read before touching anything) - **Never** run `nixos-rebuild switch|boot|test`, `nixos-install`, `parted`, `mkfs`, `mkswap`, `swapon`, `mount`, or any other destructive disk/deploy command from an agent session, even if asked indirectly. Deployment is done manually by the operator on the target host. - Validation is limited to evaluation, linting, formatting checks, and `nix build --dry-run --no-link`. - Do not add secrets, tokens, private keys, or new password hashes to the repo. - This repo currently contains **committed password hashes** in `modules/installer/common.nix` (the auto-installer's own root/nixos login — a deliberate, documented choice, see `docs/auto-installer.md`, not accidental tech debt) and **SSH public keys** in `variables.nix` (`vars.adminSshKey`, `vars.remoteBuilderAuthorizedKeys`) plus a couple of per-host `KEY` values for beszel-agent auth (`hosts/server/host.nix`, `hosts/nix-cache/host.nix`). Don't use the installer's hardcoded hash as a template for a *real* host — every other host uses sops-nix (`hashedPasswordFile`, see "Security Notes" in `README.md`). Flag any *new* secret-like string you encounter instead of committing it. - `host-keys/` is gitignored — used only by the auto-installer's own environment for pre-seeding non-LXC host keys before first boot (see `docs/auto-installer.md`). Never commit its contents; if `git status` ever shows it as trackable, something is wrong. All deployed hosts use clan vars (`vars/per-machine//openssh/`, committed and sops-encrypted) for their SSH host keys — those ARE tracked by git and belong in the repo. ### Two Proxmox nodes: `pve1.sweet.home` (production) and `pve-test.sweet.home` (sandbox) There are two SSH-reachable Proxmox nodes on the LAN, both defined in `scripts/env.sh` (`PVE1_HOST` / `PVE_TEST_HOST`), individually targetable via `scripts/proxmox/create-proxmox-resource.sh --node ` or by overriding `PROXMOX_HOST`. `PROXMOX_HOST` itself still defaults to `PVE1_HOST` (production) — that default, and every other script behavior, is unchanged from before `pve-test` existed; the only thing new is that `pve-test` can now be reached at all. They are **not interchangeable** — one is real production infrastructure, the other exists specifically so there's somewhere safe to test. The restriction below is a policy for Claude specifically, not a change to the tooling's own default or anything the operator needs to opt into. #### `pve1.sweet.home` (production — off-limits to Claude) A real, live Proxmox node hosting production VMs/containers — not a sandbox, and not Claude's to touch by default. - **Off-limits at all times unless the operator has given explicit, same-session instructions to act on this specific host.** That authorization is scoped to the task it was given for — don't carry it forward to unrelated later work in the same conversation, and never assume it from a previous session. - **Read-only for existing state is always fine, authorization or not.** You may SSH in (or use `pvesm`, `qm list`, `pct list`, `qm config`, `pct config`, the Proxmox API, etc.) to inspect the node's config, storage, and any existing VM/container — including ones this repo didn't create. - **Never** modify, stop, restart, delete, reconfigure, or create anything on this node (`qm set`, `pct set`, `qm destroy`, `pct destroy`, `qm stop`, `pct stop`, `qm create`, `pct create`, snapshot operations, storage changes, etc.) — including scratch/test resources — without that explicit go-ahead. Use `pve-test.sweet.home` for anything exploratory instead; it exists precisely so `pve1` never has to be the answer to "where do I test this." - **This is a Claude-specific policy, not something the scripts enforce.** `scripts/env.sh`/`create-proxmox-resource.sh` default to `pve1` exactly as they did before `pve-test` existed, with no extra flag or prompt required — that's deliberate, so the operator's own existing workflows don't change. Claude, however, must never rely on that default: every Proxmox action Claude takes on its own initiative — not explicitly pointed at `pve1` by the operator this session — targets `pve-test` instead (e.g. `--node "$PVE_TEST_HOST"`, or `PROXMOX_HOST=$PVE_TEST_HOST`). Claude's own default is `pve-test`, full stop, regardless of what the tooling's own unqualified default happens to be. #### `pve-test.sweet.home` (sandbox — Claude's default target) A separate Proxmox node set aside for testing. The *tooling's* default is still production (`PROXMOX_HOST` → `PVE1_HOST`, see above) — but **Claude's own default is this node**: absent an explicit, same-session instruction to use `pve1`, every Proxmox action Claude initiates targets `pve-test`. Once targeted, it's safe to create, interrogate, and destroy resources on without asking first. - **Test VMs/containers are allowed, but must be torn down.** Create a scratch VM or container here (e.g. via `scripts/proxmox/create-proxmox-resource.sh` or raw `qm`/`pct create`) to validate something. Anything created this way must be destroyed again in the same session, before ending the task — never leave a test resource running. Use a VMID/name that's obviously scratch (and doesn't collide with a real flake target) so it's unambiguous what's safe to remove. - **Node-level config is still not yours to change.** Creating/destroying your own scratch guests is fine; Proxmox host config, storage pools, and networking on `pve-test` itself are still the operator's call to make manually, same as on `pve1`. ## Commands ```bash # One-time environment bootstrap (installs Nix if missing, prints hosts) bash scripts/codex-setup.sh # Changed-files-only validation: secret grep (whole repo), nixpkgs-fmt --check # and statix on changed *.nix files, eval of the hosts/packages those changes # can affect. This is what CI runs on every push/PR. bash scripts/codex-maintenance.sh # Full sweep: nixpkgs-fmt --check/statix over the whole tree, eval every host # and package. Slow (minutes) -- CI never runs this; use it locally before a # release or after touching modules/common/*, flake.nix, or variables.nix for # extra confidence beyond the automatic full-fallback those paths already # trigger in the default mode (see below). bash scripts/codex-maintenance.sh --full-check # Either mode, plus a dry-run build (no result symlink) of every host/package # in whichever scope is active bash scripts/codex-maintenance.sh --dry-run bash scripts/codex-maintenance.sh --full-check --dry-run # List the hosts the flake currently exposes nix eval --json .#nixosConfigurations --apply builtins.attrNames | jq -r '.[]' # Evaluate a single host without building (fast sanity check) nix eval .#nixosConfigurations..config.system.build.toplevel.drvPath --raw # Dry-run build a single host nix build --dry-run --no-link .#nixosConfigurations..config.system.build.toplevel ``` Formatting/lint tools (`nixpkgs-fmt`, `statix`) are not installed locally; the maintenance script pulls them via `nix run github:NixOS/nixpkgs/nixos-25.11#`. There is no test suite — "correctness" here means the flake evaluates and `nixpkgs-fmt`/`statix` are clean. With no flags, `codex-maintenance.sh` diffs against a base ref (env `MAINT_BASE_SHA`, else the PR base SHA in CI, else `HEAD^` locally) and scopes fmt-check/statix to the changed `*.nix` files and eval to the hosts/packages those changes can affect — a `hosts//host.nix` edit only evals that host's targets, a `modules/platforms/.nix` edit only evals that platform's hosts, and so on. A change to `flake.nix`, `flake.lock`, `variables.nix`, `modules/common/*`, or any other `modules/*.nix` file outside `platforms/`/`build-types/` (whose blast radius isn't safely inferable from the path alone) falls back to evaluating every host and package, same as `--full-check` would, just without the whole-tree fmt/statix sweep. This exists because the whole-tree sweep is what was timing out CI; **CI always runs the plain, no-flag form and never passes `--full-check`.** The default mode's diff is against the working tree (uncommitted and staged edits included, not just committed ones), so it's already the right tool for an interactive session too: after editing one or two hosts/modules, plain `bash scripts/codex-maintenance.sh` naturally scopes to just what you touched. Reserve `--full-check` for changes that plausibly affect every host (`modules/common/*`, `flake.nix`, `variables.nix` — though the default mode already falls back to evaluating everything for those paths, `--full-check` additionally re-checks fmt/statix over the whole tree) or as a final check before committing. ## Scripts Beyond `codex-setup.sh`/`codex-maintenance.sh` above, `scripts/` is organized by purpose: `scripts/secrets/` (sops/age + SSH host-key management), `scripts/proxmox/` (Proxmox deployment), `scripts/installer/` (the auto-installer's own shell script, templated into the image — see below), `scripts/lib/` (shared helpers, sourced by the scripts below — not run directly), and a handful of repo-wide scripts left at the top level (`env.sh`, `bump-nixpkgs-release.sh`, plus `codex-setup.sh`/ `codex-maintenance.sh` above). When adding a new script, put it in the matching subfolder rather than the top level, and if it duplicates logic another script already has, lift the shared part into `scripts/lib/` instead of copying it. ### `scripts/installer/` - `scripts/installer/auto-install.sh` — the interactive install script baked into the auto-installer image (see `docs/auto-installer.md`), kept as a real, version-controlled shell file rather than inline in `modules/installer/common.nix`'s Nix. It sources `scripts/env.sh` itself for `LAN_DOMAIN` (`export LAN_DOMAIN`/`: "${LAN_DOMAIN:=...}"`, matching `variables.nix`'s `lanDomain` — manually kept in sync, same pattern as `NIX_CACHE_HOST` mirroring `nixCacheHost`), rather than Nix-level string substitution — that's what makes it work identically whether run straight from a git checkout or from inside the built installer image. `common.nix` bakes `scripts/env.sh` in alongside it at a matching relative path (`/etc/nixos-installer/env.sh` next to `/etc/nixos-installer/installer/auto-install.sh`) so the script's own `source "$(dirname ...)/../env.sh"` line resolves the same way in both contexts — this is also why it's invoked from `/etc/nixos-installer/installer/auto-install.sh` rather than a flat `/etc/auto-install.sh`. `#!/usr/bin/env bash`, not `#!/run/current-system/sw/bin/bash`: the latter only resolves on an already-activated NixOS system, breaking the checked-out-file case entirely (confirmed live: "cannot execute: required file not found" on a non-NixOS box); `/usr/bin/env` is reliably present on both NixOS (`environment.usrbinenv`'s own default) and any normal Linux distro. ### `scripts/secrets/` - `scripts/secrets/sync-host-keys.sh` — generates/registers SSH host keys and their `.sops.yaml`/`secrets/*.yaml` recipients for flake targets, idempotently (`--all`, ``, `--remove`, `--regenerate-all-keys`, all with `--dry-run`). Stores keys as clan vars (`vars/per-machine//openssh/`, committed and sops-encrypted) for all flake targets. The primary tool for provisioning a new host's secrets access — see "Creating a new machine" in `docs/auto-installer.md`. - `scripts/secrets/prepare-host-key.sh` — narrower predecessor: generates a key by an arbitrary name without touching `.sops.yaml`. Still useful to pre-generate a key before its flake target exists yet, since `sync-host-keys.sh` can only act on targets `nixosConfigurations` already has. - `scripts/secrets/rotate-admin-key.sh [--new-key-file ] [--dry-run]` — rotates `.sops.yaml`'s `&admin` age key: decrypts with a backed-up copy of the key currently trusted as `&admin` (verified by deriving its public key and comparing, not taken on faith), replaces the `&admin` line with a new key already present in the environment (defaults to wherever sops/age itself would look), and runs `sops updatekeys` on every `secrets/*.yaml`. One-way: the old key can no longer decrypt anything re-encrypted this way. This is the automation for the manual steps `sync-host-keys.sh`/`create-proxmox-resource.sh` print when they bootstrap a brand-new, not-yet-trusted key on a machine with no prior admin access. - `scripts/secrets/backup-admin-key.sh [--key-file ] [--force] [--dry-run]` — copies the local sops age key (source resolution matches sops/age itself: `$SOPS_AGE_KEY` inline, then `--key-file`, then `$SOPS_AGE_KEY_FILE`, then the XDG default) to an arbitrary destination path with `0600` permissions, validating it's a real age identity and round-tripping the public key before and after the write. Refuses to overwrite an existing `` without `--force`. Purely a local filesystem copy — never touches `.sops.yaml`/ `secrets/*.yaml` or the repo at all. The resulting file is exactly what `rotate-admin-key.sh` expects as its backup-key argument. - `scripts/secrets/sync-nix-cache-host-key.sh [--check] [--dry-run] [--host ]` — detects drift between the ed25519 SSH host key nix-cache is actually serving right now (via `ssh-keyscan`) and `vars.nixCacheHostKey` (`variables.nix`), the value `modules/nix-cache/remote-builder-client.nix` bakes into every real client's declarative `programs.ssh.knownHosts` and `configure-nix-cache-client.sh` hardcodes as its own default for non-NixOS clients. That value has no automatic source of truth — it's set once from whatever nix-cache's host key happened to be at the time, and silently goes stale if the host is ever rebuilt/recreated with a new key, breaking every client's distributed-build SSH trust with no error that points back here. `--check` (used by `codex-maintenance.sh`, which treats an unreachable nix-cache — e.g. from a non-LAN CI runner — as a silent skip rather than a failure) only reports drift; the no-flags form updates both files in place. Declarative clients still need a rebuild to pick up the fix. ### `scripts/proxmox/` - `scripts/proxmox/create-proxmox-resource.sh` — builds a `lxc-*`/ `proxmox-*` target's tarball/disk image and creates it on a real Proxmox node (`pct create` against the tarball as a CT template / `qm create`+ `importdisk`), or reconfigures an existing resource's cores/memory/disk size (`--modify`, always requires typing the VMID back to confirm). Checks for an already-uploaded image on the node before building (`--force-rebuild` to skip that and always rebuild), and probes nix-cache's substituter/remote-builder reachability once up front rather than letting every `nix build` call retry against it individually. Refuses to create a target whose host identity already exists live on the node (checked directly via `qm`/`pct`, not any file in this repo) unless `--allow-duplicate-host` is passed. `--dry-run` throughout both modes. The first time it has to bootstrap build tooling on a node (i.e. `nix` wasn't already on its `PATH`), it also runs `scripts/proxmox/configure-nix-cache-client.sh` there (non-fatally — a failure just falls back to building from source / `cache.nixos.org`) so the node substitutes from and can offload builds to nix-cache on every subsequent run, not just this one. - `scripts/proxmox/configure-nix-cache-client.sh [--dry-run] [--no-remote-builder] [--no-restart]` — the non-NixOS equivalent of `modules/nix-cache/client.nix`/`remote-builder-client.nix`, for a plain Debian machine with the Nix package manager (not NixOS) already installed: run as root *on that machine* to add nix-cache as a substituter in `/etc/nix/nix.conf` (`https://cache.nixos.org/` kept as fallback) via `extra-substituters`/`extra-trusted-public-keys` so it layers on top of whatever's already there instead of clobbering it, and, if `/root/.ssh/nixremote` is already present (see docs/nix-cache.md "Remote builder SSH keys"), configures it as a distributed-build machine too and trusts nix-cache's SSH host key in `/etc/ssh/ssh_known_hosts`. Idempotent (re-running replaces its own marked block rather than duplicating it); restarts `nix-daemon` by default so the change takes effect immediately. ### `scripts/lib/` Sourced by the scripts above, never run directly: - `nix-bootstrap.sh` — `NIX_CONFIG`/`ensure_nix_profile`, shared by `codex-setup.sh`/`codex-maintenance.sh` and the remote build commands `create-proxmox-resource.sh` runs over SSH. - `nix-eval.sh` — `NIX_EVAL_FLAGS` plus `list_flake_targets`/ `flake_target_hostname` flake-introspection helpers. - `ssh-host-keys.sh` — `generate_host_ed25519_key`/`ssh_pubkey_to_age`, shared by `sync-host-keys.sh` and `prepare-host-key.sh`. - `sops-age.sh` — `age_pubkey_from_identity_file`/`sops_yaml_admin_pubkey`/ `sops_updatekeys` plus the shared sops/age default key-file resolution, shared by `backup-admin-key.sh`, `rotate-admin-key.sh`, and `sync-host-keys.sh`. - `confirm.sh` — `confirm_typed`, the "type X back to confirm" destructive- action prompt shared by `create-proxmox-resource.sh` and `sync-host-keys.sh`. - `sync-host-keys-edit-sops.py` — the `.sops.yaml` anchor/key_groups editor `sync-host-keys.sh` shells out to (see that script for why: precise, idempotent YAML edits are impractical in bash). ### Top level - `scripts/env.sh` — shared config (`PROXMOX_HOST`, storage pool, bridge, default cores/memory, `NIX_CACHE_HOST`, `LAN_DOMAIN`) sourced by `create-proxmox-resource.sh` and `scripts/installer/auto-install.sh`. Add new cross-script config here instead of duplicating it per-script. - `scripts/bump-nixpkgs-release.sh` — bumps `flake.nix`'s `nixpkgs.url`/ `home-manager.url` in place. Exists because flake input URLs can't reference `variables.nix` (confirmed empirically — `nix flake metadata` errors on it), so this is the closest equivalent to a single source of truth for the tracked release. `sync-host-keys.sh`, `create-proxmox-resource.sh`, and `rotate-admin-key.sh` genuinely mutate real state when run for real (not `--dry-run`): real `secrets/*.yaml` recipients, real Proxmox VMs/ containers, real revocation of decrypt access. They require the operator's own SSH/sops access, which an agent session doesn't have — but don't suggest running any of them non-dry-run without the operator's explicit go-ahead even if it becomes technically reachable. `backup-admin-key.sh` only writes a key copy to a path the operator gives it — lower-stakes than the others, but it still handles a real private key, so treat its destination path choice as the operator's call too. ## Architecture `flake.nix` is the single entry point. It generates one `nixosConfigurations.-` attribute per target via the `mkTarget` function, composed from: ``` nixosSystem { modules = [ disko.nixosModules.disko sops-nix.nixosModules.sops ./modules/common/configuration.nix ./modules/platforms/${platform}.nix # what it runs on ./modules/build-types/${buildType}.nix # what it's for hostPath # hosts//host.nix — per-machine identity home-manager.nixosModules.home-manager { ... } ] ++ (client-only modules, for every buildType except "nix-cache" itself) } ``` Platforms: `linode`, `proxmox`, `lxc`, `baremetal`. Build types: `minimal`, `nix-cache`, `server`, `docker`, `gui`, `pxe-boot`, `tailscale-exit-node`, `tor-relay`. Not every combination is built — e.g. `pxe-boot` has no `linode` variant (PXE/DHCP/TFTP need LAN L2 adjacency a Linode VPS doesn't have), `tor-relay` currently only exists as `lxc-tor-relay`, and `baremetal` currently only exists as `baremetal-gui` (the real gui-host hardware — see `hosts/nixos/host.nix` and `modules/platforms/baremetal.nix`). Treat `flake.nix`'s `generatedTargets` as the source of truth for which hosts exist — `README.md`, `AGENTS.md`, `docs/flake-lock-automation.md`, and the CI eval workflows (`.github/workflows/check-nixos.yml`, `.gitea/workflows/check-nixos.yml`) list hosts by hand (or, for the CI workflows, evaluate the flake dynamically) and can drift from it, so re-check them against `flake.nix` when adding or removing a host. ### Composition pattern - `hosts//host.nix` — per-machine identity **only**: hostname, hostId, per-machine secrets, `system.stateVersion`. These files carry no `imports` of their own beyond narrow parameterized helpers (see `modules/beszel/host-token.nix` below) — all shared behavior comes from the platform/build-type modules composed in `flake.nix`, not from the host file. - `modules/platforms/{linode,proxmox,lxc,baremetal}.nix` — platform-specific config: boot method, guest tooling, and the hardware config, imported directly by the platform module itself — **not** wired in from `flake.nix`. VM platforms use `../hardware-configuration/vm/{proxmox,linode}.nix`; `baremetal.nix` uses `../hardware-configuration/baremetal.nix` (adapted from a real `nixos-generate-config` run on the actual hardware, not a vm/ file, since it isn't a VM) plus `hardware.enableRedistributableFirmware = true` for real wifi/GPU/microcode firmware that VMs never needed. `lxc.nix` has no hardware-configuration counterpart since containers share the host kernel; instead it imports nixpkgs' own `virtualisation/proxmox-lxc.nix`, which gives every `lxc-*` host a `config.system.build.tarball` output — a plain rootfs tarball, used as a `pct create ... vztmpl` CT template (**not** `pct restore`, which expects `vzdump` backup-archive metadata this doesn't have), no install step — see `docs/auto-installer.md`. - `modules/build-types/*.nix` — what a system is for: minimal/server/docker/gui/pxe-boot/nix-cache/tailscale-exit-node/tor-relay. - `modules/common/configuration.nix` — base NixOS config imported by every host: locale, users, nix settings, git. - `modules/common/home.nix` / `hosts/nixos/home.nix` — Home Manager config for the `nixos` user; the `nixos` workstation (`gui` build type) has its own, other hosts share `modules/common/home.nix`. - `modules/disko/proxmox.nix` — declarative disk layout (GPT: ESP + swap + ext4 root) via disko, used by all Proxmox-VM hosts (`proxmox-*`, not `lxc-*`). Also carries `imageSize`/`imageName`, letting every `proxmox-*` host be built as a standalone, `qm importdisk`-ready `.raw` image with no install step — see `docs/proxmox-images.md`. - `modules/disko/linode.nix` — `linode-*`'s disko config, deliberately different in kind from the Proxmox one: Linode provisions and sizes `/dev/sda`/`/dev/sdb` itself as whole, unpartitioned devices before the OS boots, so this declares them with `destroy = false` (disko never wipes them) and a bare `filesystem`/`swap` content type instead of a partition table — idempotent against an already-provisioned disk, never destructive. - `modules/disko/baremetal.nix` — `baremetal-gui`'s disko config: a ZFS RAID0 (striped, no redundancy — disko's zpool `mode` defaults to `""`, which is a plain stripe rather than `"mirror"`/`"raidz"`) root pool across two disks, ESP + systemd-boot on the first. Device paths (`vars.guiRootDisk1`/`guiRootDisk2`) are placeholders — fill in stable `/dev/disk/by-id/...` paths before running disko for real. `modules/platforms/baremetal.nix` also imports `modules/services/zfs/enable-service.nix` for this (the `zfs_unstable` package, autoScrub/autoSnapshot/trim) — the only other importer today is `server`'s NFS data pool, an unrelated non-root ZFS use. - `modules/boot/efi.nix` — systemd-boot + EFI vars, paired with the disko module. - `modules/installer/` — the auto-installer environment (ISO, also served as PXE netboot): `common.nix` (shared config + the generated `auto-install.sh`), `iso.nix`, `host-keys.nix` (optionally bakes `host-keys/` into the image under `--impure`). See `docs/auto-installer.md`. - `modules/pxe-boot/stage-installer-artifacts.nix` — builds the installer's netboot image and stages it on the `pxe-boot` host so its iPXE menu can chain straight to it. See `docs/pxe-boot.md`. - `modules/nix-cache/{client,server,remote-builder-client}.nix` — binary cache substituter + SSH remote-builder wiring; see `docs/nix-cache.md` for the full design (per-host local stores, no shared `/nix/store`, and how the `nixremote` signing/SSH keys fit together). - `modules/beszel/host-token.nix` — parameterized helper module (`{ name, sopsFile }`) that wires a host's beszel-agent sops secret/template and `environmentFile`; used by `hosts/server/host.nix` and `hosts/nix-cache/host.nix` to avoid duplicating that boilerplate. - `modules/tailscale/`, `modules/docker/`, `modules/networking/`, `modules/traefik/`, `modules/tor/`, `modules/services/*` — single-purpose, single-host feature modules (e.g. `docker/enable-service.nix`, `services/zfs/enable-service.nix`). Grep `modules/build-types/*.nix` for each build type's `imports` list to see which modules apply where. New host = new `hosts//host.nix` + a matching `mkTarget { platform; buildType; hostPath; }` entry added to `flake.nix`'s `generatedTargets`, composed from existing `modules/*` pieces rather than duplicating config. ### Other docs worth reading before touching these areas - `docs/nix-cache.md` — nix-cache binary cache/remote-builder design and key handling. - `docs/pxe-boot.md` — the `pxe-boot` host's iPXE/TFTP/HTTP boot chain and directory layout under `/srv/pxe`. - `docs/auto-installer.md` — the installer environment (ISO/netboot/Proxmox LXC), `host-keys/` and the sops-nix pre-seeding problem it solves, and why `lxc-*` hosts are deliberately excluded from its menu. - `docs/proxmox-images.md` — building `proxmox-*` hosts as standalone `.raw` disk images (disko's image builder) instead of installing, and deploying the result to Proxmox. - `docs/flake-lock-automation.md` — how `flake.lock` updates flow through CI (scheduled `nix flake update` PR + host-eval-on-PR workflow) and why hosts should track the committed lock file rather than `nixos-rebuild --upgrade-all`.