nixos/CLAUDE.md

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Repo purpose

Flake-based NixOS configuration for Wayne's LAN servers and workstation. There is
no application code here — changes are Nix module edits that affect real
machines when deployed.

## Safety rules (read before touching anything)

- **Never** run `nixos-rebuild switch|boot|test`, `nixos-install`, `parted`,
  `mkfs`, `mkswap`, `swapon`, `mount`, or any other destructive disk/deploy
  command from an agent session, even if asked indirectly. Deployment is done
  manually by the operator on the target host.
- Validation is limited to evaluation, linting, formatting checks, and
  `nix build --dry-run --no-link`.
- Do not add secrets, tokens, private keys, or new password hashes to the repo.
- This repo currently contains **committed password hashes** in
  `modules/installer/common.nix` (the auto-installer's own root/nixos login —
  a deliberate, documented choice, see `docs/auto-installer.md`, not
  accidental tech debt) and **SSH public keys** in `variables.nix`
  (`vars.adminSshKey`, `vars.remoteBuilderAuthorizedKeys`) plus a couple of
  per-host `KEY` values for beszel-agent auth (`hosts/server/host.nix`,
  `hosts/nix-cache/host.nix`). Don't use the installer's hardcoded hash as a
  template for a *real* host — every other host uses sops-nix
  (`hashedPasswordFile`, see "Security Notes" in `README.md`). Flag any *new*
  secret-like string you encounter instead of committing it.
- `host-keys/` is gitignored — locally-generated *private* SSH host keys for
  the auto-installer (see `docs/auto-installer.md`). Never commit its
  contents; if `git status` ever shows it as trackable, something is wrong.

## Commands

```bash
# One-time environment bootstrap (installs Nix if missing, prints hosts)
bash scripts/codex-setup.sh

# Full validation: secret grep, nixpkgs-fmt --check, statix lint, eval all hosts
bash scripts/codex-maintenance.sh

# Same, plus a dry-run build (no result symlink) of every host's toplevel
bash scripts/codex-maintenance.sh dry-run

# List the hosts the flake currently exposes
nix eval --json .#nixosConfigurations --apply builtins.attrNames | jq -r '.[]'

# Evaluate a single host without building (fast sanity check)
nix eval .#nixosConfigurations.<host>.config.system.build.toplevel.drvPath --raw

# Dry-run build a single host
nix build --dry-run --no-link .#nixosConfigurations.<host>.config.system.build.toplevel
```

Formatting/lint tools (`nixpkgs-fmt`, `statix`) are not installed locally; the
maintenance script pulls them via `nix run github:NixOS/nixpkgs/nixos-25.11#<tool>`.
There is no test suite — "correctness" here means the flake evaluates and
`nixpkgs-fmt`/`statix` are clean.

**In an interactive agent session**, prefer targeted checks over full-repo
sweeps: after editing one or two hosts/modules, evaluate just the
`nixosConfigurations.<host>` you touched (plus any `config.system.build.tarball`
/`diskoImagesScript`/package output affected) rather than looping over every
host — `codex-maintenance.sh` evaluates every `nixosConfigurations` host plus
every package/tarball/image variant and is slow to run after each small
change. Reserve a full
`codex-maintenance.sh` run for changes that plausibly affect every host
(`modules/common/*`, `flake.nix`, `variables.nix`) or as a final check before
committing. This is a session-workflow preference only — it does not apply to
CI, which should keep running the full script on every push/PR regardless of
diff size; that's the point of it.

## Scripts

Beyond `codex-setup.sh`/`codex-maintenance.sh` above, `scripts/` is
organized by purpose: `scripts/secrets/` (sops/age + SSH host-key
management), `scripts/proxmox/` (Proxmox deployment), `scripts/lib/`
(shared helpers, sourced by the scripts below — not run directly), and a
handful of repo-wide scripts left at the top level (`env.sh`,
`bump-nixpkgs-release.sh`, plus `codex-setup.sh`/`codex-maintenance.sh`
above). When adding a new script, put it in the matching subfolder rather
than the top level, and if it duplicates logic another script already has,
lift the shared part into `scripts/lib/` instead of copying it.

### `scripts/secrets/`

- `scripts/secrets/sync-host-keys.sh` — generates/registers SSH host keys
  and their `.sops.yaml`/`secrets/*.yaml` recipients for flake targets,
  idempotently (`--all`, `<target>`, `--remove`, `--regenerate-all-keys`,
  all with `--dry-run`). The primary tool for provisioning a new host's
  secrets access — see "Creating a new machine" in `docs/auto-installer.md`.
- `scripts/secrets/prepare-host-key.sh` — narrower predecessor: generates a
  key by an arbitrary name without touching `.sops.yaml`. Still useful to
  pre-generate a key before its flake target exists yet, since
  `sync-host-keys.sh` can only act on targets `nixosConfigurations` already
  has.
- `scripts/secrets/rotate-admin-key.sh <backup-admin-key> [--new-key-file
  <path>] [--dry-run]` — rotates `.sops.yaml`'s `&admin` age key: decrypts
  with a backed-up copy of the key currently trusted as `&admin` (verified
  by deriving its public key and comparing, not taken on faith), replaces
  the `&admin` line with a new key already present in the environment
  (defaults to wherever sops/age itself would look), and runs
  `sops updatekeys` on every `secrets/*.yaml`. One-way: the old key can no
  longer decrypt anything re-encrypted this way. This is the automation
  for the manual steps `sync-host-keys.sh`/`create-proxmox-resource.sh`
  print when they bootstrap a brand-new, not-yet-trusted key on a machine
  with no prior admin access.
- `scripts/secrets/backup-admin-key.sh <dest-path> [--key-file <path>]
  [--force] [--dry-run]` — copies the local sops age key (source
  resolution matches sops/age itself: `$SOPS_AGE_KEY` inline, then
  `--key-file`, then `$SOPS_AGE_KEY_FILE`, then the XDG default) to an
  arbitrary destination path with `0600` permissions, validating it's a
  real age identity and round-tripping the public key before and after the
  write. Refuses to overwrite an existing `<dest-path>` without `--force`.
  Purely a local filesystem copy — never touches `.sops.yaml`/
  `secrets/*.yaml` or the repo at all. The resulting file is exactly what
  `rotate-admin-key.sh` expects as its backup-key argument.

### `scripts/proxmox/`

- `scripts/proxmox/create-proxmox-resource.sh` — builds a `lxc-*`/
  `proxmox-*` target's tarball/disk image and creates it on a real Proxmox
  node (`pct create` against the tarball as a CT template / `qm create`+
  `importdisk`), or reconfigures an existing resource's cores/memory/disk
  size (`--modify`, always requires typing the VMID back to confirm).
  Checks for an already-uploaded image on the node before building
  (`--force-rebuild` to skip that and always rebuild), and probes
  nix-cache's substituter/remote-builder reachability once up front rather
  than letting every `nix build` call retry against it individually.
  Refuses to create a target whose host identity already exists live on
  the node (checked directly via `qm`/`pct`, not any file in this repo)
  unless `--allow-duplicate-host` is passed. `--dry-run` throughout both
  modes. The first time it has to bootstrap build tooling on a node (i.e.
  `nix` wasn't already on its `PATH`), it also runs
  `scripts/proxmox/configure-nix-cache-client.sh` there (non-fatally — a
  failure just falls back to building from source / `cache.nixos.org`) so
  the node substitutes from and can offload builds to nix-cache on every
  subsequent run, not just this one.
- `scripts/proxmox/configure-nix-cache-client.sh [--dry-run]
  [--no-remote-builder] [--no-restart]` — the non-NixOS equivalent of
  `modules/nix-cache/client.nix`/`remote-builder-client.nix`, for a plain
  Debian machine with the Nix package manager (not NixOS) already
  installed: run as root *on that machine* to add nix-cache as a
  substituter in `/etc/nix/nix.conf` (`https://cache.nixos.org/` kept as
  fallback) via `extra-substituters`/`extra-trusted-public-keys` so it
  layers on top of whatever's already there instead of clobbering it, and,
  if `/root/.ssh/nixremote` is already present (see docs/nix-cache.md
  "Remote builder SSH keys"), configures it as a distributed-build
  machine too and trusts nix-cache's SSH host key in
  `/etc/ssh/ssh_known_hosts`. Idempotent (re-running replaces its own
  marked block rather than duplicating it); restarts `nix-daemon` by
  default so the change takes effect immediately.

### `scripts/lib/`

Sourced by the scripts above, never run directly:

- `nix-bootstrap.sh` — `NIX_CONFIG`/`ensure_nix_profile`, shared by
  `codex-setup.sh`/`codex-maintenance.sh` and the remote build commands
  `create-proxmox-resource.sh` runs over SSH.
- `nix-eval.sh` — `NIX_EVAL_FLAGS` plus `list_flake_targets`/
  `flake_target_hostname` flake-introspection helpers.
- `ssh-host-keys.sh` — `generate_host_ed25519_key`/`ssh_pubkey_to_age`,
  shared by `sync-host-keys.sh` and `prepare-host-key.sh`.
- `sops-age.sh` — `age_pubkey_from_identity_file`/`sops_yaml_admin_pubkey`/
  `sops_updatekeys` plus the shared sops/age default key-file resolution,
  shared by `backup-admin-key.sh`, `rotate-admin-key.sh`, and
  `sync-host-keys.sh`.
- `confirm.sh` — `confirm_typed`, the "type X back to confirm" destructive-
  action prompt shared by `create-proxmox-resource.sh` and
  `sync-host-keys.sh`.
- `sync-host-keys-edit-sops.py` — the `.sops.yaml` anchor/key_groups editor
  `sync-host-keys.sh` shells out to (see that script for why: precise,
  idempotent YAML edits are impractical in bash).

### Top level

- `scripts/env.sh` — shared config (`PROXMOX_HOST`, storage pool, bridge,
  default cores/memory) sourced by `create-proxmox-resource.sh`. Add new
  cross-script config here instead of duplicating it per-script.
- `scripts/bump-nixpkgs-release.sh` — bumps `flake.nix`'s `nixpkgs.url`/
  `home-manager.url` in place. Exists because flake input URLs can't
  reference `variables.nix` (confirmed empirically — `nix flake metadata`
  errors on it), so this is the closest equivalent to a single source of
  truth for the tracked release.

`sync-host-keys.sh`, `create-proxmox-resource.sh`, and
`rotate-admin-key.sh` genuinely mutate real state when run for real (not
`--dry-run`): real `secrets/*.yaml` recipients, real Proxmox VMs/
containers, real revocation of decrypt access. They require the
operator's own SSH/sops access, which an agent session doesn't have — but
don't suggest running any of them non-dry-run without the operator's
explicit go-ahead even if it becomes technically reachable.
`backup-admin-key.sh` only writes a key copy to a path the operator gives
it — lower-stakes than the others, but it still handles a real private
key, so treat its destination path choice as the operator's call too.

## Architecture

`flake.nix` is the single entry point. It generates one
`nixosConfigurations.<platform>-<buildtype>` attribute per target via the
`mkTarget` function, composed from:

```
nixosSystem {
  modules = [
    disko.nixosModules.disko
    sops-nix.nixosModules.sops
    ./modules/common/configuration.nix
    ./modules/platforms/${platform}.nix        # what it runs on
    ./modules/build-types/${buildType}.nix     # what it's for
    hostPath                                   # hosts/<name>/host.nix — per-machine identity
    home-manager.nixosModules.home-manager { ... }
  ] ++ (client-only modules, for every buildType except "nix-cache" itself)
}
```

Platforms: `linode`, `proxmox`, `lxc`. Build types: `minimal`, `nix-cache`,
`server`, `docker`, `gui`, `pxe-boot`, `tailscale-exit-node`, `tor-relay`. Not
every combination is built — e.g. `pxe-boot` has no `linode` variant
(PXE/DHCP/TFTP need LAN L2 adjacency a Linode VPS doesn't have), and
`tor-relay` currently only exists as `lxc-tor-relay`. Treat `flake.nix`'s
`generatedTargets` as the source
of truth for which hosts exist — `README.md`, `AGENTS.md`,
`docs/flake-lock-automation.md`, and the CI eval workflows
(`.github/workflows/check-nixos.yml`, `.gitea/workflows/check-nixos.yml`) list
hosts by hand (or, for the CI workflows, evaluate the flake dynamically) and
can drift from it, so re-check them against `flake.nix` when adding or
removing a host.

### Composition pattern

- `hosts/<name>/host.nix` — per-machine identity **only**: hostname, hostId,
  per-machine secrets, `system.stateVersion`. These files carry no `imports`
  of their own beyond narrow parameterized helpers (see
  `modules/beszel/host-token.nix` below) — all shared behavior comes from the
  platform/build-type modules composed in `flake.nix`, not from the host file.
- `modules/platforms/{linode,proxmox,lxc}.nix` — platform-specific config:
  boot method, guest tooling, and (for linode/proxmox) the hypervisor-specific
  hardware config, imported directly by the platform module itself
  (`../hardware-configuration/vm/{proxmox,linode}.nix`) — **not** wired in
  from `flake.nix`. `lxc.nix` has no hardware-configuration counterpart since
  containers share the host kernel; instead it imports nixpkgs' own
  `virtualisation/proxmox-lxc.nix`, which gives every `lxc-*` host a
  `config.system.build.tarball` output — a plain rootfs tarball, used as a
  `pct create ... vztmpl` CT template (**not** `pct restore`, which expects
  `vzdump` backup-archive metadata this doesn't have), no install step —
  see `docs/auto-installer.md`.
- `modules/build-types/*.nix` — what a system is for:
  minimal/server/docker/gui/pxe-boot/nix-cache/tailscale-exit-node/tor-relay.
- `modules/common/configuration.nix` — base NixOS config imported by every
  host: locale, users, nix settings, git.
- `modules/common/home.nix` / `hosts/nixos/home.nix` — Home Manager config for
  the `nixos` user; the `nixos` workstation (`gui` build type) has its own,
  other hosts share `modules/common/home.nix`.
- `modules/disko/proxmox.nix` — declarative disk layout (GPT: ESP + swap +
  ext4 root) via disko, used by all Proxmox-VM hosts (`proxmox-*`, not
  `lxc-*`). Also carries `imageSize`/`imageName`, letting every `proxmox-*`
  host be built as a standalone, `qm importdisk`-ready `.raw` image with no
  install step — see `docs/proxmox-images.md`.
- `modules/disko/linode.nix` — `linode-*`'s disko config, deliberately
  different in kind from the Proxmox one: Linode provisions and sizes
  `/dev/sda`/`/dev/sdb` itself as whole, unpartitioned devices before the OS
  boots, so this declares them with `destroy = false` (disko never wipes
  them) and a bare `filesystem`/`swap` content type instead of a partition
  table — idempotent against an already-provisioned disk, never destructive.
- `modules/boot/efi.nix` — systemd-boot + EFI vars, paired with the disko module.
- `modules/installer/` — the auto-installer environment (ISO, also served as
  PXE netboot): `common.nix` (shared config + the generated
  `auto-install.sh`), `iso.nix`, `host-keys.nix` (optionally bakes
  `host-keys/` into the image under `--impure`). See
  `docs/auto-installer.md`.
- `modules/pxe-boot/stage-installer-artifacts.nix` — builds the installer's
  netboot image and stages it on the `pxe-boot` host so its iPXE menu can
  chain straight to it. See `docs/pxe-boot.md`.
- `modules/nix-cache/{client,server,remote-builder-client}.nix` — binary cache
  substituter + SSH remote-builder wiring; see `docs/nix-cache.md` for the
  full design (per-host local stores, no shared `/nix/store`, and how the
  `nixremote` signing/SSH keys fit together).
- `modules/beszel/host-token.nix` — parameterized helper module
  (`{ name, sopsFile }`) that wires a host's beszel-agent sops secret/template
  and `environmentFile`; used by `hosts/server/host.nix` and
  `hosts/nix-cache/host.nix` to avoid duplicating that boilerplate.
- `modules/tailscale/`, `modules/docker/`, `modules/networking/`,
  `modules/traefik/`, `modules/tor/`, `modules/services/*` — single-purpose,
  single-host
  feature modules (e.g. `docker/enable-service.nix`,
  `services/zfs/enable-service.nix`). Grep `modules/build-types/*.nix` for
  each build type's `imports` list to see which modules apply where.

New host = new `hosts/<name>/host.nix` + a matching
`mkTarget { platform; buildType; hostPath; }` entry added to `flake.nix`'s
`generatedTargets`, composed from existing `modules/*` pieces rather than
duplicating config.

### Other docs worth reading before touching these areas

- `docs/nix-cache.md` — nix-cache binary cache/remote-builder design and key
  handling.
- `docs/pxe-boot.md` — the `pxe-boot` host's iPXE/TFTP/HTTP boot chain and
  directory layout under `/srv/pxe`.
- `docs/auto-installer.md` — the installer environment (ISO/netboot/Proxmox
  LXC), `host-keys/` and the sops-nix pre-seeding problem it solves, and why
  `lxc-*` hosts are deliberately excluded from its menu.
- `docs/proxmox-images.md` — building `proxmox-*` hosts as standalone `.raw`
  disk images (disko's image builder) instead of installing, and deploying
  the result to Proxmox.
- `docs/flake-lock-automation.md` — how `flake.lock` updates flow through CI
  (scheduled `nix flake update` PR + host-eval-on-PR workflow) and why hosts
  should track the committed lock file rather than `nixos-rebuild --upgrade-all`.