Archived
rootandClaude Sonnet 5
86a1660adc
Check NixOS configurations / eval-hosts (pull_request) Failing after 11m19s
variables.nix's deployedTargets was a manually-maintained list with no enforcement keeping it in sync with reality -- it caused two separate false refusals in a row (naming a VM as deployed well after it had been destroyed, then matching a target against itself once the list was "corrected"). Static files can't track whether a resource still actually exists. create-proxmox-resource.sh's duplicate-host guard now queries the Proxmox node directly (qm/pct's own name/hostname config, matched against --host) instead. Also fixes a gap in that live check: it originally swallowed ssh failures and would have silently treated "can't reach the node" the same as "checked, nothing there" -- it now refuses instead of guessing when the node can't be reached. deployedTargets is removed entirely from variables.nix since nothing else in the repo consumed it once this script no longer does; README.md's Hosts table remains the sole source of truth for "(real, deployed)" status. CLAUDE.md and the script's own --help/comments updated to match. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
151 lines
8.0 KiB
Markdown
151 lines
8.0 KiB
Markdown
# NixOS LAN Configurations
|
||
|
||
Flake-based NixOS configuration repository for Wayne's LAN servers and
|
||
workstation.
|
||
|
||
## Hosts
|
||
|
||
Targets are named `<platform>-<buildtype>`, generated from two orthogonal
|
||
pieces composed in `flake.nix`:
|
||
|
||
- **Platforms** (what it runs on): `linode`, `proxmox`, `lxc`
|
||
- **Build types** (what it's for): `minimal`, `nix-cache`, `server`, `docker`,
|
||
`gui`, `pxe-boot`, `tailscale-exit-node`
|
||
|
||
Not every combination exists — `pxe-boot` has no `linode` variant, since
|
||
PXE/DHCP/TFTP need LAN L2 adjacency that a Linode VPS doesn't have. The full
|
||
list:
|
||
|
||
| Target | Purpose |
|
||
| --- | --- |
|
||
| `linode-minimal` | Minimal NixOS host profile on a Linode VPS (real, deployed) |
|
||
| `proxmox-minimal` | Minimal NixOS host profile on Proxmox (real, deployed — previously the flat `nix-minimal` target) |
|
||
| `lxc-minimal` | Minimal NixOS host profile in a Proxmox LXC container |
|
||
| `linode-nix-cache` / `proxmox-nix-cache` / `lxc-nix-cache` | Local Nix binary cache and remote builder (`lxc-nix-cache` is the real, deployed one — previously `proxmox-nix-cache`, itself previously the flat `nix-cache` target) |
|
||
| `linode-server` / `proxmox-server` / `lxc-server` | Storage, NFS, backup, and monitoring exporter host (`proxmox-server` is the real, deployed one — previously the flat `server` target) |
|
||
| `linode-docker` / `proxmox-docker` / `lxc-docker` | Docker host for the main container stack (`proxmox-docker` is the real, deployed one — previously the flat `docker` target) |
|
||
| `linode-gui` / `proxmox-gui` / `lxc-gui` | Cinnamon desktop workstation (`proxmox-gui` is the real, deployed one — previously the flat `nixos` target) |
|
||
| `proxmox-pxe-boot` / `lxc-pxe-boot` | HTTP/iPXE boot asset host (`proxmox-pxe-boot` is the real, deployed one — previously the flat `pxe-boot` target) |
|
||
| `linode-tailscale-exit-node` / `proxmox-tailscale-exit-node` / `lxc-tailscale-exit-node` | Tailscale exit node (no deployed target yet; `lxc-tailscale-exit-node` is the one planned for actual use) |
|
||
|
||
This table is the only place "(real, deployed)" status is tracked — there's
|
||
no separate machine-readable copy to keep in sync. `scripts/create-proxmox-resource.sh`
|
||
guards against creating a same-identity duplicate of an already-deployed host
|
||
by checking the Proxmox node itself (live `qm`/`pct` state) rather than any
|
||
file in this repo, since a static list can't track whether a resource still
|
||
actually exists.
|
||
|
||
Each buildtype's `hosts/<name>/host.nix` carries the per-machine identity
|
||
(hostname, hostId, per-machine secrets, `system.stateVersion`) that must stay
|
||
fixed regardless of which platform it's built for — see
|
||
`flake-target-refactor-spec.md` for the full rationale. Every deployed host
|
||
stamps its own active target name into `/etc/flake-target` at build time, so
|
||
`nixos-rebuild switch --flake .#$(cat /etc/flake-target)` always picks up the
|
||
right one even after a platform migration changes the flake attribute name.
|
||
|
||
List hosts with:
|
||
|
||
```bash
|
||
nix eval --json .#nixosConfigurations --apply builtins.attrNames | jq -r '.[]'
|
||
```
|
||
|
||
## Layout
|
||
|
||
| Path | Purpose |
|
||
| --- | --- |
|
||
| `flake.nix` | Flake inputs, the `mkTarget` platform × build-type generator, and `nixosConfigurations` outputs |
|
||
| `variables.nix` | Single source of truth for shared values (LAN domain/CIDR, hostnames, timezone, primary username, storage root, NFS share subpaths/mountpoints, service ports, ...) — passed to every module and Home Manager config as the `vars` argument via `specialArgs`/`extraSpecialArgs` |
|
||
| `hosts/<name>/host.nix` | Per-machine identity: hostname, hostId, per-machine secrets, `system.stateVersion` |
|
||
| `hosts/nixos/home.nix` | Workstation-specific Home Manager config (used by the `gui` build type) |
|
||
| `modules/platforms/` | Platform-specific config: virtualisation guest tools, boot method, hardware config (`linode.nix`, `proxmox.nix`, `lxc.nix`) |
|
||
| `modules/build-types/` | Build-type-specific config: what makes a system minimal/server/docker/gui/pxe-boot/nix-cache |
|
||
| `modules/common/` | Shared NixOS config, Home Manager, aliases imported by every host |
|
||
| `modules/nix-cache/` | Binary cache and remote builder client/server modules |
|
||
| `modules/installer/` | Auto-installer environment (ISO, also served as PXE netboot) — see `docs/auto-installer.md` |
|
||
| `host-keys/` | Gitignored, locally-generated SSH host keys for the auto-installer — see `docs/auto-installer.md` |
|
||
| `docs/` | Operational notes for cache, builders, lock updates, boot services, the auto-installer, and Proxmox image builds |
|
||
| `scripts/` | Codex setup, validation, host-key, release-bump, and Proxmox resource helpers |
|
||
|
||
## Validation
|
||
|
||
Safe validation commands for Codex and local review:
|
||
|
||
```bash
|
||
bash scripts/codex-setup.sh
|
||
bash scripts/codex-maintenance.sh dry-run
|
||
bash scripts/codex-maintenance.sh
|
||
```
|
||
|
||
For individual host evaluation:
|
||
|
||
```bash
|
||
nix eval .#nixosConfigurations.<host>.config.system.build.toplevel.drvPath --raw
|
||
```
|
||
|
||
Use `nix build --dry-run --no-link` when build planning is needed. Do not run
|
||
deployment, install, disk formatting, mount, or reboot commands from automated
|
||
review sessions.
|
||
|
||
## Operations
|
||
|
||
- Host rebuilds should consume the committed `flake.lock`.
|
||
- Routine dependency updates should happen through the flake lock automation
|
||
described in `docs/flake-lock-automation.md`.
|
||
- `nix-cache` serves substitutes over HTTP and can act as a remote builder for
|
||
client hosts.
|
||
- `pxe-boot` serves iPXE boot files over HTTP from `/srv/pxe`.
|
||
|
||
### Deploying a new host
|
||
|
||
Three different paths depending on target, none of them involving a manual
|
||
`nixos-rebuild switch` from this repo:
|
||
|
||
- Most hosts: boot the auto-installer, pick the target from its menu — see
|
||
`docs/auto-installer.md`. Every menu target has a Disko config the
|
||
installer formats unconditionally (`docs/auto-installer.md`'s "Storage"
|
||
section covers how this stays non-destructive for `linode-*`, whose disks
|
||
Linode itself provisions ahead of time).
|
||
- `lxc-*` targets: not installed at all — build a ready-to-run container
|
||
tarball and `pct create` it as a CT template directly. `docs/auto-installer.md`
|
||
covers why (and the installer's menu excludes them for the same reason).
|
||
- `proxmox-*` targets: can alternatively be built as a standalone `.raw`
|
||
disk image and attached to a new VM with no install step — see
|
||
`docs/proxmox-images.md`.
|
||
|
||
`scripts/create-proxmox-resource.sh --type lxc|vm --host <name>` automates
|
||
either of the last two end to end (build, host-key registration, upload,
|
||
`pct create`/`qm create`), with `--dry-run` and a guard against duplicating
|
||
an already-deployed host's identity. See its `--help`.
|
||
|
||
## Security Notes
|
||
|
||
Do not commit tokens, private keys, live credentials, or new password hashes
|
||
as plaintext. Secrets are managed with [sops-nix](https://github.com/Mic92/sops-nix):
|
||
encrypted files live under `secrets/`, recipients (per-host age keys derived
|
||
from each host's existing SSH host key, plus an admin key) are declared in
|
||
`.sops.yaml`. To add or edit a secret:
|
||
|
||
```bash
|
||
nix-shell -p sops --run "sops secrets/<file>.yaml"
|
||
```
|
||
|
||
then reference it from a module via `config.sops.secrets."<name>".path`
|
||
(or `sops.templates` for values that need to be embedded in a rendered
|
||
config file, e.g. `nix.conf`'s `access-tokens`). Never write a secret value
|
||
directly into a tracked `.nix` file. A pre-commit hook (`.githooks/`,
|
||
enabled via `git config core.hooksPath .githooks`, done automatically by
|
||
`scripts/codex-setup.sh`) runs `gitleaks protect --staged` to catch mistakes
|
||
before they're committed.
|
||
|
||
The auto-installer environment is the one deliberate exception to
|
||
sops-nix-everywhere: it has a hardcoded login password instead (no stable
|
||
per-boot host key for sops-nix to derive from on ephemeral media) — see
|
||
"Host keys" in `docs/auto-installer.md` for why, and how the private keys it
|
||
*does* pre-seed for target hosts stay out of git via the gitignored
|
||
`host-keys/` directory.
|
||
|
||
This repository's git *history* still contains secrets committed before this
|
||
migration (see `remove-sensetive-info-refactor.md`) — those are being
|
||
scrubbed and rotated separately; don't treat the repo as safe to make public
|
||
until that's finished.
|