diff --git a/CLAUDE.md b/CLAUDE.md index b0113f0..6351261 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -21,9 +21,8 @@ machines when deployed. `modules/installer/common.nix` (the auto-installer's own root/nixos login — a deliberate, documented choice, see `docs/auto-installer.md`, not accidental tech debt) and **SSH public keys** in `variables.nix` - (`vars.adminSshKey`, `vars.remoteBuilderAuthorizedKeys`) plus a couple of - per-host `KEY` values for beszel-agent auth (`hosts/server/host.nix`, - `hosts/nix-cache/host.nix`). Don't use the installer's hardcoded hash as a + (`vars.adminSshKey`, `vars.remoteBuilderAuthorizedKeys`) plus per-host + `KEY` values for beszel-agent auth (see `docs/beszel.md`). Don't use the installer's hardcoded hash as a template for a *real* host — every other host uses sops-nix (`hashedPasswordFile`, see "Security Notes" in `README.md`). Flag any *new* secret-like string you encounter instead of committing it. @@ -375,9 +374,8 @@ removing a host. - `hosts//host.nix` — per-machine identity **only**: hostname, hostId, per-machine secrets, `system.stateVersion`. These files carry no `imports` - of their own beyond narrow parameterized helpers (see - `modules/beszel/host-token.nix` below) — all shared behavior comes from the - platform/build-type modules composed in `flake.nix`, not from the host file. + of their own — all shared behavior comes from the platform/build-type modules + composed in `flake.nix`, not from the host file. - `modules/platforms/{linode,proxmox,lxc,baremetal}.nix` — platform-specific config: boot method, guest tooling, and the hardware config, imported directly by the platform module itself — **not** wired in from @@ -434,10 +432,10 @@ removing a host. substituter + SSH remote-builder wiring; see `docs/nix-cache.md` for the full design (per-host local stores, no shared `/nix/store`, and how the `nixremote` signing/SSH keys fit together). -- `modules/beszel/host-token.nix` — parameterized helper module - (`{ name, sopsFile }`) that wires a host's beszel-agent sops secret/template - and `environmentFile`; used by `hosts/server/host.nix` and - `hosts/nix-cache/host.nix` to avoid duplicating that boilerplate. +- `modules/beszel/enable-agent.nix` — enables beszel-agent, sets `HUB_URL`, + fixes the upstream `StateDirectory` bug, and wires the universal + `beszel-token` sops secret (from `secrets/common.yaml`) into the agent's + `environmentFile`; see `docs/beszel.md` for the full setup guide. - `modules/tailscale/`, `modules/docker/`, `modules/networking/`, `modules/traefik/`, `modules/tor/`, `modules/services/*` — single-purpose, single-host diff --git a/docs/beszel.md b/docs/beszel.md new file mode 100644 index 0000000..d52d490 --- /dev/null +++ b/docs/beszel.md @@ -0,0 +1,155 @@ +# Beszel agent + +[Beszel](https://github.com/henrygd/beszel) is the monitoring dashboard used +in this LAN. The hub runs as a Docker container on `docker.sweet.home` (port +`vars.ports.beszelHub`, 8090). Each monitored NixOS host runs a +`beszel-agent` that connects back to the hub. + +--- + +## How it works + +Everything is handled by a single module: + +**`modules/beszel/enable-agent.nix`** — imported by a build type. It: +- Enables `beszel-agent` +- Sets `HUB_URL` to `docker.sweet.home:8090` +- Reads the universal `beszel-token` from `secrets/common.yaml` via sops and + passes it to the agent as `TOKEN` in an env file +- Fixes an upstream bug where the agent couldn't persist its hub-pairing + fingerprint across restarts (adds a real `StateDirectory`) + +The only thing a host file ever needs to add is `KEY` — the hub's public key +for that agent slot, which comes from the beszel hub UI after first pairing. + +--- + +## Adding beszel to a new build type + +Add `../beszel/enable-agent.nix` to the `imports` list in +`modules/build-types/.nix`: + +```nix +imports = [ + ../beszel/enable-agent.nix + # ... other imports +]; +``` + +That's the only build-type change required. + +--- + +## Wiring the host file + +No `imports` are needed in the host file. Just set `KEY` once you've paired +the agent with the hub: + +```nix +services.beszel.agent.environment = { + KEY = "ssh-ed25519 AAAA..."; +}; +``` + +Leave `KEY` commented out until after the first pairing (see "Pairing with +the hub" below). + +--- + +## One-time setup: add the token to `secrets/common.yaml` + +The universal token is stored once in the common secrets file, shared by all +agents. You only need to do this once, not per-host: + +```sh +sops secrets/common.yaml +``` + +Add: +```yaml +beszel-token: +``` + +The token is found in the beszel hub under **Settings → Keys** or in the +"Add system" flow. + +`secrets/common.yaml` is already a sops recipient for every host via their +SSH host keys, so no additional sops recipient setup is needed for hosts that +are already provisioned. + +--- + +## Optional: monitoring extra filesystems + +To report disk usage for a mount beyond the root filesystem, add +`EXTRA_FILESYSTEMS` alongside `KEY` in the host file: + +```nix +services.beszel.agent.environment = { + KEY = "ssh-ed25519 AAAA..."; + EXTRA_FILESYSTEMS = "/mnt/data"; # colon-separated for multiple paths +}; +``` + +The `server` host uses this to expose its ZFS data pool: + +```nix +EXTRA_FILESYSTEMS = "${vars.storageRoot}/${vars.nfsShares.dockerVolumes.subpath}"; +``` + +--- + +## Optional: monitoring Docker containers + +`enable-agent.nix` has a commented-out line for Docker monitoring: + +```nix +#DOCKER_HOST = "tcp://docker-socket-proxy:2375"; +``` + +Uncomment it if the host runs docker-socket-proxy and you want per-container +stats. Hosts without Docker should leave it commented out. + +--- + +## Pairing with the hub + +1. Deploy the host with its build type importing `enable-agent.nix`. Leave + `KEY` unset (commented out) for now. +2. Open the beszel hub (`http://docker.sweet.home:8090`). +3. Go to **Systems → Add system**. The new host should appear as an unpaired + entry — copy the `KEY` value shown there. +4. In `hosts//host.nix`, set: + ```nix + services.beszel.agent.environment.KEY = ""; + ``` +5. Rebuild and deploy the host. The agent will now pair permanently. + +--- + +## Example: complete host file + +Minimal case (`hosts/tor-relay/host.nix` — one filesystem, no Docker, LXC): + +```nix +{ vars, ... }: +{ + networking = { ... }; + + services.beszel.agent.environment = { + KEY = "ssh-ed25519 AAAA..."; + }; + + system.stateVersion = "26.05"; +} +``` + +Fuller case (`hosts/server/host.nix` — extra filesystem, debug logging): + +```nix +services.beszel.agent.environment = { + KEY = "ssh-ed25519 AAAA..."; + EXTRA_FILESYSTEMS = "${vars.storageRoot}/${vars.nfsShares.dockerVolumes.subpath}"; + LOG_LEVEL = "debug"; +}; +``` diff --git a/hosts/ha-server-1/host.nix b/hosts/ha-server-1/host.nix index f1b91b0..f923f5d 100644 --- a/hosts/ha-server-1/host.nix +++ b/hosts/ha-server-1/host.nix @@ -1,12 +1,5 @@ { vars, ... }: { - imports = [ - (import ../../modules/beszel/host-token.nix { - name = "ha-server-1"; - sopsFile = ../../secrets/ha-server-1.yaml; - }) - ]; - networking = { hostName = vars.haServer1Host; hostId = "3a4b5c6d"; @@ -23,7 +16,7 @@ nameservers = [ vars.domainControllerIp ]; }; - # Set KEY after pairing this host with the beszel hub; the token is sops-managed. + # Set KEY after pairing this host with the beszel hub (see docs/beszel.md). # services.beszel.agent.environment.KEY = ""; system.stateVersion = "26.05"; diff --git a/hosts/ha-server-2/host.nix b/hosts/ha-server-2/host.nix index 4439a33..5dc038a 100644 --- a/hosts/ha-server-2/host.nix +++ b/hosts/ha-server-2/host.nix @@ -1,12 +1,5 @@ { vars, ... }: { - imports = [ - (import ../../modules/beszel/host-token.nix { - name = "ha-server-2"; - sopsFile = ../../secrets/ha-server-2.yaml; - }) - ]; - networking = { hostName = vars.haServer2Host; hostId = "7e8f9a0b"; @@ -23,7 +16,7 @@ nameservers = [ vars.domainControllerIp ]; }; - # Set KEY after pairing this host with the beszel hub; the token is sops-managed. + # Set KEY after pairing this host with the beszel hub (see docs/beszel.md). # services.beszel.agent.environment.KEY = ""; system.stateVersion = "26.05"; diff --git a/hosts/nix-cache/host.nix b/hosts/nix-cache/host.nix index d314f59..e886fb5 100644 --- a/hosts/nix-cache/host.nix +++ b/hosts/nix-cache/host.nix @@ -1,13 +1,6 @@ { vars, ... }: { - imports = [ - (import ../../modules/beszel/host-token.nix { - name = "nix-cache"; - sopsFile = ../../secrets/nix-cache.yaml; - }) - ]; - networking = { hostName = vars.nixCacheHost; useDHCP = false; diff --git a/hosts/server/host.nix b/hosts/server/host.nix index 344c2d8..775e451 100644 --- a/hosts/server/host.nix +++ b/hosts/server/host.nix @@ -1,13 +1,6 @@ { vars, ... }: { - imports = [ - (import ../../modules/beszel/host-token.nix { - name = "server"; - sopsFile = ../../secrets/server.yaml; - }) - ]; - networking = { hostName = vars.nfsServerHost; hostId = "6689f93e"; diff --git a/hosts/tailscale-router/host.nix b/hosts/tailscale-router/host.nix index 6198bf4..9f3dae7 100644 --- a/hosts/tailscale-router/host.nix +++ b/hosts/tailscale-router/host.nix @@ -1,13 +1,6 @@ { vars, ... }: { - imports = [ - (import ../../modules/beszel/host-token.nix { - name = "tailscale-router"; - sopsFile = ../../secrets/tailscale-router.yaml; - }) - ]; - networking = { hostName = "tailscale-router"; useDHCP = false; diff --git a/hosts/tor-relay/host.nix b/hosts/tor-relay/host.nix index 8cdcad3..7e894ce 100644 --- a/hosts/tor-relay/host.nix +++ b/hosts/tor-relay/host.nix @@ -1,13 +1,6 @@ { vars, ... }: { - imports = [ - (import ../../modules/beszel/host-token.nix { - name = "tor-relay"; - sopsFile = ../../secrets/tor-relay.yaml; - }) - ]; - networking = { hostName = "tor-relay"; useDHCP = false; diff --git a/modules/beszel/enable-agent.nix b/modules/beszel/enable-agent.nix index 0770f74..8e97657 100644 --- a/modules/beszel/enable-agent.nix +++ b/modules/beszel/enable-agent.nix @@ -1,10 +1,22 @@ -{ vars, ... }: +{ config, vars, ... }: { - services.beszel.agent.enable = true; - services.beszel.agent.environment = { - #DOCKER_HOST = "tcp://docker-socket-proxy:2375"; - HUB_URL = "http://${vars.dockerHost}.${vars.homeDomain}:${toString vars.ports.beszelHub}"; + # Universal token shared by all beszel agents. Add to secrets/common.yaml: + # sops secrets/common.yaml + # beszel-token: + sops.secrets."beszel-token" = { }; + + sops.templates."beszel.env".content = '' + TOKEN=${config.sops.placeholder."beszel-token"} + ''; + + services.beszel.agent = { + enable = true; + environmentFile = config.sops.templates."beszel.env".path; + environment = { + #DOCKER_HOST = "tcp://docker-socket-proxy:2375"; + HUB_URL = "http://${vars.dockerHost}.${vars.homeDomain}:${toString vars.ports.beszelHub}"; + }; }; # The upstream module runs beszel-agent under DynamicUser with diff --git a/modules/beszel/host-token.nix b/modules/beszel/host-token.nix deleted file mode 100644 index 5339b4b..0000000 --- a/modules/beszel/host-token.nix +++ /dev/null @@ -1,11 +0,0 @@ -{ name, sopsFile }: - -{ config, ... }: - -{ - sops.secrets."beszel-token".sopsFile = sopsFile; - sops.templates."${name}-beszel.env".content = '' - TOKEN=${config.sops.placeholder."beszel-token"} - ''; - services.beszel.agent.environmentFile = config.sops.templates."${name}-beszel.env".path; -}