Merge branch 'main' of https://gitea.lan.ddnsgeek.com/beatzaplenty/nixos

CLAUDE.md

@@ -21,9 +21,8 @@ machines when deployed.
   `modules/installer/common.nix` (the auto-installer's own root/nixos login —
   a deliberate, documented choice, see `docs/auto-installer.md`, not
   accidental tech debt) and **SSH public keys** in `variables.nix`
-  (`vars.adminSshKey`, `vars.remoteBuilderAuthorizedKeys`) plus a couple of
-  per-host `KEY` values for beszel-agent auth (`hosts/server/host.nix`,
-  `hosts/nix-cache/host.nix`). Don't use the installer's hardcoded hash as a
+  (`vars.adminSshKey`, `vars.remoteBuilderAuthorizedKeys`) plus per-host
+  `KEY` values for beszel-agent auth (see `docs/beszel.md`). Don't use the installer's hardcoded hash as a
   template for a *real* host — every other host uses sops-nix
   (`hashedPasswordFile`, see "Security Notes" in `README.md`). Flag any *new*
   secret-like string you encounter instead of committing it.
@@ -375,9 +374,8 @@ removing a host.
 
 - `hosts/<name>/host.nix` — per-machine identity **only**: hostname, hostId,
   per-machine secrets, `system.stateVersion`. These files carry no `imports`
-  of their own beyond narrow parameterized helpers (see
-  `modules/beszel/host-token.nix` below) — all shared behavior comes from the
-  platform/build-type modules composed in `flake.nix`, not from the host file.
+  of their own — all shared behavior comes from the platform/build-type modules
+  composed in `flake.nix`, not from the host file.
 - `modules/platforms/{linode,proxmox,lxc,baremetal}.nix` — platform-specific
   config: boot method, guest tooling, and the hardware config, imported
   directly by the platform module itself — **not** wired in from
@@ -434,10 +432,10 @@ removing a host.
   substituter + SSH remote-builder wiring; see `docs/nix-cache.md` for the
   full design (per-host local stores, no shared `/nix/store`, and how the
   `nixremote` signing/SSH keys fit together).
-- `modules/beszel/host-token.nix` — parameterized helper module
-  (`{ name, sopsFile }`) that wires a host's beszel-agent sops secret/template
-  and `environmentFile`; used by `hosts/server/host.nix` and
-  `hosts/nix-cache/host.nix` to avoid duplicating that boilerplate.
+- `modules/beszel/enable-agent.nix` — enables beszel-agent, sets `HUB_URL`,
+  fixes the upstream `StateDirectory` bug, and wires the universal
+  `beszel-token` sops secret (from `secrets/common.yaml`) into the agent's
+  `environmentFile`; see `docs/beszel.md` for the full setup guide.
 - `modules/tailscale/`, `modules/docker/`, `modules/networking/`,
   `modules/traefik/`, `modules/tor/`, `modules/services/*` — single-purpose,
   single-host

docs/beszel.md

@@ -0,0 +1,155 @@
+# Beszel agent
+
+[Beszel](https://github.com/henrygd/beszel) is the monitoring dashboard used
+in this LAN. The hub runs as a Docker container on `docker.sweet.home` (port
+`vars.ports.beszelHub`, 8090). Each monitored NixOS host runs a
+`beszel-agent` that connects back to the hub.
+
+---
+
+## How it works
+
+Everything is handled by a single module:
+
+**`modules/beszel/enable-agent.nix`** — imported by a build type. It:
+- Enables `beszel-agent`
+- Sets `HUB_URL` to `docker.sweet.home:8090`
+- Reads the universal `beszel-token` from `secrets/common.yaml` via sops and
+  passes it to the agent as `TOKEN` in an env file
+- Fixes an upstream bug where the agent couldn't persist its hub-pairing
+  fingerprint across restarts (adds a real `StateDirectory`)
+
+The only thing a host file ever needs to add is `KEY` — the hub's public key
+for that agent slot, which comes from the beszel hub UI after first pairing.
+
+---
+
+## Adding beszel to a new build type
+
+Add `../beszel/enable-agent.nix` to the `imports` list in
+`modules/build-types/<type>.nix`:
+
+```nix
+imports = [
+  ../beszel/enable-agent.nix
+  # ... other imports
+];
+```
+
+That's the only build-type change required.
+
+---
+
+## Wiring the host file
+
+No `imports` are needed in the host file. Just set `KEY` once you've paired
+the agent with the hub:
+
+```nix
+services.beszel.agent.environment = {
+  KEY = "ssh-ed25519 AAAA...";
+};
+```
+
+Leave `KEY` commented out until after the first pairing (see "Pairing with
+the hub" below).
+
+---
+
+## One-time setup: add the token to `secrets/common.yaml`
+
+The universal token is stored once in the common secrets file, shared by all
+agents. You only need to do this once, not per-host:
+
+```sh
+sops secrets/common.yaml
+```
+
+Add:
+```yaml
+beszel-token: <token from the beszel hub UI>
+```
+
+The token is found in the beszel hub under **Settings → Keys** or in the
+"Add system" flow.
+
+`secrets/common.yaml` is already a sops recipient for every host via their
+SSH host keys, so no additional sops recipient setup is needed for hosts that
+are already provisioned.
+
+---
+
+## Optional: monitoring extra filesystems
+
+To report disk usage for a mount beyond the root filesystem, add
+`EXTRA_FILESYSTEMS` alongside `KEY` in the host file:
+
+```nix
+services.beszel.agent.environment = {
+  KEY = "ssh-ed25519 AAAA...";
+  EXTRA_FILESYSTEMS = "/mnt/data";   # colon-separated for multiple paths
+};
+```
+
+The `server` host uses this to expose its ZFS data pool:
+
+```nix
+EXTRA_FILESYSTEMS = "${vars.storageRoot}/${vars.nfsShares.dockerVolumes.subpath}";
+```
+
+---
+
+## Optional: monitoring Docker containers
+
+`enable-agent.nix` has a commented-out line for Docker monitoring:
+
+```nix
+#DOCKER_HOST = "tcp://docker-socket-proxy:2375";
+```
+
+Uncomment it if the host runs docker-socket-proxy and you want per-container
+stats. Hosts without Docker should leave it commented out.
+
+---
+
+## Pairing with the hub
+
+1. Deploy the host with its build type importing `enable-agent.nix`. Leave
+   `KEY` unset (commented out) for now.
+2. Open the beszel hub (`http://docker.sweet.home:8090`).
+3. Go to **Systems → Add system**. The new host should appear as an unpaired
+   entry — copy the `KEY` value shown there.
+4. In `hosts/<name>/host.nix`, set:
+   ```nix
+   services.beszel.agent.environment.KEY = "<copied key>";
+   ```
+5. Rebuild and deploy the host. The agent will now pair permanently.
+
+---
+
+## Example: complete host file
+
+Minimal case (`hosts/tor-relay/host.nix` — one filesystem, no Docker, LXC):
+
+```nix
+{ vars, ... }:
+{
+  networking = { ... };
+
+  services.beszel.agent.environment = {
+    KEY = "ssh-ed25519 AAAA...";
+  };
+
+  system.stateVersion = "26.05";
+}
+```
+
+Fuller case (`hosts/server/host.nix` — extra filesystem, debug logging):
+
+```nix
+services.beszel.agent.environment = {
+  KEY = "ssh-ed25519 AAAA...";
+  EXTRA_FILESYSTEMS = "${vars.storageRoot}/${vars.nfsShares.dockerVolumes.subpath}";
+  LOG_LEVEL = "debug";
+};
+```

hosts/ha-server-1/host.nix

@@ -1,12 +1,5 @@
 { vars, ... }:
 {
-  imports = [
-    (import ../../modules/beszel/host-token.nix {
-      name = "ha-server-1";
-      sopsFile = ../../secrets/ha-server-1.yaml;
-    })
-  ];
-
   networking = {
     hostName = vars.haServer1Host;
     hostId = "3a4b5c6d";
@@ -23,7 +16,7 @@
     nameservers = [ vars.domainControllerIp ];
   };
 
-  # Set KEY after pairing this host with the beszel hub; the token is sops-managed.
+  # Set KEY after pairing this host with the beszel hub (see docs/beszel.md).
   # services.beszel.agent.environment.KEY = "";
 
   system.stateVersion = "26.05";

hosts/ha-server-2/host.nix

@@ -1,12 +1,5 @@
 { vars, ... }:
 {
-  imports = [
-    (import ../../modules/beszel/host-token.nix {
-      name = "ha-server-2";
-      sopsFile = ../../secrets/ha-server-2.yaml;
-    })
-  ];
-
   networking = {
     hostName = vars.haServer2Host;
     hostId = "7e8f9a0b";
@@ -23,7 +16,7 @@
     nameservers = [ vars.domainControllerIp ];
   };
 
-  # Set KEY after pairing this host with the beszel hub; the token is sops-managed.
+  # Set KEY after pairing this host with the beszel hub (see docs/beszel.md).
   # services.beszel.agent.environment.KEY = "";
 
   system.stateVersion = "26.05";

hosts/nix-cache/host.nix

@@ -1,13 +1,6 @@
 { vars, ... }:
 
 {
-  imports = [
-    (import ../../modules/beszel/host-token.nix {
-      name = "nix-cache";
-      sopsFile = ../../secrets/nix-cache.yaml;
-    })
-  ];
-
   networking = {
     hostName = vars.nixCacheHost;
     useDHCP = false;

hosts/server/host.nix

@@ -1,13 +1,6 @@
 { vars, ... }:
 
 {
-  imports = [
-    (import ../../modules/beszel/host-token.nix {
-      name = "server";
-      sopsFile = ../../secrets/server.yaml;
-    })
-  ];
-
   networking = {
     hostName = vars.nfsServerHost;
     hostId = "6689f93e";

hosts/tailscale-router/host.nix

@@ -1,13 +1,6 @@
 { vars, ... }:
 
 {
-  imports = [
-    (import ../../modules/beszel/host-token.nix {
-      name = "tailscale-router";
-      sopsFile = ../../secrets/tailscale-router.yaml;
-    })
-  ];
-
   networking = {
     hostName = "tailscale-router";
     useDHCP = false;

hosts/tor-relay/host.nix

@@ -1,13 +1,6 @@
 { vars, ... }:
 
 {
-  imports = [
-    (import ../../modules/beszel/host-token.nix {
-      name = "tor-relay";
-      sopsFile = ../../secrets/tor-relay.yaml;
-    })
-  ];
-
   networking = {
     hostName = "tor-relay";
     useDHCP = false;

modules/beszel/enable-agent.nix

@@ -1,11 +1,23 @@
-{ vars, ... }:
+{ config, vars, ... }:
 
 {
-  services.beszel.agent.enable = true;
-  services.beszel.agent.environment = {
+  # Universal token shared by all beszel agents. Add to secrets/common.yaml:
+  #   sops secrets/common.yaml
+  #   beszel-token: <value from the beszel hub UI>
+  sops.secrets."beszel-token" = { };
+
+  sops.templates."beszel.env".content = ''
+    TOKEN=${config.sops.placeholder."beszel-token"}
+  '';
+
+  services.beszel.agent = {
+    enable = true;
+    environmentFile = config.sops.templates."beszel.env".path;
+    environment = {
       #DOCKER_HOST = "tcp://docker-socket-proxy:2375";
       HUB_URL = "http://${vars.dockerHost}.${vars.homeDomain}:${toString vars.ports.beszelHub}";
     };
+  };
 
   # The upstream module runs beszel-agent under DynamicUser with
   # ProtectSystem = "strict" and no StateDirectory, so /var/lib/beszel-agent

modules/beszel/host-token.nix

@@ -1,11 +0,0 @@
-{ name, sopsFile }:
-
-{ config, ... }:
-
-{
-  sops.secrets."beszel-token".sopsFile = sopsFile;
-  sops.templates."${name}-beszel.env".content = ''
-    TOKEN=${config.sops.placeholder."beszel-token"}
-  '';
-  services.beszel.agent.environmentFile = config.sops.templates."${name}-beszel.env".path;
-}