Archived
refactor(beszel): switch to universal token via secrets/common.yaml
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m40s
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m40s
Replace per-host host-token.nix imports with a single beszel-token secret in secrets/common.yaml, wired once in enable-agent.nix. Host files now only need services.beszel.agent.environment.KEY — no imports block required. Delete modules/beszel/host-token.nix (no longer referenced anywhere). Action needed: run `sops secrets/common.yaml` and add `beszel-token: <value>` from the beszel hub UI before deploying. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -21,9 +21,8 @@ machines when deployed.
|
||||
`modules/installer/common.nix` (the auto-installer's own root/nixos login —
|
||||
a deliberate, documented choice, see `docs/auto-installer.md`, not
|
||||
accidental tech debt) and **SSH public keys** in `variables.nix`
|
||||
(`vars.adminSshKey`, `vars.remoteBuilderAuthorizedKeys`) plus a couple of
|
||||
per-host `KEY` values for beszel-agent auth (`hosts/server/host.nix`,
|
||||
`hosts/nix-cache/host.nix`). Don't use the installer's hardcoded hash as a
|
||||
(`vars.adminSshKey`, `vars.remoteBuilderAuthorizedKeys`) plus per-host
|
||||
`KEY` values for beszel-agent auth (see `docs/beszel.md`). Don't use the installer's hardcoded hash as a
|
||||
template for a *real* host — every other host uses sops-nix
|
||||
(`hashedPasswordFile`, see "Security Notes" in `README.md`). Flag any *new*
|
||||
secret-like string you encounter instead of committing it.
|
||||
@@ -375,9 +374,8 @@ removing a host.
|
||||
|
||||
- `hosts/<name>/host.nix` — per-machine identity **only**: hostname, hostId,
|
||||
per-machine secrets, `system.stateVersion`. These files carry no `imports`
|
||||
of their own beyond narrow parameterized helpers (see
|
||||
`modules/beszel/host-token.nix` below) — all shared behavior comes from the
|
||||
platform/build-type modules composed in `flake.nix`, not from the host file.
|
||||
of their own — all shared behavior comes from the platform/build-type modules
|
||||
composed in `flake.nix`, not from the host file.
|
||||
- `modules/platforms/{linode,proxmox,lxc,baremetal}.nix` — platform-specific
|
||||
config: boot method, guest tooling, and the hardware config, imported
|
||||
directly by the platform module itself — **not** wired in from
|
||||
@@ -434,10 +432,10 @@ removing a host.
|
||||
substituter + SSH remote-builder wiring; see `docs/nix-cache.md` for the
|
||||
full design (per-host local stores, no shared `/nix/store`, and how the
|
||||
`nixremote` signing/SSH keys fit together).
|
||||
- `modules/beszel/host-token.nix` — parameterized helper module
|
||||
(`{ name, sopsFile }`) that wires a host's beszel-agent sops secret/template
|
||||
and `environmentFile`; used by `hosts/server/host.nix` and
|
||||
`hosts/nix-cache/host.nix` to avoid duplicating that boilerplate.
|
||||
- `modules/beszel/enable-agent.nix` — enables beszel-agent, sets `HUB_URL`,
|
||||
fixes the upstream `StateDirectory` bug, and wires the universal
|
||||
`beszel-token` sops secret (from `secrets/common.yaml`) into the agent's
|
||||
`environmentFile`; see `docs/beszel.md` for the full setup guide.
|
||||
- `modules/tailscale/`, `modules/docker/`, `modules/networking/`,
|
||||
`modules/traefik/`, `modules/tor/`, `modules/services/*` — single-purpose,
|
||||
single-host
|
||||
|
||||
+42
-79
@@ -5,30 +5,29 @@ in this LAN. The hub runs as a Docker container on `docker.sweet.home` (port
|
||||
`vars.ports.beszelHub`, 8090). Each monitored NixOS host runs a
|
||||
`beszel-agent` that connects back to the hub.
|
||||
|
||||
This document explains how to wire beszel-agent into any build in the flake.
|
||||
|
||||
---
|
||||
|
||||
## How it is split across modules
|
||||
## How it works
|
||||
|
||||
There are two modules and they have separate jobs:
|
||||
Everything is handled by a single module:
|
||||
|
||||
| Module | What it does | Where it goes |
|
||||
|---|---|---|
|
||||
| `modules/beszel/enable-agent.nix` | Enables the systemd service, sets `HUB_URL`, fixes upstream StateDirectory bug | **Build type** (`modules/build-types/*.nix`) |
|
||||
| `modules/beszel/host-token.nix` | Declares the sops secret + env-file template that holds `TOKEN` | **Host file** (`hosts/<name>/host.nix`) |
|
||||
**`modules/beszel/enable-agent.nix`** — imported by a build type. It:
|
||||
- Enables `beszel-agent`
|
||||
- Sets `HUB_URL` to `docker.sweet.home:8090`
|
||||
- Reads the universal `beszel-token` from `secrets/common.yaml` via sops and
|
||||
passes it to the agent as `TOKEN` in an env file
|
||||
- Fixes an upstream bug where the agent couldn't persist its hub-pairing
|
||||
fingerprint across restarts (adds a real `StateDirectory`)
|
||||
|
||||
`KEY` (the hub's public key for this agent slot) is a plain string set
|
||||
directly on the host under `services.beszel.agent.environment.KEY` — it comes
|
||||
from the hub UI, not from sops, so it belongs in the host file rather than a
|
||||
secret.
|
||||
The only thing a host file ever needs to add is `KEY` — the hub's public key
|
||||
for that agent slot, which comes from the beszel hub UI after first pairing.
|
||||
|
||||
---
|
||||
|
||||
## Adding beszel to a new build type
|
||||
|
||||
Open `modules/build-types/<type>.nix` and add `../beszel/enable-agent.nix` to
|
||||
the `imports` list:
|
||||
Add `../beszel/enable-agent.nix` to the `imports` list in
|
||||
`modules/build-types/<type>.nix`:
|
||||
|
||||
```nix
|
||||
imports = [
|
||||
@@ -37,34 +36,14 @@ imports = [
|
||||
];
|
||||
```
|
||||
|
||||
That is the only change needed at the build-type level. The module sets:
|
||||
- `services.beszel.agent.enable = true`
|
||||
- `HUB_URL` pointing at `docker.sweet.home:8090`
|
||||
- a persistent `StateDirectory` for the hub-pairing fingerprint
|
||||
That's the only build-type change required.
|
||||
|
||||
---
|
||||
|
||||
## Wiring the host file
|
||||
|
||||
Every host whose build type includes `enable-agent.nix` needs two things in
|
||||
its `hosts/<name>/host.nix`:
|
||||
|
||||
### 1 — Import `host-token.nix`
|
||||
|
||||
```nix
|
||||
imports = [
|
||||
(import ../../modules/beszel/host-token.nix {
|
||||
name = "<hostname>"; # must match the sops secret file name below
|
||||
sopsFile = ../../secrets/<hostname>.yaml;
|
||||
})
|
||||
];
|
||||
```
|
||||
|
||||
This creates a sops-managed env file at runtime containing `TOKEN=<value>` and
|
||||
passes it to the agent via `environmentFile`. The `name` argument also becomes
|
||||
the sops template name (`<name>-beszel.env`), so it must be unique per host.
|
||||
|
||||
### 2 — Set `KEY` once the hub has paired the agent
|
||||
No `imports` are needed in the host file. Just set `KEY` once you've paired
|
||||
the agent with the hub:
|
||||
|
||||
```nix
|
||||
services.beszel.agent.environment = {
|
||||
@@ -72,53 +51,47 @@ services.beszel.agent.environment = {
|
||||
};
|
||||
```
|
||||
|
||||
`KEY` is the hub's SSH public key for this agent slot — copy it from the hub
|
||||
UI after the agent first connects (see "Pairing with the hub" below).
|
||||
|
||||
Until the hub has been paired, leave `KEY` commented out or omit it. The agent
|
||||
will still start and appear in the hub as an unpaired entry; fill in the key
|
||||
after that first connection.
|
||||
Leave `KEY` commented out until after the first pairing (see "Pairing with
|
||||
the hub" below).
|
||||
|
||||
---
|
||||
|
||||
## Adding the sops secret
|
||||
## One-time setup: add the token to `secrets/common.yaml`
|
||||
|
||||
The `beszel-token` value in `secrets/<hostname>.yaml` is the token the agent
|
||||
uses to authenticate to the hub. To add it for a new host:
|
||||
The universal token is stored once in the common secrets file, shared by all
|
||||
agents. You only need to do this once, not per-host:
|
||||
|
||||
1. Decrypt and edit the host's secrets file:
|
||||
```sh
|
||||
sops secrets/<hostname>.yaml
|
||||
sops secrets/common.yaml
|
||||
```
|
||||
2. Add the token:
|
||||
|
||||
Add:
|
||||
```yaml
|
||||
beszel-token: <token from the beszel hub UI>
|
||||
```
|
||||
3. Save and close — sops re-encrypts the file on exit.
|
||||
|
||||
If the host does not yet have a `secrets/<hostname>.yaml`, create one using
|
||||
the same sops recipients as the other secrets files (check `.sops.yaml`), or
|
||||
run `scripts/secrets/sync-host-keys.sh <target>` first so the host's own
|
||||
SSH key is already a valid recipient.
|
||||
The token is found in the beszel hub under **Settings → Keys** or in the
|
||||
"Add system" flow.
|
||||
|
||||
The token is found in the beszel hub under **Settings → Keys** or when you
|
||||
add a new system via the hub UI.
|
||||
`secrets/common.yaml` is already a sops recipient for every host via their
|
||||
SSH host keys, so no additional sops recipient setup is needed for hosts that
|
||||
are already provisioned.
|
||||
|
||||
---
|
||||
|
||||
## Optional: monitoring extra filesystems
|
||||
|
||||
To have the agent report disk usage for a mount point that is not the root
|
||||
filesystem, add `EXTRA_FILESYSTEMS` in the host file alongside `KEY`:
|
||||
To report disk usage for a mount beyond the root filesystem, add
|
||||
`EXTRA_FILESYSTEMS` alongside `KEY` in the host file:
|
||||
|
||||
```nix
|
||||
services.beszel.agent.environment = {
|
||||
KEY = "ssh-ed25519 AAAA...";
|
||||
EXTRA_FILESYSTEMS = "/mnt/data"; # colon-separated if more than one
|
||||
EXTRA_FILESYSTEMS = "/mnt/data"; # colon-separated for multiple paths
|
||||
};
|
||||
```
|
||||
|
||||
The `server` host uses this to expose the ZFS data pool:
|
||||
The `server` host uses this to expose its ZFS data pool:
|
||||
|
||||
```nix
|
||||
EXTRA_FILESYSTEMS = "${vars.storageRoot}/${vars.nfsShares.dockerVolumes.subpath}";
|
||||
@@ -128,48 +101,39 @@ EXTRA_FILESYSTEMS = "${vars.storageRoot}/${vars.nfsShares.dockerVolumes.subpath}
|
||||
|
||||
## Optional: monitoring Docker containers
|
||||
|
||||
The `enable-agent.nix` module has a commented-out line for Docker monitoring:
|
||||
`enable-agent.nix` has a commented-out line for Docker monitoring:
|
||||
|
||||
```nix
|
||||
#DOCKER_HOST = "tcp://docker-socket-proxy:2375";
|
||||
```
|
||||
|
||||
Uncomment it if the host runs docker-socket-proxy and you want per-container
|
||||
stats in the hub. Hosts that do not run Docker should leave this commented out.
|
||||
stats. Hosts without Docker should leave it commented out.
|
||||
|
||||
---
|
||||
|
||||
## Pairing with the hub
|
||||
|
||||
1. Deploy the host with `host-token.nix` imported and `enable-agent.nix`
|
||||
present in the build type. Leave `KEY` unset for now.
|
||||
1. Deploy the host with its build type importing `enable-agent.nix`. Leave
|
||||
`KEY` unset (commented out) for now.
|
||||
2. Open the beszel hub (`http://docker.sweet.home:8090`).
|
||||
3. Go to **Systems → Add system**. The new host should appear as an
|
||||
unpaired entry — copy the `KEY` value shown there.
|
||||
3. Go to **Systems → Add system**. The new host should appear as an unpaired
|
||||
entry — copy the `KEY` value shown there.
|
||||
4. In `hosts/<name>/host.nix`, set:
|
||||
```nix
|
||||
services.beszel.agent.environment.KEY = "<copied key>";
|
||||
```
|
||||
5. Rebuild and deploy the host. The agent will now pair permanently with the
|
||||
hub.
|
||||
5. Rebuild and deploy the host. The agent will now pair permanently.
|
||||
|
||||
---
|
||||
|
||||
## Example: complete host file
|
||||
|
||||
For reference, `hosts/tor-relay/host.nix` is the minimal case — one
|
||||
filesystem, no Docker, LXC platform:
|
||||
Minimal case (`hosts/tor-relay/host.nix` — one filesystem, no Docker, LXC):
|
||||
|
||||
```nix
|
||||
{ vars, ... }:
|
||||
{
|
||||
imports = [
|
||||
(import ../../modules/beszel/host-token.nix {
|
||||
name = "tor-relay";
|
||||
sopsFile = ../../secrets/tor-relay.yaml;
|
||||
})
|
||||
];
|
||||
|
||||
networking = { ... };
|
||||
|
||||
services.beszel.agent.environment = {
|
||||
@@ -180,8 +144,7 @@ filesystem, no Docker, LXC platform:
|
||||
}
|
||||
```
|
||||
|
||||
`hosts/server/host.nix` is the fuller case — extra filesystem path, debug
|
||||
logging:
|
||||
Fuller case (`hosts/server/host.nix` — extra filesystem, debug logging):
|
||||
|
||||
```nix
|
||||
services.beszel.agent.environment = {
|
||||
|
||||
@@ -1,12 +1,5 @@
|
||||
{ vars, ... }:
|
||||
{
|
||||
imports = [
|
||||
(import ../../modules/beszel/host-token.nix {
|
||||
name = "ha-server-1";
|
||||
sopsFile = ../../secrets/ha-server-1.yaml;
|
||||
})
|
||||
];
|
||||
|
||||
networking = {
|
||||
hostName = vars.haServer1Host;
|
||||
hostId = "3a4b5c6d";
|
||||
@@ -23,7 +16,7 @@
|
||||
nameservers = [ vars.domainControllerIp ];
|
||||
};
|
||||
|
||||
# Set KEY after pairing this host with the beszel hub; the token is sops-managed.
|
||||
# Set KEY after pairing this host with the beszel hub (see docs/beszel.md).
|
||||
# services.beszel.agent.environment.KEY = "";
|
||||
|
||||
system.stateVersion = "26.05";
|
||||
|
||||
@@ -1,12 +1,5 @@
|
||||
{ vars, ... }:
|
||||
{
|
||||
imports = [
|
||||
(import ../../modules/beszel/host-token.nix {
|
||||
name = "ha-server-2";
|
||||
sopsFile = ../../secrets/ha-server-2.yaml;
|
||||
})
|
||||
];
|
||||
|
||||
networking = {
|
||||
hostName = vars.haServer2Host;
|
||||
hostId = "7e8f9a0b";
|
||||
@@ -23,7 +16,7 @@
|
||||
nameservers = [ vars.domainControllerIp ];
|
||||
};
|
||||
|
||||
# Set KEY after pairing this host with the beszel hub; the token is sops-managed.
|
||||
# Set KEY after pairing this host with the beszel hub (see docs/beszel.md).
|
||||
# services.beszel.agent.environment.KEY = "";
|
||||
|
||||
system.stateVersion = "26.05";
|
||||
|
||||
@@ -1,13 +1,6 @@
|
||||
{ vars, ... }:
|
||||
|
||||
{
|
||||
imports = [
|
||||
(import ../../modules/beszel/host-token.nix {
|
||||
name = "nix-cache";
|
||||
sopsFile = ../../secrets/nix-cache.yaml;
|
||||
})
|
||||
];
|
||||
|
||||
networking = {
|
||||
hostName = vars.nixCacheHost;
|
||||
useDHCP = false;
|
||||
|
||||
@@ -1,13 +1,6 @@
|
||||
{ vars, ... }:
|
||||
|
||||
{
|
||||
imports = [
|
||||
(import ../../modules/beszel/host-token.nix {
|
||||
name = "server";
|
||||
sopsFile = ../../secrets/server.yaml;
|
||||
})
|
||||
];
|
||||
|
||||
networking = {
|
||||
hostName = vars.nfsServerHost;
|
||||
hostId = "6689f93e";
|
||||
|
||||
@@ -1,13 +1,6 @@
|
||||
{ vars, ... }:
|
||||
|
||||
{
|
||||
imports = [
|
||||
(import ../../modules/beszel/host-token.nix {
|
||||
name = "tailscale-router";
|
||||
sopsFile = ../../secrets/tailscale-router.yaml;
|
||||
})
|
||||
];
|
||||
|
||||
networking = {
|
||||
hostName = "tailscale-router";
|
||||
useDHCP = false;
|
||||
|
||||
@@ -1,13 +1,6 @@
|
||||
{ vars, ... }:
|
||||
|
||||
{
|
||||
imports = [
|
||||
(import ../../modules/beszel/host-token.nix {
|
||||
name = "tor-relay";
|
||||
sopsFile = ../../secrets/tor-relay.yaml;
|
||||
})
|
||||
];
|
||||
|
||||
networking = {
|
||||
hostName = "tor-relay";
|
||||
useDHCP = false;
|
||||
|
||||
@@ -1,11 +1,23 @@
|
||||
{ vars, ... }:
|
||||
{ config, vars, ... }:
|
||||
|
||||
{
|
||||
services.beszel.agent.enable = true;
|
||||
services.beszel.agent.environment = {
|
||||
# Universal token shared by all beszel agents. Add to secrets/common.yaml:
|
||||
# sops secrets/common.yaml
|
||||
# beszel-token: <value from the beszel hub UI>
|
||||
sops.secrets."beszel-token" = { };
|
||||
|
||||
sops.templates."beszel.env".content = ''
|
||||
TOKEN=${config.sops.placeholder."beszel-token"}
|
||||
'';
|
||||
|
||||
services.beszel.agent = {
|
||||
enable = true;
|
||||
environmentFile = config.sops.templates."beszel.env".path;
|
||||
environment = {
|
||||
#DOCKER_HOST = "tcp://docker-socket-proxy:2375";
|
||||
HUB_URL = "http://${vars.dockerHost}.${vars.homeDomain}:${toString vars.ports.beszelHub}";
|
||||
};
|
||||
};
|
||||
|
||||
# The upstream module runs beszel-agent under DynamicUser with
|
||||
# ProtectSystem = "strict" and no StateDirectory, so /var/lib/beszel-agent
|
||||
|
||||
@@ -1,11 +0,0 @@
|
||||
{ name, sopsFile }:
|
||||
|
||||
{ config, ... }:
|
||||
|
||||
{
|
||||
sops.secrets."beszel-token".sopsFile = sopsFile;
|
||||
sops.templates."${name}-beszel.env".content = ''
|
||||
TOKEN=${config.sops.placeholder."beszel-token"}
|
||||
'';
|
||||
services.beszel.agent.environmentFile = config.sops.templates."${name}-beszel.env".path;
|
||||
}
|
||||
Reference in New Issue
Block a user