Archived
Compare commits
55
Commits
@@ -23,3 +23,5 @@ host-keys/
|
||||
# Temporary Milestone 1 audit checklist (remove-sensetive-info-refactor.md)
|
||||
# - working notes only, never committed, deleted once every row is rotated.
|
||||
secrets-inventory.md
|
||||
.claude/worktrees/
|
||||
.claude/settings.local.json
|
||||
+2
-2
@@ -22,8 +22,8 @@ keys:
|
||||
- &proxmox-pxe-boot age1ug787sgt6st6k82fgkrug2lzltw4qsukrrqqs3w27ewwqj8rg4hsxcmylz
|
||||
- &proxmox-server age1529taqdwr6t0w7cvzmty0d5y5593wffl0krt48j6uc4u39k56g2qf6ywtp
|
||||
- &proxmox-tailscale-router age1zhfyuzlq40reuqlr34gf77852nhs3t6mqfzrqmas8z6sxk7tcfhsungrm0
|
||||
- &proxmox-ha-server-1 age1nxlnrevqs2msdatze562vz6ym4pgydt2zndye83lwqauy6flggtqrevyw5
|
||||
- &proxmox-ha-server-2 age1scfc8p53q5aq2a87tcmsazmj8sfeft0s8kxg0et4nm3ucxyp3c3s6te82y
|
||||
- &proxmox-ha-server-1 age1k73g8x47hs93wcv7qh92n3htz8pl295g49hyvlrf3570mts0hgys5g04d6
|
||||
- &proxmox-ha-server-2 age1fefy6dk8zn5c3edwmrs9vwx79quftnt784m628t9e34q3ft3cehqz8u72r
|
||||
|
||||
creation_rules:
|
||||
# Shared across every currently-deployed host: root/nixos password hash,
|
||||
|
||||
@@ -21,9 +21,7 @@ machines when deployed.
|
||||
`modules/installer/common.nix` (the auto-installer's own root/nixos login —
|
||||
a deliberate, documented choice, see `docs/auto-installer.md`, not
|
||||
accidental tech debt) and **SSH public keys** in `variables.nix`
|
||||
(`vars.adminSshKey`, `vars.remoteBuilderAuthorizedKeys`) plus a couple of
|
||||
per-host `KEY` values for beszel-agent auth (`hosts/server/host.nix`,
|
||||
`hosts/nix-cache/host.nix`). Don't use the installer's hardcoded hash as a
|
||||
(`vars.adminSshKey`, `vars.remoteBuilderAuthorizedKeys`, `vars.beszelHubKey`). Don't use the installer's hardcoded hash as a
|
||||
template for a *real* host — every other host uses sops-nix
|
||||
(`hashedPasswordFile`, see "Security Notes" in `README.md`). Flag any *new*
|
||||
secret-like string you encounter instead of committing it.
|
||||
@@ -375,9 +373,8 @@ removing a host.
|
||||
|
||||
- `hosts/<name>/host.nix` — per-machine identity **only**: hostname, hostId,
|
||||
per-machine secrets, `system.stateVersion`. These files carry no `imports`
|
||||
of their own beyond narrow parameterized helpers (see
|
||||
`modules/beszel/host-token.nix` below) — all shared behavior comes from the
|
||||
platform/build-type modules composed in `flake.nix`, not from the host file.
|
||||
of their own — all shared behavior comes from the platform/build-type modules
|
||||
composed in `flake.nix`, not from the host file.
|
||||
- `modules/platforms/{linode,proxmox,lxc,baremetal}.nix` — platform-specific
|
||||
config: boot method, guest tooling, and the hardware config, imported
|
||||
directly by the platform module itself — **not** wired in from
|
||||
@@ -434,10 +431,10 @@ removing a host.
|
||||
substituter + SSH remote-builder wiring; see `docs/nix-cache.md` for the
|
||||
full design (per-host local stores, no shared `/nix/store`, and how the
|
||||
`nixremote` signing/SSH keys fit together).
|
||||
- `modules/beszel/host-token.nix` — parameterized helper module
|
||||
(`{ name, sopsFile }`) that wires a host's beszel-agent sops secret/template
|
||||
and `environmentFile`; used by `hosts/server/host.nix` and
|
||||
`hosts/nix-cache/host.nix` to avoid duplicating that boilerplate.
|
||||
- `modules/beszel/enable-agent.nix` — enables beszel-agent, sets `HUB_URL`,
|
||||
fixes the upstream `StateDirectory` bug, and wires the universal
|
||||
`beszel-token` sops secret (from `secrets/common.yaml`) into the agent's
|
||||
`environmentFile`; see `docs/beszel.md` for the full setup guide.
|
||||
- `modules/tailscale/`, `modules/docker/`, `modules/networking/`,
|
||||
`modules/traefik/`, `modules/tor/`, `modules/services/*` — single-purpose,
|
||||
single-host
|
||||
|
||||
+113
@@ -0,0 +1,113 @@
|
||||
# Beszel agent
|
||||
|
||||
[Beszel](https://github.com/henrygd/beszel) is the monitoring dashboard used
|
||||
in this LAN. The hub runs as a Docker container on `docker.sweet.home` (port
|
||||
`vars.ports.beszelHub`, 8090). Each monitored NixOS host runs a
|
||||
`beszel-agent` that connects back to the hub.
|
||||
|
||||
---
|
||||
|
||||
## How it works
|
||||
|
||||
Everything is handled by a single module:
|
||||
|
||||
**`modules/beszel/enable-agent.nix`** — imported by a build type. It:
|
||||
- Enables `beszel-agent`
|
||||
- Sets `HUB_URL` to `docker.sweet.home:8090`
|
||||
- Sets `KEY` from `vars.beszelHubKey` (`variables.nix`) — the hub's SSH
|
||||
public key, shared by every agent. Update `beszelHubKey` if the docker
|
||||
host is ever rebuilt and the hub generates a new keypair.
|
||||
- Reads the universal `beszel-token` from `secrets/common.yaml` via sops
|
||||
and passes it to the agent as `TOKEN` in an env file
|
||||
- Fixes an upstream bug where the agent couldn't persist its hub-pairing
|
||||
fingerprint across restarts (adds a real `StateDirectory`)
|
||||
|
||||
A host file needs no beszel configuration at all — just import the module
|
||||
in the build type and add the system in the hub UI.
|
||||
|
||||
---
|
||||
|
||||
## Adding beszel to a new build type
|
||||
|
||||
Add `../beszel/enable-agent.nix` to the `imports` list in
|
||||
`modules/build-types/<type>.nix`:
|
||||
|
||||
```nix
|
||||
imports = [
|
||||
../beszel/enable-agent.nix
|
||||
# ... other imports
|
||||
];
|
||||
```
|
||||
|
||||
That's the only change required. The host file needs nothing.
|
||||
|
||||
---
|
||||
|
||||
## Adding a new system to the hub
|
||||
|
||||
1. Rebuild and deploy the host with its build type importing `enable-agent.nix`.
|
||||
2. Open the beszel hub (`http://docker.sweet.home:8090`).
|
||||
3. Go to **Systems → Add system**, enter the host's IP and the default port
|
||||
(45876). The agent will connect and the system will appear as active.
|
||||
|
||||
---
|
||||
|
||||
## One-time setup: add the token to `secrets/common.yaml`
|
||||
|
||||
The universal token is stored once in the common secrets file, shared by all
|
||||
agents. Only needed once, not per-host:
|
||||
|
||||
```sh
|
||||
sops secrets/common.yaml
|
||||
```
|
||||
|
||||
Add:
|
||||
```yaml
|
||||
beszel-token: <token from the beszel hub Settings → Keys>
|
||||
```
|
||||
|
||||
`secrets/common.yaml` is already a sops recipient for every host via their
|
||||
SSH host keys, so no additional sops recipient setup is needed.
|
||||
|
||||
---
|
||||
|
||||
## Optional: monitoring extra filesystems
|
||||
|
||||
To report disk usage for a mount beyond the root filesystem, add
|
||||
`EXTRA_FILESYSTEMS` in the host file:
|
||||
|
||||
```nix
|
||||
services.beszel.agent.environment = {
|
||||
EXTRA_FILESYSTEMS = "/mnt/data"; # colon-separated for multiple paths
|
||||
};
|
||||
```
|
||||
|
||||
The `server` host uses this to expose its ZFS data pool:
|
||||
|
||||
```nix
|
||||
services.beszel.agent.environment = {
|
||||
EXTRA_FILESYSTEMS = "${vars.storageRoot}/${vars.nfsShares.dockerVolumes.subpath}";
|
||||
LOG_LEVEL = "debug";
|
||||
};
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Optional: monitoring Docker containers
|
||||
|
||||
`enable-agent.nix` has a commented-out line for Docker monitoring:
|
||||
|
||||
```nix
|
||||
#DOCKER_HOST = "tcp://docker-socket-proxy:2375";
|
||||
```
|
||||
|
||||
Uncomment it if the host runs docker-socket-proxy and you want per-container
|
||||
stats. Hosts without Docker should leave it commented out.
|
||||
|
||||
---
|
||||
|
||||
## If the hub key changes
|
||||
|
||||
If the docker host is ever rebuilt and beszel generates a new SSH keypair,
|
||||
update `beszelHubKey` in `variables.nix` and rebuild all beszel-enabled hosts.
|
||||
The new key is visible in the beszel hub under **Settings → Keys**.
|
||||
@@ -1,12 +1,5 @@
|
||||
{ vars, ... }:
|
||||
{
|
||||
imports = [
|
||||
(import ../../modules/beszel/host-token.nix {
|
||||
name = "ha-server-1";
|
||||
sopsFile = ../../secrets/ha-server-1.yaml;
|
||||
})
|
||||
];
|
||||
|
||||
networking = {
|
||||
hostName = vars.haServer1Host;
|
||||
hostId = "3a4b5c6d";
|
||||
@@ -23,8 +16,5 @@
|
||||
nameservers = [ vars.domainControllerIp ];
|
||||
};
|
||||
|
||||
# Set KEY after pairing this host with the beszel hub; the token is sops-managed.
|
||||
services.beszel.agent.environment.KEY = "";
|
||||
|
||||
system.stateVersion = "26.05";
|
||||
}
|
||||
|
||||
@@ -1,12 +1,5 @@
|
||||
{ vars, ... }:
|
||||
{
|
||||
imports = [
|
||||
(import ../../modules/beszel/host-token.nix {
|
||||
name = "ha-server-2";
|
||||
sopsFile = ../../secrets/ha-server-2.yaml;
|
||||
})
|
||||
];
|
||||
|
||||
networking = {
|
||||
hostName = vars.haServer2Host;
|
||||
hostId = "7e8f9a0b";
|
||||
@@ -23,8 +16,5 @@
|
||||
nameservers = [ vars.domainControllerIp ];
|
||||
};
|
||||
|
||||
# Set KEY after pairing this host with the beszel hub; the token is sops-managed.
|
||||
services.beszel.agent.environment.KEY = "";
|
||||
|
||||
system.stateVersion = "26.05";
|
||||
}
|
||||
|
||||
@@ -1,13 +1,6 @@
|
||||
{ vars, ... }:
|
||||
|
||||
{
|
||||
imports = [
|
||||
(import ../../modules/beszel/host-token.nix {
|
||||
name = "nix-cache";
|
||||
sopsFile = ../../secrets/nix-cache.yaml;
|
||||
})
|
||||
];
|
||||
|
||||
networking = {
|
||||
hostName = vars.nixCacheHost;
|
||||
useDHCP = false;
|
||||
@@ -19,10 +12,6 @@
|
||||
nameservers = [ vars.domainControllerIp ];
|
||||
};
|
||||
|
||||
services.beszel.agent.environment = {
|
||||
KEY = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFPR9kwtC4TAeTRu46A7+opZsYpxqkRJ+x/ZyB2GWCeG";
|
||||
};
|
||||
|
||||
# Preserved from the pre-refactor `nix-cache` target — stateVersion must
|
||||
# never be bumped on an already-installed machine.
|
||||
system.stateVersion = "25.05";
|
||||
|
||||
@@ -26,7 +26,7 @@
|
||||
|
||||
# Optional: set environment vars
|
||||
sessionVariables = {
|
||||
EDITOR = "vim";
|
||||
EDITOR = "nano";
|
||||
SOPS_AGE_KEY_FILE = "${config.home.homeDirectory}/.config/sops/age/keys.txt";
|
||||
};
|
||||
|
||||
|
||||
@@ -1,13 +1,6 @@
|
||||
{ vars, ... }:
|
||||
|
||||
{
|
||||
imports = [
|
||||
(import ../../modules/beszel/host-token.nix {
|
||||
name = "server";
|
||||
sopsFile = ../../secrets/server.yaml;
|
||||
})
|
||||
];
|
||||
|
||||
networking = {
|
||||
hostName = vars.nfsServerHost;
|
||||
hostId = "6689f93e";
|
||||
@@ -21,7 +14,6 @@
|
||||
};
|
||||
|
||||
services.beszel.agent.environment = {
|
||||
KEY = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFPR9kwtC4TAeTRu46A7+opZsYpxqkRJ+x/ZyB2GWCeG";
|
||||
EXTRA_FILESYSTEMS = "${vars.storageRoot}/${vars.nfsShares.dockerVolumes.subpath}";
|
||||
LOG_LEVEL = "debug";
|
||||
};
|
||||
|
||||
@@ -1,13 +1,6 @@
|
||||
{ vars, ... }:
|
||||
|
||||
{
|
||||
imports = [
|
||||
(import ../../modules/beszel/host-token.nix {
|
||||
name = "tailscale-router";
|
||||
sopsFile = ../../secrets/tailscale-router.yaml;
|
||||
})
|
||||
];
|
||||
|
||||
networking = {
|
||||
hostName = "tailscale-router";
|
||||
useDHCP = false;
|
||||
@@ -19,10 +12,6 @@
|
||||
nameservers = [ vars.domainControllerIp ];
|
||||
};
|
||||
|
||||
services.beszel.agent.environment = {
|
||||
KEY = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFPR9kwtC4TAeTRu46A7+opZsYpxqkRJ+x/ZyB2GWCeG";
|
||||
};
|
||||
|
||||
# No networking.hostId: only ZFS-touching hosts (server, docker) need one
|
||||
# for pool-import safety, and this host does neither.
|
||||
|
||||
|
||||
@@ -1,13 +1,6 @@
|
||||
{ vars, ... }:
|
||||
|
||||
{
|
||||
imports = [
|
||||
(import ../../modules/beszel/host-token.nix {
|
||||
name = "tor-relay";
|
||||
sopsFile = ../../secrets/tor-relay.yaml;
|
||||
})
|
||||
];
|
||||
|
||||
networking = {
|
||||
hostName = "tor-relay";
|
||||
useDHCP = false;
|
||||
@@ -22,10 +15,6 @@
|
||||
# No networking.hostId: only ZFS-touching hosts need one for pool-import
|
||||
# safety, and this host does neither.
|
||||
|
||||
services.beszel.agent.environment = {
|
||||
KEY = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFPR9kwtC4TAeTRu46A7+opZsYpxqkRJ+x/ZyB2GWCeG";
|
||||
};
|
||||
|
||||
# A genuinely new host (not a pre-refactor carry-over), so it tracks the
|
||||
# flake's current nixpkgs release rather than being pinned to an older one.
|
||||
system.stateVersion = "26.05";
|
||||
|
||||
@@ -1,10 +1,23 @@
|
||||
{ vars, ... }:
|
||||
{ config, vars, ... }:
|
||||
|
||||
{
|
||||
services.beszel.agent.enable = true;
|
||||
services.beszel.agent.environment = {
|
||||
# Universal token shared by all beszel agents. Add to secrets/common.yaml:
|
||||
# sops secrets/common.yaml
|
||||
# beszel-token: <value from the beszel hub UI>
|
||||
sops.secrets."beszel-token" = { };
|
||||
|
||||
sops.templates."beszel.env".content = ''
|
||||
TOKEN=${config.sops.placeholder."beszel-token"}
|
||||
'';
|
||||
|
||||
services.beszel.agent = {
|
||||
enable = true;
|
||||
environmentFile = config.sops.templates."beszel.env".path;
|
||||
environment = {
|
||||
#DOCKER_HOST = "tcp://docker-socket-proxy:2375";
|
||||
HUB_URL = "http://${vars.dockerHost}.${vars.homeDomain}:${toString vars.ports.beszelHub}";
|
||||
KEY = vars.beszelHubKey;
|
||||
};
|
||||
};
|
||||
|
||||
# The upstream module runs beszel-agent under DynamicUser with
|
||||
|
||||
@@ -1,11 +0,0 @@
|
||||
{ name, sopsFile }:
|
||||
|
||||
{ config, ... }:
|
||||
|
||||
{
|
||||
sops.secrets."beszel-token".sopsFile = sopsFile;
|
||||
sops.templates."${name}-beszel.env".content = ''
|
||||
TOKEN=${config.sops.placeholder."beszel-token"}
|
||||
'';
|
||||
services.beszel.agent.environmentFile = config.sops.templates."${name}-beszel.env".path;
|
||||
}
|
||||
@@ -81,4 +81,70 @@
|
||||
programs.firefox.enable = true;
|
||||
|
||||
nixpkgs.config.allowUnfree = true;
|
||||
|
||||
# GUI-specific Home Manager additions for the IPA primary user, extending
|
||||
# the baseline in modules/ipa/client.nix with desktop apps and services
|
||||
# that only make sense on a graphical workstation.
|
||||
home-manager.users.${vars.ipaUser} = { pkgs, ... }: {
|
||||
home = {
|
||||
packages = with pkgs; [
|
||||
git
|
||||
vim
|
||||
nextcloud-client
|
||||
chromium
|
||||
claude-code
|
||||
fish
|
||||
sops
|
||||
];
|
||||
sessionVariables = {
|
||||
EDITOR = "vim";
|
||||
SOPS_AGE_KEY_FILE = "/home/${vars.ipaUser}/.config/sops/age/keys.txt";
|
||||
};
|
||||
file = {
|
||||
".local/share/applications/proxmox-chromium-app.desktop".text = ''
|
||||
[Desktop Entry]
|
||||
Type=Application
|
||||
Name=Proxmox (Chromium)
|
||||
Exec=chromium --app=https://pve.${vars.homeDomain}:${toString vars.ports.pveWeb} --window-size=1920,1080 --window-position=0,0
|
||||
Icon=/home/${vars.ipaUser}/.local/share/icons/proxmox.png
|
||||
Terminal=false
|
||||
Categories=Hypervisor;
|
||||
StartupWMClass=PVE
|
||||
'';
|
||||
".local/share/applications/pbs-chromium-app.desktop".text = ''
|
||||
[Desktop Entry]
|
||||
Type=Application
|
||||
Name=Proxmox Backup Server (Chromium)
|
||||
Exec=chromium --app=https://${vars.pbsIp}:${toString vars.ports.pbsWeb} --window-size=1920,1080 --window-position=0,0
|
||||
Icon=/home/${vars.ipaUser}/.local/share/icons/proxmox.png
|
||||
Terminal=false
|
||||
Categories=backup;
|
||||
'';
|
||||
".local/share/applications/proxmox-firefox-app.desktop".text = ''
|
||||
[Desktop Entry]
|
||||
Type=Application
|
||||
Name=Proxmox (Firefox)
|
||||
Exec=firefox --new-instance https://pve.${vars.homeDomain}:${toString vars.ports.pveWeb} --profile ProxmoxWebApp --window-size=1920,1080 --class ProxmoxWebApp
|
||||
Icon=/home/${vars.ipaUser}/.local/share/icons/proxmox.png
|
||||
Terminal=false
|
||||
Categories=Hypervisor;
|
||||
StartupWMClass=PVE
|
||||
'';
|
||||
".local/share/applications/pbs-firefox-app.desktop".text = ''
|
||||
[Desktop Entry]
|
||||
Type=Application
|
||||
Name=Proxmox Backup Server (Firefox)
|
||||
Exec=firefox --new-window https://${vars.pbsIp}:${toString vars.ports.pbsWeb} --profile PbsWebApp --window-size=1920,1080 --class PbsWebApp
|
||||
Icon=/home/${vars.ipaUser}/.local/share/icons/proxmox.png
|
||||
Terminal=false
|
||||
Categories=backup;
|
||||
StartupWMClass=PBS
|
||||
'';
|
||||
};
|
||||
};
|
||||
services.nextcloud-client = {
|
||||
enable = true;
|
||||
startInBackground = true;
|
||||
};
|
||||
};
|
||||
}
|
||||
|
||||
@@ -30,7 +30,7 @@ in
|
||||
../ha/pacemaker-stack.nix
|
||||
../ha/iscsi-target.nix
|
||||
../ha/cluster-config.nix
|
||||
../beszel/enable-agent.nix
|
||||
# ../beszel/enable-agent.nix
|
||||
];
|
||||
|
||||
# xfsprogs: mkfs.xfs/xfs_info needed by cluster-init.sh.
|
||||
@@ -45,4 +45,7 @@ in
|
||||
# Pacemaker controls nfs-server — prevent systemd from starting it at boot
|
||||
# on both nodes (only the Active node should be serving NFS).
|
||||
systemd.services.nfs-server.wantedBy = lib.mkForce [ ];
|
||||
|
||||
# Same reason as server.nix: exports use standard auth, not Kerberos.
|
||||
systemd.services.rpc-svcgssd.enable = false;
|
||||
}
|
||||
|
||||
@@ -102,6 +102,12 @@ in
|
||||
requires = [ "zfs-mount.service" ];
|
||||
};
|
||||
|
||||
# rpc-svcgssd handles Kerberos/GSS-API for NFS. Not needed: exports use
|
||||
# standard auth, not sec=krb5. On IPA-joined hosts the keytab exists (host/
|
||||
# principal only) but has no nfs/ principal, causing spurious failure.
|
||||
# Mask it so nfs-server's Wants= can't pull it in.
|
||||
systemd.services.rpc-svcgssd.enable = false;
|
||||
|
||||
services.nfs.server = {
|
||||
enable = true;
|
||||
exports = mkNfsExports vars.storageRoot;
|
||||
|
||||
@@ -1,5 +1,8 @@
|
||||
{ lib, pkgs, vars, ... }:
|
||||
|
||||
let
|
||||
gid = toString vars.dockerAccessGid;
|
||||
in
|
||||
{
|
||||
virtualisation.docker = {
|
||||
enable = true;
|
||||
@@ -16,4 +19,27 @@
|
||||
docker-compose
|
||||
docker-buildx
|
||||
];
|
||||
|
||||
# NixOS's group activation uses plain `groupmod` without --non-unique.
|
||||
# When SSSD is active it exposes the IPA "docker-access" group at
|
||||
# vars.dockerAccessGid via NSS, so groupmod sees that GID as already in
|
||||
# use and silently skips the change (warning: "not applying GID change").
|
||||
# This script runs after the normal "groups" step and applies the change
|
||||
# with --non-unique (which lets the local docker group share the GID with
|
||||
# the SSSD-provided IPA group). If the GID actually changed it also
|
||||
# restarts docker.socket so the socket is recreated with the new GID.
|
||||
system.activationScripts.docker-group-gid = {
|
||||
deps = [ "groups" ];
|
||||
text = ''
|
||||
current=$(grep "^docker:" /etc/group | cut -d: -f3)
|
||||
if [ "$current" != "${gid}" ]; then
|
||||
${pkgs.shadow}/bin/groupmod --non-unique -g ${gid} docker
|
||||
if ${pkgs.systemd}/bin/systemctl is-active --quiet docker.socket; then
|
||||
${pkgs.systemd}/bin/systemctl stop docker.service docker.socket
|
||||
rm -f /var/run/docker.sock
|
||||
${pkgs.systemd}/bin/systemctl start docker.socket docker.service
|
||||
fi
|
||||
fi
|
||||
'';
|
||||
};
|
||||
}
|
||||
|
||||
@@ -21,7 +21,26 @@
|
||||
# STONITH device during the testing phase. Switch to resource-and-stonith
|
||||
# once the fence_pve_ssh STONITH resource is active (see
|
||||
# scripts/ha/cluster-enable-stonith.sh).
|
||||
{ lib, vars, ... }:
|
||||
#
|
||||
# PATH wrapper: when the DRBD kernel module invokes the fence-peer handler
|
||||
# via the UMH (User Mode Helper) mechanism it provides a minimal PATH that
|
||||
# omits /run/current-system/sw/bin. crm-fence-peer.sh calls cibadmin,
|
||||
# crm_mon etc.; if those aren't found a pipeline in the script breaks with
|
||||
# SIGPIPE. A process killed by signal has WEXITSTATUS() == 0, so the kernel
|
||||
# sees exit code 0 and logs "fence-peer helper broken, returned 0", looping
|
||||
# forever. The writeShellScript wrappers below prepend the NixOS sw path
|
||||
# before exec-ing the real handler, giving it a working Pacemaker toolchain.
|
||||
{ lib, pkgs, vars, ... }:
|
||||
let
|
||||
fencePeerWrapper = pkgs.writeShellScript "drbd-fence-peer" ''
|
||||
export PATH="/run/current-system/sw/bin:/run/current-system/sw/sbin:$PATH"
|
||||
exec /run/current-system/sw/lib/drbd/crm-fence-peer.sh "$@"
|
||||
'';
|
||||
unfencePeerWrapper = pkgs.writeShellScript "drbd-unfence-peer" ''
|
||||
export PATH="/run/current-system/sw/bin:/run/current-system/sw/sbin:$PATH"
|
||||
exec /run/current-system/sw/lib/drbd/crm-unfence-peer.sh "$@"
|
||||
'';
|
||||
in
|
||||
{
|
||||
# Root SSH access — same key set as nixos user so all admin keys can reach root.
|
||||
users.users.root.openssh.authorizedKeys.keys = [
|
||||
@@ -37,6 +56,12 @@
|
||||
# DRBD lock-file directory (drbd-utils checks for it; missing → harmless but noisy warnings).
|
||||
systemd.tmpfiles.rules = [ "d /var/lib/drbd 0750 root root -" ];
|
||||
|
||||
# Prevent drbd.service from auto-starting at boot / nixos-rebuild switch.
|
||||
# Pacemaker's OCF drbd agent calls drbdadm up/down directly when managing
|
||||
# the resource. If drbd.service also runs drbdadm up all while DRBD is
|
||||
# already Primary under Pacemaker, apply-al fails with "device busy" (exit 20).
|
||||
systemd.services.drbd.wantedBy = lib.mkForce [];
|
||||
|
||||
services.drbd = {
|
||||
enable = true;
|
||||
config = ''
|
||||
@@ -56,15 +81,15 @@
|
||||
fencing resource-only;
|
||||
}
|
||||
handlers {
|
||||
fence-peer "/run/current-system/sw/lib/drbd/crm-fence-peer.sh";
|
||||
unfence-peer "/run/current-system/sw/lib/drbd/crm-unfence-peer.sh";
|
||||
fence-peer "${fencePeerWrapper}";
|
||||
unfence-peer "${unfencePeerWrapper}";
|
||||
}
|
||||
}
|
||||
|
||||
resource ha-data {
|
||||
volume 0 {
|
||||
device /dev/drbd0;
|
||||
disk /dev/sdb;
|
||||
disk ${vars.haServerDrbdDisk};
|
||||
meta-disk internal;
|
||||
}
|
||||
|
||||
|
||||
+17
-1
@@ -60,6 +60,15 @@ lib.mkIf enabled {
|
||||
pam.services = {
|
||||
sshd.makeHomeDir = true;
|
||||
lightdm.makeHomeDir = true;
|
||||
|
||||
# pam_unix returns PAM_AUTHINFO_UNAVAIL without prompting when the local
|
||||
# stub has "!" in shadow (account locked), so PAM_AUTHTOK is never set
|
||||
# and pam_sss's use_first_pass fails with "No authentication token".
|
||||
# Changing to try_first_pass makes pam_sss prompt independently when no
|
||||
# prior module has set the token, restoring IPA password login via
|
||||
# LightDM and su.
|
||||
login.rules.auth.sss.settings = lib.mkForce { try_first_pass = true; };
|
||||
su.rules.auth.sss.settings = lib.mkForce { try_first_pass = true; };
|
||||
};
|
||||
|
||||
# HM with useUserPackages = true (flake.nix) sets users.users.${ipaUser}.packages,
|
||||
@@ -170,6 +179,13 @@ lib.mkIf enabled {
|
||||
group = "users";
|
||||
extraGroups = [ "wheel" ];
|
||||
createHome = false;
|
||||
# "!" is not a password hash — it is the standard "account locked" marker.
|
||||
# It cannot authenticate anyone locally. It exists solely so NixOS generates
|
||||
# a shadow entry for this stub user; without one pam_unix returns
|
||||
# PAM_AUTHINFO_UNAVAIL before prompting, which means PAM_AUTHTOK is never
|
||||
# set and the subsequent pam_sss use_first_pass call has nothing to work
|
||||
# with — blocking LightDM and su logins even when IPA/SSSD auth succeeds.
|
||||
hashedPassword = "!";
|
||||
};
|
||||
|
||||
# Home Manager config for the IPA primary user, applied on every enrolled
|
||||
@@ -186,7 +202,7 @@ lib.mkIf enabled {
|
||||
homeDirectory = lib.mkForce "/home/${vars.ipaUser}";
|
||||
stateVersion = "26.05";
|
||||
packages = with pkgs; [ tmux sshfs ];
|
||||
sessionVariables.EDITOR = "nano";
|
||||
sessionVariables.EDITOR = lib.mkDefault "nano";
|
||||
};
|
||||
programs.home-manager.enable = true;
|
||||
programs.bash.enable = true;
|
||||
|
||||
Binary file not shown.
Regular → Executable
+46
-9
@@ -34,18 +34,55 @@ echo "════════════════════════
|
||||
echo " HA Cluster Acceptance Tests — $(date '+%Y-%m-%d %H:%M:%S')"
|
||||
echo "════════════════════════════════════════════════════"
|
||||
|
||||
# ── Detect Active/Standby nodes ────────────────────────────────────────────
|
||||
# Pacemaker can promote either node; determine which is currently Active
|
||||
# (DRBD Primary / holds ha-group resources) before running tests.
|
||||
# ── Pre-flight: DRBD sync must be complete ────────────────────────────────
|
||||
# Tests that check disk state, XFS mount, and iSCSI will fail or give false
|
||||
# results while the initial full sync is in progress. Block until done.
|
||||
echo ""
|
||||
echo "Detecting Active/Standby nodes..."
|
||||
if n1 "drbdadm role ha-data" 2>/dev/null | grep -q "^Primary"; then
|
||||
ACTIVE_NODE="$NODE1"; ACTIVE_IP="$NODE1_IP"
|
||||
echo "Pre-flight: verifying DRBD sync is complete..."
|
||||
DRBD_PREFLIGHT=$(n1 "drbdadm dstate ha-data 2>/dev/null" 2>/dev/null || echo "unknown")
|
||||
if ! echo "$DRBD_PREFLIGHT" | grep -q "^UpToDate/UpToDate$"; then
|
||||
echo ""
|
||||
echo " ERROR: DRBD initial sync not complete."
|
||||
echo " Current dstate on $NODE1: $DRBD_PREFLIGHT"
|
||||
echo ""
|
||||
echo " Monitor progress:"
|
||||
echo " ssh nixos@$NODE1_IP 'sudo watch -n3 cat /proc/drbd'"
|
||||
echo ""
|
||||
echo " Re-run this script once dstate shows UpToDate/UpToDate."
|
||||
exit 1
|
||||
fi
|
||||
echo " dstate: $DRBD_PREFLIGHT — ready."
|
||||
|
||||
# ── Detect Active/Standby nodes ────────────────────────────────────────────
|
||||
# Use crm_mon to detect which node holds the Promoted (Primary) DRBD resource.
|
||||
# Pacemaker is authoritative; DRBD role can briefly read as Secondary while
|
||||
# Pacemaker is mid-transition, giving a false Active/Standby swap.
|
||||
# Wait up to 90 s for Pacemaker to settle before giving up.
|
||||
echo ""
|
||||
echo "Detecting Active/Standby nodes (waiting for Pacemaker to settle)..."
|
||||
ACTIVE_NODE=""
|
||||
for i in $(seq 1 30); do
|
||||
# crm_mon -1 output contains "Promoted: [ <node> ]" for the DRBD master.
|
||||
CRM_OUT=$(n1 "crm_mon -1" 2>/dev/null || n2 "crm_mon -1" 2>/dev/null || true)
|
||||
# crm_mon 2.x formats Promoted lines as " * Promoted: [ node ]" — the * bullet
|
||||
# means ^\s*(Promoted|Masters): never matches; filter Unpromoted first instead.
|
||||
ACTIVE_NODE=$(echo "$CRM_OUT" | grep -v 'Unpromoted\|Unmanaged' | grep -E '(Promoted|Masters):' | grep -oE '\b(ha-server-[0-9]+)\b' | head -1 || true)
|
||||
[[ -n "$ACTIVE_NODE" ]] && break
|
||||
sleep 3
|
||||
done
|
||||
|
||||
if [[ -z "$ACTIVE_NODE" ]]; then
|
||||
echo " WARNING: could not determine Active node from crm_mon after 90 s — defaulting to $NODE1"
|
||||
ACTIVE_NODE="$NODE1"
|
||||
fi
|
||||
|
||||
if [[ "$ACTIVE_NODE" == "$NODE1" ]]; then
|
||||
ACTIVE_IP="$NODE1_IP"
|
||||
STANDBY_NODE="$NODE2"; STANDBY_IP="$NODE2_IP"
|
||||
na() { n1 "$@"; }
|
||||
ns() { n2 "$@"; }
|
||||
else
|
||||
ACTIVE_NODE="$NODE2"; ACTIVE_IP="$NODE2_IP"
|
||||
ACTIVE_IP="$NODE2_IP"
|
||||
STANDBY_NODE="$NODE1"; STANDBY_IP="$NODE1_IP"
|
||||
na() { n2 "$@"; }
|
||||
ns() { n1 "$@"; }
|
||||
@@ -97,7 +134,7 @@ fi
|
||||
# ── T4: iSCSI target visible on Active node ───────────────────────────────
|
||||
echo ""
|
||||
echo "[T4] iSCSI target"
|
||||
IQN_COUNT=$(na "ls /sys/kernel/config/target/iscsi/ 2>/dev/null | grep -c iqn" || echo "0")
|
||||
IQN_COUNT=$(na "bash -c 'ls /sys/kernel/config/target/iscsi/ 2>/dev/null | grep -c iqn || true'" 2>/dev/null || echo "0")
|
||||
if [[ "$IQN_COUNT" -ge 1 ]]; then
|
||||
pass "iSCSI IQN active on $ACTIVE_NODE ($IQN_COUNT target(s))"
|
||||
else
|
||||
@@ -130,7 +167,7 @@ done
|
||||
|
||||
if $MOVED; then
|
||||
pass "XFS mounted on $STANDBY_NODE after failover"
|
||||
IQN_ON_STANDBY=$(ns "ls /sys/kernel/config/target/iscsi/ 2>/dev/null | grep -c iqn" || echo "0")
|
||||
IQN_ON_STANDBY=$(ns "bash -c 'ls /sys/kernel/config/target/iscsi/ 2>/dev/null | grep -c iqn || true'" 2>/dev/null || echo "0")
|
||||
[[ "$IQN_ON_STANDBY" -ge 1 ]] \
|
||||
&& pass "iSCSI target active on $STANDBY_NODE after failover" \
|
||||
|| fail "iSCSI target NOT active on $STANDBY_NODE after failover"
|
||||
|
||||
Regular → Executable
+146
-13
@@ -32,6 +32,11 @@ ISCSI_IQN="${ISCSI_IQN:-iqn.2026-01.home.sweet:ha-storage}" # vars.haIscsiIqn
|
||||
ISCSI_LUN_FILE="${XFS_MOUNT}/iscsi-lun.img"
|
||||
ISCSI_LUN_SIZE="10G"
|
||||
DRBD_DEVICE="/dev/drbd0"
|
||||
# DRBD backing disk — by-id path that resolves correctly on both nodes
|
||||
# regardless of whether the OS-level name is sda or sdb (Proxmox VM disk
|
||||
# ordering is not guaranteed). Matches haServerDrbdDisk in variables.nix.
|
||||
# Override DRBD_DISK if your hardware uses a different controller/slot path.
|
||||
DRBD_DISK="${DRBD_DISK:-/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi1}"
|
||||
VMID_NODE1="${VMID_NODE1:-}" # set by deploy.sh; needed for STONITH
|
||||
VMID_NODE2="${VMID_NODE2:-}"
|
||||
PVE_HOST="${PVE_HOST:-pve1.sweet.home}"
|
||||
@@ -72,6 +77,18 @@ if ! command -v mkfs.xfs &>/dev/null; then
|
||||
|| die "mkfs.xfs not found — add xfsprogs to ha-server.nix environment.systemPackages and rebuild"
|
||||
fi
|
||||
|
||||
# drbdmeta lives alongside drbdadm but may not be in PATH when run via sudo.
|
||||
if ! command -v drbdmeta &>/dev/null; then
|
||||
_drbd_bin=$(dirname "$(command -v drbdadm)" 2>/dev/null || true)
|
||||
[[ -n "$_drbd_bin" ]] && export PATH="$_drbd_bin:$PATH" \
|
||||
|| die "drbdmeta not found — is drbd-utils in ha-server environment.systemPackages?"
|
||||
fi
|
||||
|
||||
# Portable 16-hex-char UUID generator (no openssl required).
|
||||
_rand_uuid() {
|
||||
cat /proc/sys/kernel/random/uuid 2>/dev/null | tr -d '-' | cut -c1-16 | tr '[:lower:]' '[:upper:]'
|
||||
}
|
||||
|
||||
# Inter-node SSH/SCP helpers — abstract over root-to-root vs nixos+sudo.
|
||||
_SSH_OPTS="-o StrictHostKeyChecking=no -o ConnectTimeout=10"
|
||||
[[ -n "$HA_KEY" ]] && _SSH_OPTS="-i $HA_KEY $_SSH_OPTS"
|
||||
@@ -103,11 +120,16 @@ n2_ssh "mkdir -p /etc/corosync"
|
||||
n2_scp "$AUTHKEY" "$AUTHKEY"
|
||||
n2_ssh "chmod 0400 '${AUTHKEY}'"
|
||||
|
||||
log "Restarting corosync on both nodes..."
|
||||
log "Restarting corosync and pacemaker on both nodes..."
|
||||
systemctl restart corosync
|
||||
n2_ssh "systemctl restart corosync"
|
||||
sleep 3
|
||||
|
||||
log "Starting pacemaker on both nodes (may have failed at boot before authkey was placed)..."
|
||||
systemctl start pacemaker 2>/dev/null || systemctl restart pacemaker 2>/dev/null || true
|
||||
n2_ssh "systemctl start pacemaker 2>/dev/null || systemctl restart pacemaker 2>/dev/null || true"
|
||||
sleep 2
|
||||
|
||||
# ── 1. Corosync quorum ────────────────────────────────────────────────────
|
||||
log "Waiting for corosync quorum..."
|
||||
for i in $(seq 1 30); do
|
||||
@@ -130,14 +152,91 @@ for i in $(seq 1 30); do
|
||||
done
|
||||
|
||||
# ── 2. DRBD initialisation ────────────────────────────────────────────────
|
||||
# Put both nodes in Pacemaker standby first so it stops managed resources
|
||||
# cleanly, then enable maintenance-mode so Pacemaker's monitor operations are
|
||||
# suspended. Without maintenance-mode, Pacemaker keeps monitoring: when it
|
||||
# sees DRBD Primary on a standby node (that it didn't start), it triggers a
|
||||
# stop action — killing the initial sync after ~10 s. Maintenance-mode
|
||||
# disables all start/stop/monitor actions for the duration of the sync; it is
|
||||
# cleared after UpToDate/UpToDate is confirmed.
|
||||
log "Setting both nodes to Pacemaker standby for DRBD metadata init..."
|
||||
crm_standby -N "$NODE1" -v on 2>/dev/null || true
|
||||
crm_standby -N "$NODE2" -v on 2>/dev/null || true
|
||||
|
||||
# Wait for Pacemaker to actually stop DRBD (if it was managing it).
|
||||
log "Waiting for DRBD to stop under Pacemaker control..."
|
||||
for i in $(seq 1 30); do
|
||||
n1_role=$(drbdadm role ha-data 2>/dev/null || echo "Unconfigured")
|
||||
n2_role=$(n2_ssh "drbdadm role ha-data 2>/dev/null" 2>/dev/null || echo "Unconfigured")
|
||||
if [[ "$n1_role" == "Unconfigured" ]] && [[ "$n2_role" == "Unconfigured" ]]; then
|
||||
log "DRBD stopped on both nodes"
|
||||
break
|
||||
fi
|
||||
[[ $i -eq 30 ]] && warn "DRBD still active after 60s standby — forcing down anyway"
|
||||
sleep 2
|
||||
done
|
||||
|
||||
log "Enabling Pacemaker maintenance-mode (suspends monitor/start/stop during sync)..."
|
||||
crm_attribute -t crm_config -n maintenance-mode -v true 2>/dev/null || true
|
||||
|
||||
log "Detaching DRBD on $NODE1 (belt-and-suspenders after standby)..."
|
||||
drbdadm down ha-data 2>/dev/null || true
|
||||
log "Detaching DRBD on $NODE2..."
|
||||
n2_ssh "drbdadm down ha-data 2>/dev/null || true"
|
||||
sleep 2
|
||||
|
||||
# Ensure /etc/drbd.conf on both nodes points to DRBD_DISK (the stable by-id
|
||||
# path). VMs built before this fix may have /dev/sda or /dev/sdb hardcoded.
|
||||
# NixOS makes /etc/drbd.conf a symlink into the read-only Nix store, so
|
||||
# sed -i on the symlink target would fail — we break the symlink first with
|
||||
# cp --remove-destination, creating a regular writable copy.
|
||||
# Rebuild+redeploy (--force-rebuild) to make this permanent.
|
||||
_PATCH_DRBD=$(mktemp)
|
||||
cat > "$_PATCH_DRBD" << 'PATCHEOF'
|
||||
#!/bin/bash
|
||||
WANT="$1"
|
||||
conf=/etc/drbd.conf
|
||||
if [[ -L "$conf" ]]; then
|
||||
cp --remove-destination "$(readlink -f "$conf")" "$conf"
|
||||
fi
|
||||
cur=$(drbdadm sh-ll-dev ha-data 2>/dev/null | head -1 || true)
|
||||
if [[ -n "$cur" && "$cur" != "$WANT" ]]; then
|
||||
echo "[cluster-init] WARNING: patching $conf: $cur → $WANT (rebuild to make permanent)"
|
||||
sed -i "s,${cur},${WANT},g" "$conf"
|
||||
fi
|
||||
PATCHEOF
|
||||
chmod +x "$_PATCH_DRBD"
|
||||
bash "$_PATCH_DRBD" "$DRBD_DISK"
|
||||
n2_scp "$_PATCH_DRBD" "/tmp/patch-drbd-disk.sh"
|
||||
n2_ssh "bash /tmp/patch-drbd-disk.sh ${DRBD_DISK}"
|
||||
n2_ssh "rm -f /tmp/patch-drbd-disk.sh"
|
||||
rm -f "$_PATCH_DRBD"
|
||||
|
||||
log "Initialising DRBD metadata on $NODE1..."
|
||||
if ! drbdadm dstate ha-data 2>/dev/null | grep -q "UpToDate\|Inconsistent\|Diskless"; then
|
||||
drbdadm create-md ha-data --force
|
||||
# Use drbdmeta --force directly for BOTH create-md and write-dev-uuid.
|
||||
# drbdadm create-md --force passes --force to drbdmeta create-md but NOT to
|
||||
# the write-dev-uuid sub-call it makes internally, so write-dev-uuid fails when
|
||||
# the backing disk is still busy and stdin is not a TTY:
|
||||
# "stdin not a TTY, not waiting for confirmation" → exit 20.
|
||||
# Calling drbdmeta --force directly bypasses the exclusive-open confirmation on
|
||||
# both steps without needing a TTY, regardless of whether the device is busy.
|
||||
# Skip metadata creation only if DRBD is UP and fully synced (UpToDate/UpToDate).
|
||||
# When the resource is down, drbdadm dstate reads metadata and returns just
|
||||
# "UpToDate" (no slash) — that must not be treated as "already synced".
|
||||
# Mismatched UUIDs from an interrupted sync cause instant WFConnection→StandAlone,
|
||||
# so we always recreate metadata unless the sync is genuinely complete.
|
||||
if [[ "$(drbdadm dstate ha-data 2>/dev/null)" != "UpToDate/UpToDate" ]]; then
|
||||
UUID1=$(_rand_uuid)
|
||||
drbdmeta --force 0 v08 "${DRBD_DISK}" internal create-md
|
||||
drbdmeta --force 0 v08 "${DRBD_DISK}" internal write-dev-uuid "$UUID1"
|
||||
fi
|
||||
|
||||
log "Initialising DRBD metadata on $NODE2..."
|
||||
# Use grep -E for ERE alternation inside the remote bash -c string (avoids \| quoting issues).
|
||||
n2_ssh "bash -c 'drbdadm dstate ha-data 2>/dev/null | grep -qE \"UpToDate|Inconsistent|Diskless\" || drbdadm create-md ha-data --force'"
|
||||
if [[ "$(n2_ssh "drbdadm dstate ha-data 2>/dev/null" 2>/dev/null)" != "UpToDate/UpToDate" ]]; then
|
||||
UUID2=$(n2_ssh "cat /proc/sys/kernel/random/uuid 2>/dev/null | tr -d '-' | cut -c1-16 | tr '[:lower:]' '[:upper:]'")
|
||||
n2_ssh "drbdmeta --force 0 v08 ${DRBD_DISK} internal create-md"
|
||||
n2_ssh "drbdmeta --force 0 v08 ${DRBD_DISK} internal write-dev-uuid ${UUID2}"
|
||||
fi
|
||||
|
||||
log "Bringing up DRBD on both nodes..."
|
||||
drbdadm up ha-data 2>/dev/null || true
|
||||
@@ -145,18 +244,49 @@ n2_ssh "drbdadm up ha-data" 2>/dev/null || true
|
||||
|
||||
log "Forcing $NODE1 to DRBD Primary for initial sync..."
|
||||
drbdadm primary ha-data --force
|
||||
# NOTE: Pacemaker standby is intentionally kept ON until after the sync
|
||||
# completes. Clearing it here races with the OCF DRBD agent: Pacemaker
|
||||
# sees DRBD in WFConnection/SyncSource and may call drbdadm-down thinking
|
||||
# something went wrong, killing the sync. Standby is cleared below, after
|
||||
# UpToDate/UpToDate is confirmed.
|
||||
|
||||
log "Waiting for DRBD to finish initial sync (this may take several minutes)..."
|
||||
for i in $(seq 1 300); do
|
||||
state=$(drbdadm dstate ha-data 2>/dev/null || echo "unknown")
|
||||
if echo "$state" | grep -q "UpToDate/UpToDate"; then
|
||||
log "DRBD sync complete: $state"
|
||||
log "Waiting for DRBD initial sync to complete (32 GB may take 10–20 min)..."
|
||||
log " (monitor with: watch -n3 cat /proc/drbd)"
|
||||
_sync_chars=('|' '/' '-' $'\\')
|
||||
_sync_iter=0
|
||||
while true; do
|
||||
_dstate=$(drbdadm dstate ha-data 2>/dev/null || echo "unknown")
|
||||
if echo "$_dstate" | grep -q "UpToDate/UpToDate"; then
|
||||
printf "\r%-80s\r" ""
|
||||
log "DRBD initial sync complete (dstate: $_dstate)"
|
||||
break
|
||||
fi
|
||||
[[ $i -eq 300 ]] && warn "DRBD not UpToDate after 300 s — continuing anyway (check drbdadm status)"
|
||||
sleep 1
|
||||
# Parse connection state from /proc/drbd (cs:SyncSource, cs:Connected, cs:StandAlone …)
|
||||
_cs=$(grep -oE 'cs:[A-Za-z]+' /proc/drbd 2>/dev/null | head -1 | sed 's/cs://' || echo "unknown")
|
||||
# /proc/drbd uses variable whitespace: "sync'ed: 5.2%" (two spaces).
|
||||
_pct=$(grep -oE "sync'ed:[[:space:]]+[0-9.]+" /proc/drbd 2>/dev/null | grep -oE "[0-9.]+" | head -1 || echo "")
|
||||
_eta=$(grep -oE "finish:[[:space:]]+[0-9:]+" /proc/drbd 2>/dev/null | grep -oE "[0-9:]+$" | head -1 || echo "")
|
||||
_spd=$(grep -oE "speed:[[:space:]]+[0-9,]+" /proc/drbd 2>/dev/null | grep -oE "[0-9,]+$" | head -1 || echo "")
|
||||
_sync_iter=$(( _sync_iter + 1 ))
|
||||
_sc="${_sync_chars[$_sync_iter % 4]}"
|
||||
if [[ "$_cs" == "StandAlone" && $_sync_iter -gt 5 ]]; then
|
||||
printf "\r%-80s\r" ""
|
||||
die "DRBD is StandAlone after 15 s — peer connection lost (dstate: $_dstate). " \
|
||||
"Check corosync/network and re-run cluster-init."
|
||||
elif [[ -n "$_pct" ]]; then
|
||||
printf "\r [%s] syncing: %s%% done — ETA %s @ %s K/s " \
|
||||
"$_sc" "$_pct" "${_eta:-??:??:??}" "${_spd:-?}"
|
||||
else
|
||||
printf "\r [%s] cs:%s dstate:%s — waiting for sync to start " "$_sc" "$_cs" "$_dstate"
|
||||
fi
|
||||
sleep 3
|
||||
done
|
||||
|
||||
log "Disabling Pacemaker maintenance-mode and clearing standby — handing DRBD back to Pacemaker..."
|
||||
crm_attribute -t crm_config -n maintenance-mode -v false 2>/dev/null || true
|
||||
crm_standby -N "$NODE1" -v off 2>/dev/null || true
|
||||
crm_standby -N "$NODE2" -v off 2>/dev/null || true
|
||||
|
||||
# ── 3. XFS filesystem ─────────────────────────────────────────────────────
|
||||
log "Creating XFS on ${DRBD_DEVICE}..."
|
||||
if ! xfs_info "${DRBD_DEVICE}" &>/dev/null; then
|
||||
@@ -218,7 +348,7 @@ log "Unmounting ${XFS_MOUNT} — Pacemaker manages it..."
|
||||
umount "${XFS_MOUNT}" || { sync; umount -l "${XFS_MOUNT}"; }
|
||||
|
||||
log "Demoting DRBD to Secondary — Pacemaker manages primary role..."
|
||||
[[ "$(drbdadm role ha-data 2>/dev/null)" == "Primary/Secondary" ]] && drbdadm secondary ha-data || true
|
||||
drbdadm role ha-data 2>/dev/null | grep -q "^Primary" && drbdadm secondary ha-data || true
|
||||
|
||||
# ── 7. Pacemaker resources ────────────────────────────────────────────────
|
||||
log "Configuring Pacemaker cluster properties..."
|
||||
@@ -306,6 +436,9 @@ cibadmin --replace --scope constraints --xml-text '<constraints>
|
||||
<rsc_colocation id="coloc-group-with-drbd" score="INFINITY" rsc="ha-group" with-rsc="ms-drbd0" with-rsc-role="Promoted"/>
|
||||
</constraints>'
|
||||
|
||||
log "Clearing stale Pacemaker failure history..."
|
||||
crm_resource --cleanup 2>/dev/null || true
|
||||
|
||||
log "Waiting for resources to start..."
|
||||
for i in $(seq 1 60); do
|
||||
if crm_resource -r vip --locate 2>/dev/null | grep -q "running on"; then
|
||||
|
||||
+93
-56
@@ -32,6 +32,7 @@
|
||||
# --skip-create-vms Skip VM creation (VMs already exist)
|
||||
# --skip-add-hardware Skip net1/scsi1 attachment (already attached)
|
||||
# --skip-boot-wait Skip boot/SSH wait (VMs already running)
|
||||
# --skip-refresh-sops-keys Skip scanning running VMs for fresh SSH host keys
|
||||
# --skip-cluster-init Skip cluster formation (cluster already configured)
|
||||
# --skip-tests Skip acceptance tests
|
||||
# --force-rebuild Pass --force-rebuild to create-proxmox-resource.sh
|
||||
@@ -70,6 +71,7 @@ SKIP_SYNC_KEYS=false
|
||||
SKIP_CREATE_VMS=false
|
||||
SKIP_ADD_HARDWARE=false
|
||||
SKIP_BOOT_WAIT=false
|
||||
SKIP_REFRESH_SOPS_KEYS=false
|
||||
SKIP_CLUSTER_INIT=false
|
||||
SKIP_TESTS=false
|
||||
FORCE_REBUILD=false
|
||||
@@ -109,6 +111,7 @@ while [[ $# -gt 0 ]]; do
|
||||
--skip-create-vms) SKIP_CREATE_VMS=true; shift ;;
|
||||
--skip-add-hardware) SKIP_ADD_HARDWARE=true; shift ;;
|
||||
--skip-boot-wait) SKIP_BOOT_WAIT=true; shift ;;
|
||||
--skip-refresh-sops-keys) SKIP_REFRESH_SOPS_KEYS=true; shift ;;
|
||||
--skip-cluster-init) SKIP_CLUSTER_INIT=true; shift ;;
|
||||
--skip-tests) SKIP_TESTS=true; shift ;;
|
||||
--force-rebuild) FORCE_REBUILD=true; shift ;;
|
||||
@@ -275,14 +278,16 @@ fi
|
||||
if ! $SKIP_CREATE_VMS; then
|
||||
log "Phase 3: Building and creating VMs on ${NODE}"
|
||||
|
||||
REBUILD_FLAG=""
|
||||
$FORCE_REBUILD && REBUILD_FLAG="--force-rebuild"
|
||||
|
||||
CREATE="${REPO_ROOT}/scripts/proxmox/create-proxmox-resource.sh"
|
||||
|
||||
for spec in "${VMID1}:ha-server-1:proxmox-ha-server-1" "${VMID2}:ha-server-2:proxmox-ha-server-2"; do
|
||||
IFS=: read -r vmid host_name flake_target <<< "$spec"
|
||||
log "Creating ${flake_target} (VMID ${vmid}) on ${NODE}..."
|
||||
# Always --force-rebuild: create-proxmox-resource.sh only calls
|
||||
# sync_remote_host_keys (which populates host-keys/ for proxmox.nix to
|
||||
# bake the clan-var SSH key into the disko image) when it actually builds.
|
||||
# Reusing a cached image skips that step, so destroy+recreate would reuse
|
||||
# an image with a stale/random key baked in → sops fails on first boot.
|
||||
run bash "$CREATE" \
|
||||
--type vm \
|
||||
--host "$host_name" \
|
||||
@@ -291,7 +296,7 @@ if ! $SKIP_CREATE_VMS; then
|
||||
--storage "$STORAGE" \
|
||||
--memory "$MEMORY_MB" \
|
||||
--cores "$CORES" \
|
||||
${REBUILD_FLAG}
|
||||
--force-rebuild
|
||||
done
|
||||
fi
|
||||
|
||||
@@ -325,6 +330,90 @@ if ! $SKIP_BOOT_WAIT; then
|
||||
sleep 10
|
||||
fi
|
||||
|
||||
# ── Phase 5.5: Refresh sops host-key registrations ───────────────────────────
|
||||
#
|
||||
# Disko builds raw disk images; each new VM boots with a freshly-generated SSH
|
||||
# host key rather than the one pre-seeded in clan vars. This phase scans the
|
||||
# actual running VMs, and if their ed25519 host keys differ from what clan vars
|
||||
# record: updates the clan var pub-key files, rewrites the .sops.yaml age-key
|
||||
# anchors, and re-encrypts all affected sops files so the nodes can decrypt
|
||||
# secrets on the next nixos-rebuild. Safe no-op when keys haven't changed.
|
||||
|
||||
if ! $SKIP_REFRESH_SOPS_KEYS; then
|
||||
if $DRY_RUN; then
|
||||
logn "[dry-run] Would scan VM host keys and refresh .sops.yaml / secrets if needed"
|
||||
else
|
||||
log "Phase 5.5: Refreshing sops host-key registrations (disko key drift fix)"
|
||||
SOPS_UPDATED=false
|
||||
|
||||
for spec in \
|
||||
"${NODE1_IP}:proxmox-ha-server-1:${NODE1_HOST}" \
|
||||
"${NODE2_IP}:proxmox-ha-server-2:${NODE2_HOST}"; do
|
||||
IFS=: read -r node_ip flake_target host_name <<< "$spec"
|
||||
CLAN_PUB="${REPO_ROOT}/vars/per-machine/${flake_target}/openssh/ssh_host_ed25519_key.pub/value"
|
||||
|
||||
logn "Scanning ed25519 host key from ${host_name} (${node_ip})..."
|
||||
RAW=$(ssh-keyscan -t ed25519 "${node_ip}" 2>/dev/null | grep -v "^#") || true
|
||||
if [[ -z "$RAW" ]]; then
|
||||
logn "WARNING: no ed25519 key returned by ssh-keyscan for ${node_ip} — skipping"
|
||||
continue
|
||||
fi
|
||||
# ssh-keyscan returns: <ip> ssh-ed25519 <b64key>
|
||||
SCANNED_TYPE=$(awk '{print $2}' <<< "$RAW")
|
||||
SCANNED_KEY=$(awk '{print $3}' <<< "$RAW")
|
||||
SCANNED_PUBKEY="${SCANNED_TYPE} ${SCANNED_KEY} ${host_name}"
|
||||
|
||||
CURRENT=$(tr -d '\n' < "$CLAN_PUB" 2>/dev/null || true)
|
||||
if [[ "$SCANNED_PUBKEY" == "$CURRENT" ]]; then
|
||||
logn "${host_name}: clan var matches running key — no update needed"
|
||||
continue
|
||||
fi
|
||||
|
||||
logn "${host_name}: key drift detected — updating clan var"
|
||||
logn " old: ${CURRENT}"
|
||||
logn " new: ${SCANNED_PUBKEY}"
|
||||
echo "$SCANNED_PUBKEY" > "$CLAN_PUB"
|
||||
SOPS_UPDATED=true
|
||||
|
||||
# Rewrite the .sops.yaml anchor for this host with the new age key.
|
||||
ANCHOR="${flake_target}" # e.g. proxmox-ha-server-1
|
||||
NEW_AGE=$(echo "$SCANNED_PUBKEY" | \
|
||||
nix run --quiet --no-warn-dirty nixpkgs#ssh-to-age 2>/dev/null)
|
||||
if [[ -z "$NEW_AGE" ]]; then
|
||||
err "ssh-to-age produced no output for ${host_name} — check nixpkgs#ssh-to-age"
|
||||
fi
|
||||
logn " new age key: ${NEW_AGE}"
|
||||
sed -i "/&${ANCHOR} /s| age[a-z0-9]*$| ${NEW_AGE}|" "${REPO_ROOT}/.sops.yaml"
|
||||
done
|
||||
|
||||
if $SOPS_UPDATED; then
|
||||
logn "Running sops updatekeys on affected secrets..."
|
||||
SOPS="nix run --quiet --no-warn-dirty nixpkgs#sops --"
|
||||
(cd "${REPO_ROOT}" && \
|
||||
$SOPS updatekeys -y secrets/common.yaml && \
|
||||
$SOPS updatekeys -y secrets/ha-server-1.yaml && \
|
||||
$SOPS updatekeys -y secrets/ha-server-2.yaml && \
|
||||
$SOPS updatekeys -y secrets/ha-server-1.keytab && \
|
||||
$SOPS updatekeys -y secrets/ha-server-2.keytab)
|
||||
# Note: ha-corosync-authkey is re-generated and re-encrypted by cluster-init below.
|
||||
|
||||
logn "Committing refreshed host keys and re-encrypted secrets..."
|
||||
(cd "${REPO_ROOT}" && \
|
||||
git add \
|
||||
vars/per-machine/proxmox-ha-server-1/openssh/ssh_host_ed25519_key.pub/value \
|
||||
vars/per-machine/proxmox-ha-server-2/openssh/ssh_host_ed25519_key.pub/value \
|
||||
.sops.yaml \
|
||||
secrets/common.yaml \
|
||||
secrets/ha-server-1.yaml \
|
||||
secrets/ha-server-2.yaml \
|
||||
secrets/ha-server-1.keytab \
|
||||
secrets/ha-server-2.keytab && \
|
||||
git commit -m "secrets(ha): refresh sops host-key registrations for new VM instances" || true)
|
||||
logn "Sops keys refreshed and committed."
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
# ── Phase 6: Cluster init ─────────────────────────────────────────────────────
|
||||
|
||||
if ! $SKIP_CLUSTER_INIT; then
|
||||
@@ -355,58 +444,6 @@ if ! $SKIP_CLUSTER_INIT; then
|
||||
"sudo mkdir -p /root/.ssh && sudo cp /tmp/cluster-init-key /root/.ssh/cluster-init-key && \
|
||||
sudo chmod 600 /root/.ssh/cluster-init-key && rm -f /tmp/cluster-init-key"
|
||||
|
||||
# Fix targetctl.service on both nodes: the iscsi-target.nix module bakes
|
||||
# pkgs.targetcli-fb for the targetctl binary, but targetctl is actually in
|
||||
# rtslib-fb (a different store path). Apply a runtime dropin that corrects
|
||||
# both ExecStart and ExecStop before Pacemaker ever touches the service.
|
||||
# The fixed iscsi-target.nix module will make this redundant on next rebuild.
|
||||
logn "Patching targetctl.service on both nodes..."
|
||||
_patch_targetctl() {
|
||||
local ip="$1"
|
||||
ssh -i ~/.ssh/id_ed25519 -o StrictHostKeyChecking=no "${HA_USER}@${ip}" sudo bash << 'PATCH'
|
||||
set -euo pipefail
|
||||
TC=$(find /nix/store -maxdepth 4 -path '*/python3*env/bin/targetctl' 2>/dev/null | head -1)
|
||||
PY=$(find /nix/store -maxdepth 4 -path '*/python3*env/bin/python3' -name 'python3' 2>/dev/null | \
|
||||
while IFS= read -r p; do "$p" -c "import rtslib_fb" 2>/dev/null && echo "$p" && break; done | head -1)
|
||||
[[ -n "$TC" && -n "$PY" ]] || { echo "targetctl or python3+rtslib_fb not found"; exit 1; }
|
||||
# Write stop script that saves LIO config then tears down kernel state
|
||||
"$PY" - "$TC" "$PY" << 'PYEOF'
|
||||
import sys, os, stat
|
||||
tc, py = sys.argv[1], sys.argv[2]
|
||||
script = f"""#!{py}
|
||||
import subprocess, sys, rtslib_fb
|
||||
root = rtslib_fb.RTSRoot()
|
||||
targets = list(root.targets)
|
||||
if targets:
|
||||
r = subprocess.run(["{tc}", "save", "/etc/target/saveconfig.json"], capture_output=True)
|
||||
print(f"saved {{len(targets)}} target(s); rc={{r.returncode}}")
|
||||
else:
|
||||
print("no active LIO targets")
|
||||
for t in targets:
|
||||
try:
|
||||
for tpg in list(t.tpgs): tpg.enable = False
|
||||
t.delete()
|
||||
except Exception as e: print(f"warn: {{e}}", file=sys.stderr)
|
||||
for so in list(root.storage_objects):
|
||||
try: so.delete()
|
||||
except Exception as e: print(f"warn: {{e}}", file=sys.stderr)
|
||||
print("LIO kernel target cleared")
|
||||
"""
|
||||
path = "/run/ha-targetctl-stop.py"
|
||||
with open(path, "w") as f: f.write(script)
|
||||
os.chmod(path, 0o755)
|
||||
print(f"wrote {path}")
|
||||
PYEOF
|
||||
mkdir -p /run/systemd/system/targetctl.service.d
|
||||
printf '[Service]\nExecStart=\nExecStart=%s restore /etc/target/saveconfig.json\nExecStop=\nExecStop=/run/ha-targetctl-stop.py\n' \
|
||||
"$TC" > /run/systemd/system/targetctl.service.d/fix-exec.conf
|
||||
systemctl daemon-reload
|
||||
echo "patched on $(hostname)"
|
||||
PATCH
|
||||
}
|
||||
_patch_targetctl "${NODE1_IP}"
|
||||
_patch_targetctl "${NODE2_IP}"
|
||||
|
||||
logn "Uploading cluster-init.sh to ${NODE1_HOST}..."
|
||||
scp -i ~/.ssh/id_ed25519 -o StrictHostKeyChecking=no \
|
||||
"$CLUSTER_INIT" "${HA_USER}@${NODE1_IP}:/tmp/cluster-init.sh"
|
||||
|
||||
Executable
+256
@@ -0,0 +1,256 @@
|
||||
#!/usr/bin/env bash
|
||||
# failover.sh — graceful HA cluster failover
|
||||
#
|
||||
# Detects which node is active and moves all resources to the other node by
|
||||
# putting the active node into Pacemaker standby. Waits for the XFS mount to
|
||||
# appear on the target before returning.
|
||||
#
|
||||
# Usage:
|
||||
# scripts/ha/failover.sh [--to node1|node2] [--force] [--timeout <s>] [--dry-run]
|
||||
#
|
||||
# --to node1|node2 target node (default: the node that is NOT currently active)
|
||||
# --force skip the interactive confirmation prompt
|
||||
# --timeout <s> seconds to wait for resources to move (default: 120)
|
||||
# --dry-run show what would be done without changing anything
|
||||
set -euo pipefail
|
||||
|
||||
# ── Configuration ─────────────────────────────────────────────────────────
|
||||
NODE1="${NODE1:-ha-server-1}"
|
||||
NODE2="${NODE2:-ha-server-2}"
|
||||
NODE1_IP="${NODE1_IP:-192.168.2.228}"
|
||||
NODE2_IP="${NODE2_IP:-192.168.2.227}"
|
||||
VIP="${VIP:-192.168.2.229}"
|
||||
XFS_MOUNT="${XFS_MOUNT:-/srv/ha-data}"
|
||||
HA_USER="${HA_USER:-nixos}"
|
||||
# ──────────────────────────────────────────────────────────────────────────
|
||||
|
||||
n1() { ssh -i ~/.ssh/id_ed25519 -o StrictHostKeyChecking=no -o ConnectTimeout=5 "${HA_USER}@${NODE1_IP}" sudo "$@" 2>/dev/null; }
|
||||
n2() { ssh -i ~/.ssh/id_ed25519 -o StrictHostKeyChecking=no -o ConnectTimeout=5 "${HA_USER}@${NODE2_IP}" sudo "$@" 2>/dev/null; }
|
||||
|
||||
# ── Argument parsing ───────────────────────────────────────────────────────
|
||||
TARGET_NODE=""
|
||||
FORCE=false
|
||||
DRY_RUN=false
|
||||
TIMEOUT=120
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--to)
|
||||
shift
|
||||
case "${1:-}" in
|
||||
node1|ha-server-1) TARGET_NODE="$NODE1" ;;
|
||||
node2|ha-server-2) TARGET_NODE="$NODE2" ;;
|
||||
*) echo "ERROR: --to must be node1, node2, ha-server-1, or ha-server-2"; exit 1 ;;
|
||||
esac
|
||||
;;
|
||||
--force) FORCE=true ;;
|
||||
--dry-run) DRY_RUN=true ;;
|
||||
--timeout) shift; TIMEOUT="${1:?--timeout requires a value}" ;;
|
||||
*) echo "Unknown argument: $1"; echo "Usage: $0 [--to node1|node2] [--force] [--timeout <s>] [--dry-run]"; exit 1 ;;
|
||||
esac
|
||||
shift
|
||||
done
|
||||
|
||||
DRY_PREFIX=""
|
||||
$DRY_RUN && DRY_PREFIX="[dry-run] "
|
||||
|
||||
echo "════════════════════════════════════════════════════"
|
||||
echo " HA Cluster Failover — $(date '+%Y-%m-%d %H:%M:%S')"
|
||||
$DRY_RUN && echo " MODE: dry-run — no changes will be made"
|
||||
echo "════════════════════════════════════════════════════"
|
||||
|
||||
# ── Detect active node ─────────────────────────────────────────────────────
|
||||
echo ""
|
||||
echo "Detecting active node..."
|
||||
|
||||
CRM_OUT=""
|
||||
if ssh -i ~/.ssh/id_ed25519 -o StrictHostKeyChecking=no -o ConnectTimeout=5 "${HA_USER}@${NODE1_IP}" true 2>/dev/null; then
|
||||
CRM_OUT=$(n1 "crm_mon -1" 2>/dev/null || true)
|
||||
fi
|
||||
if [[ -z "$CRM_OUT" ]]; then
|
||||
if ssh -i ~/.ssh/id_ed25519 -o StrictHostKeyChecking=no -o ConnectTimeout=5 "${HA_USER}@${NODE2_IP}" true 2>/dev/null; then
|
||||
CRM_OUT=$(n2 "crm_mon -1" 2>/dev/null || true)
|
||||
fi
|
||||
fi
|
||||
|
||||
# crm_mon 2.x formats Promoted lines as " * Promoted: [ node ]" — the * bullet
|
||||
# means ^\s*(Promoted|Masters): never matches; filter Unpromoted first instead.
|
||||
ACTIVE_NODE=$(echo "$CRM_OUT" | grep -v 'Unpromoted\|Unmanaged' | \
|
||||
grep -E '(Promoted|Masters):' | \
|
||||
grep -oE '\b(ha-server-[0-9]+)\b' | head -1 || true)
|
||||
|
||||
if [[ -z "$ACTIVE_NODE" ]]; then
|
||||
echo ""
|
||||
echo "ERROR: could not determine active node from crm_mon."
|
||||
echo " Is Pacemaker still settling? Try running scripts/ha/health.sh first."
|
||||
echo " If Pacemaker is down on both nodes, manual recovery is required."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [[ "$ACTIVE_NODE" == "$NODE1" ]]; then
|
||||
ACTIVE_IP="$NODE1_IP"
|
||||
STANDBY_NODE="$NODE2"
|
||||
STANDBY_IP="$NODE2_IP"
|
||||
na() { n1 "$@"; }
|
||||
ns() { n2 "$@"; }
|
||||
else
|
||||
ACTIVE_IP="$NODE2_IP"
|
||||
STANDBY_NODE="$NODE1"
|
||||
STANDBY_IP="$NODE1_IP"
|
||||
na() { n2 "$@"; }
|
||||
ns() { n1 "$@"; }
|
||||
fi
|
||||
|
||||
echo " Active: $ACTIVE_NODE ($ACTIVE_IP)"
|
||||
echo " Standby: $STANDBY_NODE ($STANDBY_IP)"
|
||||
|
||||
# ── Validate target ────────────────────────────────────────────────────────
|
||||
if [[ -n "$TARGET_NODE" ]]; then
|
||||
if [[ "$TARGET_NODE" == "$ACTIVE_NODE" ]]; then
|
||||
echo ""
|
||||
echo "ERROR: $TARGET_NODE is already the active node — nothing to do."
|
||||
exit 1
|
||||
fi
|
||||
echo " Target: $TARGET_NODE (as requested)"
|
||||
else
|
||||
echo " Target: $STANDBY_NODE (auto — the other node)"
|
||||
fi
|
||||
|
||||
# ── Pre-checks ─────────────────────────────────────────────────────────────
|
||||
echo ""
|
||||
echo "Pre-checks..."
|
||||
|
||||
DRBD_DSTATE=$(na "drbdadm dstate ha-data 2>/dev/null" 2>/dev/null || echo "unknown")
|
||||
if ! echo "$DRBD_DSTATE" | grep -q "UpToDate/UpToDate"; then
|
||||
echo ""
|
||||
echo " WARNING: DRBD dstate is '$DRBD_DSTATE' (not UpToDate/UpToDate)."
|
||||
echo " Failing over with a partially-synced disk risks split-brain."
|
||||
if ! $FORCE; then
|
||||
echo " Use --force to proceed anyway (not recommended)."
|
||||
exit 1
|
||||
fi
|
||||
echo " --force specified — proceeding despite non-ideal DRBD state."
|
||||
else
|
||||
echo " DRBD dstate: $DRBD_DSTATE — OK"
|
||||
fi
|
||||
|
||||
QUORUM_OK=$(na "corosync-quorumtool -s 2>/dev/null | grep -c 'Quorate:.*Yes'" 2>/dev/null || echo "0")
|
||||
if [[ "$QUORUM_OK" -lt 1 ]]; then
|
||||
echo " ERROR: cluster does not have quorum — failover would be unsafe."
|
||||
exit 1
|
||||
fi
|
||||
echo " Quorum: OK"
|
||||
|
||||
# ── Confirm ────────────────────────────────────────────────────────────────
|
||||
if ! $FORCE && ! $DRY_RUN; then
|
||||
echo ""
|
||||
echo " This will move all resources from $ACTIVE_NODE → $STANDBY_NODE."
|
||||
echo " VIP and services will be unreachable for ~10–30 seconds."
|
||||
printf " Proceed? [y/N] "
|
||||
read -r ANSWER
|
||||
[[ "${ANSWER,,}" == "y" || "${ANSWER,,}" == "yes" ]] || { echo "Aborted."; exit 0; }
|
||||
fi
|
||||
|
||||
# ── Capture active node's crm_node name ───────────────────────────────────
|
||||
# crm_node -n returns the node name as registered in Pacemaker (may differ
|
||||
# from hostname if Pacemaker was configured with explicit node names).
|
||||
ACTIVE_CRMD_NAME=$(na "crm_node -n 2>/dev/null" 2>/dev/null || echo "$ACTIVE_NODE")
|
||||
|
||||
# ── Perform failover ───────────────────────────────────────────────────────
|
||||
echo ""
|
||||
echo "${DRY_PREFIX}Putting $ACTIVE_NODE into standby (resources will migrate to $STANDBY_NODE)..."
|
||||
if ! $DRY_RUN; then
|
||||
na "crm_standby -N '${ACTIVE_CRMD_NAME}' -v on" 2>/dev/null || true
|
||||
fi
|
||||
|
||||
# ── Wait for resources to move ─────────────────────────────────────────────
|
||||
echo "${DRY_PREFIX}Waiting up to ${TIMEOUT}s for XFS to mount on $STANDBY_NODE..."
|
||||
MOVED=false
|
||||
SPIN_CHARS=('|' '/' '-' '\')
|
||||
SPIN_I=0
|
||||
|
||||
if $DRY_RUN; then
|
||||
echo " [dry-run] would wait for mountpoint $XFS_MOUNT on $STANDBY_NODE"
|
||||
MOVED=true
|
||||
else
|
||||
for i in $(seq 1 "$TIMEOUT"); do
|
||||
if ns "mountpoint -q '${XFS_MOUNT}' 2>/dev/null" 2>/dev/null; then
|
||||
printf "\r%-80s\r" ""
|
||||
echo " Resources moved in ${i}s"
|
||||
MOVED=true
|
||||
break
|
||||
fi
|
||||
SPIN_I=$(( SPIN_I + 1 ))
|
||||
SC="${SPIN_CHARS[$((SPIN_I % 4))]}"
|
||||
printf "\r [%s] waiting... (%ds) " "$SC" "$i"
|
||||
sleep 1
|
||||
done
|
||||
fi
|
||||
|
||||
if ! $MOVED; then
|
||||
echo ""
|
||||
echo "ERROR: XFS did not mount on $STANDBY_NODE within ${TIMEOUT}s."
|
||||
echo ""
|
||||
echo " Current resource state:"
|
||||
na "crm_mon -1 2>/dev/null" 2>/dev/null | grep -E 'Started|Stopped|Promoted|Unpromoted|FAILED' | sed 's/^/ /' || true
|
||||
echo ""
|
||||
echo " Clearing standby to restore $ACTIVE_NODE (undo the failover attempt)..."
|
||||
na "crm_standby -N '${ACTIVE_CRMD_NAME}' -v off" 2>/dev/null || true
|
||||
na "crm_resource --cleanup" 2>/dev/null || true
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# ── Clear failure history ──────────────────────────────────────────────────
|
||||
echo "${DRY_PREFIX}Clearing Pacemaker failure history..."
|
||||
if ! $DRY_RUN; then
|
||||
ns "crm_resource --cleanup 2>/dev/null" 2>/dev/null || true
|
||||
fi
|
||||
|
||||
# ── Re-enable original active node as standby ─────────────────────────────
|
||||
echo "${DRY_PREFIX}Re-enabling $ACTIVE_NODE (now standby — will not claim resources)..."
|
||||
if ! $DRY_RUN; then
|
||||
na "crm_standby -N '${ACTIVE_CRMD_NAME}' -v off" 2>/dev/null || true
|
||||
fi
|
||||
|
||||
# ── Wait briefly for DRBD resync to begin ─────────────────────────────────
|
||||
if ! $DRY_RUN; then
|
||||
sleep 5
|
||||
fi
|
||||
|
||||
# ── Final state ────────────────────────────────────────────────────────────
|
||||
echo ""
|
||||
echo "Failover complete. Final state:"
|
||||
echo ""
|
||||
|
||||
CRM_OUT_AFTER=""
|
||||
if ! $DRY_RUN; then
|
||||
CRM_OUT_AFTER=$(ns "crm_mon -1" 2>/dev/null || na "crm_mon -1" 2>/dev/null || true)
|
||||
else
|
||||
CRM_OUT_AFTER="$CRM_OUT"
|
||||
fi
|
||||
|
||||
NEW_ACTIVE=$(echo "$CRM_OUT_AFTER" | grep -v 'Unpromoted\|Unmanaged' | \
|
||||
grep -E '(Promoted|Masters):' | \
|
||||
grep -oE '\b(ha-server-[0-9]+)\b' | head -1 || true)
|
||||
|
||||
if [[ -n "$NEW_ACTIVE" ]]; then
|
||||
if [[ "$NEW_ACTIVE" == "$ACTIVE_NODE" ]]; then
|
||||
echo " WARNING: $ACTIVE_NODE is still showing as active in crm_mon."
|
||||
echo " Pacemaker may still be settling — check again in a few seconds."
|
||||
else
|
||||
echo " Active: $NEW_ACTIVE"
|
||||
echo " Standby: $ACTIVE_NODE"
|
||||
fi
|
||||
fi
|
||||
|
||||
echo ""
|
||||
RESOURCES_AFTER=$(echo "$CRM_OUT_AFTER" | awk '/Full List of Resources/,0' | tail -n +2 || true)
|
||||
[[ -z "$RESOURCES_AFTER" ]] && RESOURCES_AFTER=$(echo "$CRM_OUT_AFTER" | \
|
||||
grep -E 'Started|Stopped|Promoted|Unpromoted|FAILED|Master|Slave' || true)
|
||||
[[ -n "$RESOURCES_AFTER" ]] && echo "$RESOURCES_AFTER" | sed 's/^/ /'
|
||||
|
||||
echo ""
|
||||
echo " (DRBD resync of $ACTIVE_NODE may take a moment; monitor with:"
|
||||
echo " ssh nixos@${ACTIVE_IP} 'sudo watch -n3 cat /proc/drbd')"
|
||||
echo ""
|
||||
echo "════════════════════════════════════════════════════"
|
||||
Regular → Executable
Executable
+180
@@ -0,0 +1,180 @@
|
||||
#!/usr/bin/env bash
|
||||
# health.sh — HA cluster health snapshot (read-only, non-destructive)
|
||||
#
|
||||
# Prints a compact status panel across both nodes: SSH reachability, quorum,
|
||||
# DRBD state, Pacemaker resources, and service ports via the VIP.
|
||||
# Run from any host with SSH access to the HA nodes.
|
||||
set -euo pipefail
|
||||
|
||||
# ── Configuration ─────────────────────────────────────────────────────────
|
||||
NODE1="${NODE1:-ha-server-1}"
|
||||
NODE2="${NODE2:-ha-server-2}"
|
||||
NODE1_IP="${NODE1_IP:-192.168.2.228}" # vars.haServer1Ip
|
||||
NODE2_IP="${NODE2_IP:-192.168.2.227}" # vars.haServer2Ip
|
||||
VIP="${VIP:-192.168.2.229}" # vars.haServerVip
|
||||
XFS_MOUNT="${XFS_MOUNT:-/srv/ha-data}" # vars.haStorageRoot
|
||||
HA_USER="${HA_USER:-nixos}"
|
||||
# ──────────────────────────────────────────────────────────────────────────
|
||||
|
||||
REACHABLE_1=false
|
||||
REACHABLE_2=false
|
||||
|
||||
n1() { ssh -i ~/.ssh/id_ed25519 -o StrictHostKeyChecking=no -o ConnectTimeout=5 "${HA_USER}@${NODE1_IP}" sudo "$@" 2>/dev/null; }
|
||||
n2() { ssh -i ~/.ssh/id_ed25519 -o StrictHostKeyChecking=no -o ConnectTimeout=5 "${HA_USER}@${NODE2_IP}" sudo "$@" 2>/dev/null; }
|
||||
|
||||
probe_node() {
|
||||
local ip=$1
|
||||
ssh -i ~/.ssh/id_ed25519 -o StrictHostKeyChecking=no -o ConnectTimeout=5 "${HA_USER}@${ip}" true 2>/dev/null && echo "ONLINE" || echo "OFFLINE"
|
||||
}
|
||||
|
||||
section() { echo ""; echo "── $* ──"; }
|
||||
|
||||
echo "════════════════════════════════════════════════════"
|
||||
echo " HA Cluster Health — $(date '+%Y-%m-%d %H:%M:%S')"
|
||||
echo "════════════════════════════════════════════════════"
|
||||
|
||||
# ── Node reachability ──────────────────────────────────────────────────────
|
||||
section "Nodes"
|
||||
N1_STATUS=$(probe_node "$NODE1_IP")
|
||||
N2_STATUS=$(probe_node "$NODE2_IP")
|
||||
[[ "$N1_STATUS" == "ONLINE" ]] && REACHABLE_1=true
|
||||
[[ "$N2_STATUS" == "ONLINE" ]] && REACHABLE_2=true
|
||||
|
||||
if ! $REACHABLE_1 && ! $REACHABLE_2; then
|
||||
echo " ERROR: both nodes unreachable — cannot continue."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# ── Detect active node ─────────────────────────────────────────────────────
|
||||
# crm_mon 2.x formats the Promoted line as " * Promoted: [ node ]" — the
|
||||
# bullet * means ^\s*(Promoted|Masters): never matches. Filter out Unpromoted
|
||||
# first, then match anywhere on the line.
|
||||
ACTIVE_NODE=""
|
||||
CRM_OUT=""
|
||||
if $REACHABLE_1; then
|
||||
CRM_OUT=$(n1 "crm_mon -1" 2>/dev/null || true)
|
||||
elif $REACHABLE_2; then
|
||||
CRM_OUT=$(n2 "crm_mon -1" 2>/dev/null || true)
|
||||
fi
|
||||
ACTIVE_NODE=$(echo "${CRM_OUT:-}" | grep -v 'Unpromoted\|Unmanaged' | \
|
||||
grep -E '(Promoted|Masters):' | \
|
||||
grep -oE '\b(ha-server-[0-9]+)\b' | head -1 || true)
|
||||
|
||||
STANDBY_NODE=""
|
||||
if [[ "$ACTIVE_NODE" == "$NODE1" ]]; then
|
||||
STANDBY_NODE="$NODE2"
|
||||
elif [[ "$ACTIVE_NODE" == "$NODE2" ]]; then
|
||||
STANDBY_NODE="$NODE1"
|
||||
fi
|
||||
|
||||
n1_tag=""; n2_tag=""
|
||||
[[ "$ACTIVE_NODE" == "$NODE1" ]] && n1_tag=" [ACTIVE]" || n1_tag=" [STANDBY]"
|
||||
[[ "$ACTIVE_NODE" == "$NODE2" ]] && n2_tag=" [ACTIVE]" || n2_tag=" [STANDBY]"
|
||||
[[ -z "$ACTIVE_NODE" ]] && { n1_tag=""; n2_tag=""; }
|
||||
|
||||
printf " %-14s [%s]%s\n" "$NODE1" "$N1_STATUS" "$n1_tag"
|
||||
printf " %-14s [%s]%s\n" "$NODE2" "$N2_STATUS" "$n2_tag"
|
||||
|
||||
if [[ -z "$ACTIVE_NODE" ]]; then
|
||||
echo ""
|
||||
echo " WARNING: could not determine active node from crm_mon."
|
||||
echo " Pacemaker may still be settling, or both nodes may be in standby."
|
||||
fi
|
||||
|
||||
# ── Quorum ─────────────────────────────────────────────────────────────────
|
||||
section "Quorum"
|
||||
if $REACHABLE_1; then
|
||||
QUORUM=$(n1 "corosync-quorumtool -s 2>/dev/null" 2>/dev/null || echo "")
|
||||
elif $REACHABLE_2; then
|
||||
QUORUM=$(n2 "corosync-quorumtool -s 2>/dev/null" 2>/dev/null || echo "")
|
||||
fi
|
||||
|
||||
if [[ -z "${QUORUM:-}" ]]; then
|
||||
echo " corosync-quorumtool: unavailable"
|
||||
else
|
||||
QUORATE=$(echo "$QUORUM" | grep "Quorate:" | awk '{print $2}' || echo "?")
|
||||
VOTES=$(echo "$QUORUM" | grep "Total votes:" | awk '{print $3}' || echo "?")
|
||||
NEEDED=$(echo "$QUORUM" | grep "Quorum votes:" | awk '{print $3}' || echo "?")
|
||||
echo " Quorate: $QUORATE Votes: $VOTES / Expected: $NEEDED"
|
||||
fi
|
||||
|
||||
# ── DRBD ───────────────────────────────────────────────────────────────────
|
||||
section "DRBD (ha-data)"
|
||||
|
||||
drbd_info_from() {
|
||||
local node=$1 run=$2
|
||||
local role dstate cs pct
|
||||
role=$($run "drbdadm role ha-data 2>/dev/null" 2>/dev/null || echo "unknown")
|
||||
dstate=$($run "drbdadm dstate ha-data 2>/dev/null" 2>/dev/null || echo "unknown")
|
||||
cs=$($run "grep -oE 'cs:[A-Za-z]+' /proc/drbd 2>/dev/null | head -1 | sed 's/cs://'" 2>/dev/null || echo "unknown")
|
||||
pct=$($run "grep -oE \"sync'ed:[[:space:]]+[0-9.]+\" /proc/drbd 2>/dev/null | grep -oE '[0-9.]+$' | head -1" 2>/dev/null || echo "")
|
||||
printf " %-14s role: %-22s dstate: %-25s cs: %s" \
|
||||
"$node" "${role:-unknown}" "${dstate:-unknown}" "${cs:-unknown}"
|
||||
[[ -n "$pct" ]] && printf " syncing: %s%%" "$pct"
|
||||
echo ""
|
||||
}
|
||||
|
||||
$REACHABLE_1 && drbd_info_from "$NODE1" n1 || echo " $NODE1 [OFFLINE]"
|
||||
$REACHABLE_2 && drbd_info_from "$NODE2" n2 || echo " $NODE2 [OFFLINE]"
|
||||
|
||||
# ── Pacemaker (full crm_mon output) ───────────────────────────────────────
|
||||
section "Pacemaker"
|
||||
if [[ -n "${CRM_OUT:-}" ]]; then
|
||||
echo "$CRM_OUT" | sed 's/^/ /'
|
||||
else
|
||||
echo " crm_mon returned no output — trying again without suppression:"
|
||||
if $REACHABLE_1; then
|
||||
n1 "crm_mon -1" || true
|
||||
elif $REACHABLE_2; then
|
||||
n2 "crm_mon -1" || true
|
||||
fi
|
||||
fi
|
||||
|
||||
# ── XFS mount ─────────────────────────────────────────────────────────────
|
||||
section "XFS Mount ($XFS_MOUNT)"
|
||||
check_mount() {
|
||||
local node=$1 run=$2
|
||||
local status
|
||||
if $run "mountpoint -q '$XFS_MOUNT' 2>/dev/null" 2>/dev/null; then
|
||||
local usage
|
||||
usage=$($run "df -h '$XFS_MOUNT' 2>/dev/null | tail -1 | awk '{print \$3\"/\"\$2\" used (\"\$5\")\";}'" 2>/dev/null || echo "")
|
||||
status="mounted"
|
||||
[[ -n "$usage" ]] && status="mounted $usage"
|
||||
else
|
||||
status="not mounted"
|
||||
fi
|
||||
printf " %-14s %s\n" "$node" "$status"
|
||||
}
|
||||
|
||||
$REACHABLE_1 && check_mount "$NODE1" n1 || echo " $NODE1 [OFFLINE]"
|
||||
$REACHABLE_2 && check_mount "$NODE2" n2 || echo " $NODE2 [OFFLINE]"
|
||||
|
||||
# ── Service ports via VIP ──────────────────────────────────────────────────
|
||||
section "Services via VIP ($VIP)"
|
||||
|
||||
check_port() {
|
||||
local name=$1 port=$2
|
||||
if bash -c "echo >/dev/tcp/${VIP}/${port}" 2>/dev/null; then
|
||||
printf " %-10s port %-5s OK\n" "$name" "$port"
|
||||
else
|
||||
printf " %-10s port %-5s UNREACHABLE\n" "$name" "$port"
|
||||
fi
|
||||
}
|
||||
|
||||
if ping -c1 -W2 "$VIP" >/dev/null 2>&1; then
|
||||
echo " Ping OK"
|
||||
else
|
||||
echo " Ping UNREACHABLE"
|
||||
fi
|
||||
check_port "NFS" 2049
|
||||
check_port "iSCSI" 3260
|
||||
|
||||
echo ""
|
||||
echo "════════════════════════════════════════════════════"
|
||||
if [[ -n "$ACTIVE_NODE" ]]; then
|
||||
echo " Active: $ACTIVE_NODE Standby: $STANDBY_NODE"
|
||||
else
|
||||
echo " Active: unknown (Pacemaker not settled)"
|
||||
fi
|
||||
echo "════════════════════════════════════════════════════"
|
||||
echo ""
|
||||
Executable
+276
@@ -0,0 +1,276 @@
|
||||
#!/usr/bin/env bash
|
||||
# resize-data-disk.sh — online resize of the HA cluster data disk
|
||||
#
|
||||
# Three-phase process (all online-safe, no downtime required):
|
||||
# 1. Proxmox: grow scsi1 on both HA VMs (qm resize)
|
||||
# 2. Guest: rescan block device on both nodes so the kernel sees the new size
|
||||
# 3. DRBD + XFS: drbdadm resize, then xfs_growfs — active node only
|
||||
#
|
||||
# Usage:
|
||||
# scripts/ha/resize-data-disk.sh --size +20G [--force] [--dry-run]
|
||||
#
|
||||
# --size +NNg amount to grow scsi1 by, e.g. +20G, +50G (required)
|
||||
# XFS and DRBD cannot shrink; only positive deltas accepted
|
||||
# --force skip the interactive confirmation prompt
|
||||
# --dry-run show what would be done without changing anything
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
# shellcheck source=../env.sh
|
||||
source "${SCRIPT_DIR}/../env.sh"
|
||||
|
||||
# ── Configuration ─────────────────────────────────────────────────────────
|
||||
NODE1="${NODE1:-ha-server-1}"
|
||||
NODE2="${NODE2:-ha-server-2}"
|
||||
NODE1_IP="${NODE1_IP:-192.168.2.228}"
|
||||
NODE2_IP="${NODE2_IP:-192.168.2.227}"
|
||||
XFS_MOUNT="${XFS_MOUNT:-/srv/ha-data}"
|
||||
DRBD_RESOURCE="${DRBD_RESOURCE:-ha-data}"
|
||||
DATA_DISK_SLOT="${DATA_DISK_SLOT:-scsi1}" # Proxmox disk name (scsi1 = data disk)
|
||||
HA_USER="${HA_USER:-nixos}"
|
||||
PVE_HOST="${PVE_HOST:-${PVE1_HOST}}"
|
||||
PVE_SSH_USER="${PVE_SSH_USER:-${PROXMOX_SSH_USER}}"
|
||||
# By-id symlink for the data disk; basename resolves to the raw block device.
|
||||
# matches variables.nix's haServerDrbdDisk.
|
||||
DATA_DISK_BYID="${DATA_DISK_BYID:-scsi-0QEMU_QEMU_HARDDISK_drive-${DATA_DISK_SLOT}}"
|
||||
# ──────────────────────────────────────────────────────────────────────────
|
||||
|
||||
pve() { ssh -i ~/.ssh/id_ed25519 -o StrictHostKeyChecking=no -o ConnectTimeout=10 \
|
||||
"${PVE_SSH_USER}@${PVE_HOST}" "$@"; }
|
||||
n1() { ssh -i ~/.ssh/id_ed25519 -o StrictHostKeyChecking=no -o ConnectTimeout=5 \
|
||||
"${HA_USER}@${NODE1_IP}" sudo "$@" 2>/dev/null; }
|
||||
n2() { ssh -i ~/.ssh/id_ed25519 -o StrictHostKeyChecking=no -o ConnectTimeout=5 \
|
||||
"${HA_USER}@${NODE2_IP}" sudo "$@" 2>/dev/null; }
|
||||
|
||||
# ── Argument parsing ───────────────────────────────────────────────────────
|
||||
SIZE=""
|
||||
FORCE=false
|
||||
DRY_RUN=false
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--size) shift; SIZE="${1:?--size requires a value (e.g. +20G)}" ;;
|
||||
--force) FORCE=true ;;
|
||||
--dry-run) DRY_RUN=true ;;
|
||||
*) echo "Unknown argument: $1"
|
||||
echo "Usage: $0 --size +NNg [--force] [--dry-run]"
|
||||
exit 1 ;;
|
||||
esac
|
||||
shift
|
||||
done
|
||||
|
||||
if [[ -z "$SIZE" ]]; then
|
||||
echo "ERROR: --size is required (e.g. --size +20G)"
|
||||
echo "Usage: $0 --size +NNg [--force] [--dry-run]"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Only positive deltas — qm resize, DRBD, and XFS all refuse to shrink.
|
||||
if [[ ! "$SIZE" =~ ^\+[0-9]+(G|M|T)$ ]]; then
|
||||
echo "ERROR: --size must be a positive delta like +20G, +50G, +500M, +2T"
|
||||
echo " (qm resize, drbdadm resize, and xfs_growfs can only grow, not shrink)"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
DRY_PREFIX=""
|
||||
$DRY_RUN && DRY_PREFIX="[dry-run] "
|
||||
|
||||
echo "════════════════════════════════════════════════════"
|
||||
echo " HA Data Disk Resize — $(date '+%Y-%m-%d %H:%M:%S')"
|
||||
echo " Size delta: $SIZE • Proxmox: $PVE_HOST"
|
||||
$DRY_RUN && echo " MODE: dry-run — no changes will be made"
|
||||
echo "════════════════════════════════════════════════════"
|
||||
|
||||
# ── Detect active node ─────────────────────────────────────────────────────
|
||||
echo ""
|
||||
echo "Detecting active node..."
|
||||
|
||||
CRM_OUT=""
|
||||
if ssh -i ~/.ssh/id_ed25519 -o StrictHostKeyChecking=no -o ConnectTimeout=5 \
|
||||
"${HA_USER}@${NODE1_IP}" true 2>/dev/null; then
|
||||
CRM_OUT=$(n1 "crm_mon -1" 2>/dev/null || true)
|
||||
fi
|
||||
if [[ -z "$CRM_OUT" ]]; then
|
||||
if ssh -i ~/.ssh/id_ed25519 -o StrictHostKeyChecking=no -o ConnectTimeout=5 \
|
||||
"${HA_USER}@${NODE2_IP}" true 2>/dev/null; then
|
||||
CRM_OUT=$(n2 "crm_mon -1" 2>/dev/null || true)
|
||||
fi
|
||||
fi
|
||||
|
||||
# crm_mon 2.x formats Promoted lines as " * Promoted: [ node ]" (bullet *),
|
||||
# so ^\s*(Promoted|Masters): never matches; filter Unpromoted first instead.
|
||||
ACTIVE_NODE=$(echo "$CRM_OUT" | grep -v 'Unpromoted\|Unmanaged' | \
|
||||
grep -E '(Promoted|Masters):' | \
|
||||
grep -oE '\b(ha-server-[0-9]+)\b' | head -1 || true)
|
||||
|
||||
if [[ -z "$ACTIVE_NODE" ]]; then
|
||||
echo "ERROR: could not determine active node from crm_mon."
|
||||
echo " Is Pacemaker still settling? Try running scripts/ha/health.sh first."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [[ "$ACTIVE_NODE" == "$NODE1" ]]; then
|
||||
ACTIVE_IP="$NODE1_IP"
|
||||
na() { n1 "$@"; }
|
||||
else
|
||||
ACTIVE_IP="$NODE2_IP"
|
||||
na() { n2 "$@"; }
|
||||
fi
|
||||
|
||||
echo " Active node: $ACTIVE_NODE ($ACTIVE_IP)"
|
||||
|
||||
# ── Pre-check DRBD state ───────────────────────────────────────────────────
|
||||
echo ""
|
||||
echo "Pre-checks..."
|
||||
|
||||
DRBD_DSTATE=$(na "drbdadm dstate ${DRBD_RESOURCE} 2>/dev/null" 2>/dev/null || echo "unknown")
|
||||
if ! echo "$DRBD_DSTATE" | grep -q "UpToDate/UpToDate"; then
|
||||
echo " WARNING: DRBD dstate is '$DRBD_DSTATE' (expected UpToDate/UpToDate)."
|
||||
echo " Resizing with a partially-synced disk may cause issues."
|
||||
if ! $FORCE; then
|
||||
echo " Use --force to proceed anyway."
|
||||
exit 1
|
||||
fi
|
||||
echo " --force specified — proceeding despite non-ideal DRBD state."
|
||||
else
|
||||
echo " DRBD dstate: $DRBD_DSTATE — OK"
|
||||
fi
|
||||
|
||||
# ── Find VMIDs on Proxmox ─────────────────────────────────────────────────
|
||||
echo ""
|
||||
echo "Looking up VM IDs on ${PVE_HOST}..."
|
||||
|
||||
QM_LIST=$(pve "qm list 2>/dev/null" || true)
|
||||
VMID1=$(echo "$QM_LIST" | awk -v name="$NODE1" '$0 ~ name {print $1}' | head -1)
|
||||
VMID2=$(echo "$QM_LIST" | awk -v name="$NODE2" '$0 ~ name {print $1}' | head -1)
|
||||
|
||||
if [[ -z "$VMID1" ]]; then
|
||||
echo " ERROR: could not find VMID for $NODE1 on $PVE_HOST"
|
||||
echo " qm list output:"
|
||||
echo "$QM_LIST" | sed 's/^/ /'
|
||||
exit 1
|
||||
fi
|
||||
if [[ -z "$VMID2" ]]; then
|
||||
echo " ERROR: could not find VMID for $NODE2 on $PVE_HOST"
|
||||
echo " qm list output:"
|
||||
echo "$QM_LIST" | sed 's/^/ /'
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo " $NODE1: VMID $VMID1"
|
||||
echo " $NODE2: VMID $VMID2"
|
||||
|
||||
# ── Confirm ────────────────────────────────────────────────────────────────
|
||||
if ! $FORCE && ! $DRY_RUN; then
|
||||
echo ""
|
||||
echo " Plan:"
|
||||
echo " Phase 1 — qm resize $VMID1 ${DATA_DISK_SLOT} ${SIZE} (on $PVE_HOST)"
|
||||
echo " qm resize $VMID2 ${DATA_DISK_SLOT} ${SIZE} (on $PVE_HOST)"
|
||||
echo " Phase 2 — block device rescan on $NODE1 and $NODE2"
|
||||
echo " Phase 3 — drbdadm resize + xfs_growfs on $ACTIVE_NODE"
|
||||
echo " No downtime required (all operations are online-safe)."
|
||||
printf " Proceed? [y/N] "
|
||||
read -r ANSWER
|
||||
[[ "${ANSWER,,}" == "y" || "${ANSWER,,}" == "yes" ]] || { echo "Aborted."; exit 0; }
|
||||
fi
|
||||
|
||||
# ═══════════════════════════════════════════════════════════════
|
||||
# Phase 1 — Resize both VM data disks in Proxmox
|
||||
# ═══════════════════════════════════════════════════════════════
|
||||
echo ""
|
||||
echo "── Phase 1 — Proxmox disk resize (${DATA_DISK_SLOT} ${SIZE} on both VMs) ──"
|
||||
|
||||
echo " ${DRY_PREFIX}qm resize $VMID1 ${DATA_DISK_SLOT} ${SIZE} ($NODE1 on ${PVE_HOST})"
|
||||
if ! $DRY_RUN; then
|
||||
pve "qm resize $VMID1 ${DATA_DISK_SLOT} ${SIZE}"
|
||||
fi
|
||||
|
||||
echo " ${DRY_PREFIX}qm resize $VMID2 ${DATA_DISK_SLOT} ${SIZE} ($NODE2 on ${PVE_HOST})"
|
||||
if ! $DRY_RUN; then
|
||||
pve "qm resize $VMID2 ${DATA_DISK_SLOT} ${SIZE}"
|
||||
fi
|
||||
|
||||
echo " Phase 1 done."
|
||||
|
||||
# ═══════════════════════════════════════════════════════════════
|
||||
# Phase 2 — Rescan block device on both guest nodes
|
||||
# ═══════════════════════════════════════════════════════════════
|
||||
echo ""
|
||||
echo "── Phase 2 — Block device rescan (both nodes) ──"
|
||||
|
||||
rescan_node() {
|
||||
local node_name=$1 run_fn=$2
|
||||
|
||||
# Resolve block device name from the stable by-id symlink on the guest.
|
||||
local blk_dev=""
|
||||
if ! $DRY_RUN; then
|
||||
blk_dev=$($run_fn "bash -c 'basename \$(readlink -f /dev/disk/by-id/${DATA_DISK_BYID})'" 2>/dev/null || true)
|
||||
fi
|
||||
if [[ -z "$blk_dev" ]]; then
|
||||
# Fall back: scsi1 → index 1 → sdb, scsi2 → sdc, etc.
|
||||
local slot_idx
|
||||
slot_idx=$(echo "$DATA_DISK_SLOT" | grep -oE '[0-9]+$' || echo "1")
|
||||
blk_dev=$(printf "sd%s" "$(echo "abcdefghij" | cut -c$((slot_idx + 1)))")
|
||||
echo " WARNING: could not resolve /dev/disk/by-id/${DATA_DISK_BYID} on $node_name;" \
|
||||
"falling back to /dev/${blk_dev}" >&2
|
||||
fi
|
||||
|
||||
echo " ${DRY_PREFIX}Rescanning /dev/${blk_dev} on ${node_name}..."
|
||||
if ! $DRY_RUN; then
|
||||
$run_fn "bash -c 'echo 1 > /sys/block/${blk_dev}/device/rescan'" 2>/dev/null
|
||||
local new_size
|
||||
new_size=$($run_fn "lsblk -nd -o SIZE /dev/${blk_dev} 2>/dev/null" 2>/dev/null || echo "unknown")
|
||||
echo " /dev/${blk_dev} on $node_name now reports: $new_size"
|
||||
fi
|
||||
}
|
||||
|
||||
rescan_node "$NODE1" n1
|
||||
rescan_node "$NODE2" n2
|
||||
|
||||
echo " Phase 2 done."
|
||||
|
||||
# ═══════════════════════════════════════════════════════════════
|
||||
# Phase 3 — Grow DRBD metadata, then XFS (active node only)
|
||||
# ═══════════════════════════════════════════════════════════════
|
||||
echo ""
|
||||
echo "── Phase 3 — DRBD resize + XFS grow (on active node: $ACTIVE_NODE) ──"
|
||||
|
||||
echo " ${DRY_PREFIX}drbdadm resize ${DRBD_RESOURCE}"
|
||||
if ! $DRY_RUN; then
|
||||
na "drbdadm resize ${DRBD_RESOURCE}"
|
||||
fi
|
||||
|
||||
echo " ${DRY_PREFIX}xfs_growfs ${XFS_MOUNT}"
|
||||
if ! $DRY_RUN; then
|
||||
na "xfs_growfs ${XFS_MOUNT}"
|
||||
fi
|
||||
|
||||
echo " Phase 3 done."
|
||||
|
||||
# ── Verify ────────────────────────────────────────────────────────────────
|
||||
echo ""
|
||||
echo "── Verify ──"
|
||||
|
||||
if ! $DRY_RUN; then
|
||||
DF_OUT=$(na "df -h '${XFS_MOUNT}' 2>/dev/null" 2>/dev/null || echo "")
|
||||
if [[ -n "$DF_OUT" ]]; then
|
||||
echo " ${XFS_MOUNT}:"
|
||||
echo "$DF_OUT" | sed 's/^/ /'
|
||||
fi
|
||||
|
||||
DRBD_DSTATE_AFTER=$(na "drbdadm dstate ${DRBD_RESOURCE} 2>/dev/null" 2>/dev/null || echo "unknown")
|
||||
echo " DRBD dstate: $DRBD_DSTATE_AFTER"
|
||||
if ! echo "$DRBD_DSTATE_AFTER" | grep -q "UpToDate/UpToDate"; then
|
||||
echo " NOTE: DRBD is resyncing — normal immediately after resize."
|
||||
echo " Monitor: ssh nixos@${ACTIVE_IP} 'sudo watch -n3 cat /proc/drbd'"
|
||||
fi
|
||||
else
|
||||
echo " [dry-run] would verify df -h ${XFS_MOUNT} and drbdadm dstate on $ACTIVE_NODE"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "════════════════════════════════════════════════════"
|
||||
echo " Resize complete."
|
||||
echo " Active node: $ACTIVE_NODE"
|
||||
echo "════════════════════════════════════════════════════"
|
||||
echo ""
|
||||
+127
-127
@@ -5,229 +5,229 @@ sops:
|
||||
age:
|
||||
- enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBJR3NKa2kxZGRHd3RWdWRR
|
||||
WDkreDcxZzE1RVZ5NFFUN1pLUktzVHZSUVd3CmJkMzVsRVVraGZWVEZLWkQydVJI
|
||||
NGF5Ui9ranRZU1lqS2RKZzRmWGJVYmMKLS0tIGdIRlRMOFBDMnNnT3hZdjcvUVA3
|
||||
cUxXa0tjWWk3K0ZTNUx2OU9MRHpobzQKMZLQH+z8o27s1bAXyJI8HD8jHnU5JaZV
|
||||
WLHwwptGYz7pYyCkWc25IkA0nR3KR4lfHT2o5eGn2BBivpI5o/QQMw==
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBiU2IxZ056SUs3SVZiUTlQ
|
||||
NXZEYmg3VzJzeWJPRnZXcUtOMGQ1MElBU0dVCm5xSURZZ0sxTjVxQnhpVVNEU092
|
||||
bDBHbFAzZ3hzejc5NmM2U0FQTkRNSTQKLS0tIC9OOURPRlcvQXIxMVZXR1JjUWRX
|
||||
bE5ja3FoWXFIZ2ZLTklqWHhMRHBTRWcK2Z4sEcdyvx1rdBIGrbNw7bYvEPXdobvF
|
||||
h0aNihn817DvtVr8eeucj22iYCS40etElQNcO5i68t5wab4/d3rUbQ==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
recipient: age1njap586hc0q43kr03g6c8eqhdsmk8zcafkl3f83xwlc2gqhlmfgs4tmwad
|
||||
- enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBpOVVzNDZvcE1Kb1Zua2pF
|
||||
cjllRWZkTDZUb0JtVW84Z01OOHF0Q3JsZW1RCmswV0lxUW02NEZzY3lWeHNXQ1Ez
|
||||
K0VILzFhemZjTTFLbEpVZVdJY21HTTAKLS0tIHVqZ2E5aU1NVk10bEJ5UXkvMVFK
|
||||
dXdSWEx2OXBSZG9aM0VGNVQzTjVRUkUKYgZjGG7I0ea9I+gG4Ah1VSkiONNAHdDQ
|
||||
5zG9LnKQZ5faY4tWsX7JHFWDX2Y83mXbHV6jMCutAfQG2zt/5psTMw==
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBrN0lNNGtYbE8rRTB0U2hL
|
||||
eUNOZUNFRGk1RzE3R1VjV0Q3ckszcDNHWGxJCkhFN25JbDVSaWJIZ09uNzczbkM1
|
||||
ZWZ1Y2wrTHZ6VzlnVWRoT2JOdDZHM0UKLS0tIHQ2ZE9BaFRCeDNqbldRQkpLclI4
|
||||
dXZqOXNaR2Q2QWJtRlZUa0hLaTh2YlkKeCh3+yIt0sjh2Ig8m/YQifhMDHnTpuZE
|
||||
zHBE74SUyWFmqLgtd1b8Rx5gn9vBcS/3SMv3yKWispg4DZ3bCOeNFA==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
recipient: age19m0m7vdfg86yqy8l5mmle5jdd0unrn3f55t232w8h5ey42cqw34sfpt32n
|
||||
- enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSAxS3VuaHlaVGJObVNPb3Rh
|
||||
UEdBMk9meDQvWi85NVRHR3dXSlhkcFZVa1RBClZDR1MxOUdwVjhzSjFaWVFWelVX
|
||||
cUgzMWNrUzl3L1R6elc1ZTRGamxDU2MKLS0tIDFwUUJFQ0hIUWNTVmFwdGt0czZi
|
||||
V3VGSndCT0tQb2Nzd2w2TWxBbEFBeUEK8kQQVqISb3h0snOtqM0w/mhWEpjIxlhk
|
||||
N84QzgxhD40tvutfO+57HsfOIqjKi4Yhdmbd8+iW9GzGnrbfrDuVGw==
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSB5c2luYmpYcVVxRHROQlpk
|
||||
cTM4cm9uNEtVS21pK2pDRVRySmxYS3NwdGg0CnQvS3g4Q2tSdlpVeWZxbzArcTM1
|
||||
YWRMcDAxcWgySFRqRC9XSElsTVorVEEKLS0tIFd6YjdrY2ZXQVBaZ01wMTIvV2sv
|
||||
ZEk0TG0ybENlTlhpZnovMXh0OGd5V1kKEbjrmKP6Su16KRDQ4PMqi0tHRS6+PfuT
|
||||
0v5DEyi8EDtIYSMzzdF7Jk2C0h5XuRux45Z6E6JhKW5c788dGQHz/A==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
recipient: age1rrxqea6q6pn39sw8y5te63h2py8jgjl9v0jyper86w3ggtn67upqg3ah39
|
||||
- enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBoYXBpZlpjT2oxMURTcW9t
|
||||
NTdNWkMyT0d0SVBCSFBqVC9vNkxYUUk5Wnd3CmlyNUhIa0FmR0lwSFEwK2QzRjlO
|
||||
ajNKMitzZGlDaXpRS1pNNDhhaFdPYjgKLS0tIFVtSjNXSWdlQTVwOEUzcjdEVExu
|
||||
aklINzU2QUFsZmxPaUI5aFR2Z3V0bXMK/rDuNYW2r022tHjuk4KqIEOxqvjQsBIg
|
||||
D2lG4ih5h3idm++gnhCSlP/9tKSaVHoq+BpunsmFObrrWzZRhKI6jw==
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSB1eW5kd1FrSGJaSDkwTzdY
|
||||
SnJhR2hqRkNTTGtSTWsvYmZpSlVWV0xvYkRZCmtqR1FQOTRQR3dGYU8zNFpMdkUr
|
||||
SWN2KzhXaE1rYWZYNHJsOVVvRDY3R00KLS0tIG1aUldSWHlHeHkrcjF2TENvazR3
|
||||
YXJLUXV3VVNSWEdseFlZQUJPQUF6OFkKOPKH52muUha+MV2leWDiqpuTaIj0ZKzV
|
||||
keEdyqkIck6KKfIjphNKTW5ujiuKjGQbRliquBknfCRGMnRlMNfmkA==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
recipient: age1adur9g330gua4l6ndk8cqjg35qc8yxwgme6wrl2hpylcc7vxm38q05ejuy
|
||||
- enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSArbkhEaEhoUk1SUHNjVm5C
|
||||
Q2svcmpselJ1eU5qMVJKV2JWV1J0eUpSWlV3CnhIWDF2NXFMM0VMbDQyeHdkNDJV
|
||||
aForNXJ0UFd0aXFGZzZhekNTdUd2U3cKLS0tIFljclNTa2I5RHJTUU1yMm5XakVo
|
||||
amlHVmZzSDZRTndFSEpjQlVOUTVhRVEKz4Tx0PceUCxw8MsREB2HRPDyC/lIskhs
|
||||
/HWmseMlnsoEMu8E1OIO0TpWY3qowHrdFFr4njpxMJUTXusGvRK8Qw==
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBmTDExNFpCZHZmWnpwYis4
|
||||
bFJGZXV5UksyS01MT0d3cXE2Wno2Zzh6OFFnCm9hT3MyZ0hTbnkzdXk3NmN1TUVF
|
||||
OGtCZnlJRmZJaWJWaE1pMGcrTzJzZm8KLS0tIFE1L3ZxeUl6WVA2UXN2b0lIcTBl
|
||||
YmtXZnZaMml6ejY1Y3lRUFFtTEVMS0UKZbcuunA9GRqctCfC0Fh2foKt7n3l8KkE
|
||||
FNjjxUzL7OQjNG9M1aAtRMMO5KXL7YKXRUJWOPcpiLeLUNvDLeLXvA==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
recipient: age17e89ty6p0fw24daanen57wg8uald9s025t3wwxsw269svwpmgvrshfvfvt
|
||||
- enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBXbm1jWW1DZDVNdnVZOE5E
|
||||
bTRLTzRhckVMNjV5VkMrcllLZk9NeVRiT0NrCkh3RDRicXFTQTNqSmw4VEQ0QlNt
|
||||
cGJNRlFTSjU5YWFXWmM1MjBjVGZSeVkKLS0tIHFPOERUTGVOVSs5U0VMalAwUXcx
|
||||
QTYvK2RScDErVENFdUhpYUVsejBYR0UKCNBN3n4q2X1YLturxpjDv4IlnLXoPtOM
|
||||
DqXMNEgGubagOBhfVFrOjpFxonP7JThSfzEJT/pmL+S7bCvgyosYXA==
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBQYlp5U0FvVkdIKzBGZVRl
|
||||
UEtOOG5kVHVLQ0ptVTVpdG9Lb1Y3bzlOODNzCmNYN0Y5ZGNzTGFGYmxJcWFxaExn
|
||||
NkkrT2RLSlZTdmFiNks3RVZXQlBEYnMKLS0tIDdCVlBxUG8vYU05d3c2MXhjQjhM
|
||||
TTVJUGxlWUxWalBRYzFKQUQzd1R6dHcKPz1ciNICph7qnqSH5WYmOjUCNHsIAmTX
|
||||
lRnbFCZdD/ZIFRK8OYAyCGfQ42UYSTf16kH2jDdTFUy7Fk2+O7lNIA==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
recipient: age1hrx8qj02fj2ea6d4g9vqhyj9hl7fppkjqfdx2l37py3h6pdkr95s8n8rvs
|
||||
- enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBXdDRCMlVyaTdPWFQ5VVhh
|
||||
UWMzUTBkZVYrSDRQWUpWLytDeHVmcVh0L0g0CjVIZE1jZHA0N3RtSmZKVThrcmt5
|
||||
N3VrenFGTS9LUDdZRHBiNDhORE4wa2sKLS0tIGdieDQxQ0lWZnVMZUZzSExKWEJU
|
||||
azl2Q01YdHZacDRSaElpZ0h3SG9xcGsK4s30qRC2eXbuKqPUHfRUJrE8FCMdz5EQ
|
||||
25UhdrspVBadt0G92hHV06Uwm/KKnG4Mi5crLKIMI+HAF+5Uxrh/xA==
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBiemVqWW9LdlV0RjRuRHpF
|
||||
U2MwN3RDeHU5SWhuQ2l4YStRVVE2eU9XSUc4ClNGMkNXQVdkWFkwcys0aFVhZnZv
|
||||
Y3J6RmxwakdaZVo5VWZhb0lVTUdTYzQKLS0tIEQ2NW0yb2lGQ0NLZm1PSWNFVkQ1
|
||||
a0Y1NlYwZ0dnbHhsQVB5eS9kZVdqbmsKgFLUpZMqGywO9Ext5WRwco2bpenVNrzL
|
||||
XrGCqhn/7TW9DrJlPnFe8r4fl4DFP6GWByn7axDl58Qy1OzeBZwnNg==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
recipient: age1e7l8dusgmgfzd2cxrrzwepzjxt69hzqj4epee0cs27u6yg4kxcuqm34ncx
|
||||
- enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBwL2xrRnJyOWxRT01YbHI0
|
||||
OG8vU2xQaEJySkZkQnp6ck90dmkzQlV1RER3CmtGc3duTGRxZXhlbDlHd2hGUDdH
|
||||
QmxYTjBVbkVWRVlIcEhpaml2MlVzc0kKLS0tIFRWTnZGQm94VDhJK0J6SS9oa2FB
|
||||
aGpJUGFydUhvQlpFaURWSm9VS1AzM28KWvU5knwB/ViUrSxbP0zPR4iUE4LXxYi3
|
||||
qraWrbv3jXZwu4Sgv0H+/k6St/Xo5RU+nIqSOCKEy/kpACwk4BhMXg==
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBLRzlzWEcwTUx6M1JNYWgr
|
||||
ZWxnVXhFWkhQVjBrSndGTU9oY2MrOTQxTGhFCjhUdERPc3hzQlkzMjBEa3AvaWVX
|
||||
T0RJWXFFZHdmUVBObGhpQktCQ2NPbWcKLS0tIDN2YkE4bEVtbjJlcndWVmlrQ3ZT
|
||||
dlVvRmt4RklOdUJyZ25ia0ZoYjU4ZHcK5fL1yvRcty6+EMzfSkwwJM3qvrr6h46d
|
||||
d7Zs+EElXrsGDQvL0fevB/zDcICAtudYnVv4nP3Jf8GQnL19zWlwbg==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
recipient: age1jcx3yajjhghn8qh8za3yeu8nxykzlg3p4nrv03vnfvzl0mzayg2qmg940e
|
||||
- enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSB3K2tIY25TZCtReTQ2Zkc5
|
||||
d2w2aGZqYXNBSmZ2VVc5WWtZNDU4YlJpNm5VCkpGUkNOdGVSeXU1WG1YV2NQYjhu
|
||||
c2NKd0czSm5FUWxmNTdHVGVVWGlUMWcKLS0tIDJmRy9vbGJDNGxMVDM3b25TRVVI
|
||||
blpZTkZsTzNrdmFwUGQzR2RIOVJINlkKwa06bE6TmYtXPx0daHadbwL9/u1iYuGm
|
||||
n62kEwTKdyfncODl895qqjWkiA3JnbwQxzsaVDtEGfauzUph9DPY2g==
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBhZElRNXNLK0tVUjQ0ektQ
|
||||
Tzc5NDJReldiYVFFZnRLS3l5dWw3UVQ0WWhJCm1DYVloZGpYeVJtNUNRcEJvVUZE
|
||||
Q0xMcmR0WWdVVTBaWko0dGI5V1hqaUkKLS0tIFBBeXdNY1pFai9QT2t3NWFLV2sw
|
||||
cWpRU2RISSs1UXh3Wkl1dWI3VEk1bXMKP6v3Lm0XZkp3Gn3iJkB0K0Y0E5BeilqR
|
||||
plmPh1rKbT/wkefFwpa8e2Wwmhx80YK+bjn633MH7mLQDf6AiK83Xw==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
recipient: age1sweerhrga9yf8x6sv0apz4ed4g48rnlcq34rpv20t0rcelwgpgeqwvndzz
|
||||
- enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSA1ZlVxYkZ4UFJmRVBHeExi
|
||||
OWZ0akNwc0dMenlHQ2dUSDlERGxSNzZtdFJJCnRYK2cwNHJRK0xtY0NnYlh1a1pG
|
||||
aTk0WURSYVdUY1ZTSUhBNVpZZUFyaUUKLS0tIHZKTFZmS2ZsNlZ6UjZUWmhNK0Q4
|
||||
RC9TREh3ZWZRTzc0WmFkQm5qVlZJcFkKUngIUXCVV1CKpUKfAlal1KjoJwk83mq5
|
||||
1MvU9mjx8Oq/suUlW4axFfAO99mUc9yCwb8TCMUZDhQp802v5FjBcw==
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBMVk8yT1FtMVoxYjF2YVBs
|
||||
UTFVbmhzblBCRno2V3YvaERRN0xWcHd2NVE4CmVyK3p1SWtodk5iWjdkQjJRc0dM
|
||||
Vm8zZHczYlFFSG91QThmUGNaWHBoN0UKLS0tIDZMMXoxbUM0TUVaNFUzRExLc2VM
|
||||
OVFrNGZNL256cjg0Q3ZBQW9xUFcxSUEKv4HqaTcTzNYRJCmNEeUQLFfEoFXyOBEC
|
||||
yZrg8D+ROBn98484bxNeoyOjQdcVn7OYiCv1y6utnyeA9sdMUB0ong==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
recipient: age1f7usptjx9rv4rxauasve200gxtdt9jkqhhdqstlf20wvlm7u75rsjfw50m
|
||||
- enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBRM0hrc0I3UCtBOXJWUm1H
|
||||
MW5QWHAwV3lETFp2TGtPWVZHZ2s0ZHJmQ2w4CnBlQmlTVmEvRTlLMjVXMkRJeWIw
|
||||
QUR3aXpPUndocCt4aWUybndxK3RUN00KLS0tIDJrNkY0d1Jqby9pa0tXWVVuTER5
|
||||
YTlVNklTakFlaGQxdjJqRks5cUZoK3MKkV199oB7lPBzNYd30nSY0J6Cd09ViBA8
|
||||
uZWlmMGKlveBQ98qOt/vDTouRDxedASz0ZGg7+jxUXkUKbujkCYYXw==
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSA1Rkl0UXduS3k3WVU1RzdG
|
||||
N2IybE1TZ1kwVmM4Y2lvUS9DVG0zSkdpZGhRCnFkZnJVS2FBZm9Xa0V4SjRwaGdR
|
||||
TWpaaTh4VEk4cUFrb2pURERzbWtSR0UKLS0tIEsybHdvUWg3UlUvL0lhdHIrUE0y
|
||||
R0pMdnI5QlpIcWVWdGRYQ3JCaElLWTgK5cTfvehj4BPU22tYiR5piZdouUdeh5Gr
|
||||
x/xSbFWqYVauDB6rpzZvhGMI4KCJh2RmDdt7FrlCb/YCB3QoSgvl8w==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
recipient: age17jqc66x9yeshfgd9v78mj483r4zzarqdtuxtrkxe4x5mw679gphshd94th
|
||||
- enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBwVmlDOHh6dUFaR1YxOHJM
|
||||
N09LdWxscStGSTYvMm1QaWRkZVVod1JTTlJnCkp0Z1pSa3lVU2JqUnU0ZnAzWWEv
|
||||
bHc3S2ZjK3E5Z1BRL2pYWklvSFoxSnMKLS0tIGxTd29MK0dRWFFnaGVwa2NVQXM2
|
||||
TjRQdHRZNDFTTWNWR2FvZXhQMVZORncKdAcSOB50bNQsaTGtAqnvjYyFpcjdMGzO
|
||||
NvwI5ZquoT/B37Xsg4NQ7++/ZkemzkuX5VYoXLVAQ9WJwQxewoMFKg==
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSAwL0NNbGlGS3lTZjEydGJm
|
||||
MWZRK0Z5YktuNVNudk84MndLUUdXR0RlRFF3ClB4MWRUTFBWYUdwbityeUU2SlR4
|
||||
azZMb012YXQzWHUxZnZzeVp3MmdzcU0KLS0tIFJ2SUNyMTBEY1NWejdVQ3hyUytQ
|
||||
VnlQWHRDYWZFaU1ON2RQTXBwNFNGYUEKwnRUWCdsiStia1jLHJ6UNP/kgEt4Aw8U
|
||||
pA8PKKu6TZVPI8pw7svtI/uvJuW+8ARuNcyb1FdYmX+sOHnOEqHyjg==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
recipient: age1px0h5l9zp2dww0m8fncrc82kfdmzplsfv2ltat7sna28xpg09pqqcl3s2k
|
||||
- enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSA3V0M5bnBzamdGemU4Y0g0
|
||||
TEdZdzQvUVpYSkozZlRsMWM4MStmN2lxckYwCms0ZTF3cTB5OTRzWWE0a1FXWkhG
|
||||
SGptVCtiOHptV3V0aVNQUUJ4NkE0dnMKLS0tIEEvK0lwa1pnc1YwWXhkYmh3cy94
|
||||
S2E0SUk4RDdSWktQTjZoTkNUallLTVEKNqTmhQo74Q00ZhrSrD+/JB3TwxL9OQdC
|
||||
XSONu6XXzPzfhllNIwe3BnmRVJM/1ru8rvllk98Cvle0k4K5+xBJ/g==
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBxM0FyaFlTR2FzMkhGTzZx
|
||||
d1d3dWNGOG1ESllETEVxRGJlYWkrTUNIdkYwCmI4andLMUUvNVU3R0N2WCsvWWhV
|
||||
UmlkcHdMSFA3bFp0ZG5OckE3YmJrejAKLS0tIDhEVEJYOFc5VWVHcGcvMm5yanBl
|
||||
eGxaNEFxdEN4NWh3cnlGeElBUTJmUzAKvCSJF+LGf1JqEdkzT/Wk6hEzferseyWO
|
||||
vAYR8Z+u6GDwQi7JdFz9iZKD21LAs0IMzd7cZqXKIXVedyXuaIhPtg==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
recipient: age1ufg390ydrmma849t9xfkxxl5xvdkk6mngnlzhmy7mvuaje8sgcmsmnq6l7
|
||||
- enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBKYnFtM29UUzVRYWpTN0w2
|
||||
MDVmQk83THEwR3liN0hmemxBZlF3eDVZV0RRCjRFcjZKQk8yaXViWm0vTUdrSlM3
|
||||
UDNnemoyMG0wbnpzK0FRYXNiRTRnaW8KLS0tIHd1QVYvWWl2QlcxVGZzbFBLbjlu
|
||||
djV4dVpnQzF6LzFiWTREaTYrMHAreUkK82l9Njt3B7HJBLtdLjNSnQ4TDTvBgG9/
|
||||
iUEn3fd9OmvfSGcbp4hhwB7q0UJqb7AP47ctwDOEg/FYeF7eSBvVgg==
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBGZ2M0cG5nMU5nYVliRzlx
|
||||
TVpDcHpwclhXNkxuc2VKanBvcnRxRW04UXh3CmNJeEl3K0FrWWVHQWRwK3Q4Tzk2
|
||||
MEZQbFV1QURLZEhCdEhaMjBRUzllaVUKLS0tIFhSUTJ4SXJwRHhVbkZFc2Rwb3Fh
|
||||
QzAxTnpWSHZaWlF3NUlFMlBHUkRUa2sK3Kim0mp2adSvruI6Necd3UpZHow0EFmT
|
||||
JFOIpIK+3Pc06gsiqrDtDAx2Qtc10Ja4ILTutPlp4va6zzmzrPABiw==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
recipient: age16j42pdc5dr6wnj7xayhkqdj2rny9u68fcqejs50hqq42scssh4gsnrrnlt
|
||||
- enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBoZnVZekVieVVRT2ViWUpN
|
||||
bFk3ckJUU1RHRUJrbkNkTUZkbVEzRGd1VWtRCk03MFEweENBOWtOS3gvM3ZpNTF2
|
||||
WG1UWTM3OFp3RERNNC9iV05aOTZnWmMKLS0tIFZqZ0NhZVhxODMyWENhVlI3SUR0
|
||||
b2doaGp1VHBOUVFacG51Rk02d2o1MWMKnNYktA42DgSWWGuy0XTOtUN9ry49WOBd
|
||||
NaGq27ke16XhQ0ZC+r2a1g7YAFaL+qWJxpaH+0ojhUjti3x9O3pWqw==
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBwTUZ1RzJKcVF5bnJGamlU
|
||||
bWJvcU1sTTJuME94YjhMZWRWRWR0aGJ3UUNvCjh6U1BkaEg0TDlCeTNyUlNTRWV6
|
||||
Nnd0NjE3NmplSWFjVWpSRy9qWjBUbnMKLS0tIGdMZUFzbGdMejZoMlFBVUJXSE13
|
||||
TW4rUHFjZk5oVkw0UGtpaG5mOHRRbHcKFX4ZyNQ0TUeqwXEaz1qmAXQRIOctDkvz
|
||||
TIwxJseAjS3qVxCslnxzgaqmCh8DzCfjrRxLUS/HVTnwOaE1YAh0vg==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
recipient: age1nruncs4l0ufk7yuc4des8p99c0alfndl0lhsws8tycl5pplfp56s30af5f
|
||||
- enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBRTkQ0cm9MSTMzZjF5T0Vh
|
||||
TWlOdVJzZ1JvQWdLTzZsYWZ6djJhTjk2dnpFCmxUZkhrOW5OTWtrS2FLOUdsWm9F
|
||||
WVB2VzhTdGwxNzZIZ2lSRmRielAzZkUKLS0tIEd1RlIwQXhocmJkVWtvVGMvTlVB
|
||||
WWtyVjUrWjYrWDFmc3VtdzJUaEFPeGMKvWONIbN6B9Ims3f0l/lfUTU116TFPXew
|
||||
K6kAvyySbh9Z9JK8Uyp31WLSbqVy56eGr2Su8Hj6rAGIoJdo0v5NoQ==
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBTWStVRDhlblZDMDJ3ejIr
|
||||
MkwyeHRwcnpWQVBmOXY0a3NCU1JnTjkvVGtVClVvVUpLTVhUNGNNLzY4U0o2SDRD
|
||||
bGZteUU1Uzlzc0NjTW56OVNCcnVpUUEKLS0tIC8wN1V2Nkc0cVU3OHNYa1NBVlhq
|
||||
YTV5QUpsQWppNHRpT1dMb2JDQnpoR3MKCoBngnNg5qpktg5j4CcuTUKiooOVI8IN
|
||||
04plE+1lra8q5ZK8q4r1/K8hBM23poyfq+XushO1GZ17dPtWs9wITg==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
recipient: age1k7d2du5mejsmv5rzavm4xwgpthqvcfsehduquv28nzs53zppa3kqngfxq2
|
||||
- enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBlSWtxbHJsQllITit4NVZv
|
||||
b2Z3YzJZTHFOTGU3VzkwWEFZUjdWTC96UjNVCi83SkYyaFdoTVN3eFhLQnA0bDVa
|
||||
OExZRFl1M28yV0VTUC9hZUJJR3dYTWMKLS0tIGNuc00rZDI2dHFqWEc2R1ZVb1E5
|
||||
Vm45YUpvWlMvMXUxdDB0NHpncWpIMjgKcMIZK2ww/VPuuBXlL8klSTm8ySME2njC
|
||||
vbhCzg9VIqIV3q2i8WEbWFmo7uyiyF89Z52xf/3uUc/TUEqjVVDnZQ==
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBUU2ZSazhZNWFZVXk0UXdx
|
||||
Mjh3R1NLMThlaHUvWVFMZEQ4Z0NYOW83dWgwCkVnd2dhdzQrZkxaL1NRbDJodERQ
|
||||
SkM5S0kxcVBtY3laNWp5OXpXZHhqZ0UKLS0tIHVOT2VUZU5DZjJjVW1kcU5qS3FD
|
||||
YXNhRkpaREMzRDB5aWgrdVNvUk1qcWsKQnPqE+5xSBGeDMmLfKB0tEBt2eylBOxT
|
||||
oi26mC+EoO5T2pCIpbD+DODAAlWKLattqCYHneUO9wRk9DnhuEXRBg==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
recipient: age16kqfmvz4e23hmdlqresnyw69ej604s320mmd49h4hm3fhqchtgyqrws0k2
|
||||
- enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBZRkRvN2dBZ1ErZmNhSktm
|
||||
LzNLdzJLR3RhR2VqS0xCdlVSVUJtRnpLTkVNCjFmZlp3U05Vb2FwbDBBdmhjVUJt
|
||||
Uk01dDVzS1lWeEZueDlmcUw4MlFFTHMKLS0tIGJ1bENFSFZPa0tNNFZsblFOM05x
|
||||
THAyY3draFB5eHpydk94Um1qU1BZb0UKUVszMlSqUPsEj3vVseGksI+SEEjEblNo
|
||||
zK5fwkYH0aDubsotkVcJjoXc2ZArHyygIMImLNpRBJYIScDGzzTBRA==
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBZUlFlenlOVDk1NmxQVGUx
|
||||
UHhVK3ZtMHdzaXdtWmE1TXhOekpsTmUwSlJ3CmdncjMxOUw0alBQZkhZZjA0YzJZ
|
||||
bFNVOHUySXRuYW8wVjdHMDN4ZThuQmsKLS0tIGdvUUdHcXlpWlZkL0xBcVpMN1E1
|
||||
NGZCQURYQVY5ZzdDODNNY0o3SG04eDAKiYVuKsJLaSC7/bQRB51tX282/yqpAjWT
|
||||
XNnqPQBQ7H3xuK0tDVdCAUMruOSlqAe9YRZwmiou1irpg9vZYwgTPA==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
recipient: age1arhf2q45zw6wf2uevju4savp575x3m2tfvved5zzq3ay92ynua9s3cm92c
|
||||
- enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBic1JIWi9xKzVkVUtqdHBx
|
||||
OVU1djJDbDcxYTFCSEQ0MTVRaS8zYm5QWFhRCjVNNFE0RUtPTlE5U3pRTlh2eVNy
|
||||
OUU0cnpLNlluemdkVVNjWGxhV1NTRUEKLS0tIDNkWmhoSTNkWExWek4zdjduaFBB
|
||||
UEJjYnZwYVN3L3p5Vk9sRGVQM3dld0kKsFjQKYOClBjWBaU3kCEP5yYGBphUfgOO
|
||||
E1epPU6UI3asa2AC/svfFBZI3u/7EpxDH9jaSSpPec4QTwkheiVVrw==
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBnUUlPS3ZPS1lSVmxwV0lq
|
||||
WWE2QXltb1BzU3p3cFRuMzVieGJta05nWno4Cnk1RG1CUXZjM3dNQzlJK1ZGaHpZ
|
||||
b3drMTB2L3hidkpoTXdMQ05vOUtTb28KLS0tIHhyUllpQnlzL093MCs4YWx4VlVn
|
||||
S3Q3QnpVNEx6ek54RGx5bmlqUk5hbVEKHVUvfk/xKtlOk2o73LodweIivSB4Oii/
|
||||
6dwnE8wa+Qx/OU0vDJVelKOUE6JiLwckX/0u2YIUOybkx49n+tYs/Q==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
recipient: age19mn8zrxl8zpps9yvrh4euquvygpp4fp8queg7xc6qhtnl4ng8c9qx02qwn
|
||||
- enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBEb0VlcGQrWnZHWlM4YjhG
|
||||
NlN6c2I3UnpSdkhYTXg4NlM5aG82K3YzUGwwCkdsM0JLUm03RTk0cm5SSitzR1ls
|
||||
bVBLZ3kzaElDaTRHR1Q3bjdXYm9BMk0KLS0tIEpHNE9MTDJuMDEzV002YTVpT2J5
|
||||
Rm9nem5FSnA3M1NzNi91NnBtRi9XMmMKaGB7uE6HCE2cYHfI1VscO8meeq6O5JPi
|
||||
7bTansMUKCPKhDTBYlSnxxjlDE3DRB8ZdxW7LiG+8iVI7HUbsjTejQ==
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSA3MHlrTTdxYit0MTkxa293
|
||||
eFFIcmRLMUpkZjd3eWdsOXYySEljNDB2NUdNCnAyanVkS0dmenZSa2dpSWZTSkZw
|
||||
Tm5LMkRoWEk3L2ExVktnZDZMM09hZ0EKLS0tIGVGcHFuMjh4OTUxdHNJQSsrR1pq
|
||||
WU1jVG5UN2hMbW9iQ3JmbHlwc2VveFUK4JC32glesv/9nMWBR0lFeX7MPgIt9sIo
|
||||
9HsAkOMTxPTI0NZW8q79+G5hq8qG+QvQhwO6wQtwz/KEHPXntKDX+Q==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
recipient: age1jlltcv5jcnm40z5k0q6hv053k2rqpqvemtuecdwn527uw8uqz4es3x7m68
|
||||
- enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBlSnYrVVFObDRmUE5UN0xY
|
||||
K0ZIMzNtUWM5RE8xMmlhTm5pcHNEemVwYVZNCnVLWGRObkxPcWdaOGVMcExqRlRw
|
||||
VngrNGJFWXBHZGh3YXVBQmtSZXBsTzgKLS0tIFovVG43V0FBbmRJYW1qRGJCRUZp
|
||||
RENmSmFuMmhFV0xHUHB2N2xWbHVvSWcKJXaXCfz3Zh4SRgTJkMuPabOq3laRorIo
|
||||
F4R4bJDkao5L5QWDH1BWSi97kTTLz3QgmsEGPHm/SpODM7Llx7EPMQ==
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBRbEZGNmtlUjJVZ2dlVGZC
|
||||
MGVEQXV1SFRpMjFqWks1Y2F6c29TY25oL1MwCjVFRk90Q2dJRXhUMmIrRVdHVFRD
|
||||
TWgvQS91VWdFWERJWEYrL3VnRXFraUkKLS0tIExUU01VRWlENlpBSGt3V1Izc1dS
|
||||
RFBhSTVKTGV2KzFidDQ3b3lmUHZ3d0UKHWvWO/Om6ahwBkONreGeENqdFG9UI6JB
|
||||
fRA3gRHy98EorEiia1him6lGY+ewYkTtVNo6iEt0abncW5x5xMQywQ==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
recipient: age1ug787sgt6st6k82fgkrug2lzltw4qsukrrqqs3w27ewwqj8rg4hsxcmylz
|
||||
- enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSAyR3VNVisxbnoyQXZGaTJV
|
||||
Y0lMemtYK1F1QjAyWFVqSm9VRzhhbHRCRFRJCnZINVpiS0ZCRXUrVEJhcVlSZk9H
|
||||
TnhaeDU4b0loQjdOazJVbkpEbDZ4K3cKLS0tIGFKRjd0Rkh3RTk2WG9uQUVpWU1K
|
||||
c3IxVmU0Tnc5dnBBWUZhYUgxV0NaaE0K+00bh1AiHdTL3gsA98fvFI2/IDsnWiiK
|
||||
3tOBK4UB5OE5PRupqVXlpEK3ZqKXju/MQbu0/KdaVwHxlL+WXdjAGA==
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSA2dFFpeDU2Y2dtQU12cytZ
|
||||
WUJjKzZ5S2RiZmtvYVJScStYVHI5d2FvcmpzCm9COGZmRllyaElqb2NsczhKRjhO
|
||||
ZGhwZCtoaFcrWmNZbGVHMUNLMFQ3Ym8KLS0tIHJQR1JOblZlY0cydnhNUnBCT1Fh
|
||||
aHJiZVpMU3Y3NXd0ajc0MGMxSGp4MmcKky1oaS8yFydszcvZE2pAL90ZUKoN9MnJ
|
||||
1TdqatDSMtmo3PWkdFeHbWxPjTizQGIcKEcsv6GmO/5KCAJDCHDNMg==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
recipient: age1529taqdwr6t0w7cvzmty0d5y5593wffl0krt48j6uc4u39k56g2qf6ywtp
|
||||
- enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBjSHFHbmU0WGg2a1poMzRF
|
||||
Ry85ZWZCYVdVWkVoNnRWWTZXWWNqQURtendJCndHcmtIaFNPRGMrblJndElMKy9Q
|
||||
NHlaeDVOS2k1OSt1bWs0RlU3a0dJTFEKLS0tIFRrMGJWYThKbVQ5SzdjLzlQbk9a
|
||||
b0xlZGFvTVJGRjZHQk5XM3VObFRnaUkKMavcISNlQh+5yHpA1M5JIkQEF2qasnXH
|
||||
Pd/JWKhnvk3Lyd45ZBJEqFV2wOknJF7v4Z0jdbo4WiUaLh5shqOp2A==
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBrd3QwWE4yaVlIakdiYkNX
|
||||
bFVmR01yNjdWa0d6V29tcWh4M0hWRmQwR3pBCkFRV2gzSVdPQmQ5dXZYc0F1dW1r
|
||||
OFZNTkRwSzRFR0tlUXE2UzFxeU1Ld0kKLS0tIEZ6RlZkMFFiZytPZ05DSEI1Ni9X
|
||||
WEtpUlMycVhkMERsWDRGQ2lrNHdQcDAKRqVAOzQY4ZM1uH7LyZSqX1T6+1PwNhBY
|
||||
5M0RxSuI43E8YKC9axcppvKFJMQI1GAGvgfQxg2J0CEpnmPSVsO5DA==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
recipient: age1zhfyuzlq40reuqlr34gf77852nhs3t6mqfzrqmas8z6sxk7tcfhsungrm0
|
||||
- enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSA3ckNVU2lUQS94RkRGeTFo
|
||||
aWNJRTdpcWxUOWZEOHF5ODIrRkZ0MkpnRndBClREbDNNUW9RZWFSRHdON3pOT3VZ
|
||||
bUlFZnFGTnJpSzhFVjFLMVhKZTNDWUkKLS0tIG44Ui9YL2RPYzJLck1UVVBiRTRJ
|
||||
MWxURWJlVEFBQ3RSZzZFYWJvaW1FVW8KX8/o/4LP+Wp/qOfF4wt7cTt0O+kAZbWR
|
||||
Xu+dthboRmZBV21zzrEvzKGRFBj2T3EMyGDqqPcfX/vuhbVEJJgZyg==
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBvb0xNMloyRW5tTUUvZGph
|
||||
YWxIc0ZhSWI5RHNmaEIvdjZJNHFDM2xBMkNzCldlM2RUeFZCNE9tMVp0MFlWM2sr
|
||||
Sm44MjkrWlNkY0U0eUlyNlE2ODl5SFkKLS0tIEdISElQa2VBZlduajhsV0ZaU3dZ
|
||||
ci9COWE4bldUMS9lSVB1eEk4TG1sVjQKYu9zby36DP+41dTV8sRdtDil3Az40pf0
|
||||
vIhqhCTxQymnJHEwEukiEzM/w3Z85R/W/MbOKyzEZmxzG0a5O5svhg==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
recipient: age1nxlnrevqs2msdatze562vz6ym4pgydt2zndye83lwqauy6flggtqrevyw5
|
||||
recipient: age1k73g8x47hs93wcv7qh92n3htz8pl295g49hyvlrf3570mts0hgys5g04d6
|
||||
- enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBYck0zNDJaMmhLZUxxVUxR
|
||||
OGJqdTBHMFpzb2tQV3NsUVhPc1JTeGxCWWkwCmtWeWhqRXJFNVRoVVBTcjJtWlM2
|
||||
UmQ1OXh3SnJVQWQ2NnloVjRCdUZnaDQKLS0tIGZITFdwcjJFMU5iRHhvRllKaTA0
|
||||
dW1xN2ZhUmZ3VlNzWFprYWNOVG90aVUKH8Gd6gkbDIZydnb1mL0tpujmucLhbPbm
|
||||
sUm8KZpNUfWpP+t7SK5M91HFaAZiqGJOARb31oszwqLGcNDliV40MQ==
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBCZlFtOHI0NnlMeWdjZEsr
|
||||
UnQ0UFJXY1pHTm4yLyszY0dBaldtUkIxdzJvCkNQbk9mN0wzK3BtN2pEa3JSWWFk
|
||||
RXNoemFFbDZ1SlUxOEZMUm9mY1RaZW8KLS0tIEk3bCtyaUVGM3MwNUwxS3QyTjA4
|
||||
YVpTTFlOdzljbmI2cnZUNG1nd1NrVFkKxARm3bpBL685Wky4Y06eA18VvUZRkAl3
|
||||
BBYPJMn3lfDgiQPvfXINfhJI6O+bWyjt0WnfodCGFM6EJLARXiTaqA==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
recipient: age1scfc8p53q5aq2a87tcmsazmj8sfeft0s8kxg0et4nm3ucxyp3c3s6te82y
|
||||
recipient: age1fefy6dk8zn5c3edwmrs9vwx79quftnt784m628t9e34q3ft3cehqz8u72r
|
||||
lastmodified: "2026-07-23T21:15:41Z"
|
||||
mac: ENC[AES256_GCM,data:qFhnPra6IE3wyKQ4WKweON0S0YtD5I0adGZVfA0m6BVilN6bX5oC/1j5NK2oHrsz920hSl0SOF8LrpqOrUyGjSRkPsN4kq8qr9bJcrX4URiktP0oRden5LLt6hf+ZRP7WmRXFqixPkPHJnZIoAvkNnTFce7cDq5NEAHkKUEKG7k=,iv:nyblUDGeu3TUfFivYylOn3C/HITj99qiPI2+mh8AGh4=,tag:FrtRzSCylC4wlIoqZdfx7w==,type:str]
|
||||
unencrypted_suffix: _unencrypted
|
||||
|
||||
@@ -1,22 +1,22 @@
|
||||
{
|
||||
"data": "ENC[AES256_GCM,data:olqZm/3yxtVnlDeYzAWnXjUP9pgkruFEpeS6qONHSfdrj0lJOch44zn1pbG6Jg5QYcksny0rHki9PJEZSbRIyrZ2mSKfTJing3n9M5ODm9xPMbNYn8cpVB4AzZmCSAH7mN4p/98+5hJ1qzopGwCgKTcOOJ9DMUl9AStXORY=,iv:R2K3OehIfyKZXRFH3Y9KYnvEp6CjHPT4Ki2P1YrC6+I=,tag:1FM/E0n9teARAZ1HZM4b3A==,type:str]",
|
||||
"data": "ENC[AES256_GCM,data:JwjmYg0qzoer+8/jv7KOfK/BxmpObHOn04GNsUFQd6hSFBWbQlRbODfluL54hnWQ4gsS9MHgwWh8nrLciBhye05kRQCqADB7Crm0CZYubNb65WzM/devh14jmerc3MYIp1M3VmDisML3x5IqhJkFMSqKM/VaNAC3f0otcVR8TTgZw/fJyvPlfqpPTjikGmo5xg++Yk7Cy8A1Dj70kYk10+EQjQ78jf4k/agLoaS+YvMmKXFk4EljtHg8Fe5H6Rn9nfMSSKGTZeAH3Riix2e+fIH+nWEZiPHC0UlPjJrB3SeBqqGRSUUJshVKHFIdKI1Tq2Ea+GDWrAyIRI0qO8e7WQ==,iv:mr39NxbZuYUDHDw0e+fJDshbt9R1hLgVqkUxUVX/+l0=,tag:u31xZRPfvH5j3Sw5U8/nqg==,type:str]",
|
||||
"sops": {
|
||||
"age": [
|
||||
{
|
||||
"enc": "-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSA3RFRoRW1rRklxbXYybUtu\nL1lEOXJBMDNCN1paU1k5N0hQRTJYQ1dQcTBVCk9OYnloOFZad2toOHo0UXFYSzNR\nZDUwV00vKzVnckMwSE9MMnE4RUozZTgKLS0tIFByL3Z3SzJFeEVNZjhuRDNTMzFo\nNU5tZ2tQU1ZORE5qNm1JUEJrTGdzUHMKypYeJz6BeUUY4aPKazB1nxncOA3DGkal\nfemLr9uwCJw+D3xfXzrIKrI0w3OH7bu2LmWqrNDSz3Bwa5VDuHoLqQ==\n-----END AGE ENCRYPTED FILE-----\n",
|
||||
"enc": "-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBtVW5ENXlkUW83elc3ZnhE\nOEZ4MWIvQWZ4aXhuUmZGYXZNQnhJemsvOWlVCjIwNmFjS3pJOWEwRnhBOGNRamlM\ncXZid1RNYVBXOFZpMlpnU3ZnVE1scnMKLS0tIGFhWFZFd2JPTFNrV1VHSDVKVHRK\nbmhoeFh6YlUySU1ZRVhHbjZnM1IxMFkKo7aHkz2pEeV64m+OEkBZ2V1e+PUzoChu\nUc/Mnh37dNXSSJtg2KnocHdyzDGi1yQbA72xKTxx6QjYJWrAC/jurg==\n-----END AGE ENCRYPTED FILE-----\n",
|
||||
"recipient": "age1njap586hc0q43kr03g6c8eqhdsmk8zcafkl3f83xwlc2gqhlmfgs4tmwad"
|
||||
},
|
||||
{
|
||||
"enc": "-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBqTzhtOHM1Q1lWRXdadHB6\nZ1VrMXFsQWZEbDBMMnQvYnZSN090VnpyQnd3CkdJaHBsbzQ4SERmU0dBS2RZMUpi\nZVYrZmtSNXhiaDNTZkNMVEZud3ZzT3cKLS0tIFhETER5eWJRK05ValFhenRnSGg1\nc1prdWVTVmMyWWt2UkNXWVRLcWRHcm8KjVn1faGmsWiFzcNg4PnxZQfeQONFKz/i\nyJGJc7w2KSZ3La+44jfitMziJQ3AFRUlAhGDX25kWZJ7PtjmFvYwuQ==\n-----END AGE ENCRYPTED FILE-----\n",
|
||||
"recipient": "age1nxlnrevqs2msdatze562vz6ym4pgydt2zndye83lwqauy6flggtqrevyw5"
|
||||
"enc": "-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBReWRLb09reXUxS0YyMUVV\nQlR2bEdYY08rK0RoR2pIelpLVEtYL0pqNUVNCklRaDNoSjBWT20xbk1UdmNqWHNt\nR3JoSlVwODVMUVBzMERUNktHMVViZm8KLS0tIHc0SXRIS1IreDF1VCthb0FBR0Na\ncGFRWTROUHF3clFtTmY1azEwNUpPNzQKsgCND/BZcMTgBTAcnHunQcT6LG1jNrtO\n+W7Yx7bFtFBajWnRYiNpUZPibQJlv5SE9os47WDH1gs86xftgV+uyA==\n-----END AGE ENCRYPTED FILE-----\n",
|
||||
"recipient": "age1k73g8x47hs93wcv7qh92n3htz8pl295g49hyvlrf3570mts0hgys5g04d6"
|
||||
},
|
||||
{
|
||||
"enc": "-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBkVXFQazlvUEx2NzVUNlF1\nN3FQNTdFL1FtSDhCdmNQclRMdG1vRVhvcEhzClF4Ykh1ZS9JekFzOHkvOVphejVy\nd2s0a0IrZ1d4V3diOGNvMlBHUVlwTmMKLS0tIEFhTE13Q2F1di9ESXdJVDhJUVl4\na1kzNDBLcHhlRGg2Y3hSSmdEc2k5MlEKpU2sIv5Pq0LaKkvqA/fRkiB9PcvsSjvq\no4iEFzGNDi63QzXqftWnzengEy/6nWGNA31fzScQftESesGlKfYYlA==\n-----END AGE ENCRYPTED FILE-----\n",
|
||||
"recipient": "age1scfc8p53q5aq2a87tcmsazmj8sfeft0s8kxg0et4nm3ucxyp3c3s6te82y"
|
||||
"enc": "-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBGbllNTlhxNVVkd01GSUNi\nR0lxQWZoRXI5NWxoTExRUnBSb2FPR1BONnpzCkxCN3pNOXVrTW5GR2J0WFhZUE9r\ncm1QTThINDVVVlZEalJiS1RXcHJRS28KLS0tIHV6aGh5QjB0M1VPWDVMT1k3cTBh\nNTNGY25NV0tWSUl3UHlJSExFQVdpVDQK3ARj8xhFRYU6oqYxNQ5+Ryza86ALNUwX\n7yK/8ATnquYC8/ZIYUbgTPxzsocmXX9lVT0+ktIALqjtG8PgNTlOow==\n-----END AGE ENCRYPTED FILE-----\n",
|
||||
"recipient": "age1fefy6dk8zn5c3edwmrs9vwx79quftnt784m628t9e34q3ft3cehqz8u72r"
|
||||
}
|
||||
],
|
||||
"lastmodified": "2026-07-28T06:46:00Z",
|
||||
"mac": "ENC[AES256_GCM,data:xJHyh4JuY+qnKljS3mtfN4Cy0pGPUszhU4ud2L8JxTFGUHTXhZzwQ3s1vM88efLTwM7yDK7oBMBUH2xligAzglDnqkE1T0l7R9k+hyldm4nYqOK6Y8GyPt/aald6oLYGCx3J7Ap0Oqq/crU+s4NiY9enf+R+08jliFyrPBHuRSY=,iv:FGtURIfSrf1tRGqpPfn40GqybhNYFBcvR7n1YkcAMGY=,tag:ir0TU7YuxpPuew+nyC9/eg==,type:str]",
|
||||
"lastmodified": "2026-07-28T16:04:51Z",
|
||||
"mac": "ENC[AES256_GCM,data:9X0dkAEGJiug16LDy/8//QOkHITTzt4zwTKiIU13tRgSRXRpt/7b27+3eDauscXDMamg5wbYo1YF0+VwLl21Ip4icnnwrZBHd6HWleP30HTgef8rmo21SstshnclZz6RH8SEGVDO/vjrMDChaZ3A2WD+nbcomw47DpK3cG3kBIk=,iv:G7CScSD5Z1Z9WMEMydyeGK/RD4W43xA0PlcpmTCxLOc=,tag:DZmby/xkxpinoJztZZRWwA==,type:str]",
|
||||
"version": "3.13.3"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,18 +1,19 @@
|
||||
{
|
||||
"data": "ENC[AES256_GCM,data:4Plrw4SFb7Qx7ofyHwXkHHiCaY+54QT+XLnqoMullnxoTEwf4aZGmjZKx+J5eNnXO/+aPcheg50pe6vXggIgA02a736l3xkIU8mmRdqDSXwROwy7DWBfpcsT5bcL5umGpJ3tnJUAoA8TWVvxoj14J2k=,iv:IwoLgVeWJ+1E1yjeVLjMGEcEDNK0MD1caaP0hcbYJhs=,tag:KPT4mFN2MitDcWAggBV7Mg==,type:str]",
|
||||
"data": "ENC[AES256_GCM,data:VpxnzrdX5FY2gfDcZpqg1f2wqL6mAawcrdE5A9A3fQOKr4BWXmCT/hjQEtt8qG+yVOTApdG+vF8SKDj1AKZ25KsZ253WS7W5C4j8oiT/xHo0CtvVani4fZqN60lB/9381h7aUZ+lcBMPaQcNn35zoQqYthuCGrC9PH1qQgD2VVHr0v8J6kceXKp0KTJldtDsQxky6ROC9Zyc44wwkrFPgPss4/yGaRWWqiojDvK5fK8238hmR8HyfD6Tf5KhgbqCsSGS2g6yt1muUhmegaZwtpi/KpoqBmf1XFaObuHGRPKvblq4Fa0x/sSjpPogBZyNuvRKIwRhla0FkKOH1bX4gC9bzOlgiUUfVcRJT6Nr9jocYuhIv9wvIT4TujGbgHyorYaDSQLFPJoJEUa3fgpGq+AC6p1DJJE3b/wqUSngNbdOtAPE/imaEuprOfnbS7cRx+Ti/Yu+ZYZuAlDtLkSVdXi1hQ4fLQ==,iv:rq/BzePXa/w/Gqewz8JfM/NdU+x4ShiH1OuQT7wRyCQ=,tag:IcO3ddoj4M57CkndHEmNkQ==,type:str]",
|
||||
"sops": {
|
||||
"age": [
|
||||
{
|
||||
"enc": "-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBIaDJyeGJKeDhVNU9xYzhk\nb2Z5K0VTbnM2OExVWWZPaVBoYkNrOVVwbjJBCnBmQlRsdjdPZHZydU14UDl2Nm5G\nUkJWd1N0SXljTW9Kb2VMVXpETGZlS1kKLS0tIEZaWkdrWkVqMENQUmpQOERSY0hQ\ncHU5anBjeUQ0U2ZFYmFaa3ozSmpGWXcKU8QpWAk7Zkc3N/ZmWn5xLsOSTAtMxU5W\nIQOvdDNzIn7Ecoj4zQ91T9puAhaO6PZ1wWjo+rAN0TyWjM+TAwTuFw==\n-----END AGE ENCRYPTED FILE-----\n",
|
||||
"enc": "-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBRdWFEUlRJMlZId2Yvc29F\nNnkwREkwQkllY25vZnRjV3Nob3BvNE1Zd21NCm56UVg5YUFGbEc2ak5MakxDbCs2\nTFVsRFhjVlRwMXdUbGpMKzQ1N2hBUUEKLS0tIEhmdm9hZEJqUEhRN0dsSk80bDY2\nY1JpajNOVFpEejhibjdKZlNHVjE1bGcKycyDfEBZ1WYc4EfAK2y5x/nKRqq7mnb4\nDlpR1Som4bSt7+B+OCZa48mC0Zv05HbP8PnoZdCe83swGNljvRONcQ==\n-----END AGE ENCRYPTED FILE-----\n",
|
||||
"recipient": "age1njap586hc0q43kr03g6c8eqhdsmk8zcafkl3f83xwlc2gqhlmfgs4tmwad"
|
||||
},
|
||||
{
|
||||
"enc": "-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBMbGg2WW1nOUw0RzhaNXNa\nbk1iWmRFWng5SlgwY1V1ZXZIcEp1SWlKYnlBCmJ1Q3Z1KzJoY2ZZWi9UckFOTWZC\nUGxZeUlJZDdRci9ZbzhsdTM4cXk0SjgKLS0tIEhWUy91bTZkTyt2ZFRIb2VKL1Rn\nMm0wN2RjUUJ3dHAxQU1nd0VRTGlQbUEKsb6AFoMZw+FxZW1A1rf4YUlNaODUgbS3\npZfGZIocS2y9uMTaXWQ56Y078zDD/F40wLYsuknm0ECVfNysSVacVw==\n-----END AGE ENCRYPTED FILE-----\n",
|
||||
"recipient": "age1nxlnrevqs2msdatze562vz6ym4pgydt2zndye83lwqauy6flggtqrevyw5"
|
||||
"enc": "-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSB0SGFVbXYycTE3amR3ZVhi\nVFJHVVhWNWVQQlJKZUJvZ2p4ditqdnYrWnd3ClZaaUVLUXFKaWlDdDQ4QW5SQUZY\nbG05c0RGNnZUenI1QTBlemVjT0szRncKLS0tIHBWTmNpd2FjWHc3MmR3cWt4SCtH\nWS8xZnVON1lGZVc4YzZFRmRBamdSVlUKdlQTlwiExTUVsiY05MXFG2/IQt1bZwKU\nlFDnuy6YCQzawBLQZuSUAp7WSGedupexlZKQKNUzPf+dis26XEYZoA==\n-----END AGE ENCRYPTED FILE-----\n",
|
||||
"recipient": "age1k73g8x47hs93wcv7qh92n3htz8pl295g49hyvlrf3570mts0hgys5g04d6"
|
||||
}
|
||||
],
|
||||
"lastmodified": "2026-07-28T06:44:56Z",
|
||||
"mac": "ENC[AES256_GCM,data:N95kOsHFH2nKVhr79EB/zILMpJ0rgh+QZ1weYbQJgvxRYjLMl+AK/ALjtx7Mrj9ufOnIWnndrUpD7X2cod2oPrlAPT3c8P0IY8LaJTNDx6TUljKHFe9Nvm/JcnI9i5skioY0X8sRC1oosL0loVTmVb9yaf5xsf+wursIsj7pI3A=,iv:kQLx4FLeiKZIevOOOPS3NAu2NUUhnRw/jpws3FZ6zcM=,tag:rCp0h+wujci4N5VFuzCuLA==,type:str]",
|
||||
"lastmodified": "2026-07-28T11:53:06Z",
|
||||
"mac": "ENC[AES256_GCM,data:PMsPSffQkxRmqT6HwFezIoT4WKJ9WDlAcflWynsFLtzbqVTREN0YGOJgMwLd0yBR2sbGBWogG5xAuKjO16aiWUVpCFPRvP97um351679ZoXIlrOL+GlFn4NKwoLH9/eCp8BteB9mv5huhEEN631lqCr+jtYO6yS0ULk6QS4+7/c=,iv:oYWV4J1JUuGryvqntVXkj/HOzRDRr9E2C1RKNvr2bT4=,tag:e4p+I40y1LrgdlK9tLRicA==,type:str]",
|
||||
"unencrypted_suffix": "_unencrypted",
|
||||
"version": "3.13.3"
|
||||
}
|
||||
}
|
||||
|
||||
+11
-11
@@ -3,22 +3,22 @@ sops:
|
||||
age:
|
||||
- enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSAzeEVTTVZ1ZHdVNWNNUW53
|
||||
amN2d1NhQ2lydVNJUXdyaG52ditNTHNJZmdZCnljakFpaW92a0xEbjNZSEhXdDcr
|
||||
ZUN1MmN2dlA0T2lYTVNibjB6UExqU1UKLS0tIGliYmVicTBrWmE3dFZWVzhwUm9r
|
||||
T2dUUUdUQUVzTWRvbzlWSExKRUorTGMKktkw3uuydGeChy+9pxtysHLvssZ5LKoK
|
||||
8i+CNr76/nyKCRns6tHj5lHYefY/B8IpdmlQ4fXQKSrna/dDjkdWTA==
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBLT0oxSDR6WHJGT0VTY211
|
||||
d3hYcU42SzRIQzhvdUxWSnZhRitoNGFlclVBClFEZ0xXVnU2OFV3TmRXZHJyOVp4
|
||||
aDZjQ0g0RlV5bnlzU0Y4ZnYxWm4zK1UKLS0tIDJYZ1F6OG8vdUxnS2NuaG9SL1gv
|
||||
bURjNjRWQXIvQlNyVVZieGpCblV6bmsKQrCQQz7b4+w2WB3E56lEZoZa8/Yx8mn5
|
||||
vWtJa6nvqlE4miJdRpbjQWUeFHghr0A6FsYDg5yf+Mcpf2otTw8psQ==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
recipient: age1njap586hc0q43kr03g6c8eqhdsmk8zcafkl3f83xwlc2gqhlmfgs4tmwad
|
||||
- enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSB0dmd5ZkdDcEM1akJna3Yv
|
||||
RnZFYVJNZ1IvY2R0QnZ6WC9PQnFmVkdWNHpFClVJWnNaRFV1UVBQTWlXb1NEZEVp
|
||||
Z0ZiUC8yTXVyMFdmRnB5OWdwQVVTSUEKLS0tIFFxLzdyNlZ2WlFPdEJFTTl0Mnhh
|
||||
TXVMdVIvKzRTeGRIRXBqTUNxM0Z1cmsKMLHPN6n2i18LHkgjinq033qZsrt7BdFG
|
||||
NuQA4EbCqg5uiHPCSGU6/EVk2XmfROrJz5eyMWIwrwCbt53jsUNTFw==
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSB5cVFzS3hFVkZpcGRWcUc3
|
||||
YnRqVWUrSU1mUGRaaWhZTFQyRGRLMm9BSWdFCnhzYVhtWExGMFlRdU40K05TV0li
|
||||
L09BK1h6SFlWSGRtQ2F1QW9iM3lDVWMKLS0tIGNQSE9aN0xHNytlOHBzM3MvMVpB
|
||||
VEdyUFc0WWgzVEswRG1XSyt4dThTWGMKRkR+FifFDDKzrMrlJgZj/lbJDjfHsvjI
|
||||
BGfJj1x0whWiozg/SZ6BsYYYlVkcCBwVTmPYx2iQHk4jLqHcMF36dw==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
recipient: age1nxlnrevqs2msdatze562vz6ym4pgydt2zndye83lwqauy6flggtqrevyw5
|
||||
recipient: age1k73g8x47hs93wcv7qh92n3htz8pl295g49hyvlrf3570mts0hgys5g04d6
|
||||
lastmodified: "2026-07-28T06:44:32Z"
|
||||
mac: ENC[AES256_GCM,data:rtDzbONqaNpbEKo1btGiHArAJVCVsStHhgvM8MTgkRMUNGU8p9W4tiuQLmobDpFh81kj/mDnkx4cCpScuiMhQojz9H74l7P9o9b1Wxy0fSiCtGoQhaq6JoIVJoKtEZoHfMStdIio5EBW9DBB5abvjPLTaC0afgHo5pNn3A9qr6g=,iv:SqtJhel/5qW0V7IV6V8dCRWqPnsK6QCzbC15/BKeq1k=,tag:vF7RJJccm0q7BSyNGYnplQ==,type:str]
|
||||
unencrypted_suffix: _unencrypted
|
||||
|
||||
@@ -1,18 +1,19 @@
|
||||
{
|
||||
"data": "ENC[AES256_GCM,data:Gi6HP0EuWQcTESaFOEq7wgAvuvlyevuFIKAnShDXJIUBg7xNNlGJ3PmBfx+YyFvChCqOqThAS8TxStogMVQqyWxm0vtCoVshkHG+9Q9xHV72AiuTcmJdsZ0DiSf9jwJYLZ80x1VI9a++Mqg0yLs5few=,iv:H9YYUQ4nOhQTBaonA/H7F7KroMRyU7kGul/bL/AnQ2Y=,tag:7ESjzSEH+XTqvvtxpinKJg==,type:str]",
|
||||
"data": "ENC[AES256_GCM,data:Ea5AGwloFMyRgmQhJaDNTW19mpDvx0+dSc3ibENEXEniEIBV4Wv135mAwE6jQPbN8wF6f5ki3B5mDXHOMuhDQEqUXg6jcs4uEm2nwz7wAKYXEX4T8p0gUj7FPISiSQrw8O3jwzt4qF7AVcPsNhMtZLgrx9ssMtVmGct5do+E61Y7dFe0XbpJLynfzeiMKYvaBngTbHwQjv1mn9AMpX98apsp3tyzg8avLDR+WXRcMusds2mdov0jCoKuWPLG/srhMwjKOp1Z1Tu+aZCr9lHTT76mytpGIWGSbrP5l3UTp3oPLmy87Kwi/FJStZYdYikMnvmSo5qovePiYsLKVD74dS+d3oFGARyW4TILd9OQdXEjzdDRak5HcPv2/HhR638kvukVMdEdH0hpeJcqcV0h+BoLyIHe1e037iF1ZkCjuXE+A8LtlzRR0wqP202uzBIyg21ca4X5bRiHkkAOX4P3riaEcMgKfw==,iv:IXq4sHHsjnK6maVd3RRVZzhnxx1hOnxfsvYX8IKLn+4=,tag:jRZinAVLH8nGVf3gE4aSKA==,type:str]",
|
||||
"sops": {
|
||||
"age": [
|
||||
{
|
||||
"enc": "-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSA1QVJsNHBYaUt6WWFDRjFX\nYVkrMHRtd1pTVFJsano1TG9hVyt4NGZyUEY0Ck1RU2U3MTQ1YTVkaXVhNlM5clg5\nWDZKZ1duTHZrM3FxWldnVXB1alJBeGsKLS0tIC9wUmZWcG1yLzBIL05qQ3lVV0xp\nS2lhWWVaTU9NWjd0K2hEaTcwNWE5YjgKmy3SHvYur/2sn6PjVLZFYzHnVMTFe66r\nm7MsWeUXU6zs4o4RWPPNBrpsVXIGZJx5Gbl+lp9Zubn61Wt/KVqyfw==\n-----END AGE ENCRYPTED FILE-----\n",
|
||||
"enc": "-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSByOTkrVVd0TXlBRzdkTktO\nalU3WlVOb2ltc1k5RG5sRE9EN2M2WUlnQVRnClFGZFYvSHdpVlhtNXExSXZMZnU3\ncnl6aHRzU3k2V3R3dU9RVDA3VWp3VkkKLS0tIHcwREM1dDREK0h0UFp0bU5JTW9Q\ncWdyanNMVjY0SnZUWUl3Y2R2SXNyc1UKI1FPIE66to19oQK2TwM5B4snGOVMLFfx\nl6JMEilDVSgy2RR/tSiovmo4NwOsv0hmmF1t34547rbZb+dB5KrMEQ==\n-----END AGE ENCRYPTED FILE-----\n",
|
||||
"recipient": "age1njap586hc0q43kr03g6c8eqhdsmk8zcafkl3f83xwlc2gqhlmfgs4tmwad"
|
||||
},
|
||||
{
|
||||
"enc": "-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBRNG5SQ0FvWWVvaGhOTmI1\naTE2anlXbTJDcHVBdWlxamlBam9jQlVyL2kwCmVJMkx2NnU4dno2T2dQOGUrRFRp\nQnl3TkJ2R0ZOS3Q2K2g3VDZQZ2VWS2cKLS0tIHgvdjJ3WURmcHVSQytHUGlwSnpm\nc1k2UHZORWt5TlJvaVp0YlpzWDdob00KKJ4bSM2CQD675b0ceEv9tz+wBtCIx/wz\n7Nr2CJoY68cB8KjM1eMzioZI1Rlt49tTQuD4XSs2/9Ej/dDDbP8p6A==\n-----END AGE ENCRYPTED FILE-----\n",
|
||||
"recipient": "age1scfc8p53q5aq2a87tcmsazmj8sfeft0s8kxg0et4nm3ucxyp3c3s6te82y"
|
||||
"enc": "-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBnbWp2UWphSnFibnA4N0ZV\nRHlNUnBNYUNzMzNZakFVc3JXT1d0R3Yyakg0CkowMFlOcEhPcHE5dXd4ZUd3cDAx\nbjQ2T1NqejNRMnFvbTdNRGFGdW1pM2sKLS0tIDhGb1JBQ0RFQ3NxbmpkdXg2WVF6\nS25uZW1hMVo4Q1JGdGNQWWRyWE84b0EK0FmFZB6sN7uZynhzYU932x461zSZIUWU\n8oyDEKNoUuPqpWK8RktgjWlKHtB0oLXC+SLAOJEhtnhWF+GSqloGfw==\n-----END AGE ENCRYPTED FILE-----\n",
|
||||
"recipient": "age1fefy6dk8zn5c3edwmrs9vwx79quftnt784m628t9e34q3ft3cehqz8u72r"
|
||||
}
|
||||
],
|
||||
"lastmodified": "2026-07-28T06:45:59Z",
|
||||
"mac": "ENC[AES256_GCM,data:13BJaaM1Zrcg8/SZToXdlEM8J4GtFZnkoVt6vBWoeLU58wsGi58jdEAqMzWpnT3Vf2Sb8OjNUB1cbBVv9uMitbffFR27y06diKkAICnVc4shS0czoO/6u9O0k9gXicU04jUr/FKLL+sSiNi2ypCFSZT1C3WY44sTB15RNYQR7XQ=,iv:/4owaf5HYndcojBy5rZv6LZp/ipe3UuXy34dq3XI4Qc=,tag:i3Qk8jdl5hUmLScLCirm9w==,type:str]",
|
||||
"lastmodified": "2026-07-28T11:53:32Z",
|
||||
"mac": "ENC[AES256_GCM,data:EhWEEIamG62xHI6SqO4tzdK6gcEXxUU/UGcF25m+X11kUVXzHd6yLcfiM2DvqcwGLmwmOOf3SpFLABlQUzTka4QRzTMFIAJi7x1rzQV4J0YIGITHk9rgvy0V50TI0loONBb2du+vDt8IlTBNvF9WMbNCki+fHPAjIemyoRBdubo=,iv:wfsx0H4fD3L7WeGzz57SzZtajBr/xTRDljSpz22DD7Y=,tag:naDyGH0WTKkqgd954Ya8jA==,type:str]",
|
||||
"unencrypted_suffix": "_unencrypted",
|
||||
"version": "3.13.3"
|
||||
}
|
||||
}
|
||||
|
||||
+11
-11
@@ -3,22 +3,22 @@ sops:
|
||||
age:
|
||||
- enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBhdmNmSFlmei9KbDRWU3lj
|
||||
TVJoSW9FRE9sWUdBRkNIMTB3aVNERUp0U0c4CnZUUzIvYm1xL1JNMGFKcWlBK2xT
|
||||
SCt4TTdlU0ZralArcUVFZEh1YTRQaWcKLS0tIEtKbnBZVlkrU2U3VzlycS9odXVi
|
||||
VjhXeWtZdDhpTG12cUZwVDhKeWZPMmsKOtXLVB8Z0faO05BPCVQXBn8gbMOZeHxC
|
||||
sHmt3vMWO0jrxM0WLql59VWAFsczNwb1fw89NdZMfd/XoVq2EUV8dA==
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBlMWJCTUNSVVA0Rk84VWl5
|
||||
emJ5ck0rMTFRMmljUmVOTlpUT1dlM2VFekZnCmRZbWcvL214YjUzakNDYkovQm5q
|
||||
cVJQZnJvVGpIMkpqUnZEenkxZHNDSkkKLS0tIE5tUFl4a0FtZFNNTXFudlFNbjcv
|
||||
a1dEbnl1NTR2T1lWN2wweVZ2ZnErZEEKbn9bLqIsMwSGKBjR97kMfFDuh1l7Hu5o
|
||||
zdLjnAHzCEAsTW6V6wPB1rvi++FO04lqq1RfEiWXJkB2KWSRf+jdTw==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
recipient: age1njap586hc0q43kr03g6c8eqhdsmk8zcafkl3f83xwlc2gqhlmfgs4tmwad
|
||||
- enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBucCtBR3N3Qm9JbTlLdTZT
|
||||
eTlxWEsxUWZsUnMrWi9zSDdRVWZWejBUWFNVCnpyVEVLbTllVHg1a2RXRkVRZ05z
|
||||
ZWlsSTBDWFF3bzkzaTFMVDQrVW1yczgKLS0tIFhKMll2OVdzWTcyeWdoaU85Vmla
|
||||
WWEwRFgxNDF5czA3Q01LZ0NjY2toaTQKKawEiHUWss0NPuJng6UimekEOlxipiyC
|
||||
ielJbLxma8WPLeLwBukFOBOTyNFxgsVBvwmBJ2k1iMIg1lCjhLD7RA==
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBDYUhMZWZCVS9MbnlscVBQ
|
||||
aENENGFLTjJ3MVg5SGJCeXZsNVZ2ZFM4aVZZClYyRUhYTFZENEkvQjJqNGp6b3Q2
|
||||
SWthQ01EMDdQYkVBQ0ZRdmRIMTFnUE0KLS0tIEJHV2YzRmw1MDVsVWhHRW9LMnVH
|
||||
cU9qdUQzcjlNWFdQZmVydHdpamdQQ2MKndSN6wQbwyGqnyS819tHJRqDLRryXonC
|
||||
CPuBYrTwxosRPwgh+nyJto4lj1uZqCmFVMXGjn87em8ykY5arp9o8Q==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
recipient: age1scfc8p53q5aq2a87tcmsazmj8sfeft0s8kxg0et4nm3ucxyp3c3s6te82y
|
||||
recipient: age1fefy6dk8zn5c3edwmrs9vwx79quftnt784m628t9e34q3ft3cehqz8u72r
|
||||
lastmodified: "2026-07-28T06:45:50Z"
|
||||
mac: ENC[AES256_GCM,data:f50JYTzMJiEjB9+fNIs734CXrkN1IRgJKcoJ0GHoI5p75cfLfPxuzUifty9kQ/4DbM7LbWa5w+9occoV/yNOa86BKpPtU0LmlzXf904/SgOCUKhyHqhyO5mp8w71GoyLxx3ld8CbqfBu8t5l4gwbQamftsmQRxnUdU/Oa/CzdZQ=,iv:EewXBjeihw89h8ypH5u/G4YfoKZzkHqRWgSSIXU3/a0=,tag:lEWMvAkebMexDGGpE+wqkg==,type:str]
|
||||
unencrypted_suffix: _unencrypted
|
||||
|
||||
+17
-5
@@ -40,17 +40,24 @@
|
||||
# from scratch.
|
||||
nixCacheHostKey = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICuHUxGNH6ei3BZD+EfZs3l4X8uJNcjQiOsM/G4yo4O/ lxc-nix-cache";
|
||||
|
||||
# Beszel hub's SSH public key — used by every agent to authenticate the
|
||||
# hub's incoming connection. Update if the docker host is ever rebuilt and
|
||||
# the hub generates a new keypair.
|
||||
beszelHubKey = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFPR9kwtC4TAeTRu46A7+opZsYpxqkRJ+x/ZyB2GWCeG";
|
||||
|
||||
# Public keys authorized to SSH in as remoteBuilderUser on the nix-cache
|
||||
# host (modules/nix-cache/server.nix) — one per client host that's allowed
|
||||
# to use it as a distributed builder.
|
||||
remoteBuilderAuthorizedKeys = [
|
||||
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFDEA1S2ikpObREgbP5uVBWMxIOGbY8B+Wx7VTZK1m6t root@server"
|
||||
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPAYIT9ormlmxZ0SziyDQaUntnKI8HK9/s3Qac1ZKjP2 root@docker"
|
||||
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKKKzoEPl/ZW9KBRHBcp6/ThOngGpwMv5EhkTlgC4aDf root@nixos"
|
||||
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIGtOWOCS+ImHc7NehguoyD7PbonGosKMZqc9+QR3v/h root@nixos"
|
||||
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHxXTQxFnArK5HXG7czeoybZebCGfxpUdusJkPn+BCSp root@server"
|
||||
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIK+ioWPhHixlgCB9KIQ0QTHTz6A+Oo2F3uKiINLip5rO root@docker"
|
||||
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICMJhrfFayLBG+gWtO6oAvgambw5nWWgztiTFEaaaVRH debian@surface"
|
||||
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJEj26SL/emsVjW2YhRucJVp2kTz8WgcEQgjBEBLRikk root@claude"
|
||||
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAII/rLceRhnDobVXQYiPceuhDHHvVjFQ1pc9A6un/eUlA root@server"
|
||||
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHj11bLPpRzH2oslnwFzEvY9cSgfEFtSZbLQaDm4nZMK root@pxe-boot"
|
||||
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIH4/Sesm8NYpj73R0cbGhI0Ubvz73vIVWAnbEDTlBTdh root@tor-relay"
|
||||
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIBCFUUtePndW7pqtlawft1QCdHmBVs3O/c8EJO+RcXV root@tailscale-router"
|
||||
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBZ9WKKAlP9Z7GQdgaZ1Xgw9C+vja2lqEZO5rJFpVqYN root@ha-server-1"
|
||||
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEGNKlaaMckd8nLWNGz4B2QokXjnnIvM+rEUv+R6h0sp root@ha-server-2"
|
||||
];
|
||||
|
||||
# Admin SSH public key, authorized on the primary user of every host and
|
||||
@@ -108,6 +115,11 @@
|
||||
haStoragePrefixLength = 29; # storage subnet prefix length (/29)
|
||||
haStorageRoot = "/srv/ha-data"; # XFS-over-DRBD mount point on the Active node
|
||||
haIscsiIqn = "iqn.2026-01.home.sweet:ha-storage";
|
||||
# DRBD backing disk — identified by SCSI controller path so it resolves to the
|
||||
# correct block device regardless of OS-level naming (sda vs sdb can differ
|
||||
# between Proxmox VMs depending on disk-add order). drive-scsi1 is always the
|
||||
# dedicated data disk on all HA nodes; drive-scsi0 is the OS disk.
|
||||
haServerDrbdDisk = "/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi1";
|
||||
|
||||
# Storage
|
||||
storageRoot = "/tank"; # ZFS pool root on `server`
|
||||
|
||||
@@ -1 +1 @@
|
||||
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILdvVsj5cw/y6s2q5cPqKnULgnkEnsIQahrhm9DyCdd/ proxmox-ha-server-1
|
||||
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIN7IABG7x9ejHSENmHsL/BF9MQadGuBnHZa1eUEFMYnH ha-server-1
|
||||
|
||||
@@ -1 +1 @@
|
||||
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILjgkgUFZfpTLkFQeEbqXlvktE641EexWLv2I4iVKyQi proxmox-ha-server-2
|
||||
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINWuFwPKW2jwQEj6c//xb95DbtandPkJ7M0ceyTNd35G ha-server-2
|
||||
|
||||
Reference in New Issue
Block a user