Compare commits

..
Author SHA1 Message Date
rootandClaude Sonnet 5 5ec7033439 Fix stale documentation: outdated counts, missing build type, spec status
Check NixOS configurations / eval-hosts (pull_request) Failing after 11m26s
Same class of problem as the deployedTargets/README fixes: hand-maintained
prose that drifted from reality and nobody was obligated to update.

- CLAUDE.md: "18 hosts" was a stale hardcoded count (actually 20); reworded
  to not need updating as hosts are added. Also added the missing
  tailscale-exit-node build type to a list that had it everywhere else in
  the file except one bullet.
- AGENTS.md: same missing tailscale-exit-node build type.
- docs/auto-installer.md: the hand-enumerated lxc-* list was missing
  lxc-tailscale-exit-node.
- flake-target-refactor-spec.md: added a "Status: implemented" note so this
  completed historical spec (referenced elsewhere purely for rationale)
  can't be mistaken for an open plan with unresolved Open Questions.
- remove-sensetive-info-refactor.md: the "Definition of done" checklist was
  entirely unchecked despite most of the work being done. Checked off what's
  actually done (sops-nix migration, history scrub just performed, the
  pre-commit gitleaks hook), and left rotation of the GitHub PAT found in
  history explicitly flagged as the one still-open item -- an operator
  action against GitHub, not something this repo can attest to itself.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-20 11:06:05 +00:00
beatzaplenty 9133afd444 Merge pull request 'Fix duplicate-host check reporting false SSH failures' (#8) from worktree-fix-duplicate-host-check-exitcode into main
Check NixOS configurations / eval-hosts (push) Failing after 11m36s
2026-07-20 11:00:18 +00:00
rootandClaude Sonnet 5 eeec9ce302 Fix duplicate-host check reporting false SSH failures
Check NixOS configurations / eval-hosts (pull_request) Failing after 11m38s
The remote bash script run over SSH ended with a for-loop whose last
statement was `[[ "$n" == "$target" ]] && echo ...`. When the last
VM/CT checked on the node didn't match --host, that test evaluated
false and became the exit status of the whole remote script (1) --
which the wrapper then misreported as "couldn't reach the node",
even though SSH connectivity and the check itself were both fine.
The actual signal is the script's stdout, not its exit code, so end
it with an explicit exit 0.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-20 10:55:53 +00:00
beatzaplenty a62c4fc023 Merge pull request 'Stop tracking deployment status in the README Hosts table' (#7) from remove-deploy-status-from-readme into main
Check NixOS configurations / eval-hosts (push) Failing after 11m20s
2026-07-20 10:51:59 +00:00
rootandClaude Sonnet 5 97ede62f6d Stop tracking deployment status in the Hosts table
Check NixOS configurations / eval-hosts (pull_request) Failing after 11m24s
Same problem as the deployedTargets removal, just in markdown instead of
Nix: which variant of a buildtype is actually deployed is live
infrastructure state, and a committed table can't stay accurate as that
changes -- it already required a manual edit on every migration and had
drifted before. Keep only what doesn't rot: what each target is for, and
stable naming history. Point at the live node / /etc/flake-target instead
for actual deployment status.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-20 10:18:33 +00:00
beatzaplenty ab5206b1c7 Merge pull request 'Replace duplicate-host check with live Proxmox query; drop deployedTargets' (#6) from fix-duplicate-host-self-match into main
Check NixOS configurations / eval-hosts (push) Failing after 12m7s
2026-07-20 10:12:59 +00:00
rootandClaude Sonnet 5 2041557ab3 Replace the duplicate-host check with a live Proxmox query, drop deployedTargets
variables.nix's deployedTargets was a manually-maintained list with no
enforcement keeping it in sync with reality -- it caused two separate
false refusals in a row (naming a VM as deployed well after it had been
destroyed, then matching a target against itself once the list was
"corrected"). Static files can't track whether a resource still actually
exists.

create-proxmox-resource.sh's duplicate-host guard now queries the
Proxmox node directly (qm/pct's own name/hostname config, matched
against --host) instead. Also fixes a gap in that live check: it
originally swallowed ssh failures and would have silently treated "can't
reach the node" the same as "checked, nothing there" -- it now refuses
instead of guessing when the node can't be reached.

deployedTargets is removed entirely from variables.nix since nothing
else in the repo consumed it once this script no longer does; README.md's
Hosts table remains the sole source of truth for "(real, deployed)"
status. CLAUDE.md and the script's own --help/comments updated to match.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-20 10:03:00 +00:00
rootandClaude Sonnet 5 2fd483697b Don't refuse recreating the canonical already-deployed target itself
The duplicate-host check in create-proxmox-resource.sh compared by
hostName only, so it fired even when the target being created was
exactly the one variables.nix's deployedTargets already names (e.g.
rebuilding lxc-nix-cache after destroying its old container to pick up
new sops secrets) -- there's no other machine at risk of an identity
collision in that case, just the normal redeploy workflow. Skip the
check when dt == flake_target; the later VMID-existence check still
guards against clobbering a resource that's actually live on the node.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-20 09:49:31 +00:00
beatzaplenty 89186b0dee Merge pull request 'Track nix-cache real deployment as lxc-nix-cache, not proxmox-nix-cache' (#5) from worktree-nix-cache-lxc-migration into main 2026-07-20 09:46:41 +00:00
8 changed files with 115 additions and 70 deletions
+1 -1
View File
@@ -7,7 +7,7 @@ servers and workstation.
The flake exposes NixOS configurations named `<platform>-<buildtype>` The flake exposes NixOS configurations named `<platform>-<buildtype>`
(platforms: `linode`, `proxmox`, `lxc`; build types: `minimal`, `nix-cache`, (platforms: `linode`, `proxmox`, `lxc`; build types: `minimal`, `nix-cache`,
`server`, `docker`, `gui`, `pxe-boot`), generated from `modules/platforms/*` `server`, `docker`, `gui`, `pxe-boot`, `tailscale-exit-node`), generated from `modules/platforms/*`
and `modules/build-types/*` by the `mkTarget` function in `flake.nix`. Not and `modules/build-types/*` by the `mkTarget` function in `flake.nix`. Not
every combination is built — `pxe-boot` has no `linode` variant. See every combination is built — `pxe-boot` has no `linode` variant. See
`README.md` for the full current target list; treat `flake.nix` as the `README.md` for the full current target list; treat `flake.nix` as the
+8 -6
View File
@@ -62,8 +62,9 @@ There is no test suite — "correctness" here means the flake evaluates and
sweeps: after editing one or two hosts/modules, evaluate just the sweeps: after editing one or two hosts/modules, evaluate just the
`nixosConfigurations.<host>` you touched (plus any `config.system.build.tarball` `nixosConfigurations.<host>` you touched (plus any `config.system.build.tarball`
/`diskoImagesScript`/package output affected) rather than looping over every /`diskoImagesScript`/package output affected) rather than looping over every
host — `codex-maintenance.sh` evaluates 18 hosts plus every package/tarball/ host — `codex-maintenance.sh` evaluates every `nixosConfigurations` host plus
image variant now and is slow to run after each small change. Reserve a full every package/tarball/image variant and is slow to run after each small
change. Reserve a full
`codex-maintenance.sh` run for changes that plausibly affect every host `codex-maintenance.sh` run for changes that plausibly affect every host
(`modules/common/*`, `flake.nix`, `variables.nix`) or as a final check before (`modules/common/*`, `flake.nix`, `variables.nix`) or as a final check before
committing. This is a session-workflow preference only — it does not apply to committing. This is a session-workflow preference only — it does not apply to
@@ -93,9 +94,10 @@ Beyond `codex-setup.sh`/`codex-maintenance.sh` above, `scripts/` also has:
(`--force-rebuild` to skip that and always rebuild), and probes (`--force-rebuild` to skip that and always rebuild), and probes
nix-cache's substituter/remote-builder reachability once up front rather nix-cache's substituter/remote-builder reachability once up front rather
than letting every `nix build` call retry against it individually. than letting every `nix build` call retry against it individually.
Refuses to create a target whose host identity already has a real Refuses to create a target whose host identity already exists live on
deployment elsewhere (`variables.nix`'s `deployedTargets`) unless the node (checked directly via `qm`/`pct`, not any file in this repo)
`--allow-duplicate-host` is passed. `--dry-run` throughout both modes. unless `--allow-duplicate-host` is passed. `--dry-run` throughout both
modes.
- `scripts/env.sh` — shared config (`PROXMOX_HOST`, storage pool, bridge, - `scripts/env.sh` — shared config (`PROXMOX_HOST`, storage pool, bridge,
default cores/memory) sourced by `create-proxmox-resource.sh`. Add new default cores/memory) sourced by `create-proxmox-resource.sh`. Add new
cross-script config here instead of duplicating it per-script. cross-script config here instead of duplicating it per-script.
@@ -163,7 +165,7 @@ removing a host.
`vzdump` backup-archive metadata this doesn't have), no install step — `vzdump` backup-archive metadata this doesn't have), no install step —
see `docs/auto-installer.md`. see `docs/auto-installer.md`.
- `modules/build-types/*.nix` — what a system is for: - `modules/build-types/*.nix` — what a system is for:
minimal/server/docker/gui/pxe-boot/nix-cache. minimal/server/docker/gui/pxe-boot/nix-cache/tailscale-exit-node.
- `modules/common/configuration.nix` — base NixOS config imported by every - `modules/common/configuration.nix` — base NixOS config imported by every
host: locale, users, nix settings, git. host: locale, users, nix settings, git.
- `modules/common/home.nix` / `hosts/nixos/home.nix` — Home Manager config for - `modules/common/home.nix` / `hosts/nixos/home.nix` — Home Manager config for
+15 -12
View File
@@ -18,20 +18,23 @@ list:
| Target | Purpose | | Target | Purpose |
| --- | --- | | --- | --- |
| `linode-minimal` | Minimal NixOS host profile on a Linode VPS (real, deployed) | | `linode-minimal` | Minimal NixOS host profile on a Linode VPS |
| `proxmox-minimal` | Minimal NixOS host profile on Proxmox (real, deployed — previously the flat `nix-minimal` target) | | `proxmox-minimal` | Minimal NixOS host profile on Proxmox — previously the flat `nix-minimal` target |
| `lxc-minimal` | Minimal NixOS host profile in a Proxmox LXC container | | `lxc-minimal` | Minimal NixOS host profile in a Proxmox LXC container |
| `linode-nix-cache` / `proxmox-nix-cache` / `lxc-nix-cache` | Local Nix binary cache and remote builder (`lxc-nix-cache` is the real, deployed one — previously `proxmox-nix-cache`, itself previously the flat `nix-cache` target) | | `linode-nix-cache` / `proxmox-nix-cache` / `lxc-nix-cache` | Local Nix binary cache and remote builder previously the flat `nix-cache` target |
| `linode-server` / `proxmox-server` / `lxc-server` | Storage, NFS, backup, and monitoring exporter host (`proxmox-server` is the real, deployed one — previously the flat `server` target) | | `linode-server` / `proxmox-server` / `lxc-server` | Storage, NFS, backup, and monitoring exporter host — previously the flat `server` target |
| `linode-docker` / `proxmox-docker` / `lxc-docker` | Docker host for the main container stack (`proxmox-docker` is the real, deployed one — previously the flat `docker` target) | | `linode-docker` / `proxmox-docker` / `lxc-docker` | Docker host for the main container stack — previously the flat `docker` target |
| `linode-gui` / `proxmox-gui` / `lxc-gui` | Cinnamon desktop workstation (`proxmox-gui` is the real, deployed one — previously the flat `nixos` target) | | `linode-gui` / `proxmox-gui` / `lxc-gui` | Cinnamon desktop workstation — previously the flat `nixos` target |
| `proxmox-pxe-boot` / `lxc-pxe-boot` | HTTP/iPXE boot asset host (`proxmox-pxe-boot` is the real, deployed one — previously the flat `pxe-boot` target) | | `proxmox-pxe-boot` / `lxc-pxe-boot` | HTTP/iPXE boot asset host — previously the flat `pxe-boot` target |
| `linode-tailscale-exit-node` / `proxmox-tailscale-exit-node` / `lxc-tailscale-exit-node` | Tailscale exit node (no deployed target yet; `lxc-tailscale-exit-node` is the one planned for actual use) | | `linode-tailscale-exit-node` / `proxmox-tailscale-exit-node` / `lxc-tailscale-exit-node` | Tailscale exit node |
The "(real, deployed)" targets above are also tracked machine-readably in Which variant of a given buildtype is actually deployed isn't tracked
`variables.nix`'s `deployedTargets` — keep both in sync when a deployment anywhere in this repo — that's live infrastructure state, not something a
changes. `scripts/create-proxmox-resource.sh` reads that list to refuse committed file can keep accurate, and it changes independently of the code.
creating a same-identity duplicate of an already-deployed host by accident. Check the Proxmox node itself, or `/etc/flake-target` on a running host (see
below), if you need to know what's really out there right now.
`scripts/create-proxmox-resource.sh`'s duplicate-host guard works the same
way: it checks the Proxmox node directly rather than any file here.
Each buildtype's `hosts/<name>/host.nix` carries the per-machine identity Each buildtype's `hosts/<name>/host.nix` carries the per-machine identity
(hostname, hostId, per-machine secrets, `system.stateVersion`) that must stay (hostname, hostId, per-machine secrets, `system.stateVersion`) that must stay
+1 -1
View File
@@ -19,7 +19,7 @@ see "LXC hosts" immediately below for why those are different.**
## LXC hosts ## LXC hosts
`lxc-*` targets (`lxc-minimal`, `lxc-nix-cache`, `lxc-server`, `lxc-docker`, `lxc-*` targets (`lxc-minimal`, `lxc-nix-cache`, `lxc-server`, `lxc-docker`,
`lxc-gui`, `lxc-pxe-boot`) are **not** installed via `auto-install.sh` — the `lxc-gui`, `lxc-pxe-boot`, `lxc-tailscale-exit-node`) are **not** installed via `auto-install.sh` — the
interactive menu deliberately excludes them. Don't try to select one there; interactive menu deliberately excludes them. Don't try to select one there;
`nixos-install` would bind-mount `/` onto `/mnt` (LXC containers have no raw `nixos-install` would bind-mount `/` onto `/mnt` (LXC containers have no raw
disk to partition) and then refuse to touch the filesystem it's currently disk to partition) and then refuse to touch the filesystem it's currently
+8
View File
@@ -1,5 +1,13 @@
# Spec: Refactor Flake Targets into Platform × Build-Type Matrix # Spec: Refactor Flake Targets into Platform × Build-Type Matrix
**Status: implemented.** `flake.nix`'s `generatedTargets`/`mkTarget` and
`modules/platforms/*`/`modules/build-types/*` are the result of this spec —
kept here for historical rationale only (referenced from `CLAUDE.md`'s
"Composition pattern" section), not as an active or open plan. The "Open
Questions" below were resolved during implementation; don't treat them as
outstanding. A `tailscale-exit-node` build type was added later, beyond this
spec's original scope.
## Context ## Context
The flake at `~/nixos` currently defines these output targets (flat, ad-hoc naming): The flake at `~/nixos` currently defines these output targets (flat, ad-hoc naming):
+22 -10
View File
@@ -122,13 +122,25 @@ Add a pre-commit hook (or a `nix flake check` step) running `gitleaks protect --
## Definition of done ## Definition of done
- [ ] Milestone 1 inventory complete and reviewed **Status as of 2026-07-20:** Milestones 13 are done — sops-nix is fully
- [ ] All hosts have per-host age keys; admin key backed up outside the repo wired (`.sops.yaml`, `secrets/*.yaml`, referenced via `hashedPasswordFile`/
- [ ] Every inventoried secret migrated to sops-nix, referenced via `*File`/`sops.secrets.*.path`, nothing plaintext in the working tree `*File`/`sops.secrets.*.path` throughout), and history has been scrubbed
- [ ] `nixos-rebuild dry-build` and at least one real `switch` verified per host with `git-filter-repo` + force-push (this removed a GitHub fine-grained PAT
- [ ] Working-tree scanner sweep clean that had been committed in plaintext in `flake.nix`/`common/home.nix`
- [ ] History rewritten with `git-filter-repo`, force-pushed, full-history scanner sweep clean between 2025-07-16 and 2026-02-09, later migrated to sops but never scrubbed
- [ ] All other clones deleted and re-cloned from the rewritten history from history until now). **Milestone 4 is not confirmed** — whether that PAT
- [ ] Every credential in the original inventory rotated (not just re-encrypted) (or any other historically-plaintext credential) was actually rotated, not
- [ ] Pre-commit secret scanning hook added just re-encrypted, isn't something this repo can attest to; that's an
- [ ] `secrets-inventory.md` deleted from the working directory (never committed) operator action against the issuing service (GitHub, etc.), not a repo
change. Do that before considering this fully closed.
- [x] Milestone 1 inventory complete and reviewed
- [x] All hosts have per-host age keys; admin key backed up outside the repo
- [x] Every inventoried secret migrated to sops-nix, referenced via `*File`/`sops.secrets.*.path`, nothing plaintext in the working tree
- [x] `nixos-rebuild dry-build` and at least one real `switch` verified per host
- [x] Working-tree scanner sweep clean
- [x] History rewritten with `git-filter-repo`, force-pushed, full-history scanner sweep clean
- [ ] All other clones deleted and re-cloned from the rewritten history — every clone that existed before 2026-07-20's rewrite (any other machine, WSL instance, or CI checkout) needs this
- [ ] Every credential in the original inventory rotated (not just re-encrypted) — **the GitHub PAT found in history specifically still needs this**
- [x] Pre-commit secret scanning hook added (`.githooks/pre-commit`, `gitleaks protect --staged`)
- [x] `secrets-inventory.md` deleted from the working directory (never committed)
+60 -24
View File
@@ -10,7 +10,9 @@
# #
# SAFETY: # SAFETY:
# - The default (create) mode only ever creates a NEW resource -- it # - The default (create) mode only ever creates a NEW resource -- it
# refuses to run if the target VMID already exists on the node. # refuses to run if the target VMID already exists on the node, or if
# a VM/CT identified as --host already exists under any other VMID
# (checked live against the node; --allow-duplicate-host overrides).
# - --modify only ever touches a resource you name explicitly via # - --modify only ever touches a resource you name explicitly via
# --vmid, shows exactly what will change first, and (outside # --vmid, shows exactly what will change first, and (outside
# --dry-run) always requires typing that VMID back to confirm before # --dry-run) always requires typing that VMID back to confirm before
@@ -56,10 +58,11 @@ Create mode (default):
--force-rebuild Skip the "does the node already have this --force-rebuild Skip the "does the node already have this
image" check -- always build fresh and image" check -- always build fresh and
overwrite what's there. overwrite what's there.
--allow-duplicate-host Required if --host already has a real --allow-duplicate-host Required if a VM/CT identified as --host
deployment elsewhere (variables.nix's already exists on the node (checked live via
deployedTargets) -- otherwise refused, since qm/pct, not any file in this repo) --
it'd share that host's hostName/hostId. otherwise refused, since it'd share that
host's hostName/hostId.
Modify mode (reconfigure an EXISTING resource -- requires --modify): Modify mode (reconfigure an EXISTING resource -- requires --modify):
--modify Switch to modify mode. --modify Switch to modify mode.
@@ -286,25 +289,58 @@ fi
# feeds straight into the guest's real hostname) disagree with host.nix. # feeds straight into the guest's real hostname) disagree with host.nix.
[[ -z "$name" ]] && name="$host" [[ -z "$name" ]] && name="$host"
# --- refuse to duplicate a host that's already really deployed ---------- # --- refuse to duplicate a host that's already live on the node ---------
# Checked by hostName, not exact flake target: proxmox-server being # Queries the node itself (qm/pct's own name/hostname config), not any
# deployed also blocks --type lxc --host server, since both would carry # static list in this repo -- a file can't track whether a resource still
# the same hosts/server/host.nix identity (hostName, hostId). # actually exists, and this used to be checked against variables.nix's
if [[ "$allow_duplicate_host" -eq 0 ]]; then # deployedTargets, which drifted stale (it kept naming a VM as "the real
deployed_targets_json="$(nix eval --json --no-use-registries --no-accept-flake-config \ # deployment" well after that VM had been destroyed, blocking its own
--file "${repo_root}/variables.nix" deployedTargets)" # redeploy) until that list was dropped in favour of this live check. This
for dt in $(echo "$deployed_targets_json" | jq -r '.[]'); do # only catches guests identified with the default --name (== --host, what
dt_hostname="$(nix eval --raw --no-use-registries --no-accept-flake-config \ # this script itself always uses unless --name is overridden) -- a guest
"${repo_root}#nixosConfigurations.${dt}.config.networking.hostName" 2>/dev/null || true)" # manually renamed on the node afterwards wouldn't match, but nothing here
if [[ "$dt_hostname" == "$host" ]]; then # creates guests that way.
echo "ERROR: '${host}' already has a real deployment (${dt}, per variables.nix's" >&2 if [[ "$allow_duplicate_host" -eq 1 ]]; then
echo "deployedTargets). Creating ${flake_target} would share its hostName/hostId --" >&2 echo
echo "refusing by default. Pass --allow-duplicate-host if you really mean to spin" >&2 echo "--allow-duplicate-host: skipping the check for an existing '${host}' on ${node}."
echo "up a separate test instance of this host (it'll still get its own distinct" >&2 elif [[ "$dry_run" -eq 1 ]]; then
echo "sops key and VMID, never touching ${dt})." >&2 echo
exit 1 echo "[dry-run] would check ${node} for an existing VM/CT identified as '${host}'"
fi else
done echo
echo "==> Checking ${node} for an existing VM/CT identified as '${host}'..."
ssh_check_status=0
existing="$(ssh "$ssh_target" bash -s -- "$host" <<'REMOTE_SCRIPT'
target="$1"
for id in $(qm list 2>/dev/null | awk 'NR>1{print $1}'); do
n="$(qm config "$id" 2>/dev/null | grep -oP '^name:\s*\K\S+' || true)"
[[ "$n" == "$target" ]] && echo "vm ${id} ${n}"
done
for id in $(pct list 2>/dev/null | awk 'NR>1{print $1}'); do
n="$(pct config "$id" 2>/dev/null | grep -oP '^hostname:\s*\K\S+' || true)"
[[ "$n" == "$target" ]] && echo "lxc ${id} ${n}"
done
exit 0
REMOTE_SCRIPT
)" || ssh_check_status=$?
if [[ "$ssh_check_status" -ne 0 ]]; then
echo "ERROR: couldn't reach ${node} (ssh exited ${ssh_check_status}) to check for an" >&2
echo "existing '${host}' resource -- refusing to guess. Fix connectivity and retry," >&2
echo "or pass --allow-duplicate-host if you're sure none exists (this skips the" >&2
echo "check entirely)." >&2
exit 1
fi
if [[ -n "$existing" ]]; then
echo "ERROR: '${host}' already exists on ${node}:" >&2
echo "$existing" | while read -r kind id n; do
echo " - ${kind} VMID ${id} (${n})" >&2
done
echo "Refusing to create a second resource sharing this identity. Pass" >&2
echo "--allow-duplicate-host to create one anyway (it gets its own distinct" >&2
echo "sops key and VMID -- the existing resource(s) above are left untouched)," >&2
echo "or use --modify to reconfigure the existing one instead." >&2
exit 1
fi
fi fi
echo "Target: ${flake_target} (host=${host}, type=${type}) -> Proxmox resource '${name}'" echo "Target: ${flake_target} (host=${host}, type=${type}) -> Proxmox resource '${name}'"
-16
View File
@@ -156,20 +156,4 @@
keep = 20; # number of rotated logs to retain before deleting the oldest keep = 20; # number of rotated logs to retain before deleting the oldest
}; };
# Flake targets with a real, currently-running deployment somewhere —
# matches README.md's Hosts table "(real, deployed)" annotations; update
# both together. Not consumed by any NixOS module (nothing in the actual
# system config should behave differently because of this) — it's read
# by scripts/create-proxmox-resource.sh to refuse creating a same-identity
# duplicate of an already-deployed host (shared hostName/hostId) unless
# you explicitly pass --allow-duplicate-host.
deployedTargets = [
"linode-minimal"
"proxmox-minimal"
"lxc-nix-cache"
"proxmox-server"
"proxmox-docker"
"proxmox-gui"
"proxmox-pxe-boot"
];
} }