Archived
Compare commits
13
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
222a3ced69 | ||
|
|
03137eef9a | ||
|
|
af0fe5bdfd | ||
|
|
23b910a011 | ||
|
|
19f076bba1 | ||
|
|
9a1d6842d7 | ||
|
|
f5ef3194d4 | ||
|
|
90e3397b42 | ||
|
|
be5812d5bb | ||
|
|
84f7e038cb | ||
|
|
91d8f8fab1 | ||
|
|
723212a81f | ||
|
|
a5990ccf7d |
Submodule
+1
Submodule .claude/worktrees/proxmox-remote-build added at a5990ccf7d
@@ -7,7 +7,7 @@ servers and workstation.
|
|||||||
|
|
||||||
The flake exposes NixOS configurations named `<platform>-<buildtype>`
|
The flake exposes NixOS configurations named `<platform>-<buildtype>`
|
||||||
(platforms: `linode`, `proxmox`, `lxc`; build types: `minimal`, `nix-cache`,
|
(platforms: `linode`, `proxmox`, `lxc`; build types: `minimal`, `nix-cache`,
|
||||||
`server`, `docker`, `gui`, `pxe-boot`, `tailscale-exit-node`), generated from `modules/platforms/*`
|
`server`, `docker`, `gui`, `pxe-boot`, `tailscale-exit-node`, `tor-relay`), generated from `modules/platforms/*`
|
||||||
and `modules/build-types/*` by the `mkTarget` function in `flake.nix`. Not
|
and `modules/build-types/*` by the `mkTarget` function in `flake.nix`. Not
|
||||||
every combination is built — `pxe-boot` has no `linode` variant. See
|
every combination is built — `pxe-boot` has no `linode` variant. See
|
||||||
`README.md` for the full current target list; treat `flake.nix` as the
|
`README.md` for the full current target list; treat `flake.nix` as the
|
||||||
|
|||||||
@@ -97,10 +97,29 @@ Beyond `codex-setup.sh`/`codex-maintenance.sh` above, `scripts/` also has:
|
|||||||
Refuses to create a target whose host identity already exists live on
|
Refuses to create a target whose host identity already exists live on
|
||||||
the node (checked directly via `qm`/`pct`, not any file in this repo)
|
the node (checked directly via `qm`/`pct`, not any file in this repo)
|
||||||
unless `--allow-duplicate-host` is passed. `--dry-run` throughout both
|
unless `--allow-duplicate-host` is passed. `--dry-run` throughout both
|
||||||
modes.
|
modes. The first time it has to bootstrap build tooling on a node (i.e.
|
||||||
|
`nix` wasn't already on its `PATH`), it also runs
|
||||||
|
`scripts/configure-nix-cache-client.sh` there (non-fatally — a failure
|
||||||
|
just falls back to building from source / `cache.nixos.org`) so the
|
||||||
|
node substitutes from and can offload builds to nix-cache on every
|
||||||
|
subsequent run, not just this one.
|
||||||
- `scripts/env.sh` — shared config (`PROXMOX_HOST`, storage pool, bridge,
|
- `scripts/env.sh` — shared config (`PROXMOX_HOST`, storage pool, bridge,
|
||||||
default cores/memory) sourced by `create-proxmox-resource.sh`. Add new
|
default cores/memory) sourced by `create-proxmox-resource.sh`. Add new
|
||||||
cross-script config here instead of duplicating it per-script.
|
cross-script config here instead of duplicating it per-script.
|
||||||
|
- `scripts/configure-nix-cache-client.sh [--dry-run] [--no-remote-builder]
|
||||||
|
[--no-restart]` — the non-NixOS equivalent of
|
||||||
|
`modules/nix-cache/client.nix`/`remote-builder-client.nix`, for a plain
|
||||||
|
Debian machine with the Nix package manager (not NixOS) already
|
||||||
|
installed: run as root *on that machine* to add nix-cache as a
|
||||||
|
substituter in `/etc/nix/nix.conf` (`https://cache.nixos.org/` kept as
|
||||||
|
fallback) via `extra-substituters`/`extra-trusted-public-keys` so it
|
||||||
|
layers on top of whatever's already there instead of clobbering it, and,
|
||||||
|
if `/root/.ssh/nixremote` is already present (see docs/nix-cache.md
|
||||||
|
"Remote builder SSH keys"), configures it as a distributed-build
|
||||||
|
machine too and trusts nix-cache's SSH host key in
|
||||||
|
`/etc/ssh/ssh_known_hosts`. Idempotent (re-running replaces its own
|
||||||
|
marked block rather than duplicating it); restarts `nix-daemon` by
|
||||||
|
default so the change takes effect immediately.
|
||||||
- `scripts/bump-nixpkgs-release.sh` — bumps `flake.nix`'s `nixpkgs.url`/
|
- `scripts/bump-nixpkgs-release.sh` — bumps `flake.nix`'s `nixpkgs.url`/
|
||||||
`home-manager.url` in place. Exists because flake input URLs can't
|
`home-manager.url` in place. Exists because flake input URLs can't
|
||||||
reference `variables.nix` (confirmed empirically — `nix flake metadata`
|
reference `variables.nix` (confirmed empirically — `nix flake metadata`
|
||||||
@@ -160,9 +179,10 @@ nixosSystem {
|
|||||||
```
|
```
|
||||||
|
|
||||||
Platforms: `linode`, `proxmox`, `lxc`. Build types: `minimal`, `nix-cache`,
|
Platforms: `linode`, `proxmox`, `lxc`. Build types: `minimal`, `nix-cache`,
|
||||||
`server`, `docker`, `gui`, `pxe-boot`, `tailscale-exit-node`. Not every
|
`server`, `docker`, `gui`, `pxe-boot`, `tailscale-exit-node`, `tor-relay`. Not
|
||||||
combination is built — e.g. `pxe-boot` has no `linode` variant (PXE/DHCP/TFTP
|
every combination is built — e.g. `pxe-boot` has no `linode` variant
|
||||||
need LAN L2 adjacency a Linode VPS doesn't have). Treat `flake.nix`'s
|
(PXE/DHCP/TFTP need LAN L2 adjacency a Linode VPS doesn't have), and
|
||||||
|
`tor-relay` currently only exists as `lxc-tor-relay`. Treat `flake.nix`'s
|
||||||
`generatedTargets` as the source
|
`generatedTargets` as the source
|
||||||
of truth for which hosts exist — `README.md`, `AGENTS.md`,
|
of truth for which hosts exist — `README.md`, `AGENTS.md`,
|
||||||
`docs/flake-lock-automation.md`, and the CI eval workflows
|
`docs/flake-lock-automation.md`, and the CI eval workflows
|
||||||
@@ -190,7 +210,7 @@ removing a host.
|
|||||||
`vzdump` backup-archive metadata this doesn't have), no install step —
|
`vzdump` backup-archive metadata this doesn't have), no install step —
|
||||||
see `docs/auto-installer.md`.
|
see `docs/auto-installer.md`.
|
||||||
- `modules/build-types/*.nix` — what a system is for:
|
- `modules/build-types/*.nix` — what a system is for:
|
||||||
minimal/server/docker/gui/pxe-boot/nix-cache/tailscale-exit-node.
|
minimal/server/docker/gui/pxe-boot/nix-cache/tailscale-exit-node/tor-relay.
|
||||||
- `modules/common/configuration.nix` — base NixOS config imported by every
|
- `modules/common/configuration.nix` — base NixOS config imported by every
|
||||||
host: locale, users, nix settings, git.
|
host: locale, users, nix settings, git.
|
||||||
- `modules/common/home.nix` / `hosts/nixos/home.nix` — Home Manager config for
|
- `modules/common/home.nix` / `hosts/nixos/home.nix` — Home Manager config for
|
||||||
@@ -225,7 +245,8 @@ removing a host.
|
|||||||
and `environmentFile`; used by `hosts/server/host.nix` and
|
and `environmentFile`; used by `hosts/server/host.nix` and
|
||||||
`hosts/nix-cache/host.nix` to avoid duplicating that boilerplate.
|
`hosts/nix-cache/host.nix` to avoid duplicating that boilerplate.
|
||||||
- `modules/tailscale/`, `modules/docker/`, `modules/networking/`,
|
- `modules/tailscale/`, `modules/docker/`, `modules/networking/`,
|
||||||
`modules/traefik/`, `modules/services/*` — single-purpose, single-host
|
`modules/traefik/`, `modules/tor/`, `modules/services/*` — single-purpose,
|
||||||
|
single-host
|
||||||
feature modules (e.g. `docker/enable-service.nix`,
|
feature modules (e.g. `docker/enable-service.nix`,
|
||||||
`services/zfs/enable-service.nix`). Grep `modules/build-types/*.nix` for
|
`services/zfs/enable-service.nix`). Grep `modules/build-types/*.nix` for
|
||||||
each build type's `imports` list to see which modules apply where.
|
each build type's `imports` list to see which modules apply where.
|
||||||
|
|||||||
@@ -10,11 +10,11 @@ pieces composed in `flake.nix`:
|
|||||||
|
|
||||||
- **Platforms** (what it runs on): `linode`, `proxmox`, `lxc`
|
- **Platforms** (what it runs on): `linode`, `proxmox`, `lxc`
|
||||||
- **Build types** (what it's for): `minimal`, `nix-cache`, `server`, `docker`,
|
- **Build types** (what it's for): `minimal`, `nix-cache`, `server`, `docker`,
|
||||||
`gui`, `pxe-boot`, `tailscale-exit-node`
|
`gui`, `pxe-boot`, `tailscale-exit-node`, `tor-relay`
|
||||||
|
|
||||||
Not every combination exists — `pxe-boot` has no `linode` variant, since
|
Not every combination exists — `pxe-boot` has no `linode` variant, since
|
||||||
PXE/DHCP/TFTP need LAN L2 adjacency that a Linode VPS doesn't have. The full
|
PXE/DHCP/TFTP need LAN L2 adjacency that a Linode VPS doesn't have, and
|
||||||
list:
|
`tor-relay` currently only exists as `lxc-tor-relay`. The full list:
|
||||||
|
|
||||||
| Target | Purpose |
|
| Target | Purpose |
|
||||||
| --- | --- |
|
| --- | --- |
|
||||||
@@ -27,6 +27,7 @@ list:
|
|||||||
| `linode-gui` / `proxmox-gui` / `lxc-gui` | Cinnamon desktop workstation — previously the flat `nixos` target |
|
| `linode-gui` / `proxmox-gui` / `lxc-gui` | Cinnamon desktop workstation — previously the flat `nixos` target |
|
||||||
| `proxmox-pxe-boot` / `lxc-pxe-boot` | HTTP/iPXE boot asset host — previously the flat `pxe-boot` target |
|
| `proxmox-pxe-boot` / `lxc-pxe-boot` | HTTP/iPXE boot asset host — previously the flat `pxe-boot` target |
|
||||||
| `linode-tailscale-exit-node` / `proxmox-tailscale-exit-node` / `lxc-tailscale-exit-node` | Tailscale exit node |
|
| `linode-tailscale-exit-node` / `proxmox-tailscale-exit-node` / `lxc-tailscale-exit-node` | Tailscale exit node |
|
||||||
|
| `lxc-tor-relay` | Tor middle relay |
|
||||||
|
|
||||||
Which variant of a given buildtype is actually deployed isn't tracked
|
Which variant of a given buildtype is actually deployed isn't tracked
|
||||||
anywhere in this repo — that's live infrastructure state, not something a
|
anywhere in this repo — that's live infrastructure state, not something a
|
||||||
@@ -114,9 +115,10 @@ Three different paths depending on target, none of them involving a manual
|
|||||||
`docs/proxmox-images.md`.
|
`docs/proxmox-images.md`.
|
||||||
|
|
||||||
`scripts/create-proxmox-resource.sh --type lxc|vm --host <name>` automates
|
`scripts/create-proxmox-resource.sh --type lxc|vm --host <name>` automates
|
||||||
either of the last two end to end (build, host-key registration, upload,
|
either of the last two end to end (host-key registration, building the
|
||||||
`pct create`/`qm create`), with `--dry-run` and a guard against duplicating
|
image directly on the Proxmox node itself, `pct create`/`qm create`), with
|
||||||
an already-deployed host's identity. See its `--help`.
|
`--dry-run` and a guard against duplicating an already-deployed host's
|
||||||
|
identity. See its `--help`.
|
||||||
|
|
||||||
## Security Notes
|
## Security Notes
|
||||||
|
|
||||||
|
|||||||
@@ -19,7 +19,7 @@ see "LXC hosts" immediately below for why those are different.**
|
|||||||
## LXC hosts
|
## LXC hosts
|
||||||
|
|
||||||
`lxc-*` targets (`lxc-minimal`, `lxc-nix-cache`, `lxc-server`, `lxc-docker`,
|
`lxc-*` targets (`lxc-minimal`, `lxc-nix-cache`, `lxc-server`, `lxc-docker`,
|
||||||
`lxc-gui`, `lxc-pxe-boot`, `lxc-tailscale-exit-node`) are **not** installed via `auto-install.sh` — the
|
`lxc-gui`, `lxc-pxe-boot`, `lxc-tailscale-exit-node`, `lxc-tor-relay`) are **not** installed via `auto-install.sh` — the
|
||||||
interactive menu deliberately excludes them. Don't try to select one there;
|
interactive menu deliberately excludes them. Don't try to select one there;
|
||||||
`nixos-install` would bind-mount `/` onto `/mnt` (LXC containers have no raw
|
`nixos-install` would bind-mount `/` onto `/mnt` (LXC containers have no raw
|
||||||
disk to partition) and then refuse to touch the filesystem it's currently
|
disk to partition) and then refuse to touch the filesystem it's currently
|
||||||
@@ -74,8 +74,8 @@ booting one:
|
|||||||
First boot runs `boot.postBootCommands` (registers the Nix store DB and
|
First boot runs `boot.postBootCommands` (registers the Nix store DB and
|
||||||
system profile) — there's no separate activation step to run yourself.
|
system profile) — there's no separate activation step to run yourself.
|
||||||
`scripts/create-proxmox-resource.sh --type lxc --host <name>` automates all
|
`scripts/create-proxmox-resource.sh --type lxc --host <name>` automates all
|
||||||
of this (build, host-key handling, upload, `pct create` with the flags
|
of this (host-key handling, building the tarball directly on the Proxmox
|
||||||
above) — see its `--help`.
|
node itself, `pct create` with the flags above) — see its `--help`.
|
||||||
|
|
||||||
Host keys still need pre-seeding the same way as any other host — the
|
Host keys still need pre-seeding the same way as any other host — the
|
||||||
sops-nix activation-vs-first-boot race is identical regardless of how the
|
sops-nix activation-vs-first-boot race is identical regardless of how the
|
||||||
|
|||||||
@@ -9,9 +9,11 @@ install, so there's nothing host-specific to write; it's available for every
|
|||||||
|
|
||||||
`scripts/create-proxmox-resource.sh --type vm --host <name>` automates the
|
`scripts/create-proxmox-resource.sh --type vm --host <name>` automates the
|
||||||
whole walkthrough below (and the equivalent LXC one) end to end, including
|
whole walkthrough below (and the equivalent LXC one) end to end, including
|
||||||
host-key handling and upload — see its `--help`. The steps here are what it
|
host-key handling and building the image directly on the Proxmox node
|
||||||
runs under the hood, useful for doing any of it by hand or understanding
|
itself (no local build, no image transfer) — see its `--help`. The steps
|
||||||
what it does before you trust it against real infrastructure.
|
here are what it runs under the hood, useful for doing any of it by hand
|
||||||
|
or understanding what it does before you trust it against real
|
||||||
|
infrastructure.
|
||||||
|
|
||||||
## Building
|
## Building
|
||||||
|
|
||||||
|
|||||||
@@ -1,151 +0,0 @@
|
|||||||
# Spec: Refactor Flake Targets into Platform × Build-Type Matrix
|
|
||||||
|
|
||||||
**Status: implemented.** `flake.nix`'s `generatedTargets`/`mkTarget` and
|
|
||||||
`modules/platforms/*`/`modules/build-types/*` are the result of this spec —
|
|
||||||
kept here for historical rationale only (referenced from `CLAUDE.md`'s
|
|
||||||
"Composition pattern" section), not as an active or open plan. The "Open
|
|
||||||
Questions" below were resolved during implementation; don't treat them as
|
|
||||||
outstanding. A `tailscale-exit-node` build type was added later, beyond this
|
|
||||||
spec's original scope.
|
|
||||||
|
|
||||||
## Context
|
|
||||||
|
|
||||||
The flake at `~/nixos` currently defines these output targets (flat, ad-hoc naming):
|
|
||||||
|
|
||||||
- `docker`
|
|
||||||
- `linode-minimal`
|
|
||||||
- `nix-cache`
|
|
||||||
- `nix-minimal`
|
|
||||||
- `nixos`
|
|
||||||
- `server`
|
|
||||||
- `pxe-boot`
|
|
||||||
|
|
||||||
Some already follow a `platform-buildtype` convention (`linode-minimal`), most don't.
|
|
||||||
`~/nix-auto-installer` is a related repo and should be checked for any coupling to
|
|
||||||
these target names (scripts, docs, CI, or install automation that reference them by
|
|
||||||
name) before renaming anything.
|
|
||||||
|
|
||||||
## Goal
|
|
||||||
|
|
||||||
Restructure the flake so targets are generated from two orthogonal concepts:
|
|
||||||
|
|
||||||
**Build types** (what the system is for):
|
|
||||||
- `minimal`
|
|
||||||
- `nix-cache`
|
|
||||||
- `server`
|
|
||||||
- `docker`
|
|
||||||
- `pxe-boot`
|
|
||||||
- `gui`
|
|
||||||
|
|
||||||
**Platforms** (what it's deployed on):
|
|
||||||
- `linode` (Linode VM)
|
|
||||||
- `proxmox` (Proxmox VM)
|
|
||||||
- `lxc` (Proxmox LXC container)
|
|
||||||
|
|
||||||
Final targets should be named consistently as `<platform>-<buildtype>`, e.g.:
|
|
||||||
|
|
||||||
```
|
|
||||||
linode-minimal proxmox-minimal lxc-minimal
|
|
||||||
linode-nix-cache proxmox-nix-cache lxc-nix-cache
|
|
||||||
linode-server proxmox-server lxc-server
|
|
||||||
linode-docker proxmox-docker lxc-docker
|
|
||||||
linode-pxe-boot proxmox-pxe-boot lxc-pxe-boot
|
|
||||||
linode-gui proxmox-gui lxc-gui
|
|
||||||
```
|
|
||||||
|
|
||||||
That's the full matrix (18 targets) if every build type applies to every platform.
|
|
||||||
See **Open Questions** below — some combinations may not make sense and should be
|
|
||||||
confirmed with me before being built out, not silently included or dropped.
|
|
||||||
|
|
||||||
## Migration mapping (old → new)
|
|
||||||
|
|
||||||
| Old target | New target | Notes |
|
|
||||||
|--------------------|------------------------------------------------------|-------|
|
|
||||||
| `linode-minimal` | `linode-minimal` | Already correct, keep as-is |
|
|
||||||
| `nix-minimal` | likely `proxmox-minimal` or a platform-less base module | Ambiguous — see Open Questions |
|
|
||||||
| `nix-cache` | base module consumed by `linode-nix-cache`, `proxmox-nix-cache`, `lxc-nix-cache` | Currently platform-less; needs to become a build-type module, not a standalone target |
|
|
||||||
| `server` | base module consumed by `linode-server`, `proxmox-server`, `lxc-server` | Same as above |
|
|
||||||
| `docker` | base module consumed by `linode-docker`, `proxmox-docker`, `lxc-docker` | Confirm docker actually makes sense as an LXC/VM guest build vs. a standalone container image — see Open Questions |
|
|
||||||
| `pxe-boot` | TBD — may stay a single target rather than a per-platform one | See Open Questions |
|
|
||||||
| `nixos` | TBD — unclear what this maps to in the new scheme | See Open Questions |
|
|
||||||
|
|
||||||
## Open Questions (Claude Code: raise these with me before implementing, don't guess)
|
|
||||||
|
|
||||||
1. **`nixos` target** — what is this currently used for (bare metal install, dev
|
|
||||||
shell, template)? It doesn't obviously map to any of the six build types.
|
|
||||||
2. **`nix-minimal` vs `linode-minimal`** — are these two different things, or is
|
|
||||||
`nix-minimal` a leftover/duplicate?
|
|
||||||
3. **`pxe-boot` and `gui` across all three platforms** — does PXE boot make sense
|
|
||||||
for an LXC container or a cloud VM (Linode), or is it inherently bare-metal/
|
|
||||||
network-boot only and should remain a single non-platform target? Does `gui`
|
|
||||||
make sense inside an LXC container?
|
|
||||||
4. **`docker` as a build type** — is this "a NixOS host configured to run Docker"
|
|
||||||
(which would sensibly have linode/proxmox/lxc variants), or "a Docker container
|
|
||||||
image built by the flake" (which wouldn't take a platform prefix at all, since
|
|
||||||
it doesn't run on Linode/Proxmox/LXC as a guest OS)? These are structurally
|
|
||||||
different and change how it should be wired in.
|
|
||||||
5. Confirm whether all 18 combinations should actually exist, or whether this is
|
|
||||||
meant to produce only the combinations that are genuinely useful (e.g. maybe no
|
|
||||||
one needs `lxc-pxe-boot`).
|
|
||||||
|
|
||||||
## Implementation approach
|
|
||||||
|
|
||||||
1. **Inventory first.** Read the current `flake.nix` and any `nixosConfigurations`/
|
|
||||||
`modules` structure. Map every existing target to what module(s) it actually
|
|
||||||
pulls in. Don't assume — confirm against the real file contents.
|
|
||||||
2. **Separate build-type and platform into their own module directories**, e.g.:
|
|
||||||
```
|
|
||||||
modules/build-types/minimal.nix
|
|
||||||
modules/build-types/nix-cache.nix
|
|
||||||
modules/build-types/server.nix
|
|
||||||
modules/build-types/docker.nix
|
|
||||||
modules/build-types/pxe-boot.nix
|
|
||||||
modules/build-types/gui.nix
|
|
||||||
|
|
||||||
modules/platforms/linode.nix
|
|
||||||
modules/platforms/proxmox.nix
|
|
||||||
modules/platforms/lxc.nix
|
|
||||||
```
|
|
||||||
Build-type modules should contain only what makes a system "minimal" vs
|
|
||||||
"server" vs "gui", etc. Platform modules should contain only what's specific
|
|
||||||
to running as a Linode VM vs Proxmox VM vs LXC container (virtualisation
|
|
||||||
guest tools, boot method, filesystem/image format, LXC-specific constraints
|
|
||||||
like no kernel modules, etc).
|
|
||||||
3. **Generate the target matrix programmatically** in `flake.nix` rather than
|
|
||||||
hand-writing 18 near-identical `nixosConfigurations` entries — e.g. a small
|
|
||||||
function that takes a platform name and build-type name, composes the two
|
|
||||||
modules plus any shared base module, and produces the named output. This
|
|
||||||
keeps future build types/platforms a one-line addition rather than a copy-paste
|
|
||||||
job.
|
|
||||||
4. **Only build combinations we've confirmed make sense** (see Open Questions) —
|
|
||||||
don't emit all 18 by default if some are structurally invalid.
|
|
||||||
5. **Preserve existing working configs during the transition.** Don't delete the
|
|
||||||
old target names until their replacements build successfully — rename/alias
|
|
||||||
at the end, not the start, so there's no window where the flake is broken.
|
|
||||||
|
|
||||||
## Verification
|
|
||||||
|
|
||||||
For every new target produced:
|
|
||||||
```bash
|
|
||||||
nix flake check
|
|
||||||
nix build .#nixosConfigurations.<target>.config.system.build.toplevel
|
|
||||||
```
|
|
||||||
Confirm each builds without evaluation errors before considering it done. If a
|
|
||||||
target fails to build, report which one and why rather than silently skipping it.
|
|
||||||
|
|
||||||
## Deliverables
|
|
||||||
|
|
||||||
- Refactored `flake.nix` using the composed module + generated-matrix approach.
|
|
||||||
- New `modules/build-types/*.nix` and `modules/platforms/*.nix` files.
|
|
||||||
- Old flat target names removed only after their replacements are verified.
|
|
||||||
- A short `README.md` (or section in existing docs) listing the final target
|
|
||||||
names and what each one is for.
|
|
||||||
- A summary at the end of what changed, what was removed, and any of the Open
|
|
||||||
Questions above that got resolved differently than expected.
|
|
||||||
|
|
||||||
## Out of scope
|
|
||||||
|
|
||||||
- Don't touch `~/nix-auto-installer` contents beyond checking it for references
|
|
||||||
to the old target names — if changes there are needed, flag them, don't make
|
|
||||||
them without confirming.
|
|
||||||
- Don't add new build types or platforms beyond the ones listed here.
|
|
||||||
@@ -98,6 +98,8 @@
|
|||||||
linode-tailscale-exit-node = mkTarget { platform = "linode"; buildType = "tailscale-exit-node"; hostPath = ./hosts/tailscale-exit-node/host.nix; };
|
linode-tailscale-exit-node = mkTarget { platform = "linode"; buildType = "tailscale-exit-node"; hostPath = ./hosts/tailscale-exit-node/host.nix; };
|
||||||
proxmox-tailscale-exit-node = mkTarget { platform = "proxmox"; buildType = "tailscale-exit-node"; hostPath = ./hosts/tailscale-exit-node/host.nix; };
|
proxmox-tailscale-exit-node = mkTarget { platform = "proxmox"; buildType = "tailscale-exit-node"; hostPath = ./hosts/tailscale-exit-node/host.nix; };
|
||||||
lxc-tailscale-exit-node = mkTarget { platform = "lxc"; buildType = "tailscale-exit-node"; hostPath = ./hosts/tailscale-exit-node/host.nix; };
|
lxc-tailscale-exit-node = mkTarget { platform = "lxc"; buildType = "tailscale-exit-node"; hostPath = ./hosts/tailscale-exit-node/host.nix; };
|
||||||
|
|
||||||
|
lxc-tor-relay = mkTarget { platform = "lxc"; buildType = "tor-relay"; hostPath = ./hosts/tor-relay/host.nix; };
|
||||||
};
|
};
|
||||||
|
|
||||||
# Auto-install environments (migrated from the former nix-auto-installer
|
# Auto-install environments (migrated from the former nix-auto-installer
|
||||||
|
|||||||
@@ -0,0 +1,12 @@
|
|||||||
|
_:
|
||||||
|
|
||||||
|
{
|
||||||
|
networking.hostName = "tor-relay";
|
||||||
|
|
||||||
|
# No networking.hostId: only ZFS-touching hosts (server, docker) need one
|
||||||
|
# for pool-import safety, and this host does neither.
|
||||||
|
|
||||||
|
# A genuinely new host (not a pre-refactor carry-over), so it tracks the
|
||||||
|
# flake's current nixpkgs release rather than being pinned to an older one.
|
||||||
|
system.stateVersion = "26.05";
|
||||||
|
}
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
{ ... }:
|
||||||
|
|
||||||
|
{
|
||||||
|
imports = [
|
||||||
|
../tor/enable-relay.nix
|
||||||
|
];
|
||||||
|
}
|
||||||
@@ -0,0 +1,35 @@
|
|||||||
|
{ pkgs, vars, ... }:
|
||||||
|
|
||||||
|
{
|
||||||
|
services.tor = {
|
||||||
|
enable = true;
|
||||||
|
|
||||||
|
# Opens settings.ORPort (and DirPort, unset here) in the firewall —
|
||||||
|
# see the nixpkgs tor module's own networking.firewall.mkIf block.
|
||||||
|
openFirewall = true;
|
||||||
|
|
||||||
|
relay = {
|
||||||
|
enable = true;
|
||||||
|
# Plain middle/guard relay, not "exit" — relays onion traffic between
|
||||||
|
# other Tor nodes without ever making requests to the public internet
|
||||||
|
# on a user's behalf, avoiding the abuse complaints and legal exposure
|
||||||
|
# an exit node invites.
|
||||||
|
role = "relay";
|
||||||
|
};
|
||||||
|
|
||||||
|
settings.ORPort = vars.ports.torRelayOrPort;
|
||||||
|
|
||||||
|
# Unix control socket at /run/tor/control (GroupWritable, group "tor")
|
||||||
|
# -- what nyx below actually monitors the relay through. Nyx's own
|
||||||
|
# default control-socket path (/var/run/tor/control) resolves to the
|
||||||
|
# same place, so no extra nyx config is needed.
|
||||||
|
controlSocket.enable = true;
|
||||||
|
};
|
||||||
|
|
||||||
|
# Lets the primary user's shell session read/write the control socket
|
||||||
|
# above without being root -- otherwise nyx fails to authenticate against
|
||||||
|
# it at all.
|
||||||
|
users.users.${vars.primaryUser}.extraGroups = [ "tor" ];
|
||||||
|
|
||||||
|
environment.systemPackages = [ pkgs.nyx ];
|
||||||
|
}
|
||||||
@@ -1,146 +0,0 @@
|
|||||||
# Spec: Remove Sensitive Information from NixOS Flake
|
|
||||||
|
|
||||||
## Goal
|
|
||||||
|
|
||||||
Every secret currently readable in plaintext anywhere in this repo (working tree *and* git history) gets removed, replaced with `sops-nix`-managed encrypted references, and rotated. When this is done, the repo should be safe to make public without exposing anything about the systems it configures.
|
|
||||||
|
|
||||||
Treat this as three sequential milestones. Do not start git history rewriting (Milestone 3) until Milestones 1 and 2 are fully verified and the flake still builds. This should be its own branch (`refactor/secrets`) until fully verified, then merged.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Milestone 1 — Audit
|
|
||||||
|
|
||||||
Before touching anything, produce a complete inventory. Do not guess at scope — grep the whole tree and the whole history.
|
|
||||||
|
|
||||||
1. Run a secret scanner across the working tree and full history. Use both, since they catch different things:
|
|
||||||
- `gitleaks detect --source . -v --log-opts="--all"` (scans history too)
|
|
||||||
- `trufflehog git file://. --since-commit=$(git rev-list --max-parents=0 HEAD) --only-verified=false`
|
|
||||||
If neither is installed, add them via a temporary `nix-shell -p gitleaks trufflehog` — don't install anything globally on the host.
|
|
||||||
|
|
||||||
2. Manually grep for the categories below, since scanners miss config-specific patterns:
|
|
||||||
- `hashedPassword`, `password`, `initialPassword`, `initialHashedPassword` in any `users.users.*` block
|
|
||||||
- `age.secrets`, `sops.secrets` (if any partial secrets work already exists — check for it)
|
|
||||||
- PSK / `preSharedKey`, `privateKeyFile` inline values (vs. file references) for WireGuard
|
|
||||||
- `authKey`, `apiToken`, `api_key`, `token =`, `secret =` in service modules (Tailscale, Cloudflare, backup tools, etc.)
|
|
||||||
- SSH private key material: search for `BEGIN OPENSSH PRIVATE KEY` / `BEGIN RSA PRIVATE KEY` literals
|
|
||||||
- TLS cert/key pairs committed under e.g. `secrets/`, `certs/`, `pki/`
|
|
||||||
- Real name, personal email, home address, or anything in comments/hostnames that maps a machine to your physical identity or network layout (e.g. hostnames like `wayne-desktop`, static LAN IPs, ISP-identifying info)
|
|
||||||
- `.env` files, `secrets.nix`, `secrets.yaml`, or any file that looks like it was meant to be gitignored but wasn't
|
|
||||||
|
|
||||||
3. Produce `secrets-inventory.md` (temporary, delete before finishing) listing: file path, line, secret type, and which host/service it belongs to. This becomes the checklist for Milestone 2 — every row must be either migrated to sops or deleted, with nothing left unaccounted for.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Milestone 2 — Migrate to sops-nix
|
|
||||||
|
|
||||||
### 2.1 Set up sops-nix
|
|
||||||
|
|
||||||
1. Add the flake input:
|
|
||||||
```nix
|
|
||||||
sops-nix.url = "github:Mic92/sops-nix";
|
|
||||||
sops-nix.inputs.nixpkgs.follows = "nixpkgs";
|
|
||||||
```
|
|
||||||
2. Import `sops-nix.nixosModules.sops` into each host's module list (or into a shared `common.nix` if all hosts use it).
|
|
||||||
3. Generate an age keypair **per host** (not one shared key for everything — a compromised host shouldn't decrypt every other host's secrets):
|
|
||||||
```
|
|
||||||
nix-shell -p age --run "age-keygen -o /var/lib/sops-nix/key.txt"
|
|
||||||
```
|
|
||||||
Print the public key (`age-keygen -y`) for each host — you'll need it for `.sops.yaml`.
|
|
||||||
4. Also generate one age key for yourself (your admin workstation) so you can edit secrets without needing to SSH into a host: store it at `~/.config/sops/age/keys.txt`, back it up somewhere outside this repo (password manager, offline). **If this key is lost, every secret encrypted with it is unrecoverable — losing the age key is equivalent to losing the secrets.**
|
|
||||||
5. Create `.sops.yaml` at the repo root defining creation rules: which age public keys can decrypt which secrets files, keyed by path regex, so e.g. `secrets/hostA.yaml` is decryptable by your admin key + hostA's key, `secrets/hostB.yaml` by your admin key + hostB's key.
|
|
||||||
|
|
||||||
### 2.2 Migrate each secret category from the inventory
|
|
||||||
|
|
||||||
For each row in `secrets-inventory.md`:
|
|
||||||
|
|
||||||
- **Password hashes**: generate hash with `mkpasswd -m sha-512` (or `bcrypt` if your setup wants that), store under `sops.secrets."<name>/hashedPassword"`, reference via `users.users.<name>.hashedPasswordFile = config.sops.secrets."<name>/hashedPassword".path;`. Do not put the *plaintext* password anywhere, only the hash, and only the hash goes into the encrypted sops file.
|
|
||||||
- **API tokens / auth keys**: move the raw value into the per-host sops YAML, reference in the module via `config.sops.secrets."<service>/token".path` — most NixOS service modules that take a token also accept a `*File` variant (e.g. `environmentFile`, `tokenFile`); use that instead of passing the value directly.
|
|
||||||
- **Private keys / certs**: move the PEM/key content wholesale into a sops secret, output as a file with appropriate `sops.secrets.<name>.path`, `owner`, `mode`, `restartUnits` so the depending service (sshd, wireguard, nginx) reloads when the secret changes.
|
|
||||||
- **Personal/identifying info**: this doesn't belong in sops (it's not "secret," it's just information you don't want public). Replace real names/emails with placeholders or move to a small untracked `local.nix` that's `.gitignore`'d and imported conditionally, with a documented template (`local.nix.example`) committed instead.
|
|
||||||
|
|
||||||
### 2.3 Verify before moving on
|
|
||||||
|
|
||||||
- `nixos-rebuild dry-build --flake .#<host>` succeeds for every host.
|
|
||||||
- `sudo nixos-rebuild switch --flake .#<host>` on at least one real machine (or a VM) confirms secrets decrypt and services start.
|
|
||||||
- Confirm decrypted secrets land under `/run/secrets/` (not the Nix store — anything placed in `/nix/store` is world-readable by design, so sops-nix's runtime-only placement is the whole point; double check no module accidentally pulls a secret path into a store-built config file).
|
|
||||||
- Re-run the grep/scanner sweep from Milestone 1 against the *working tree only* (not history yet) — it should now come back clean.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Milestone 3 — Scrub git history
|
|
||||||
|
|
||||||
Do this only after Milestone 2 is merged to your main branch and confirmed working, since it rewrites every commit SHA from the point of the earliest offending commit onward.
|
|
||||||
|
|
||||||
**This is destructive and irreversible on your local clone. Back up first:**
|
|
||||||
```
|
|
||||||
cp -r /path/to/nixos-repo /path/to/nixos-repo-backup-$(date +%F)
|
|
||||||
```
|
|
||||||
|
|
||||||
1. Install `git-filter-repo` (not the older `git filter-branch` / BFG — filter-repo is the currently maintained, faster, safer tool):
|
|
||||||
```
|
|
||||||
nix-shell -p git-filter-repo
|
|
||||||
```
|
|
||||||
2. Use the `secrets-inventory.md` list to build a list of literal strings/paths to strip. Two approaches, use both:
|
|
||||||
- Path-based: if whole files were secret (e.g. `secrets.nix`, a `.env`, a private key file), remove them entirely from history:
|
|
||||||
```
|
|
||||||
git filter-repo --path secrets.nix --path .env --invert-paths
|
|
||||||
```
|
|
||||||
- Value-based: for secrets embedded inline in files you're keeping (not deleting the whole file), use `--replace-text` with a file listing each literal secret string to replace with `***REMOVED***`:
|
|
||||||
```
|
|
||||||
git filter-repo --replace-text expressions.txt
|
|
||||||
```
|
|
||||||
3. After filtering, verify: run the Milestone 1 scanners again against full history (`--log-opts="--all"`). They must come back clean.
|
|
||||||
4. Force-push the rewritten history:
|
|
||||||
```
|
|
||||||
git push origin --force --all
|
|
||||||
git push origin --force --tags
|
|
||||||
```
|
|
||||||
5. **Every other clone of this repo (other machines, WSL instances, CI) must be deleted and re-cloned fresh** — a `git pull` against rewritten history will not work cleanly and risks resurrecting the old commits. Don't try to reconcile old clones; throw them away and re-clone.
|
|
||||||
6. If this repo has ever been pushed to a public host (GitHub, etc.) or a fork/mirror exists, treat every secret that was ever in history as **permanently compromised regardless of the rewrite** — caches, forks, and Wayback-style archives can retain old commits indefinitely. History scrubbing prevents *future* exposure via `git clone`; it does not undo past exposure.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Milestone 4 — Rotate everything
|
|
||||||
|
|
||||||
Because the secrets were exposed in history (even briefly, even in a private repo), the migration is not complete until every credential in the inventory has been **rotated**, not just re-encrypted. Re-encrypting an already-leaked value protects it going forward but doesn't undo the leak.
|
|
||||||
|
|
||||||
For each row in the original inventory:
|
|
||||||
- Password hashes → change the actual account password, regenerate the hash, update the sops file.
|
|
||||||
- API tokens/auth keys → revoke the old token in the issuing service's dashboard (Cloudflare, Tailscale, backup provider, etc.) and generate a new one.
|
|
||||||
- SSH/WireGuard private keys → generate new keypairs, update the corresponding public key wherever it's trusted (authorized_keys, peer configs, etc.), retire the old ones.
|
|
||||||
- TLS certs → reissue if the private key was exposed.
|
|
||||||
|
|
||||||
Keep `secrets-inventory.md` open during this step and check off each row as rotated. Delete the file only once every row is checked off — it should not be committed.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Ongoing prevention
|
|
||||||
|
|
||||||
Add a pre-commit hook (or a `nix flake check` step) running `gitleaks protect --staged` so a secret can't be committed again by accident. Document in the repo README (briefly) that new secrets go through `sops <file>` to edit, never as plaintext in a tracked file.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Definition of done
|
|
||||||
|
|
||||||
**Status as of 2026-07-20:** Milestones 1–3 are done — sops-nix is fully
|
|
||||||
wired (`.sops.yaml`, `secrets/*.yaml`, referenced via `hashedPasswordFile`/
|
|
||||||
`*File`/`sops.secrets.*.path` throughout), and history has been scrubbed
|
|
||||||
with `git-filter-repo` + force-push (this removed a GitHub fine-grained PAT
|
|
||||||
that had been committed in plaintext in `flake.nix`/`common/home.nix`
|
|
||||||
between 2025-07-16 and 2026-02-09, later migrated to sops but never scrubbed
|
|
||||||
from history until now). **Milestone 4 is not confirmed** — whether that PAT
|
|
||||||
(or any other historically-plaintext credential) was actually rotated, not
|
|
||||||
just re-encrypted, isn't something this repo can attest to; that's an
|
|
||||||
operator action against the issuing service (GitHub, etc.), not a repo
|
|
||||||
change. Do that before considering this fully closed.
|
|
||||||
|
|
||||||
- [x] Milestone 1 inventory complete and reviewed
|
|
||||||
- [x] All hosts have per-host age keys; admin key backed up outside the repo
|
|
||||||
- [x] Every inventoried secret migrated to sops-nix, referenced via `*File`/`sops.secrets.*.path`, nothing plaintext in the working tree
|
|
||||||
- [x] `nixos-rebuild dry-build` and at least one real `switch` verified per host
|
|
||||||
- [x] Working-tree scanner sweep clean
|
|
||||||
- [x] History rewritten with `git-filter-repo`, force-pushed, full-history scanner sweep clean
|
|
||||||
- [ ] All other clones deleted and re-cloned from the rewritten history — every clone that existed before 2026-07-20's rewrite (any other machine, WSL instance, or CI checkout) needs this
|
|
||||||
- [ ] Every credential in the original inventory rotated (not just re-encrypted) — **the GitHub PAT found in history specifically still needs this**
|
|
||||||
- [x] Pre-commit secret scanning hook added (`.githooks/pre-commit`, `gitleaks protect --staged`)
|
|
||||||
- [x] `secrets-inventory.md` deleted from the working directory (never committed)
|
|
||||||
@@ -41,6 +41,17 @@ warn-dirty = false
|
|||||||
build-users-group = nixbld
|
build-users-group = nixbld
|
||||||
EOF
|
EOF
|
||||||
|
|
||||||
|
# The official installer's single-user root path still shells out to
|
||||||
|
# `sudo` to create /nix even though it already knows it's running as
|
||||||
|
# root -- confirmed live against a sudo-less minimal Debian/Proxmox
|
||||||
|
# node, where it fails with "sudo: not found" and prints this exact
|
||||||
|
# mkdir/chown as the manual fix. Pre-create it so that branch of the
|
||||||
|
# installer is skipped entirely.
|
||||||
|
if [ ! -d /nix ]; then
|
||||||
|
mkdir -m 0755 /nix
|
||||||
|
chown root /nix
|
||||||
|
fi
|
||||||
|
|
||||||
sh <(curl -L https://nixos.org/nix/install) --no-daemon
|
sh <(curl -L https://nixos.org/nix/install) --no-daemon
|
||||||
else
|
else
|
||||||
sh <(curl -L https://nixos.org/nix/install) --no-daemon
|
sh <(curl -L https://nixos.org/nix/install) --no-daemon
|
||||||
|
|||||||
Executable
+172
@@ -0,0 +1,172 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Points a non-NixOS Debian machine's Nix install at nix-cache: adds it as
|
||||||
|
# a substituter (with cache.nixos.org kept as fallback) and, once the
|
||||||
|
# remote-builder private key is installed, as a distributed-build machine
|
||||||
|
# too.
|
||||||
|
#
|
||||||
|
# This is the non-NixOS equivalent of modules/nix-cache/client.nix +
|
||||||
|
# modules/nix-cache/remote-builder-client.nix -- those two only apply to
|
||||||
|
# hosts built from this flake. A plain Debian box with Nix installed
|
||||||
|
# (single- or multi-user install, nix-daemon running) has no NixOS module
|
||||||
|
# system to pick that config up, so this edits /etc/nix/nix.conf by hand
|
||||||
|
# instead. Run this ON the target Debian machine, as root.
|
||||||
|
#
|
||||||
|
# The values below mirror variables.nix / modules/nix-cache/client.nix in
|
||||||
|
# this repo -- update both if nix-cache is ever rebuilt with a new host
|
||||||
|
# key or the cache signing key is rotated (see docs/nix-cache.md).
|
||||||
|
#
|
||||||
|
# Usage:
|
||||||
|
# sudo ./configure-nix-cache-client.sh [--dry-run] [--no-remote-builder] [--no-restart]
|
||||||
|
#
|
||||||
|
# Env overrides (defaults match variables.nix):
|
||||||
|
# NIX_CACHE_HOST, NIX_CACHE_HOST_KEY, REMOTE_BUILDER_USER, REMOTE_BUILDER_KEY
|
||||||
|
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
: "${NIX_CACHE_HOST:=nix-cache}"
|
||||||
|
: "${NIX_CACHE_HOST_KEY:=ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHrMKZlIGUd3pH9G3AqbsruqUGjxIXMAZw52u9MwiBCn lxc-nix-cache}"
|
||||||
|
: "${REMOTE_BUILDER_USER:=nixremote}"
|
||||||
|
: "${REMOTE_BUILDER_KEY:=/root/.ssh/nixremote}"
|
||||||
|
|
||||||
|
CACHE_PUB_KEY="cache.local-1:usoWYanY3Kpq2+kDIS2nhWoLZiRxanmdysdzqCFBHW4="
|
||||||
|
FALLBACK_URL="https://cache.nixos.org/"
|
||||||
|
FALLBACK_PUB_KEY="cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY="
|
||||||
|
|
||||||
|
NIX_CONF="/etc/nix/nix.conf"
|
||||||
|
KNOWN_HOSTS="/etc/ssh/ssh_known_hosts"
|
||||||
|
MARKER_BEGIN="# BEGIN nix-cache client config (configure-nix-cache-client.sh)"
|
||||||
|
MARKER_END="# END nix-cache client config"
|
||||||
|
|
||||||
|
dry_run=0
|
||||||
|
with_remote_builder=1
|
||||||
|
restart_daemon=1
|
||||||
|
|
||||||
|
for arg in "$@"; do
|
||||||
|
case "$arg" in
|
||||||
|
--dry-run) dry_run=1 ;;
|
||||||
|
--no-remote-builder) with_remote_builder=0 ;;
|
||||||
|
--no-restart) restart_daemon=0 ;;
|
||||||
|
-h|--help)
|
||||||
|
sed -n '2,20p' "$0"
|
||||||
|
exit 0
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
echo "ERROR: unknown argument: $arg" >&2
|
||||||
|
exit 1
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
if [[ "$dry_run" -eq 0 && "$EUID" -ne 0 ]]; then
|
||||||
|
echo "ERROR: must run as root (writes $NIX_CONF and, unless --no-remote-builder, $KNOWN_HOSTS)." >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
if ! command -v nix >/dev/null 2>&1; then
|
||||||
|
echo "ERROR: no 'nix' binary on PATH -- install the Nix package manager first." >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ ! -f "$NIX_CONF" ]]; then
|
||||||
|
echo "ERROR: $NIX_CONF not found -- expected an existing multi-user Nix install." >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
builder_line=""
|
||||||
|
if [[ "$with_remote_builder" -eq 1 ]]; then
|
||||||
|
if [[ -f "$REMOTE_BUILDER_KEY" ]]; then
|
||||||
|
case "$(uname -m)" in
|
||||||
|
x86_64) nix_system="x86_64-linux" ;;
|
||||||
|
aarch64) nix_system="aarch64-linux" ;;
|
||||||
|
*)
|
||||||
|
echo "WARNING: unrecognized architecture '$(uname -m)' -- skipping remote builder, keeping substituter config." >&2
|
||||||
|
with_remote_builder=0
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
if [[ "$with_remote_builder" -eq 1 ]]; then
|
||||||
|
builder_line="builders = ssh://${REMOTE_BUILDER_USER}@${NIX_CACHE_HOST} ${nix_system} ${REMOTE_BUILDER_KEY} 4 2 big-parallel,kvm,nixos-test,benchmark"
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
echo "WARNING: $REMOTE_BUILDER_KEY not found -- skipping remote builder config (substituter still configured)." >&2
|
||||||
|
echo " See docs/nix-cache.md 'Remote builder SSH keys' for how to install it, then re-run this script." >&2
|
||||||
|
with_remote_builder=0
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
block="$(cat <<EOF
|
||||||
|
$MARKER_BEGIN
|
||||||
|
extra-substituters = http://${NIX_CACHE_HOST} ${FALLBACK_URL}
|
||||||
|
extra-trusted-public-keys = ${CACHE_PUB_KEY} ${FALLBACK_PUB_KEY}
|
||||||
|
EOF
|
||||||
|
)"
|
||||||
|
if [[ "$with_remote_builder" -eq 1 ]]; then
|
||||||
|
block="${block}
|
||||||
|
builders-use-substitutes = true
|
||||||
|
${builder_line}"
|
||||||
|
fi
|
||||||
|
block="${block}
|
||||||
|
$MARKER_END"
|
||||||
|
|
||||||
|
echo "== nix.conf block to install =="
|
||||||
|
echo "$block"
|
||||||
|
echo "================================"
|
||||||
|
|
||||||
|
if [[ "$dry_run" -eq 1 ]]; then
|
||||||
|
echo "(--dry-run: not writing $NIX_CONF)"
|
||||||
|
else
|
||||||
|
tmp_conf="$(mktemp)"
|
||||||
|
trap 'rm -f "$tmp_conf"' EXIT
|
||||||
|
|
||||||
|
if grep -qF "$MARKER_BEGIN" "$NIX_CONF"; then
|
||||||
|
awk -v begin="$MARKER_BEGIN" -v end="$MARKER_END" -v block="$block" '
|
||||||
|
$0 == begin { print block; skip = 1; next }
|
||||||
|
$0 == end { skip = 0; next }
|
||||||
|
skip { next }
|
||||||
|
{ print }
|
||||||
|
' "$NIX_CONF" > "$tmp_conf"
|
||||||
|
else
|
||||||
|
cp "$NIX_CONF" "$tmp_conf"
|
||||||
|
printf '\n%s\n' "$block" >> "$tmp_conf"
|
||||||
|
fi
|
||||||
|
|
||||||
|
cp "$NIX_CONF" "${NIX_CONF}.bak.$(date +%Y%m%d%H%M%S)"
|
||||||
|
install -m 0644 "$tmp_conf" "$NIX_CONF"
|
||||||
|
echo "Updated $NIX_CONF (backup saved alongside it)."
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ "$with_remote_builder" -eq 1 ]]; then
|
||||||
|
known_hosts_line="${NIX_CACHE_HOST} ${NIX_CACHE_HOST_KEY}"
|
||||||
|
if [[ "$dry_run" -eq 1 ]]; then
|
||||||
|
echo "(--dry-run: would ensure this line is present in $KNOWN_HOSTS)"
|
||||||
|
echo " $known_hosts_line"
|
||||||
|
else
|
||||||
|
mkdir -p "$(dirname "$KNOWN_HOSTS")"
|
||||||
|
touch "$KNOWN_HOSTS"
|
||||||
|
if ! grep -qF "$known_hosts_line" "$KNOWN_HOSTS" 2>/dev/null; then
|
||||||
|
echo "$known_hosts_line" >> "$KNOWN_HOSTS"
|
||||||
|
echo "Added nix-cache's SSH host key to $KNOWN_HOSTS."
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ "$dry_run" -eq 0 && "$restart_daemon" -eq 1 ]]; then
|
||||||
|
if command -v systemctl >/dev/null 2>&1 && systemctl is-active --quiet nix-daemon 2>/dev/null; then
|
||||||
|
systemctl restart nix-daemon
|
||||||
|
echo "Restarted nix-daemon to pick up the new config."
|
||||||
|
else
|
||||||
|
echo "nix-daemon not managed by systemd (or not running) -- restart it manually to pick up the new config."
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
cat <<EOF
|
||||||
|
|
||||||
|
Done. Verify with:
|
||||||
|
curl http://${NIX_CACHE_HOST}/nix-cache-info
|
||||||
|
nix show-config | grep -E 'substituters|trusted-public-keys|builders'
|
||||||
|
EOF
|
||||||
|
if [[ "$with_remote_builder" -eq 1 ]]; then
|
||||||
|
cat <<EOF
|
||||||
|
ssh -i ${REMOTE_BUILDER_KEY} ${REMOTE_BUILDER_USER}@${NIX_CACHE_HOST} nix-store --version
|
||||||
|
nix build nixpkgs#hello -L
|
||||||
|
EOF
|
||||||
|
fi
|
||||||
@@ -3,6 +3,15 @@
|
|||||||
# existing ones -- the manual workflows in docs/proxmox-images.md (VM) and
|
# existing ones -- the manual workflows in docs/proxmox-images.md (VM) and
|
||||||
# docs/auto-installer.md's "LXC hosts" section (container), automated.
|
# docs/auto-installer.md's "LXC hosts" section (container), automated.
|
||||||
#
|
#
|
||||||
|
# Images are built directly on the Proxmox node (PROXMOX_REMOTE_REPO_DIR /
|
||||||
|
# --remote-repo-dir in scripts/env.sh), not on whatever machine runs this
|
||||||
|
# script -- there's no multi-gigabyte image to transfer afterward. The first
|
||||||
|
# time a node doesn't have that repo path yet, it's bootstrapped: cloned from
|
||||||
|
# this checkout's own `origin` remote, then scripts/codex-setup.sh installs
|
||||||
|
# the build tooling (Nix, etc.). Every run after that just `git pull`s it and
|
||||||
|
# copies over the locally-managed host-keys/ (gitignored, so a git pull
|
||||||
|
# alone wouldn't carry it) before building.
|
||||||
|
#
|
||||||
# Usage:
|
# Usage:
|
||||||
# scripts/create-proxmox-resource.sh --type lxc|vm --host <name> [options]
|
# scripts/create-proxmox-resource.sh --type lxc|vm --host <name> [options]
|
||||||
# scripts/create-proxmox-resource.sh --type lxc|vm --list
|
# scripts/create-proxmox-resource.sh --type lxc|vm --list
|
||||||
@@ -64,11 +73,15 @@ Create mode (default):
|
|||||||
Refuses to run if this ID already exists.
|
Refuses to run if this ID already exists.
|
||||||
--disk-size <GB> lxc only: rootfs size for \`pct create\`
|
--disk-size <GB> lxc only: rootfs size for \`pct create\`
|
||||||
(default: \$PROXMOX_DEFAULT_LXC_DISK_GB, ${PROXMOX_DEFAULT_LXC_DISK_GB}).
|
(default: \$PROXMOX_DEFAULT_LXC_DISK_GB, ${PROXMOX_DEFAULT_LXC_DISK_GB}).
|
||||||
--image <path> Use this local image/tarball instead of
|
--image <path> Use this local image/tarball (uploaded to the
|
||||||
checking the node / building one from the flake.
|
node via scp) instead of checking the node /
|
||||||
|
building one there from the flake.
|
||||||
--force-rebuild Skip the "does the node already have this
|
--force-rebuild Skip the "does the node already have this
|
||||||
image" check -- always build fresh and
|
image" check -- always build fresh and
|
||||||
overwrite what's there.
|
overwrite what's there.
|
||||||
|
--remote-repo-dir <path> Where this flake repo lives (or gets
|
||||||
|
cloned) on the node, and is built from
|
||||||
|
(default: \$PROXMOX_REMOTE_REPO_DIR, ${PROXMOX_REMOTE_REPO_DIR}).
|
||||||
--allow-duplicate-host Required if a VM/CT identified as --host
|
--allow-duplicate-host Required if a VM/CT identified as --host
|
||||||
already exists on the node (checked live via
|
already exists on the node (checked live via
|
||||||
qm/pct, not any file in this repo) --
|
qm/pct, not any file in this repo) --
|
||||||
@@ -131,6 +144,7 @@ storage="$PROXMOX_STORAGE"
|
|||||||
iso_storage="$PROXMOX_ISO_STORAGE"
|
iso_storage="$PROXMOX_ISO_STORAGE"
|
||||||
bridge="$PROXMOX_BRIDGE"
|
bridge="$PROXMOX_BRIDGE"
|
||||||
node="$PROXMOX_HOST"
|
node="$PROXMOX_HOST"
|
||||||
|
remote_repo_dir="$PROXMOX_REMOTE_REPO_DIR"
|
||||||
do_list=0
|
do_list=0
|
||||||
allow_duplicate_host=0
|
allow_duplicate_host=0
|
||||||
force_rebuild=0
|
force_rebuild=0
|
||||||
@@ -151,6 +165,7 @@ while [[ $# -gt 0 ]]; do
|
|||||||
--iso-storage) iso_storage="$2"; shift 2 ;;
|
--iso-storage) iso_storage="$2"; shift 2 ;;
|
||||||
--bridge) bridge="$2"; shift 2 ;;
|
--bridge) bridge="$2"; shift 2 ;;
|
||||||
--node) node="$2"; shift 2 ;;
|
--node) node="$2"; shift 2 ;;
|
||||||
|
--remote-repo-dir) remote_repo_dir="$2"; shift 2 ;;
|
||||||
--list) do_list=1; shift ;;
|
--list) do_list=1; shift ;;
|
||||||
--allow-duplicate-host) allow_duplicate_host=1; shift ;;
|
--allow-duplicate-host) allow_duplicate_host=1; shift ;;
|
||||||
--force-rebuild) force_rebuild=1; shift ;;
|
--force-rebuild) force_rebuild=1; shift ;;
|
||||||
@@ -498,6 +513,99 @@ if [[ "$type" == "lxc" ]]; then
|
|||||||
fi
|
fi
|
||||||
remote_path="${remote_dir}/${remote_filename}"
|
remote_path="${remote_dir}/${remote_filename}"
|
||||||
|
|
||||||
|
# --- ensure the flake repo (+ tooling) exists on the node, and is current --
|
||||||
|
# Bootstraps once (git clone from this checkout's own `origin`, then
|
||||||
|
# scripts/codex-setup.sh installs Nix + friends) if ${remote_repo_dir}
|
||||||
|
# doesn't exist yet on the node; otherwise just `git pull`s it, so the image
|
||||||
|
# built there reflects what's actually committed and pushed. Only called
|
||||||
|
# right before an actual remote build below -- reusing an image already on
|
||||||
|
# the node, or an explicit --image, never touch the node's checkout at all.
|
||||||
|
ensure_remote_repo() {
|
||||||
|
echo
|
||||||
|
echo "==> Ensuring ${remote_repo_dir} exists and is current on ${node}..."
|
||||||
|
if [[ "$dry_run" -eq 1 ]]; then
|
||||||
|
echo "[dry-run] would ensure ${remote_repo_dir} exists on ${node} (clone if missing, git pull if present), and would verify/bootstrap build tooling there (scripts/codex-setup.sh) if \`nix\` isn't already on PATH -- and if that bootstrap actually ran, would also configure ${node} as a nix-cache client (scripts/configure-nix-cache-client.sh)"
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
|
||||||
|
if ssh "$ssh_target" "test -d '${remote_repo_dir}/.git'"; then
|
||||||
|
echo "Repo present -- pulling latest..."
|
||||||
|
ssh "$ssh_target" "cd '${remote_repo_dir}' && git pull --ff-only"
|
||||||
|
else
|
||||||
|
local origin_url
|
||||||
|
origin_url="$(git -C "$repo_root" remote get-url origin 2>/dev/null || true)"
|
||||||
|
if [[ -z "$origin_url" ]]; then
|
||||||
|
echo "ERROR: ${remote_repo_dir} doesn't exist on ${node}, and this checkout has no" >&2
|
||||||
|
echo "'origin' remote to clone from. Set one (git remote add origin <url>) or create" >&2
|
||||||
|
echo "${remote_repo_dir} on ${node} yourself (e.g. git clone), then re-run." >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
echo "Not present -- cloning from ${origin_url}..."
|
||||||
|
ssh "$ssh_target" "git clone '${origin_url}' '${remote_repo_dir}'"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Trivial check, run every time (not just right after a fresh clone) --
|
||||||
|
# confirmed live: a first bootstrap can clone the repo successfully and
|
||||||
|
# still leave the node without a working `nix` (e.g. the node had no
|
||||||
|
# `sudo`, which the Nix installer's root path depends on -- see the fix
|
||||||
|
# in scripts/codex-setup.sh), and a later run with the repo already
|
||||||
|
# present would otherwise never retry it. Sources
|
||||||
|
# scripts/lib/nix-bootstrap.sh's ensure_nix_profile first -- a
|
||||||
|
# single-user Nix install typically only gets sourced into login shells,
|
||||||
|
# and ssh's non-interactive command execution is neither, so a
|
||||||
|
# freshly-installed `nix` still wouldn't be on PATH here without it.
|
||||||
|
#
|
||||||
|
# Just `nix` today -- the only thing the remote build commands below
|
||||||
|
# actually invoke -- but a list (not a single hardcoded check) so a
|
||||||
|
# future remote step needing another tool can add itself here instead of
|
||||||
|
# growing a parallel check.
|
||||||
|
local remote_required_cmds=(nix)
|
||||||
|
local tooling_check_cmd="cd '${remote_repo_dir}' && . scripts/lib/nix-bootstrap.sh && ensure_nix_profile"
|
||||||
|
local cmd
|
||||||
|
for cmd in "${remote_required_cmds[@]}"; do
|
||||||
|
tooling_check_cmd="${tooling_check_cmd} && command -v ${cmd}"
|
||||||
|
done
|
||||||
|
|
||||||
|
if ssh "$ssh_target" "$tooling_check_cmd" >/dev/null 2>&1; then
|
||||||
|
echo "Build tooling already present on ${node}."
|
||||||
|
else
|
||||||
|
echo "==> Bootstrapping build tooling on ${node} (scripts/codex-setup.sh)..."
|
||||||
|
ssh "$ssh_target" "cd '${remote_repo_dir}' && bash scripts/codex-setup.sh"
|
||||||
|
|
||||||
|
# Only on this first-time bootstrap, not every run -- a node that
|
||||||
|
# already has tooling either already went through this once, or had
|
||||||
|
# it configured some other way, and re-running is harmless but
|
||||||
|
# pointless. Non-fatal: this only makes the node's own builds faster
|
||||||
|
# (substitute from nix-cache instead of building from source) and
|
||||||
|
# offloadable to it as a remote builder -- worth trying, not worth
|
||||||
|
# aborting the image build over if nix-cache happens to be down right
|
||||||
|
# now. Needs ensure_nix_profile first, same as the tooling_check_cmd
|
||||||
|
# above -- ssh's non-interactive command execution won't have picked
|
||||||
|
# up a freshly single-user-installed `nix` otherwise.
|
||||||
|
echo "==> Configuring ${node} as a nix-cache substituter/remote-builder client..."
|
||||||
|
if ! ssh "$ssh_target" "cd '${remote_repo_dir}' && . scripts/lib/nix-bootstrap.sh && ensure_nix_profile && bash scripts/configure-nix-cache-client.sh"; then
|
||||||
|
echo "WARNING: configure-nix-cache-client.sh failed on ${node} -- continuing without it (${node} will build from source / against cache.nixos.org only)." >&2
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# --- sync locally-managed host-keys/ to the node ---------------------------
|
||||||
|
# Gitignored (see .gitignore), so `git pull` above never carries it -- both
|
||||||
|
# build paths need it present as NIXOS_HOST_KEYS_DIR / --pre-format-files
|
||||||
|
# input on the node itself now that the build runs there. scp (not rsync,
|
||||||
|
# not already a dependency anywhere else in this repo) mirrors how this
|
||||||
|
# script already transfers the --image case below.
|
||||||
|
sync_remote_host_keys() {
|
||||||
|
echo
|
||||||
|
echo "==> Syncing host-keys/ to ${node}..."
|
||||||
|
if [[ "$dry_run" -eq 1 ]]; then
|
||||||
|
echo "[dry-run] would copy ${repo_root}/host-keys/ to ${ssh_target}:${remote_repo_dir}/host-keys/"
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
ssh "$ssh_target" "mkdir -p '${remote_repo_dir}/host-keys'"
|
||||||
|
scp -pr "${repo_root}/host-keys/." "${ssh_target}:${remote_repo_dir}/host-keys/"
|
||||||
|
}
|
||||||
|
|
||||||
# --- build (or reuse an image already on the node) ------------------------
|
# --- build (or reuse an image already on the node) ------------------------
|
||||||
echo
|
echo
|
||||||
local_image=""
|
local_image=""
|
||||||
@@ -514,7 +622,7 @@ else
|
|||||||
if [[ "$dry_run" -eq 1 ]]; then
|
if [[ "$dry_run" -eq 1 ]]; then
|
||||||
echo "[dry-run] would check: ssh ${ssh_target} -- test -f ${remote_path}"
|
echo "[dry-run] would check: ssh ${ssh_target} -- test -f ${remote_path}"
|
||||||
elif ssh "$ssh_target" "test -f '${remote_path}'" 2>/dev/null; then
|
elif ssh "$ssh_target" "test -f '${remote_path}'" 2>/dev/null; then
|
||||||
echo "Found it -- reusing, skipping build and upload (use --force-rebuild to override)."
|
echo "Found it -- reusing, skipping build (use --force-rebuild to override)."
|
||||||
image_already_remote=1
|
image_already_remote=1
|
||||||
else
|
else
|
||||||
echo "Not found -- will build."
|
echo "Not found -- will build."
|
||||||
@@ -522,70 +630,132 @@ else
|
|||||||
fi
|
fi
|
||||||
|
|
||||||
if [[ "$image_already_remote" -eq 0 && -z "$local_image" ]]; then
|
if [[ "$image_already_remote" -eq 0 && -z "$local_image" ]]; then
|
||||||
# Mirrors the real build commands' "${NIX_OPTS[@]}" below -- nix_extra_opts
|
ensure_remote_repo
|
||||||
# (called earlier, once) has already decided whether nix-cache is in play,
|
sync_remote_host_keys
|
||||||
# and the dry-run preview needs to reflect that decision instead of always
|
|
||||||
# printing the same command regardless of outcome.
|
# Relayed into the remote build below exactly as decided by the local
|
||||||
|
# nix_extra_opts call earlier in this script -- that decision (whether
|
||||||
|
# nix-cache is reachable) is made once, locally, same as it always has
|
||||||
|
# been; only *where* the resulting "${NIX_OPTS[@]}" gets used as a `nix
|
||||||
|
# build` flag moves to the node. NIX_EXTRA_OPTS is already a %q-quoted
|
||||||
|
# string built for exactly this eval-based reconstruction (see env.sh).
|
||||||
nix_opts_display=""
|
nix_opts_display=""
|
||||||
if [[ ${#NIX_OPTS[@]} -gt 0 ]]; then
|
if [[ ${#NIX_OPTS[@]} -gt 0 ]]; then
|
||||||
printf -v nix_opts_display '%q ' "${NIX_OPTS[@]}"
|
printf -v nix_opts_display '%q ' "${NIX_OPTS[@]}"
|
||||||
nix_opts_display=" ${nix_opts_display% }"
|
nix_opts_display=" ${nix_opts_display% }"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
if [[ "$type" == "lxc" ]]; then
|
if [[ "$type" == "lxc" ]]; then
|
||||||
if [[ "$dry_run" -eq 1 ]]; then
|
if [[ "$dry_run" -eq 1 ]]; then
|
||||||
echo "[dry-run] would build: NIXOS_HOST_KEYS_DIR=${repo_root}/host-keys nix build --impure \\"
|
echo "[dry-run] would build on ${node}: NIXOS_HOST_KEYS_DIR=\$(pwd)/host-keys nix build --impure \\"
|
||||||
echo "[dry-run] --no-use-registries --no-accept-flake-config${nix_opts_display} \\"
|
echo "[dry-run] --no-use-registries --no-accept-flake-config${nix_opts_display} \\"
|
||||||
echo "[dry-run] .#nixosConfigurations.${flake_target}.config.system.build.tarball"
|
echo "[dry-run] .#nixosConfigurations.${flake_target}.config.system.build.tarball"
|
||||||
|
echo "[dry-run] would stage the result at ${remote_path}"
|
||||||
local_image="<built-tarball>"
|
local_image="<built-tarball>"
|
||||||
else
|
else
|
||||||
echo "==> Building LXC tarball for ${flake_target}..."
|
echo "==> Building LXC tarball for ${flake_target} on ${node}..."
|
||||||
NIXOS_HOST_KEYS_DIR="${repo_root}/host-keys" nix build --impure \
|
# Built as a single already-%q-quoted command string, not separate ssh
|
||||||
|
# argv elements -- ssh joins remote command args with plain spaces and
|
||||||
|
# hands the result to the remote shell to re-split, which would
|
||||||
|
# otherwise scatter NIX_EXTRA_OPTS (itself several space-separated,
|
||||||
|
# %q-quoted tokens) across the wrong positional parameters below.
|
||||||
|
printf -v remote_cmd 'bash -s -- %q %q %q %q %q' \
|
||||||
|
"$remote_repo_dir" "$flake_target" "$remote_dir" "$remote_filename" "$NIX_EXTRA_OPTS"
|
||||||
|
ssh "$ssh_target" "$remote_cmd" <<'REMOTE_SCRIPT'
|
||||||
|
set -euo pipefail
|
||||||
|
repo_dir="$1"; target="$2"; dest_dir="$3"; dest_name="$4"; nix_extra_opts_str="$5"
|
||||||
|
declare -a NIX_OPTS=()
|
||||||
|
[[ -n "$nix_extra_opts_str" ]] && eval "NIX_OPTS=(${nix_extra_opts_str})"
|
||||||
|
cd "$repo_dir"
|
||||||
|
# A single-user Nix install only gets sourced into login shells; this ssh
|
||||||
|
# session is neither, so `nix` wouldn't otherwise be on PATH here even
|
||||||
|
# right after a successful install.
|
||||||
|
. scripts/lib/nix-bootstrap.sh
|
||||||
|
ensure_nix_profile
|
||||||
|
NIXOS_HOST_KEYS_DIR="$(pwd)/host-keys" nix build --impure \
|
||||||
--no-use-registries --no-accept-flake-config "${NIX_OPTS[@]}" \
|
--no-use-registries --no-accept-flake-config "${NIX_OPTS[@]}" \
|
||||||
".#nixosConfigurations.${flake_target}.config.system.build.tarball" \
|
".#nixosConfigurations.${target}.config.system.build.tarball" \
|
||||||
--out-link "${repo_root}/result-${flake_target}"
|
--out-link "result-${target}"
|
||||||
local_image="$(find "${repo_root}/result-${flake_target}/tarball" -maxdepth 1 -type f | head -1)"
|
built="$(find "result-${target}/tarball" -maxdepth 1 -type f | head -1)"
|
||||||
echo "Built: ${local_image}"
|
if [[ -z "$built" ]]; then
|
||||||
|
echo "ERROR: no tarball found under result-${target}/tarball after build." >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
mkdir -p "$dest_dir"
|
||||||
|
cp "$built" "${dest_dir}/${dest_name}"
|
||||||
|
echo "Built and staged: ${dest_dir}/${dest_name}"
|
||||||
|
REMOTE_SCRIPT
|
||||||
|
local_image="$remote_path"
|
||||||
|
echo "Built on ${node}: ${remote_path}"
|
||||||
fi
|
fi
|
||||||
else
|
else
|
||||||
|
# PROXMOX_SSH_USER defaults to root (env.sh), which needs no sudo and
|
||||||
|
# can't assume it's even installed on a minimal node -- only shell out
|
||||||
|
# through sudo when actually running as a non-root SSH user.
|
||||||
|
sudo_prefix="sudo"
|
||||||
|
sudo_display="sudo "
|
||||||
|
if [[ "$PROXMOX_SSH_USER" == "root" ]]; then
|
||||||
|
sudo_prefix=""
|
||||||
|
sudo_display=""
|
||||||
|
fi
|
||||||
if [[ "$dry_run" -eq 1 ]]; then
|
if [[ "$dry_run" -eq 1 ]]; then
|
||||||
echo "[dry-run] would build: nix build --no-use-registries --no-accept-flake-config${nix_opts_display} \\"
|
echo "[dry-run] would build on ${node}: nix build --no-use-registries --no-accept-flake-config${nix_opts_display} \\"
|
||||||
echo "[dry-run] .#nixosConfigurations.${flake_target}.config.system.build.diskoImagesScript"
|
echo "[dry-run] .#nixosConfigurations.${flake_target}.config.system.build.diskoImagesScript"
|
||||||
echo "[dry-run] would run: sudo ./result-${flake_target} \\"
|
echo "[dry-run] would run: ${sudo_display}./result-${flake_target} \\"
|
||||||
echo "[dry-run] --pre-format-files host-keys/${flake_target}_ssh_host_ed25519_key /etc/ssh/ssh_host_ed25519_key \\"
|
echo "[dry-run] --pre-format-files host-keys/${flake_target}_ssh_host_ed25519_key /etc/ssh/ssh_host_ed25519_key \\"
|
||||||
echo "[dry-run] --pre-format-files host-keys/${flake_target}_ssh_host_ed25519_key.pub /etc/ssh/ssh_host_ed25519_key.pub \\"
|
echo "[dry-run] --pre-format-files host-keys/${flake_target}_ssh_host_ed25519_key.pub /etc/ssh/ssh_host_ed25519_key.pub \\"
|
||||||
echo "[dry-run] --build-memory 2048"
|
echo "[dry-run] --build-memory 2048"
|
||||||
|
echo "[dry-run] would stage the result at ${remote_path}"
|
||||||
local_image="<built-image>.raw"
|
local_image="<built-image>.raw"
|
||||||
else
|
else
|
||||||
echo "==> Building Disko image script for ${flake_target}..."
|
echo "==> Building Disko image for ${flake_target} on ${node}..."
|
||||||
|
# See the LXC branch above for why this is one %q-quoted command
|
||||||
|
# string rather than separate ssh argv elements.
|
||||||
|
printf -v remote_cmd 'bash -s -- %q %q %q %q %q %q' \
|
||||||
|
"$remote_repo_dir" "$flake_target" "$remote_dir" "$remote_filename" "$NIX_EXTRA_OPTS" "$sudo_prefix"
|
||||||
|
ssh "$ssh_target" "$remote_cmd" <<'REMOTE_SCRIPT'
|
||||||
|
set -euo pipefail
|
||||||
|
repo_dir="$1"; target="$2"; dest_dir="$3"; dest_name="$4"; nix_extra_opts_str="$5"; sudo_prefix="$6"
|
||||||
|
declare -a NIX_OPTS=()
|
||||||
|
[[ -n "$nix_extra_opts_str" ]] && eval "NIX_OPTS=(${nix_extra_opts_str})"
|
||||||
|
cd "$repo_dir"
|
||||||
|
. scripts/lib/nix-bootstrap.sh
|
||||||
|
ensure_nix_profile
|
||||||
nix build --no-use-registries --no-accept-flake-config "${NIX_OPTS[@]}" \
|
nix build --no-use-registries --no-accept-flake-config "${NIX_OPTS[@]}" \
|
||||||
".#nixosConfigurations.${flake_target}.config.system.build.diskoImagesScript" \
|
".#nixosConfigurations.${target}.config.system.build.diskoImagesScript" \
|
||||||
--out-link "${repo_root}/result-${flake_target}"
|
--out-link "result-${target}"
|
||||||
echo "==> Running it (builds the .raw image in a temporary QEMU VM, needs sudo)..."
|
$sudo_prefix "./result-${target}" \
|
||||||
( cd "$repo_root" && sudo "./result-${flake_target}" \
|
--pre-format-files "host-keys/${target}_ssh_host_ed25519_key" /etc/ssh/ssh_host_ed25519_key \
|
||||||
--pre-format-files "host-keys/${flake_target}_ssh_host_ed25519_key" /etc/ssh/ssh_host_ed25519_key \
|
--pre-format-files "host-keys/${target}_ssh_host_ed25519_key.pub" /etc/ssh/ssh_host_ed25519_key.pub \
|
||||||
--pre-format-files "host-keys/${flake_target}_ssh_host_ed25519_key.pub" /etc/ssh/ssh_host_ed25519_key.pub \
|
--build-memory 2048
|
||||||
--build-memory 2048 )
|
built="$(find . -maxdepth 1 -name '*.raw' -newer "result-${target}" | head -1)"
|
||||||
local_image="$(find "$repo_root" -maxdepth 1 -name "*.raw" -newer "${repo_root}/result-${flake_target}" | head -1)"
|
if [[ -z "$built" ]]; then
|
||||||
if [[ -z "$local_image" ]]; then
|
echo "ERROR: no .raw image found in ${repo_dir} after build." >&2
|
||||||
echo "ERROR: expected a .raw image after the build but didn't find one in ${repo_root}." >&2
|
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
echo "Built: ${local_image}"
|
mkdir -p "$dest_dir"
|
||||||
|
mv "$built" "${dest_dir}/${dest_name}"
|
||||||
|
echo "Built and staged: ${dest_dir}/${dest_name}"
|
||||||
|
REMOTE_SCRIPT
|
||||||
|
local_image="$remote_path"
|
||||||
|
echo "Built on ${node}: ${remote_path}"
|
||||||
fi
|
fi
|
||||||
fi
|
fi
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# --- upload (skip entirely if reusing an image already on the node) ------
|
# --- upload -- only for an explicit --image; a build above stages its
|
||||||
|
# result directly at ${remote_path} on the node already, and reusing an
|
||||||
|
# image already on the node needs nothing transferred either. ------------
|
||||||
echo
|
echo
|
||||||
if [[ "$image_already_remote" -eq 1 ]]; then
|
if [[ -n "$image" ]]; then
|
||||||
: # nothing to upload
|
if [[ "$dry_run" -eq 1 ]]; then
|
||||||
elif [[ "$dry_run" -eq 1 ]]; then
|
|
||||||
echo "[dry-run] would upload: scp ${local_image} ${ssh_target}:${remote_path}"
|
echo "[dry-run] would upload: scp ${local_image} ${ssh_target}:${remote_path}"
|
||||||
else
|
else
|
||||||
echo "==> Uploading to ${node}:${remote_path}..."
|
echo "==> Uploading to ${node}:${remote_path}..."
|
||||||
ssh "$ssh_target" "mkdir -p ${remote_dir}"
|
ssh "$ssh_target" "mkdir -p ${remote_dir}"
|
||||||
scp "$local_image" "${ssh_target}:${remote_path}"
|
scp "$local_image" "${ssh_target}:${remote_path}"
|
||||||
fi
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
# --- create -----------------------------------------------------------------
|
# --- create -----------------------------------------------------------------
|
||||||
echo
|
echo
|
||||||
|
|||||||
+10
-1
@@ -14,6 +14,14 @@
|
|||||||
: "${PROXMOX_HOST:=pve.sweet.home}"
|
: "${PROXMOX_HOST:=pve.sweet.home}"
|
||||||
: "${PROXMOX_SSH_USER:=root}"
|
: "${PROXMOX_SSH_USER:=root}"
|
||||||
|
|
||||||
|
# Where this flake repo lives on the Proxmox node itself.
|
||||||
|
# scripts/create-proxmox-resource.sh builds images directly on the node
|
||||||
|
# instead of transferring them over the network -- it clones the repo here
|
||||||
|
# (from this checkout's own `origin` remote) the first time it doesn't
|
||||||
|
# find it, installing build tooling via scripts/codex-setup.sh, then
|
||||||
|
# `git pull`s it before every subsequent build.
|
||||||
|
: "${PROXMOX_REMOTE_REPO_DIR:=/root/nixos}"
|
||||||
|
|
||||||
# Storage pool names -- Proxmox's own stock-install defaults, but this
|
# Storage pool names -- Proxmox's own stock-install defaults, but this
|
||||||
# varies a lot by setup (ZFS pool name, custom LVM-thin volume, etc.).
|
# varies a lot by setup (ZFS pool name, custom LVM-thin volume, etc.).
|
||||||
# Verify with `pvesm status` on the node and correct these if wrong.
|
# Verify with `pvesm status` on the node and correct these if wrong.
|
||||||
@@ -57,7 +65,8 @@
|
|||||||
|
|
||||||
export PROXMOX_HOST PROXMOX_SSH_USER PROXMOX_STORAGE PROXMOX_ISO_STORAGE \
|
export PROXMOX_HOST PROXMOX_SSH_USER PROXMOX_STORAGE PROXMOX_ISO_STORAGE \
|
||||||
PROXMOX_BRIDGE PROXMOX_DEFAULT_CORES PROXMOX_DEFAULT_MEMORY_MB \
|
PROXMOX_BRIDGE PROXMOX_DEFAULT_CORES PROXMOX_DEFAULT_MEMORY_MB \
|
||||||
PROXMOX_DEFAULT_LXC_DISK_GB PROXMOX_DEFAULT_LXC_FEATURES
|
PROXMOX_DEFAULT_LXC_DISK_GB PROXMOX_DEFAULT_LXC_FEATURES \
|
||||||
|
PROXMOX_REMOTE_REPO_DIR
|
||||||
|
|
||||||
# Matches variables.nix's nixCacheHost -- update both if it ever changes.
|
# Matches variables.nix's nixCacheHost -- update both if it ever changes.
|
||||||
: "${NIX_CACHE_HOST:=nix-cache}"
|
: "${NIX_CACHE_HOST:=nix-cache}"
|
||||||
|
|||||||
+16
-16
@@ -4,31 +4,31 @@ sops:
|
|||||||
age:
|
age:
|
||||||
- enc: |
|
- enc: |
|
||||||
-----BEGIN AGE ENCRYPTED FILE-----
|
-----BEGIN AGE ENCRYPTED FILE-----
|
||||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBmOHlPcGZvN3o5aEZCcFdz
|
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBPWE1HTUhiSUp5ZEUwWEpI
|
||||||
WlYydFdoQ0ZHbElMdTRkZWljMmoxTHZqNjJZCnE3cmgzKzZya3FROEczbHVveDND
|
bGpkZlBIMUo5ZlYrQ09SN3Q1a0ZkQ0ZnOEhVCnJPNEZQenVWWGZiODlzQzNEc1Zq
|
||||||
VnEzRW12cnRKdzhkZm5uTXpkTEtrNUEKLS0tIHFDbkc4Mk4wVlM2R29zZXYwR2Ri
|
c3l4OWZJTElJc2Y2UE15OGtEUzhyY1EKLS0tIHNJUStyWnlQWjZBbEZjQ3UwdUpz
|
||||||
ODBML1p4eUZiZldYUERQTUhTU1llV0UKxjvH6zbW6wKghzR1o34CyKPEa2FqZmo0
|
ZndoUDR6bisrNGJCUHk3TGI4bTZaMFUK87fFsm9ne9s+PK2pcwtrDjqyGBss2r2E
|
||||||
PxgqyuXkIwas9soXVAkScx7ElaV09Fjaj+mDrKwi4a+DwdoSP7czyA==
|
8lhqoeiKZ2j96z8kP/7ChzovwTCmqdcmAQuyNQD+ZAFijseipSvfbQ==
|
||||||
-----END AGE ENCRYPTED FILE-----
|
-----END AGE ENCRYPTED FILE-----
|
||||||
recipient: age1njap586hc0q43kr03g6c8eqhdsmk8zcafkl3f83xwlc2gqhlmfgs4tmwad
|
recipient: age1njap586hc0q43kr03g6c8eqhdsmk8zcafkl3f83xwlc2gqhlmfgs4tmwad
|
||||||
- enc: |
|
- enc: |
|
||||||
-----BEGIN AGE ENCRYPTED FILE-----
|
-----BEGIN AGE ENCRYPTED FILE-----
|
||||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSB0VDBXYUZNbnpEZHlqV05C
|
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBUYi9SRFFGV3Z6cFd2Znk5
|
||||||
WWxQZjVYcWlZeXlGc0RiaUpERzR0T21CWWtBCkM0dW5kYlFNK1RGSzRudFZ2Z0t3
|
b2FLbWtzTllJMDBUaGk0NTViOTNBa2hQclVZClZHKzNhbGVjQUJhWkFWdTFBMG5a
|
||||||
aVBreElGM1BIT0RzQkxTVTA3S2NhQncKLS0tIERyTU8zWWRTRm95SmRZQ1BhalVV
|
cUFJdUdyVG5HQXJRRnJId3hqRTN2cXMKLS0tIEFMRjh3WE1ON0U2TTNTZ3hxMTR4
|
||||||
SVdBajN1V1BuM2s4K216S3c0VHczNlkKM5jvsSEfCBA5uZRjBJNbM91lLRQkj+jK
|
ZGRlemlIbDZKeExmVHROc3Eyak5DdzQKaLwIVDi6BN4cxpVxJoqTYvJETPOp4thc
|
||||||
rM5uSfGLTvSjPgXIMIq03OXxH1CE7GoKxAPwFrJdAFMMQcutIethhw==
|
l9uVMvIGuEsEZgDsvShw1dYLljd+uGy/A+dXbcxIUCP/mmPkwmd1Pw==
|
||||||
-----END AGE ENCRYPTED FILE-----
|
-----END AGE ENCRYPTED FILE-----
|
||||||
recipient: age120le4a5l8dh3lyfgvmj3d9ksmej6ajs5mer5y7r0vfg3x9fn69dqf8xgzu
|
recipient: age120le4a5l8dh3lyfgvmj3d9ksmej6ajs5mer5y7r0vfg3x9fn69dqf8xgzu
|
||||||
- enc: |
|
- enc: |
|
||||||
-----BEGIN AGE ENCRYPTED FILE-----
|
-----BEGIN AGE ENCRYPTED FILE-----
|
||||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSAvbk9tRE9jYkpxNThQeElH
|
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSArOWovSW9DeFpxL0VDUDQ3
|
||||||
dWpKYThDb1ZkUWtMWlRxc2tYbjhBbEFEcWtVCkNiK0NBRko2TkJQdUVtdGtHNGcv
|
SHUwTzJVZUtPV01ZRkdCUXZGL2lTRCtCNFNnCjBSNExqRW5mTEN5SFVucHJHSzZt
|
||||||
andSaUlKSFA4THRyZXNTYmI0Yk5WbmsKLS0tIHlQTUtpWDdPeTVZL0M1RElRdFNk
|
cDlNc3BjY3M1c1k1Z2tkVEg4R1pacGsKLS0tIEFWbHNKZW0vbVh1Y2VhQW93OUwx
|
||||||
QWlGdFo5NkZWSmY0YXdpNzNUQnlsK3MKtzC0bM7Ek+K73nMranOA1Mc98RUnYnq1
|
MWV0eW9sOXdQd0l2ZjlWOEVVc1dwcTgK2s4p9xoNkawH2OkGsl80bNIo3ad5vn4W
|
||||||
hAt0QEFKWK4QVKubaN/rG3AzE0U7qPKWHTzoxgnAiL3WyV9teLW+iA==
|
Z2w+jwppSoUmbQnD3WFbLmSSxmuobmU8HILwElv6SZu+KE3aspF6XA==
|
||||||
-----END AGE ENCRYPTED FILE-----
|
-----END AGE ENCRYPTED FILE-----
|
||||||
recipient: age164px2a8e48ptsf9ngtan38aa6jls4jdl26mzrgzf6sn3vcvt49hqjrgr8w
|
recipient: age1xjst4frdh0th6q8m7p7u9g5af7ty5jqeum0p6z8a52a9q7st7ewqw8yl9j
|
||||||
lastmodified: "2026-07-19T23:30:21Z"
|
lastmodified: "2026-07-19T23:30:21Z"
|
||||||
mac: ENC[AES256_GCM,data:kLGE2xawQT7mx+sfw68hmGk5nCEGiEjZrqTEl9B1dtQmTrMwmoVr/1RISi4LfJrwxy31mDgff4lcIL4wIJuM373uk3X8j4RNyYQNTfKEkORT6r8NHeepNs267O77pKGd7OmcM4MT/BqOnB8ELS7Wlf2ect7CAlvUUVyc8icxgZE=,iv:EYLDsHYHZ1XOQXafOTqHHWpk/OBNq/R6IJnOBYV33E4=,tag:thxrCPC5oGvDjhK7Dz87YA==,type:str]
|
mac: ENC[AES256_GCM,data:kLGE2xawQT7mx+sfw68hmGk5nCEGiEjZrqTEl9B1dtQmTrMwmoVr/1RISi4LfJrwxy31mDgff4lcIL4wIJuM373uk3X8j4RNyYQNTfKEkORT6r8NHeepNs267O77pKGd7OmcM4MT/BqOnB8ELS7Wlf2ect7CAlvUUVyc8icxgZE=,iv:EYLDsHYHZ1XOQXafOTqHHWpk/OBNq/R6IJnOBYV33E4=,tag:thxrCPC5oGvDjhK7Dz87YA==,type:str]
|
||||||
unencrypted_suffix: _unencrypted
|
unencrypted_suffix: _unencrypted
|
||||||
|
|||||||
@@ -137,6 +137,14 @@
|
|||||||
# shortcuts on the gui build type (hosts/nixos/home.nix).
|
# shortcuts on the gui build type (hosts/nixos/home.nix).
|
||||||
pveWeb = 8006;
|
pveWeb = 8006;
|
||||||
pbsWeb = 8007;
|
pbsWeb = 8007;
|
||||||
|
|
||||||
|
# Tor relay's ORPort — the port other Tor relays connect to for onion
|
||||||
|
# routing traffic (modules/tor/enable-relay.nix). Tor's own conventional
|
||||||
|
# default; opened via services.tor.openFirewall rather than
|
||||||
|
# networking.firewall.allowedTCPPorts directly, but kept here anyway so
|
||||||
|
# it's not a bare literal duplicated between the relay's settings and
|
||||||
|
# anything else that ever needs to reference it.
|
||||||
|
torRelayOrPort = 9001;
|
||||||
};
|
};
|
||||||
|
|
||||||
# .raw disk image size for every proxmox-* host's standalone Disko image
|
# .raw disk image size for every proxmox-* host's standalone Disko image
|
||||||
|
|||||||
Reference in New Issue
Block a user