Compare commits

...
Author SHA1 Message Date
beatzaplentyandClaude Sonnet 4.6 5497a5b0ae feat: add nixos@nixos workstation SSH key to all hosts
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m44s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Rcq7iXrN9eYUTUEWrbWqyX
2026-07-23 11:05:51 +10:00
beatzaplenty e10e493ddd Merge pull request 'Worktree harmonic jingling bee' (#47) from worktree-harmonic-jingling-bee into main
Check NixOS configurations / eval-hosts (push) Successful in 10m20s
Reviewed-on: #47
2026-07-23 00:40:36 +00:00
beatzaplentyandClaude Sonnet 4.6 ae9acecbf3 fix: sudo the tarball/image staging into /var/lib/vz
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m20s
The post-build cp/mv into /var/lib/vz/template/cache (LXC) and
/var/lib/vz/import (VM) are Proxmox-owned root directories -- they need
sudo_pfx just like pct/qm/pvesh do. nix build writes to the nix store
as the SSH user, but staging into /var/lib/vz/ requires root.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-23 10:39:09 +10:00
beatzaplentyandClaude Sonnet 4.6 a3be05538b fix: support single-user (non-root) nix in configure-nix-cache-client.sh
The script was root-only and hard-coded /etc/nix/nix.conf and
/etc/ssh/ssh_known_hosts, making it always fail (non-fatally) when
called as a non-root SSH user from create-proxmox-resource.sh.

Add dual-mode detection based on EUID:
- root (multi-user/daemon): existing behavior unchanged -- writes
  /etc/nix/nix.conf, /etc/ssh/ssh_known_hosts, restarts nix-daemon
- non-root (single-user): writes ~/.config/nix/nix.conf and
  ~/.ssh/known_hosts, creates the config file if missing, skips the
  daemon restart (single-user has no daemon), defaults REMOTE_BUILDER_KEY
  to ~/.ssh/id_ed25519 instead of /root/.ssh/id_ed25519

create-proxmox-resource.sh already calls the script without sudo (as the
SSH user), so no change is needed there -- the script now handles both
cases on its own.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-23 10:37:57 +10:00
beatzaplenty 289163c712 Merge pull request 'Worktree harmonic jingling bee' (#46) from worktree-harmonic-jingling-bee into main
Check NixOS configurations / eval-hosts (push) Successful in 10m22s
Reviewed-on: #46
2026-07-23 00:29:56 +00:00
beatzaplentyandClaude Sonnet 4.6 2123e4ad69 fix: reinstall nix as SSH user, not root, on Proxmox nodes
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m27s
nix was installed as root on pve1 (the codex-setup.sh root path, meant
for container/Codex environments), making nix build require sudo there.
After cleaning up the root install and reinstalling as the SSH user
(wayne), nix is owned by that user and runs directly without sudo.

create-proxmox-resource.sh: drop sudo_pfx from nix build in both
remote scripts. The SSH user owns the store after reinstall; nix build
goes through the nix daemon-or-store directly. sudo stays on pct/qm/pvesh
(cluster IPC) and the disko image-writer script (writes to disk).

codex-setup.sh: add build-users-group = (empty) to the user nix.conf
written by the non-root install path. Guards against a stale
/etc/nix/nix.conf from a prior root install (which sets
build-users-group = nixbld) silently breaking single-user builds.

Manual cleanup required once on each Proxmox node that had root's nix:
  sudo rm -rf /nix /etc/nix
  sudo rm -f /etc/profile.d/nix.sh /etc/profile.d/nix-daemon.sh
  for i in $(seq 1 10); do sudo userdel nixbld$i 2>/dev/null||true; done
  sudo groupdel nixbld 2>/dev/null || true
After that, the next create-proxmox-resource.sh run auto-reinstalls
nix as the SSH user via codex-setup.sh.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-23 10:25:43 +10:00
beatzaplentyandClaude Sonnet 4.6 177950dd3d revert: restore sudo for nix build in remote scripts
nix on pve1 was installed as root (single-user), so wayne can't access
/nix/var/nix/db/big-lock without root -- nix build genuinely needs sudo
there. The previous fix to drop sudo_pfx was wrong.

The real fix is node config: add nix to wayne's NOPASSWD rules in
sudoers on pve1 (see below). pct/qm/pvesh already have NOPASSWD and
work fine in non-interactive SSH heredocs; nix was just missing from
that list.

On pve1 as root:
  echo 'wayne ALL=(root) NOPASSWD: ALL' | tee /etc/sudoers.d/wayne-nopasswd
  chmod 440 /etc/sudoers.d/wayne-nopasswd

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-23 10:18:56 +10:00
beatzaplenty cbf1239be4 Merge pull request 'fix: don't sudo nix build in remote scripts' (#45) from worktree-harmonic-jingling-bee into main
Check NixOS configurations / eval-hosts (push) Successful in 10m33s
Reviewed-on: #45
2026-07-23 00:13:51 +00:00
beatzaplentyandClaude Sonnet 4.6 8a282ee32e fix: don't sudo nix build in remote scripts
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m33s
nix build runs through the nix daemon and doesn't need root; the
tooling-check step already confirms the SSH user can run nix directly
(ensure_nix_profile + command -v nix). sudo without a TTY blocks
non-interactive SSH heredoc sessions with "a terminal is required".

Keep sudo on pct/qm/pvesh (cluster IPC) and the disko image-writer
script (writes to block devices) -- those actually require root.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-23 10:10:29 +10:00
beatzaplentyandClaude Sonnet 4.6 25079a7f0a fix: use sudo for nix build on non-root SSH user
Check NixOS configurations / eval-hosts (push) Successful in 10m18s
Single-user Nix installations are owned by root. When PROXMOX_SSH_USER
is not root, prefix the remote nix build command with sudo_prefix, same
as the Proxmox tool invocations. Passes sudo_prefix as an extra arg to
both the LXC tarball and VM disko image build heredocs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-23 09:49:14 +10:00
beatzaplentyandClaude Sonnet 4.6 4952e5224d fix: default PROXMOX_REMOTE_REPO_DIR to SSH user's home dir
Check NixOS configurations / eval-hosts (push) Successful in 10m20s
/root/nixos was only correct when PROXMOX_SSH_USER=root. Now that it
defaults to wayne, use /home/${PROXMOX_SSH_USER}/nixos so git clone
goes somewhere the SSH user can actually write to.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-23 09:44:14 +10:00
beatzaplenty 98409f4502 Merge pull request 'fix: prefix Proxmox commands with sudo for non-root SSH user' (#44) from worktree-reactive-gliding-map into main
Check NixOS configurations / eval-hosts (push) Successful in 10m20s
Reviewed-on: #44
2026-07-22 22:59:35 +00:00
beatzaplentyandClaude Sonnet 4.6 7779f3e137 fix: prefix Proxmox commands with sudo for non-root SSH user
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m18s
PROXMOX_SSH_USER was changed from root to wayne, but all remote pvesh/qm/pct
invocations assumed root. When run as a non-root user these commands fail with
ipcc_send_rec errors because they can't reach the pve-cluster IPC socket.

Adds a global sudo_prefix (empty when PROXMOX_SSH_USER=root, "sudo" otherwise)
and applies it to every remote Proxmox command in the script, including the
duplicate-host heredoc check, pvesh nextid, vmid existence checks, resource
destruction, and all create/start commands. Removes the now-redundant local
sudo_prefix definition that was previously only in the VM image build branch.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-23 08:47:00 +10:00
beatzaplenty 1462829aa6 fix syntax
Check NixOS configurations / eval-hosts (push) Failing after 11m48s
2026-07-23 08:38:36 +10:00
beatzaplenty 48ce2c4097 added sops key path and updated pve1 SSH user
Check NixOS configurations / eval-hosts (push) Failing after 9m45s
2026-07-23 08:37:44 +10:00
beatzaplenty bcae177d8e added jq
Check NixOS configurations / eval-hosts (push) Failing after 9m45s
2026-07-23 06:59:37 +10:00
beatzaplenty d35aca3138 added jq and direnv 2026-07-23 06:59:00 +10:00
beatzaplenty 147cb3803a Update hosts/nixos/home.nix
Check NixOS configurations / eval-hosts (push) Failing after 11m59s
2026-07-22 19:57:14 +00:00
beatzaplenty 98445565d6 Update hosts/nixos/home.nix
Check NixOS configurations / eval-hosts (push) Failing after 11m10s
2026-07-22 19:56:07 +00:00
beatzaplenty eef4b05254 Merge pull request 'Unmount everything under /mnt before zpool export, not just chroot dirs' (#43) from worktree-baremetal-esp-unmount into main
Check NixOS configurations / eval-hosts (push) Successful in 10m20s
Reviewed-on: #43
2026-07-22 04:30:05 +00:00
beatzaplenty 619324589a Unmount everything under /mnt before zpool export, not just the chroot dirs
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m28s
The previous fix (#42) only unmounted /mnt/{dev,proc,sys,run}, but disko
also mounts the ESP at /mnt/boot (modules/disko/baremetal.nix) -- another
nested mount blocking ZFS from unmounting its own root dataset at /mnt
the same way. Confirmed live: zpool export still failed with "cannot
unmount '/mnt': pool or dataset busy" after the chroot-only fix.

Replace the manual dev/proc/sys/run list with a single recursive
`umount -R /mnt`, which clears everything nested under /mnt -- current
and future mountpoints alike -- rather than needing to keep enumerating
whatever nixos-install/disko happen to leave mounted.
2026-07-22 04:23:42 +00:00
beatzaplenty 400af07154 Merge pull request 'Unmount leftover chroot bind mounts before zpool export' (#42) from worktree-baremetal-emergency-access into main
Check NixOS configurations / eval-hosts (push) Successful in 10m26s
Reviewed-on: #42
2026-07-22 04:10:08 +00:00
beatzaplenty 9bb626327f Unmount nixos-install's leftover chroot bind mounts before zpool export
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m20s
nixos-install bind-mounts /dev, /proc, /sys (and usually /run) into
/mnt to run the target's activation script in a chroot, and doesn't
unmount them again afterward. Left in place, those nested mounts made
ZFS refuse to unmount its own root dataset at /mnt: zpool export
failed with "cannot unmount '/mnt': pool or dataset busy", and because
of this script's set -e, that killed the script before it ever reached
reboot -- silently defeating the export-before-reboot fix from #40 on
every real run, which is why the ZFS-import stall kept recurring.
2026-07-22 04:07:32 +00:00
beatzaplenty 1f8bf8c852 Merge pull request 'Allow initrd emergency shell access on baremetal-gui' (#41) from worktree-baremetal-emergency-access into main
Check NixOS configurations / eval-hosts (push) Successful in 10m35s
Reviewed-on: #41
2026-07-22 03:48:48 +00:00
beatzaplenty 5d7a6327b7 Allow initrd emergency shell access on baremetal-gui
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m45s
The systemd-based initrd (default here, since this host has a ZFS
root) locks the root account by default, so sulogin refuses a shell
if something in the initrd fails and it drops to emergency mode --
confirmed live: it just loops re-entering the target instead of
prompting, making an initrd-level ZFS import failure impossible to
diagnose from the console. Only affects the pre-switch-root initrd
shell, not the installed system's own login.
2026-07-22 03:47:54 +00:00
beatzaplenty 0b9f124713 Merge pull request 'Export ZFS root pool before rebooting from the auto-installer' (#40) from worktree-fix-zfs-install-export into main
Check NixOS configurations / eval-hosts (push) Successful in 10m25s
Reviewed-on: #40
2026-07-22 03:22:32 +00:00
9 changed files with 129 additions and 77 deletions
+2
View File
@@ -19,11 +19,13 @@
nextcloud-client nextcloud-client
# vscode # vscode
chromium chromium
claude-code
]; ];
# Optional: set environment vars # Optional: set environment vars
sessionVariables = { sessionVariables = {
EDITOR = "vim"; EDITOR = "vim";
SOPS_AGE_KEY_FILE = "/home/nixos/Nextcloud/Filing Cabinet/keys/nixos-sops-age-key-txt";
}; };
file = { file = {
+1 -1
View File
@@ -18,7 +18,7 @@
]; ];
boot.loader.grub.useOSProber = true; boot.loader.grub.useOSProber = true;
programs.direnv.enable = true;
services = { services = {
xserver = { xserver = {
enable = true; enable = true;
+2
View File
@@ -31,6 +31,7 @@
btop btop
git git
gcr gcr
jq
]; ];
# Secrets shared by every host, decrypted at activation via each host's # Secrets shared by every host, decrypted at activation via each host's
@@ -76,6 +77,7 @@
openssh.authorizedKeys.keys = [ openssh.authorizedKeys.keys = [
vars.adminSshKey vars.adminSshKey
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICMJhrfFayLBG+gWtO6oAvgambw5nWWgztiTFEaaaVRH debian@surface" "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICMJhrfFayLBG+gWtO6oAvgambw5nWWgztiTFEaaaVRH debian@surface"
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGygkCljN6uKpdJbHTOQtn8ZnH+wKXDLAwrDFbLrE/65 nixos@nixos"
]; ];
}; };
+10
View File
@@ -25,4 +25,14 @@
enable = true; enable = true;
enable32Bit = true; enable32Bit = true;
}; };
# The systemd-based initrd (default here since this host has a ZFS root --
# see modules/disko/baremetal.nix) locks the root account by default, so
# sulogin refuses to hand over a shell if something in the initrd (e.g.
# the ZFS pool import) fails and it drops to emergency mode -- confirmed
# live: it just loops re-entering the target instead of prompting. This
# only affects the pre-switch-root initrd shell, not the installed
# system's own login, and is worth the tradeoff on a box already reachable
# at the physical console.
boot.initrd.systemd.emergencyAccess = true;
} }
+1
View File
@@ -68,6 +68,7 @@ cat > "$HOME/.config/nix/nix.conf" <<'EOF'
experimental-features = nix-command flakes experimental-features = nix-command flakes
accept-flake-config = false accept-flake-config = false
warn-dirty = false warn-dirty = false
build-users-group =
EOF EOF
echo "Nix version:" echo "Nix version:"
+2 -2
View File
@@ -22,7 +22,7 @@
: "${PVE1_HOST:=pve1.sweet.home}" : "${PVE1_HOST:=pve1.sweet.home}"
: "${PVE_TEST_HOST:=pve-test.sweet.home}" : "${PVE_TEST_HOST:=pve-test.sweet.home}"
: "${PROXMOX_HOST:=$PVE1_HOST}" : "${PROXMOX_HOST:=$PVE1_HOST}"
: "${PROXMOX_SSH_USER:=root}" : "${PROXMOX_SSH_USER:=wayne}"
# Where this flake repo lives on the Proxmox node itself. # Where this flake repo lives on the Proxmox node itself.
# scripts/proxmox/create-proxmox-resource.sh builds images directly on the node # scripts/proxmox/create-proxmox-resource.sh builds images directly on the node
@@ -30,7 +30,7 @@
# (from this checkout's own `origin` remote) the first time it doesn't # (from this checkout's own `origin` remote) the first time it doesn't
# find it, installing build tooling via scripts/codex-setup.sh, then # find it, installing build tooling via scripts/codex-setup.sh, then
# `git pull`s it before every subsequent build. # `git pull`s it before every subsequent build.
: "${PROXMOX_REMOTE_REPO_DIR:=/root/nixos}" : "${PROXMOX_REMOTE_REPO_DIR:=/home/${PROXMOX_SSH_USER}/nixos}"
# Storage pool names -- Proxmox's own stock-install defaults, but this # Storage pool names -- Proxmox's own stock-install defaults, but this
# varies a lot by setup (ZFS pool name, custom LVM-thin volume, etc.). # varies a lot by setup (ZFS pool name, custom LVM-thin volume, etc.).
+18
View File
@@ -179,6 +179,24 @@ nixos-install \
# makes boot stall waiting on the ZFS import. Exporting here (a no-op # makes boot stall waiting on the ZFS import. Exporting here (a no-op
# if the chosen host has no ZFS root, e.g. proxmox-*/linode-*) clears # if the chosen host has no ZFS root, e.g. proxmox-*/linode-*) clears
# that in-use state so the next import, from any hostid, succeeds. # that in-use state so the next import, from any hostid, succeeds.
#
# Anything still mounted under /mnt -- nixos-install's own leftover
# chroot bind mounts for running the target's activation script
# (/mnt/dev, /mnt/proc, /mnt/sys, /mnt/run), and disko's own /mnt/boot
# ESP mount (modules/disko/baremetal.nix) -- blocks ZFS from unmounting
# its root dataset at /mnt, the same way any nested mount blocks
# unmounting its parent. Confirmed live: zpool export failed with
# "cannot unmount '/mnt': pool or dataset busy" even after handling the
# chroot mounts alone, because /mnt/boot was still mounted too. Because
# of this script's `set -e`, that killed the script before it ever
# reached reboot, silently defeating the whole point of exporting first.
# Unmounting everything under /mnt up front (recursively, so nested
# mounts like /mnt/dev/pts come along for free) sidesteps needing to
# enumerate every mount disko/nixos-install might leave behind.
if mountpoint -q /mnt; then
umount -R /mnt
fi
if [[ -n "$(zpool list -H -o name 2>/dev/null)" ]]; then if [[ -n "$(zpool list -H -o name 2>/dev/null)" ]]; then
echo "Exporting ZFS pool(s) before reboot..." echo "Exporting ZFS pool(s) before reboot..."
zpool export -a zpool export -a
+43 -26
View File
@@ -6,27 +6,31 @@
# #
# This is the non-NixOS equivalent of modules/nix-cache/client.nix + # This is the non-NixOS equivalent of modules/nix-cache/client.nix +
# modules/nix-cache/remote-builder-client.nix -- those two only apply to # modules/nix-cache/remote-builder-client.nix -- those two only apply to
# hosts built from this flake. A plain Debian box with Nix installed # hosts built from this flake. A plain Debian box with Nix installed has no
# (single- or multi-user install, nix-daemon running) has no NixOS module # NixOS module system to pick that config up, so this edits nix.conf by hand.
# system to pick that config up, so this edits /etc/nix/nix.conf by hand #
# instead. Run this ON the target Debian machine, as root. # Two modes depending on who runs it:
#
# root (multi-user / daemon install):
# Writes /etc/nix/nix.conf, /etc/ssh/ssh_known_hosts, restarts nix-daemon.
# Requires /etc/nix/nix.conf to already exist (i.e. nix-daemon is set up).
# Run as: sudo ./configure-nix-cache-client.sh [options]
#
# non-root (single-user install):
# Writes ~/.config/nix/nix.conf, ~/.ssh/known_hosts. No daemon to restart.
# Run as: ./configure-nix-cache-client.sh [options]
# #
# The values below mirror variables.nix / modules/nix-cache/client.nix in # The values below mirror variables.nix / modules/nix-cache/client.nix in
# this repo -- update both if nix-cache is ever rebuilt with a new host # this repo -- update both if nix-cache is ever rebuilt with a new host
# key or the cache signing key is rotated (see docs/nix-cache.md). # key or the cache signing key is rotated (see docs/nix-cache.md).
# #
# REMOTE_BUILDER_KEY defaults to this machine's own default root SSH # REMOTE_BUILDER_KEY defaults to the running user's default SSH identity
# identity (matches modules/nix-cache/remote-builder-client.nix's # (root: /root/.ssh/id_ed25519, other user: ~/.ssh/id_ed25519). That key
# convention for real NixOS clients: authenticate as nixremote with the # must be listed in vars.remoteBuilderAuthorizedKeys in this repo and
# host's own default key, added individually to # nix-cache rebuilt before remote building works.
# vars.remoteBuilderAuthorizedKeys, rather than a separately-named or
# shared keypair) -- generate one with
# `ssh-keygen -t ed25519 -N '' -f /root/.ssh/id_ed25519` if this machine
# doesn't have one yet, then add its .pub to vars.remoteBuilderAuthorizedKeys
# and rebuild nix-cache.
# #
# Usage: # Usage:
# sudo ./configure-nix-cache-client.sh [--dry-run] [--no-remote-builder] [--no-restart] # ./configure-nix-cache-client.sh [--dry-run] [--no-remote-builder] [--no-restart]
# #
# Env overrides (defaults match variables.nix): # Env overrides (defaults match variables.nix):
# NIX_CACHE_HOST, NIX_CACHE_HOST_KEY, REMOTE_BUILDER_USER, REMOTE_BUILDER_KEY # NIX_CACHE_HOST, NIX_CACHE_HOST_KEY, REMOTE_BUILDER_USER, REMOTE_BUILDER_KEY
@@ -36,17 +40,28 @@ set -euo pipefail
: "${NIX_CACHE_HOST:=nix-cache}" : "${NIX_CACHE_HOST:=nix-cache}"
: "${NIX_CACHE_HOST_KEY:=ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPeWgMsdaiz4axT/deFc1+0B5bN+GX/NOeW9bbQ0c/IT lxc-nix-cache}" : "${NIX_CACHE_HOST_KEY:=ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPeWgMsdaiz4axT/deFc1+0B5bN+GX/NOeW9bbQ0c/IT lxc-nix-cache}"
: "${REMOTE_BUILDER_USER:=nixremote}" : "${REMOTE_BUILDER_USER:=nixremote}"
: "${REMOTE_BUILDER_KEY:=/root/.ssh/id_ed25519}"
CACHE_PUB_KEY="cache.local-1:usoWYanY3Kpq2+kDIS2nhWoLZiRxanmdysdzqCFBHW4=" CACHE_PUB_KEY="cache.local-1:usoWYanY3Kpq2+kDIS2nhWoLZiRxanmdysdzqCFBHW4="
FALLBACK_URL="https://cache.nixos.org/" FALLBACK_URL="https://cache.nixos.org/"
FALLBACK_PUB_KEY="cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=" FALLBACK_PUB_KEY="cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY="
NIX_CONF="/etc/nix/nix.conf"
KNOWN_HOSTS="/etc/ssh/ssh_known_hosts"
MARKER_BEGIN="# BEGIN nix-cache client config (configure-nix-cache-client.sh)" MARKER_BEGIN="# BEGIN nix-cache client config (configure-nix-cache-client.sh)"
MARKER_END="# END nix-cache client config" MARKER_END="# END nix-cache client config"
# Mode: root uses system-wide paths and restarts the daemon; non-root uses
# user-level paths and has no daemon to restart.
if [[ "$EUID" -eq 0 ]]; then
install_mode="multi"
NIX_CONF="/etc/nix/nix.conf"
KNOWN_HOSTS="/etc/ssh/ssh_known_hosts"
: "${REMOTE_BUILDER_KEY:=/root/.ssh/id_ed25519}"
else
install_mode="single"
NIX_CONF="${XDG_CONFIG_HOME:-$HOME/.config}/nix/nix.conf"
KNOWN_HOSTS="$HOME/.ssh/known_hosts"
: "${REMOTE_BUILDER_KEY:=$HOME/.ssh/id_ed25519}"
fi
dry_run=0 dry_run=0
with_remote_builder=1 with_remote_builder=1
restart_daemon=1 restart_daemon=1
@@ -57,7 +72,7 @@ for arg in "$@"; do
--no-remote-builder) with_remote_builder=0 ;; --no-remote-builder) with_remote_builder=0 ;;
--no-restart) restart_daemon=0 ;; --no-restart) restart_daemon=0 ;;
-h|--help) -h|--help)
sed -n '2,20p' "$0" sed -n '2,37p' "$0"
exit 0 exit 0
;; ;;
*) *)
@@ -67,21 +82,22 @@ for arg in "$@"; do
esac esac
done done
if [[ "$dry_run" -eq 0 && "$EUID" -ne 0 ]]; then
echo "ERROR: must run as root (writes $NIX_CONF and, unless --no-remote-builder, $KNOWN_HOSTS)." >&2
exit 1
fi
if ! command -v nix >/dev/null 2>&1; then if ! command -v nix >/dev/null 2>&1; then
echo "ERROR: no 'nix' binary on PATH -- install the Nix package manager first." >&2 echo "ERROR: no 'nix' binary on PATH -- install the Nix package manager first." >&2
exit 1 exit 1
fi fi
if [[ ! -f "$NIX_CONF" ]]; then if [[ "$install_mode" == "multi" && ! -f "$NIX_CONF" ]]; then
echo "ERROR: $NIX_CONF not found -- expected an existing multi-user Nix install." >&2 echo "ERROR: $NIX_CONF not found -- expected an existing multi-user Nix install." >&2
exit 1 exit 1
fi fi
# Single-user: create the config file if it doesn't exist yet.
if [[ "$install_mode" == "single" && "$dry_run" -eq 0 ]]; then
mkdir -p "$(dirname "$NIX_CONF")"
[[ -f "$NIX_CONF" ]] || touch "$NIX_CONF"
fi
builder_line="" builder_line=""
if [[ "$with_remote_builder" -eq 1 ]]; then if [[ "$with_remote_builder" -eq 1 ]]; then
if [[ -f "$REMOTE_BUILDER_KEY" ]]; then if [[ -f "$REMOTE_BUILDER_KEY" ]]; then
@@ -117,7 +133,7 @@ fi
block="${block} block="${block}
$MARKER_END" $MARKER_END"
echo "== nix.conf block to install ==" echo "== nix.conf block to install ($NIX_CONF) =="
echo "$block" echo "$block"
echo "================================" echo "================================"
@@ -159,7 +175,8 @@ if [[ "$with_remote_builder" -eq 1 ]]; then
fi fi
fi fi
if [[ "$dry_run" -eq 0 && "$restart_daemon" -eq 1 ]]; then # Only restart the daemon for multi-user installs -- single-user has no daemon.
if [[ "$dry_run" -eq 0 && "$restart_daemon" -eq 1 && "$install_mode" == "multi" ]]; then
if command -v systemctl >/dev/null 2>&1 && systemctl is-active --quiet nix-daemon 2>/dev/null; then if command -v systemctl >/dev/null 2>&1 && systemctl is-active --quiet nix-daemon 2>/dev/null; then
systemctl restart nix-daemon systemctl restart nix-daemon
echo "Restarted nix-daemon to pick up the new config." echo "Restarted nix-daemon to pick up the new config."
+50 -48
View File
@@ -193,6 +193,16 @@ done
ssh_target="${PROXMOX_SSH_USER}@${node}" ssh_target="${PROXMOX_SSH_USER}@${node}"
# Proxmox tools (pvesh, qm, pct) require root access to the cluster IPC
# socket. When SSH-ing as a non-root user with sudo, prefix every remote
# Proxmox command with sudo.
sudo_prefix=""
sudo_display=""
if [[ "$PROXMOX_SSH_USER" != "root" ]]; then
sudo_prefix="sudo"
sudo_display="sudo "
fi
remote() { remote() {
if [[ "$dry_run" -eq 1 ]]; then if [[ "$dry_run" -eq 1 ]]; then
echo "[dry-run] ssh ${ssh_target} -- $*" echo "[dry-run] ssh ${ssh_target} -- $*"
@@ -214,10 +224,10 @@ cmd_modify() {
echo "Looking up VMID ${vmid} on ${node}..." echo "Looking up VMID ${vmid} on ${node}..."
local kind current_cores current_memory disk_key local kind current_cores current_memory disk_key
if ssh "$ssh_target" "qm status ${vmid}" >/dev/null 2>&1; then if ssh "$ssh_target" "${sudo_prefix} qm status ${vmid}" >/dev/null 2>&1; then
kind="vm" kind="vm"
disk_key="scsi0" disk_key="scsi0"
elif ssh "$ssh_target" "pct status ${vmid}" >/dev/null 2>&1; then elif ssh "$ssh_target" "${sudo_prefix} pct status ${vmid}" >/dev/null 2>&1; then
kind="lxc" kind="lxc"
disk_key="rootfs" disk_key="rootfs"
else else
@@ -225,8 +235,8 @@ cmd_modify() {
exit 1 exit 1
fi fi
local config_cmd="qm config ${vmid}" local config_cmd="${sudo_prefix} qm config ${vmid}"
[[ "$kind" == "lxc" ]] && config_cmd="pct config ${vmid}" [[ "$kind" == "lxc" ]] && config_cmd="${sudo_prefix} pct config ${vmid}"
local current_config local current_config
current_config="$(ssh "$ssh_target" "$config_cmd")" current_config="$(ssh "$ssh_target" "$config_cmd")"
current_cores="$(echo "$current_config" | grep -oP '^cores:\s*\K\S+' || echo '?')" current_cores="$(echo "$current_config" | grep -oP '^cores:\s*\K\S+' || echo '?')"
@@ -250,9 +260,9 @@ cmd_modify() {
exit 1 exit 1
fi fi
local set_cmd="qm set" local set_cmd="${sudo_prefix} qm set"
local resize_cmd="qm resize" local resize_cmd="${sudo_prefix} qm resize"
[[ "$kind" == "lxc" ]] && set_cmd="pct set" && resize_cmd="pct resize" [[ "$kind" == "lxc" ]] && set_cmd="${sudo_prefix} pct set" && resize_cmd="${sudo_prefix} pct resize"
if [[ -n "$cores" || -n "$memory" ]]; then if [[ -n "$cores" || -n "$memory" ]]; then
local args="" local args=""
@@ -359,14 +369,15 @@ else
echo echo
echo "==> Checking ${node} for an existing VM/CT identified as '${host}'..." echo "==> Checking ${node} for an existing VM/CT identified as '${host}'..."
ssh_check_status=0 ssh_check_status=0
existing="$(ssh "$ssh_target" bash -s -- "$host" <<'REMOTE_SCRIPT' existing="$(ssh "$ssh_target" bash -s -- "$host" "$sudo_prefix" <<'REMOTE_SCRIPT'
target="$1" target="$1"
for id in $(qm list 2>/dev/null | awk 'NR>1{print $1}'); do sudo_pfx="$2"
n="$(qm config "$id" 2>/dev/null | grep -oP '^name:\s*\K\S+' || true)" for id in $($sudo_pfx qm list 2>/dev/null | awk 'NR>1{print $1}'); do
n="$($sudo_pfx qm config "$id" 2>/dev/null | grep -oP '^name:\s*\K\S+' || true)"
[[ "$n" == "$target" ]] && echo "vm ${id} ${n}" [[ "$n" == "$target" ]] && echo "vm ${id} ${n}"
done done
for id in $(pct list 2>/dev/null | awk 'NR>1{print $1}'); do for id in $($sudo_pfx pct list 2>/dev/null | awk 'NR>1{print $1}'); do
n="$(pct config "$id" 2>/dev/null | grep -oP '^hostname:\s*\K\S+' || true)" n="$($sudo_pfx pct config "$id" 2>/dev/null | grep -oP '^hostname:\s*\K\S+' || true)"
[[ "$n" == "$target" ]] && echo "lxc ${id} ${n}" [[ "$n" == "$target" ]] && echo "lxc ${id} ${n}"
done done
exit 0 exit 0
@@ -453,12 +464,12 @@ REMOTE_SCRIPT
if [[ "$kind" == "vm" ]]; then if [[ "$kind" == "vm" ]]; then
# qm destroy has no --force to stop-then-destroy in one call (pct's # qm destroy has no --force to stop-then-destroy in one call (pct's
# does) -- stop explicitly first if it's running. # does) -- stop explicitly first if it's running.
if ssh "$ssh_target" "qm status ${id}" 2>/dev/null | grep -q running; then if ssh "$ssh_target" "${sudo_prefix} qm status ${id}" 2>/dev/null | grep -q running; then
ssh "$ssh_target" "qm stop ${id}" ssh "$ssh_target" "${sudo_prefix} qm stop ${id}"
fi fi
ssh "$ssh_target" "qm destroy ${id} --purge 1" ssh "$ssh_target" "${sudo_prefix} qm destroy ${id} --purge 1"
else else
ssh "$ssh_target" "pct destroy ${id} --force 1 --purge 1" ssh "$ssh_target" "${sudo_prefix} pct destroy ${id} --force 1 --purge 1"
fi fi
done done
fi fi
@@ -487,7 +498,7 @@ if [[ -z "$vmid" ]]; then
vmid="<next-free-vmid>" vmid="<next-free-vmid>"
echo "[dry-run] would ask ${node} for the next free VMID (pvesh get /cluster/nextid)" echo "[dry-run] would ask ${node} for the next free VMID (pvesh get /cluster/nextid)"
else else
vmid="$(ssh "$ssh_target" "pvesh get /cluster/nextid" | tr -d '[:space:]')" vmid="$(ssh "$ssh_target" "${sudo_prefix} pvesh get /cluster/nextid" | tr -d '[:space:]')"
echo "Auto-assigned VMID: ${vmid}" echo "Auto-assigned VMID: ${vmid}"
fi fi
else else
@@ -501,8 +512,8 @@ if [[ "$dry_run" -eq 0 ]]; then
# both. Any success here means something is already using this ID -- # both. Any success here means something is already using this ID --
# refuse to go anywhere near it. (Reconfiguring an existing resource is # refuse to go anywhere near it. (Reconfiguring an existing resource is
# --modify's job, not this one's.) # --modify's job, not this one's.)
if ssh "$ssh_target" "qm status ${vmid}" >/dev/null 2>&1 \ if ssh "$ssh_target" "${sudo_prefix} qm status ${vmid}" >/dev/null 2>&1 \
|| ssh "$ssh_target" "pct status ${vmid}" >/dev/null 2>&1; then || ssh "$ssh_target" "${sudo_prefix} pct status ${vmid}" >/dev/null 2>&1; then
echo "ERROR: VMID ${vmid} already exists on ${node}. Refusing to touch an" >&2 echo "ERROR: VMID ${vmid} already exists on ${node}. Refusing to touch an" >&2
echo "existing resource here -- use --modify to reconfigure it, pick a" >&2 echo "existing resource here -- use --modify to reconfigure it, pick a" >&2
echo "different --vmid, or omit it to auto-assign." >&2 echo "different --vmid, or omit it to auto-assign." >&2
@@ -672,11 +683,11 @@ if [[ "$image_already_remote" -eq 0 && -z "$local_image" ]]; then
# hands the result to the remote shell to re-split, which would # hands the result to the remote shell to re-split, which would
# otherwise scatter NIX_EXTRA_OPTS (itself several space-separated, # otherwise scatter NIX_EXTRA_OPTS (itself several space-separated,
# %q-quoted tokens) across the wrong positional parameters below. # %q-quoted tokens) across the wrong positional parameters below.
printf -v remote_cmd 'bash -s -- %q %q %q %q %q' \ printf -v remote_cmd 'bash -s -- %q %q %q %q %q %q' \
"$remote_repo_dir" "$flake_target" "$remote_dir" "$remote_filename" "$NIX_EXTRA_OPTS" "$remote_repo_dir" "$flake_target" "$remote_dir" "$remote_filename" "$NIX_EXTRA_OPTS" "$sudo_prefix"
ssh "$ssh_target" "$remote_cmd" <<'REMOTE_SCRIPT' ssh "$ssh_target" "$remote_cmd" <<'REMOTE_SCRIPT'
set -euo pipefail set -euo pipefail
repo_dir="$1"; target="$2"; dest_dir="$3"; dest_name="$4"; nix_extra_opts_str="$5" repo_dir="$1"; target="$2"; dest_dir="$3"; dest_name="$4"; nix_extra_opts_str="$5"; sudo_pfx="$6"
declare -a NIX_OPTS=() declare -a NIX_OPTS=()
[[ -n "$nix_extra_opts_str" ]] && eval "NIX_OPTS=(${nix_extra_opts_str})" [[ -n "$nix_extra_opts_str" ]] && eval "NIX_OPTS=(${nix_extra_opts_str})"
cd "$repo_dir" cd "$repo_dir"
@@ -694,23 +705,14 @@ if [[ -z "$built" ]]; then
echo "ERROR: no tarball found under result-${target}/tarball after build." >&2 echo "ERROR: no tarball found under result-${target}/tarball after build." >&2
exit 1 exit 1
fi fi
mkdir -p "$dest_dir" $sudo_pfx mkdir -p "$dest_dir"
cp "$built" "${dest_dir}/${dest_name}" $sudo_pfx cp "$built" "${dest_dir}/${dest_name}"
echo "Built and staged: ${dest_dir}/${dest_name}" echo "Built and staged: ${dest_dir}/${dest_name}"
REMOTE_SCRIPT REMOTE_SCRIPT
local_image="$remote_path" local_image="$remote_path"
echo "Built on ${node}: ${remote_path}" echo "Built on ${node}: ${remote_path}"
fi fi
else else
# PROXMOX_SSH_USER defaults to root (env.sh), which needs no sudo and
# can't assume it's even installed on a minimal node -- only shell out
# through sudo when actually running as a non-root SSH user.
sudo_prefix="sudo"
sudo_display="sudo "
if [[ "$PROXMOX_SSH_USER" == "root" ]]; then
sudo_prefix=""
sudo_display=""
fi
if [[ "$dry_run" -eq 1 ]]; then if [[ "$dry_run" -eq 1 ]]; then
echo "[dry-run] would build on ${node}: nix build --no-use-registries --no-accept-flake-config${nix_opts_display} \\" echo "[dry-run] would build on ${node}: nix build --no-use-registries --no-accept-flake-config${nix_opts_display} \\"
echo "[dry-run] .#nixosConfigurations.${flake_target}.config.system.build.diskoImagesScript" echo "[dry-run] .#nixosConfigurations.${flake_target}.config.system.build.diskoImagesScript"
@@ -728,7 +730,7 @@ REMOTE_SCRIPT
"$remote_repo_dir" "$flake_target" "$remote_dir" "$remote_filename" "$NIX_EXTRA_OPTS" "$sudo_prefix" "$remote_repo_dir" "$flake_target" "$remote_dir" "$remote_filename" "$NIX_EXTRA_OPTS" "$sudo_prefix"
ssh "$ssh_target" "$remote_cmd" <<'REMOTE_SCRIPT' ssh "$ssh_target" "$remote_cmd" <<'REMOTE_SCRIPT'
set -euo pipefail set -euo pipefail
repo_dir="$1"; target="$2"; dest_dir="$3"; dest_name="$4"; nix_extra_opts_str="$5"; sudo_prefix="$6" repo_dir="$1"; target="$2"; dest_dir="$3"; dest_name="$4"; nix_extra_opts_str="$5"; sudo_pfx="$6"
declare -a NIX_OPTS=() declare -a NIX_OPTS=()
[[ -n "$nix_extra_opts_str" ]] && eval "NIX_OPTS=(${nix_extra_opts_str})" [[ -n "$nix_extra_opts_str" ]] && eval "NIX_OPTS=(${nix_extra_opts_str})"
cd "$repo_dir" cd "$repo_dir"
@@ -737,7 +739,7 @@ ensure_nix_profile
nix build --no-use-registries --no-accept-flake-config "${NIX_OPTS[@]}" \ nix build --no-use-registries --no-accept-flake-config "${NIX_OPTS[@]}" \
".#nixosConfigurations.${target}.config.system.build.diskoImagesScript" \ ".#nixosConfigurations.${target}.config.system.build.diskoImagesScript" \
--out-link "result-${target}" --out-link "result-${target}"
$sudo_prefix "./result-${target}" \ $sudo_pfx "./result-${target}" \
--pre-format-files "host-keys/${target}_ssh_host_ed25519_key" /etc/ssh/ssh_host_ed25519_key \ --pre-format-files "host-keys/${target}_ssh_host_ed25519_key" /etc/ssh/ssh_host_ed25519_key \
--pre-format-files "host-keys/${target}_ssh_host_ed25519_key.pub" /etc/ssh/ssh_host_ed25519_key.pub \ --pre-format-files "host-keys/${target}_ssh_host_ed25519_key.pub" /etc/ssh/ssh_host_ed25519_key.pub \
--build-memory 2048 --build-memory 2048
@@ -746,8 +748,8 @@ if [[ -z "$built" ]]; then
echo "ERROR: no .raw image found in ${repo_dir} after build." >&2 echo "ERROR: no .raw image found in ${repo_dir} after build." >&2
exit 1 exit 1
fi fi
mkdir -p "$dest_dir" $sudo_pfx mkdir -p "$dest_dir"
mv "$built" "${dest_dir}/${dest_name}" $sudo_pfx mv "$built" "${dest_dir}/${dest_name}"
echo "Built and staged: ${dest_dir}/${dest_name}" echo "Built and staged: ${dest_dir}/${dest_name}"
REMOTE_SCRIPT REMOTE_SCRIPT
local_image="$remote_path" local_image="$remote_path"
@@ -812,9 +814,9 @@ if [[ "$type" == "lxc" ]]; then
# hands the whole string to `ssh` as a single command for the *remote* # hands the whole string to `ssh` as a single command for the *remote*
# shell to parse -- unquoted, that `;` would be read as a remote # shell to parse -- unquoted, that `;` would be read as a remote
# command separator and silently truncate this into two commands. # command separator and silently truncate this into two commands.
create_cmd="pct create ${vmid} ${iso_storage}:vztmpl/${remote_filename} --unprivileged ${unprivileged_flag} --features '${PROXMOX_DEFAULT_LXC_FEATURES}' --rootfs ${storage}:${local_disk_size} --hostname ${name} --cores ${cores} --memory ${memory} --swap ${local_swap} --net0 name=eth0,bridge=${bridge},ip=dhcp" create_cmd="${sudo_prefix} pct create ${vmid} ${iso_storage}:vztmpl/${remote_filename} --unprivileged ${unprivileged_flag} --features '${PROXMOX_DEFAULT_LXC_FEATURES}' --rootfs ${storage}:${local_disk_size} --hostname ${name} --cores ${cores} --memory ${memory} --swap ${local_swap} --net0 name=eth0,bridge=${bridge},ip=dhcp"
remote "$create_cmd" remote "$create_cmd"
remote "pct start ${vmid}" remote "${sudo_prefix} pct start ${vmid}"
else else
echo "==> Creating VM ${vmid} (${name})..." echo "==> Creating VM ${vmid} (${name})..."
# pre-enrolled-keys=0 disables OVMF's Secure Boot key pre-enrollment -- # pre-enrolled-keys=0 disables OVMF's Secure Boot key pre-enrollment --
@@ -825,29 +827,29 @@ else
# without this flag Proxmox never creates the channel it listens on, so # without this flag Proxmox never creates the channel it listens on, so
# `qm guest exec`/`qm agent` and the UI's IP-address display silently # `qm guest exec`/`qm agent` and the UI's IP-address display silently
# never work for any VM this script creates. # never work for any VM this script creates.
remote "qm create ${vmid} --name ${name} --memory ${memory} --cores ${cores} \ remote "${sudo_prefix} qm create ${vmid} --name ${name} --memory ${memory} --cores ${cores} \
--net0 virtio,bridge=${bridge} --bios ovmf --machine q35 --scsihw virtio-scsi-pci \ --net0 virtio,bridge=${bridge} --bios ovmf --machine q35 --scsihw virtio-scsi-pci \
--efidisk0 ${storage}:1,efitype=4m,pre-enrolled-keys=0 --agent enabled=1" --efidisk0 ${storage}:1,efitype=4m,pre-enrolled-keys=0 --agent enabled=1"
if [[ "$dry_run" -eq 1 ]]; then if [[ "$dry_run" -eq 1 ]]; then
echo "[dry-run] ssh ${ssh_target} -- qm importdisk ${vmid} ${remote_path} ${storage}" echo "[dry-run] ssh ${ssh_target} -- ${sudo_display}qm importdisk ${vmid} ${remote_path} ${storage}"
echo "[dry-run] (would parse the resulting disk identifier from that output)" echo "[dry-run] (would parse the resulting disk identifier from that output)"
echo "[dry-run] ssh ${ssh_target} -- qm set ${vmid} --scsi0 ${storage}:<parsed-disk-id>" echo "[dry-run] ssh ${ssh_target} -- ${sudo_display}qm set ${vmid} --scsi0 ${storage}:<parsed-disk-id>"
else else
importdisk_output="$(ssh "$ssh_target" "qm importdisk ${vmid} ${remote_path} ${storage}")" importdisk_output="$(ssh "$ssh_target" "${sudo_prefix} qm importdisk ${vmid} ${remote_path} ${storage}")"
echo "$importdisk_output" echo "$importdisk_output"
disk_id="$(echo "$importdisk_output" | grep -oP "(?<=Successfully imported disk as ')[^']+" | sed 's/^unused[0-9]*://')" disk_id="$(echo "$importdisk_output" | grep -oP "(?<=Successfully imported disk as ')[^']+" | sed 's/^unused[0-9]*://')"
if [[ -z "$disk_id" ]]; then if [[ -z "$disk_id" ]]; then
echo "ERROR: couldn't parse the imported disk identifier from qm importdisk's output above." >&2 echo "ERROR: couldn't parse the imported disk identifier from qm importdisk's output above." >&2
echo "The VM shell (${vmid}) and imported disk both exist -- finish attaching it by hand:" >&2 echo "The VM shell (${vmid}) and imported disk both exist -- finish attaching it by hand:" >&2
echo " ssh ${ssh_target} -- qm set ${vmid} --scsi0 ${storage}:<disk-id-from-output-above>" >&2 echo " ssh ${ssh_target} -- ${sudo_display}qm set ${vmid} --scsi0 ${storage}:<disk-id-from-output-above>" >&2
echo " ssh ${ssh_target} -- qm set ${vmid} --boot order=scsi0" >&2 echo " ssh ${ssh_target} -- ${sudo_display}qm set ${vmid} --boot order=scsi0" >&2
exit 1 exit 1
fi fi
remote "qm set ${vmid} --scsi0 ${disk_id}" remote "${sudo_prefix} qm set ${vmid} --scsi0 ${disk_id}"
fi fi
remote "qm set ${vmid} --boot order=scsi0" remote "${sudo_prefix} qm set ${vmid} --boot order=scsi0"
remote "qm start ${vmid}" remote "${sudo_prefix} qm start ${vmid}"
fi fi
echo echo