Compare commits

..
Author SHA1 Message Date
beatzaplentyandClaude Sonnet 4.6 ae9acecbf3 fix: sudo the tarball/image staging into /var/lib/vz
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m20s
The post-build cp/mv into /var/lib/vz/template/cache (LXC) and
/var/lib/vz/import (VM) are Proxmox-owned root directories -- they need
sudo_pfx just like pct/qm/pvesh do. nix build writes to the nix store
as the SSH user, but staging into /var/lib/vz/ requires root.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-23 10:39:09 +10:00
beatzaplentyandClaude Sonnet 4.6 a3be05538b fix: support single-user (non-root) nix in configure-nix-cache-client.sh
The script was root-only and hard-coded /etc/nix/nix.conf and
/etc/ssh/ssh_known_hosts, making it always fail (non-fatally) when
called as a non-root SSH user from create-proxmox-resource.sh.

Add dual-mode detection based on EUID:
- root (multi-user/daemon): existing behavior unchanged -- writes
  /etc/nix/nix.conf, /etc/ssh/ssh_known_hosts, restarts nix-daemon
- non-root (single-user): writes ~/.config/nix/nix.conf and
  ~/.ssh/known_hosts, creates the config file if missing, skips the
  daemon restart (single-user has no daemon), defaults REMOTE_BUILDER_KEY
  to ~/.ssh/id_ed25519 instead of /root/.ssh/id_ed25519

create-proxmox-resource.sh already calls the script without sudo (as the
SSH user), so no change is needed there -- the script now handles both
cases on its own.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-23 10:37:57 +10:00
beatzaplentyandClaude Sonnet 4.6 2123e4ad69 fix: reinstall nix as SSH user, not root, on Proxmox nodes
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m27s
nix was installed as root on pve1 (the codex-setup.sh root path, meant
for container/Codex environments), making nix build require sudo there.
After cleaning up the root install and reinstalling as the SSH user
(wayne), nix is owned by that user and runs directly without sudo.

create-proxmox-resource.sh: drop sudo_pfx from nix build in both
remote scripts. The SSH user owns the store after reinstall; nix build
goes through the nix daemon-or-store directly. sudo stays on pct/qm/pvesh
(cluster IPC) and the disko image-writer script (writes to disk).

codex-setup.sh: add build-users-group = (empty) to the user nix.conf
written by the non-root install path. Guards against a stale
/etc/nix/nix.conf from a prior root install (which sets
build-users-group = nixbld) silently breaking single-user builds.

Manual cleanup required once on each Proxmox node that had root's nix:
  sudo rm -rf /nix /etc/nix
  sudo rm -f /etc/profile.d/nix.sh /etc/profile.d/nix-daemon.sh
  for i in $(seq 1 10); do sudo userdel nixbld$i 2>/dev/null||true; done
  sudo groupdel nixbld 2>/dev/null || true
After that, the next create-proxmox-resource.sh run auto-reinstalls
nix as the SSH user via codex-setup.sh.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-23 10:25:43 +10:00
beatzaplentyandClaude Sonnet 4.6 177950dd3d revert: restore sudo for nix build in remote scripts
nix on pve1 was installed as root (single-user), so wayne can't access
/nix/var/nix/db/big-lock without root -- nix build genuinely needs sudo
there. The previous fix to drop sudo_pfx was wrong.

The real fix is node config: add nix to wayne's NOPASSWD rules in
sudoers on pve1 (see below). pct/qm/pvesh already have NOPASSWD and
work fine in non-interactive SSH heredocs; nix was just missing from
that list.

On pve1 as root:
  echo 'wayne ALL=(root) NOPASSWD: ALL' | tee /etc/sudoers.d/wayne-nopasswd
  chmod 440 /etc/sudoers.d/wayne-nopasswd

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-23 10:18:56 +10:00
beatzaplentyandClaude Sonnet 4.6 8a282ee32e fix: don't sudo nix build in remote scripts
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m33s
nix build runs through the nix daemon and doesn't need root; the
tooling-check step already confirms the SSH user can run nix directly
(ensure_nix_profile + command -v nix). sudo without a TTY blocks
non-interactive SSH heredoc sessions with "a terminal is required".

Keep sudo on pct/qm/pvesh (cluster IPC) and the disko image-writer
script (writes to block devices) -- those actually require root.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-23 10:10:29 +10:00
beatzaplentyandClaude Sonnet 4.6 25079a7f0a fix: use sudo for nix build on non-root SSH user
Check NixOS configurations / eval-hosts (push) Successful in 10m18s
Single-user Nix installations are owned by root. When PROXMOX_SSH_USER
is not root, prefix the remote nix build command with sudo_prefix, same
as the Proxmox tool invocations. Passes sudo_prefix as an extra arg to
both the LXC tarball and VM disko image build heredocs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-23 09:49:14 +10:00
beatzaplentyandClaude Sonnet 4.6 4952e5224d fix: default PROXMOX_REMOTE_REPO_DIR to SSH user's home dir
Check NixOS configurations / eval-hosts (push) Successful in 10m20s
/root/nixos was only correct when PROXMOX_SSH_USER=root. Now that it
defaults to wayne, use /home/${PROXMOX_SSH_USER}/nixos so git clone
goes somewhere the SSH user can actually write to.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-23 09:44:14 +10:00
beatzaplenty 98409f4502 Merge pull request 'fix: prefix Proxmox commands with sudo for non-root SSH user' (#44) from worktree-reactive-gliding-map into main
Check NixOS configurations / eval-hosts (push) Successful in 10m20s
Reviewed-on: #44
2026-07-22 22:59:35 +00:00
beatzaplentyandClaude Sonnet 4.6 7779f3e137 fix: prefix Proxmox commands with sudo for non-root SSH user
Check NixOS configurations / eval-hosts (pull_request) Successful in 10m18s
PROXMOX_SSH_USER was changed from root to wayne, but all remote pvesh/qm/pct
invocations assumed root. When run as a non-root user these commands fail with
ipcc_send_rec errors because they can't reach the pve-cluster IPC socket.

Adds a global sudo_prefix (empty when PROXMOX_SSH_USER=root, "sudo" otherwise)
and applies it to every remote Proxmox command in the script, including the
duplicate-host heredoc check, pvesh nextid, vmid existence checks, resource
destruction, and all create/start commands. Removes the now-redundant local
sudo_prefix definition that was previously only in the VM image build branch.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-23 08:47:00 +10:00
beatzaplenty 1462829aa6 fix syntax
Check NixOS configurations / eval-hosts (push) Failing after 11m48s
2026-07-23 08:38:36 +10:00
beatzaplenty 48ce2c4097 added sops key path and updated pve1 SSH user
Check NixOS configurations / eval-hosts (push) Failing after 9m45s
2026-07-23 08:37:44 +10:00
beatzaplenty bcae177d8e added jq
Check NixOS configurations / eval-hosts (push) Failing after 9m45s
2026-07-23 06:59:37 +10:00
beatzaplenty d35aca3138 added jq and direnv 2026-07-23 06:59:00 +10:00
beatzaplenty 147cb3803a Update hosts/nixos/home.nix
Check NixOS configurations / eval-hosts (push) Failing after 11m59s
2026-07-22 19:57:14 +00:00
beatzaplenty 98445565d6 Update hosts/nixos/home.nix
Check NixOS configurations / eval-hosts (push) Failing after 11m10s
2026-07-22 19:56:07 +00:00
beatzaplenty eef4b05254 Merge pull request 'Unmount everything under /mnt before zpool export, not just chroot dirs' (#43) from worktree-baremetal-esp-unmount into main
Check NixOS configurations / eval-hosts (push) Successful in 10m20s
Reviewed-on: #43
2026-07-22 04:30:05 +00:00
7 changed files with 100 additions and 77 deletions
+2
View File
@@ -19,11 +19,13 @@
nextcloud-client nextcloud-client
# vscode # vscode
chromium chromium
claude-code
]; ];
# Optional: set environment vars # Optional: set environment vars
sessionVariables = { sessionVariables = {
EDITOR = "vim"; EDITOR = "vim";
SOPS_AGE_KEY_FILE = "/home/nixos/Nextcloud/Filing Cabinet/keys/nixos-sops-age-key-txt";
}; };
file = { file = {
+1 -1
View File
@@ -18,7 +18,7 @@
]; ];
boot.loader.grub.useOSProber = true; boot.loader.grub.useOSProber = true;
programs.direnv.enable = true;
services = { services = {
xserver = { xserver = {
enable = true; enable = true;
+1
View File
@@ -31,6 +31,7 @@
btop btop
git git
gcr gcr
jq
]; ];
# Secrets shared by every host, decrypted at activation via each host's # Secrets shared by every host, decrypted at activation via each host's
+1
View File
@@ -68,6 +68,7 @@ cat > "$HOME/.config/nix/nix.conf" <<'EOF'
experimental-features = nix-command flakes experimental-features = nix-command flakes
accept-flake-config = false accept-flake-config = false
warn-dirty = false warn-dirty = false
build-users-group =
EOF EOF
echo "Nix version:" echo "Nix version:"
+2 -2
View File
@@ -22,7 +22,7 @@
: "${PVE1_HOST:=pve1.sweet.home}" : "${PVE1_HOST:=pve1.sweet.home}"
: "${PVE_TEST_HOST:=pve-test.sweet.home}" : "${PVE_TEST_HOST:=pve-test.sweet.home}"
: "${PROXMOX_HOST:=$PVE1_HOST}" : "${PROXMOX_HOST:=$PVE1_HOST}"
: "${PROXMOX_SSH_USER:=root}" : "${PROXMOX_SSH_USER:=wayne}"
# Where this flake repo lives on the Proxmox node itself. # Where this flake repo lives on the Proxmox node itself.
# scripts/proxmox/create-proxmox-resource.sh builds images directly on the node # scripts/proxmox/create-proxmox-resource.sh builds images directly on the node
@@ -30,7 +30,7 @@
# (from this checkout's own `origin` remote) the first time it doesn't # (from this checkout's own `origin` remote) the first time it doesn't
# find it, installing build tooling via scripts/codex-setup.sh, then # find it, installing build tooling via scripts/codex-setup.sh, then
# `git pull`s it before every subsequent build. # `git pull`s it before every subsequent build.
: "${PROXMOX_REMOTE_REPO_DIR:=/root/nixos}" : "${PROXMOX_REMOTE_REPO_DIR:=/home/${PROXMOX_SSH_USER}/nixos}"
# Storage pool names -- Proxmox's own stock-install defaults, but this # Storage pool names -- Proxmox's own stock-install defaults, but this
# varies a lot by setup (ZFS pool name, custom LVM-thin volume, etc.). # varies a lot by setup (ZFS pool name, custom LVM-thin volume, etc.).
+43 -26
View File
@@ -6,27 +6,31 @@
# #
# This is the non-NixOS equivalent of modules/nix-cache/client.nix + # This is the non-NixOS equivalent of modules/nix-cache/client.nix +
# modules/nix-cache/remote-builder-client.nix -- those two only apply to # modules/nix-cache/remote-builder-client.nix -- those two only apply to
# hosts built from this flake. A plain Debian box with Nix installed # hosts built from this flake. A plain Debian box with Nix installed has no
# (single- or multi-user install, nix-daemon running) has no NixOS module # NixOS module system to pick that config up, so this edits nix.conf by hand.
# system to pick that config up, so this edits /etc/nix/nix.conf by hand #
# instead. Run this ON the target Debian machine, as root. # Two modes depending on who runs it:
#
# root (multi-user / daemon install):
# Writes /etc/nix/nix.conf, /etc/ssh/ssh_known_hosts, restarts nix-daemon.
# Requires /etc/nix/nix.conf to already exist (i.e. nix-daemon is set up).
# Run as: sudo ./configure-nix-cache-client.sh [options]
#
# non-root (single-user install):
# Writes ~/.config/nix/nix.conf, ~/.ssh/known_hosts. No daemon to restart.
# Run as: ./configure-nix-cache-client.sh [options]
# #
# The values below mirror variables.nix / modules/nix-cache/client.nix in # The values below mirror variables.nix / modules/nix-cache/client.nix in
# this repo -- update both if nix-cache is ever rebuilt with a new host # this repo -- update both if nix-cache is ever rebuilt with a new host
# key or the cache signing key is rotated (see docs/nix-cache.md). # key or the cache signing key is rotated (see docs/nix-cache.md).
# #
# REMOTE_BUILDER_KEY defaults to this machine's own default root SSH # REMOTE_BUILDER_KEY defaults to the running user's default SSH identity
# identity (matches modules/nix-cache/remote-builder-client.nix's # (root: /root/.ssh/id_ed25519, other user: ~/.ssh/id_ed25519). That key
# convention for real NixOS clients: authenticate as nixremote with the # must be listed in vars.remoteBuilderAuthorizedKeys in this repo and
# host's own default key, added individually to # nix-cache rebuilt before remote building works.
# vars.remoteBuilderAuthorizedKeys, rather than a separately-named or
# shared keypair) -- generate one with
# `ssh-keygen -t ed25519 -N '' -f /root/.ssh/id_ed25519` if this machine
# doesn't have one yet, then add its .pub to vars.remoteBuilderAuthorizedKeys
# and rebuild nix-cache.
# #
# Usage: # Usage:
# sudo ./configure-nix-cache-client.sh [--dry-run] [--no-remote-builder] [--no-restart] # ./configure-nix-cache-client.sh [--dry-run] [--no-remote-builder] [--no-restart]
# #
# Env overrides (defaults match variables.nix): # Env overrides (defaults match variables.nix):
# NIX_CACHE_HOST, NIX_CACHE_HOST_KEY, REMOTE_BUILDER_USER, REMOTE_BUILDER_KEY # NIX_CACHE_HOST, NIX_CACHE_HOST_KEY, REMOTE_BUILDER_USER, REMOTE_BUILDER_KEY
@@ -36,17 +40,28 @@ set -euo pipefail
: "${NIX_CACHE_HOST:=nix-cache}" : "${NIX_CACHE_HOST:=nix-cache}"
: "${NIX_CACHE_HOST_KEY:=ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPeWgMsdaiz4axT/deFc1+0B5bN+GX/NOeW9bbQ0c/IT lxc-nix-cache}" : "${NIX_CACHE_HOST_KEY:=ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPeWgMsdaiz4axT/deFc1+0B5bN+GX/NOeW9bbQ0c/IT lxc-nix-cache}"
: "${REMOTE_BUILDER_USER:=nixremote}" : "${REMOTE_BUILDER_USER:=nixremote}"
: "${REMOTE_BUILDER_KEY:=/root/.ssh/id_ed25519}"
CACHE_PUB_KEY="cache.local-1:usoWYanY3Kpq2+kDIS2nhWoLZiRxanmdysdzqCFBHW4=" CACHE_PUB_KEY="cache.local-1:usoWYanY3Kpq2+kDIS2nhWoLZiRxanmdysdzqCFBHW4="
FALLBACK_URL="https://cache.nixos.org/" FALLBACK_URL="https://cache.nixos.org/"
FALLBACK_PUB_KEY="cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=" FALLBACK_PUB_KEY="cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY="
NIX_CONF="/etc/nix/nix.conf"
KNOWN_HOSTS="/etc/ssh/ssh_known_hosts"
MARKER_BEGIN="# BEGIN nix-cache client config (configure-nix-cache-client.sh)" MARKER_BEGIN="# BEGIN nix-cache client config (configure-nix-cache-client.sh)"
MARKER_END="# END nix-cache client config" MARKER_END="# END nix-cache client config"
# Mode: root uses system-wide paths and restarts the daemon; non-root uses
# user-level paths and has no daemon to restart.
if [[ "$EUID" -eq 0 ]]; then
install_mode="multi"
NIX_CONF="/etc/nix/nix.conf"
KNOWN_HOSTS="/etc/ssh/ssh_known_hosts"
: "${REMOTE_BUILDER_KEY:=/root/.ssh/id_ed25519}"
else
install_mode="single"
NIX_CONF="${XDG_CONFIG_HOME:-$HOME/.config}/nix/nix.conf"
KNOWN_HOSTS="$HOME/.ssh/known_hosts"
: "${REMOTE_BUILDER_KEY:=$HOME/.ssh/id_ed25519}"
fi
dry_run=0 dry_run=0
with_remote_builder=1 with_remote_builder=1
restart_daemon=1 restart_daemon=1
@@ -57,7 +72,7 @@ for arg in "$@"; do
--no-remote-builder) with_remote_builder=0 ;; --no-remote-builder) with_remote_builder=0 ;;
--no-restart) restart_daemon=0 ;; --no-restart) restart_daemon=0 ;;
-h|--help) -h|--help)
sed -n '2,20p' "$0" sed -n '2,37p' "$0"
exit 0 exit 0
;; ;;
*) *)
@@ -67,21 +82,22 @@ for arg in "$@"; do
esac esac
done done
if [[ "$dry_run" -eq 0 && "$EUID" -ne 0 ]]; then
echo "ERROR: must run as root (writes $NIX_CONF and, unless --no-remote-builder, $KNOWN_HOSTS)." >&2
exit 1
fi
if ! command -v nix >/dev/null 2>&1; then if ! command -v nix >/dev/null 2>&1; then
echo "ERROR: no 'nix' binary on PATH -- install the Nix package manager first." >&2 echo "ERROR: no 'nix' binary on PATH -- install the Nix package manager first." >&2
exit 1 exit 1
fi fi
if [[ ! -f "$NIX_CONF" ]]; then if [[ "$install_mode" == "multi" && ! -f "$NIX_CONF" ]]; then
echo "ERROR: $NIX_CONF not found -- expected an existing multi-user Nix install." >&2 echo "ERROR: $NIX_CONF not found -- expected an existing multi-user Nix install." >&2
exit 1 exit 1
fi fi
# Single-user: create the config file if it doesn't exist yet.
if [[ "$install_mode" == "single" && "$dry_run" -eq 0 ]]; then
mkdir -p "$(dirname "$NIX_CONF")"
[[ -f "$NIX_CONF" ]] || touch "$NIX_CONF"
fi
builder_line="" builder_line=""
if [[ "$with_remote_builder" -eq 1 ]]; then if [[ "$with_remote_builder" -eq 1 ]]; then
if [[ -f "$REMOTE_BUILDER_KEY" ]]; then if [[ -f "$REMOTE_BUILDER_KEY" ]]; then
@@ -117,7 +133,7 @@ fi
block="${block} block="${block}
$MARKER_END" $MARKER_END"
echo "== nix.conf block to install ==" echo "== nix.conf block to install ($NIX_CONF) =="
echo "$block" echo "$block"
echo "================================" echo "================================"
@@ -159,7 +175,8 @@ if [[ "$with_remote_builder" -eq 1 ]]; then
fi fi
fi fi
if [[ "$dry_run" -eq 0 && "$restart_daemon" -eq 1 ]]; then # Only restart the daemon for multi-user installs -- single-user has no daemon.
if [[ "$dry_run" -eq 0 && "$restart_daemon" -eq 1 && "$install_mode" == "multi" ]]; then
if command -v systemctl >/dev/null 2>&1 && systemctl is-active --quiet nix-daemon 2>/dev/null; then if command -v systemctl >/dev/null 2>&1 && systemctl is-active --quiet nix-daemon 2>/dev/null; then
systemctl restart nix-daemon systemctl restart nix-daemon
echo "Restarted nix-daemon to pick up the new config." echo "Restarted nix-daemon to pick up the new config."
+50 -48
View File
@@ -193,6 +193,16 @@ done
ssh_target="${PROXMOX_SSH_USER}@${node}" ssh_target="${PROXMOX_SSH_USER}@${node}"
# Proxmox tools (pvesh, qm, pct) require root access to the cluster IPC
# socket. When SSH-ing as a non-root user with sudo, prefix every remote
# Proxmox command with sudo.
sudo_prefix=""
sudo_display=""
if [[ "$PROXMOX_SSH_USER" != "root" ]]; then
sudo_prefix="sudo"
sudo_display="sudo "
fi
remote() { remote() {
if [[ "$dry_run" -eq 1 ]]; then if [[ "$dry_run" -eq 1 ]]; then
echo "[dry-run] ssh ${ssh_target} -- $*" echo "[dry-run] ssh ${ssh_target} -- $*"
@@ -214,10 +224,10 @@ cmd_modify() {
echo "Looking up VMID ${vmid} on ${node}..." echo "Looking up VMID ${vmid} on ${node}..."
local kind current_cores current_memory disk_key local kind current_cores current_memory disk_key
if ssh "$ssh_target" "qm status ${vmid}" >/dev/null 2>&1; then if ssh "$ssh_target" "${sudo_prefix} qm status ${vmid}" >/dev/null 2>&1; then
kind="vm" kind="vm"
disk_key="scsi0" disk_key="scsi0"
elif ssh "$ssh_target" "pct status ${vmid}" >/dev/null 2>&1; then elif ssh "$ssh_target" "${sudo_prefix} pct status ${vmid}" >/dev/null 2>&1; then
kind="lxc" kind="lxc"
disk_key="rootfs" disk_key="rootfs"
else else
@@ -225,8 +235,8 @@ cmd_modify() {
exit 1 exit 1
fi fi
local config_cmd="qm config ${vmid}" local config_cmd="${sudo_prefix} qm config ${vmid}"
[[ "$kind" == "lxc" ]] && config_cmd="pct config ${vmid}" [[ "$kind" == "lxc" ]] && config_cmd="${sudo_prefix} pct config ${vmid}"
local current_config local current_config
current_config="$(ssh "$ssh_target" "$config_cmd")" current_config="$(ssh "$ssh_target" "$config_cmd")"
current_cores="$(echo "$current_config" | grep -oP '^cores:\s*\K\S+' || echo '?')" current_cores="$(echo "$current_config" | grep -oP '^cores:\s*\K\S+' || echo '?')"
@@ -250,9 +260,9 @@ cmd_modify() {
exit 1 exit 1
fi fi
local set_cmd="qm set" local set_cmd="${sudo_prefix} qm set"
local resize_cmd="qm resize" local resize_cmd="${sudo_prefix} qm resize"
[[ "$kind" == "lxc" ]] && set_cmd="pct set" && resize_cmd="pct resize" [[ "$kind" == "lxc" ]] && set_cmd="${sudo_prefix} pct set" && resize_cmd="${sudo_prefix} pct resize"
if [[ -n "$cores" || -n "$memory" ]]; then if [[ -n "$cores" || -n "$memory" ]]; then
local args="" local args=""
@@ -359,14 +369,15 @@ else
echo echo
echo "==> Checking ${node} for an existing VM/CT identified as '${host}'..." echo "==> Checking ${node} for an existing VM/CT identified as '${host}'..."
ssh_check_status=0 ssh_check_status=0
existing="$(ssh "$ssh_target" bash -s -- "$host" <<'REMOTE_SCRIPT' existing="$(ssh "$ssh_target" bash -s -- "$host" "$sudo_prefix" <<'REMOTE_SCRIPT'
target="$1" target="$1"
for id in $(qm list 2>/dev/null | awk 'NR>1{print $1}'); do sudo_pfx="$2"
n="$(qm config "$id" 2>/dev/null | grep -oP '^name:\s*\K\S+' || true)" for id in $($sudo_pfx qm list 2>/dev/null | awk 'NR>1{print $1}'); do
n="$($sudo_pfx qm config "$id" 2>/dev/null | grep -oP '^name:\s*\K\S+' || true)"
[[ "$n" == "$target" ]] && echo "vm ${id} ${n}" [[ "$n" == "$target" ]] && echo "vm ${id} ${n}"
done done
for id in $(pct list 2>/dev/null | awk 'NR>1{print $1}'); do for id in $($sudo_pfx pct list 2>/dev/null | awk 'NR>1{print $1}'); do
n="$(pct config "$id" 2>/dev/null | grep -oP '^hostname:\s*\K\S+' || true)" n="$($sudo_pfx pct config "$id" 2>/dev/null | grep -oP '^hostname:\s*\K\S+' || true)"
[[ "$n" == "$target" ]] && echo "lxc ${id} ${n}" [[ "$n" == "$target" ]] && echo "lxc ${id} ${n}"
done done
exit 0 exit 0
@@ -453,12 +464,12 @@ REMOTE_SCRIPT
if [[ "$kind" == "vm" ]]; then if [[ "$kind" == "vm" ]]; then
# qm destroy has no --force to stop-then-destroy in one call (pct's # qm destroy has no --force to stop-then-destroy in one call (pct's
# does) -- stop explicitly first if it's running. # does) -- stop explicitly first if it's running.
if ssh "$ssh_target" "qm status ${id}" 2>/dev/null | grep -q running; then if ssh "$ssh_target" "${sudo_prefix} qm status ${id}" 2>/dev/null | grep -q running; then
ssh "$ssh_target" "qm stop ${id}" ssh "$ssh_target" "${sudo_prefix} qm stop ${id}"
fi fi
ssh "$ssh_target" "qm destroy ${id} --purge 1" ssh "$ssh_target" "${sudo_prefix} qm destroy ${id} --purge 1"
else else
ssh "$ssh_target" "pct destroy ${id} --force 1 --purge 1" ssh "$ssh_target" "${sudo_prefix} pct destroy ${id} --force 1 --purge 1"
fi fi
done done
fi fi
@@ -487,7 +498,7 @@ if [[ -z "$vmid" ]]; then
vmid="<next-free-vmid>" vmid="<next-free-vmid>"
echo "[dry-run] would ask ${node} for the next free VMID (pvesh get /cluster/nextid)" echo "[dry-run] would ask ${node} for the next free VMID (pvesh get /cluster/nextid)"
else else
vmid="$(ssh "$ssh_target" "pvesh get /cluster/nextid" | tr -d '[:space:]')" vmid="$(ssh "$ssh_target" "${sudo_prefix} pvesh get /cluster/nextid" | tr -d '[:space:]')"
echo "Auto-assigned VMID: ${vmid}" echo "Auto-assigned VMID: ${vmid}"
fi fi
else else
@@ -501,8 +512,8 @@ if [[ "$dry_run" -eq 0 ]]; then
# both. Any success here means something is already using this ID -- # both. Any success here means something is already using this ID --
# refuse to go anywhere near it. (Reconfiguring an existing resource is # refuse to go anywhere near it. (Reconfiguring an existing resource is
# --modify's job, not this one's.) # --modify's job, not this one's.)
if ssh "$ssh_target" "qm status ${vmid}" >/dev/null 2>&1 \ if ssh "$ssh_target" "${sudo_prefix} qm status ${vmid}" >/dev/null 2>&1 \
|| ssh "$ssh_target" "pct status ${vmid}" >/dev/null 2>&1; then || ssh "$ssh_target" "${sudo_prefix} pct status ${vmid}" >/dev/null 2>&1; then
echo "ERROR: VMID ${vmid} already exists on ${node}. Refusing to touch an" >&2 echo "ERROR: VMID ${vmid} already exists on ${node}. Refusing to touch an" >&2
echo "existing resource here -- use --modify to reconfigure it, pick a" >&2 echo "existing resource here -- use --modify to reconfigure it, pick a" >&2
echo "different --vmid, or omit it to auto-assign." >&2 echo "different --vmid, or omit it to auto-assign." >&2
@@ -672,11 +683,11 @@ if [[ "$image_already_remote" -eq 0 && -z "$local_image" ]]; then
# hands the result to the remote shell to re-split, which would # hands the result to the remote shell to re-split, which would
# otherwise scatter NIX_EXTRA_OPTS (itself several space-separated, # otherwise scatter NIX_EXTRA_OPTS (itself several space-separated,
# %q-quoted tokens) across the wrong positional parameters below. # %q-quoted tokens) across the wrong positional parameters below.
printf -v remote_cmd 'bash -s -- %q %q %q %q %q' \ printf -v remote_cmd 'bash -s -- %q %q %q %q %q %q' \
"$remote_repo_dir" "$flake_target" "$remote_dir" "$remote_filename" "$NIX_EXTRA_OPTS" "$remote_repo_dir" "$flake_target" "$remote_dir" "$remote_filename" "$NIX_EXTRA_OPTS" "$sudo_prefix"
ssh "$ssh_target" "$remote_cmd" <<'REMOTE_SCRIPT' ssh "$ssh_target" "$remote_cmd" <<'REMOTE_SCRIPT'
set -euo pipefail set -euo pipefail
repo_dir="$1"; target="$2"; dest_dir="$3"; dest_name="$4"; nix_extra_opts_str="$5" repo_dir="$1"; target="$2"; dest_dir="$3"; dest_name="$4"; nix_extra_opts_str="$5"; sudo_pfx="$6"
declare -a NIX_OPTS=() declare -a NIX_OPTS=()
[[ -n "$nix_extra_opts_str" ]] && eval "NIX_OPTS=(${nix_extra_opts_str})" [[ -n "$nix_extra_opts_str" ]] && eval "NIX_OPTS=(${nix_extra_opts_str})"
cd "$repo_dir" cd "$repo_dir"
@@ -694,23 +705,14 @@ if [[ -z "$built" ]]; then
echo "ERROR: no tarball found under result-${target}/tarball after build." >&2 echo "ERROR: no tarball found under result-${target}/tarball after build." >&2
exit 1 exit 1
fi fi
mkdir -p "$dest_dir" $sudo_pfx mkdir -p "$dest_dir"
cp "$built" "${dest_dir}/${dest_name}" $sudo_pfx cp "$built" "${dest_dir}/${dest_name}"
echo "Built and staged: ${dest_dir}/${dest_name}" echo "Built and staged: ${dest_dir}/${dest_name}"
REMOTE_SCRIPT REMOTE_SCRIPT
local_image="$remote_path" local_image="$remote_path"
echo "Built on ${node}: ${remote_path}" echo "Built on ${node}: ${remote_path}"
fi fi
else else
# PROXMOX_SSH_USER defaults to root (env.sh), which needs no sudo and
# can't assume it's even installed on a minimal node -- only shell out
# through sudo when actually running as a non-root SSH user.
sudo_prefix="sudo"
sudo_display="sudo "
if [[ "$PROXMOX_SSH_USER" == "root" ]]; then
sudo_prefix=""
sudo_display=""
fi
if [[ "$dry_run" -eq 1 ]]; then if [[ "$dry_run" -eq 1 ]]; then
echo "[dry-run] would build on ${node}: nix build --no-use-registries --no-accept-flake-config${nix_opts_display} \\" echo "[dry-run] would build on ${node}: nix build --no-use-registries --no-accept-flake-config${nix_opts_display} \\"
echo "[dry-run] .#nixosConfigurations.${flake_target}.config.system.build.diskoImagesScript" echo "[dry-run] .#nixosConfigurations.${flake_target}.config.system.build.diskoImagesScript"
@@ -728,7 +730,7 @@ REMOTE_SCRIPT
"$remote_repo_dir" "$flake_target" "$remote_dir" "$remote_filename" "$NIX_EXTRA_OPTS" "$sudo_prefix" "$remote_repo_dir" "$flake_target" "$remote_dir" "$remote_filename" "$NIX_EXTRA_OPTS" "$sudo_prefix"
ssh "$ssh_target" "$remote_cmd" <<'REMOTE_SCRIPT' ssh "$ssh_target" "$remote_cmd" <<'REMOTE_SCRIPT'
set -euo pipefail set -euo pipefail
repo_dir="$1"; target="$2"; dest_dir="$3"; dest_name="$4"; nix_extra_opts_str="$5"; sudo_prefix="$6" repo_dir="$1"; target="$2"; dest_dir="$3"; dest_name="$4"; nix_extra_opts_str="$5"; sudo_pfx="$6"
declare -a NIX_OPTS=() declare -a NIX_OPTS=()
[[ -n "$nix_extra_opts_str" ]] && eval "NIX_OPTS=(${nix_extra_opts_str})" [[ -n "$nix_extra_opts_str" ]] && eval "NIX_OPTS=(${nix_extra_opts_str})"
cd "$repo_dir" cd "$repo_dir"
@@ -737,7 +739,7 @@ ensure_nix_profile
nix build --no-use-registries --no-accept-flake-config "${NIX_OPTS[@]}" \ nix build --no-use-registries --no-accept-flake-config "${NIX_OPTS[@]}" \
".#nixosConfigurations.${target}.config.system.build.diskoImagesScript" \ ".#nixosConfigurations.${target}.config.system.build.diskoImagesScript" \
--out-link "result-${target}" --out-link "result-${target}"
$sudo_prefix "./result-${target}" \ $sudo_pfx "./result-${target}" \
--pre-format-files "host-keys/${target}_ssh_host_ed25519_key" /etc/ssh/ssh_host_ed25519_key \ --pre-format-files "host-keys/${target}_ssh_host_ed25519_key" /etc/ssh/ssh_host_ed25519_key \
--pre-format-files "host-keys/${target}_ssh_host_ed25519_key.pub" /etc/ssh/ssh_host_ed25519_key.pub \ --pre-format-files "host-keys/${target}_ssh_host_ed25519_key.pub" /etc/ssh/ssh_host_ed25519_key.pub \
--build-memory 2048 --build-memory 2048
@@ -746,8 +748,8 @@ if [[ -z "$built" ]]; then
echo "ERROR: no .raw image found in ${repo_dir} after build." >&2 echo "ERROR: no .raw image found in ${repo_dir} after build." >&2
exit 1 exit 1
fi fi
mkdir -p "$dest_dir" $sudo_pfx mkdir -p "$dest_dir"
mv "$built" "${dest_dir}/${dest_name}" $sudo_pfx mv "$built" "${dest_dir}/${dest_name}"
echo "Built and staged: ${dest_dir}/${dest_name}" echo "Built and staged: ${dest_dir}/${dest_name}"
REMOTE_SCRIPT REMOTE_SCRIPT
local_image="$remote_path" local_image="$remote_path"
@@ -812,9 +814,9 @@ if [[ "$type" == "lxc" ]]; then
# hands the whole string to `ssh` as a single command for the *remote* # hands the whole string to `ssh` as a single command for the *remote*
# shell to parse -- unquoted, that `;` would be read as a remote # shell to parse -- unquoted, that `;` would be read as a remote
# command separator and silently truncate this into two commands. # command separator and silently truncate this into two commands.
create_cmd="pct create ${vmid} ${iso_storage}:vztmpl/${remote_filename} --unprivileged ${unprivileged_flag} --features '${PROXMOX_DEFAULT_LXC_FEATURES}' --rootfs ${storage}:${local_disk_size} --hostname ${name} --cores ${cores} --memory ${memory} --swap ${local_swap} --net0 name=eth0,bridge=${bridge},ip=dhcp" create_cmd="${sudo_prefix} pct create ${vmid} ${iso_storage}:vztmpl/${remote_filename} --unprivileged ${unprivileged_flag} --features '${PROXMOX_DEFAULT_LXC_FEATURES}' --rootfs ${storage}:${local_disk_size} --hostname ${name} --cores ${cores} --memory ${memory} --swap ${local_swap} --net0 name=eth0,bridge=${bridge},ip=dhcp"
remote "$create_cmd" remote "$create_cmd"
remote "pct start ${vmid}" remote "${sudo_prefix} pct start ${vmid}"
else else
echo "==> Creating VM ${vmid} (${name})..." echo "==> Creating VM ${vmid} (${name})..."
# pre-enrolled-keys=0 disables OVMF's Secure Boot key pre-enrollment -- # pre-enrolled-keys=0 disables OVMF's Secure Boot key pre-enrollment --
@@ -825,29 +827,29 @@ else
# without this flag Proxmox never creates the channel it listens on, so # without this flag Proxmox never creates the channel it listens on, so
# `qm guest exec`/`qm agent` and the UI's IP-address display silently # `qm guest exec`/`qm agent` and the UI's IP-address display silently
# never work for any VM this script creates. # never work for any VM this script creates.
remote "qm create ${vmid} --name ${name} --memory ${memory} --cores ${cores} \ remote "${sudo_prefix} qm create ${vmid} --name ${name} --memory ${memory} --cores ${cores} \
--net0 virtio,bridge=${bridge} --bios ovmf --machine q35 --scsihw virtio-scsi-pci \ --net0 virtio,bridge=${bridge} --bios ovmf --machine q35 --scsihw virtio-scsi-pci \
--efidisk0 ${storage}:1,efitype=4m,pre-enrolled-keys=0 --agent enabled=1" --efidisk0 ${storage}:1,efitype=4m,pre-enrolled-keys=0 --agent enabled=1"
if [[ "$dry_run" -eq 1 ]]; then if [[ "$dry_run" -eq 1 ]]; then
echo "[dry-run] ssh ${ssh_target} -- qm importdisk ${vmid} ${remote_path} ${storage}" echo "[dry-run] ssh ${ssh_target} -- ${sudo_display}qm importdisk ${vmid} ${remote_path} ${storage}"
echo "[dry-run] (would parse the resulting disk identifier from that output)" echo "[dry-run] (would parse the resulting disk identifier from that output)"
echo "[dry-run] ssh ${ssh_target} -- qm set ${vmid} --scsi0 ${storage}:<parsed-disk-id>" echo "[dry-run] ssh ${ssh_target} -- ${sudo_display}qm set ${vmid} --scsi0 ${storage}:<parsed-disk-id>"
else else
importdisk_output="$(ssh "$ssh_target" "qm importdisk ${vmid} ${remote_path} ${storage}")" importdisk_output="$(ssh "$ssh_target" "${sudo_prefix} qm importdisk ${vmid} ${remote_path} ${storage}")"
echo "$importdisk_output" echo "$importdisk_output"
disk_id="$(echo "$importdisk_output" | grep -oP "(?<=Successfully imported disk as ')[^']+" | sed 's/^unused[0-9]*://')" disk_id="$(echo "$importdisk_output" | grep -oP "(?<=Successfully imported disk as ')[^']+" | sed 's/^unused[0-9]*://')"
if [[ -z "$disk_id" ]]; then if [[ -z "$disk_id" ]]; then
echo "ERROR: couldn't parse the imported disk identifier from qm importdisk's output above." >&2 echo "ERROR: couldn't parse the imported disk identifier from qm importdisk's output above." >&2
echo "The VM shell (${vmid}) and imported disk both exist -- finish attaching it by hand:" >&2 echo "The VM shell (${vmid}) and imported disk both exist -- finish attaching it by hand:" >&2
echo " ssh ${ssh_target} -- qm set ${vmid} --scsi0 ${storage}:<disk-id-from-output-above>" >&2 echo " ssh ${ssh_target} -- ${sudo_display}qm set ${vmid} --scsi0 ${storage}:<disk-id-from-output-above>" >&2
echo " ssh ${ssh_target} -- qm set ${vmid} --boot order=scsi0" >&2 echo " ssh ${ssh_target} -- ${sudo_display}qm set ${vmid} --boot order=scsi0" >&2
exit 1 exit 1
fi fi
remote "qm set ${vmid} --scsi0 ${disk_id}" remote "${sudo_prefix} qm set ${vmid} --scsi0 ${disk_id}"
fi fi
remote "qm set ${vmid} --boot order=scsi0" remote "${sudo_prefix} qm set ${vmid} --boot order=scsi0"
remote "qm start ${vmid}" remote "${sudo_prefix} qm start ${vmid}"
fi fi
echo echo