Archived
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
 beatzaplenty
|
6e601ce85c | ||
|
beatzaplenty
|
cbdb288bc3 |
@@ -256,33 +256,6 @@ lift the shared part into `scripts/lib/` instead of copying it.
|
||||
`/etc/ssh/ssh_known_hosts`. Idempotent (re-running replaces its own
|
||||
marked block rather than duplicating it); restarts `nix-daemon` by
|
||||
default so the change takes effect immediately.
|
||||
- `scripts/proxmox/clone-pve1-to-pve-test.sh --vmid <n> [options]` — ad hoc
|
||||
copy of a single VM/CT from one node to another via `vzdump` +
|
||||
`qmrestore`/`pct restore`, relayed through this machine
|
||||
(`ssh src cat ... | ssh dst cat > ...`, no on-disk staging copy).
|
||||
Defaults `pve1` → `pve-test` (see "Two Proxmox nodes" above); source is
|
||||
left running throughout (`--mode snapshot` default) and untouched
|
||||
beyond the vzdump + a typed-VMID confirm, restored on the target with a
|
||||
fresh MAC (`--unique 1`, always) since the source is expected to still
|
||||
be live on the same LAN. Backup archives are deleted from both nodes
|
||||
afterward unless `--keep-backup`. `--dry-run` throughout.
|
||||
- `scripts/proxmox/migrate-vm.sh --vmid <n> [options]` — moves (rather
|
||||
than clones) a single VM/CT between nodes, built on the same
|
||||
vzdump/relay/restore pattern as `clone-pve1-to-pve-test.sh` above, plus
|
||||
a cutover step (stop the guest on the source node, start it on the
|
||||
target) and, optionally, `--remove-source` to destroy it on the source
|
||||
node afterward (its own separate typed confirmation, since that step is
|
||||
irreversible) — the default instead leaves the stopped source guest
|
||||
intact as a safety net. Restores with the *original* MAC (no
|
||||
`--unique`) unless `--unique` is passed, since the source is expected to
|
||||
go away rather than keep running alongside the target. **Not** a true
|
||||
`qm migrate`-style live migration — pve1/pve-test aren't a clustered
|
||||
pair with shared storage, so this is vzdump-based and, with the default
|
||||
`--mode snapshot`, any writes on the source between the snapshot and
|
||||
cutover are lost (use `--mode stop` if that's not acceptable). Generic
|
||||
node-to-node (not pve1/pve-test-specific) via
|
||||
`--source-node`/`--target-node`, defaulting to pve1 → pve-test like the
|
||||
clone script. `--dry-run` throughout.
|
||||
|
||||
### `scripts/lib/`
|
||||
|
||||
|
||||
@@ -150,11 +150,7 @@ use case.
|
||||
|
||||
The `pxe` variant is also built automatically as part of the `pxe-boot` host
|
||||
itself (`modules/pxe-boot/stage-installer-artifacts.nix`) and served over
|
||||
iPXE as the menu's "NixOS Auto-Installer" entry — see `docs/pxe-boot.md`.
|
||||
That same host also builds and serves `packages.x86_64-linux.pxe-minimal`,
|
||||
a vanilla NixOS minimal netboot image with none of this auto-installer's
|
||||
wiring, as a separate "NixOS Minimal" menu entry — also documented in
|
||||
`docs/pxe-boot.md`, not covered further here since it's not this installer.
|
||||
iPXE — see `docs/pxe-boot.md`.
|
||||
|
||||
## Host keys
|
||||
|
||||
|
||||
+16
-35
@@ -1,10 +1,9 @@
|
||||
# pxe-boot
|
||||
|
||||
The `pxe-boot` host serves HTTP boot assets for iPXE clients — including
|
||||
self-staged copies of both this flake's own auto-installer netboot image
|
||||
(see `docs/auto-installer.md` for what that image actually is and does once
|
||||
booted) and a vanilla, unmodified NixOS minimal netboot image for plain
|
||||
rescue/inspection use.
|
||||
The `pxe-boot` host serves HTTP boot assets for iPXE clients — including a
|
||||
self-staged copy of this flake's own auto-installer netboot image, see
|
||||
`docs/auto-installer.md` for what that image actually is and does once
|
||||
booted.
|
||||
|
||||
## Host Role
|
||||
|
||||
@@ -29,8 +28,7 @@ The host creates these directories with systemd tmpfiles:
|
||||
/srv/pxe
|
||||
/srv/pxe/http
|
||||
/srv/pxe/http/images
|
||||
/srv/pxe/http/auto-installer
|
||||
/srv/pxe/http/nixos-minimal
|
||||
/srv/pxe/http/nixos
|
||||
/srv/pxe/http/systemrescue
|
||||
/srv/pxe/http/ubuntu
|
||||
/srv/pxe/http/rescue
|
||||
@@ -39,7 +37,7 @@ The host creates these directories with systemd tmpfiles:
|
||||
|
||||
Mount shared image storage under `/srv/pxe/http`, preferably
|
||||
`/srv/pxe/http/images` unless a menu entry expects files in a specific
|
||||
directory such as `/srv/pxe/http/auto-installer`.
|
||||
directory such as `/srv/pxe/http/nixos`.
|
||||
|
||||
The HTTP iPXE chain is:
|
||||
|
||||
@@ -52,37 +50,20 @@ undionly.kpxe or ipxe.efi
|
||||
|
||||
The generated menu currently exposes entries for:
|
||||
|
||||
- NixOS Auto-Installer
|
||||
- NixOS Minimal
|
||||
- NixOS installer
|
||||
- SystemRescue environment
|
||||
- iPXE shell
|
||||
- Reboot
|
||||
|
||||
Both NixOS entries chain-load a `netboot.ipxe` staged into their own
|
||||
directory (`/srv/pxe/http/auto-installer/netboot.ipxe` and
|
||||
`/srv/pxe/http/nixos-minimal/netboot.ipxe`), each nixpkgs' own generated
|
||||
netboot iPXE script (correct `init=`/`initrd=` kernel parameters included)
|
||||
rather than a hand-rolled boot line — that script in turn expects its
|
||||
kernel/initrd siblings in the same directory. Each directory's three files
|
||||
(`bzImage`, `initrd`, `netboot.ipxe`) are built from source and staged
|
||||
automatically by `modules/pxe-boot/stage-installer-artifacts.nix` via
|
||||
`systemd.tmpfiles.rules` — no manual operator step required:
|
||||
|
||||
- `auto-installer` is this flake's own `netbootSystem` (`flake.nix`) — the
|
||||
same auto-installer image `nix build .#pxe` produces. See
|
||||
`docs/auto-installer.md`.
|
||||
- `nixos-minimal` is `netbootMinimalSystem` (`flake.nix`) — nixpkgs'
|
||||
`netboot-minimal.nix` composed on its own, with none of this flake's
|
||||
auto-installer wiring (no `common.nix`, no `auto-install.sh`, no baked
|
||||
host keys or custom users). Same `nix build .#pxe-minimal` mechanism as
|
||||
the auto-installer image, just a different module composition. Useful
|
||||
as a plain rescue/inspection shell that doesn't assume anything about
|
||||
this flake.
|
||||
|
||||
Both images set `networking.hostName` to match their menu entry/staged
|
||||
directory name (`auto-installer` / `nixos-minimal`), so each one's
|
||||
generated system name (`nixos-system-<name>-*`) is self-describing rather
|
||||
than the nixpkgs default of `nixos-system-nixos-*` for both.
|
||||
The NixOS installer entry chain-loads `/srv/pxe/http/nixos/netboot.ipxe`,
|
||||
which is nixpkgs' own generated netboot iPXE script (correct `init=`/`initrd=`
|
||||
kernel parameters included) rather than a hand-rolled boot line — that script
|
||||
in turn expects its kernel/initrd siblings in the same directory. All three
|
||||
files (`bzImage`, `initrd`, `netboot.ipxe`) are built from this flake's own
|
||||
`modules/installer/iso.nix` netboot image (the same one `nix build .#pxe`
|
||||
produces) and staged automatically by
|
||||
`modules/pxe-boot/stage-installer-artifacts.nix` via `systemd.tmpfiles.rules`
|
||||
— no manual operator step required.
|
||||
|
||||
The SystemRescue entry expects the source ISO at:
|
||||
|
||||
|
||||
@@ -65,7 +65,7 @@
|
||||
# file without a same-option circular dependency (a module
|
||||
# contributing to environment.etc can't read the merged
|
||||
# environment.etc it's itself contributing to).
|
||||
specialArgs = { inherit inputs vars netbootSystem netbootMinimalSystem flakeTarget; };
|
||||
specialArgs = { inherit inputs vars netbootSystem flakeTarget; };
|
||||
};
|
||||
|
||||
# Generated platform x build-type matrix. pxe-boot has no linode
|
||||
@@ -119,61 +119,19 @@
|
||||
|
||||
# Same installer environment, built as netboot (kernel + initrd +
|
||||
# iPXE script) instead of an ISO — this is what packages.pxe bundles.
|
||||
#
|
||||
# Deliberately imports common.nix directly, NOT ./modules/installer/iso.nix
|
||||
# (which pulls in nixpkgs' installation-cd-minimal.nix) -- confirmed live
|
||||
# that composing the ISO module together with netboot-minimal.nix hangs
|
||||
# every boot waiting for a device that can never exist on a netboot
|
||||
# client ("A start job is running for /dev/disk/by-label/nixos-minimal-...").
|
||||
# Both installation-cd-base.nix and netboot.nix set fileSystems."/" via
|
||||
# the identical lib.mkImageMediaOverride (mkOverride 60) priority --
|
||||
# genuinely conflicting root-filesystem strategies (ISO-by-label vs.
|
||||
# netboot-tmpfs) at the same priority, and the ISO one was winning.
|
||||
# netboot-minimal.nix's own chain (netboot-base.nix) already imports
|
||||
# profiles/installation-device.nix independently, so common.nix's
|
||||
# initialHashedPassword override (which assumes that profile is
|
||||
# present) still applies correctly without iso.nix in the mix.
|
||||
#
|
||||
# networking.hostName is set explicitly (rather than left at nixpkgs'
|
||||
# own "nixos" default) so this image's generated system name
|
||||
# (nixos-system-auto-installer-*) matches its iPXE menu entry —
|
||||
# see modules/build-types/pxe-boot.nix's :auto-installer item — and
|
||||
# its staged directory, /srv/pxe/http/auto-installer.
|
||||
netbootSystem = nixpkgs.lib.nixosSystem {
|
||||
inherit system;
|
||||
modules = [
|
||||
./modules/installer/common.nix
|
||||
./modules/installer/iso.nix
|
||||
({ modulesPath, ... }: {
|
||||
imports = [
|
||||
(modulesPath + "/installer/netboot/netboot-minimal.nix")
|
||||
];
|
||||
})
|
||||
{ networking.hostName = "auto-installer"; }
|
||||
];
|
||||
specialArgs = { inherit vars; };
|
||||
};
|
||||
|
||||
# A genuinely vanilla NixOS minimal netboot image: nixpkgs'
|
||||
# netboot-minimal.nix on its own, with none of this flake's
|
||||
# auto-installer wiring (no common.nix — no auto-install.sh, no
|
||||
# baked host keys, no custom users/passwords). Built from source via
|
||||
# the same nixosSystem + netboot-minimal.nix path as netbootSystem
|
||||
# above, so both go through an identical build mechanism; the only
|
||||
# difference is what's composed in. hostName again matches this
|
||||
# image's iPXE menu entry (:nixos-minimal) and staged directory
|
||||
# (/srv/pxe/http/nixos-minimal).
|
||||
netbootMinimalSystem = nixpkgs.lib.nixosSystem {
|
||||
inherit system;
|
||||
modules = [
|
||||
({ modulesPath, ... }: {
|
||||
imports = [
|
||||
(modulesPath + "/installer/netboot/netboot-minimal.nix")
|
||||
];
|
||||
})
|
||||
{ networking.hostName = "nixos-minimal"; }
|
||||
];
|
||||
};
|
||||
|
||||
in
|
||||
{
|
||||
|
||||
@@ -195,15 +153,6 @@
|
||||
{ name = "initrd"; path = netbootSystem.config.system.build.netbootRamdisk; }
|
||||
{ name = "kernel"; path = netbootSystem.config.system.build.kernel; }
|
||||
];
|
||||
|
||||
# Vanilla NixOS minimal netboot bundle — see netbootMinimalSystem
|
||||
# above. Staged onto the pxe-boot host alongside packages.pxe by
|
||||
# modules/pxe-boot/stage-installer-artifacts.nix.
|
||||
pxe-minimal = pkgs.linkFarm "pxe-minimal" [
|
||||
{ name = "netboot.ipxe"; path = netbootMinimalSystem.config.system.build.netbootIpxeScript; }
|
||||
{ name = "initrd"; path = netbootMinimalSystem.config.system.build.netbootRamdisk; }
|
||||
{ name = "kernel"; path = netbootMinimalSystem.config.system.build.kernel; }
|
||||
];
|
||||
};
|
||||
};
|
||||
}
|
||||
|
||||
@@ -68,19 +68,15 @@ let
|
||||
set base ${pxeBaseUrl}
|
||||
|
||||
menu PXE Boot Menu
|
||||
item auto-installer NixOS Auto-Installer
|
||||
item nixos-minimal NixOS Minimal
|
||||
item nixos NixOS Installer
|
||||
item rescue Rescue Environment
|
||||
item shell iPXE Shell
|
||||
item reboot Reboot
|
||||
|
||||
choose target && goto ''${target}
|
||||
|
||||
:auto-installer
|
||||
chain ''${base}/auto-installer/netboot.ipxe
|
||||
|
||||
:nixos-minimal
|
||||
chain ''${base}/nixos-minimal/netboot.ipxe
|
||||
:nixos
|
||||
chain ''${base}/nixos/netboot.ipxe
|
||||
|
||||
:rescue
|
||||
chain ''${base}/systemrescue.ipxe
|
||||
@@ -133,8 +129,7 @@ in
|
||||
"d ${pxeRoot} 0755 root root -"
|
||||
"d ${httpRoot} 0755 root root -"
|
||||
"d ${httpRoot}/images 0755 root root -"
|
||||
"d ${httpRoot}/auto-installer 0755 root root -"
|
||||
"d ${httpRoot}/nixos-minimal 0755 root root -"
|
||||
"d ${httpRoot}/nixos 0755 root root -"
|
||||
"d ${httpRoot}/systemrescue 0755 root root -"
|
||||
"d ${httpRoot}/ubuntu 0755 root root -"
|
||||
"d ${httpRoot}/rescue 0755 root root -"
|
||||
|
||||
@@ -1,32 +1,23 @@
|
||||
{ netbootSystem, netbootMinimalSystem, ... }:
|
||||
{ netbootSystem, ... }:
|
||||
|
||||
let
|
||||
# config.system.build.kernel and .netbootRamdisk are directories, not the
|
||||
# files themselves — nixpkgs' own system.build.kexecTree does the same
|
||||
# ${...}/<file> dereference for the same reason.
|
||||
mkStageRules = { dirName, system }:
|
||||
let
|
||||
inherit (system.config.system.boot.loader) kernelFile;
|
||||
dir = "/srv/pxe/http/${dirName}";
|
||||
in
|
||||
[
|
||||
# Declared here too (not just in build-types/pxe-boot.nix) so this
|
||||
# module's C+ rules don't depend on cross-module list-merge ordering —
|
||||
# tmpfiles' C type needs the target directory to already exist.
|
||||
"d ${dir} 0755 root root -"
|
||||
"C+ ${dir}/${kernelFile} 0644 root root - ${system.config.system.build.kernel}/${kernelFile}"
|
||||
"C+ ${dir}/initrd 0644 root root - ${system.config.system.build.netbootRamdisk}/initrd"
|
||||
"C+ ${dir}/netboot.ipxe 0644 root root - ${system.config.system.build.netbootIpxeScript}/netboot.ipxe"
|
||||
];
|
||||
inherit (netbootSystem.config.system.boot.loader) kernelFile;
|
||||
in
|
||||
{
|
||||
# Builds this flake's own installer netboot image (the same one
|
||||
# `nix build .#pxe` produces) plus the vanilla NixOS minimal netboot image
|
||||
# (`nix build .#pxe-minimal`), and stages both where menu.ipxe's
|
||||
# :auto-installer / :nixos-minimal entries expect them, so the pxe-boot
|
||||
# host is self-contained — no manual operator step to populate
|
||||
# /srv/pxe/http after deploy.
|
||||
systemd.tmpfiles.rules =
|
||||
mkStageRules { dirName = "auto-installer"; system = netbootSystem; }
|
||||
++ mkStageRules { dirName = "nixos-minimal"; system = netbootMinimalSystem; };
|
||||
# `nix build .#pxe` produces) and stages it where menu.ipxe's :nixos
|
||||
# entry expects it, so the pxe-boot host is self-contained — no manual
|
||||
# operator step to populate /srv/pxe/http/nixos after deploy.
|
||||
systemd.tmpfiles.rules = [
|
||||
# Declared here too (not just in build-types/pxe-boot.nix) so this
|
||||
# module's C+ rules don't depend on cross-module list-merge ordering —
|
||||
# tmpfiles' C type needs the target directory to already exist.
|
||||
"d /srv/pxe/http/nixos 0755 root root -"
|
||||
"C+ /srv/pxe/http/nixos/${kernelFile} 0644 root root - ${netbootSystem.config.system.build.kernel}/${kernelFile}"
|
||||
"C+ /srv/pxe/http/nixos/initrd 0644 root root - ${netbootSystem.config.system.build.netbootRamdisk}/initrd"
|
||||
"C+ /srv/pxe/http/nixos/netboot.ipxe 0644 root root - ${netbootSystem.config.system.build.netbootIpxeScript}/netboot.ipxe"
|
||||
];
|
||||
}
|
||||
|
||||
@@ -1,221 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Ad hoc clone of a single VM/CT from pve1 (production) to pve-test
|
||||
# (sandbox), via vzdump + qmrestore/pct restore -- not a general-purpose
|
||||
# backup tool, just a quick "give me a disposable copy of this thing on
|
||||
# pve-test" for testing against real-ish data without touching prod.
|
||||
#
|
||||
# Flow:
|
||||
# 1. vzdump the resource on pve1 into its "local" storage (--mode
|
||||
# snapshot by default, so the source keeps running throughout --
|
||||
# see --mode below for when that's not possible).
|
||||
# 2. Stream the resulting archive straight from pve1 to pve-test
|
||||
# (ssh pve1 cat ... | ssh pve-test cat > ...) -- this machine is
|
||||
# just the relay, no separate on-disk staging copy here.
|
||||
# 3. qmrestore / pct restore it on pve-test under --new-vmid (default:
|
||||
# same VMID as the source -- pve-test is a separate node/cluster, so
|
||||
# no collision unless that VMID is already in use there too).
|
||||
# Always restored with --unique 1 (fresh MAC addresses) since the
|
||||
# source is typically still running on the same LAN -- restoring
|
||||
# with the *same* MAC would put two live guests on the wire with
|
||||
# identical hardware addresses.
|
||||
# 4. Delete the vzdump archive from pve1's local storage and the
|
||||
# relayed copy on pve-test, so neither node accumulates ad hoc
|
||||
# backup files from this script. Only the pve1 original is
|
||||
# preserved on any failure after step 1, so a failed
|
||||
# transfer/restore can be retried without re-running the backup.
|
||||
#
|
||||
# This script's own defaults are pve1 -> pve-test, unlike
|
||||
# create-proxmox-resource.sh's --node (which defaults to production) --
|
||||
# see CLAUDE.md's "Two Proxmox nodes" section. pve1 is only ever touched
|
||||
# here after typing the source VMID back to confirm; pve-test is treated
|
||||
# as disposable, matching this repo's usual policy for that node.
|
||||
#
|
||||
# See --help for the full option list.
|
||||
set -euo pipefail
|
||||
|
||||
repo_root="$(cd "$(dirname "$0")/../.." && pwd)"
|
||||
# shellcheck source=../env.sh
|
||||
source "${repo_root}/scripts/env.sh"
|
||||
# shellcheck source=../lib/confirm.sh
|
||||
source "${repo_root}/scripts/lib/confirm.sh"
|
||||
|
||||
usage() {
|
||||
cat <<EOF
|
||||
Usage: $0 --vmid <n> [options]
|
||||
|
||||
--vmid <n> Required: VMID on the source node to clone.
|
||||
Kind (qemu VM vs LXC CT) is auto-detected.
|
||||
--new-vmid <n> VMID to restore as on the target node
|
||||
(default: same as --vmid).
|
||||
--mode snapshot|suspend|stop
|
||||
vzdump backup mode (default: snapshot -- the
|
||||
source resource keeps running throughout;
|
||||
requires snapshot-capable storage, e.g.
|
||||
ZFS/LVM-thin/Ceph/qcow2). Fall back to
|
||||
"suspend" (brief pause) or "stop" (source
|
||||
goes down for the duration) if the source's
|
||||
storage doesn't support live snapshots --
|
||||
vzdump's own error will say so.
|
||||
--source-node <host> (default: \$PVE1_HOST, ${PVE1_HOST})
|
||||
--target-node <host> (default: \$PVE_TEST_HOST, ${PVE_TEST_HOST})
|
||||
--source-storage <pool> Where vzdump writes the backup on the
|
||||
source node (default: local).
|
||||
--target-storage <pool> Where the restored disk/rootfs lands on
|
||||
the target node (default: \$PROXMOX_STORAGE, ${PROXMOX_STORAGE}).
|
||||
--keep-backup Don't delete the vzdump archive from
|
||||
either node afterward (debugging aid).
|
||||
--yes Skip the typed VMID confirmation
|
||||
before touching the source node.
|
||||
--dry-run Print the full plan and skip every
|
||||
mutating step (vzdump, transfer,
|
||||
restore, delete) and the confirm
|
||||
prompt. Still makes read-only SSH
|
||||
calls to look up the source kind
|
||||
and check the target VMID is free
|
||||
-- harmless on either node.
|
||||
-h, --help
|
||||
EOF
|
||||
}
|
||||
|
||||
vmid=""
|
||||
new_vmid=""
|
||||
mode="snapshot"
|
||||
source_node="$PVE1_HOST"
|
||||
target_node="$PVE_TEST_HOST"
|
||||
source_storage="local"
|
||||
target_storage="$PROXMOX_STORAGE"
|
||||
keep_backup=0
|
||||
skip_confirm=0
|
||||
dry_run=0
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--vmid) vmid="$2"; shift 2 ;;
|
||||
--new-vmid) new_vmid="$2"; shift 2 ;;
|
||||
--mode) mode="$2"; shift 2 ;;
|
||||
--source-node) source_node="$2"; shift 2 ;;
|
||||
--target-node) target_node="$2"; shift 2 ;;
|
||||
--source-storage) source_storage="$2"; shift 2 ;;
|
||||
--target-storage) target_storage="$2"; shift 2 ;;
|
||||
--keep-backup) keep_backup=1; shift ;;
|
||||
--yes) skip_confirm=1; shift ;;
|
||||
--dry-run) dry_run=1; shift ;;
|
||||
-h | --help) usage; exit 0 ;;
|
||||
*) echo "Unknown option: $1" >&2; usage >&2; exit 1 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [[ -z "$vmid" ]]; then
|
||||
echo "ERROR: --vmid is required." >&2
|
||||
usage >&2
|
||||
exit 1
|
||||
fi
|
||||
if [[ "$mode" != "snapshot" && "$mode" != "suspend" && "$mode" != "stop" ]]; then
|
||||
echo "ERROR: --mode must be snapshot, suspend, or stop." >&2
|
||||
exit 1
|
||||
fi
|
||||
[[ -z "$new_vmid" ]] && new_vmid="$vmid"
|
||||
|
||||
source_target="${PROXMOX_SSH_USER}@${source_node}"
|
||||
target_target="${PROXMOX_SSH_USER}@${target_node}"
|
||||
|
||||
# No dry-run wrapper needed for the calls below: every mutating step
|
||||
# (vzdump, transfer, restore, delete) is reached only after the --dry-run
|
||||
# early-exit further down, so a plain `ssh` call is never in the dry-run
|
||||
# path.
|
||||
|
||||
# --- identify the resource kind on the source node -----------------------
|
||||
echo "==> Looking up VMID ${vmid} on ${source_node}..."
|
||||
kind=""
|
||||
if ssh "$source_target" "qm status ${vmid}" >/dev/null 2>&1; then
|
||||
kind="vm"
|
||||
elif ssh "$source_target" "pct status ${vmid}" >/dev/null 2>&1; then
|
||||
kind="lxc"
|
||||
else
|
||||
echo "ERROR: VMID ${vmid} doesn't exist on ${source_node} as either a VM or CT." >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "VMID ${vmid} on ${source_node} is a ${kind}."
|
||||
|
||||
# --- refuse to clobber an existing resource on the target node -----------
|
||||
if ssh "$target_target" "qm status ${new_vmid}" >/dev/null 2>&1 \
|
||||
|| ssh "$target_target" "pct status ${new_vmid}" >/dev/null 2>&1; then
|
||||
echo "ERROR: VMID ${new_vmid} already exists on ${target_node}. Pass --new-vmid" >&2
|
||||
echo "with a free ID, or remove the existing resource there first." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo
|
||||
echo "Plan:"
|
||||
echo " source: ${kind} VMID ${vmid} on ${source_node} (storage: ${source_storage}, mode: ${mode})"
|
||||
echo " target: VMID ${new_vmid} on ${target_node} (storage: ${target_storage}, fresh MAC via --unique)"
|
||||
[[ "$keep_backup" -eq 1 ]] && echo " backup archives are kept on both nodes afterward (--keep-backup)"
|
||||
|
||||
if [[ "$dry_run" -eq 1 ]]; then
|
||||
echo
|
||||
echo "[dry-run] No backup, transfer, restore, or delete was performed."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if [[ "$skip_confirm" -ne 1 ]]; then
|
||||
echo
|
||||
if ! confirm_typed "$vmid" "Type the source VMID (${vmid}) to confirm backing it up from ${source_node}: "; then
|
||||
echo "Cancelled -- input didn't match ${vmid}." >&2
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
# --- vzdump on the source node --------------------------------------------
|
||||
echo
|
||||
echo "==> Backing up VMID ${vmid} on ${source_node} (mode=${mode}, storage=${source_storage})..."
|
||||
vzdump_log="$(ssh "$source_target" \
|
||||
"vzdump ${vmid} --mode ${mode} --storage ${source_storage} --compress zstd" 2>&1)" \
|
||||
|| {
|
||||
echo "$vzdump_log" >&2
|
||||
echo "ERROR: vzdump failed on ${source_node}." >&2
|
||||
exit 1
|
||||
}
|
||||
echo "$vzdump_log"
|
||||
|
||||
archive="$(echo "$vzdump_log" | grep -oP "creating vzdump archive '\K[^']+" | tail -n1)"
|
||||
if [[ -z "$archive" ]]; then
|
||||
echo "ERROR: couldn't find the archive path in vzdump's output above." >&2
|
||||
exit 1
|
||||
fi
|
||||
archive_basename="$(basename "$archive")"
|
||||
target_tmp_archive="/var/tmp/${archive_basename}"
|
||||
echo "Archive: ${archive}"
|
||||
|
||||
# Always clean up the relayed copy on the target node, success or failure
|
||||
# -- it's only ever a working copy, restored or not.
|
||||
cleanup_target_tmp() {
|
||||
if [[ "$keep_backup" -ne 1 ]]; then
|
||||
ssh "$target_target" "rm -f '${target_tmp_archive}'" >/dev/null 2>&1 || true
|
||||
fi
|
||||
}
|
||||
trap cleanup_target_tmp EXIT
|
||||
|
||||
# --- relay the archive from source to target ------------------------------
|
||||
echo
|
||||
echo "==> Transferring archive to ${target_node}..."
|
||||
ssh "$source_target" "cat '${archive}'" | ssh "$target_target" "cat > '${target_tmp_archive}'"
|
||||
|
||||
# --- restore on the target node --------------------------------------------
|
||||
echo
|
||||
echo "==> Restoring as VMID ${new_vmid} on ${target_node} (storage=${target_storage})..."
|
||||
if [[ "$kind" == "vm" ]]; then
|
||||
ssh "$target_target" "qmrestore '${target_tmp_archive}' ${new_vmid} --storage ${target_storage} --unique 1"
|
||||
else
|
||||
ssh "$target_target" "pct restore ${new_vmid} '${target_tmp_archive}' --storage ${target_storage} --unique 1"
|
||||
fi
|
||||
|
||||
# --- clean up the source backup now that the restore succeeded -----------
|
||||
if [[ "$keep_backup" -ne 1 ]]; then
|
||||
echo
|
||||
echo "==> Deleting backup archive from ${source_node}'s ${source_storage} storage..."
|
||||
ssh "$source_target" "rm -f '${archive}' '${archive}.notes' '${archive}.log'" >/dev/null 2>&1 || true
|
||||
fi
|
||||
|
||||
echo
|
||||
echo "Done. VMID ${new_vmid} (${kind}) is now on ${target_node}, cloned from" \
|
||||
"VMID ${vmid} on ${source_node}."
|
||||
@@ -1,347 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Move a single VM/CT from one Proxmox node to another via vzdump +
|
||||
# qmrestore/pct restore -- the same relay pattern as
|
||||
# clone-pve1-to-pve-test.sh, but for a *migration* (the guest ends up
|
||||
# living on the target node only) rather than a *clone* (a disposable
|
||||
# copy, source untouched, fresh MAC).
|
||||
#
|
||||
# There is no real `qm migrate` here: that command only works between
|
||||
# nodes in the same Proxmox cluster with shared/replicated storage, which
|
||||
# pve1 and pve-test are not (see CLAUDE.md's "Two Proxmox nodes" section --
|
||||
# they're deliberately separate, unclustered nodes). This script is the
|
||||
# closest equivalent across two independent nodes, built out of the same
|
||||
# primitives as the clone script. It is NOT a true live migration -- read
|
||||
# the caveat below before using it for anything where losing a few
|
||||
# seconds/minutes of writes is unacceptable.
|
||||
#
|
||||
# Flow:
|
||||
# 1. vzdump the resource on the source node (--mode snapshot by
|
||||
# default, so the source keeps running throughout the bulk copy --
|
||||
# this is the part that's "live": the guest stays reachable while
|
||||
# its disk data is captured and shipped to the target).
|
||||
# 2. Stream the archive straight from the source node to the target
|
||||
# node (ssh source cat ... | ssh target cat > ...), same relay this
|
||||
# machine does in the clone script -- no separate on-disk staging
|
||||
# copy here either.
|
||||
# 3. qmrestore / pct restore it on the target node under the same VMID
|
||||
# by default (--new-vmid to pick a different one). Restored WITHOUT
|
||||
# --unique (i.e. original MAC preserved) unless --unique is passed
|
||||
# explicitly -- unlike the clone script, the source is being retired,
|
||||
# not left running alongside a copy, so there's no address collision
|
||||
# to avoid. qmrestore/pct restore don't start the guest, so nothing
|
||||
# is live on the target node yet.
|
||||
# 4. Cutover: stop the guest on the source node, then (unless --no-start)
|
||||
# start it on the target node. This is the only real downtime window
|
||||
# -- everything before this point runs with the source guest still
|
||||
# up.
|
||||
# 5. Unless --remove-source is passed, the now-stopped guest is LEFT ON
|
||||
# the source node as a safety net (config + disk intact, just
|
||||
# powered off) -- easy to start back up if the target copy turns out
|
||||
# to be broken. Pass --remove-source to actually destroy it there
|
||||
# once you've verified the target is good; this step gets its own
|
||||
# typed confirmation since qm destroy/pct destroy is irreversible.
|
||||
# 6. Delete the vzdump archive from both nodes' storage (unless
|
||||
# --keep-backup), same as the clone script -- neither node
|
||||
# accumulates ad hoc backup files from this script.
|
||||
#
|
||||
# --- IMPORTANT CAVEAT: this is not byte-perfect live migration ---
|
||||
# vzdump is not incremental. With the default --mode snapshot, the source
|
||||
# guest keeps running (and can keep writing to disk) for the entire time
|
||||
# between when the snapshot is taken and when this script stops it at
|
||||
# cutover. Any writes in that window are NOT captured in the archive and
|
||||
# will NOT exist on the target. For a large disk this window can be
|
||||
# minutes. If the workload can't tolerate that:
|
||||
# - use --mode stop (or --mode suspend) instead, which makes the source
|
||||
# guest go down *before* vzdump reads its disk, so the archive is
|
||||
# exactly the state being migrated and cutover has nothing left to
|
||||
# lose -- at the cost of the guest being down for the whole backup
|
||||
# duration instead of just the final cutover.
|
||||
# - or don't use this script -- put both nodes in an actual Proxmox
|
||||
# cluster with shared storage and use `qm migrate --online` instead,
|
||||
# which is the real thing this script is only approximating.
|
||||
#
|
||||
# This script's own defaults are pve1 -> pve-test, matching
|
||||
# clone-pve1-to-pve-test.sh and CLAUDE.md's "Two Proxmox nodes" section --
|
||||
# but unlike that script, this one is a generic node-to-node mover: pass
|
||||
# --source-node/--target-node for any other pair. Regardless of node
|
||||
# names, the source resource is only ever touched here after typing the
|
||||
# source VMID back to confirm, and destroying it on the source node
|
||||
# (--remove-source) requires a second, separate typed confirmation.
|
||||
#
|
||||
# See --help for the full option list.
|
||||
set -euo pipefail
|
||||
|
||||
repo_root="$(cd "$(dirname "$0")/../.." && pwd)"
|
||||
# shellcheck source=../env.sh
|
||||
source "${repo_root}/scripts/env.sh"
|
||||
# shellcheck source=../lib/confirm.sh
|
||||
source "${repo_root}/scripts/lib/confirm.sh"
|
||||
|
||||
usage() {
|
||||
cat <<EOF
|
||||
Usage: $0 --vmid <n> [options]
|
||||
|
||||
--vmid <n> Required: VMID on the source node to migrate.
|
||||
Kind (qemu VM vs LXC CT) is auto-detected.
|
||||
--new-vmid <n> VMID to restore as on the target node
|
||||
(default: same as --vmid).
|
||||
--mode snapshot|suspend|stop
|
||||
vzdump backup mode (default: snapshot -- the
|
||||
source resource keeps running until cutover;
|
||||
requires snapshot-capable storage, e.g.
|
||||
ZFS/LVM-thin/Ceph/qcow2). Use "stop" for a
|
||||
byte-perfect migration with no post-snapshot
|
||||
write gap (source goes down for the whole
|
||||
backup duration instead of just cutover) --
|
||||
see the caveat at the top of this script.
|
||||
--source-node <host> (default: \$PVE1_HOST, ${PVE1_HOST})
|
||||
--target-node <host> (default: \$PVE_TEST_HOST, ${PVE_TEST_HOST})
|
||||
--source-storage <pool> Where vzdump writes the backup on the
|
||||
source node (default: local).
|
||||
--target-storage <pool> Where the restored disk/rootfs lands on
|
||||
the target node (default: \$PROXMOX_STORAGE, ${PROXMOX_STORAGE}).
|
||||
--unique Restore with a fresh MAC address
|
||||
(--unique 1), as the clone script always
|
||||
does. Off by default here since the
|
||||
source is being retired, not left
|
||||
running alongside the target.
|
||||
--no-start Don't start the guest on the target
|
||||
node after cutover (default: start
|
||||
it).
|
||||
--remove-source Destroy the guest on the source
|
||||
node (qm destroy/pct destroy) after
|
||||
a successful cutover, instead of
|
||||
just leaving it stopped there.
|
||||
Irreversible -- prompts for its own
|
||||
typed confirmation unless --yes.
|
||||
--keep-backup Don't delete the vzdump archive from
|
||||
either node afterward (debugging aid).
|
||||
--yes Skip all typed confirmations (initial
|
||||
backup, cutover, and --remove-source
|
||||
if passed).
|
||||
--dry-run Print the full plan and skip every
|
||||
mutating step (vzdump, transfer,
|
||||
restore, cutover, destroy, delete)
|
||||
and every confirm prompt. Still makes
|
||||
read-only SSH calls to look up the
|
||||
source kind and check the target
|
||||
VMID is free -- harmless on either
|
||||
node.
|
||||
-h, --help
|
||||
EOF
|
||||
}
|
||||
|
||||
vmid=""
|
||||
new_vmid=""
|
||||
mode="snapshot"
|
||||
source_node="$PVE1_HOST"
|
||||
target_node="$PVE_TEST_HOST"
|
||||
source_storage="local"
|
||||
target_storage="$PROXMOX_STORAGE"
|
||||
unique=0
|
||||
start_target=1
|
||||
remove_source=0
|
||||
keep_backup=0
|
||||
skip_confirm=0
|
||||
dry_run=0
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--vmid) vmid="$2"; shift 2 ;;
|
||||
--new-vmid) new_vmid="$2"; shift 2 ;;
|
||||
--mode) mode="$2"; shift 2 ;;
|
||||
--source-node) source_node="$2"; shift 2 ;;
|
||||
--target-node) target_node="$2"; shift 2 ;;
|
||||
--source-storage) source_storage="$2"; shift 2 ;;
|
||||
--target-storage) target_storage="$2"; shift 2 ;;
|
||||
--unique) unique=1; shift ;;
|
||||
--no-start) start_target=0; shift ;;
|
||||
--remove-source) remove_source=1; shift ;;
|
||||
--keep-backup) keep_backup=1; shift ;;
|
||||
--yes) skip_confirm=1; shift ;;
|
||||
--dry-run) dry_run=1; shift ;;
|
||||
-h | --help) usage; exit 0 ;;
|
||||
*) echo "Unknown option: $1" >&2; usage >&2; exit 1 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [[ -z "$vmid" ]]; then
|
||||
echo "ERROR: --vmid is required." >&2
|
||||
usage >&2
|
||||
exit 1
|
||||
fi
|
||||
if [[ "$mode" != "snapshot" && "$mode" != "suspend" && "$mode" != "stop" ]]; then
|
||||
echo "ERROR: --mode must be snapshot, suspend, or stop." >&2
|
||||
exit 1
|
||||
fi
|
||||
[[ -z "$new_vmid" ]] && new_vmid="$vmid"
|
||||
|
||||
source_target="${PROXMOX_SSH_USER}@${source_node}"
|
||||
target_target="${PROXMOX_SSH_USER}@${target_node}"
|
||||
|
||||
# No dry-run wrapper needed for the calls below: every mutating step
|
||||
# (vzdump, transfer, restore, cutover, destroy, delete) is reached only
|
||||
# after the --dry-run early-exit further down, so a plain `ssh` call is
|
||||
# never in the dry-run path.
|
||||
|
||||
# --- identify the resource kind on the source node -----------------------
|
||||
echo "==> Looking up VMID ${vmid} on ${source_node}..."
|
||||
kind=""
|
||||
if ssh "$source_target" "qm status ${vmid}" >/dev/null 2>&1; then
|
||||
kind="vm"
|
||||
elif ssh "$source_target" "pct status ${vmid}" >/dev/null 2>&1; then
|
||||
kind="lxc"
|
||||
else
|
||||
echo "ERROR: VMID ${vmid} doesn't exist on ${source_node} as either a VM or CT." >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "VMID ${vmid} on ${source_node} is a ${kind}."
|
||||
|
||||
# --- refuse to clobber an existing resource on the target node -----------
|
||||
if ssh "$target_target" "qm status ${new_vmid}" >/dev/null 2>&1 \
|
||||
|| ssh "$target_target" "pct status ${new_vmid}" >/dev/null 2>&1; then
|
||||
echo "ERROR: VMID ${new_vmid} already exists on ${target_node}. Pass --new-vmid" >&2
|
||||
echo "with a free ID, or remove the existing resource there first." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo
|
||||
echo "Plan:"
|
||||
echo " source: ${kind} VMID ${vmid} on ${source_node} (storage: ${source_storage}, mode: ${mode})"
|
||||
echo " target: VMID ${new_vmid} on ${target_node} (storage: ${target_storage}," \
|
||||
"$([[ "$unique" -eq 1 ]] && echo "fresh MAC via --unique" || echo "original MAC preserved"))"
|
||||
echo " cutover: stop VMID ${vmid} on ${source_node}," \
|
||||
"$([[ "$start_target" -eq 1 ]] && echo "then start VMID ${new_vmid} on ${target_node}" || echo "target left stopped (--no-start)")"
|
||||
if [[ "$remove_source" -eq 1 ]]; then
|
||||
echo " after cutover: DESTROY VMID ${vmid} on ${source_node} (--remove-source, irreversible)"
|
||||
else
|
||||
echo " after cutover: source VMID ${vmid} left stopped (but intact) on ${source_node}"
|
||||
fi
|
||||
[[ "$keep_backup" -eq 1 ]] && echo " backup archives are kept on both nodes afterward (--keep-backup)"
|
||||
|
||||
if [[ "$dry_run" -eq 1 ]]; then
|
||||
echo
|
||||
echo "[dry-run] No backup, transfer, restore, cutover, destroy, or delete was performed."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if [[ "$skip_confirm" -ne 1 ]]; then
|
||||
echo
|
||||
if ! confirm_typed "$vmid" "Type the source VMID (${vmid}) to confirm backing it up from ${source_node} for migration: "; then
|
||||
echo "Cancelled -- input didn't match ${vmid}." >&2
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
# --- vzdump on the source node --------------------------------------------
|
||||
echo
|
||||
echo "==> Backing up VMID ${vmid} on ${source_node} (mode=${mode}, storage=${source_storage})..."
|
||||
vzdump_log="$(ssh "$source_target" \
|
||||
"vzdump ${vmid} --mode ${mode} --storage ${source_storage} --compress zstd" 2>&1)" \
|
||||
|| {
|
||||
echo "$vzdump_log" >&2
|
||||
echo "ERROR: vzdump failed on ${source_node}." >&2
|
||||
exit 1
|
||||
}
|
||||
echo "$vzdump_log"
|
||||
|
||||
archive="$(echo "$vzdump_log" | grep -oP "creating vzdump archive '\K[^']+" | tail -n1)"
|
||||
if [[ -z "$archive" ]]; then
|
||||
echo "ERROR: couldn't find the archive path in vzdump's output above." >&2
|
||||
exit 1
|
||||
fi
|
||||
archive_basename="$(basename "$archive")"
|
||||
target_tmp_archive="/var/tmp/${archive_basename}"
|
||||
echo "Archive: ${archive}"
|
||||
|
||||
# Always clean up the relayed copy on the target node, success or failure
|
||||
# -- it's only ever a working copy, restored or not.
|
||||
cleanup_target_tmp() {
|
||||
if [[ "$keep_backup" -ne 1 ]]; then
|
||||
ssh "$target_target" "rm -f '${target_tmp_archive}'" >/dev/null 2>&1 || true
|
||||
fi
|
||||
}
|
||||
trap cleanup_target_tmp EXIT
|
||||
|
||||
# --- relay the archive from source to target ------------------------------
|
||||
echo
|
||||
echo "==> Transferring archive to ${target_node}..."
|
||||
ssh "$source_target" "cat '${archive}'" | ssh "$target_target" "cat > '${target_tmp_archive}'"
|
||||
|
||||
# --- restore on the target node --------------------------------------------
|
||||
echo
|
||||
echo "==> Restoring as VMID ${new_vmid} on ${target_node} (storage=${target_storage})..."
|
||||
restore_unique_flag=0
|
||||
[[ "$unique" -eq 1 ]] && restore_unique_flag=1
|
||||
if [[ "$kind" == "vm" ]]; then
|
||||
ssh "$target_target" "qmrestore '${target_tmp_archive}' ${new_vmid} --storage ${target_storage} --unique ${restore_unique_flag}"
|
||||
else
|
||||
ssh "$target_target" "pct restore ${new_vmid} '${target_tmp_archive}' --storage ${target_storage} --unique ${restore_unique_flag}"
|
||||
fi
|
||||
echo "Restored on ${target_node}. Source VMID ${vmid} on ${source_node} is still up --" \
|
||||
"the guest hasn't moved yet."
|
||||
|
||||
# --- cutover: stop source, start target -----------------------------------
|
||||
if [[ "$skip_confirm" -ne 1 ]]; then
|
||||
echo
|
||||
if ! confirm_typed "$vmid" "Type the source VMID (${vmid}) again to confirm CUTOVER (stop it on ${source_node}$([[ "$start_target" -eq 1 ]] && echo ", start VMID ${new_vmid} on ${target_node}")): "; then
|
||||
echo "Cancelled before cutover -- input didn't match ${vmid}." >&2
|
||||
echo "The restored (but not started) copy remains on ${target_node} as VMID ${new_vmid};" >&2
|
||||
echo "the source on ${source_node} is untouched. Re-run cutover manually, or clean up" >&2
|
||||
echo "the target copy if you no longer want it." >&2
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
echo
|
||||
echo "==> Stopping VMID ${vmid} on ${source_node}..."
|
||||
if [[ "$kind" == "vm" ]]; then
|
||||
ssh "$source_target" "qm stop ${vmid}"
|
||||
else
|
||||
ssh "$source_target" "pct stop ${vmid}"
|
||||
fi
|
||||
|
||||
if [[ "$start_target" -eq 1 ]]; then
|
||||
echo
|
||||
echo "==> Starting VMID ${new_vmid} on ${target_node}..."
|
||||
if [[ "$kind" == "vm" ]]; then
|
||||
ssh "$target_target" "qm start ${new_vmid}"
|
||||
else
|
||||
ssh "$target_target" "pct start ${new_vmid}"
|
||||
fi
|
||||
fi
|
||||
|
||||
# --- optionally destroy the now-stopped source resource -------------------
|
||||
if [[ "$remove_source" -eq 1 ]]; then
|
||||
if [[ "$skip_confirm" -ne 1 ]]; then
|
||||
echo
|
||||
if ! confirm_typed "$vmid" "Type the source VMID (${vmid}) one more time to permanently DESTROY it on ${source_node}: "; then
|
||||
echo "Cancelled -- input didn't match ${vmid}." >&2
|
||||
echo "VMID ${vmid} on ${source_node} is left stopped (not destroyed). VMID ${new_vmid}" >&2
|
||||
echo "on ${target_node} is up and running." >&2
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
echo
|
||||
echo "==> Destroying VMID ${vmid} on ${source_node}..."
|
||||
if [[ "$kind" == "vm" ]]; then
|
||||
ssh "$source_target" "qm destroy ${vmid} --purge"
|
||||
else
|
||||
ssh "$source_target" "pct destroy ${vmid} --purge"
|
||||
fi
|
||||
fi
|
||||
|
||||
# --- clean up backup archives now that the migration succeeded -----------
|
||||
if [[ "$keep_backup" -ne 1 ]]; then
|
||||
echo
|
||||
echo "==> Deleting backup archive from ${source_node}'s ${source_storage} storage..."
|
||||
ssh "$source_target" "rm -f '${archive}' '${archive}.notes' '${archive}.log'" >/dev/null 2>&1 || true
|
||||
fi
|
||||
|
||||
echo
|
||||
echo "Done. VMID ${new_vmid} (${kind}) is now on ${target_node}, migrated from" \
|
||||
"VMID ${vmid} on ${source_node}."
|
||||
if [[ "$remove_source" -ne 1 ]]; then
|
||||
echo "The source copy is stopped but still present on ${source_node} -- re-run with" \
|
||||
"--remove-source once you've verified the target, or remove it manually."
|
||||
fi
|
||||
Reference in New Issue
Block a user