Archived
beatzaplentyandClaude Sonnet 5
a5990ccf7d
Check NixOS configurations / eval-hosts (pull_request) Failing after 11m39s
create-proxmox-resource.sh no longer builds locally and scp's a multi-gigabyte image over -- it now clones/pulls this repo onto the Proxmox node itself (bootstrapping build tooling via the existing codex-setup.sh on first use) and runs the nix build / disko image script there, staging the result straight into the node's own import directory. host-keys/ (gitignored) is copied over separately since a git pull doesn't carry it. --image still uploads an explicit local file for the case where you don't want a build at all. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
117 lines
4.4 KiB
Markdown
117 lines
4.4 KiB
Markdown
# Proxmox VM disk images
|
|
|
|
`proxmox-*` hosts (VM platform, not `lxc-*`) can be built as standalone,
|
|
ready-to-attach `.raw` disk images via disko's own image-builder — no
|
|
`nixos-install`, no live installer boot. This uses the same `disko.devices`
|
|
config (`modules/disko/proxmox.nix`) already used to format a real disk on
|
|
install, so there's nothing host-specific to write; it's available for every
|
|
`proxmox-*` target automatically.
|
|
|
|
`scripts/create-proxmox-resource.sh --type vm --host <name>` automates the
|
|
whole walkthrough below (and the equivalent LXC one) end to end, including
|
|
host-key handling and building the image directly on the Proxmox node
|
|
itself (no local build, no image transfer) — see its `--help`. The steps
|
|
here are what it runs under the hood, useful for doing any of it by hand
|
|
or understanding what it does before you trust it against real
|
|
infrastructure.
|
|
|
|
## Building
|
|
|
|
```sh
|
|
nix build .#nixosConfigurations.proxmox-server.config.system.build.diskoImagesScript
|
|
sudo ./result --build-memory 2048
|
|
```
|
|
|
|
This produces `<hostname>.raw` in the current directory (e.g. `server.raw`
|
|
for `proxmox-server`, matching `networking.hostName`, not the flake attribute
|
|
name — every `proxmox-*` host gets a distinctly named image instead of all
|
|
of them producing an identical `main.raw`). The script builds inside a
|
|
temporary QEMU VM and moves the finished image out to the working directory
|
|
when done; `--build-memory` controls how much RAM that build VM gets.
|
|
|
|
`disko.devices.disk.main.imageSize` (currently `20G`, in
|
|
`modules/disko/proxmox.nix`) sets the image's total size — disko doesn't
|
|
support auto-resizing, so this needs to comfortably fit ESP + swap + root at
|
|
build time. Grow the virtual disk (and resize the filesystem) in Proxmox
|
|
after attaching if a host needs more than that; this is the normal way to
|
|
size these images, not a one-time decision to get exactly right up front.
|
|
|
|
## Host keys
|
|
|
|
The disko image script runs a real activation pass inside its temporary
|
|
build VM while constructing the image — the same sops-nix
|
|
activation-before-first-boot problem the installer and LXC tarball workflows
|
|
have (see `docs/auto-installer.md`) applies here too, unmodified. Disko has
|
|
a native mechanism for it:
|
|
|
|
```sh
|
|
sudo ./result \
|
|
--pre-format-files host-keys/server_ssh_host_ed25519_key /etc/ssh/ssh_host_ed25519_key \
|
|
--pre-format-files host-keys/server_ssh_host_ed25519_key.pub /etc/ssh/ssh_host_ed25519_key.pub \
|
|
--build-memory 2048
|
|
```
|
|
|
|
Generate the key first with `scripts/sync-host-keys.sh <hostname>`, same
|
|
as any other host — see `docs/auto-installer.md` for the full walkthrough
|
|
(it registers the new key in `.sops.yaml` and re-encrypts the affected
|
|
`secrets/*.yaml` files too, no manual editing needed).
|
|
|
|
## Deploying to Proxmox
|
|
|
|
The image needs **UEFI (OVMF)**, not Proxmox's default SeaBIOS —
|
|
`modules/boot/efi.nix` uses `systemd-boot`, which only works with UEFI
|
|
firmware. `virtio-scsi` is safe to use as the disk bus:
|
|
`hardware-configuration/vm/proxmox.nix` already includes `virtio_scsi` in
|
|
its initrd kernel modules.
|
|
|
|
1. Copy the image to the Proxmox host:
|
|
|
|
```sh
|
|
scp server.raw root@<proxmox-host>:/var/lib/vz/import/
|
|
```
|
|
|
|
2. Create an empty VM shell (no disk yet) — replace `<vmid>` with a free ID
|
|
and `<storage>` with your storage pool's name (`pvesm status` or
|
|
Datacenter → Storage in the web UI):
|
|
|
|
```sh
|
|
qm create <vmid> --name proxmox-server --memory 2048 --cores 2 \
|
|
--net0 virtio,bridge=vmbr0 \
|
|
--bios ovmf --machine q35 \
|
|
--scsihw virtio-scsi-pci \
|
|
--efidisk0 <storage>:1,efitype=4m,pre-enrolled-keys=0
|
|
```
|
|
|
|
(`--efidisk0` is required for UEFI — it's where OVMF persists boot-entry
|
|
NVRAM; without it, systemd-boot's boot entry may not survive a reboot.)
|
|
|
|
3. Import the raw disk into storage:
|
|
|
|
```sh
|
|
qm importdisk <vmid> /var/lib/vz/import/server.raw <storage>
|
|
```
|
|
|
|
This prints the resulting disk identifier (e.g. `vm-<vmid>-disk-1`).
|
|
|
|
4. Attach it and set it as the boot disk:
|
|
|
|
```sh
|
|
qm set <vmid> --scsi0 <storage>:vm-<vmid>-disk-1
|
|
qm set <vmid> --boot order=scsi0
|
|
```
|
|
|
|
5. Boot it:
|
|
|
|
```sh
|
|
qm start <vmid>
|
|
```
|
|
|
|
No install step — it boots straight into the already-activated system.
|
|
|
|
## Why not `nix build .#nixosConfigurations.<host>.config.system.build.vm`?
|
|
|
|
That's a different, unrelated feature — `system.build.vm` (`nixos-rebuild
|
|
build-vm`) produces an ephemeral QEMU script for locally testing a
|
|
configuration, not a distributable disk image. It's not part of this
|
|
workflow.
|