From 120240f14a7e6cbe69c417af0a611b63c3926713 Mon Sep 17 00:00:00 2001 From: beatzaplenty Date: Mon, 20 Jul 2026 05:58:56 +1000 Subject: [PATCH] Fix LXC deployment path and clean up remaining eval warnings MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit LXC hosts (device busy fix): modules/platforms/lxc.nix now imports nixpkgs' own virtualisation/proxmox-lxc.nix, giving every lxc-* host a real config.system.build.tarball output — a directly `pct restore`-able Proxmox container image. This is the actual bug fix behind the "cannot remove real root directory: device busy or in use" error: lxc-* targets were only reachable through nixos-install, which bind-mounts / onto /mnt for containers (no raw disk to partition) and then correctly refuses to modify the filesystem it's currently running on. auto-install.sh's menu now excludes lxc-* targets entirely (they deploy via nix build + pct restore instead, see docs/auto-installer.md) — and, on the same reasoning, also excludes `installer`/`proxmox-lxc`, which are the installer image's own flake targets, not deployable hosts. manageHostName = true keeps host.nix's declared hostnames (upstream's default would let Proxmox's ambient container config win instead); privileged = false matches how these containers are actually created. Eval warnings, now zero across all 19 nixosConfigurations + 4 packages: - Multiple password options (root/nixos in the installer): nixpkgs' own installer profile sets initialHashedPassword = "" for passwordless login, conflicting with our explicit hashedPassword. Force-nulled the upstream option rather than adopting passwordless login, since this image now also boots over LAN PXE with PasswordAuthentication enabled. - boot.zfs.forceImportRoot default value: set explicitly to false (matching the two places that already did) in modules/common/configuration.nix and modules/installer/common.nix, covering every host and the installer alike. - Deprecated pkgs.system in modules/build-types/gui.nix: switched to pkgs.stdenv.hostPlatform.system. All confirmed non-behavioral where it matters: unrelated hosts' drvPaths are byte-identical to their pre-existing baselines throughout. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01La55Nsss8jZ7ZuzUV9mfot --- docs/auto-installer.md | 32 +++++++++++++++++ modules/build-types/gui.nix | 2 +- modules/common/configuration.nix | 6 ++++ modules/installer/common.nix | 37 +++++++++++++++++++- modules/platforms/lxc.nix | 60 +++++++++++++++++--------------- 5 files changed, 106 insertions(+), 31 deletions(-) diff --git a/docs/auto-installer.md b/docs/auto-installer.md index 3f0080e..d3f8022 100644 --- a/docs/auto-installer.md +++ b/docs/auto-installer.md @@ -12,6 +12,38 @@ discovers available hosts from this same flake, lets the operator choose a target, applies that host's Disko storage configuration, installs NixOS, and reboots. +**This applies to every `nixosConfigurations` target except `lxc-*` hosts — +see "LXC hosts" immediately below for why those are different.** + +## LXC hosts + +`lxc-*` targets (`lxc-minimal`, `lxc-nix-cache`, `lxc-server`, `lxc-docker`, +`lxc-gui`, `lxc-pxe-boot`) are **not** installed via `auto-install.sh` — the +interactive menu deliberately excludes them. Don't try to select one there; +`nixos-install` would bind-mount `/` onto `/mnt` (LXC containers have no raw +disk to partition) and then refuse to touch the filesystem it's currently +running on — it's designed to protect exactly this case, so it just fails. + +`modules/platforms/lxc.nix` imports nixpkgs' own +`virtualisation/proxmox-lxc.nix` module, which gives every `lxc-*` host a +`config.system.build.tarball` output — a complete, directly Proxmox-importable +container image, no install step at all: + +```sh +nix build .#nixosConfigurations.lxc-minimal.config.system.build.tarball +``` + +Then, on the Proxmox host, `pct restore` (or the GUI's "Create CT" → upload +template flow) that tarball directly as a new container. First boot runs +`boot.postBootCommands` (registers the Nix store DB and system profile) — +there's no separate activation step to run yourself. + +Host keys still need pre-seeding the same way as any other host (see "Host +keys" below) — the sops-nix activation-vs-first-boot race is identical +regardless of how the image reaches the machine. `NIXOS_HOST_KEYS_DIR=... +nix build ... --impure` bakes the matching key into the tarball the same way +it does for the ISO/PXE installer images. + ## Layout - `modules/installer/common.nix` — shared by every installer target: SSH diff --git a/modules/build-types/gui.nix b/modules/build-types/gui.nix index 37007d0..ba20255 100644 --- a/modules/build-types/gui.nix +++ b/modules/build-types/gui.nix @@ -2,7 +2,7 @@ { environment.systemPackages = with pkgs; [ - inputs.nixos-conf-editor.packages.${pkgs.system}.nixos-conf-editor + inputs.nixos-conf-editor.packages.${pkgs.stdenv.hostPlatform.system}.nixos-conf-editor nodejs appimage-run seahorse diff --git a/modules/common/configuration.nix b/modules/common/configuration.nix index 316255a..716a848 100644 --- a/modules/common/configuration.nix +++ b/modules/common/configuration.nix @@ -13,6 +13,12 @@ networking.networkmanager.enable = true; # Easiest to use and most distros use this by default. + # Recommended over the true default (bypasses ZFS's own import safeguards) + # per the option's own docs; matches hosts/docker/host.nix and + # modules/services/zfs/enable-service.nix, which already set this + # explicitly. Harmless no-op on hosts that don't use ZFS at all. + boot.zfs.forceImportRoot = false; + # Set your time zone. time.timeZone = vars.timeZone; diff --git a/modules/installer/common.nix b/modules/installer/common.nix index 8c58dc1..6ad2626 100644 --- a/modules/installer/common.nix +++ b/modules/installer/common.nix @@ -7,6 +7,12 @@ networking.useDHCP = lib.mkDefault true; + # Recommended over the true default (bypasses ZFS's own import safeguards) + # per the option's own docs. This installer environment has no ZFS pools + # of its own to import, so this is a no-op here — just silences the + # eval-time warning, matching modules/common/configuration.nix. + boot.zfs.forceImportRoot = false; + time.timeZone = vars.timeZone; # Without this, the installer only ever sees cache.nixos.org, which @@ -50,11 +56,26 @@ export FLAKE_BASE_URL="git+https://${vars.lanDomain}/beatzaplenty/nixos.git" echo "Fetching available NixOS hosts from flake..." + # Two categories deliberately excluded from the menu: + # lxc-* — these build a config.system.build.tarball + # meant for `pct restore` on Proxmox + # directly, not an install. Running + # nixos-install against one here would + # bind-mount / onto /mnt and then refuse to + # touch the filesystem it's currently + # running on — see docs/auto-installer.md. + # installer/proxmox-lxc — these *are* the installer image's own + # flake targets, not deployable hosts; + # "installing" one means nixos-install-ing + # a copy of the installer into itself. mapfile -t options < <( nix eval --json --no-use-registries --no-accept-flake-config --extra-experimental-features "flakes nix-command" \ "''${FLAKE_BASE_URL}#nixosConfigurations" \ --apply builtins.attrNames \ - | jq -r '.[]' + | jq -r '.[] + | select(startswith("lxc-") | not) + | select(. != "installer") + | select(. != "proxmox-lxc")' ) if [[ ''${#options[@]} -eq 0 ]]; then @@ -62,6 +83,10 @@ exit 1 fi + echo "Note: lxc-* targets aren't installed this way — build them with" + echo " nix build .#nixosConfigurations..config.system.build.tarball" + echo "and 'pct restore' the result on Proxmox directly. See docs/auto-installer.md." + echo "Choose the flake profile to install:" select choice in "''${options[@]}"; do if [[ -n "$choice" ]]; then @@ -190,9 +215,18 @@ PasswordAuthentication = true; }; + # nixpkgs' own installer profile (profiles/installation-device.nix, pulled + # in via installation-cd-minimal.nix) sets initialHashedPassword = "" for + # both users — its own passwordless-login convention for install media. + # That's a second, non-null password option alongside our hashedPassword + # below, which NixOS warns about as ambiguous precedence. Force it null + # rather than adopting passwordless login: this image now also boots over + # LAN PXE with PasswordAuthentication enabled, so passwordless root SSH + # would be reachable by anyone on the LAN, not just local console. users.users.root = { hashedPassword = "$6$Kwv9KAyvcurAViQF$H4.u3feqGE7lVoNgkFXhE3n2Pmo//9JYDTCz8ifrVHBxPjwa1xMby7tEZ8Bpt5MXs9Rkx6/YbZWxs5CpH0s/70"; + initialHashedPassword = lib.mkForce null; }; users.users.${vars.primaryUser} = { @@ -206,6 +240,7 @@ hashedPassword = "$6$Kwv9KAyvcurAViQF$H4.u3feqGE7lVoNgkFXhE3n2Pmo//9JYDTCz8ifrVHBxPjwa1xMby7tEZ8Bpt5MXs9Rkx6/YbZWxs5CpH0s/70"; + initialHashedPassword = lib.mkForce null; openssh.authorizedKeys.keys = [ vars.adminSshKey diff --git a/modules/platforms/lxc.nix b/modules/platforms/lxc.nix index a596244..31d6fe7 100644 --- a/modules/platforms/lxc.nix +++ b/modules/platforms/lxc.nix @@ -1,36 +1,38 @@ -{ lib, ... }: +{ lib, modulesPath, ... }: { - boot = { - isContainer = true; + # LXC containers share the host kernel — Proxmox starts them by exec'ing + # /sbin/init directly, no bootloader/initrd involved — and Proxmox has its + # own container hostname/network provisioning outside Nix. nixpkgs' own + # virtualisation/proxmox-lxc.nix module already handles all of this + # correctly (boot.isContainer, loader.initScript, systemd-networkd) and, + # critically, provides config.system.build.tarball — a directly + # `pct restore`-able container image, no nixos-install/bind-mount needed + # (nixos-install refuses to touch the filesystem it's currently running + # on, which is exactly what bind-mounting / onto /mnt for an installer + # LXC container does). + imports = [ + (modulesPath + "/virtualisation/proxmox-lxc.nix") + ]; - loader = { - grub.enable = false; - systemd-boot.enable = false; - - # LXC containers share the host kernel — Proxmox starts them by exec'ing - # /sbin/init directly, no bootloader/initrd involved. Without this, that - # file doesn't correctly launch the current generation, so even a - # correctly-installed system can fail to come up after reboot. - initScript.enable = true; - }; + proxmoxLXC = { + # host.nix declares each host's real hostname (networking.hostName); + # keep that instead of letting Proxmox's ambient container config win. + manageHostName = true; + # Unprivileged matches how these containers are actually created. + privileged = false; }; - networking = { - # boot.isContainer disables services.udev (see nixpkgs' - # virtualisation/container-config.nix), and NetworkManager depends on a - # running udevd to enumerate/classify devices. That leaves NM unable to - # reliably manage the container's veth interface, which is what broke - # DHCP-hostname registration in Pi-hole. systemd-networkd talks to the - # kernel over rtnetlink directly and doesn't have that dependency. - networkmanager.enable = lib.mkForce false; - useNetworkd = true; - - # container-config.nix also defaults this to true, which assumes a - # systemd-nspawn-style host bind-mount of /etc/resolv.conf. Real Proxmox - # LXC doesn't do that (nixpkgs' own virtualisation/proxmox-lxc.nix module - # forces this false for the same reason), so leaving it true silently - # breaks DNS instead of falling back to networkd/DHCP-provided servers. - useHostResolvConf = lib.mkForce false; + boot.loader = { + grub.enable = false; + systemd-boot.enable = false; }; + + # NetworkManager depends on a running udevd to enumerate/classify devices, + # which boot.isContainer disables (see nixpkgs' container-config.nix) — + # that's what broke DHCP-hostname registration in Pi-hole. The imported + # proxmox-lxc.nix module already switches networking to systemd-networkd + # for the same reason; it just doesn't disable NetworkManager itself, + # which modules/common/configuration.nix enables for every host. + networking.networkmanager.enable = lib.mkForce false; }