virtualisation/proxmox-lxc.nix registers the Nix store DB via a systemd
service, never an activation script -- so neededForUsers sops secrets
(password hashes) and the user-creation step that consumes them never ran
on a real first boot, leaving /etc/shadow stuck with build-time placeholder
entries. boot.postBootCommands looked like the right hook (stage-2-init.sh
does invoke it) but switch-to-configuration behaves unreliably that early,
before systemd itself is up. Fixed with a genuine oneshot systemd service,
gated by ConditionPathExists so it only ever runs once.
Confirmed live via a from-scratch destroy+rebuild+redeploy of the
lxc-nix-cache test container: real password hashes applied automatically,
systemctl is-system-running -> running, zero failed units.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01La55Nsss8jZ7ZuzUV9mfot
The real root cause behind the original nix-cache 502, traced all the way
through: modules/installer/host-keys.nix (which NIXOS_HOST_KEYS_DIR=...
--impure actually wires up) is only ever imported by the installer's own
modules/installer/common.nix -- modules/platforms/lxc.nix, which every
real lxc-* host build actually uses, never imported anything like it.
docs/auto-installer.md previously claimed NIXOS_HOST_KEYS_DIR bakes a key
into lxc-* tarballs "the same way it does for the ISO/PXE installer
images" -- that was never actually true; I wrote it without verifying the
mechanism existed for lxc.nix specifically.
In practice this meant every lxc-* container booted with a freshly
self-generated SSH host key that could never match whatever .sops.yaml
actually trusts for that target, so *every* secret -- not just
cache-priv-key -- silently failed to decrypt. No error surfaces in the
boot log for this: the activation step that installs secrets only runs
on a genuinely fresh first activation and silently no-ops once
/run/current-system already exists, so by the time anyone looks the
window has closed. Found by manually invoking sops-install-secrets
directly: "Error getting data key: 0 successful groups required, got 0".
Fixed by giving modules/platforms/lxc.nix the same key-baking mechanism
the installer has, but keyed to its own exact flake target and placing
the key directly at /etc/ssh/ssh_host_ed25519_key (no copy step to stage
for, unlike the installer's /etc/host-keys/ staging area -- an lxc-*
tarball has no install step). The target name comes in via
specialArgs.flakeTarget (new, set by flake.nix's mkTarget) rather than
being read back from config.environment.etc."flake-target" -- reading
that back from within a module that also contributes to
environment.etc is circular (confirmed: "infinite recursion
encountered").
Verified live end-to-end against the real test container (lxc-nix-cache,
VMID 100 on pve.sweet.home): destroyed it, rebuilt the tarball fresh with
the fix, recreated it, and confirmed /run/secrets/ now has all three
secrets this host needs (beszel-token, cache-priv-key, nix-github-token),
nix-serve is active (running), and curl http://localhost/nix-cache-info
succeeds both directly and through nginx.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01La55Nsss8jZ7ZuzUV9mfot
LXC hosts (device busy fix):
modules/platforms/lxc.nix now imports nixpkgs' own
virtualisation/proxmox-lxc.nix, giving every lxc-* host a real
config.system.build.tarball output — a directly `pct restore`-able
Proxmox container image. This is the actual bug fix behind the
"cannot remove real root directory: device busy or in use" error:
lxc-* targets were only reachable through nixos-install, which
bind-mounts / onto /mnt for containers (no raw disk to partition)
and then correctly refuses to modify the filesystem it's currently
running on. auto-install.sh's menu now excludes lxc-* targets
entirely (they deploy via nix build + pct restore instead, see
docs/auto-installer.md) — and, on the same reasoning, also excludes
`installer`/`proxmox-lxc`, which are the installer image's own flake
targets, not deployable hosts.
manageHostName = true keeps host.nix's declared hostnames (upstream's
default would let Proxmox's ambient container config win instead);
privileged = false matches how these containers are actually created.
Eval warnings, now zero across all 19 nixosConfigurations + 4 packages:
- Multiple password options (root/nixos in the installer): nixpkgs'
own installer profile sets initialHashedPassword = "" for
passwordless login, conflicting with our explicit hashedPassword.
Force-nulled the upstream option rather than adopting passwordless
login, since this image now also boots over LAN PXE with
PasswordAuthentication enabled.
- boot.zfs.forceImportRoot default value: set explicitly to false
(matching the two places that already did) in
modules/common/configuration.nix and modules/installer/common.nix,
covering every host and the installer alike.
- Deprecated pkgs.system in modules/build-types/gui.nix: switched to
pkgs.stdenv.hostPlatform.system.
All confirmed non-behavioral where it matters: unrelated hosts'
drvPaths are byte-identical to their pre-existing baselines throughout.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01La55Nsss8jZ7ZuzUV9mfot
Zero W20 (repeated attribute keys), W10 (empty { ... }: variadic
pattern, use _: instead), and W04 (a = x.a instead of inherit)
warnings remain anywhere in the tree, and nixpkgs-fmt --check is
clean on all 46 .nix files.
Repeated-key merges go as deep as statix actually flags per file
(e.g. boot.loader.* nested under boot.loader = { ... } once the
outer boot.* merge exposed it as its own repeat) — every merge is a
pure attribute-path restructuring with no value changes, verified by
comparing config.system.build.toplevel.drvPath before/after for a
representative host per changed module plus a full 19-host + 4-package
eval sweep.
One indentation slip caught and fixed during this pass: nesting
modules/installer/common.nix's environment.etc."auto-install.sh".text
under an environment = { ... } block initially normalized the
script's shebang/set line indentation, which actually changes the
rendered file (Nix's '' string dedent treats it as real content, not
cosmetic whitespace) — reproduced the original's exact indentation
and reverified the rendered script is byte-identical to before.
modules/services/zfs/auto-mount-volumes.nix picked up formatting too;
worth noting it isn't imported by anything in this flake at all.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01La55Nsss8jZ7ZuzUV9mfot
boot.isContainer disables services.udev, which NetworkManager depends on
to enumerate devices — this left NM unable to reliably manage the
container veth, breaking DHCP-hostname registration in Pi-hole. It also
defaulted networking.useHostResolvConf to true, which assumes a
systemd-nspawn-style resolv.conf bind-mount that real Proxmox LXC doesn't
provide (nixpkgs' own proxmox-lxc.nix module forces this false for the
same reason). Also guard the networkmanager extraGroups membership in the
minimal/gui build-types, since that group only exists when NM is enabled.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01La55Nsss8jZ7ZuzUV9mfot
LXC containers share the host kernel — Proxmox starts them by exec'ing
/sbin/init directly in the container's rootfs, no bootloader or initrd
involved at all. Without boot.loader.initScript.enable, that file
isn't wired to launch the current generation, so even a correctly
installed system (see the nix-auto-installer bind-mount fix, same
underlying issue) could still fail to come up after reboot. This is
exactly what nixpkgs' own virtualisation/proxmox-lxc.nix module sets
for the same reason.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Generates all nixosConfigurations from mkTarget(platform, buildType,
hostPath) instead of hand-written per-host blocks, so adding a new
platform or build type is a one-line addition. Per-machine identity
(hostname, hostId, secrets, stateVersion) moves into hosts/<name>/host.nix;
platform-specific config (hardware, boot, networking) into
modules/platforms/*.nix; build-type config (minimal/server/docker/gui/
nix-cache/pxe-boot) into modules/build-types/*.nix.
Old flat targets (nixos, docker, server, nix-cache, nix-minimal, pxe-boot)
are replaced by the 17-target <platform>-<buildtype> matrix; each new
target was verified to evaluate before its old counterpart was removed.
CI workflows and docs/aliases now discover hosts dynamically via
nixosConfigurations attrNames and /etc/flake-target instead of hardcoded
lists, so they can't drift from flake.nix again.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
beatzaplentyandClaude Sonnet 5