Fix ZFS root import stall on baremetal-gui reboot #41

2026-07-22T03:48:04Z

beatzaplenty commented

2026-07-22 03:48:04 +00:00

Two fixes for the auto-installer leaving baremetal-gui's ZFS root pool unable to import on its first real boot:

The actual root cause: nixos-install bind-mounts /dev, /proc, /sys (and usually /run) into /mnt to run the target's activation script in a chroot, and doesn't unmount them again afterward. Left in place, those nested mounts made ZFS refuse to unmount its own root dataset at /mnt -- confirmed live: zpool export -a failed with cannot unmount '/mnt': pool or dataset busy, and because auto-install.sh runs under set -e, that killed the script before it ever reached reboot. So the pool was never actually exported, silently defeating the export-before-reboot fix from #40 on every real run -- which is why the ZFS-import stall kept recurring even after that PR merged. Fixed by unmounting those leftover chroot mounts before attempting the export.
Debuggability follow-up: the systemd-based initrd locks the root account by default, so when the ZFS import unit did fail and dropped to emergency mode, sulogin refused a shell (looped re-entering the target instead of prompting) -- there was no way to see the real error without rebooting into separate installer media. Adds boot.initrd.systemd.emergencyAccess = true; to modules/platforms/baremetal.nix (scoped to this host only -- not promoted to common config, since it trades away console-lockout protection and the other hosts (linode-*/proxmox-*/server) don't have the same boot fragility) so the next occurrence is debuggable directly on the real boot.

Both verified with bash -n and scripts/codex-maintenance.sh (clean, only pre-existing documented secret-grep exceptions).

Two fixes for the auto-installer leaving baremetal-gui's ZFS root pool unable to import on its first real boot: 1. **The actual root cause:** `nixos-install` bind-mounts `/dev`, `/proc`, `/sys` (and usually `/run`) into `/mnt` to run the target's activation script in a chroot, and doesn't unmount them again afterward. Left in place, those nested mounts made ZFS refuse to unmount its own root dataset at `/mnt` -- confirmed live: `zpool export -a` failed with `cannot unmount '/mnt': pool or dataset busy`, and because `auto-install.sh` runs under `set -e`, that killed the script before it ever reached `reboot`. So the pool was never actually exported, silently defeating the export-before-reboot fix from #40 on every real run -- which is why the ZFS-import stall kept recurring even after that PR merged. Fixed by unmounting those leftover chroot mounts before attempting the export. 2. **Debuggability follow-up:** the systemd-based initrd locks the root account by default, so when the ZFS import unit *did* fail and dropped to emergency mode, `sulogin` refused a shell (looped re-entering the target instead of prompting) -- there was no way to see the real error without rebooting into separate installer media. Adds `boot.initrd.systemd.emergencyAccess = true;` to `modules/platforms/baremetal.nix` (scoped to this host only -- not promoted to common config, since it trades away console-lockout protection and the other hosts (`linode-*`/`proxmox-*`/`server`) don't have the same boot fragility) so the next occurrence is debuggable directly on the real boot. Both verified with `bash -n` and `scripts/codex-maintenance.sh` (clean, only pre-existing documented secret-grep exceptions).

beatzaplenty added 1 commit 2026-07-22 03:48:04 +00:00

Allow initrd emergency shell access on baremetal-gui

Check NixOS configurations / eval-hosts (pull_request) Successful in 10m45s

Details

5d7a6327b7

The systemd-based initrd (default here, since this host has a ZFS
root) locks the root account by default, so sulogin refuses a shell
if something in the initrd fails and it drops to emergency mode --
confirmed live: it just loops re-entering the target instead of
prompting, making an initrd-level ZFS import failure impossible to
diagnose from the console. Only affects the pre-switch-root initrd
shell, not the installed system's own login.

beatzaplenty merged commit 1f8bf8c852 into main

2026-07-22 03:48:48 +00:00

beatzaplenty changed title from ~~Allow initrd emergency shell access on baremetal-gui~~ to Fix ZFS root import stall on baremetal-gui reboot

2026-07-22 04:07:48 +00:00

beatzaplenty referenced this pull request

2026-07-22 04:09:49 +00:00

Unmount leftover chroot bind mounts before zpool export #42

This repo is archived. You cannot comment on pull requests.

1 Participants

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: beatzaplenty/nixos#41