Archived
Fix LXC container creation: unprivileged, nesting/keyctl, swap sizing
Found and fixed live against a real test container (VMID 100, lxc-nix-cache on pve.sweet.home) after the previous pct-restore-to-pct-create fix still produced a container that booted into garbled console output: 1. pct create's own CLI default for --unprivileged is privileged (unlike the web UI, whose checkbox defaults the other way), but modules/platforms/lxc.nix sets proxmoxLXC.privileged = false, so the image assumes it's running unprivileged. Real mismatch -- now passes --unprivileged 1 explicitly. 2. The actual root cause of the garbled console: modern (v247+) systemd routinely uses nested user namespaces and credential mounts (even plain getty units, via LoadCredential=-style mechanisms), which AppArmor's default LXC confinement denies without --features nesting=1,keyctl=1. Confirmed via the host's kernel audit log: every getty unit was crash-looping on a denied /run/credentials/* mount every ~3s, and core services like nsncd failed userns_create the same way -- the system never finished activating. Fixed live (pct set + restart on the running test container) before committing the script change: systemctl is-system-running went from never completing to "running" with zero failed units. 3. --memory doesn't touch swap -- confirmed live it silently stayed at Proxmox's own 512M default with --memory 2048. Now defaults --swap to whatever --memory resolves to. docs/auto-installer.md's manual pct create walkthrough gets the same fixes, with the "why" for each flag, since a human following it by hand would hit the identical bugs. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01La55Nsss8jZ7ZuzUV9mfot
This commit is contained in:
+24
-2
@@ -45,15 +45,37 @@ of its own:
|
||||
|
||||
```sh
|
||||
pct create <vmid> local:vztmpl/<file>.tar.xz \
|
||||
--rootfs local-lvm:8 --hostname <name> --cores 2 --memory 2048 \
|
||||
--unprivileged 1 --features nesting=1,keyctl=1 \
|
||||
--rootfs local-lvm:8 --hostname <name> --cores 2 --memory 2048 --swap 2048 \
|
||||
--net0 name=eth0,bridge=vmbr0,ip=dhcp
|
||||
pct start <vmid>
|
||||
```
|
||||
|
||||
Every one of those extra flags is load-bearing, confirmed by actually
|
||||
booting one:
|
||||
|
||||
- `--unprivileged 1` — `modules/platforms/lxc.nix` sets
|
||||
`proxmoxLXC.privileged = false`, so the image assumes it's running
|
||||
unprivileged. `pct create`'s own CLI default for this flag is
|
||||
privileged (unlike the web UI, whose checkbox defaults the other way)
|
||||
— omit it and you get a privileged container running a NixOS config
|
||||
that assumes unprivileged, a real mismatch.
|
||||
- `--features nesting=1,keyctl=1` — required for a modern (v247+)
|
||||
systemd guest to boot unprivileged at all. Without it, AppArmor denies
|
||||
the nested user namespaces and credential mounts systemd routinely
|
||||
uses (even plain getty units) — every getty crash-loops on a denied
|
||||
`/run/credentials/*` mount every ~3s (this is what garbage on the
|
||||
console turns out to be) while core services like `nsncd` fail the
|
||||
same way, and the system never finishes activating.
|
||||
- `--swap 2048` — `--memory` doesn't touch swap; it silently stays at
|
||||
Proxmox's own 512M default otherwise. Match it to `--memory` unless
|
||||
you deliberately want otherwise.
|
||||
|
||||
First boot runs `boot.postBootCommands` (registers the Nix store DB and
|
||||
system profile) — there's no separate activation step to run yourself.
|
||||
`scripts/create-proxmox-resource.sh --type lxc --host <name>` automates all
|
||||
of this (build, host-key handling, upload, `pct create`) — see its `--help`.
|
||||
of this (build, host-key handling, upload, `pct create` with the flags
|
||||
above) — see its `--help`.
|
||||
|
||||
Host keys still need pre-seeding the same way as any other host (see "Host
|
||||
keys" below) — the sops-nix activation-vs-first-boot race is identical
|
||||
|
||||
@@ -73,6 +73,10 @@ Shared:
|
||||
modify: omit to leave unchanged.
|
||||
--memory <MB> create: default \$PROXMOX_DEFAULT_MEMORY_MB (${PROXMOX_DEFAULT_MEMORY_MB}).
|
||||
modify: omit to leave unchanged.
|
||||
--swap <MB> lxc only, create time: \`--memory\` doesn't
|
||||
touch swap -- it silently stays at Proxmox's
|
||||
own 512M default otherwise. (default: matches
|
||||
whatever --memory resolves to)
|
||||
--storage <pool> (default: \$PROXMOX_STORAGE, ${PROXMOX_STORAGE})
|
||||
--iso-storage <pool> (default: \$PROXMOX_ISO_STORAGE, ${PROXMOX_ISO_STORAGE})
|
||||
--bridge <bridge> (default: \$PROXMOX_BRIDGE, ${PROXMOX_BRIDGE})
|
||||
@@ -95,6 +99,7 @@ name=""
|
||||
vmid=""
|
||||
cores=""
|
||||
memory=""
|
||||
swap=""
|
||||
disk_size=""
|
||||
grow_disk=""
|
||||
image=""
|
||||
@@ -114,6 +119,7 @@ while [[ $# -gt 0 ]]; do
|
||||
--vmid) vmid="$2"; shift 2 ;;
|
||||
--cores) cores="$2"; shift 2 ;;
|
||||
--memory) memory="$2"; shift 2 ;;
|
||||
--swap) swap="$2"; shift 2 ;;
|
||||
--disk-size) disk_size="$2"; shift 2 ;;
|
||||
--grow-disk) grow_disk="$2"; shift 2 ;;
|
||||
--image) image="$2"; shift 2 ;;
|
||||
@@ -436,7 +442,25 @@ echo
|
||||
if [[ "$type" == "lxc" ]]; then
|
||||
echo "==> Creating LXC container ${vmid} (${name})..."
|
||||
local_disk_size="${disk_size:-$PROXMOX_DEFAULT_LXC_DISK_GB}"
|
||||
create_cmd="pct create ${vmid} ${iso_storage}:vztmpl/${remote_filename} --rootfs ${storage}:${local_disk_size} --hostname ${name} --cores ${cores} --memory ${memory} --net0 name=eth0,bridge=${bridge},ip=dhcp"
|
||||
# --memory doesn't touch swap -- it silently stays at Proxmox's own
|
||||
# 512M default otherwise (confirmed live: --memory 2048 left swap at
|
||||
# 512). Default to matching whatever --memory resolved to above.
|
||||
local_swap="${swap:-$memory}"
|
||||
# --unprivileged 1: modules/platforms/lxc.nix sets proxmoxLXC.privileged
|
||||
# = false, so the NixOS config inside the image assumes it's running as
|
||||
# an unprivileged container (cgroup/capability/mount expectations baked
|
||||
# in at boot). `pct create`'s own CLI default for this flag is
|
||||
# privileged (unlike the web UI, which defaults its checkbox the other
|
||||
# way) -- leaving it unset creates a privileged container running a
|
||||
# NixOS config that assumes unprivileged, a real mismatch.
|
||||
#
|
||||
# --features nesting=1,keyctl=1: required for a modern (v247+) systemd
|
||||
# guest to actually boot unprivileged -- confirmed live: without this,
|
||||
# AppArmor denies the nested user namespaces and credential mounts
|
||||
# systemd routinely uses (even plain getty units), and every getty
|
||||
# crash-loops on a denied mount every ~3s (visible as garbage on the
|
||||
# console) while core services like nsncd fail the same way.
|
||||
create_cmd="pct create ${vmid} ${iso_storage}:vztmpl/${remote_filename} --unprivileged 1 --features ${PROXMOX_DEFAULT_LXC_FEATURES} --rootfs ${storage}:${local_disk_size} --hostname ${name} --cores ${cores} --memory ${memory} --swap ${local_swap} --net0 name=eth0,bridge=${bridge},ip=dhcp"
|
||||
remote "$create_cmd"
|
||||
remote "pct start ${vmid}"
|
||||
else
|
||||
|
||||
+19
-1
@@ -30,8 +30,26 @@
|
||||
# no backup metadata to infer it from. Matches Proxmox's own GUI default.
|
||||
: "${PROXMOX_DEFAULT_LXC_DISK_GB:=8}"
|
||||
|
||||
# `pct create --memory` only sets RAM -- swap is a wholly separate
|
||||
# parameter that otherwise silently stays at Proxmox's own 512M default
|
||||
# regardless of --memory (confirmed: creating with --memory 2048 left
|
||||
# swap at 512). create-proxmox-resource.sh defaults --swap to whatever
|
||||
# --memory resolves to at runtime rather than a static value here, so it
|
||||
# tracks a --memory picked at the CLI too, not just the default above.
|
||||
|
||||
# Required for a modern (v247+) systemd guest to actually boot as an
|
||||
# unprivileged container: systemd's routine use of nested user namespaces
|
||||
# and credential mounts (LoadCredential=, DynamicUser=, etc. -- used even
|
||||
# by plain getty units) gets denied by AppArmor's default LXC confinement
|
||||
# without these. Confirmed live: without them, every getty unit
|
||||
# crash-loops on a denied `/run/credentials/*` mount every ~3s (visible
|
||||
# as garbage on the console) and core services like nsncd fail the same
|
||||
# way on userns_create; system.build.tarball never finishes activating.
|
||||
: "${PROXMOX_DEFAULT_LXC_FEATURES:=nesting=1,keyctl=1}"
|
||||
|
||||
export PROXMOX_HOST PROXMOX_SSH_USER PROXMOX_STORAGE PROXMOX_ISO_STORAGE \
|
||||
PROXMOX_BRIDGE PROXMOX_DEFAULT_CORES PROXMOX_DEFAULT_MEMORY_MB PROXMOX_DEFAULT_LXC_DISK_GB
|
||||
PROXMOX_BRIDGE PROXMOX_DEFAULT_CORES PROXMOX_DEFAULT_MEMORY_MB \
|
||||
PROXMOX_DEFAULT_LXC_DISK_GB PROXMOX_DEFAULT_LXC_FEATURES
|
||||
|
||||
# Matches variables.nix's nixCacheHost -- update both if it ever changes.
|
||||
: "${NIX_CACHE_HOST:=nix-cache}"
|
||||
|
||||
Reference in New Issue
Block a user