Archived
Fix LXC container creation: unprivileged, nesting/keyctl, swap sizing
Found and fixed live against a real test container (VMID 100, lxc-nix-cache on pve.sweet.home) after the previous pct-restore-to-pct-create fix still produced a container that booted into garbled console output: 1. pct create's own CLI default for --unprivileged is privileged (unlike the web UI, whose checkbox defaults the other way), but modules/platforms/lxc.nix sets proxmoxLXC.privileged = false, so the image assumes it's running unprivileged. Real mismatch -- now passes --unprivileged 1 explicitly. 2. The actual root cause of the garbled console: modern (v247+) systemd routinely uses nested user namespaces and credential mounts (even plain getty units, via LoadCredential=-style mechanisms), which AppArmor's default LXC confinement denies without --features nesting=1,keyctl=1. Confirmed via the host's kernel audit log: every getty unit was crash-looping on a denied /run/credentials/* mount every ~3s, and core services like nsncd failed userns_create the same way -- the system never finished activating. Fixed live (pct set + restart on the running test container) before committing the script change: systemctl is-system-running went from never completing to "running" with zero failed units. 3. --memory doesn't touch swap -- confirmed live it silently stayed at Proxmox's own 512M default with --memory 2048. Now defaults --swap to whatever --memory resolves to. docs/auto-installer.md's manual pct create walkthrough gets the same fixes, with the "why" for each flag, since a human following it by hand would hit the identical bugs. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01La55Nsss8jZ7ZuzUV9mfot
This commit is contained in:
+24
-2
@@ -45,15 +45,37 @@ of its own:
|
|||||||
|
|
||||||
```sh
|
```sh
|
||||||
pct create <vmid> local:vztmpl/<file>.tar.xz \
|
pct create <vmid> local:vztmpl/<file>.tar.xz \
|
||||||
--rootfs local-lvm:8 --hostname <name> --cores 2 --memory 2048 \
|
--unprivileged 1 --features nesting=1,keyctl=1 \
|
||||||
|
--rootfs local-lvm:8 --hostname <name> --cores 2 --memory 2048 --swap 2048 \
|
||||||
--net0 name=eth0,bridge=vmbr0,ip=dhcp
|
--net0 name=eth0,bridge=vmbr0,ip=dhcp
|
||||||
pct start <vmid>
|
pct start <vmid>
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Every one of those extra flags is load-bearing, confirmed by actually
|
||||||
|
booting one:
|
||||||
|
|
||||||
|
- `--unprivileged 1` — `modules/platforms/lxc.nix` sets
|
||||||
|
`proxmoxLXC.privileged = false`, so the image assumes it's running
|
||||||
|
unprivileged. `pct create`'s own CLI default for this flag is
|
||||||
|
privileged (unlike the web UI, whose checkbox defaults the other way)
|
||||||
|
— omit it and you get a privileged container running a NixOS config
|
||||||
|
that assumes unprivileged, a real mismatch.
|
||||||
|
- `--features nesting=1,keyctl=1` — required for a modern (v247+)
|
||||||
|
systemd guest to boot unprivileged at all. Without it, AppArmor denies
|
||||||
|
the nested user namespaces and credential mounts systemd routinely
|
||||||
|
uses (even plain getty units) — every getty crash-loops on a denied
|
||||||
|
`/run/credentials/*` mount every ~3s (this is what garbage on the
|
||||||
|
console turns out to be) while core services like `nsncd` fail the
|
||||||
|
same way, and the system never finishes activating.
|
||||||
|
- `--swap 2048` — `--memory` doesn't touch swap; it silently stays at
|
||||||
|
Proxmox's own 512M default otherwise. Match it to `--memory` unless
|
||||||
|
you deliberately want otherwise.
|
||||||
|
|
||||||
First boot runs `boot.postBootCommands` (registers the Nix store DB and
|
First boot runs `boot.postBootCommands` (registers the Nix store DB and
|
||||||
system profile) — there's no separate activation step to run yourself.
|
system profile) — there's no separate activation step to run yourself.
|
||||||
`scripts/create-proxmox-resource.sh --type lxc --host <name>` automates all
|
`scripts/create-proxmox-resource.sh --type lxc --host <name>` automates all
|
||||||
of this (build, host-key handling, upload, `pct create`) — see its `--help`.
|
of this (build, host-key handling, upload, `pct create` with the flags
|
||||||
|
above) — see its `--help`.
|
||||||
|
|
||||||
Host keys still need pre-seeding the same way as any other host (see "Host
|
Host keys still need pre-seeding the same way as any other host (see "Host
|
||||||
keys" below) — the sops-nix activation-vs-first-boot race is identical
|
keys" below) — the sops-nix activation-vs-first-boot race is identical
|
||||||
|
|||||||
@@ -73,6 +73,10 @@ Shared:
|
|||||||
modify: omit to leave unchanged.
|
modify: omit to leave unchanged.
|
||||||
--memory <MB> create: default \$PROXMOX_DEFAULT_MEMORY_MB (${PROXMOX_DEFAULT_MEMORY_MB}).
|
--memory <MB> create: default \$PROXMOX_DEFAULT_MEMORY_MB (${PROXMOX_DEFAULT_MEMORY_MB}).
|
||||||
modify: omit to leave unchanged.
|
modify: omit to leave unchanged.
|
||||||
|
--swap <MB> lxc only, create time: \`--memory\` doesn't
|
||||||
|
touch swap -- it silently stays at Proxmox's
|
||||||
|
own 512M default otherwise. (default: matches
|
||||||
|
whatever --memory resolves to)
|
||||||
--storage <pool> (default: \$PROXMOX_STORAGE, ${PROXMOX_STORAGE})
|
--storage <pool> (default: \$PROXMOX_STORAGE, ${PROXMOX_STORAGE})
|
||||||
--iso-storage <pool> (default: \$PROXMOX_ISO_STORAGE, ${PROXMOX_ISO_STORAGE})
|
--iso-storage <pool> (default: \$PROXMOX_ISO_STORAGE, ${PROXMOX_ISO_STORAGE})
|
||||||
--bridge <bridge> (default: \$PROXMOX_BRIDGE, ${PROXMOX_BRIDGE})
|
--bridge <bridge> (default: \$PROXMOX_BRIDGE, ${PROXMOX_BRIDGE})
|
||||||
@@ -95,6 +99,7 @@ name=""
|
|||||||
vmid=""
|
vmid=""
|
||||||
cores=""
|
cores=""
|
||||||
memory=""
|
memory=""
|
||||||
|
swap=""
|
||||||
disk_size=""
|
disk_size=""
|
||||||
grow_disk=""
|
grow_disk=""
|
||||||
image=""
|
image=""
|
||||||
@@ -114,6 +119,7 @@ while [[ $# -gt 0 ]]; do
|
|||||||
--vmid) vmid="$2"; shift 2 ;;
|
--vmid) vmid="$2"; shift 2 ;;
|
||||||
--cores) cores="$2"; shift 2 ;;
|
--cores) cores="$2"; shift 2 ;;
|
||||||
--memory) memory="$2"; shift 2 ;;
|
--memory) memory="$2"; shift 2 ;;
|
||||||
|
--swap) swap="$2"; shift 2 ;;
|
||||||
--disk-size) disk_size="$2"; shift 2 ;;
|
--disk-size) disk_size="$2"; shift 2 ;;
|
||||||
--grow-disk) grow_disk="$2"; shift 2 ;;
|
--grow-disk) grow_disk="$2"; shift 2 ;;
|
||||||
--image) image="$2"; shift 2 ;;
|
--image) image="$2"; shift 2 ;;
|
||||||
@@ -436,7 +442,25 @@ echo
|
|||||||
if [[ "$type" == "lxc" ]]; then
|
if [[ "$type" == "lxc" ]]; then
|
||||||
echo "==> Creating LXC container ${vmid} (${name})..."
|
echo "==> Creating LXC container ${vmid} (${name})..."
|
||||||
local_disk_size="${disk_size:-$PROXMOX_DEFAULT_LXC_DISK_GB}"
|
local_disk_size="${disk_size:-$PROXMOX_DEFAULT_LXC_DISK_GB}"
|
||||||
create_cmd="pct create ${vmid} ${iso_storage}:vztmpl/${remote_filename} --rootfs ${storage}:${local_disk_size} --hostname ${name} --cores ${cores} --memory ${memory} --net0 name=eth0,bridge=${bridge},ip=dhcp"
|
# --memory doesn't touch swap -- it silently stays at Proxmox's own
|
||||||
|
# 512M default otherwise (confirmed live: --memory 2048 left swap at
|
||||||
|
# 512). Default to matching whatever --memory resolved to above.
|
||||||
|
local_swap="${swap:-$memory}"
|
||||||
|
# --unprivileged 1: modules/platforms/lxc.nix sets proxmoxLXC.privileged
|
||||||
|
# = false, so the NixOS config inside the image assumes it's running as
|
||||||
|
# an unprivileged container (cgroup/capability/mount expectations baked
|
||||||
|
# in at boot). `pct create`'s own CLI default for this flag is
|
||||||
|
# privileged (unlike the web UI, which defaults its checkbox the other
|
||||||
|
# way) -- leaving it unset creates a privileged container running a
|
||||||
|
# NixOS config that assumes unprivileged, a real mismatch.
|
||||||
|
#
|
||||||
|
# --features nesting=1,keyctl=1: required for a modern (v247+) systemd
|
||||||
|
# guest to actually boot unprivileged -- confirmed live: without this,
|
||||||
|
# AppArmor denies the nested user namespaces and credential mounts
|
||||||
|
# systemd routinely uses (even plain getty units), and every getty
|
||||||
|
# crash-loops on a denied mount every ~3s (visible as garbage on the
|
||||||
|
# console) while core services like nsncd fail the same way.
|
||||||
|
create_cmd="pct create ${vmid} ${iso_storage}:vztmpl/${remote_filename} --unprivileged 1 --features ${PROXMOX_DEFAULT_LXC_FEATURES} --rootfs ${storage}:${local_disk_size} --hostname ${name} --cores ${cores} --memory ${memory} --swap ${local_swap} --net0 name=eth0,bridge=${bridge},ip=dhcp"
|
||||||
remote "$create_cmd"
|
remote "$create_cmd"
|
||||||
remote "pct start ${vmid}"
|
remote "pct start ${vmid}"
|
||||||
else
|
else
|
||||||
|
|||||||
+19
-1
@@ -30,8 +30,26 @@
|
|||||||
# no backup metadata to infer it from. Matches Proxmox's own GUI default.
|
# no backup metadata to infer it from. Matches Proxmox's own GUI default.
|
||||||
: "${PROXMOX_DEFAULT_LXC_DISK_GB:=8}"
|
: "${PROXMOX_DEFAULT_LXC_DISK_GB:=8}"
|
||||||
|
|
||||||
|
# `pct create --memory` only sets RAM -- swap is a wholly separate
|
||||||
|
# parameter that otherwise silently stays at Proxmox's own 512M default
|
||||||
|
# regardless of --memory (confirmed: creating with --memory 2048 left
|
||||||
|
# swap at 512). create-proxmox-resource.sh defaults --swap to whatever
|
||||||
|
# --memory resolves to at runtime rather than a static value here, so it
|
||||||
|
# tracks a --memory picked at the CLI too, not just the default above.
|
||||||
|
|
||||||
|
# Required for a modern (v247+) systemd guest to actually boot as an
|
||||||
|
# unprivileged container: systemd's routine use of nested user namespaces
|
||||||
|
# and credential mounts (LoadCredential=, DynamicUser=, etc. -- used even
|
||||||
|
# by plain getty units) gets denied by AppArmor's default LXC confinement
|
||||||
|
# without these. Confirmed live: without them, every getty unit
|
||||||
|
# crash-loops on a denied `/run/credentials/*` mount every ~3s (visible
|
||||||
|
# as garbage on the console) and core services like nsncd fail the same
|
||||||
|
# way on userns_create; system.build.tarball never finishes activating.
|
||||||
|
: "${PROXMOX_DEFAULT_LXC_FEATURES:=nesting=1,keyctl=1}"
|
||||||
|
|
||||||
export PROXMOX_HOST PROXMOX_SSH_USER PROXMOX_STORAGE PROXMOX_ISO_STORAGE \
|
export PROXMOX_HOST PROXMOX_SSH_USER PROXMOX_STORAGE PROXMOX_ISO_STORAGE \
|
||||||
PROXMOX_BRIDGE PROXMOX_DEFAULT_CORES PROXMOX_DEFAULT_MEMORY_MB PROXMOX_DEFAULT_LXC_DISK_GB
|
PROXMOX_BRIDGE PROXMOX_DEFAULT_CORES PROXMOX_DEFAULT_MEMORY_MB \
|
||||||
|
PROXMOX_DEFAULT_LXC_DISK_GB PROXMOX_DEFAULT_LXC_FEATURES
|
||||||
|
|
||||||
# Matches variables.nix's nixCacheHost -- update both if it ever changes.
|
# Matches variables.nix's nixCacheHost -- update both if it ever changes.
|
||||||
: "${NIX_CACHE_HOST:=nix-cache}"
|
: "${NIX_CACHE_HOST:=nix-cache}"
|
||||||
|
|||||||
Reference in New Issue
Block a user