# pve-test's Wifi-Primary Network

Non-standard, deliberately chosen against normal Proxmox guidance (a
hypervisor's management network is not supposed to be wireless). Applies
**only** to `pve-test` as a standalone sandbox node — see
`05-node-roles.md` for why this and cluster membership don't mix, and
never apply this pattern to a node that's joined (or might join) a
cluster.

## Why this works at all

Wifi (802.11) normally cannot be bridged the way Ethernet can: an access
point only accepts frames whose source MAC matches the MAC that
associated with it. A Linux bridge with a wifi port as a member sends
frames tagged with whatever MAC actually generated them (the bridge's own
MAC, a VM's MAC, etc.) — none of which match the wifi card's hardware
MAC, so the AP silently drops them. This isn't a Linux/Proxmox limitation,
it's how WiFi association works.

**4addr (WDS) client mode** is the exception: if both the wifi driver and
the AP support it, the client tags frames with a 4th address field
carrying the original MAC, and the AP forwards them like a real Ethernet
segment. Most consumer routers do **not** support this (it's common on
OpenWrt/DD-WRT/enterprise APs, rare on stock ISP hardware) — it has to be
confirmed empirically, per-AP, before trusting it with a management IP.

pve-test's card (Intel AX200, `iwlwifi` driver, interface `wlp3s0`)
supports 4addr client-side generically (this is a `mac80211` core
feature, not driver-specific). The home AP it associates to
(`nbn-fttp-net-5G`) was confirmed to honor it — see validation method
below.

## Validate before ever touching a live management IP

Don't add the wifi NIC straight into your real bridge to "see if it
works" — if 4addr isn't actually honored, you lose access to the box with
no diagnostic trail (association succeeds; only the *data plane* silently
fails). Test in an isolated scratch namespace first:

```bash
# assumes wlp3s0 already associated with 4addr on (see systemd unit below)
ip link add testbr0 type bridge && ip link set testbr0 up
ip link set wlp3s0 master testbr0 && ip link set wlp3s0 up

ip netns add wifitest
ip link add veth-host type veth peer name veth-ns
ip link set veth-host master testbr0 && ip link set veth-host up
ip link set veth-ns netns wifitest
ip netns exec wifitest ip link set veth-ns address 02:11:22:33:44:55  # deliberately NOT the wifi card's MAC
ip netns exec wifitest ip link set veth-ns up
ip netns exec wifitest ip link set lo up

ip netns exec wifitest dhclient -v -1 veth-ns   # real DHCP lease = AP forwards foreign MACs = 4addr genuinely works
```

If that gets a lease (or at minimum a ping reply from the gateway) from a
MAC that was never the one that associated, the AP is forwarding
arbitrary-MAC frames — real bridge mode will work. If not, don't proceed
to bridge mode; fall back to a routed/NAT design instead (wifi gets its
own IP via DHCP, VMs NAT out through it — never covered here since it
wasn't needed, but keep this fallback in mind if reproducing on different
hardware/AP).

Tear the test namespace/bridge down afterwards (`ip netns del wifitest;
ip link del veth-host; ip link set wlp3s0 nomaster; ip link del testbr0`)
— it's scratch, not part of the real config.

## Final architecture

```
wlp3s0 (wifi, 4addr client mode) ─┐
                                    ├─ bond0 (active-backup, wlp3s0 primary) ─ vmbr0 (192.168.2.251/24)
nic0 (wired, LAN backup)          ─┘
```

- **`wpa-4addr-<iface>.service`** (systemd unit, `Before=network-pre.target`) —
  sets 4addr mode and starts `wpa_supplicant` *before* ifupdown2 processes
  `/etc/network/interfaces`. Deliberately not using `wpasupplicant`'s own
  `wpa-conf` ifupdown2 hook integration — its ordering relative to a
  custom `pre-up iw ... set 4addr on` line is implementation-specific and
  not worth gambling on for something this hard to debug if it's wrong.
  The systemd unit gives full, deterministic control over sequencing
  instead.
- **`bond0`** — active-backup bonding, `wlp3s0` as `bond-primary`,
  `nic0` as backup, `bond-miimon 100`. Bonding a wifi interface works here
  because `mac80211`/`iwlwifi` properly reports carrier state
  (`netif_carrier_on/off`) on association/disassociation, which is what
  bonding's `miimon` (with the default `use_carrier=1`) actually watches
  — it doesn't require a "real" MII-capable NIC.
- **`vmbr0`** — unchanged IP (`192.168.2.251/24`), bridged over `bond0`
  instead of directly over a physical NIC.

Reproduce with `scripts/setup-wifi-bond-network.sh` — see its header
comment for usage and required env vars (SSID/passphrase, management
address/gateway). It stages `/etc/network/interfaces` and the systemd
unit but does **not** auto-apply (`ifreload -a`) — see the next section
for why that step needs care, not automation.

## Applying a live bridge-port change: what went wrong once, and what worked

Moving `vmbr0`'s underlying port live — while SSH'd in over the very
address that lives on that bridge — is inherently risky: if the new port
doesn't actually pass traffic, you lose the connection you're using to
fix it.

**What failed**: doing the port swap via raw `ip link set nic0 nomaster`
+ `ip link set nic0 down` (with `wlp3s0` already added as a second bridge
member) caused an extended outage that needed a physical power-cycle to
recover — `nic0` never came back up on its own even well past a 60s
auto-revert watchdog's deadline. Importantly, the *host itself* never
crashed — `journalctl -b -1` showed completely normal operation (a
`pvestatd` polling loop, no gaps, no panic/OOM/watchdog trigger) right up
until a keyboard was plugged in and it was manually reset. So the failure
was specifically in the live network transition, not the OS — root cause
unconfirmed, but plausibly bridge FDB/MAC-identity handling when a port
carrying an already-live IP is swapped out via raw `ip link` rather than
a coordinated reconfiguration.

**What worked**: writing the final state into `/etc/network/interfaces`
and applying with `ifreload -a` (Proxmox's own supported hot-reload path)
instead. Same end state, same risk window, but ifupdown2 evidently
sequences the transition in a way raw `ip link` surgery didn't. This
succeeded on the first retry using this method.

**Either way, expect ~30-60 seconds of apparent breakage even on a
successful change** — the upstream switch/AP needs to relearn which port
`192.168.2.251`'s MAC now lives behind. Don't judge success/failure
before that window passes; a `ping`/SSH failure at the 5-10s mark is not
yet a sign anything is wrong.

**Recommended safety net for any future live change here**: arm a
backgrounded auto-revert-on-timeout before applying, e.g.:

```bash
cp /etc/network/interfaces /etc/network/interfaces.bak
cat > /root/revert.sh <<'EOF'
#!/bin/bash
cp /etc/network/interfaces.bak /etc/network/interfaces
ifreload -a
EOF
chmod +x /root/revert.sh
setsid nohup bash -c "sleep 45 && /root/revert.sh" >/root/revert.log 2>&1 < /dev/null &
disown
# now apply the real change, e.g.: ifreload -a
# once confirmed working: pkill -f "sleep 45 && /root/revert.sh"
```

This only self-heals if the box stays responsive enough for the
backgrounded job to run to completion — it is not a substitute for
physical/console access being available, just a way to avoid needing it
for the common case.

## Troubleshooting: "can't ping it, web UI won't load" isn't necessarily an outage

Two independent things can make `pve-test` look dead when it isn't:

1. **Stale ARP after any change to `vmbr0`'s active port.** `vmbr0`'s MAC
   address follows whichever interface is currently active in `bond0`
   (`wlp3s0`'s MAC while wifi is primary, `nic0`'s if it fails over) — it
   is *not* fixed. Other devices on the LAN (including your own
   workstation) that cached the old MAC before a change won't notice
   until their ARP entry naturally expires or gets flushed
   (`ip neigh flush <ip>` / `arp -d <ip>`). Symptom: intermittent or
   totally dead connectivity from one specific device while others (or
   the host itself) are fine.
2. **ICMP ping is blocked by the management firewall, unrelated to
   network health.** See `04-security-hardening.md`'s firewall section —
   `cluster.fw`'s `policy_in: DROP` only explicitly allows TCP 22/8006
   (now also ICMP echo-request, after this was hit) from the mgmt
   network. Before that rule was added, `ping` failed 100% *even with a
   perfectly correct, fresh ARP entry and full SSH/web UI access* —
   nothing to do with wifi, bonding, or the network being down. This is
   exactly what happened once: alarming "can't ping, web UI not loading"
   turned out to be a stale-ARP moment (which resolved itself) plus a
   pre-existing firewall policy (ping was never going to work,
   regardless of wifi).

**When `pve-test` seems unreachable, check in this order before assuming
a real outage**: SSH (`ssh root@pve-test.sweet.home`) → web UI via `curl
-sk -o /dev/null -w '%{http_code}' https://pve-test.sweet.home:8006/` →
only then worry about `ping` specifically, and check
`/etc/pve/firewall/cluster.fw` before blaming the network.

## Known limitation history

`nic0` (the bond's backup slave) initially had no cable physically
connected when this was first set up — the bond was correctly configured
but inert until a cable was plugged in. As of this writing a cable is
connected and the backup slave is live (confirm with `cat
/proc/net/bonding/bond0`) — if reproducing this setup, verify the same
before assuming LAN failover will actually work.