fix(ha/cluster-init): fix DRBD metadata skip condition and StandAlone grace period
Check NixOS configurations / eval-hosts (push) Successful in 10m27s

The metadata creation check used grep -q "UpToDate" which matches when the
resource is DOWN — drbdadm dstate reads metadata directly and returns just
"UpToDate" (no slash) for a down-but-previously-synced resource.  This caused
metadata creation to be silently skipped, leaving mismatched UUIDs from an
interrupted sync which cause DRBD to go WFConnection→StandAlone immediately.

Fix: require exact "UpToDate/UpToDate" (with slash, only possible when the
resource is UP and both disks are confirmed synced) before skipping.

Also give the StandAlone detection a 5-iteration (15 s) grace period before
dying, to avoid false-positive on the first few ticks while the peer is still
coming up.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-07-29 01:45:03 +10:00
co-authored by Claude Sonnet 4.6
parent b88ea49880
commit d7e63cd80e
+9 -4
View File
@@ -215,14 +215,19 @@ log "Initialising DRBD metadata on $NODE1..."
# "stdin not a TTY, not waiting for confirmation" → exit 20. # "stdin not a TTY, not waiting for confirmation" → exit 20.
# Calling drbdmeta --force directly bypasses the exclusive-open confirmation on # Calling drbdmeta --force directly bypasses the exclusive-open confirmation on
# both steps without needing a TTY, regardless of whether the device is busy. # both steps without needing a TTY, regardless of whether the device is busy.
if ! drbdadm dstate ha-data 2>/dev/null | grep -q "UpToDate"; then # Skip metadata creation only if DRBD is UP and fully synced (UpToDate/UpToDate).
# When the resource is down, drbdadm dstate reads metadata and returns just
# "UpToDate" (no slash) — that must not be treated as "already synced".
# Mismatched UUIDs from an interrupted sync cause instant WFConnection→StandAlone,
# so we always recreate metadata unless the sync is genuinely complete.
if [[ "$(drbdadm dstate ha-data 2>/dev/null)" != "UpToDate/UpToDate" ]]; then
UUID1=$(_rand_uuid) UUID1=$(_rand_uuid)
drbdmeta --force 0 v08 "${DRBD_DISK}" internal create-md drbdmeta --force 0 v08 "${DRBD_DISK}" internal create-md
drbdmeta --force 0 v08 "${DRBD_DISK}" internal write-dev-uuid "$UUID1" drbdmeta --force 0 v08 "${DRBD_DISK}" internal write-dev-uuid "$UUID1"
fi fi
log "Initialising DRBD metadata on $NODE2..." log "Initialising DRBD metadata on $NODE2..."
if ! n2_ssh "drbdadm dstate ha-data 2>/dev/null | grep -q UpToDate" 2>/dev/null; then if [[ "$(n2_ssh "drbdadm dstate ha-data 2>/dev/null" 2>/dev/null)" != "UpToDate/UpToDate" ]]; then
UUID2=$(n2_ssh "cat /proc/sys/kernel/random/uuid 2>/dev/null | tr -d '-' | cut -c1-16 | tr '[:lower:]' '[:upper:]'") UUID2=$(n2_ssh "cat /proc/sys/kernel/random/uuid 2>/dev/null | tr -d '-' | cut -c1-16 | tr '[:lower:]' '[:upper:]'")
n2_ssh "drbdmeta --force 0 v08 ${DRBD_DISK} internal create-md" n2_ssh "drbdmeta --force 0 v08 ${DRBD_DISK} internal create-md"
n2_ssh "drbdmeta --force 0 v08 ${DRBD_DISK} internal write-dev-uuid ${UUID2}" n2_ssh "drbdmeta --force 0 v08 ${DRBD_DISK} internal write-dev-uuid ${UUID2}"
@@ -259,9 +264,9 @@ while true; do
_spd=$(grep -oE "speed:[[:space:]]+[0-9,]+" /proc/drbd 2>/dev/null | grep -oE "[0-9,]+$" | head -1 || echo "") _spd=$(grep -oE "speed:[[:space:]]+[0-9,]+" /proc/drbd 2>/dev/null | grep -oE "[0-9,]+$" | head -1 || echo "")
_sync_iter=$(( _sync_iter + 1 )) _sync_iter=$(( _sync_iter + 1 ))
_sc="${_sync_chars[$_sync_iter % 4]}" _sc="${_sync_chars[$_sync_iter % 4]}"
if [[ "$_cs" == "StandAlone" ]]; then if [[ "$_cs" == "StandAlone" && $_sync_iter -gt 5 ]]; then
printf "\r%-80s\r" "" printf "\r%-80s\r" ""
die "DRBD is StandAlone — peer connection lost (dstate: $_dstate). " \ die "DRBD is StandAlone after 15 s — peer connection lost (dstate: $_dstate). " \
"Check corosync/network and re-run cluster-init." "Check corosync/network and re-run cluster-init."
elif [[ -n "$_pct" ]]; then elif [[ -n "$_pct" ]]; then
printf "\r [%s] syncing: %s%% done — ETA %s @ %s K/s " \ printf "\r [%s] syncing: %s%% done — ETA %s @ %s K/s " \