Archived
fix(ha/cluster-init): enable Pacemaker maintenance-mode during DRBD sync
Check NixOS configurations / eval-hosts (push) Successful in 10m23s
Check NixOS configurations / eval-hosts (push) Successful in 10m23s
Even with both nodes in standby, Pacemaker's monitor operations keep running. When the monitor sees DRBD is Primary on a standby node (that it didn't start), it triggers a stop action — killing the initial sync after ~10 s. Enable maintenance-mode after standby stops the existing resources but before DRBD is manually brought up for the sync. Maintenance-mode suspends all start/stop/monitor actions so Pacemaker is completely hands-off during the sync. Disable it alongside crm_standby -v off once UpToDate/UpToDate is confirmed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -152,11 +152,13 @@ for i in $(seq 1 30); do
|
||||
done
|
||||
|
||||
# ── 2. DRBD initialisation ────────────────────────────────────────────────
|
||||
# Put both nodes in Pacemaker standby before touching DRBD metadata.
|
||||
# Without this, the OCF DRBD agent races: it sees drbdadm-down as a failure
|
||||
# and immediately calls drbdadm-up again, leaving the backing disk busy when
|
||||
# create-md / write-dev-uuid runs. On a fresh cluster with no resources
|
||||
# configured this is a no-op; on a re-run it stops the race.
|
||||
# Put both nodes in Pacemaker standby first so it stops managed resources
|
||||
# cleanly, then enable maintenance-mode so Pacemaker's monitor operations are
|
||||
# suspended. Without maintenance-mode, Pacemaker keeps monitoring: when it
|
||||
# sees DRBD Primary on a standby node (that it didn't start), it triggers a
|
||||
# stop action — killing the initial sync after ~10 s. Maintenance-mode
|
||||
# disables all start/stop/monitor actions for the duration of the sync; it is
|
||||
# cleared after UpToDate/UpToDate is confirmed.
|
||||
log "Setting both nodes to Pacemaker standby for DRBD metadata init..."
|
||||
crm_standby -N "$NODE1" -v on 2>/dev/null || true
|
||||
crm_standby -N "$NODE2" -v on 2>/dev/null || true
|
||||
@@ -174,6 +176,9 @@ for i in $(seq 1 30); do
|
||||
sleep 2
|
||||
done
|
||||
|
||||
log "Enabling Pacemaker maintenance-mode (suspends monitor/start/stop during sync)..."
|
||||
crm_attribute -t crm_config -n maintenance-mode -v true 2>/dev/null || true
|
||||
|
||||
log "Detaching DRBD on $NODE1 (belt-and-suspenders after standby)..."
|
||||
drbdadm down ha-data 2>/dev/null || true
|
||||
log "Detaching DRBD on $NODE2..."
|
||||
@@ -277,7 +282,8 @@ while true; do
|
||||
sleep 3
|
||||
done
|
||||
|
||||
log "Clearing Pacemaker standby — sync complete, handing DRBD back to Pacemaker..."
|
||||
log "Disabling Pacemaker maintenance-mode and clearing standby — handing DRBD back to Pacemaker..."
|
||||
crm_attribute -t crm_config -n maintenance-mode -v false 2>/dev/null || true
|
||||
crm_standby -N "$NODE1" -v off 2>/dev/null || true
|
||||
crm_standby -N "$NODE2" -v off 2>/dev/null || true
|
||||
|
||||
|
||||
Reference in New Issue
Block a user