# Node 1 Hardware Layout

Build node 1 so nodes 2/3 are drop-in identical later — don't re-architect
disks or network when the cluster grows.

## Disks — three separate roles, physically separate devices

1. **Boot/OS pool (`rpool`)** — 2x small SSDs (240-480GB plenty), ZFS
   mirror. Proxmox itself only. Never share with Ceph OSDs or bulk ZFS
   data pools.
2. **Future Ceph OSD disks** — must end up as raw, unformatted devices —
   no ZFS/RAID/LVM underneath (Ceph does its own replication; anything
   underneath just doubles copy-on-write/checksumming and hurts
   performance). Use enterprise SATA/NVMe SSDs with power-loss protection
   (PLP) — matters far more for Ceph write latency than for general ZFS
   use. Ceph needs 3 nodes minimum to go live, so on node 1 these disks
   either sit idle or run as a temporary local ZFS pool to be wiped and
   handed to Ceph once nodes 2/3 exist.
3. **Local ZFS "replicated tier" disks** — separate set of disks
   (mirror or small raidz) for VMs kept on local storage + PVE
   replication rather than Ceph (latency-sensitive or non-critical
   workloads). This pool is permanent, not a placeholder.

If budget only allows one extra disk set right now: prioritize the
future-Ceph disks, run everything on ZFS locally until nodes 2/3 arrive,
then split workloads out. Avoid consumer QLC SSDs for either role — Ceph
punishes it on latency, ZFS on sync writes/scrub.

## Networking — cable and provision for the final topology now

Logically separate networks (ideally separate NICs/VLANs):

- **Management** — web UI / SSH
- **Corosync** — cluster quorum traffic, low-latency, unshared
- **Ceph public** — VM-to-OSD traffic
- **Ceph cluster/backend** — OSD-to-OSD replication (heaviest load)

Practical layout: 2x 10/25GbE bonded or split — one pair for Ceph, one
for mgmt + corosync + VM traffic, with corosync on its own VLAN even if
sharing a physical NIC. Get switch/cabling right on node 1 so nodes 2/3
are identical drops.

## CPU / RAM sizing

Size for the end state, not day one — RAM is the hardest thing to
retrofit. Budget covers:

- OS + ZFS ARC (ZFS wants RAM, not just disk)
- Ceph OSD daemons — realistically 3-5GB per OSD once running
- Actual VM workloads

Roughly a core per OSD on top of what VMs need. If OSDs won't be active
for a while, that's headway, but buy for 3 nodes' worth of eventual OSD
load.

## Backup target (PBS)

Keep it off the Ceph/compute nodes if possible — its failure domain
should be independent of the cluster. Modest separate machine or NAS:
ZFS mirror or raidz2, ECC RAM if possible, capacity for retention policy.
If it has to run as a VM inside the cluster short-term, that's a known
compromise, not the end state.

## Node 1 install sequence

1. Install Proxmox VE fresh onto the ZFS boot mirror.
2. Create the local ZFS "tier 2" pool for VM disks.
3. Leave Ceph-earmarked disks idle, or provision as a temporary ZFS pool
   (to be wiped once Ceph goes live at 3 nodes).