Archived
Initial infrastructure mono-repo scaffold
Consolidates nixos, docker, raspi, and debian-configuration into a single infrastructure-as-code repo. Includes: - ansible/: full inventory + proxmox-hardening, freeipa, and raspberrypi roles (converted from debian-configuration bash scripts) - terraform/: Proxmox VMs, Dynu DNS, Pi-hole (decommissioned stub), Docker container catalog — migrated from docker/infrastructure/terraform/ - stacks/docker/, stacks/raspi/, nixos/: placeholder READMEs pending git subtree population (see implementation plan) - docs/: internal MkDocs site with architecture, network topology, runbooks, and drift-detection guide; external sanitized site - scripts/: drift-detect.sh, docs-build.sh, install-hooks.sh, check-secrets.sh - CI: secret-scan (push/PR), drift-detect (daily), docs-build (on change) - Pi-hole removed throughout — DNS is FreeIPA, DHCP is router See docs/internal/implementation-plan.md for the phased rollout after pushing to Gitea. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UvNjoxTWEDkhXsd1Dq2ETP
This commit is contained in:
beatzaplentyandClaude Sonnet 4.6 committed
commit
f85c65870f
125 files changed
+8186
No files matched your search
Vendored
+31
@@ -0,0 +1,31 @@
|
||||
# Homelab
|
||||
|
||||
A self-hosted infrastructure running on modest home hardware.
|
||||
|
||||
## Self-hosted services
|
||||
|
||||
| Service | What it does |
|
||||
|---------|-------------|
|
||||
| Nextcloud | Private cloud storage and sync |
|
||||
| Passbolt | Team password manager |
|
||||
| Gitea | Self-hosted Git with CI/CD |
|
||||
| SearXNG | Privacy-respecting meta-search engine |
|
||||
| Gramps Web | Family tree / genealogy |
|
||||
| Uptime Kuma | Service availability monitoring |
|
||||
|
||||
## Infrastructure
|
||||
|
||||
Built on:
|
||||
- **Proxmox VE** hypervisor running virtual machines and containers
|
||||
- **NixOS** for declarative, reproducible server configuration
|
||||
- **Docker** for containerised applications
|
||||
- **FreeIPA** for centralised identity and authentication
|
||||
- **Terraform** and **Ansible** for infrastructure as code
|
||||
- **Traefik** for reverse proxy with automatic TLS
|
||||
|
||||
## Tech stack highlights
|
||||
|
||||
- All infrastructure is declared in code — no manual configuration
|
||||
- Automated drift detection runs daily to catch any state inconsistencies
|
||||
- Secrets are encrypted at rest using SOPS + age
|
||||
- Automated documentation generated from live inventory
|
||||
Whitespace-only changes.
@@ -0,0 +1,77 @@
|
||||
# Architecture
|
||||
|
||||
## Infrastructure layers
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ pve1.sweet.home (Proxmox VE — production hypervisor) │
|
||||
│ │
|
||||
│ ┌────────────┐ ┌──────────┐ ┌──────────────────────┐ │
|
||||
│ │ server- │ │ docker │ │ nix-cache pxe-boot │ │
|
||||
│ │ nixos VM │ │ LXC │ │ LXC LXC │ │
|
||||
│ │ (baremetal │ │ 192.168 │ │ .224 .223 │ │
|
||||
│ │ GUI host) │ │ .2.225 │ └──────────────────────┘ │
|
||||
│ └────────────┘ └──────────┘ │
|
||||
│ ┌────────────┐ ┌──────────┐ ┌──────────────────────┐ │
|
||||
│ │ domain- │ │ pxe-boot │ │ ha-server-1 ha- │ │
|
||||
│ │ controller │ │ LXC │ │ VM .2.228 server-2│ │
|
||||
│ │ FreeIPA VM │ │ PXE DHCP │ │ VM .2.227│ │
|
||||
│ │ .2.253 │ │ .2.223 │ └──────────────────────┘ │
|
||||
│ └────────────┘ └──────────┘ │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ pve-test.sweet.home (Proxmox VE — sandbox) │
|
||||
│ WiFi-connected. Safe for scratch VMs/LXCs. │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ raspberrypi.tail13f623.ts.net (Raspberry Pi 4) │
|
||||
│ Edge monitoring via Tailscale. │
|
||||
│ Traefik · Uptime Kuma · CrowdSec · Beszel agent │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Configuration management matrix
|
||||
|
||||
| Host | Managed by | How |
|
||||
|------|-----------|-----|
|
||||
| pve1, pve-test | Ansible (`proxmox-hardening` role) | `ansible/playbooks/proxmox.yml` |
|
||||
| domain-controller | Ansible (`freeipa` role) | `ansible/playbooks/freeipa.yml` |
|
||||
| pihole | Ansible (`pihole` role) | `ansible/playbooks/pihole.yml` |
|
||||
| raspberrypi | Ansible (`raspberrypi` role) | `ansible/playbooks/raspi.yml` |
|
||||
| docker LXC | NixOS flake | `nixos/` — `proxmox-docker` target |
|
||||
| nix-cache LXC | NixOS flake | `nixos/` — `proxmox-nix-cache` target |
|
||||
| pxe-boot LXC | NixOS flake | `nixos/` — `proxmox-pxe-boot` target |
|
||||
| ha-server-1/2 | NixOS flake | `nixos/` — `proxmox-ha-server-1/2` targets |
|
||||
| baremetal workstation | NixOS flake | `nixos/` — `baremetal-gui` target |
|
||||
|
||||
## Authentication and DNS backbone
|
||||
|
||||
FreeIPA (`SWEET.HOME` realm) provides:
|
||||
- Kerberos SSO for all IPA-enrolled hosts
|
||||
- LDAP user/group directory (`admins`, `docker-access` groups)
|
||||
- Authoritative DNS for the entire LAN (Pi-hole decommissioned; FreeIPA is the sole resolver)
|
||||
- Certificate authority for internal TLS
|
||||
|
||||
All hosts (Proxmox nodes, Raspberry Pi, Docker host) are IPA-enrolled via SSSD.
|
||||
The `admins` group has passwordless sudo on all enrolled hosts.
|
||||
The `docker-access` group (GID 50010) grants Docker socket access on Docker hosts.
|
||||
|
||||
DHCP is handled by the router. PXE-specific DHCP options are served by the `pxe-boot` LXC.
|
||||
|
||||
## Monitoring stack
|
||||
|
||||
```
|
||||
Beszel agents (every host)
|
||||
└─→ Beszel hub (stacks/docker — beszel.lan.ddnsgeek.com)
|
||||
|
||||
Uptime Kuma (stacks/docker — monitor-kuma.lan.ddnsgeek.com)
|
||||
└─→ monitors all public endpoints
|
||||
|
||||
Gotify (stacks/docker)
|
||||
└─→ receives: WUD alerts, Docker health checks, drift detection notifications
|
||||
|
||||
WUD (stacks/docker)
|
||||
└─→ watches: local docker host + Raspberry Pi (via Tailscale)
|
||||
```
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,372 @@
|
||||
# Implementation Plan
|
||||
|
||||
Phased rollout of the `infrastructure` mono-repo. Work through these phases in order
|
||||
after the initial push to Gitea. Each phase is independently completeable — the repo
|
||||
is usable after Phase 0 even if later phases aren't done yet.
|
||||
|
||||
---
|
||||
|
||||
## Phase 0: Initial push and git history migration
|
||||
|
||||
**Goal:** Get the repo to Gitea and bring in existing repos with their commit histories.
|
||||
|
||||
### 0.1 — Create Gitea repo and push
|
||||
|
||||
```bash
|
||||
cd /home/wayne/repos/infrastructure
|
||||
git add -A
|
||||
git commit -m "Initial infrastructure mono-repo scaffold"
|
||||
|
||||
# Create the repo on Gitea first (via web UI), then:
|
||||
git remote add origin git@gitea.lan.ddnsgeek.com:wayne/infrastructure.git
|
||||
git push -u origin main
|
||||
```
|
||||
|
||||
### 0.2 — Migrate existing repos via git subtree
|
||||
|
||||
`git subtree` merges another repo's history into a subdirectory. Run these in order.
|
||||
`--squash` collapses history into one merge commit (cleaner). Remove `--squash` if you
|
||||
want full per-commit history (makes the log noisier but preserves full blame).
|
||||
|
||||
```bash
|
||||
# Main docker stack → stacks/docker/
|
||||
git subtree add --prefix=stacks/docker /home/wayne/repos/docker main --squash
|
||||
git push
|
||||
|
||||
# Raspberry Pi stack → stacks/raspi/
|
||||
git subtree add --prefix=stacks/raspi /home/wayne/repos/raspi main --squash
|
||||
git push
|
||||
```
|
||||
|
||||
> **Note:** `debian-configuration` is NOT imported via subtree — it's being dissolved.
|
||||
> Its content has been converted into Ansible roles in `ansible/roles/`. The original
|
||||
> repo should be archived in Gitea (Settings → Danger Zone → Archive) after Phase 3
|
||||
> is complete.
|
||||
|
||||
### 0.3 — Install pre-commit hook locally
|
||||
|
||||
```bash
|
||||
./scripts/install-hooks.sh
|
||||
```
|
||||
|
||||
### 0.4 — Configure Gitea CI secrets
|
||||
|
||||
In the Gitea repo → Settings → Secrets, add:
|
||||
|
||||
| Secret name | Value |
|
||||
|-------------|-------|
|
||||
| `PROXMOX_ENDPOINT` | `https://pve1.sweet.home:8006/` |
|
||||
| `PROXMOX_API_TOKEN_ID` | Terraform API token ID (create in PVE if not exists) |
|
||||
| `PROXMOX_API_TOKEN_SECRET` | Terraform API token secret |
|
||||
| `DYNU_API_KEY` | Dynu API key |
|
||||
| `ANSIBLE_SSH_KEY` | Private SSH key for Ansible connections (base64 or raw PEM) |
|
||||
| `GOTIFY_URL` | `https://gotify.lan.ddnsgeek.com` |
|
||||
| `GOTIFY_TOKEN` | Gotify app token |
|
||||
|
||||
**Creating a Proxmox API token for Terraform (if not already done):**
|
||||
```bash
|
||||
# On pve1 as root:
|
||||
pveum user add terraform@pve --comment "Terraform service account"
|
||||
pveum role add TerraformRole --privs "VM.Allocate VM.Clone VM.Config.CDROM VM.Config.CPU VM.Config.Cloudinit VM.Config.Disk VM.Config.HWType VM.Config.Memory VM.Config.Network VM.Config.Options VM.Monitor VM.Audit VM.PowerMgmt Datastore.AllocateSpace Datastore.Audit Pool.Allocate Sys.Audit Sys.Console Sys.Modify"
|
||||
pveum aclmod / -user terraform@pve -role TerraformRole
|
||||
pveum user token add terraform@pve tf --privsep=0
|
||||
# Copy the token secret — it's shown only once
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Ansible — verify connectivity and run check mode
|
||||
|
||||
**Goal:** Confirm Ansible can reach all managed hosts and produce clean check-mode output.
|
||||
|
||||
### 1.1 — Install Ansible and collections
|
||||
|
||||
```bash
|
||||
pip install ansible
|
||||
cd ansible
|
||||
ansible-galaxy collection install -r collections/requirements.yml
|
||||
```
|
||||
|
||||
### 1.2 — Verify inventory and connectivity
|
||||
|
||||
```bash
|
||||
cd ansible
|
||||
# List all hosts
|
||||
ansible all --list-hosts
|
||||
|
||||
# Ping everything
|
||||
ansible-playbook playbooks/ping.yml
|
||||
```
|
||||
|
||||
Fix any unreachable hosts (SSH key, hostname, `ansible_user`) before proceeding.
|
||||
|
||||
### 1.3 — Run check mode against each host group
|
||||
|
||||
```bash
|
||||
# Proxmox nodes
|
||||
ansible-playbook playbooks/proxmox.yml --check --diff --limit proxmox
|
||||
|
||||
# FreeIPA (check only — install tasks are gated by ipa_installed check)
|
||||
ansible-playbook playbooks/freeipa.yml --check --diff
|
||||
|
||||
# Raspberry Pi
|
||||
ansible-playbook playbooks/raspi.yml --check --diff
|
||||
```
|
||||
|
||||
**Expected result:** `check` mode should show either "no changes" (already hardened) or
|
||||
show exactly the diffs you'd expect. If it shows unexpected changes, review the task
|
||||
and update the role defaults before applying.
|
||||
|
||||
### 1.4 — Apply to pve-test first (sandbox validation)
|
||||
|
||||
```bash
|
||||
ansible-playbook playbooks/proxmox.yml --limit pve-test.sweet.home
|
||||
```
|
||||
|
||||
Verify the node is still reachable and the PVE web UI works before applying to pve1.
|
||||
|
||||
### 1.5 — Apply to production hosts
|
||||
|
||||
```bash
|
||||
# Confirm check mode looks clean, then:
|
||||
ansible-playbook playbooks/proxmox.yml --limit pve1.sweet.home
|
||||
ansible-playbook playbooks/raspi.yml
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Terraform — import existing infrastructure state
|
||||
|
||||
**Goal:** Bring existing Proxmox VMs and DNS records under Terraform state management,
|
||||
enabling drift detection.
|
||||
|
||||
### 2.1 — Configure credentials
|
||||
|
||||
```bash
|
||||
export TF_VAR_proxmox_endpoint="https://pve1.sweet.home:8006/"
|
||||
export TF_VAR_proxmox_api_token_id="terraform@pve!tf"
|
||||
export TF_VAR_proxmox_api_token_secret="<token>"
|
||||
```
|
||||
|
||||
Or create `terraform/proxmox/terraform.tfvars` (this file is git-ignored):
|
||||
```hcl
|
||||
proxmox_endpoint = "https://pve1.sweet.home:8006/"
|
||||
proxmox_api_token_id = "terraform@pve!tf"
|
||||
proxmox_api_token_secret = "<token>"
|
||||
```
|
||||
|
||||
### 2.2 — Initialise Terraform workspaces
|
||||
|
||||
```bash
|
||||
cd terraform/proxmox && terraform init
|
||||
cd ../dns && terraform init
|
||||
```
|
||||
|
||||
### 2.3 — Verify existing resource blocks match live state
|
||||
|
||||
The `terraform/proxmox/` workspace already has resource blocks for existing VMs
|
||||
(`docker.tf`, `nix-cache.tf`, `server-nixos.tf`, etc.) — these were generated from
|
||||
live state previously. Verify they're accurate:
|
||||
|
||||
```bash
|
||||
cd terraform/proxmox
|
||||
# Check if state is already populated (it may not be on first run)
|
||||
terraform state list
|
||||
|
||||
# If state is empty, import each existing VM:
|
||||
terraform import proxmox_virtual_environment_vm.docker pve/qemu/103
|
||||
terraform import proxmox_virtual_environment_vm.nix-cache pve/qemu/105
|
||||
terraform import proxmox_virtual_environment_vm.server-nixos pve/qemu/104
|
||||
# Add imports for any other VMs not yet in state
|
||||
# VMID reference: check 'qm list' on pve1
|
||||
```
|
||||
|
||||
### 2.4 — Run plan and reconcile any drift
|
||||
|
||||
```bash
|
||||
terraform plan
|
||||
```
|
||||
|
||||
If plan shows changes, the `.tf` resource block differs from live state.
|
||||
Either update the `.tf` file to match (if the live state is correct) or apply
|
||||
to enforce the declared state. **Do not apply to production VMs without reviewing
|
||||
every planned change.**
|
||||
|
||||
### 2.5 — DNS workspace
|
||||
|
||||
```bash
|
||||
cd terraform/dns
|
||||
# Configure Dynu credentials if not already set:
|
||||
export TF_VAR_dynu_api_key="<api_key>"
|
||||
terraform plan
|
||||
```
|
||||
|
||||
The DNS workspace already has the existing records imported. Verify no drift.
|
||||
|
||||
### 2.6 — Verify drift detection works end-to-end
|
||||
|
||||
Manually introduce a small change in the PVE UI (e.g. add a VM note) and run:
|
||||
```bash
|
||||
./scripts/drift-detect.sh --terraform
|
||||
```
|
||||
Confirm it detects the change. Revert it or update the `.tf` to match.
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: NixOS flake migration
|
||||
|
||||
**Goal:** Move the `nixos` repo into this mono-repo while keeping the flake fully functional.
|
||||
|
||||
### 3.1 — Import with git subtree
|
||||
|
||||
```bash
|
||||
git subtree add --prefix=nixos /home/wayne/repos/nixos main --squash
|
||||
git push
|
||||
```
|
||||
|
||||
### 3.2 — Update the remote flake URL on all NixOS hosts
|
||||
|
||||
The `Switch-nix` and `Test-nix` aliases on NixOS hosts reference the old Gitea URL.
|
||||
Update `nixos/variables.nix` (the `flakeUrl` variable) to point at the new path:
|
||||
|
||||
```nix
|
||||
# Old:
|
||||
flakeUrl = "git+ssh://gitea.lan.ddnsgeek.com/wayne/nixos";
|
||||
# New:
|
||||
flakeUrl = "git+ssh://gitea.lan.ddnsgeek.com/wayne/infrastructure?dir=nixos";
|
||||
```
|
||||
|
||||
Deploy the change to the `docker` LXC first (lowest risk), then remaining hosts.
|
||||
|
||||
### 3.3 — Validate flake builds from subdirectory
|
||||
|
||||
```bash
|
||||
nix build ./nixos#proxmox-docker
|
||||
nix build ./nixos#proxmox-nix-cache
|
||||
```
|
||||
|
||||
### 3.4 — Archive the old nixos repo
|
||||
|
||||
Once all hosts are rebuilding successfully from the new URL, archive the old repo:
|
||||
Gitea → `wayne/nixos` → Settings → Danger Zone → Archive Repository.
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Activate drift detection CI
|
||||
|
||||
**Goal:** Daily automated drift checks running in Gitea CI with Gotify notifications.
|
||||
|
||||
### 4.1 — Verify secrets are configured (from Phase 0.4)
|
||||
|
||||
Trigger the `drift-detect` workflow manually from Gitea CI → Actions → Drift Detection.
|
||||
|
||||
### 4.2 — Confirm first successful run
|
||||
|
||||
Review the step summary in Gitea. Expected outcome on a clean run:
|
||||
- Terraform: "No changes. Infrastructure matches configuration."
|
||||
- Ansible: `changed=0` on all hosts
|
||||
|
||||
### 4.3 — Test the notification path
|
||||
|
||||
Temporarily modify a `.tf` resource (without applying) and trigger a manual run.
|
||||
Confirm the Gotify notification arrives.
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: Documentation pipeline
|
||||
|
||||
**Goal:** Auto-generated service catalog and host inventory, with both internal and external sites building on every push.
|
||||
|
||||
### 5.1 — Install docs dependencies
|
||||
|
||||
```bash
|
||||
pip install mkdocs mkdocs-material jinja2 pyyaml
|
||||
```
|
||||
|
||||
### 5.2 — Test local build
|
||||
|
||||
```bash
|
||||
./scripts/docs-build.sh
|
||||
./scripts/docs-build.sh --serve # preview at http://localhost:8000
|
||||
```
|
||||
|
||||
### 5.3 — Implement service catalog generator
|
||||
|
||||
Create `scripts/generate-service-catalog.py`:
|
||||
- Parses all `docker-compose.yml` files under `stacks/`
|
||||
- Extracts service name, image, Traefik domain (from labels), network memberships
|
||||
- Outputs `docs/generated/service-catalog.md`
|
||||
|
||||
Template for the output:
|
||||
```markdown
|
||||
# Service Catalog
|
||||
Auto-generated from compose files in stacks/.
|
||||
|
||||
| Service | Image | Domain | Stack |
|
||||
|---------|-------|--------|-------|
|
||||
| traefik | traefik:latest | — | stacks/docker/core/traefik |
|
||||
| nextcloud-webapp | nextcloud:production | nextcloud.lan.ddnsgeek.com | stacks/docker/apps/nextcloud |
|
||||
...
|
||||
```
|
||||
|
||||
### 5.4 — Configure docs deploy target
|
||||
|
||||
Set these Gitea secrets (from Phase 0.4):
|
||||
- `PAGES_SSH_KEY` — key for rsync to the docs server
|
||||
- `PAGES_HOST` — user@host for rsync
|
||||
- `PAGES_PATH_INTERNAL` — path on the docs server for internal site
|
||||
|
||||
Or use Gitea Pages if self-hosted Gitea supports it.
|
||||
|
||||
---
|
||||
|
||||
## Phase 6: Cleanup and archive
|
||||
|
||||
**Goal:** Retire the old repos cleanly.
|
||||
|
||||
### 6.1 — Verify nothing still references old repos
|
||||
|
||||
- All NixOS hosts rebuilding from `infrastructure?dir=nixos` ✓
|
||||
- Docker stack deployed from `stacks/docker/` ✓
|
||||
- Ansible roles replacing debian-configuration scripts ✓
|
||||
|
||||
### 6.2 — Archive deprecated repos
|
||||
|
||||
Archive these repos on Gitea (makes them read-only, preserves history):
|
||||
- `wayne/debian-configuration` — content dissolved into `ansible/roles/`
|
||||
- `wayne/docker` — content migrated to `stacks/docker/`
|
||||
- `wayne/raspi` — content migrated to `stacks/raspi/`
|
||||
- `wayne/nixos` — migrated to `infrastructure/nixos/`
|
||||
|
||||
Do NOT delete them — they have git history that may be useful for reference.
|
||||
|
||||
### 6.3 — Update any external references
|
||||
|
||||
- CLAUDE.md files in the old repos — add a note pointing to the new mono-repo
|
||||
- Any README links, bookmarks, or documentation that references the old repo URLs
|
||||
|
||||
---
|
||||
|
||||
## Ongoing: Adding new infrastructure
|
||||
|
||||
### Adding a new Proxmox VM
|
||||
|
||||
1. Add a resource block in `terraform/proxmox/<name>.tf`
|
||||
2. Run `terraform plan` → `terraform apply`
|
||||
3. Add the host to `ansible/inventory/hosts.yml`
|
||||
4. If NixOS: add to `nixos/hosts/<name>/host.nix`, run `./scripts/sync-host-keys.sh`
|
||||
5. Deploy: `nixos-rebuild switch --flake ./nixos#<target> --target-host <hostname>`
|
||||
|
||||
### Adding a new Docker service
|
||||
|
||||
1. Add a `docker-compose.yml` under `stacks/docker/apps/<service>/` or `stacks/docker/monitoring/<service>/`
|
||||
2. Add Traefik labels, networks, secrets (following existing service patterns)
|
||||
3. Update `stacks/docker/default-environment.env` if new env vars are needed
|
||||
4. Test: `./services-up.sh config` (validates interpolation), then `./services-up.sh up -d <service>`
|
||||
|
||||
### Rotating credentials
|
||||
|
||||
- Proxmox API token: create new, update Gitea secrets, delete old
|
||||
- Ansible SSH key: replace `ANSIBLE_SSH_KEY` secret, re-add public key to all hosts
|
||||
- Dynu API key: update `DYNU_API_KEY` secret in Gitea
|
||||
- NixOS SOPS secrets: use `nixos/scripts/rotate-admin-key.sh`
|
||||
@@ -0,0 +1,25 @@
|
||||
# Infrastructure Documentation
|
||||
|
||||
Internal documentation for the `sweet.home` homelab. Contains full topology, network
|
||||
details, credentials catalog (names only), and operational runbooks.
|
||||
|
||||
## Quick links
|
||||
|
||||
- [Architecture](architecture.md) — host map and service relationships
|
||||
- [Network Topology](network-topology.md) — VLANs, IPs, firewall rules
|
||||
- [Implementation Plan](implementation-plan.md) — phased rollout roadmap
|
||||
- [Host Inventory](generated/host-inventory.md) — auto-generated from Ansible inventory
|
||||
- [Service Catalog](generated/service-catalog.md) — auto-generated from compose files
|
||||
|
||||
## Repo structure
|
||||
|
||||
```
|
||||
infrastructure/
|
||||
├── terraform/ Infrastructure state (Proxmox VMs, DNS, Pi-hole)
|
||||
├── ansible/ Configuration management (non-NixOS hosts)
|
||||
├── nixos/ NixOS flake (all NixOS hosts)
|
||||
├── stacks/
|
||||
│ ├── docker/ Main app stack (Traefik, Nextcloud, Passbolt, Gitea…)
|
||||
│ └── raspi/ Raspberry Pi edge monitoring stack
|
||||
└── docs/ This documentation
|
||||
```
|
||||
@@ -0,0 +1,53 @@
|
||||
# Network Topology
|
||||
|
||||
## LAN segments
|
||||
|
||||
| Subnet | VLAN | Purpose |
|
||||
|--------|------|---------|
|
||||
| `192.168.2.0/24` | (untagged/management) | Primary LAN — all host management interfaces |
|
||||
| `192.168.10.0/29` | VLAN 10 | HA cluster replication (Corosync ring, DRBD) — internal to pve1 |
|
||||
| `192.168.20.0/24` | VLAN 20 | HA storage-client (iSCSI/NFS) — internal to pve1 |
|
||||
|
||||
## Host IP assignments
|
||||
|
||||
| Host | IP | Role |
|
||||
|------|----|------|
|
||||
| `pve1.sweet.home` | (assigned by router) | Proxmox hypervisor |
|
||||
| `domain-controller.sweet.home` | `192.168.2.253` | FreeIPA server |
|
||||
| `pihole.sweet.home` | see Pi-hole admin | DNS/DHCP server |
|
||||
| `docker.sweet.home` | `192.168.2.225` | Docker app stack host |
|
||||
| `nix-cache.sweet.home` | `192.168.2.224` | Nix binary cache |
|
||||
| `pxe-boot.sweet.home` | `192.168.2.223` | PXE/TFTP/HTTP boot server |
|
||||
| `tailscale-router.sweet.home` | `192.168.2.222` | Tailscale subnet router |
|
||||
| `tor-relay.sweet.home` | `192.168.2.221` | Tor middle relay |
|
||||
| `ha-server-1.sweet.home` | `192.168.2.228` | HA cluster node 1 |
|
||||
| `ha-server-2.sweet.home` | `192.168.2.227` | HA cluster node 2 |
|
||||
| HA LAN VIP (Pacemaker) | `192.168.2.229` | Floating NFS/service VIP |
|
||||
| HA storage VIP (Pacemaker) | `192.168.20.229` | Docker iSCSI/NFS floating VIP |
|
||||
| `raspberrypi.tail13f623.ts.net` | Tailscale | Raspberry Pi (reachable via Tailnet) |
|
||||
|
||||
## DNS architecture
|
||||
|
||||
Pi-hole has been decommissioned. FreeIPA is now the sole DNS server for the LAN.
|
||||
|
||||
```
|
||||
All LAN clients → FreeIPA (domain-controller.sweet.home — 192.168.2.253)
|
||||
│
|
||||
├── *.sweet.home → IPA integrated DNS (authoritative)
|
||||
└── Everything else → upstream resolvers (forwarded by FreeIPA)
|
||||
```
|
||||
|
||||
PXE DHCP is handled by the `pxe-boot` LXC (dnsmasq in proxy mode for PXE chainloading only).
|
||||
General DHCP is handled by the router.
|
||||
|
||||
## External access
|
||||
|
||||
- Domain: `*.lan.ddnsgeek.com` → Dynamic DNS via Dynu → home WAN IP
|
||||
- TLS: LetsEncrypt via Traefik ACME (HTTP challenge)
|
||||
- Tailscale VPN: subnet router at `192.168.2.222` bridges Tailnet to LAN
|
||||
|
||||
## Proxmox firewall
|
||||
|
||||
Default-deny inbound on all Proxmox nodes.
|
||||
Management access (SSH port 22, web UI port 8006) from `192.168.2.0/24` only.
|
||||
See `ansible/roles/proxmox-hardening/templates/cluster-fw.j2` for the ruleset.
|
||||
@@ -0,0 +1,64 @@
|
||||
# Runbook: Drift Detection
|
||||
|
||||
How to interpret and respond to drift detected by the CI pipeline or local script.
|
||||
|
||||
## What is drift?
|
||||
|
||||
**Terraform drift** — a Proxmox VM, DNS record, or Pi-hole entry exists in a different
|
||||
state than what's declared in the Terraform workspace. Common causes:
|
||||
- A VM was resized or reconfigured manually in the PVE web UI
|
||||
- A DNS record was added directly via Dynu or Pi-hole admin
|
||||
- A VM was deleted outside of Terraform
|
||||
|
||||
**Ansible drift** — a host's configuration differs from what the Ansible role declares.
|
||||
Common causes:
|
||||
- A config file was edited by hand on the host
|
||||
- A package was installed/removed manually
|
||||
- A service was stopped/disabled without updating the role
|
||||
|
||||
## CI workflow
|
||||
|
||||
The `drift-detect` workflow runs daily at 06:00 AEST. If drift is detected, a Gotify
|
||||
notification is sent to the push notification dashboard. The step summary in Gitea CI
|
||||
contains the full `terraform plan` diff.
|
||||
|
||||
## Responding to Terraform drift
|
||||
|
||||
```bash
|
||||
# See exactly what changed
|
||||
cd terraform/proxmox
|
||||
terraform plan
|
||||
|
||||
# Option A: The manual change was intentional — update the .tf file to match reality,
|
||||
# then run terraform apply to reconcile state
|
||||
vim proxmox/docker.tf # update the resource to match current state
|
||||
terraform apply
|
||||
|
||||
# Option B: The manual change was accidental — apply to restore declared state
|
||||
terraform apply # will revert the manual change
|
||||
```
|
||||
|
||||
## Responding to Ansible drift
|
||||
|
||||
```bash
|
||||
# See exactly what would change
|
||||
cd ansible
|
||||
ansible-playbook playbooks/site.yml --check --diff --limit <drifted_host>
|
||||
|
||||
# Apply to restore declared state
|
||||
ansible-playbook playbooks/site.yml --limit <drifted_host>
|
||||
```
|
||||
|
||||
## Required Gitea secrets
|
||||
|
||||
The following secrets must be configured in the Gitea repo for CI drift detection to work:
|
||||
|
||||
| Secret | Used by |
|
||||
|--------|---------|
|
||||
| `PROXMOX_ENDPOINT` | Terraform proxmox workspace |
|
||||
| `PROXMOX_API_TOKEN_ID` | Terraform proxmox workspace |
|
||||
| `PROXMOX_API_TOKEN_SECRET` | Terraform proxmox workspace |
|
||||
| `DYNU_API_KEY` | Terraform dns workspace |
|
||||
| `ANSIBLE_SSH_KEY` | Ansible drift check |
|
||||
| `GOTIFY_URL` | Drift notifications |
|
||||
| `GOTIFY_TOKEN` | Drift notifications |
|
||||
@@ -0,0 +1,59 @@
|
||||
# Runbook: Provisioning a New Host
|
||||
|
||||
Steps for adding a new machine to the infrastructure.
|
||||
|
||||
## NixOS host on Proxmox
|
||||
|
||||
```bash
|
||||
# 1. Use the PXE boot menu or installer ISO
|
||||
# The auto-installer at pxe-boot.sweet.home presents all flake targets.
|
||||
|
||||
# 2. Add to nixos/ flake if a new build-type is needed
|
||||
# Otherwise pick the closest existing target (e.g. proxmox-minimal)
|
||||
|
||||
# 3. Add SSH host key to clan vars (enables SOPS decryption)
|
||||
cd nixos
|
||||
./scripts/sync-host-keys.sh <target>
|
||||
|
||||
# 4. Create the VM/LXC via Terraform
|
||||
cd ../terraform/proxmox
|
||||
# Add a resource block in the appropriate .tf file
|
||||
terraform plan
|
||||
terraform apply
|
||||
|
||||
# 5. Deploy NixOS
|
||||
nixos-rebuild switch --flake ./nixos#<target> --target-host <hostname>
|
||||
```
|
||||
|
||||
## Non-NixOS host (Proxmox VM)
|
||||
|
||||
```bash
|
||||
# 1. Create VM in Proxmox (manually or via terraform)
|
||||
# 2. Add to ansible/inventory/hosts.yml with correct group
|
||||
# 3. Add SSH key, verify connectivity:
|
||||
ansible-playbook ansible/playbooks/ping.yml --limit <hostname>
|
||||
# 4. Apply the role:
|
||||
ansible-playbook ansible/playbooks/<role>.yml --limit <hostname> --check --diff
|
||||
ansible-playbook ansible/playbooks/<role>.yml --limit <hostname>
|
||||
```
|
||||
|
||||
## Adding Ansible inventory entry
|
||||
|
||||
```yaml
|
||||
# ansible/inventory/hosts.yml — under the appropriate group:
|
||||
new-host.sweet.home:
|
||||
ansible_user: wayne
|
||||
# any role-specific variables
|
||||
```
|
||||
|
||||
## IPA enrollment (for any new host)
|
||||
|
||||
```bash
|
||||
# On the new host (after OS install):
|
||||
sudo ipa-client-install \
|
||||
--server=domain-controller.sweet.home \
|
||||
--domain=sweet.home \
|
||||
--realm=SWEET.HOME \
|
||||
--principal=admin \
|
||||
--mkhomedir
|
||||
```
|
||||
@@ -0,0 +1,52 @@
|
||||
# Runbook: Proxmox VE Hardening
|
||||
|
||||
Applies Stage 1 hardening to a fresh Proxmox VE node using the `proxmox-hardening` Ansible role.
|
||||
|
||||
## Pre-conditions
|
||||
|
||||
- Fresh PVE install with root SSH access
|
||||
- SSH key already added to root's `authorized_keys`
|
||||
- Host added to `ansible/inventory/hosts.yml` under the `proxmox` group
|
||||
|
||||
## Steps
|
||||
|
||||
```bash
|
||||
# 1. Verify connectivity
|
||||
cd ansible
|
||||
ansible-playbook playbooks/ping.yml --limit <hostname>
|
||||
|
||||
# 2. Dry-run to preview changes
|
||||
ansible-playbook playbooks/proxmox.yml --limit <hostname> --check --diff
|
||||
|
||||
# 3. Review the diff output, then apply
|
||||
ansible-playbook playbooks/proxmox.yml --limit <hostname>
|
||||
|
||||
# 4. Verify hardening state
|
||||
ansible-playbook playbooks/proxmox.yml --limit <hostname> --tags audit --check
|
||||
```
|
||||
|
||||
## What gets applied
|
||||
|
||||
1. **APT repos** — enterprise source disabled, no-subscription source enabled
|
||||
2. **SSH hardening** — `PermitRootLogin prohibit-password`, `PasswordAuthentication no`, fail2ban
|
||||
3. **Datacenter firewall** — default-deny inbound; SSH (22) and web UI (8006) from management CIDR only
|
||||
4. **Unattended upgrades** — security-only origins, no auto-reboot
|
||||
5. **Named admin user** — `<proxmox_admin_username>@pve` with Administrator role (one-time)
|
||||
6. **IPA sudo** — `admins` group gets NOPASSWD sudo for shell and PVE tools
|
||||
7. **Subscription nag** — cosmetic patch to remove the nag dialog
|
||||
|
||||
## Variables to set per-host
|
||||
|
||||
In `ansible/inventory/hosts.yml`:
|
||||
```yaml
|
||||
pve1.sweet.home:
|
||||
proxmox_mgmt_cidr: "192.168.2.0/24"
|
||||
proxmox_admin_username: wayne
|
||||
```
|
||||
|
||||
## After hardening
|
||||
|
||||
- Log into the PVE web UI as the named admin, change the initial password
|
||||
- Enable TOTP/2FA for both the named admin and root@pam
|
||||
- Verify with the audit script: `cd /home/wayne/repos/debian-configuration && proxmox/scripts/audit.sh`
|
||||
(will migrate to an Ansible audit task in a future iteration)
|
||||
@@ -0,0 +1,15 @@
|
||||
site_name: "Homelab"
|
||||
site_description: "Self-hosted infrastructure overview"
|
||||
docs_dir: external
|
||||
|
||||
nav:
|
||||
- Home: index.md
|
||||
- Services: services.md
|
||||
|
||||
theme:
|
||||
name: material
|
||||
palette:
|
||||
scheme: default
|
||||
primary: indigo
|
||||
|
||||
# No internal IPs, hostnames, or credential references in external docs.
|
||||
@@ -0,0 +1,30 @@
|
||||
site_name: "Infrastructure — Internal Docs"
|
||||
site_description: "Internal documentation for the sweet.home homelab infrastructure"
|
||||
docs_dir: internal
|
||||
|
||||
nav:
|
||||
- Home: index.md
|
||||
- Architecture: architecture.md
|
||||
- Network Topology: network-topology.md
|
||||
- Runbooks:
|
||||
- Proxmox Hardening: runbooks/proxmox-hardening.md
|
||||
- FreeIPA Setup: runbooks/freeipa-setup.md
|
||||
- New Host Provisioning: runbooks/new-host.md
|
||||
- Drift Detection: runbooks/drift-detection.md
|
||||
- Generated:
|
||||
- Host Inventory: generated/host-inventory.md
|
||||
- Service Catalog: generated/service-catalog.md
|
||||
|
||||
theme:
|
||||
name: material
|
||||
palette:
|
||||
scheme: slate
|
||||
primary: indigo
|
||||
|
||||
validation:
|
||||
nav:
|
||||
omitted_files: ignore
|
||||
|
||||
exclude_docs: |
|
||||
README.md
|
||||
*.example
|
||||
Reference in new issue
Block a user