docker/README.md

# Homelab Docker + Terraform Inventory Repository

This repository is both:

1. **operational** (Docker Compose application/runtime definition), and
2. **documentary/inventory-oriented** (Terraform capture of Proxmox VMs, host metadata, and selected Docker objects).

If you only read one section, read **[Source-of-truth boundaries](docs/source-of-truth.md)** first.

---

## Quick navigation

- Architecture overview: [docs/architecture.md](docs/architecture.md)
- Repository layout: [docs/repo-structure.md](docs/repo-structure.md)
- Source-of-truth boundaries and guardrails: [docs/source-of-truth.md](docs/source-of-truth.md)
- Docker environment composition and `services-up.sh`: [docs/docker-environment.md](docs/docker-environment.md)
- Terraform workflows (brownfield import/reconciliation): [docs/terraform-workflows.md](docs/terraform-workflows.md)
- Infrastructure inventory intent and outputs: [docs/infrastructure-inventory.md](docs/infrastructure-inventory.md)
- Dynu DNS read-only inventory workflow: [docs/dynu-dns-inventory.md](docs/dynu-dns-inventory.md)
- Generated host topology doc: [docs/generated/host-topology.md](docs/generated/host-topology.md)
- Ansible bootstrap workflows: [docs/ansible-workflows.md](docs/ansible-workflows.md)
- Deployment prerequisites and secrets setup: [docs/deployment-prerequisites.md](docs/deployment-prerequisites.md)
- Secrets inventory: [docs/security-secrets.md](docs/security-secrets.md)

Codex helper scripts:

- Initial Codex environment/bootstrap setup: [scripts/codex-setup.sh](scripts/codex-setup.sh)
- Codex environment maintenance/refresh: [scripts/codex-maintenance.sh](scripts/codex-maintenance.sh)

Infrastructure subtrees:

- Ansible foundation docs: [infrastructure/ansible/README.md](infrastructure/ansible/README.md)
- Terraform root docs: [infrastructure/terraform/README.md](infrastructure/terraform/README.md)
- Terraform Docker mirror: [infrastructure/terraform/docker/README.md](infrastructure/terraform/docker/README.md)
- Terraform Proxmox inventory: [infrastructure/terraform/proxmox/README.md](infrastructure/terraform/proxmox/README.md)

---

## Operating model

### Docker Compose (runtime authority)

- Compose files under `core/`, `apps/`, and `monitoring/` describe runtime services.
- `services-up.sh` composes the environment by discovering compose files and applying common env/network inputs.
- For service runtime behavior, start from Compose files and `services-up.sh` (not Terraform).


### Ansible (bootstrap foundation)

- Ansible under `infrastructure/ansible/` is a phase-1 foundation for inventory/configuration scaffolding.
- It supports safe validation (inventory parsing and playbook syntax checks) while hosts/devices are onboarded gradually.
- It does not replace Compose runtime authority or Terraform reconciliation authority at this stage.

### Terraform (inventory and reconciliation authority)

- Terraform under `infrastructure/terraform/` is used to codify and reconcile existing infrastructure.
- Current repo usage emphasizes **brownfield import-first workflows** and safe reconciliation.
- Terraform captures:
  - Proxmox VM configuration for existing VMs.
  - Physical host metadata in locals/outputs.
  - Documentation-oriented Docker container mirroring (limited, selective).

Terraform here is **not** a replacement for Docker Compose deployment.

---

## Guardrails

- Do not run destructive Terraform commands casually.
- Do not treat generated Terraform config as final without manual review.
- Do not commit real secrets, credentials, or local state.
- Keep one-resource-per-file patterns where already established in Terraform subdirectories.
- Prefer shaping outputs for documentation/tooling consumption over dumping raw provider objects.

See [docs/source-of-truth.md](docs/source-of-truth.md) and [docs/terraform-workflows.md](docs/terraform-workflows.md) for concrete do/don't guidance.

---

## High-level architecture

```mermaid
flowchart TB
  Internet((Internet Clients)) -->|HTTPS 443 / HTTP 80| Traefik[Traefik Ingress\nACME TLS + Security Middlewares]

  subgraph DockerHost[Primary Docker Host]
    Traefik
    Authelia[Authelia SSO / ForwardAuth]
    CrowdSec[CrowdSec + Traefik Bouncer]
    ErrPages[Error Pages Fallback]

    subgraph Apps[Business / User Applications]
      Nextcloud[Nextcloud]
      Passbolt[Passbolt]
      Gitea[Gitea]
      FamilyTree[Gramps Web]
      Searxng[SearXNG]
    end

    subgraph Ops[Operations & Monitoring]
      Grafana[Grafana]
      Prometheus[Prometheus]
      InfluxDB[InfluxDB]
      NodeRED[Node-RED]
      Portainer[Portainer]
      UptimeKuma[Uptime Kuma]
      Gotify[Gotify Notifications]
    end
  end

  Traefik --> Apps
  Traefik --> Ops
  Traefik -->|ForwardAuth for selected routes| Authelia
  Traefik -->|Threat decisions| CrowdSec
  Traefik -->|4xx/5xx fallback| ErrPages

  Prometheus --> Grafana
  Prometheus --> Gotify
```

For request-flow and network detail, see [docs/architecture.md](docs/architecture.md).

### Regenerating architecture docs (Prometheus + Dynu DNS)

```bash
# Refresh Dynu live inventory and generated resources/import helpers
cd infrastructure/terraform/dynu
terraform apply -refresh-only
python3 scripts/generate-brownfield-records.py --overwrite

# Regenerate architecture docs from Prometheus + Dynu inventory
cd ../../..
python3 scripts/render_prometheus_docs.py \
  --inventory-file docs/runtime/prometheus-inventory.json \
  --dynu-dns-inventory-file infrastructure/terraform/dynu/generated/dynu_dns_records_inventory.json
```

---

## Codex setup and maintenance scripts

The repository includes helper scripts for Codex sessions that need local tooling and safe placeholder secret material for validation-only workflows:

- `scripts/codex-setup.sh`
  - Installs baseline CLI dependencies (shell/yaml/terraform/ansible tooling).
  - Prepares `secrets/stack-secrets.env` from templates and creates dummy file-based secret placeholders based on `secrets/inventory.json`.
  - Installs/refreshed baseline Ansible collections when `infrastructure/ansible/collections/requirements.yml` is present.
  - Runs safe Ansible bootstrap checks (version, inventory parse, playbook syntax check) without live connectivity operations.
  - Prints installed tool versions for quick verification.

- `scripts/codex-maintenance.sh`
  - Refreshes Python-based linting/automation tooling.
  - Reconciles placeholder secret files against current `secrets/inventory.json` (creates missing, removes stale).
  - Rebuilds `secrets/stack-secrets.env` with dummy values for compose-config validation.
  - Refreshes Ansible collections and repeats safe inventory/syntax validation checks.

Both scripts are intended for local validation environments and should not be treated as production provisioning automation.