| Component | Technology | Location | |-----------|-----------|---------| | Backend API | Spring Boot on k3s | TransIP VPS 136.144.174.219 | | Database | PostgreSQL 15 (Bitnami Helm chart) | Same VPS, auditpicdb | | Object storage | MinIO | Same VPS, ports 9004/9005 | | Container registry | GHCR ghcr.io/mahmoudholding/auditpic/* | GitHub | | Secrets | 1Password → Sealed Secrets | 1Password vault + k3s |

3. Backup Strategy

3.1 PostgreSQL

Method: pg_dump compressed dump via CronJob in k3s.
Schedule: Hourly incremental (WAL archiving), daily full dump.
Retention: 7 daily + 4 weekly dumps.
Storage: Off-site MinIO bucket or S3-compatible remote (configure BACKUP_S3_ENDPOINT env var).
Validation: Weekly restore test to a staging namespace — automated via scripts/validate-backup.sh.

3.2 MinIO (Photo Objects)

Method: MinIO mc mirror to a secondary MinIO instance or S3-compatible bucket.
Schedule: Continuous replication (MinIO site replication) or hourly mirror job.
Retention: Objects retained according to the photo retention policy (default 365 days).

3.3 Kubernetes State

All k8s manifests are in git (helm/ and k8s/). The cluster state is fully reproducible from the repository.
Sealed Secrets are re-generated from 1Password; no cluster backup of secrets is needed.

4. Disaster Scenarios & Recovery Procedures

Scenario A — Single Pod / Deployment Failure

RTO: < 5 minutes (Kubernetes self-healing)

k3s restarts failed pods automatically.
If stuck: kubectl rollout restart deployment/audit-pic-backend -n auditpic
If image is broken: roll back to previous tag via Helm (helm rollback auditpic -n auditpic).

Scenario B — Database Corruption or Accidental Data Loss

RTO: ≤ 2 h | RPO: ≤ 1 h

Stop backend traffic: scale backend to 0 replicas.
Identify the latest clean backup (hourly dumps in the backup store).

Restore:

pg_restore -h <db-host> -U auditpic -d auditpicdb --clean /backups/auditpicdb_<timestamp>.dump

Validate: run Flyway repair, spot-check 5 random photo signatures.
Scale backend back to 1+ replicas.
Notify affected users if data loss > 0.

Scenario C — Full VPS Loss

RTO: ≤ 8 h | RPO: ≤ 1 h

Provision a new VPS (TransIP API or manual).
Install k3s: curl -sfL https://get.k3s.io | sh -
Restore Sealed Secrets from 1Password: OP_SERVICE_ACCOUNT_TOKEN=<token> ./scripts/seal-secrets.sh
Apply all Helm charts: helm upgrade --install auditpic ./helm/auditpic -n auditpic --create-namespace
Restore PostgreSQL dump (Scenario B, step 3).
Restore MinIO objects from mirror.
Update DNS A record to point to new VPS IP.
Validate end-to-end: upload a test photo, verify signature, check AI detection status.

Scenario D — Compromised Signing Keys

RTO: ≤ 2 h (key rotation, no data loss)

Rotate JWT_SECRET and HMAC_SECRET in 1Password vault.
Re-seal: OP_SERVICE_ACCOUNT_TOKEN=<token> ./scripts/seal-secrets.sh
Apply: kubectl apply -f k8s/sealed-secrets/
Roll restart backend: kubectl rollout restart deployment/audit-pic-backend -n auditpic
Impact: All existing JWT tokens are invalidated. Users must log in again.
Note: Existing photo signatures use the old HMAC key. The verify-signature endpoint will return false for all pre-rotation photos. Post-rotation re-signing is out of scope; advise affected users to re-upload if verification is required.

5. Communication Plan

| Audience | Channel | Trigger | |----------|---------|---------| | Development team | Internal chat incident channel | Any P1/P2 | | End users | In-app banner + email | Service unavailable > 2 h or data loss | | Supervisory authority | Email per GDPR Art.33 | Personal data breach |

6. BCP Test Schedule

| Test | Frequency | Owner | |------|-----------|-------| | Backup restore drill (Scenario B) | Monthly | On-call engineer | | Full DR simulation (Scenario C) | Quarterly | GloryLabs infrastructure lead | | Key rotation drill (Scenario D) | Semi-annual | GloryLabs security lead |

Test results are logged in reports/bcp-tests/ and linked to the GitHub Issue tracker.

7. Dependencies & Single Points of Failure

| Dependency | Risk | Mitigation | |-----------|------|-----------| | Single VPS | Full outage if VPS is lost | Backups off-site; recovery < 8 h via Scenario C | | GitHub GHCR | Cannot deploy new images | Cache last 3 image tags locally on VPS | | 1Password | Cannot unseal secrets | Emergency plaintext backup in encrypted USB (locked in office safe) | | Hive AI API | AI detection unavailable | Detection fails gracefully; aiDetectionStatus = FAILED; core verification still works |

Reacties

Nog geen reacties

Athena — auditPic/disaster-recovery.md

Business Continuity & Disaster Recovery Plan — AuditPic

1. Objectives

2. Architecture Overview