AuditPic is a Spring Boot 3 backend (signed-photo SaaS) deployed to k3s via Helm. Images are built by GitHub Actions and pushed to GHCR; CD runs helm upgrade against the project namespace.

| | Production | Staging | |---|---|---| | Namespace | audit-pic | audit-pic-staging | | Ingress host | api.auditpic.com | auditpic.staging.glorylabs.nl | | Backend image | ghcr.io/mahmoudholding/auditpic/audit-pic-backend | same | | Helm chart | helm/audit-pic | helm/audit-pic (values-staging.yaml) | | Backend port | 8093 | 8093 |

Cluster operations & runbooks

AuditPic deploys to the shared single-node k3s staging cluster. Before touching staging, read the shared ops docs — they are the source of truth for the cluster's quirks:

Staging bring-up recipe — proven step-by-step to get a backend green (sealed secret → two-user Postgres + runtime CREATEROLE → Redis RESP2 → NetworkPolicy → actuator probe paths → manual CD dispatch when CI ends failure on a non-blocking job). AuditPic's two-user Postgres pattern is one of the confirmed cases.
k3s cluster / ops reference — node facts (~4 GB, CPU-bound), staggered one-at-a-time bring-up, memory budget, TLS/cert-manager state.
Node-reboot → PV recovery runbook — after a VPS reboot the node re-registers under a new name and strands every local-path PV; this is the recovery (quiesce → recreate stranded PVCs → re-run the CREATEROLE grant for the auditpic Flyway role → staggered bring-up).

CI/CD (automatic)

PR merged to develop → staging auto-deploys (namespace audit-pic-staging)
PR merged to main    → production auto-deploys (namespace audit-pic)

CI builds + pushes the image (.github/workflows/ci-backend.yml), then CD (.github/workflows/deploy-backend.yml, "CD — Backend") runs on CI success.

⚠️ CD often skips because CI ends failure on a non-blocking job (Trivy / Flutter / Maestro) even when Build & Push Image succeeded. When that happens, deploy the built image manually:

gh workflow run "CD — Backend" --ref develop \
  -f image_tag=sha-<develop-head-7> \
  -f environment=staging

Deploying manually with Helm

# Staging
helm upgrade --install audit-pic helm/audit-pic \
  --namespace audit-pic-staging \
  -f helm/audit-pic/values-staging.yaml \
  --set image.backend.tag=sha-<commit>

# Production
helm upgrade --install audit-pic helm/audit-pic \
  --namespace audit-pic \
  --set image.backend.tag=sha-<commit>

Secrets — 1Password → Sealed Secrets → k3s

Sealed from the AuditPic vault (mahmoudholdingbv.1password.com), items audit-pic-secrets-{staging,production}:

| Field | Purpose | |---|---| | database-flyway-password | schema-owner migration user (auditpic) | | database-app-password | DML-only app runtime user (auditpic_app) | | jwt-secret | JWT signing (≥32 chars) | | hmac-secret-v1 | HMAC rotation v1 key | | field-encryption-key | AES-256-GCM field encryption (32-byte base64) | | minio-access-key / minio-secret-key | object storage | | ai-detection-api-key | Hive AI detection (or disabled) |

Regenerate the env-specific sealed manifest (needs a live op session or OP_SERVICE_ACCOUNT_TOKEN):

OP_SERVICE_ACCOUNT_TOKEN=<token> ./scripts/seal-secrets.sh staging   # or: production
git add k8s/sealed-secrets/backend-secrets-staging.yaml
git commit -m "chore: update sealed secrets (staging)"
git push

CD applies k8s/sealed-secrets/backend-secrets-${ENV}.yaml before helm upgrade. Never sed-rename a SealedSecret's namespace — they are namespace-scoped and the cluster can't decrypt a re-targeted manifest.

Verify

kubectl get pods -n audit-pic-staging          # backend should be 1/1
# health (no public DNS for staging — use Host-header curl or port-forward):
curl -k -H "Host: auditpic.staging.glorylabs.nl" https://136.144.174.219/actuator/health

{"status":"UP"} = healthy. The k8s probes hit /actuator/health/liveness and /actuator/health/readiness, so SecurityConfig must permit /actuator/health/** (see recipe step 6b).

Rollback

helm rollback audit-pic -n audit-pic            # or audit-pic-staging

Reacties

Nog geen reacties

Athena — auditPic/deployment.md

Deployment — AuditPic