Athena — ops/analytics-posthog-plan.md

Analytics (PostHog) — Hosting + Integration Plan

STATUS: PROPOSAL — pending owner decision on hosting. No infra deployed, no SDK wired, no keys committed. This document only recommends a path and pre-bakes the integration so it's a small follow-up once hosting is decided.

Owner preference on record: self-hosted open-source if feasible, not cloud. Bottom line up front: PostHog self-hosted does NOT fit our current cluster. See §1.


0. TL;DR

| Question | Answer | |---|---| | Can we self-host PostHog OSS on our cluster today? | No. It needs ClickHouse + Kafka + Zookeeper + Postgres + Redis + MinIO + Django + workers (15+ containers). PostHog's own floor is 4 vCPU / 8 GB, realistically 16 GB+. Our only node is a 4 GB TransIP VPS at ~92% memory. It will not fit, full stop. | | Does the owner's "self-hosted OSS" goal survive? | Yes — just not with PostHog. A lighter self-hostable OSS analytics tool does fit our existing node. | | Recommendation | Self-host Umami (OSS, ~200–400 MB RAM, reuses our existing Postgres) for web analytics now → revisit PostHog later only if we get a bigger/separate host and actually need product-analytics depth (funnels, session replay, feature flags). PostHog Cloud free tier is the interim fallback only if the owner accepts cloud. | | First product to instrument | auditPic (webapp Next.js + Flutter app), host+key fully env/secret-driven. |


1. PostHog self-hosted OSS — the reality check

PostHog is product analytics, not just web analytics, and the OSS self-host stack reflects that. A self-hosted deployment runs 15+ containers:

  • Django web app + 2 Celery workers + plugin/capture server
  • ClickHouse (columnar event store — the RAM hog)
  • Kafka + Zookeeper (event ingestion pipeline)
  • PostgreSQL (app metadata)
  • Redis (cache/queues)
  • MinIO (object storage for replay/exports)

Stated requirements (2026):

  • PostHog docs floor: 4 vCPU / 8 GB RAM minimum.
  • Community/benchmark reality: ClickHouse + Kafka idle alone ≈ 1.5 GB; stable production reports land at 16 GB+, some say 32 GB for comfort.
  • PostHog has also de-emphasised small self-hosting — the hobby docker-compose path is explicitly "not for production scale," and they steer users toward Cloud or large k8s clusters.

Our infrastructure (from CLAUDE.md + capacity guardrail):

  • One k3s node: a 4 GB TransIP VPS (136.144.174.219), single node, local-path storage only, no Prometheus CRD.
  • Already running 5 products' staging/prod workloads and sitting at ~92% memory with a ~90% capacity guardrail (see docs/ops/node-capacity-guardrail-proposal.md).

Verdict: PostHog self-host is INFEASIBLE on the current cluster. Even its 8 GB floor is 2× our entire node, before any of our own services. Forcing it on would OOM the node and take down live products. Not a tuning problem — a fundamental size mismatch.


2. The three paths (pros / cons / pick)

Path A — Separate / bigger host for self-hosted PostHog

Stand up PostHog OSS on a new, dedicated box (≥ 8 GB, realistically 16 GB) or managed k8s, kept off the product cluster.

  • Pros: Owner gets the real thing — funnels, retention, session replay, feature flags, A/B, both web + product analytics in one tool. Stays OSS + self-hosted (meets the literal preference). GDPR-friendly (data stays on our infra, EU region).
  • Cons: New recurring cost (a 16 GB VPS ≫ our current spend) and a whole second cluster to operate, patch, back up, and secure — real ops burden for a 1-person shop. Overkill for current traffic (pre-revenue / low DAU).
  • When it makes sense: later, once a product has real user volume and we genuinely need funnels/replay/flags — not now.

Path B — Lighter self-hostable OSS analytics that DOES fit ✅ PICK

Run a lean OSS analytics tool on the existing cluster (or even alongside a product's Postgres).

| Tool | Stack | RAM (idle→load) | Fits our node? | Notes | |---|---|---|---|---| | Umami ✅ | Node.js + Postgres (reuse existing) | ~200→400 MB | Yes | Lightest. Privacy-first, cookieless, GDPR-friendly, no consent banner needed. MIT/OSS. | | Plausible | Elixir + ClickHouse | ~1–2 GB (ClickHouse) | Tight | Great UX but ClickHouse dependency adds a heavy component to a 4 GB node. | | Matomo | PHP + MySQL + archive cron | 2 GB min, 1–2 GB+ practical | No (too tight) | Most features (GA-replacement depth) but heaviest of the three; needs its own DB + cron headroom. |

  • Pros: Stays OSS + self-hosted (honours owner preference), fits today's node with no new spend, GDPR/EU by construction, one small container.
  • Cons: Web analytics only — no product funnels, session replay, or feature flags. (If we later need those, that's the trigger for Path A.)
  • Pick within B: Umami. It's the only one that fits comfortably at ~92% memory headroom, and it can reuse a product's existing Postgres rather than adding a database. Deploy in its own analytics namespace, behind the shared ingress (e.g. analytics.glorylabs.nl), staging-first per standing policy.

Path C — PostHog Cloud free tier (interim)

Use PostHog Cloud (EU region) free tier (~1M events/mo free).

  • Pros: Zero infra, real PostHog features immediately, EU data residency.
  • Cons: It's cloud — contradicts the owner's stated preference; data leaves our infra (a DPIA/processor-agreement item for GDPR). Free tier limits.
  • Use only if the owner explicitly relaxes the self-hosted requirement and wants PostHog's depth before investing in Path A.

Recommendation

  1. Now: self-host Umami on the existing cluster for web analytics (Path B). Honours the OSS/self-host preference, fits the node, no new cost.
  2. Keep PostHog warm: if/when a product needs product-analytics depth (funnels, replay, flags), go Path A on a dedicated ≥16 GB host — owner-gated, as a deliberate cost decision.
  3. Path C only as a stop-gap if the owner accepts cloud.

The SDK plan in §3 is written PostHog-first because that's what was asked and it keeps the door open, but the same event taxonomy maps cleanly onto Umami custom events if we go Path B first.


3. SDK integration plan — auditPic FIRST

auditPic = signed-photo SaaS. Two clients: a Next.js webapp and a Flutter mobile app. Instrument both with a host + key that are 100% env/secret-driven. No key is ever committed — staging/prod keys live in the AuditPic 1Password vault → Sealed Secrets → injected at build/runtime.

3.1 Config (never hardcode)

  • Webapp (posthog-js): read from public env at build:
    • NEXT_PUBLIC_POSTHOG_HOST (e.g. our self-hosted analytics host, or https://eu.posthog.com for Path C)
    • NEXT_PUBLIC_POSTHOG_KEY (project API key — public client key, still injected via env, never literal in repo)
  • Flutter (posthog_flutter): pass via --dart-define at build (POSTHOG_API_KEY, POSTHOG_HOST) wired through CI / Codemagic, sourced from 1Password. Never in pubspec.yaml or committed config.
  • Default to disabled when key is empty (no-op init) so dev/CI never emits events to a real project.

3.2 Privacy / GDPR

  • EU host only; IP anonymisation on; person_profiles: 'identified_only'.
  • Mask all PII in autocapture; never send photo content, file contents, or the signed image itself — only event metadata.
  • Honour cookie/consent state before init on the webapp.

3.3 Key events to capture (initial taxonomy)

Stable snake_case names, minimal non-PII properties:

| Event | Where | Key properties (no PII) | |---|---|---| | signup_completed | web + app | method, plan | | photo_captured | app (primary), web | source (camera/upload), device_type | | photo_signed | app/web | sign_method, duration_ms | | photo_uploaded | app/web | size_bucket, success, retry_count | | verification_viewed | web (verify-view) | result (valid/invalid/tampered), via (qr/link) |

Supporting: app_opened / page_viewed (autocapture or manual), login_succeeded, upload_failed (error_code). Funnel of interest: signup → photo_captured → photo_signed → photo_uploaded → verification_viewed.

3.4 Rollout order

  1. Owner decides hosting (this doc's open question).
  2. Seal the key into AuditPic staging vault item; wire env into webapp + Flutter builds (CI/Codemagic), key absent → analytics no-op.
  3. Add the 5 core events above behind a feature flag / env guard.
  4. Verify events land in staging project; only then promote to prod (owner-gated, per standing staging-first policy).
  5. Repeat taxonomy for the next product once auditPic is proven.

4. Open question for the owner (blocking)

Pick the hosting path:

  • B (recommended): self-host Umami now on the existing node (OSS, self-hosted, fits, free) — web analytics only.
  • A: stand up self-hosted PostHog on a new ≥16 GB host (real product analytics, new cost + ops) — later/owner-gated.
  • C: PostHog Cloud free tier interim (cloud — contradicts stated preference; needs processor agreement/DPIA).

Until then: nothing is deployed and no keys exist.


Sources

Reacties

Nog geen reacties