Athena — mahmoud-consultancy/compliance/processes/data-flow.md

Process Documentation

Process: Data Flow — Personal Data Lifecycle in InterimPlaza Project: InterimPlaza — Recruitment & Staffing Platform (Mahmoud Consultancy B.V.) Owner: GloryLabs Date: 2026-03-27


1. Purpose

This document describes how personal data moves through the InterimPlaza platform from collection to deletion. It covers all actors, systems, and data stores involved, and serves as the primary reference for GDPR Art. 30 record-keeping, DPIA §3 data flows, and incident investigation. It also illustrates the flow of vacancy data from external scraping sources through to candidate-facing matching results.


2. Scope

Systems involved:

  • Angular 20 frontend (candidate and employer web app)
  • Astro marketing/landing site (no personal data processing)
  • Spring Boot 3 REST API (backend, business logic, auth)
  • PostgreSQL (persistent data store — all personal data)
  • Redis (ephemeral session cache and match result cache)
  • Firecrawl API (external job scraping — no personal data sent)
  • Gmail SMTP / Google Workspace (transactional email)
  • GHCR (container images only — no personal data)
  • 1Password + Bitnami Sealed Secrets (secrets management — no personal data)
  • TransIP VPS 136.144.174.219, Netherlands (hosting infrastructure)

Data involved:

  • Candidate personal data: name, email, phone, CV, work history, skills, certifications, hourly/day rate
  • Employer data: company name, KvK number, contact person, email, phone, billing address
  • Vacancy data: title, description, skills, rate range, employer identity, source URL
  • Session tokens, match scores, audit log entries

People involved:

  • Candidates (IT professionals)
  • Employers / HR contacts
  • Platform administrators (GloryLabs, Mahmoud Consultancy staff)
  • Automated processes (scraper scheduler, cleanup jobs)

3. Trigger

The data flow is initiated by one of three events:

  1. A candidate or employer submits a registration form in the Angular app.
  2. The scheduled Firecrawl scraping job runs (configurable cron, default: every 6 hours).
  3. A candidate or employer logs in and interacts with the platform (profile update, vacancy search, match review).

4. Process Steps

4a — Candidate Registration & Profile Creation

| Step | Actor | Action | System | Output | |------|-------|--------|--------|--------| | 1 | Candidate | Completes registration form (name, email, password, phone) | Angular frontend | Form data submitted via HTTPS POST to /api/v1/auth/register | | 2 | Spring Boot API | Validates input, hashes password (bcrypt), creates candidate record | PostgreSQL | New candidates row; JWT access + refresh tokens returned | | 3 | Spring Boot API | Sends welcome/confirmation email | Gmail SMTP | Email delivered to candidate; send event logged in email_log table | | 4 | Candidate | Uploads CV (PDF or DOCX) and fills in profile (skills, work history, rates) | Angular frontend | HTTPS PUT to /api/v1/me/profile | | 5 | Spring Boot API | Stores CV file reference and structured profile fields | PostgreSQL (candidate_profiles, candidate_documents) | Profile record updated; CV stored as binary or object reference | | 6 | Candidate (optional) | Requests AI CV rewrite | Angular frontend → Spring Boot API | API calls internal LLM endpoint; rewritten CV text stored alongside original in candidate_documents | | 7 | Spring Boot API | Writes audit log entry for registration and profile creation | PostgreSQL (audit_log) | Audit record: user ID, action, timestamp, IP address |

4b — Employer Registration & Vacancy Posting

| Step | Actor | Action | System | Output | |------|-------|--------|--------|--------| | 1 | Employer | Completes employer registration (company name, KvK, contact, email) | Angular frontend | HTTPS POST to /api/v1/auth/register/employer | | 2 | Spring Boot API | Validates, creates employer account | PostgreSQL (employers) | Employer record created; JWT tokens returned | | 3 | Employer | Posts a vacancy (title, description, required skills, rate, location) | Angular frontend | HTTPS POST to /api/v1/vacancies | | 4 | Spring Boot API | Stores vacancy; triggers async matching job | PostgreSQL (vacancies) | Vacancy record created; matching queue entry added | | 5 | Spring Boot API (async) | Runs skills-based matching algorithm against active candidate profiles | PostgreSQL + Redis | Match scores computed; top matches cached in Redis (TTL 1h); match log written to match_log table | | 6 | Spring Boot API | Sends email notifications to matched candidates (configurable) | Gmail SMTP | Notification email with vacancy summary sent to matched candidates |

4c — Automated Vacancy Scraping (Firecrawl)

| Step | Actor | Action | System | Output | |------|-------|--------|--------|--------| | 1 | Spring Boot scheduler | Triggers scraping job on cron schedule (every 6 hours) | Spring Boot (internal) | Job initiated | | 2 | Spring Boot API | Sends scraping request with list of target job board URLs to Firecrawl API | Firecrawl API (HTTPS, external) | Firecrawl returns structured vacancy data (title, description, skills, URL) | | 3 | Spring Boot API | Deduplicates results against existing scraped vacancies (by URL hash) | PostgreSQL (scraped_vacancies) | New vacancies inserted; existing records updated if content changed | | 4 | Spring Boot API (async) | Runs matching against newly scraped vacancies; notifies relevant candidates | PostgreSQL + Redis | Match scores and notifications as per step 4b.5–6 | | 5 | Spring Boot scheduler | Deletes scraped vacancy records older than 90 days | PostgreSQL | Expired vacancy records removed; no personal data in scraped records |

4d — Data Subject Access & Deletion

| Step | Actor | Action | System | Output | |------|-------|--------|--------|--------| | 1 | Candidate | Requests data export via "My Profile → Export my data" | Angular frontend | HTTPS GET to /api/v1/me/export | | 2 | Spring Boot API | Queries all personal data linked to candidate ID | PostgreSQL | JSON export returned to browser; includes profile, CV text, match history, email log references | | 3 | Candidate | Requests account deletion via "My Profile → Delete account" | Angular frontend | HTTPS DELETE to /api/v1/me | | 4 | Spring Boot API | Soft-deletes account (status = PENDING_DELETION); schedules hard delete in 24h | PostgreSQL | Account hidden from matching; session tokens invalidated in Redis | | 5 | Spring Boot API (scheduled) | Hard-deletes all personal data linked to candidate after 24h grace period | PostgreSQL | Cascade delete: profile, CV, match logs, email logs, audit logs (redacted to anonymous user ID) | | 6 | Spring Boot API | Sends deletion confirmation email | Gmail SMTP | Confirmation to candidate's email (last email sent before deletion) |


5. Data Flow

EXTERNAL                          PLATFORM (TransIP VPS 136.144.174.219)
                                  ┌─────────────────────────────────────────────────┐
                                  │  k3s cluster                                    │
[Candidate browser]               │                                                 │
  │                               │  ┌──────────────┐    ┌───────────────────────┐ │
  │──HTTPS POST (registration,    │  │ Spring Boot  │───▶│ PostgreSQL            │ │
  │  profile, CV upload)─────────▶│  │ REST API     │    │ - candidates          │ │
  │◀─JWT token, profile data──────│  │ (port 8080)  │◀───│ - employers           │ │
  │                               │  │              │    │ - vacancies           │ │
[Employer browser]                │  │              │    │ - scraped_vacancies   │ │
  │──HTTPS POST (vacancy)────────▶│  │              │    │ - match_log           │ │
  │                               │  │              │    │ - audit_log           │ │
[Admin browser]                   │  └──────┬───────┘    └───────────────────────┘ │
  │──HTTPS (admin API)───────────▶│         │                                       │
  │                               │         │──session/cache──▶ ┌───────────────┐  │
                                  │         │                    │ Redis         │  │
[Firecrawl API]                   │         │                    │ (TTL 24h)     │  │
  │◀─scrape request (URLs only)───│         │                    └───────────────┘  │
  │──scraped vacancy JSON────────▶│         │                                       │
                                  │         │──SMTP─────────────▶ [Gmail SMTP]     │
[Gmail / candidate inbox]         │         │                         (external)    │
  │◀─transactional emails─────────│─────────┘                                      │
                                  │                                                 │
                                  └─────────────────────────────────────────────────┘

SECRETS FLOW (not data flow):
  1Password Secrets Automation ──▶ Bitnami Sealed Secrets ──▶ k3s pod env vars
  (DB password, JWT secret, Firecrawl API key, Gmail credentials)

DATA LEAVING THE EER:
  - Firecrawl (US): only public vacancy URLs — no personal data
  - GitHub GHCR (US): container images only — no personal data
  - Gmail SMTP (EU data region): candidate/employer name + email in email body

Data at rest: All persistent personal data resides in PostgreSQL on the LUKS-encrypted TransIP VPS. Redis data is ephemeral and also on the encrypted volume.

Data in transit: All external communication is TLS 1.3. Internal cluster communication is currently unencrypted within the k3s pod network (mTLS planned Q3 2026).

No personal data leaves the EER during normal operations, with the narrow exception of transactional email routing through Gmail SMTP (Google EU data region, covered by Google Workspace DPA).


6. Error Handling

| Step | Failure scenario | Behaviour | |------|-----------------|-----------| | Candidate registration | Email delivery fails (Gmail SMTP down) | Registration succeeds; welcome email queued for retry (max 3 attempts, 10-min backoff); candidate can still log in | | CV upload | File too large (>10 MB) or unsupported format | API returns HTTP 422 with user-friendly error message; no partial data stored | | AI CV rewrite | LLM endpoint unavailable or times out | API returns HTTP 503; candidate profile remains unchanged; original CV unaffected; error logged to audit log | | Firecrawl scraping | Firecrawl API returns error or rate-limit | Scraping job logs warning, skips batch, retries on next scheduled run; no data loss; existing scraped vacancies unaffected | | Matching job | PostgreSQL query timeout | Job logs error, partial results discarded; full re-run triggered on next cron cycle (15-min interval) | | Data export | PostgreSQL unavailable during export | API returns HTTP 503; no partial export delivered; candidate retries when service recovers | | Account deletion | Hard-delete job fails (DB error) | Job retries up to 5 times with exponential backoff; alert raised to GloryLabs on-call if all retries fail; account remains in PENDING_DELETION state (hidden from platform) | | Backup job | pg_dump fails or TransIP object storage unreachable | Failure alert sent to GloryLabs monitoring email; backup gap noted; manual backup initiated if failure persists >24h |


7. Related Documents

Reacties

Nog geen reacties