Athena — mahmoud-consultancy/planning/vacancy-content-features-scoping-2026-08.md

Vacancy Content Features — Scoping Proposal (2026-08-03)

Owner directive: scope 3 feature requests before implementation. All findings below are from real codebase/data investigation, not assumptions.


1. Source-visibility flag per assignment

Corrects a false premise in the original request: there is no "2+ sources" threshold anywhere in the code. The feature already exists and is unconditional.

What already exists today:

  • Backend: JobSourceLink entity (job_source_links table), populated by JobIngestService.createSourceLink()/ensureSourceLink() from every scraper. GET /api/jobs/{id}/sources (JobController.getJobSources()) is @PreAuthorize("permitAll()") — visible to everyone, no role gating.
  • Frontend (frontend/recruitment-portal only — interimplaza-web marketing site has none of this): job-detail.component renders, gated only on jobSources.length > 0 (not 2+):
    1. An attribution line ("Ook op:" / "Also on").
    2. A full broker-comparison table ("Beschikbaar via N aanbieder(s)") with provider name, rate, deadline, view-link. PROVIDER_SINGULAR/PROVIDER_PLURAL i18n keys already handle the 1-source grammatical case.

What's genuinely missing:

  1. No feature flag exists — the display is unconditional. config/feature-flags.ts is an established boolean-flag convention already used elsewhere (cvFeature, jobBoard); a new flag hooks in trivially.
  2. The indicator only exists on the job detail page — absent from job-list/search-result cards. If "per assignment" means visible while browsing (a badge on the card), that's a real, non-trivial gap.
  3. Absent entirely from interimplaza-web (marketing site) — net-new integration if wanted there.
  4. No role-based visibility control exists (admin vs. candidate).

Effort:

  • If the ask is "just wrap the existing always-on display in a toggle": trivial, <0.5 day.
  • If the ask is "a badge visible on job-list/cards, not just after opening a job": the real gap, ~1-2 days (N+1 fetch or new batch backend endpoint, plus a new compact badge component).
  • Needs owner clarification on which of these two is actually meant — they're very different sizes of work.

2. Strip source links from vacancy description text

This is not a link-stripping problem — it's a self-inflicted duplication bug. No AI pass needed; no generic regex/HTML-stripper needed either.

Real findings (sampled directly from interimplaza-staging Postgres):

  • Striive (960/960 rows), StarApple (420/420), Flextender (47/182, detail pages) — every description has a fixed HTML attribution block (<a href="...">bekijk de originele opdracht op X</a> or <strong>Bron:</strong> <a href="...">X</a>) injected by our own mapper code (StriiveJobMapper, StarAppleVacancyParser, FlextenderJobParser.buildDescription()), not scraped from source prose.
  • Freelance.nl (2845/2845 rows, plain text): every description ends with a fixed trailer Bron: Freelance.nl — <url>, same self-injected pattern.
  • Circle8: already clean — its parser extracts via Jsoup .text() (tag-stripping), so no links reach the DB.
  • Only 2/2845 Freelance.nl rows had any other embedded URL (a third-party ATS apply link, not a source-site link) — negligible.

Fix: delete the ~2-5 lines of attribution-injection code in each of 4 files (StriiveJobMapper.java ~line 141, StarAppleVacancyParser.java, FlextenderJobParser.java ~line 138-172, Freelance.nl's mapper trailer-append). This duplicates the existing structured JobSourceLink "also found via" feature (item 1 above), which already covers attribution correctly. New ingests then never carry the link.

For the ~4,272 already-affected existing rows (960+420+47+2845), a one-off backfill script doing a targeted regex strip of the exact known fixed-template patterns (safe, since the pattern is code-generated and fixed, not free-form prose).

Effort: small, well under a day. ~30 min for the 4-file code fix + test-fixture updates; 1-2 hours for the backfill script.


3. Single unified format for all vacancy descriptions

Real connector list confirmed (10 + 1 generic route): Circle8, Randstad, Ncim, SourcePower (WP-REST), Freelance, ItContracts (RSS), Striive, KvK, Flextender, StarApple, plus FirecrawlSourceScraper for ad-hoc sources.

Format survey (real DB excerpts, interimplaza-staging):

  • 9 of 10 mappers share one pattern: synthetic <h3>title</h3> + <p><strong>Label:</strong> value</p> metadata block + <hr> + raw source HTML dumped verbatim, unstructured. None split into Eisen/Wensen at parse time unless the source itself already wrote it that way (NCIM has native "Eisen."/"Wensen" headers; Flextender/Striive have native requirement lists but wrapped in messy Word-export HTML).
  • FreelanceJobMapper is the outlier: plain text, no HTML, with a real sample containing 14k+ chars of dense text with native "Eisen 1..."/"Wensen W1..." sections embedded inline.
  • KvkJobMapper: the KvK API's DTO has a native eisen field, but the mapper currently writes it into Job.requirements, not Job.eisen — a field-naming mismatch to reconcile.
  • Lengths vary wildly: 311 chars (ItContracts/RSS) to 14.7k chars (Flextender).

eisen/wensen AI extraction coverage: NOT universal — effectively Flextender-only, and mostly empty even there. AiFieldExtractionService.refineJobAsync() is called from exactly one place codewide: FlextenderScraperService. Staging DB proof: ~19 of ~4,637 jobs (~0.4%) have any eisen/wensen populated at all. This matches the already-logged "AI-extraction charges but persists nothing" bug pattern in this workspace's history — real coverage is negligible today, so "reuse existing extraction" currently means reusing almost nothing until extraction is wired onto the other connectors' persist paths.

Proposed unified template (structured fields first, raw HTML fallback): TitelKerngegevens (opdrachtgever/locatie/tarief/duur/uren, from existing structured Job columns) → Omschrijving (raw description HTML, stripped of the synthetic header/metadata block that's already duplicated in Kerngegevens) → Eisen (from Job.eisen, with KvK's Job.requirements folded in) → Wensen (from Job.wensen) → Bron attribution footer (already present everywhere; becomes even cleaner once item 2 above is fixed).

Effort:

  • Cheap (same builder pattern, mechanical strip/restructure): Circle8, ItContracts, Randstad, Ncim, SourcePower, Striive, Flextender, StarApple — 8 of 10.
  • Expensive: Freelance.nl (no HTML structure, needs a text-section parser to find its native Eisen/Wensen sections); KvK (field-name reconciliation).
  • The real blocker for the whole "reuse eisen/wensen" goal: extending AiFieldExtractionService.refineJobAsync from Flextender-only to all persist paths. Best done by moving the call into JobConstructionListener's @PrePersist hook, which already fires for all 11 routes — this is the highest-leverage single change since it unblocks the eisen/wensen field for the template across every connector at once.

Recommended sequencing

  1. Item 2 (link-strip bug fix) first — smallest, cleanest, well under a day, directly improves data quality regardless of what happens with items 1/3.
  2. Item 1, after owner clarifies which of the two scopes (detail-page toggle vs. list-card badge) is actually wanted.
  3. Item 3 last and in two phases — Phase A: wire AiFieldExtractionService universally via JobConstructionListener (the real blocker, unlocks eisen/wensen data for all connectors). Phase B: build the unified template once real eisen/wensen coverage exists broadly, starting with the 8 "cheap" connectors and treating Freelance.nl/KvK as follow-up.

Findings via 3 parallel read-only investigation agents, 2026-08-03. Sourced from live staging-database sampling (interimplaza-staging namespace, recruitment-platform-postgresql-0) and direct codebase inspection, not assumptions.

Reacties

Nog geen reacties