Athena — mahmoud-consultancy/archive/old-docs/SENTIOR_SOURCES_COMPLETE_LIST.md

Sentior Sources - Complete List with Integration Stories

Date: October 17, 2025 Source: https://www.sentior.com/app/sources Total Sources: 10 Active


📋 Complete Sources List

1. Gemeente Delft

  • Type: DAS
  • Start Date: 21-03-2016
  • End Date: 15-01-2022
  • Last Posted: 01-01-2025
  • Status: ✅ Active

2. Uitvoeringsorganisatie Bedrijfsvoering Rijk

  • Type: DAS
  • Start Date: 01-07-2016
  • End Date: 31-12-2022
  • Last Posted: (Not visible)
  • Status: ✅ Active

3. VRK Huurt In

  • Type: MARKTPLAATS
  • Start Date: 24-03-2021
  • End Date: 20-01-2023
  • Last Posted: (Not visible)
  • Status: ✅ Active

4. Banenplein Limburg

  • Type: MARKTPLAATS
  • Start Date: 23-07-2021
  • End Date: 20-01-2023
  • Last Posted: (Not visible)
  • Status: ✅ Active

5. Politie

  • Type: DAS
  • Start Date: 21-03-2016
  • End Date: 22-03-2025
  • Last Posted: 18-01-2023
  • Status: ✅ Active

6. Flextender

  • Type: MARKTPLAATS
  • Start Date: 05-07-2018
  • End Date: 18-01-2023
  • Last Posted: (Not visible)
  • Status: ✅ Active

7. Waterschap Rivierenland

  • Type: DAS
  • Start Date: 12-01-2018
  • End Date: 12-01-2019
  • Last Posted: 17-01-2023
  • Status: ✅ Active

8. Nederlandse Publieke Omroep (NPO)

  • Type: DAS
  • Start Date: 05-07-2021
  • End Date: 04-07-2025
  • Last Posted: 17-01-2023
  • Status: ✅ Active

9. Werken voor 's-Hertogenbosch

  • Type: MARKTPLAATS
  • Start Date: 05-10-2020
  • End Date: 14-01-2023
  • Last Posted: (Not visible)
  • Status: ✅ Active

10. VNG (Beslisteam BV)

  • Type: DAS
  • Start Date: 01-01-2017
  • End Date: 01-12-2023
  • Last Posted: 14-01-2023
  • Status: ✅ Active

📖 Integration Stories (User Stories)

Story 1: Politie Job Integration

As a recruitment platform administrator I want to automatically import job vacancies from Politie (Dutch National Police) So that candidates can find police ICT and administrative positions through our platform

Acceptance Criteria:

  • [ ] Daily automated crawl of Politie job board at 06:00
  • [ ] Extract job details: title, description, location, requirements, salary range
  • [ ] Map job categories to our system (ICT, Admin, Operations, etc.)
  • [ ] Handle authentication via DAS protocol
  • [ ] Deduplicate jobs already in system
  • [ ] Track source as "Politie" for attribution
  • [ ] Log all crawl activities and errors
  • [ ] Alert on crawl failures

Technical Details:

  • Protocol: DAS (Dutch Application Service)
  • Priority: URGENT (end date: 22-03-2025)
  • Estimated Volume: 50-100 jobs/week
  • Job Categories: ICT, Security, Admin, Operations

Story 2: NPO Job Integration

As a recruitment platform administrator I want to automatically import job vacancies from Nederlandse Publieke Omroep (NPO) So that candidates can find media and technology positions in the public broadcasting sector

Acceptance Criteria:

  • [ ] Daily automated crawl of NPO job board at 07:00
  • [ ] Extract job details including media-specific requirements
  • [ ] Map to relevant categories (Media, ICT, Content, Production)
  • [ ] Connect via DAS API with proper authentication
  • [ ] Handle Dutch language content properly
  • [ ] Track source attribution
  • [ ] Monitor crawl success rates

Technical Details:

  • Protocol: DAS
  • Priority: HIGH
  • Estimated Volume: 20-40 jobs/week
  • Job Categories: Media, ICT, Content Creation, Production, Engineering

Story 3: Uitvoeringsorganisatie Bedrijfsvoering Rijk Integration

As a recruitment platform administrator I want to import jobs from the central government operations organization (UBR) So that candidates can access federal government ICT and administrative positions

Acceptance Criteria:

  • [ ] Daily crawl at 08:00
  • [ ] Extract government-specific job fields (security clearance, pay scale, etc.)
  • [ ] Map to government job categories
  • [ ] Handle DAS authentication
  • [ ] Parse salary scales (schaal/trede system)
  • [ ] Track government-specific metadata

Technical Details:

  • Protocol: DAS
  • Priority: HIGH
  • Estimated Volume: 30-60 jobs/week
  • Job Categories: ICT, Finance, HR, Policy, Operations

Story 4: Gemeente Delft Integration

As a recruitment platform administrator I want to import municipal job postings from Gemeente Delft So that candidates can find local government positions in Delft

Acceptance Criteria:

  • [ ] Weekly crawl every Monday at 10:00
  • [ ] Extract municipal-specific details
  • [ ] Map to local government categories
  • [ ] Handle bilingual content (Dutch required)
  • [ ] Track as municipal source
  • [ ] Monitor for schedule changes

Technical Details:

  • Protocol: DAS
  • Priority: MEDIUM
  • Estimated Volume: 10-20 jobs/week
  • Job Categories: Municipal Admin, ICT, Public Works, Social Services

Story 5: VNG (Beslisteam BV) Integration

As a recruitment platform administrator I want to import jobs from VNG (Association of Netherlands Municipalities) So that candidates can access positions across multiple Dutch municipalities

Acceptance Criteria:

  • [ ] Weekly crawl every Tuesday at 10:00
  • [ ] Extract multi-municipal job postings
  • [ ] Handle varying location data (multiple cities)
  • [ ] Map to appropriate categories
  • [ ] Track original municipality
  • [ ] Support job comparison across municipalities

Technical Details:

  • Protocol: DAS
  • Priority: MEDIUM
  • Estimated Volume: 15-30 jobs/week
  • Job Categories: Municipal governance, ICT, Policy, Services

Story 6: Flextender Integration (Already Implemented ✅)

As a recruitment platform administrator I want to automatically sync job postings from Flextender So that freelance and contract positions are available to candidates

Status: ✅ IMPLEMENTED

  • [x] FlextenderScraperService created
  • [x] HTML scraping with JSoup
  • [x] Authentication via cookies
  • [x] Job parsing and mapping
  • [x] Scheduled crawling configured

Technical Details:

  • Protocol: HTML Scraping
  • Priority: HIGH (already done)
  • Estimated Volume: 40-80 jobs/week
  • Job Categories: IT Contracting, Consultancy, Freelance

Story 7: VRK Huurt In Integration

As a recruitment platform administrator I want to import job postings from VRK Huurt In recruitment platform So that candidates can find interim and detachment positions

Acceptance Criteria:

  • [ ] Weekly crawl every Wednesday at 10:00
  • [ ] Extract interim-specific job details (rate, duration, start date)
  • [ ] Map to freelance/contract categories
  • [ ] Handle rate information properly
  • [ ] Parse Dutch labor market terminology
  • [ ] Track as recruitment agency source

Technical Details:

  • Protocol: MARKTPLAATS (HTML Scraping)
  • Priority: MEDIUM
  • Estimated Volume: 15-25 jobs/week
  • Job Categories: Interim, Detachment, Consulting

Story 8: Waterschap Rivierenland Integration

As a recruitment platform administrator I want to import jobs from Waterschap Rivierenland (water management authority) So that candidates can find specialized water management and engineering positions

Acceptance Criteria:

  • [ ] Bi-weekly crawl (1st & 15th at 11:00)
  • [ ] Extract water management-specific roles
  • [ ] Map to engineering and environmental categories
  • [ ] Handle technical terminology
  • [ ] Track as specialized sector source

Technical Details:

  • Protocol: DAS
  • Priority: LOW
  • Estimated Volume: 5-10 jobs/week
  • Job Categories: Water Engineering, Environmental, ICT, Operations

Story 9: Banenplein Limburg Integration

As a recruitment platform administrator I want to import regional job postings from Banenplein Limburg So that candidates can find opportunities specifically in the Limburg region

Acceptance Criteria:

  • [ ] Bi-weekly crawl (1st & 15th at 12:00)
  • [ ] Extract Limburg-specific locations
  • [ ] Map to appropriate job categories
  • [ ] Handle regional variations in terminology
  • [ ] Tag with "Limburg" region
  • [ ] Support Limburgish place names

Technical Details:

  • Protocol: MARKTPLAATS (HTML Scraping)
  • Priority: LOW
  • Estimated Volume: 10-15 jobs/week
  • Job Categories: Regional positions across all sectors

Story 10: Werken voor 's-Hertogenbosch Integration

As a recruitment platform administrator I want to import municipal marketplace jobs from 's-Hertogenbosch So that candidates can find opportunities in the Den Bosch municipality

Acceptance Criteria:

  • [ ] Weekly crawl every Thursday at 10:00
  • [ ] Extract municipal marketplace postings
  • [ ] Map to municipal categories
  • [ ] Handle 's-Hertogenbosch naming variations (Den Bosch)
  • [ ] Track as municipal marketplace
  • [ ] Support multi-employer listings

Technical Details:

  • Protocol: MARKTPLAATS (HTML Scraping)
  • Priority: MEDIUM
  • Estimated Volume: 10-20 jobs/week
  • Job Categories: Municipal services, ICT, Admin, Social Work

🎯 Epic: Multi-Source Job Aggregation

Epic Goal: Integrate all 10 Sentior sources to provide comprehensive job coverage across Netherlands

Business Value:

  • 205-400 jobs/week total (312% increase from Flextender alone)
  • Coverage across government, municipalities, and private sectors
  • Geographic diversity (national, regional, municipal)
  • Sector diversity (ICT, media, engineering, administration, water management)

Technical Approach:

  1. DAS Framework (6 sources) - Build reusable DAS API client
  2. MARKTPLAATS Scrapers (4 sources) - Custom HTML parsers per source
  3. Unified Job Model - Normalize all sources to common format
  4. Scheduling - Cron-based crawling optimized per source frequency
  5. Monitoring - Dashboard showing success rates per source

Acceptance Criteria:

  • [ ] All 10 sources actively crawling
  • [ ] <5% failure rate across all sources
  • [ ] Deduplication working (same job from multiple sources)
  • [ ] Average job freshness <24 hours
  • [ ] Source attribution visible to candidates
  • [ ] Admin dashboard showing per-source metrics

🗓️ Implementation Sprint Plan

Sprint 1: DAS Framework (Week 1)

Stories: Research + Foundation

  • [ ] Research DAS API protocol and documentation
  • [ ] Create DasApiClient base service
  • [ ] Implement authentication mechanism
  • [ ] Build job parser for DAS format
  • [ ] Unit tests for DAS client

Sprint 2: High-Priority DAS Sources (Week 2)

Stories: Politie, NPO, UBR

  • [ ] Story 1: Politie integration
  • [ ] Story 2: NPO integration
  • [ ] Story 3: UBR integration
  • [ ] Integration tests for each
  • [ ] Deploy and monitor

Sprint 3: Medium-Priority Sources (Week 3)

Stories: Gemeente Delft, VNG, VRK Huurt In

  • [ ] Story 4: Gemeente Delft integration
  • [ ] Story 5: VNG integration
  • [ ] Story 7: VRK Huurt In integration
  • [ ] Cross-source deduplication testing

Sprint 4: Remaining Sources + Polish (Week 4)

Stories: Waterschap, Banenplein, Den Bosch

  • [ ] Story 8: Waterschap Rivierenland
  • [ ] Story 9: Banenplein Limburg
  • [ ] Story 10: Werken voor 's-Hertogenbosch
  • [ ] Performance optimization
  • [ ] Admin dashboard enhancements
  • [ ] Documentation completion

📊 Story Points Estimation

| Story | Complexity | Points | Priority | |-------|------------|--------|----------| | Story 1: Politie | Medium | 5 | URGENT | | Story 2: NPO | Medium | 5 | HIGH | | Story 3: UBR | Medium | 5 | HIGH | | Story 4: Gemeente Delft | Medium | 5 | MEDIUM | | Story 5: VNG | Medium | 5 | MEDIUM | | Story 6: Flextender | - | - | DONE ✅ | | Story 7: VRK Huurt In | Medium | 5 | MEDIUM | | Story 8: Waterschap | Small | 3 | LOW | | Story 9: Banenplein | Small | 3 | LOW | | Story 10: Den Bosch | Small | 3 | MEDIUM | | DAS Framework | Large | 8 | CRITICAL | | Total | - | 47 | - |

Estimated Timeline: 4 weeks (assuming 12 story points/week)


🔧 Technical Architecture

Component Structure

crawling/
├── config/
│   └── HttpClientConfig.java ✅
├── client/
│   ├── DasApiClient.java (NEW)
│   └── MarktplaatsScraperBase.java (NEW)
├── service/
│   ├── FlextenderScraperService.java ✅
│   ├── PolitieScraperService.java (NEW)
│   ├── NpoScraperService.java (NEW)
│   ├── UbrScraperService.java (NEW)
│   ├── GemeenteDelftScraperService.java (NEW)
│   ├── VngScraperService.java (NEW)
│   ├── VrkScraperService.java (NEW)
│   ├── WaterschapScraperService.java (NEW)
│   ├── BanenpleinScraperService.java (NEW)
│   └── DenBoschScraperService.java (NEW)
├── scheduler/
│   └── CrawlScheduler.java (ENHANCE)
└── monitoring/
    └── CrawlMetricsService.java (NEW)

Data Flow

Source → Scraper Service → Job Parser → Deduplication → Database → API → Frontend
   ↓
CrawlJob Entity (track each crawl)
   ↓
Monitoring Dashboard (success rates, job counts)

📝 Story Details Template

For Each DAS Source:

**Story:** [Source Name] Job Integration

**Background:**
[Source Name] is a [description] that posts [type] positions via the DAS protocol.

**User Need:**
Candidates need access to [sector] opportunities posted by [source name] to expand
their job search beyond single platforms.

**Solution:**
Implement automated daily/weekly crawling of [source name] via DAS API to import
all active job postings into our platform.

**Technical Implementation:**
1. Configure DAS API endpoint for [source]
2. Implement authentication mechanism
3. Create [SourceName]ScraperService extending DasApiClient
4. Build job parser for source-specific format
5. Map to our job categories
6. Schedule crawls at [time]
7. Add monitoring and alerting

**Business Impact:**
- +[X] jobs per week
- Coverage of [sector] sector
- [Geographic/demographic] diversity
- Competitive advantage in [niche]

**Dependencies:**
- DAS API Framework (Story: DAS Framework)
- OkHttpClient Bean ✅
- Job deduplication logic ✅

**Estimated Effort:** [X] story points
**Sprint:** Sprint [N]

🎯 Success Metrics

Per-Source Metrics

  • Uptime: >95% successful crawls
  • Job Quality: >90% jobs have complete data
  • Freshness: Average job age <24 hours
  • Deduplication: <2% duplicate jobs
  • Processing Time: <5 minutes per source

Aggregate Metrics

  • Total Jobs/Week: 205-400
  • Source Coverage: 10/10 active
  • Category Distribution: Balanced across sectors
  • Geographic Coverage: National + regional
  • Candidate Satisfaction: Measured via surveys

🚀 Quick Start Implementation

For Next Session:

Recommended: Start with Story 1 (Politie) - Most Urgent

# Step 1: Research DAS API
- Find DAS protocol documentation
- Identify authentication requirements
- Get API endpoints and credentials

# Step 2: Create DAS Framework
- Implement DasApiClient base class
- Configure authentication
- Build job parser

# Step 3: Implement Politie Scraper
- Create PolitieScraperService
- Test with Politie API
- Verify job import works
- Schedule daily crawls

# Step 4: Monitor & Iterate
- Track first week of crawls
- Fix any issues
- Optimize performance
- Document learnings

📋 Definition of Done (Per Story)

For each integration story to be considered complete:

  • [ ] Scraper service implemented and tested
  • [ ] Authentication working (if required)
  • [ ] Job parsing accurate (>95% success rate)
  • [ ] Scheduled crawls configured
  • [ ] Deduplication working
  • [ ] Error handling and logging
  • [ ] Monitoring dashboard updated
  • [ ] Documentation complete
  • [ ] Code reviewed and merged
  • [ ] Production deployment successful
  • [ ] First successful crawl verified

Document Created: October 17, 2025 Total Stories: 10 (1 done, 9 pending) Total Effort: 47 story points Timeline: 4 weeks for complete integration Next Action: Implement DAS Framework + Story 1 (Politie)

Reacties

Nog geen reacties