Duration: ~2 hours Goal: Complete local Kubernetes deployment that was started November 2nd Status: ✅ COMPLETE SUCCESS - All pods running and healthy!

🎉 Final Result

ALL SERVICES RUNNING IN KUBERNETES:

NAME                                          READY   STATUS    RESTARTS   AGE
glorylabs-postgresql-0                        1/1     Running   0          10m
glorylabs-recruitment-platform-backend        1/1     Running   0          3m
glorylabs-recruitment-platform-frontend       1/1     Running   0          30s
glorylabs-redis-master-0                      1/1     Running   0          10m

Services Created:

service/glorylabs-postgresql                      ClusterIP   10.111.59.5      5432/TCP
service/glorylabs-recruitment-platform-backend    ClusterIP   10.105.234.142   8080/TCP
service/glorylabs-recruitment-platform-frontend   ClusterIP   10.102.138.234   80/TCP
service/glorylabs-redis-master                    ClusterIP   10.102.48.62     6379/TCP

🔧 Issues Fixed Today

1. Backend Logback File System Error ❌ → ✅

Problem: Backend crashing with "Read-only file system" error when trying to write log files to /tmp

Solution:

Added Kubernetes-specific profile to logback-spring.xml
Kubernetes profile uses console-only logging (JSON format for log aggregation)
Updated backend-configmap.yaml to set SPRING_PROFILES_ACTIVE=kubernetes

Files Changed:

backend/src/main/resources/logback-spring.xml - Added kubernetes profile
helm/recruitment-platform/templates/backend-configmap.yaml - Added SPRING_PROFILES_ACTIVE

2. H2 Driver Being Used Instead of PostgreSQL ❌ → ✅

Problem: Backend trying to use H2 driver for PostgreSQL URL

Root Cause: Missing SPRING_DATASOURCE_DRIVER_CLASS_NAME environment variable

Solution:

Added SPRING_DATASOURCE_DRIVER_CLASS_NAME=org.postgresql.Driver to backend ConfigMap

Files Changed:

helm/recruitment-platform/templates/backend-configmap.yaml

3. Database Connection Failures ❌ → ✅

Problem: UnknownHostException: recruitment-postgresql

Root Cause: Incorrect hostname in values file (didn't match actual Kubernetes service names)

Solution:

Updated database host: recruitment-postgresql → glorylabs-postgresql
Updated Redis host: recruitment-redis-master → glorylabs-redis-master

Files Changed:

helm/recruitment-platform/values-local.yaml

4. Actuator Health Check Returning 401 ❌ → ✅

Problem: Kubernetes readiness/liveness probes failing because actuator endpoints required authentication

Root Cause: Security config only allowed /actuator/health/** but Kubernetes was calling /api/actuator/health/**

Solution:

Added .requestMatchers("/api/actuator/health/**").permitAll() to SecurityConfig

Files Changed:

backend/src/main/java/nl/glorylabs/config/SecurityConfig.java

5. Frontend Nginx Can't Find Backend ❌ → ✅

Problem: Frontend crashing: "host not found in upstream 'glorylabs-recruitment-platform-backend'"

Root Cause: Missing Kubernetes Service resources! Helm templates didn't include service definitions

Solution:

Created helm/recruitment-platform/templates/backend-service.yaml
Created helm/recruitment-platform/templates/frontend-service.yaml

Files Created:

helm/recruitment-platform/templates/backend-service.yaml ⭐
helm/recruitment-platform/templates/frontend-service.yaml ⭐

6. PostgreSQL/Redis Image Pull Errors ❌ → ✅

Problem: ImagePullBackOff for PostgreSQL and Redis (specific Bitnami tags didn't exist)

Solution:

Updated to use latest tag for both PostgreSQL and Redis in values-local.yaml
Pre-pulled images with docker pull bitnami/postgresql:latest and docker pull bitnami/redis:latest

Files Changed:

helm/recruitment-platform/values-local.yaml

📝 Configuration Summary

Backend Configuration

Environment Variables:
  SPRING_PROFILES_ACTIVE: kubernetes
  SPRING_DATASOURCE_DRIVER_CLASS_NAME: org.postgresql.Driver
  SPRING_DATASOURCE_URL: jdbc:postgresql://glorylabs-postgresql:5432/recruitment_local
  SPRING_DATASOURCE_USERNAME: recruitment
  SPRING_REDIS_HOST: glorylabs-redis-master
  SPRING_REDIS_PORT: 6379

Database/Redis Configuration

postgresql:
  enabled: true
  image:
    repository: bitnami/postgresql
    tag: "latest"
  auth:
    username: recruitment
    password: "localpass123"
    database: recruitment_local

redis:
  enabled: true
  image:
    repository: bitnami/redis
    tag: "latest"
  auth:
    password: "localredis123"

🚀 Deployment Commands

Build Docker Images

cd backend
docker build -t recruitment-backend:local .

cd ../frontend/recruitment-portal
docker build --target production -t recruitment-frontend:local .

Deploy to Kubernetes

# Install/Upgrade Helm release
helm upgrade --install glorylabs ./helm/recruitment-platform \
  -f ./helm/recruitment-platform/values-local.yaml \
  -n glorylabs-local --create-namespace

# Check status
kubectl get pods -n glorylabs-local
kubectl get svc -n glorylabs-local

# View logs
kubectl logs -f deployment/glorylabs-recruitment-platform-backend -n glorylabs-local
kubectl logs -f deployment/glorylabs-recruitment-platform-frontend -n glorylabs-local

Access Application

# Port-forward frontend
kubectl port-forward -n glorylabs-local svc/glorylabs-recruitment-platform-frontend 4300:80

# Port-forward backend
kubectl port-forward -n glorylabs-local svc/glorylabs-recruitment-platform-backend 8090:8080

# Access application
open http://localhost:4300

# Check backend health
curl http://localhost:8090/api/actuator/health

📊 Deployment Architecture

┌─────────────────────────────────────────────────────────────┐
│                    Kubernetes Cluster                        │
│                    (glorylabs-local)                         │
│                                                              │
│  ┌──────────────┐      ┌──────────────┐                    │
│  │   Frontend   │──────│   Backend    │                    │
│  │  nginx:80    │      │  Java:8080   │                    │
│  └──────────────┘      └──────┬───────┘                    │
│                               │                              │
│                        ┌──────┴───────┐                     │
│                        │              │                      │
│                  ┌─────▼─────┐  ┌────▼────┐                │
│                  │ PostgreSQL │  │  Redis  │                │
│                  │   :5432    │  │  :6379  │                │
│                  └────────────┘  └─────────┘                │
│                                                              │
└─────────────────────────────────────────────────────────────┘

📁 Files Modified/Created

Created (6 files)

helm/recruitment-platform/templates/backend-service.yaml ⭐
helm/recruitment-platform/templates/frontend-service.yaml ⭐
helm/recruitment-platform/templates/serviceaccount.yaml
helm/recruitment-platform/Chart.lock
helm/recruitment-platform/charts/postgresql-18.1.3.tgz
helm/recruitment-platform/charts/redis-23.2.2.tgz

Modified (10 files)

backend/src/main/java/nl/glorylabs/config/SecurityConfig.java
backend/src/main/resources/logback-spring.xml
backend/Dockerfile
frontend/recruitment-portal/Dockerfile
helm/recruitment-platform/Chart.yaml
helm/recruitment-platform/templates/backend-configmap.yaml
helm/recruitment-platform/templates/backend-deployment.yaml
helm/recruitment-platform/templates/cv-service-deployment.yaml
helm/recruitment-platform/templates/frontend-deployment.yaml
helm/recruitment-platform/values-local.yaml

🎓 Key Learnings

Kubernetes requires Services - Deployments alone aren't enough; Services are needed for pod-to-pod communication
File system is read-only in K8s - Configure apps for console/stdout logging only
Health endpoints must be public - Kubernetes probes can't authenticate
Service naming matters - Hostnames must exactly match Kubernetes Service names
Bitnami image tags - Some specific version tags don't exist; latest is safer for local dev
Driver class must be explicit - Spring Boot doesn't always auto-detect the right JDBC driver
Helm subcharts are complex - Better to use simple values that work than fight with subchart overrides

✅ Testing Checklist

[x] Backend pod starts successfully
[x] Frontend pod starts successfully
[x] PostgreSQL pod starts successfully
[x] Redis pod starts successfully
[x] All pods pass readiness checks (1/1 READY)
[x] Backend health endpoint returns 200 OK
[x] Services created for backend and frontend
[ ] Frontend accessible via port-forward (next step)
[ ] Backend API accessible via port-forward (next step)
[ ] Can register and login (next step)

🎯 Next Steps

Test application functionality via port-forward
Create production-ready values (values-prod.yaml)
Set up Ingress for external access (instead of port-forward)
Add persistent volumes for PostgreSQL and Redis
Configure monitoring (Prometheus/Grafana)
Set up CI/CD pipeline to auto-deploy on commit

⏱️ Time Breakdown

Docker Image Building: ~15 min
Fixing logback configuration: ~10 min
Fixing datasource configuration: ~10 min
Fixing database hostnames: ~10 min
Fixing health endpoint security: ~10 min
Fixing PostgreSQL/Redis images: ~15 min
Creating missing Service templates: ~15 min (critical fix!)
Testing and verification: ~20 min
Documentation: ~15 min
Total: ~2 hours

📚 References

Previous session: bin/SESSION_2025-11-02_K8S_DEPLOYMENT.md
Helm documentation: https://helm.sh/docs/
Kubernetes services: https://kubernetes.io/docs/concepts/services-networking/service/
Bitnami charts: https://github.com/bitnami/charts

Session End: November 3, 2025 Status: ✅ DEPLOYMENT SUCCESSFUL - Ready for application testing Next Session: Test application functionality and create production deployment config

Reacties

Nog geen reacties

Athena — mahmoud-consultancy/archive/sessions/SESSION_2025-11-03_K8S_SUCCESS.md

Kubernetes Local Deployment - SUCCESS! - November 3, 2025

Session Overview

🎉 Final Result

🔧 Issues Fixed Today

1. Backend Logback File System Error ❌ → ✅

2. H2 Driver Being Used Instead of PostgreSQL ❌ → ✅

3. Database Connection Failures ❌ → ✅

4. Actuator Health Check Returning 401 ❌ → ✅

5. Frontend Nginx Can't Find Backend ❌ → ✅

6. PostgreSQL/Redis Image Pull Errors ❌ → ✅

📝 Configuration Summary

Backend Configuration

Database/Redis Configuration

🚀 Deployment Commands

Build Docker Images

Deploy to Kubernetes

Access Application

📊 Deployment Architecture

📁 Files Modified/Created

Created (6 files)

Modified (10 files)

🎓 Key Learnings

✅ Testing Checklist

🎯 Next Steps

⏱️ Time Breakdown

📚 References

Reacties