Files

Tour 9857f053a1 Fix mock tests

2025-12-04 04:30:44 +01:00

13 KiB

Raw Blame History

Quarkus Auction Monitor - Complete Guide

🚀 Overview

The Troostwijk Auction Monitor now runs on Quarkus, a Kubernetes-native Java framework optimized for fast startup and low memory footprint.

Key Features

✅ Quarkus Scheduler - Built-in cron-based scheduling ✅ REST API - Control and monitor via HTTP endpoints ✅ Health Checks - Kubernetes-ready liveness/readiness probes ✅ CDI/Dependency Injection - Type-safe service management ✅ Fast Startup - 0.5s startup time ✅ Low Memory - ~50MB RSS memory footprint ✅ Hot Reload - Development mode with live coding

📦 Quick Start

Option 1: Run with Maven (Development)

# Start in dev mode with live reload
mvn quarkus:dev

# Access application
# API: http://localhost:8081/api/monitor/status
# Health: http://localhost:8081/health

Option 2: Build and Run JAR

# Build
mvn clean package

# Run
java -jar target/quarkus-app/quarkus-run.jar

# Or use fast-jar (recommended for production)
mvn clean package -Dquarkus.package.jar.type=fast-jar
java -jar target/quarkus-app/quarkus-run.jar

Option 3: Docker

# Build image
docker build -t auction-monitor:latest .

# Run container
docker run -p 8081:8081 \
  -v $(pwd)/data:/mnt/okcomputer/output \
  auction-monitor:latest

Option 4: Docker Compose (Recommended)

# Start services
docker-compose up -d

# View logs
docker-compose logs -f

# Stop services
docker-compose down

🔧 Configuration

application.properties

All configuration is in src/main/resources/application.properties:

# Database
auction.database.path=C:\\mnt\\okcomputer\\output\\cache.db
auction.images.path=C:\\mnt\\okcomputer\\output\\images

# Notifications
auction.notification.config=desktop
# Or for email: smtp:your@gmail.com:app_password:recipient@example.com

# YOLO Models (optional)
auction.yolo.config=models/yolov4.cfg
auction.yolo.weights=models/yolov4.weights
auction.yolo.classes=models/coco.names

# Workflow Schedules (cron expressions)
auction.workflow.scraper-import.cron=0 */30 * * * ?      # Every 30 min
auction.workflow.image-processing.cron=0 0 * * * ?       # Every 1 hour
auction.workflow.bid-monitoring.cron=0 */15 * * * ?      # Every 15 min
auction.workflow.closing-alerts.cron=0 */5 * * * ?       # Every 5 min

# HTTP Server
quarkus.http.port=8081
quarkus.http.host=0.0.0.0

Environment Variables

Override configuration with environment variables:

export AUCTION_DATABASE_PATH=/path/to/cache.db
export AUCTION_NOTIFICATION_CONFIG=desktop
export QUARKUS_HTTP_PORT=8081

📅 Scheduled Workflows

Quarkus automatically runs these workflows based on cron expressions:

Workflow	Schedule	Cron Expression	Description
Scraper Import	Every 30 min	`0 /30 * * ?`	Import auctions/lots from external scraper
Image Processing	Every 1 hour	`0 0 * * * ?`	Download images & run object detection
Bid Monitoring	Every 15 min	`0 /15 * * ?`	Check for bid changes
Closing Alerts	Every 5 min	`0 /5 * * ?`	Send alerts for lots closing soon

Cron Expression Format

┌───────────── second (0-59)
│ ┌───────────── minute (0-59)
│ │ ┌───────────── hour (0-23)
│ │ │ ┌───────────── day of month (1-31)
│ │ │ │ ┌───────────── month (1-12)
│ │ │ │ │ ┌───────────── day of week (0-6, Sunday=0)
│ │ │ │ │ │
0 */30 * * * ?  = Every 30 minutes
0 0 * * * ?     = Every hour at minute 0
0 0 0 * * ?     = Every day at midnight

🌐 REST API

Base URL

http://localhost:8081/api/monitor

Endpoints

1. Get Status

GET /api/monitor/status

# Example
curl http://localhost:8081/api/monitor/status

# Response
{
  "running": true,
  "auctions": 25,
  "lots": 150,
  "images": 300,
  "closingSoon": 5
}

2. Get Statistics

GET /api/monitor/statistics

# Example
curl http://localhost:8081/api/monitor/statistics

# Response
{
  "totalAuctions": 25,
  "totalLots": 150,
  "totalImages": 300,
  "activeLots": 120,
  "lotsWithBids": 80,
  "totalBidValue": "€125,450.00",
  "averageBid": "€1,568.13"
}

3. Trigger Workflows Manually

# Scraper Import
POST /api/monitor/trigger/scraper-import
curl -X POST http://localhost:8081/api/monitor/trigger/scraper-import

# Image Processing
POST /api/monitor/trigger/image-processing
curl -X POST http://localhost:8081/api/monitor/trigger/image-processing

# Bid Monitoring
POST /api/monitor/trigger/bid-monitoring
curl -X POST http://localhost:8081/api/monitor/trigger/bid-monitoring

# Closing Alerts
POST /api/monitor/trigger/closing-alerts
curl -X POST http://localhost:8081/api/monitor/trigger/closing-alerts

4. Get Auctions

# All auctions
GET /api/monitor/auctions
curl http://localhost:8081/api/monitor/auctions

# Filter by country
GET /api/monitor/auctions?country=NL
curl http://localhost:8081/api/monitor/auctions?country=NL

5. Get Lots

# Active lots
GET /api/monitor/lots
curl http://localhost:8081/api/monitor/lots

# Lots closing soon (within 30 minutes by default)
GET /api/monitor/lots/closing-soon
curl http://localhost:8081/api/monitor/lots/closing-soon

# Custom minutes threshold
GET /api/monitor/lots/closing-soon?minutes=60
curl http://localhost:8081/api/monitor/lots/closing-soon?minutes=60

6. Get Lot Images

GET /api/monitor/lots/{lotId}/images

# Example
curl http://localhost:8081/api/monitor/lots/12345/images

7. Test Notification

POST /api/monitor/test-notification
Content-Type: application/json

{
  "message": "Test message",
  "title": "Test Title",
  "priority": "0"
}

# Example
curl -X POST http://localhost:8081/api/monitor/test-notification \
  -H "Content-Type: application/json" \
  -d '{"message":"Test notification","title":"Test","priority":"0"}'

🏥 Health Checks

Quarkus provides built-in health checks for Kubernetes/Docker:

Liveness Probe

GET /health/live

# Example
curl http://localhost:8081/health/live

# Response
{
  "status": "UP",
  "checks": [
    {
      "name": "Auction Monitor is alive",
      "status": "UP"
    }
  ]
}

Readiness Probe

GET /health/ready

# Example
curl http://localhost:8081/health/ready

# Response
{
  "status": "UP",
  "checks": [
    {
      "name": "database",
      "status": "UP",
      "data": {
        "auctions": 25
      }
    }
  ]
}

Startup Probe

GET /health/started

# Example
curl http://localhost:8081/health/started

Combined Health

GET /health

# Returns all health checks
curl http://localhost:8081/health

🐳 Docker Deployment

Build Image

docker build -t auction-monitor:1.0 .

Run Container

docker run -d \
  --name auction-monitor \
  -p 8081:8081 \
  -v $(pwd)/data:/mnt/okcomputer/output \
  -e AUCTION_NOTIFICATION_CONFIG=desktop \
  auction-monitor:1.0

Docker Compose

version: '3.8'
services:
  auction-monitor:
    image: auction-monitor:1.0
    ports:
      - "8081:8081"
    volumes:
      - ./data:/mnt/okcomputer/output
    environment:
      - AUCTION_DATABASE_PATH=/mnt/okcomputer/output/cache.db
      - AUCTION_NOTIFICATION_CONFIG=desktop
    healthcheck:
      test: ["CMD", "wget", "--spider", "http://localhost:8081/health/live"]
      interval: 30s
      timeout: 3s
      retries: 3

☸️ Kubernetes Deployment

deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: auction-monitor
spec:
  replicas: 1
  selector:
    matchLabels:
      app: auction-monitor
  template:
    metadata:
      labels:
        app: auction-monitor
    spec:
      containers:
      - name: auction-monitor
        image: auction-monitor:1.0
        ports:
        - containerPort: 8081
        env:
        - name: AUCTION_DATABASE_PATH
          value: /data/cache.db
        - name: QUARKUS_HTTP_PORT
          value: "8081"
        volumeMounts:
        - name: data
          mountPath: /mnt/okcomputer/output
        livenessProbe:
          httpGet:
            path: /health/live
            port: 8081
          initialDelaySeconds: 10
          periodSeconds: 30
        readinessProbe:
          httpGet:
            path: /health/ready
            port: 8081
          initialDelaySeconds: 5
          periodSeconds: 10
        startupProbe:
          httpGet:
            path: /health/started
            port: 8081
          failureThreshold: 30
          periodSeconds: 10
      volumes:
      - name: data
        persistentVolumeClaim:
          claimName: auction-data-pvc
---
apiVersion: v1
kind: Service
metadata:
  name: auction-monitor
spec:
  selector:
    app: auction-monitor
  ports:
  - port: 8081
    targetPort: 8081
  type: LoadBalancer

🔄 Development Mode

Quarkus dev mode provides live reload for rapid development:

# Start dev mode
mvn quarkus:dev

# Features available:
# - Live reload (no restart needed)
# - Dev UI: http://localhost:8081/q/dev/
# - Continuous testing
# - Debug on port 5005

Dev UI

Access at: http://localhost:8081/q/dev/

Features:

Configuration editor
Scheduler dashboard
Health checks
REST endpoints explorer
Continuous testing

🧪 Testing

Run All Tests

mvn test

Run Quarkus Tests

mvn test -Dtest=*QuarkusTest

Integration Test with Running Application

# Terminal 1: Start application
mvn quarkus:dev

# Terminal 2: Run integration tests
curl http://localhost:8081/api/monitor/status
curl http://localhost:8081/health/live
curl -X POST http://localhost:8081/api/monitor/trigger/scraper-import

📊 Monitoring & Logging

View Logs

# Docker
docker logs -f auction-monitor

# Docker Compose
docker-compose logs -f

# Kubernetes
kubectl logs -f deployment/auction-monitor

Log Levels

Configure in application.properties:

# Production
quarkus.log.console.level=INFO

# Development
%dev.quarkus.log.console.level=DEBUG

# Specific logger
quarkus.log.category."com.auction".level=DEBUG

Scheduled Job Logs

14:30:00 INFO  [com.auc.Qua] (executor-thread-1) 📥 [WORKFLOW 1] Importing scraper data...
14:30:00 INFO  [com.auc.Qua] (executor-thread-1)   → Imported 5 auctions
14:30:00 INFO  [com.auc.Qua] (executor-thread-1)   → Imported 25 lots
14:30:00 INFO  [com.auc.Qua] (executor-thread-1)   ✓ Scraper import completed in 1250ms

⚙️ Performance

Startup Time

JVM Mode: ~0.5 seconds
Native Image: ~0.014 seconds

Memory Footprint

JVM Mode: ~50MB RSS
Native Image: ~15MB RSS

Build Native Image (Optional)

# Requires GraalVM
mvn package -Pnative

# Run native executable
./target/troostwijk-scraper-1.0-SNAPSHOT-runner

🔐 Security

Environment Variables for Secrets

# Don't commit credentials!
export AUCTION_NOTIFICATION_CONFIG=smtp:user@gmail.com:SECRET_PASSWORD:recipient@example.com

# Or use Kubernetes secrets
kubectl create secret generic auction-secrets \
  --from-literal=notification-config='smtp:user@gmail.com:password:recipient@example.com'

Kubernetes Secret

apiVersion: v1
kind: Secret
metadata:
  name: auction-secrets
type: Opaque
stringData:
  notification-config: smtp:user@gmail.com:app_password:recipient@example.com

🛠️ Troubleshooting

Issue: Schedulers not running

Check scheduler status:

curl http://localhost:8081/health/ready

Enable debug logging:

quarkus.log.category."io.quarkus.scheduler".level=DEBUG

Issue: Database not found

Check file permissions:

ls -la C:/mnt/okcomputer/output/cache.db

Create directory:

mkdir -p C:/mnt/okcomputer/output

Issue: Port 8081 already in use

Change port:

mvn quarkus:dev -Dquarkus.http.port=8082
# Or
export QUARKUS_HTTP_PORT=8082

Issue: Health check failing

Check application logs:

docker logs auction-monitor

Verify database connection:

curl http://localhost:8081/health/ready

📚 Additional Resources

Summary

✅ Quarkus Framework integrated for modern Java development ✅ CDI/Dependency Injection for clean architecture ✅ @Scheduled annotations for cron-based workflows ✅ REST API for control and monitoring ✅ Health Checks for Kubernetes/Docker ✅ Fast Startup and low memory footprint ✅ Docker/Kubernetes ready ✅ Production optimized

Run and enjoy! 🎉

13 KiB Raw Blame History