651 lines
13 KiB
Markdown
651 lines
13 KiB
Markdown
# Quarkus Auction Monitor - Complete Guide
|
|
|
|
## 🚀 Overview
|
|
|
|
The Troostwijk Auction Monitor now runs on **Quarkus**, a Kubernetes-native Java framework optimized for fast startup and low memory footprint.
|
|
|
|
### Key Features
|
|
|
|
✅ **Quarkus Scheduler** - Built-in cron-based scheduling
|
|
✅ **REST API** - Control and monitor via HTTP endpoints
|
|
✅ **Health Checks** - Kubernetes-ready liveness/readiness probes
|
|
✅ **CDI/Dependency Injection** - Type-safe service management
|
|
✅ **Fast Startup** - 0.5s startup time
|
|
✅ **Low Memory** - ~50MB RSS memory footprint
|
|
✅ **Hot Reload** - Development mode with live coding
|
|
|
|
---
|
|
|
|
## 📦 Quick Start
|
|
|
|
### Option 1: Run with Maven (Development)
|
|
|
|
```bash
|
|
# Start in dev mode with live reload
|
|
mvn quarkus:dev
|
|
|
|
# Access application
|
|
# API: http://localhost:8081/api/monitor/status
|
|
# Health: http://localhost:8081/health
|
|
```
|
|
|
|
### Option 2: Build and Run JAR
|
|
|
|
```bash
|
|
# Build
|
|
mvn clean package
|
|
|
|
# Run
|
|
java -jar target/quarkus-app/quarkus-run.jar
|
|
|
|
# Or use fast-jar (recommended for production)
|
|
mvn clean package -Dquarkus.package.jar.type=fast-jar
|
|
java -jar target/quarkus-app/quarkus-run.jar
|
|
```
|
|
|
|
### Option 3: Docker
|
|
|
|
```bash
|
|
# Build image
|
|
docker build -t auction-monitor:latest .
|
|
|
|
# Run container
|
|
docker run -p 8081:8081 \
|
|
-v $(pwd)/data:/mnt/okcomputer/output \
|
|
auction-monitor:latest
|
|
```
|
|
|
|
### Option 4: Docker Compose (Recommended)
|
|
|
|
```bash
|
|
# Start services
|
|
docker-compose up -d
|
|
|
|
# View logs
|
|
docker-compose logs -f
|
|
|
|
# Stop services
|
|
docker-compose down
|
|
```
|
|
|
|
---
|
|
|
|
## 🔧 Configuration
|
|
|
|
### application.properties
|
|
|
|
All configuration is in `src/main/resources/application.properties`:
|
|
|
|
```properties
|
|
# Database
|
|
auction.database.path=C:\\mnt\\okcomputer\\output\\cache.db
|
|
auction.images.path=C:\\mnt\\okcomputer\\output\\images
|
|
|
|
# Notifications
|
|
auction.notification.config=desktop
|
|
# Or for email: smtp:your@gmail.com:app_password:recipient@example.com
|
|
|
|
# YOLO Models (optional)
|
|
auction.yolo.config=models/yolov4.cfg
|
|
auction.yolo.weights=models/yolov4.weights
|
|
auction.yolo.classes=models/coco.names
|
|
|
|
# Workflow Schedules (cron expressions)
|
|
auction.workflow.scraper-import.cron=0 */30 * * * ? # Every 30 min
|
|
auction.workflow.image-processing.cron=0 0 * * * ? # Every 1 hour
|
|
auction.workflow.bid-monitoring.cron=0 */15 * * * ? # Every 15 min
|
|
auction.workflow.closing-alerts.cron=0 */5 * * * ? # Every 5 min
|
|
|
|
# HTTP Server
|
|
quarkus.http.port=8081
|
|
quarkus.http.host=0.0.0.0
|
|
```
|
|
|
|
### Environment Variables
|
|
|
|
Override configuration with environment variables:
|
|
|
|
```bash
|
|
export AUCTION_DATABASE_PATH=/path/to/cache.db
|
|
export AUCTION_NOTIFICATION_CONFIG=desktop
|
|
export QUARKUS_HTTP_PORT=8081
|
|
```
|
|
|
|
---
|
|
|
|
## 📅 Scheduled Workflows
|
|
|
|
Quarkus automatically runs these workflows based on cron expressions:
|
|
|
|
| Workflow | Schedule | Cron Expression | Description |
|
|
|----------|----------|-----------------|-------------|
|
|
| **Scraper Import** | Every 30 min | `0 */30 * * * ?` | Import auctions/lots from external scraper |
|
|
| **Image Processing** | Every 1 hour | `0 0 * * * ?` | Download images & run object detection |
|
|
| **Bid Monitoring** | Every 15 min | `0 */15 * * * ?` | Check for bid changes |
|
|
| **Closing Alerts** | Every 5 min | `0 */5 * * * ?` | Send alerts for lots closing soon |
|
|
|
|
### Cron Expression Format
|
|
|
|
```
|
|
┌───────────── second (0-59)
|
|
│ ┌───────────── minute (0-59)
|
|
│ │ ┌───────────── hour (0-23)
|
|
│ │ │ ┌───────────── day of month (1-31)
|
|
│ │ │ │ ┌───────────── month (1-12)
|
|
│ │ │ │ │ ┌───────────── day of week (0-6, Sunday=0)
|
|
│ │ │ │ │ │
|
|
0 */30 * * * ? = Every 30 minutes
|
|
0 0 * * * ? = Every hour at minute 0
|
|
0 0 0 * * ? = Every day at midnight
|
|
```
|
|
|
|
---
|
|
|
|
## 🌐 REST API
|
|
|
|
### Base URL
|
|
```
|
|
http://localhost:8081/api/monitor
|
|
```
|
|
|
|
### Endpoints
|
|
|
|
#### 1. Get Status
|
|
```bash
|
|
GET /api/monitor/status
|
|
|
|
# Example
|
|
curl http://localhost:8081/api/monitor/status
|
|
|
|
# Response
|
|
{
|
|
"running": true,
|
|
"auctions": 25,
|
|
"lots": 150,
|
|
"images": 300,
|
|
"closingSoon": 5
|
|
}
|
|
```
|
|
|
|
#### 2. Get Statistics
|
|
```bash
|
|
GET /api/monitor/statistics
|
|
|
|
# Example
|
|
curl http://localhost:8081/api/monitor/statistics
|
|
|
|
# Response
|
|
{
|
|
"totalAuctions": 25,
|
|
"totalLots": 150,
|
|
"totalImages": 300,
|
|
"activeLots": 120,
|
|
"lotsWithBids": 80,
|
|
"totalBidValue": "€125,450.00",
|
|
"averageBid": "€1,568.13"
|
|
}
|
|
```
|
|
|
|
#### 3. Trigger Workflows Manually
|
|
|
|
```bash
|
|
# Scraper Import
|
|
POST /api/monitor/trigger/scraper-import
|
|
curl -X POST http://localhost:8081/api/monitor/trigger/scraper-import
|
|
|
|
# Image Processing
|
|
POST /api/monitor/trigger/image-processing
|
|
curl -X POST http://localhost:8081/api/monitor/trigger/image-processing
|
|
|
|
# Bid Monitoring
|
|
POST /api/monitor/trigger/bid-monitoring
|
|
curl -X POST http://localhost:8081/api/monitor/trigger/bid-monitoring
|
|
|
|
# Closing Alerts
|
|
POST /api/monitor/trigger/closing-alerts
|
|
curl -X POST http://localhost:8081/api/monitor/trigger/closing-alerts
|
|
```
|
|
|
|
#### 4. Get Auctions
|
|
```bash
|
|
# All auctions
|
|
GET /api/monitor/auctions
|
|
curl http://localhost:8081/api/monitor/auctions
|
|
|
|
# Filter by country
|
|
GET /api/monitor/auctions?country=NL
|
|
curl http://localhost:8081/api/monitor/auctions?country=NL
|
|
```
|
|
|
|
#### 5. Get Lots
|
|
```bash
|
|
# Active lots
|
|
GET /api/monitor/lots
|
|
curl http://localhost:8081/api/monitor/lots
|
|
|
|
# Lots closing soon (within 30 minutes by default)
|
|
GET /api/monitor/lots/closing-soon
|
|
curl http://localhost:8081/api/monitor/lots/closing-soon
|
|
|
|
# Custom minutes threshold
|
|
GET /api/monitor/lots/closing-soon?minutes=60
|
|
curl http://localhost:8081/api/monitor/lots/closing-soon?minutes=60
|
|
```
|
|
|
|
#### 6. Get Lot Images
|
|
```bash
|
|
GET /api/monitor/lots/{lotId}/images
|
|
|
|
# Example
|
|
curl http://localhost:8081/api/monitor/lots/12345/images
|
|
```
|
|
|
|
#### 7. Test Notification
|
|
```bash
|
|
POST /api/monitor/test-notification
|
|
Content-Type: application/json
|
|
|
|
{
|
|
"message": "Test message",
|
|
"title": "Test Title",
|
|
"priority": "0"
|
|
}
|
|
|
|
# Example
|
|
curl -X POST http://localhost:8081/api/monitor/test-notification \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"message":"Test notification","title":"Test","priority":"0"}'
|
|
```
|
|
|
|
---
|
|
|
|
## 🏥 Health Checks
|
|
|
|
Quarkus provides built-in health checks for Kubernetes/Docker:
|
|
|
|
### Liveness Probe
|
|
```bash
|
|
GET /health/live
|
|
|
|
# Example
|
|
curl http://localhost:8081/health/live
|
|
|
|
# Response
|
|
{
|
|
"status": "UP",
|
|
"checks": [
|
|
{
|
|
"name": "Auction Monitor is alive",
|
|
"status": "UP"
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
### Readiness Probe
|
|
```bash
|
|
GET /health/ready
|
|
|
|
# Example
|
|
curl http://localhost:8081/health/ready
|
|
|
|
# Response
|
|
{
|
|
"status": "UP",
|
|
"checks": [
|
|
{
|
|
"name": "database",
|
|
"status": "UP",
|
|
"data": {
|
|
"auctions": 25
|
|
}
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
### Startup Probe
|
|
```bash
|
|
GET /health/started
|
|
|
|
# Example
|
|
curl http://localhost:8081/health/started
|
|
```
|
|
|
|
### Combined Health
|
|
```bash
|
|
GET /health
|
|
|
|
# Returns all health checks
|
|
curl http://localhost:8081/health
|
|
```
|
|
|
|
---
|
|
|
|
## 🐳 Docker Deployment
|
|
|
|
### Build Image
|
|
|
|
```bash
|
|
docker build -t auction-monitor:1.0 .
|
|
```
|
|
|
|
### Run Container
|
|
|
|
```bash
|
|
docker run -d \
|
|
--name auction-monitor \
|
|
-p 8081:8081 \
|
|
-v $(pwd)/data:/mnt/okcomputer/output \
|
|
-e AUCTION_NOTIFICATION_CONFIG=desktop \
|
|
auction-monitor:1.0
|
|
```
|
|
|
|
### Docker Compose
|
|
|
|
```yaml
|
|
version: '3.8'
|
|
services:
|
|
auction-monitor:
|
|
image: auction-monitor:1.0
|
|
ports:
|
|
- "8081:8081"
|
|
volumes:
|
|
- ./data:/mnt/okcomputer/output
|
|
environment:
|
|
- AUCTION_DATABASE_PATH=/mnt/okcomputer/output/cache.db
|
|
- AUCTION_NOTIFICATION_CONFIG=desktop
|
|
healthcheck:
|
|
test: ["CMD", "wget", "--spider", "http://localhost:8081/health/live"]
|
|
interval: 30s
|
|
timeout: 3s
|
|
retries: 3
|
|
```
|
|
|
|
---
|
|
|
|
## ☸️ Kubernetes Deployment
|
|
|
|
### deployment.yaml
|
|
|
|
```yaml
|
|
apiVersion: apps/v1
|
|
kind: Deployment
|
|
metadata:
|
|
name: auction-monitor
|
|
spec:
|
|
replicas: 1
|
|
selector:
|
|
matchLabels:
|
|
app: auction-monitor
|
|
template:
|
|
metadata:
|
|
labels:
|
|
app: auction-monitor
|
|
spec:
|
|
containers:
|
|
- name: auction-monitor
|
|
image: auction-monitor:1.0
|
|
ports:
|
|
- containerPort: 8081
|
|
env:
|
|
- name: AUCTION_DATABASE_PATH
|
|
value: /data/cache.db
|
|
- name: QUARKUS_HTTP_PORT
|
|
value: "8081"
|
|
volumeMounts:
|
|
- name: data
|
|
mountPath: /mnt/okcomputer/output
|
|
livenessProbe:
|
|
httpGet:
|
|
path: /health/live
|
|
port: 8081
|
|
initialDelaySeconds: 10
|
|
periodSeconds: 30
|
|
readinessProbe:
|
|
httpGet:
|
|
path: /health/ready
|
|
port: 8081
|
|
initialDelaySeconds: 5
|
|
periodSeconds: 10
|
|
startupProbe:
|
|
httpGet:
|
|
path: /health/started
|
|
port: 8081
|
|
failureThreshold: 30
|
|
periodSeconds: 10
|
|
volumes:
|
|
- name: data
|
|
persistentVolumeClaim:
|
|
claimName: auction-data-pvc
|
|
---
|
|
apiVersion: v1
|
|
kind: Service
|
|
metadata:
|
|
name: auction-monitor
|
|
spec:
|
|
selector:
|
|
app: auction-monitor
|
|
ports:
|
|
- port: 8081
|
|
targetPort: 8081
|
|
type: LoadBalancer
|
|
```
|
|
|
|
---
|
|
|
|
## 🔄 Development Mode
|
|
|
|
Quarkus dev mode provides live reload for rapid development:
|
|
|
|
```bash
|
|
# Start dev mode
|
|
mvn quarkus:dev
|
|
|
|
# Features available:
|
|
# - Live reload (no restart needed)
|
|
# - Dev UI: http://localhost:8081/q/dev/
|
|
# - Continuous testing
|
|
# - Debug on port 5005
|
|
```
|
|
|
|
### Dev UI
|
|
|
|
Access at: `http://localhost:8081/q/dev/`
|
|
|
|
Features:
|
|
- Configuration editor
|
|
- Scheduler dashboard
|
|
- Health checks
|
|
- REST endpoints explorer
|
|
- Continuous testing
|
|
|
|
---
|
|
|
|
## 🧪 Testing
|
|
|
|
### Run All Tests
|
|
```bash
|
|
mvn test
|
|
```
|
|
|
|
### Run Quarkus Tests
|
|
```bash
|
|
mvn test -Dtest=*QuarkusTest
|
|
```
|
|
|
|
### Integration Test with Running Application
|
|
```bash
|
|
# Terminal 1: Start application
|
|
mvn quarkus:dev
|
|
|
|
# Terminal 2: Run integration tests
|
|
curl http://localhost:8081/api/monitor/status
|
|
curl http://localhost:8081/health/live
|
|
curl -X POST http://localhost:8081/api/monitor/trigger/scraper-import
|
|
```
|
|
|
|
---
|
|
|
|
## 📊 Monitoring & Logging
|
|
|
|
### View Logs
|
|
|
|
```bash
|
|
# Docker
|
|
docker logs -f auction-monitor
|
|
|
|
# Docker Compose
|
|
docker-compose logs -f
|
|
|
|
# Kubernetes
|
|
kubectl logs -f deployment/auction-monitor
|
|
```
|
|
|
|
### Log Levels
|
|
|
|
Configure in `application.properties`:
|
|
|
|
```properties
|
|
# Production
|
|
quarkus.log.console.level=INFO
|
|
|
|
# Development
|
|
%dev.quarkus.log.console.level=DEBUG
|
|
|
|
# Specific logger
|
|
quarkus.log.category."com.auction".level=DEBUG
|
|
```
|
|
|
|
### Scheduled Job Logs
|
|
|
|
```
|
|
14:30:00 INFO [com.auc.Qua] (executor-thread-1) 📥 [WORKFLOW 1] Importing scraper data...
|
|
14:30:00 INFO [com.auc.Qua] (executor-thread-1) → Imported 5 auctions
|
|
14:30:00 INFO [com.auc.Qua] (executor-thread-1) → Imported 25 lots
|
|
14:30:00 INFO [com.auc.Qua] (executor-thread-1) ✓ Scraper import completed in 1250ms
|
|
```
|
|
|
|
---
|
|
|
|
## ⚙️ Performance
|
|
|
|
### Startup Time
|
|
- **JVM Mode**: ~0.5 seconds
|
|
- **Native Image**: ~0.014 seconds
|
|
|
|
### Memory Footprint
|
|
- **JVM Mode**: ~50MB RSS
|
|
- **Native Image**: ~15MB RSS
|
|
|
|
### Build Native Image (Optional)
|
|
|
|
```bash
|
|
# Requires GraalVM
|
|
mvn package -Pnative
|
|
|
|
# Run native executable
|
|
./target/troostwijk-scraper-1.0-SNAPSHOT-runner
|
|
```
|
|
|
|
---
|
|
|
|
## 🔐 Security
|
|
|
|
### Environment Variables for Secrets
|
|
|
|
```bash
|
|
# Don't commit credentials!
|
|
export AUCTION_NOTIFICATION_CONFIG=smtp:user@gmail.com:SECRET_PASSWORD:recipient@example.com
|
|
|
|
# Or use Kubernetes secrets
|
|
kubectl create secret generic auction-secrets \
|
|
--from-literal=notification-config='smtp:user@gmail.com:password:recipient@example.com'
|
|
```
|
|
|
|
### Kubernetes Secret
|
|
|
|
```yaml
|
|
apiVersion: v1
|
|
kind: Secret
|
|
metadata:
|
|
name: auction-secrets
|
|
type: Opaque
|
|
stringData:
|
|
notification-config: smtp:user@gmail.com:app_password:recipient@example.com
|
|
```
|
|
|
|
---
|
|
|
|
## 🛠️ Troubleshooting
|
|
|
|
### Issue: Schedulers not running
|
|
|
|
**Check scheduler status:**
|
|
```bash
|
|
curl http://localhost:8081/health/ready
|
|
```
|
|
|
|
**Enable debug logging:**
|
|
```properties
|
|
quarkus.log.category."io.quarkus.scheduler".level=DEBUG
|
|
```
|
|
|
|
### Issue: Database not found
|
|
|
|
**Check file permissions:**
|
|
```bash
|
|
ls -la C:/mnt/okcomputer/output/cache.db
|
|
```
|
|
|
|
**Create directory:**
|
|
```bash
|
|
mkdir -p C:/mnt/okcomputer/output
|
|
```
|
|
|
|
### Issue: Port 8081 already in use
|
|
|
|
**Change port:**
|
|
```bash
|
|
mvn quarkus:dev -Dquarkus.http.port=8082
|
|
# Or
|
|
export QUARKUS_HTTP_PORT=8082
|
|
```
|
|
|
|
### Issue: Health check failing
|
|
|
|
**Check application logs:**
|
|
```bash
|
|
docker logs auction-monitor
|
|
```
|
|
|
|
**Verify database connection:**
|
|
```bash
|
|
curl http://localhost:8081/health/ready
|
|
```
|
|
|
|
---
|
|
|
|
## 📚 Additional Resources
|
|
|
|
- [Quarkus Official Guide](https://quarkus.io/guides/)
|
|
- [Quarkus Scheduler](https://quarkus.io/guides/scheduler)
|
|
- [Quarkus REST](https://quarkus.io/guides/rest)
|
|
- [Quarkus Health](https://quarkus.io/guides/smallrye-health)
|
|
- [Quarkus Docker](https://quarkus.io/guides/container-image)
|
|
|
|
---
|
|
|
|
## Summary
|
|
|
|
✅ **Quarkus Framework** integrated for modern Java development
|
|
✅ **CDI/Dependency Injection** for clean architecture
|
|
✅ **@Scheduled** annotations for cron-based workflows
|
|
✅ **REST API** for control and monitoring
|
|
✅ **Health Checks** for Kubernetes/Docker
|
|
✅ **Fast Startup** and low memory footprint
|
|
✅ **Docker/Kubernetes** ready
|
|
✅ **Production** optimized
|
|
|
|
**Run and enjoy! 🎉**
|