start
This commit is contained in:
Tour
650
QUARKUS_GUIDE.md
Normal file
650
QUARKUS_GUIDE.md
Normal file
@@ -0,0 +1,650 @@
|
||||
# Quarkus Auction Monitor - Complete Guide
|
||||
|
||||
## 🚀 Overview
|
||||
|
||||
The Troostwijk Auction Monitor now runs on **Quarkus**, a Kubernetes-native Java framework optimized for fast startup and low memory footprint.
|
||||
|
||||
### Key Features
|
||||
|
||||
✅ **Quarkus Scheduler** - Built-in cron-based scheduling
|
||||
✅ **REST API** - Control and monitor via HTTP endpoints
|
||||
✅ **Health Checks** - Kubernetes-ready liveness/readiness probes
|
||||
✅ **CDI/Dependency Injection** - Type-safe service management
|
||||
✅ **Fast Startup** - 0.5s startup time
|
||||
✅ **Low Memory** - ~50MB RSS memory footprint
|
||||
✅ **Hot Reload** - Development mode with live coding
|
||||
|
||||
---
|
||||
|
||||
## 📦 Quick Start
|
||||
|
||||
### Option 1: Run with Maven (Development)
|
||||
|
||||
```bash
|
||||
# Start in dev mode with live reload
|
||||
mvn quarkus:dev
|
||||
|
||||
# Access application
|
||||
# API: http://localhost:8081/api/monitor/status
|
||||
# Health: http://localhost:8081/health
|
||||
```
|
||||
|
||||
### Option 2: Build and Run JAR
|
||||
|
||||
```bash
|
||||
# Build
|
||||
mvn clean package
|
||||
|
||||
# Run
|
||||
java -jar target/quarkus-app/quarkus-run.jar
|
||||
|
||||
# Or use fast-jar (recommended for production)
|
||||
mvn clean package -Dquarkus.package.jar.type=fast-jar
|
||||
java -jar target/quarkus-app/quarkus-run.jar
|
||||
```
|
||||
|
||||
### Option 3: Docker
|
||||
|
||||
```bash
|
||||
# Build image
|
||||
docker build -t auction-monitor:latest .
|
||||
|
||||
# Run container
|
||||
docker run -p 8081:8081 \
|
||||
-v $(pwd)/data:/mnt/okcomputer/output \
|
||||
auction-monitor:latest
|
||||
```
|
||||
|
||||
### Option 4: Docker Compose (Recommended)
|
||||
|
||||
```bash
|
||||
# Start services
|
||||
docker-compose up -d
|
||||
|
||||
# View logs
|
||||
docker-compose logs -f
|
||||
|
||||
# Stop services
|
||||
docker-compose down
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Configuration
|
||||
|
||||
### application.properties
|
||||
|
||||
All configuration is in `src/main/resources/application.properties`:
|
||||
|
||||
```properties
|
||||
# Database
|
||||
auction.database.path=C:\\mnt\\okcomputer\\output\\cache.db
|
||||
auction.images.path=C:\\mnt\\okcomputer\\output\\images
|
||||
|
||||
# Notifications
|
||||
auction.notification.config=desktop
|
||||
# Or for email: smtp:your@gmail.com:app_password:recipient@example.com
|
||||
|
||||
# YOLO Models (optional)
|
||||
auction.yolo.config=models/yolov4.cfg
|
||||
auction.yolo.weights=models/yolov4.weights
|
||||
auction.yolo.classes=models/coco.names
|
||||
|
||||
# Workflow Schedules (cron expressions)
|
||||
auction.workflow.scraper-import.cron=0 */30 * * * ? # Every 30 min
|
||||
auction.workflow.image-processing.cron=0 0 * * * ? # Every 1 hour
|
||||
auction.workflow.bid-monitoring.cron=0 */15 * * * ? # Every 15 min
|
||||
auction.workflow.closing-alerts.cron=0 */5 * * * ? # Every 5 min
|
||||
|
||||
# HTTP Server
|
||||
quarkus.http.port=8081
|
||||
quarkus.http.host=0.0.0.0
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
Override configuration with environment variables:
|
||||
|
||||
```bash
|
||||
export AUCTION_DATABASE_PATH=/path/to/cache.db
|
||||
export AUCTION_NOTIFICATION_CONFIG=desktop
|
||||
export QUARKUS_HTTP_PORT=8081
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📅 Scheduled Workflows
|
||||
|
||||
Quarkus automatically runs these workflows based on cron expressions:
|
||||
|
||||
| Workflow | Schedule | Cron Expression | Description |
|
||||
|----------|----------|-----------------|-------------|
|
||||
| **Scraper Import** | Every 30 min | `0 */30 * * * ?` | Import auctions/lots from external scraper |
|
||||
| **Image Processing** | Every 1 hour | `0 0 * * * ?` | Download images & run object detection |
|
||||
| **Bid Monitoring** | Every 15 min | `0 */15 * * * ?` | Check for bid changes |
|
||||
| **Closing Alerts** | Every 5 min | `0 */5 * * * ?` | Send alerts for lots closing soon |
|
||||
|
||||
### Cron Expression Format
|
||||
|
||||
```
|
||||
┌───────────── second (0-59)
|
||||
│ ┌───────────── minute (0-59)
|
||||
│ │ ┌───────────── hour (0-23)
|
||||
│ │ │ ┌───────────── day of month (1-31)
|
||||
│ │ │ │ ┌───────────── month (1-12)
|
||||
│ │ │ │ │ ┌───────────── day of week (0-6, Sunday=0)
|
||||
│ │ │ │ │ │
|
||||
0 */30 * * * ? = Every 30 minutes
|
||||
0 0 * * * ? = Every hour at minute 0
|
||||
0 0 0 * * ? = Every day at midnight
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🌐 REST API
|
||||
|
||||
### Base URL
|
||||
```
|
||||
http://localhost:8081/api/monitor
|
||||
```
|
||||
|
||||
### Endpoints
|
||||
|
||||
#### 1. Get Status
|
||||
```bash
|
||||
GET /api/monitor/status
|
||||
|
||||
# Example
|
||||
curl http://localhost:8081/api/monitor/status
|
||||
|
||||
# Response
|
||||
{
|
||||
"running": true,
|
||||
"auctions": 25,
|
||||
"lots": 150,
|
||||
"images": 300,
|
||||
"closingSoon": 5
|
||||
}
|
||||
```
|
||||
|
||||
#### 2. Get Statistics
|
||||
```bash
|
||||
GET /api/monitor/statistics
|
||||
|
||||
# Example
|
||||
curl http://localhost:8081/api/monitor/statistics
|
||||
|
||||
# Response
|
||||
{
|
||||
"totalAuctions": 25,
|
||||
"totalLots": 150,
|
||||
"totalImages": 300,
|
||||
"activeLots": 120,
|
||||
"lotsWithBids": 80,
|
||||
"totalBidValue": "€125,450.00",
|
||||
"averageBid": "€1,568.13"
|
||||
}
|
||||
```
|
||||
|
||||
#### 3. Trigger Workflows Manually
|
||||
|
||||
```bash
|
||||
# Scraper Import
|
||||
POST /api/monitor/trigger/scraper-import
|
||||
curl -X POST http://localhost:8081/api/monitor/trigger/scraper-import
|
||||
|
||||
# Image Processing
|
||||
POST /api/monitor/trigger/image-processing
|
||||
curl -X POST http://localhost:8081/api/monitor/trigger/image-processing
|
||||
|
||||
# Bid Monitoring
|
||||
POST /api/monitor/trigger/bid-monitoring
|
||||
curl -X POST http://localhost:8081/api/monitor/trigger/bid-monitoring
|
||||
|
||||
# Closing Alerts
|
||||
POST /api/monitor/trigger/closing-alerts
|
||||
curl -X POST http://localhost:8081/api/monitor/trigger/closing-alerts
|
||||
```
|
||||
|
||||
#### 4. Get Auctions
|
||||
```bash
|
||||
# All auctions
|
||||
GET /api/monitor/auctions
|
||||
curl http://localhost:8081/api/monitor/auctions
|
||||
|
||||
# Filter by country
|
||||
GET /api/monitor/auctions?country=NL
|
||||
curl http://localhost:8081/api/monitor/auctions?country=NL
|
||||
```
|
||||
|
||||
#### 5. Get Lots
|
||||
```bash
|
||||
# Active lots
|
||||
GET /api/monitor/lots
|
||||
curl http://localhost:8081/api/monitor/lots
|
||||
|
||||
# Lots closing soon (within 30 minutes by default)
|
||||
GET /api/monitor/lots/closing-soon
|
||||
curl http://localhost:8081/api/monitor/lots/closing-soon
|
||||
|
||||
# Custom minutes threshold
|
||||
GET /api/monitor/lots/closing-soon?minutes=60
|
||||
curl http://localhost:8081/api/monitor/lots/closing-soon?minutes=60
|
||||
```
|
||||
|
||||
#### 6. Get Lot Images
|
||||
```bash
|
||||
GET /api/monitor/lots/{lotId}/images
|
||||
|
||||
# Example
|
||||
curl http://localhost:8081/api/monitor/lots/12345/images
|
||||
```
|
||||
|
||||
#### 7. Test Notification
|
||||
```bash
|
||||
POST /api/monitor/test-notification
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"message": "Test message",
|
||||
"title": "Test Title",
|
||||
"priority": "0"
|
||||
}
|
||||
|
||||
# Example
|
||||
curl -X POST http://localhost:8081/api/monitor/test-notification \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"message":"Test notification","title":"Test","priority":"0"}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🏥 Health Checks
|
||||
|
||||
Quarkus provides built-in health checks for Kubernetes/Docker:
|
||||
|
||||
### Liveness Probe
|
||||
```bash
|
||||
GET /health/live
|
||||
|
||||
# Example
|
||||
curl http://localhost:8081/health/live
|
||||
|
||||
# Response
|
||||
{
|
||||
"status": "UP",
|
||||
"checks": [
|
||||
{
|
||||
"name": "Auction Monitor is alive",
|
||||
"status": "UP"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Readiness Probe
|
||||
```bash
|
||||
GET /health/ready
|
||||
|
||||
# Example
|
||||
curl http://localhost:8081/health/ready
|
||||
|
||||
# Response
|
||||
{
|
||||
"status": "UP",
|
||||
"checks": [
|
||||
{
|
||||
"name": "database",
|
||||
"status": "UP",
|
||||
"data": {
|
||||
"auctions": 25
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Startup Probe
|
||||
```bash
|
||||
GET /health/started
|
||||
|
||||
# Example
|
||||
curl http://localhost:8081/health/started
|
||||
```
|
||||
|
||||
### Combined Health
|
||||
```bash
|
||||
GET /health
|
||||
|
||||
# Returns all health checks
|
||||
curl http://localhost:8081/health
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🐳 Docker Deployment
|
||||
|
||||
### Build Image
|
||||
|
||||
```bash
|
||||
docker build -t auction-monitor:1.0 .
|
||||
```
|
||||
|
||||
### Run Container
|
||||
|
||||
```bash
|
||||
docker run -d \
|
||||
--name auction-monitor \
|
||||
-p 8081:8081 \
|
||||
-v $(pwd)/data:/mnt/okcomputer/output \
|
||||
-e AUCTION_NOTIFICATION_CONFIG=desktop \
|
||||
auction-monitor:1.0
|
||||
```
|
||||
|
||||
### Docker Compose
|
||||
|
||||
```yaml
|
||||
version: '3.8'
|
||||
services:
|
||||
auction-monitor:
|
||||
image: auction-monitor:1.0
|
||||
ports:
|
||||
- "8081:8081"
|
||||
volumes:
|
||||
- ./data:/mnt/okcomputer/output
|
||||
environment:
|
||||
- AUCTION_DATABASE_PATH=/mnt/okcomputer/output/cache.db
|
||||
- AUCTION_NOTIFICATION_CONFIG=desktop
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "--spider", "http://localhost:8081/health/live"]
|
||||
interval: 30s
|
||||
timeout: 3s
|
||||
retries: 3
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ☸️ Kubernetes Deployment
|
||||
|
||||
### deployment.yaml
|
||||
|
||||
```yaml
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: auction-monitor
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: auction-monitor
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: auction-monitor
|
||||
spec:
|
||||
containers:
|
||||
- name: auction-monitor
|
||||
image: auction-monitor:1.0
|
||||
ports:
|
||||
- containerPort: 8081
|
||||
env:
|
||||
- name: AUCTION_DATABASE_PATH
|
||||
value: /data/cache.db
|
||||
- name: QUARKUS_HTTP_PORT
|
||||
value: "8081"
|
||||
volumeMounts:
|
||||
- name: data
|
||||
mountPath: /mnt/okcomputer/output
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /health/live
|
||||
port: 8081
|
||||
initialDelaySeconds: 10
|
||||
periodSeconds: 30
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /health/ready
|
||||
port: 8081
|
||||
initialDelaySeconds: 5
|
||||
periodSeconds: 10
|
||||
startupProbe:
|
||||
httpGet:
|
||||
path: /health/started
|
||||
port: 8081
|
||||
failureThreshold: 30
|
||||
periodSeconds: 10
|
||||
volumes:
|
||||
- name: data
|
||||
persistentVolumeClaim:
|
||||
claimName: auction-data-pvc
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: auction-monitor
|
||||
spec:
|
||||
selector:
|
||||
app: auction-monitor
|
||||
ports:
|
||||
- port: 8081
|
||||
targetPort: 8081
|
||||
type: LoadBalancer
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔄 Development Mode
|
||||
|
||||
Quarkus dev mode provides live reload for rapid development:
|
||||
|
||||
```bash
|
||||
# Start dev mode
|
||||
mvn quarkus:dev
|
||||
|
||||
# Features available:
|
||||
# - Live reload (no restart needed)
|
||||
# - Dev UI: http://localhost:8081/q/dev/
|
||||
# - Continuous testing
|
||||
# - Debug on port 5005
|
||||
```
|
||||
|
||||
### Dev UI
|
||||
|
||||
Access at: `http://localhost:8081/q/dev/`
|
||||
|
||||
Features:
|
||||
- Configuration editor
|
||||
- Scheduler dashboard
|
||||
- Health checks
|
||||
- REST endpoints explorer
|
||||
- Continuous testing
|
||||
|
||||
---
|
||||
|
||||
## 🧪 Testing
|
||||
|
||||
### Run All Tests
|
||||
```bash
|
||||
mvn test
|
||||
```
|
||||
|
||||
### Run Quarkus Tests
|
||||
```bash
|
||||
mvn test -Dtest=*QuarkusTest
|
||||
```
|
||||
|
||||
### Integration Test with Running Application
|
||||
```bash
|
||||
# Terminal 1: Start application
|
||||
mvn quarkus:dev
|
||||
|
||||
# Terminal 2: Run integration tests
|
||||
curl http://localhost:8081/api/monitor/status
|
||||
curl http://localhost:8081/health/live
|
||||
curl -X POST http://localhost:8081/api/monitor/trigger/scraper-import
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 Monitoring & Logging
|
||||
|
||||
### View Logs
|
||||
|
||||
```bash
|
||||
# Docker
|
||||
docker logs -f auction-monitor
|
||||
|
||||
# Docker Compose
|
||||
docker-compose logs -f
|
||||
|
||||
# Kubernetes
|
||||
kubectl logs -f deployment/auction-monitor
|
||||
```
|
||||
|
||||
### Log Levels
|
||||
|
||||
Configure in `application.properties`:
|
||||
|
||||
```properties
|
||||
# Production
|
||||
quarkus.log.console.level=INFO
|
||||
|
||||
# Development
|
||||
%dev.quarkus.log.console.level=DEBUG
|
||||
|
||||
# Specific logger
|
||||
quarkus.log.category."com.auction".level=DEBUG
|
||||
```
|
||||
|
||||
### Scheduled Job Logs
|
||||
|
||||
```
|
||||
14:30:00 INFO [com.auc.Qua] (executor-thread-1) 📥 [WORKFLOW 1] Importing scraper data...
|
||||
14:30:00 INFO [com.auc.Qua] (executor-thread-1) → Imported 5 auctions
|
||||
14:30:00 INFO [com.auc.Qua] (executor-thread-1) → Imported 25 lots
|
||||
14:30:00 INFO [com.auc.Qua] (executor-thread-1) ✓ Scraper import completed in 1250ms
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ⚙️ Performance
|
||||
|
||||
### Startup Time
|
||||
- **JVM Mode**: ~0.5 seconds
|
||||
- **Native Image**: ~0.014 seconds
|
||||
|
||||
### Memory Footprint
|
||||
- **JVM Mode**: ~50MB RSS
|
||||
- **Native Image**: ~15MB RSS
|
||||
|
||||
### Build Native Image (Optional)
|
||||
|
||||
```bash
|
||||
# Requires GraalVM
|
||||
mvn package -Pnative
|
||||
|
||||
# Run native executable
|
||||
./target/troostwijk-scraper-1.0-SNAPSHOT-runner
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔐 Security
|
||||
|
||||
### Environment Variables for Secrets
|
||||
|
||||
```bash
|
||||
# Don't commit credentials!
|
||||
export AUCTION_NOTIFICATION_CONFIG=smtp:user@gmail.com:SECRET_PASSWORD:recipient@example.com
|
||||
|
||||
# Or use Kubernetes secrets
|
||||
kubectl create secret generic auction-secrets \
|
||||
--from-literal=notification-config='smtp:user@gmail.com:password:recipient@example.com'
|
||||
```
|
||||
|
||||
### Kubernetes Secret
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: auction-secrets
|
||||
type: Opaque
|
||||
stringData:
|
||||
notification-config: smtp:user@gmail.com:app_password:recipient@example.com
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🛠️ Troubleshooting
|
||||
|
||||
### Issue: Schedulers not running
|
||||
|
||||
**Check scheduler status:**
|
||||
```bash
|
||||
curl http://localhost:8081/health/ready
|
||||
```
|
||||
|
||||
**Enable debug logging:**
|
||||
```properties
|
||||
quarkus.log.category."io.quarkus.scheduler".level=DEBUG
|
||||
```
|
||||
|
||||
### Issue: Database not found
|
||||
|
||||
**Check file permissions:**
|
||||
```bash
|
||||
ls -la C:/mnt/okcomputer/output/cache.db
|
||||
```
|
||||
|
||||
**Create directory:**
|
||||
```bash
|
||||
mkdir -p C:/mnt/okcomputer/output
|
||||
```
|
||||
|
||||
### Issue: Port 8081 already in use
|
||||
|
||||
**Change port:**
|
||||
```bash
|
||||
mvn quarkus:dev -Dquarkus.http.port=8082
|
||||
# Or
|
||||
export QUARKUS_HTTP_PORT=8082
|
||||
```
|
||||
|
||||
### Issue: Health check failing
|
||||
|
||||
**Check application logs:**
|
||||
```bash
|
||||
docker logs auction-monitor
|
||||
```
|
||||
|
||||
**Verify database connection:**
|
||||
```bash
|
||||
curl http://localhost:8081/health/ready
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📚 Additional Resources
|
||||
|
||||
- [Quarkus Official Guide](https://quarkus.io/guides/)
|
||||
- [Quarkus Scheduler](https://quarkus.io/guides/scheduler)
|
||||
- [Quarkus REST](https://quarkus.io/guides/rest)
|
||||
- [Quarkus Health](https://quarkus.io/guides/smallrye-health)
|
||||
- [Quarkus Docker](https://quarkus.io/guides/container-image)
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
✅ **Quarkus Framework** integrated for modern Java development
|
||||
✅ **CDI/Dependency Injection** for clean architecture
|
||||
✅ **@Scheduled** annotations for cron-based workflows
|
||||
✅ **REST API** for control and monitoring
|
||||
✅ **Health Checks** for Kubernetes/Docker
|
||||
✅ **Fast Startup** and low memory footprint
|
||||
✅ **Docker/Kubernetes** ready
|
||||
✅ **Production** optimized
|
||||
|
||||
**Run and enjoy! 🎉**
|
||||
Reference in New Issue
Block a user