auctiora/wiki/WORKFLOW_GUIDE.md

## Troostwijk Auction Monitor - Workflow Integration Guide

Complete guide for running the auction monitoring system with scheduled workflows, cron jobs, and event-driven triggers.

---

## Table of Contents

1. [Overview](#overview)
2. [Running Modes](#running-modes)
3. [Workflow Orchestration](#workflow-orchestration)
4. [Windows Scheduling](#windows-scheduling)
5. [Event-Driven Triggers](#event-driven-triggers)
6. [Configuration](#configuration)
7. [Monitoring & Debugging](#monitoring--debugging)

---

## Overview

The Troostwijk Auction Monitor supports multiple execution modes:

- **Workflow Mode** (Recommended): Continuous operation with built-in scheduling
- **Once Mode**: Single execution for external schedulers (Windows Task Scheduler, cron)
- **Legacy Mode**: Original monitoring approach
- **Status Mode**: Quick status check

---

## Running Modes

### 1. Workflow Mode (Default - Recommended)

**Runs all workflows continuously with built-in scheduling.**

```bash
# Windows
java -jar target\troostwijk-scraper-1.0-SNAPSHOT-jar-with-dependencies.jar workflow

# Or simply (workflow is default)
java -jar target\troostwijk-scraper-1.0-SNAPSHOT-jar-with-dependencies.jar

# Using batch script
run-workflow.bat
```

**What it does:**
- ✅ Imports scraper data every 30 minutes
- ✅ Processes images every 1 hour
- ✅ Monitors bids every 15 minutes
- ✅ Checks closing times every 5 minutes

**Best for:**
- Production deployment
- Long-running services
- Development/testing

---

### 2. Once Mode (For External Schedulers)

**Runs complete workflow once and exits.**

```bash
# Windows
java -jar target\troostwijk-scraper-1.0-SNAPSHOT-jar-with-dependencies.jar once

# Using batch script
run-once.bat
```

**What it does:**
1. Imports scraper data
2. Processes pending images
3. Monitors bids
4. Checks closing times
5. Exits

**Best for:**
- Windows Task Scheduler
- Cron jobs (Linux/Mac)
- Manual execution
- Testing

---

### 3. Legacy Mode

**Original monitoring approach (backward compatibility).**

```bash
java -jar target\troostwijk-scraper-1.0-SNAPSHOT-jar-with-dependencies.jar legacy
```

**Best for:**
- Maintaining existing deployments
- Troubleshooting

---

### 4. Status Mode

**Shows current status and exits.**

```bash
java -jar target\troostwijk-scraper-1.0-SNAPSHOT-jar-with-dependencies.jar status

# Using batch script
check-status.bat
```

**Output:**
```
📊 Workflow Status:
  Running: No
  Auctions: 25
  Lots: 150
  Images: 300
  Closing soon (< 30 min): 5
```

---

## Workflow Orchestration

The `WorkflowOrchestrator` coordinates 4 scheduled workflows:

### Workflow 1: Scraper Data Import
**Frequency:** Every 30 minutes
**Purpose:** Import new auctions and lots from external scraper

**Process:**
1. Import auctions from scraper database
2. Import lots from scraper database
3. Import image URLs
4. Send notification if significant data imported

**Code Location:** `WorkflowOrchestrator.java:110`

---

### Workflow 2: Image Processing
**Frequency:** Every 1 hour
**Purpose:** Download images and run object detection

**Process:**
1. Get unprocessed images from database
2. Download each image
3. Run YOLO object detection
4. Save labels to database
5. Send notification for interesting detections (3+ objects)

**Code Location:** `WorkflowOrchestrator.java:150`

---

### Workflow 3: Bid Monitoring
**Frequency:** Every 15 minutes
**Purpose:** Check for bid changes and send notifications

**Process:**
1. Get all active lots
2. Check for bid changes (via external scraper updates)
3. Send notifications for bid increases

**Code Location:** `WorkflowOrchestrator.java:210`

**Note:** The external scraper updates bids; this workflow monitors and notifies.

---

### Workflow 4: Closing Alerts
**Frequency:** Every 5 minutes
**Purpose:** Send alerts for lots closing soon

**Process:**
1. Get all active lots
2. Check closing times
3. Send high-priority notification for lots closing in < 5 min
4. Mark as notified to prevent duplicates

**Code Location:** `WorkflowOrchestrator.java:240`

---

## Windows Scheduling

### Option A: Use Built-in Workflow Mode (Recommended)

**Run as a Windows Service or startup application:**

1. Create shortcut to `run-workflow.bat`
2. Place in: `C:\Users\[YourUser]\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup`
3. Monitor will start automatically on login

---

### Option B: Windows Task Scheduler (Once Mode)

**Automated setup:**

```powershell
# Run PowerShell as Administrator
.\setup-windows-task.ps1
```

This creates two tasks:
- `TroostwijkMonitor-Workflow`: Runs every 30 minutes
- `TroostwijkMonitor-StatusCheck`: Runs every 6 hours

**Manual setup:**

1. Open Task Scheduler
2. Create Basic Task
3. Configure:
   - **Name:** `TroostwijkMonitor`
   - **Trigger:** Every 30 minutes
   - **Action:** Start a program
   - **Program:** `java`
   - **Arguments:** `-jar "C:\path\to\troostwijk-scraper.jar" once`
   - **Start in:** `C:\path\to\project`

---

### Option C: Multiple Scheduled Tasks (Fine-grained Control)

Create separate tasks for each workflow:

| Task | Frequency | Command |
|------|-----------|---------|
| Import Data | Every 30 min | `run-once.bat` |
| Process Images | Every 1 hour | `run-once.bat` |
| Check Bids | Every 15 min | `run-once.bat` |
| Closing Alerts | Every 5 min | `run-once.bat` |

---

## Event-Driven Triggers

The orchestrator supports event-driven execution:

### 1. New Auction Discovered

```java
orchestrator.onNewAuctionDiscovered(auctionInfo);
```

**Triggered when:**
- External scraper finds new auction

**Actions:**
- Insert to database
- Send notification

---

### 2. Bid Change Detected

```java
orchestrator.onBidChange(lot, previousBid, newBid);
```

**Triggered when:**
- Bid increases on monitored lot

**Actions:**
- Update database
- Send notification: "Nieuw bod op kavel X: €Y (was €Z)"

---

### 3. Objects Detected

```java
orchestrator.onObjectsDetected(lotId, labels);
```

**Triggered when:**
- YOLO detects 2+ objects in image

**Actions:**
- Send notification: "Lot X contains: car, truck, machinery"

---

## Configuration

### Environment Variables

```bash
# Database location
set DATABASE_FILE=C:\mnt\okcomputer\output\cache.db

# Notification configuration
set NOTIFICATION_CONFIG=desktop

# Or for email notifications
set NOTIFICATION_CONFIG=smtp:your@gmail.com:app_password:recipient@example.com
```

### Configuration Files

**YOLO Model Paths** (`Main.java:35-37`):
```java
String yoloCfg = "models/yolov4.cfg";
String yoloWeights = "models/yolov4.weights";
String yoloClasses = "models/coco.names";
```

### Customizing Schedules

Edit `WorkflowOrchestrator.java` to change frequencies:

```java
// Change from 30 minutes to 15 minutes
scheduler.scheduleAtFixedRate(() -> {
    // ... scraper import logic
}, 0, 15, TimeUnit.MINUTES);  // Changed from 30
```

---

## Monitoring & Debugging

### Check Status

```bash
# Quick status check
java -jar troostwijk-monitor.jar status

# Or
check-status.bat
```

### View Logs

Workflows print timestamped logs:

```
📥 [WORKFLOW 1] Importing scraper data...
  → Imported 5 auctions
  → Imported 25 lots
  → Found 50 unprocessed images
  ✓ Scraper import completed in 1250ms

🖼️  [WORKFLOW 2] Processing pending images...
  → Processing 50 images
  ✓ Processed 50 images, detected objects in 12 (15.3s)
```

### Common Issues

#### 1. No data being imported

**Problem:** External scraper not running

**Solution:**
```bash
# Check if scraper is running and populating database
sqlite3 C:\mnt\okcomputer\output\cache.db "SELECT COUNT(*) FROM auctions;"
```

#### 2. Images not downloading

**Problem:** No internet connection or invalid URLs

**Solution:**
- Check network connectivity
- Verify image URLs in database
- Check firewall settings

#### 3. Notifications not showing

**Problem:** System tray not available

**Solution:**
- Use email notifications instead
- Check notification permissions in Windows

#### 4. Workflows not running

**Problem:** Application crashed or was stopped

**Solution:**
- Check Task Scheduler logs
- Review application logs
- Restart in workflow mode

---

## Integration Examples

### Example 1: Complete Automated Workflow

**Setup:**
1. External scraper runs continuously, populating database
2. This monitor runs in workflow mode
3. Notifications sent to desktop + email

**Result:**
- New auctions → Notification within 30 min
- New images → Processed within 1 hour
- Bid changes → Notification within 15 min
- Closing alerts → Notification within 5 min

---

### Example 2: On-Demand Processing

**Setup:**
1. External scraper runs once per day (cron/Task Scheduler)
2. This monitor runs in once mode after scraper completes

**Script:**
```bash
# run-daily.bat
@echo off
REM Run scraper first
python scraper.py

REM Wait for completion
timeout /t 30

REM Run monitor once
java -jar troostwijk-monitor.jar once
```

---

### Example 3: Event-Driven with External Integration

**Setup:**
1. External system calls orchestrator events
2. Workflows run on-demand

**Java code:**
```java
WorkflowOrchestrator orchestrator = new WorkflowOrchestrator(...);

// When external scraper finds new auction
AuctionInfo newAuction = parseScraperData();
orchestrator.onNewAuctionDiscovered(newAuction);

// When bid detected
orchestrator.onBidChange(lot, 100.0, 150.0);
```

---

## Advanced Topics

### Custom Workflows

Add custom workflows to `WorkflowOrchestrator`:

```java
// Workflow 5: Value Estimation (every 2 hours)
scheduler.scheduleAtFixedRate(() -> {
    try {
        Console.println("💰 [WORKFLOW 5] Estimating values...");

        var lotsWithImages = db.getLotsWithImages();
        for (var lot : lotsWithImages) {
            var images = db.getImagesForLot(lot.lotId());
            double estimatedValue = estimateValue(images);

            // Update database
            db.updateLotEstimatedValue(lot.lotId(), estimatedValue);

            // Notify if high value
            if (estimatedValue > 5000) {
                notifier.sendNotification(
                    String.format("High value lot detected: %d (€%.2f)",
                        lot.lotId(), estimatedValue),
                    "Value Alert", 1
                );
            }
        }
    } catch (Exception e) {
        Console.println("  ❌ Value estimation failed: " + e.getMessage());
    }
}, 10, 120, TimeUnit.MINUTES);
```

### Webhook Integration

Trigger workflows via HTTP webhooks:

```java
// In a separate web server (e.g., using Javalin)
Javalin app = Javalin.create().start(7070);

app.post("/webhook/new-auction", ctx -> {
    AuctionInfo auction = ctx.bodyAsClass(AuctionInfo.class);
    orchestrator.onNewAuctionDiscovered(auction);
    ctx.result("OK");
});

app.post("/webhook/bid-change", ctx -> {
    BidChange change = ctx.bodyAsClass(BidChange.class);
    orchestrator.onBidChange(change.lot, change.oldBid, change.newBid);
    ctx.result("OK");
});
```

---

## Summary

| Mode | Use Case | Scheduling | Best For |
|------|----------|------------|----------|
| **workflow** | Continuous operation | Built-in (Java) | Production, development |
| **once** | Single execution | External (Task Scheduler) | Cron jobs, on-demand |
| **legacy** | Backward compatibility | Built-in (Java) | Existing deployments |
| **status** | Quick check | Manual/External | Health checks, debugging |

**Recommended Setup for Windows:**
1. Install as Windows Service OR
2. Add to Startup folder (workflow mode) OR
3. Use Task Scheduler (once mode, every 30 min)

**All workflows automatically:**
- Import data from scraper
- Process images
- Detect objects
- Monitor bids
- Send notifications
- Handle errors gracefully

---

## Support

For issues or questions:
- Check `TEST_SUITE_SUMMARY.md` for test coverage
- Review code in `WorkflowOrchestrator.java`
- Run `java -jar troostwijk-monitor.jar status` for diagnostics