auctiora/wiki/WORKFLOW_GUIDE.md

Troostwijk Auction Monitor - Workflow Integration Guide

Complete guide for running the auction monitoring system with scheduled workflows, cron jobs, and event-driven triggers.

---

Table of Contents

1. Overview
2. Running Modes
3. Workflow Orchestration
4. Windows Scheduling
5. Event-Driven Triggers
6. Configuration
7. Monitoring & Debugging

---

Overview

The Troostwijk Auction Monitor supports multiple execution modes:

• Workflow Mode (Recommended): Continuous operation with built-in scheduling
• Once Mode: Single execution for external schedulers (Windows Task Scheduler, cron)
• Legacy Mode: Original monitoring approach
• Status Mode: Quick status check

---

Running Modes

1. Workflow Mode (Default - Recommended)

Runs all workflows continuously with built-in scheduling.

# Windows
java -jar target\troostwijk-scraper-1.0-SNAPSHOT-jar-with-dependencies.jar workflow

# Or simply (workflow is default)
java -jar target\troostwijk-scraper-1.0-SNAPSHOT-jar-with-dependencies.jar

# Using batch script
run-workflow.bat

What it does:

• ✅ Imports scraper data every 30 minutes
• ✅ Processes images every 1 hour
• ✅ Monitors bids every 15 minutes
• ✅ Checks closing times every 5 minutes

Best for:

• Production deployment
• Long-running services
• Development/testing

---

2. Once Mode (For External Schedulers)

Runs complete workflow once and exits.

# Windows
java -jar target\troostwijk-scraper-1.0-SNAPSHOT-jar-with-dependencies.jar once

# Using batch script
run-once.bat

What it does:

1. Imports scraper data
2. Processes pending images
3. Monitors bids
4. Checks closing times
5. Exits

Best for:

• Windows Task Scheduler
• Cron jobs (Linux/Mac)
• Manual execution
• Testing

---

3. Legacy Mode

Original monitoring approach (backward compatibility).

java -jar target\troostwijk-scraper-1.0-SNAPSHOT-jar-with-dependencies.jar legacy

Best for:

• Maintaining existing deployments
• Troubleshooting

---

4. Status Mode

Shows current status and exits.

java -jar target\troostwijk-scraper-1.0-SNAPSHOT-jar-with-dependencies.jar status

# Using batch script
check-status.bat

Output:

📊 Workflow Status:
  Running: No
  Auctions: 25
  Lots: 150
  Images: 300
  Closing soon (< 30 min): 5

---

Workflow Orchestration

The WorkflowOrchestrator coordinates 4 scheduled workflows:

Workflow 1: Scraper Data Import

Frequency: Every 30 minutes Purpose: Import new auctions and lots from external scraper

Process:

1. Import auctions from scraper database
2. Import lots from scraper database
3. Import image URLs
4. Send notification if significant data imported

Code Location: WorkflowOrchestrator.java:110

---

Workflow 2: Image Processing

Frequency: Every 1 hour Purpose: Download images and run object detection

Process:

1. Get unprocessed images from database
2. Download each image
3. Run YOLO object detection
4. Save labels to database
5. Send notification for interesting detections (3+ objects)

Code Location: WorkflowOrchestrator.java:150

---

Workflow 3: Bid Monitoring

Frequency: Every 15 minutes Purpose: Check for bid changes and send notifications

Process:

1. Get all active lots
2. Check for bid changes (via external scraper updates)
3. Send notifications for bid increases

Code Location: WorkflowOrchestrator.java:210

Note: The external scraper updates bids; this workflow monitors and notifies.

---

Workflow 4: Closing Alerts

Frequency: Every 5 minutes Purpose: Send alerts for lots closing soon

Process:

1. Get all active lots
2. Check closing times
3. Send high-priority notification for lots closing in < 5 min
4. Mark as notified to prevent duplicates

Code Location: WorkflowOrchestrator.java:240

---

Windows Scheduling

Option A: Use Built-in Workflow Mode (Recommended)

Run as a Windows Service or startup application:

1. Create shortcut to run-workflow.bat
2. Place in: C:\Users\[YourUser]\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup
3. Monitor will start automatically on login

---

Option B: Windows Task Scheduler (Once Mode)

Automated setup:

# Run PowerShell as Administrator
.\setup-windows-task.ps1

This creates two tasks:

• TroostwijkMonitor-Workflow: Runs every 30 minutes
• TroostwijkMonitor-StatusCheck: Runs every 6 hours

Manual setup:

1. Open Task Scheduler
2. Create Basic Task
3. Configure:
   • Name: TroostwijkMonitor
   • Trigger: Every 30 minutes
   • Action: Start a program
   • Program: java
   • Arguments: -jar "C:\path\to\troostwijk-scraper.jar" once
   • Start in: C:\path\to\project

---

Option C: Multiple Scheduled Tasks (Fine-grained Control)

Create separate tasks for each workflow:

Task	Frequency	Command
Import Data	Every 30 min	run-once.bat
Process Images	Every 1 hour	run-once.bat
Check Bids	Every 15 min	run-once.bat
Closing Alerts	Every 5 min	run-once.bat

---

Event-Driven Triggers

The orchestrator supports event-driven execution:

1. New Auction Discovered

orchestrator.onNewAuctionDiscovered(auctionInfo);

Triggered when:

• External scraper finds new auction

Actions:

• Insert to database
• Send notification

---

2. Bid Change Detected

orchestrator.onBidChange(lot, previousBid, newBid);

Triggered when:

• Bid increases on monitored lot

Actions:

• Update database
• Send notification: "Nieuw bod op kavel X: €Y (was €Z)"

---

3. Objects Detected

orchestrator.onObjectsDetected(lotId, labels);

Triggered when:

• YOLO detects 2+ objects in image

Actions:

• Send notification: "Lot X contains: car, truck, machinery"

---

Configuration

Environment Variables

# Database location
set DATABASE_FILE=C:\mnt\okcomputer\output\cache.db

# Notification configuration
set NOTIFICATION_CONFIG=desktop

# Or for email notifications
set NOTIFICATION_CONFIG=smtp:your@gmail.com:app_password:recipient@example.com

Configuration Files

YOLO Model Paths (Main.java:35-37):

String yoloCfg = "models/yolov4.cfg";
String yoloWeights = "models/yolov4.weights";
String yoloClasses = "models/coco.names";

Customizing Schedules

Edit WorkflowOrchestrator.java to change frequencies:

// Change from 30 minutes to 15 minutes
scheduler.scheduleAtFixedRate(() -> {
    // ... scraper import logic
}, 0, 15, TimeUnit.MINUTES);  // Changed from 30

---

Monitoring & Debugging

Check Status

# Quick status check
java -jar troostwijk-monitor.jar status

# Or
check-status.bat

View Logs

Workflows print timestamped logs:

📥 [WORKFLOW 1] Importing scraper data...
  → Imported 5 auctions
  → Imported 25 lots
  → Found 50 unprocessed images
  ✓ Scraper import completed in 1250ms

🖼️  [WORKFLOW 2] Processing pending images...
  → Processing 50 images
  ✓ Processed 50 images, detected objects in 12 (15.3s)

Common Issues

1. No data being imported

Problem: External scraper not running

Solution:

# Check if scraper is running and populating database
sqlite3 C:\mnt\okcomputer\output\cache.db "SELECT COUNT(*) FROM auctions;"

2. Images not downloading

Problem: No internet connection or invalid URLs

Solution:

• Check network connectivity
• Verify image URLs in database
• Check firewall settings

3. Notifications not showing

Problem: System tray not available

Solution:

• Use email notifications instead
• Check notification permissions in Windows

4. Workflows not running

Problem: Application crashed or was stopped

Solution:

• Check Task Scheduler logs
• Review application logs
• Restart in workflow mode

---

Integration Examples

Example 1: Complete Automated Workflow

Setup:

1. External scraper runs continuously, populating database
2. This monitor runs in workflow mode
3. Notifications sent to desktop + email

Result:

• New auctions → Notification within 30 min
• New images → Processed within 1 hour
• Bid changes → Notification within 15 min
• Closing alerts → Notification within 5 min

---

Example 2: On-Demand Processing

Setup:

1. External scraper runs once per day (cron/Task Scheduler)
2. This monitor runs in once mode after scraper completes

Script:

# run-daily.bat
@echo off
REM Run scraper first
python scraper.py

REM Wait for completion
timeout /t 30

REM Run monitor once
java -jar troostwijk-monitor.jar once

---

Example 3: Event-Driven with External Integration

Setup:

1. External system calls orchestrator events
2. Workflows run on-demand

Java code:

WorkflowOrchestrator orchestrator = new WorkflowOrchestrator(...);

// When external scraper finds new auction
AuctionInfo newAuction = parseScraperData();
orchestrator.onNewAuctionDiscovered(newAuction);

// When bid detected
orchestrator.onBidChange(lot, 100.0, 150.0);

---

Advanced Topics

Custom Workflows

Add custom workflows to WorkflowOrchestrator:

// Workflow 5: Value Estimation (every 2 hours)
scheduler.scheduleAtFixedRate(() -> {
    try {
        Console.println("💰 [WORKFLOW 5] Estimating values...");

        var lotsWithImages = db.getLotsWithImages();
        for (var lot : lotsWithImages) {
            var images = db.getImagesForLot(lot.lotId());
            double estimatedValue = estimateValue(images);

            // Update database
            db.updateLotEstimatedValue(lot.lotId(), estimatedValue);

            // Notify if high value
            if (estimatedValue > 5000) {
                notifier.sendNotification(
                    String.format("High value lot detected: %d (€%.2f)",
                        lot.lotId(), estimatedValue),
                    "Value Alert", 1
                );
            }
        }
    } catch (Exception e) {
        Console.println("  ❌ Value estimation failed: " + e.getMessage());
    }
}, 10, 120, TimeUnit.MINUTES);

Webhook Integration

Trigger workflows via HTTP webhooks:

// In a separate web server (e.g., using Javalin)
Javalin app = Javalin.create().start(7070);

app.post("/webhook/new-auction", ctx -> {
    AuctionInfo auction = ctx.bodyAsClass(AuctionInfo.class);
    orchestrator.onNewAuctionDiscovered(auction);
    ctx.result("OK");
});

app.post("/webhook/bid-change", ctx -> {
    BidChange change = ctx.bodyAsClass(BidChange.class);
    orchestrator.onBidChange(change.lot, change.oldBid, change.newBid);
    ctx.result("OK");
});

---

Summary

Mode	Use Case	Scheduling	Best For
workflow	Continuous operation	Built-in (Java)	Production, development
once	Single execution	External (Task Scheduler)	Cron jobs, on-demand
legacy	Backward compatibility	Built-in (Java)	Existing deployments
status	Quick check	Manual/External	Health checks, debugging

Recommended Setup for Windows:

1. Install as Windows Service OR
2. Add to Startup folder (workflow mode) OR
3. Use Task Scheduler (once mode, every 30 min)

All workflows automatically:

• Import data from scraper
• Process images
• Detect objects
• Monitor bids
• Send notifications
• Handle errors gracefully

---

Support

For issues or questions:

• Check TEST_SUITE_SUMMARY.md for test coverage
• Review code in WorkflowOrchestrator.java
• Run java -jar troostwijk-monitor.jar status for diagnostics