119 lines
3.8 KiB
Markdown
119 lines
3.8 KiB
Markdown
# Refactoring Summary: Troostwijk Auction Monitor
|
|
|
|
## Overview
|
|
This project has been refactored to focus on **image processing and monitoring**, removing all auction/lot scraping functionality which is now handled by the external `ARCHITECTURE-TROOSTWIJK-SCRAPER` process.
|
|
|
|
## Architecture Changes
|
|
|
|
### Removed Components
|
|
- ❌ **TroostwijkScraper.java** - Removed (replaced by TroostwijkMonitor)
|
|
- ❌ Auction discovery and scraping logic
|
|
- ❌ Lot scraping via Playwright/JSoup
|
|
- ❌ CacheDatabase (can be removed if not used elsewhere)
|
|
|
|
### New/Updated Components
|
|
|
|
#### New Classes
|
|
- ✅ **TroostwijkMonitor.java** - Monitors bids and coordinates services (no scraping)
|
|
- ✅ **ImageProcessingService.java** - Downloads images and runs object detection
|
|
- ✅ **Console.java** - Simple output utility (renamed from IO to avoid Java 25 conflict)
|
|
|
|
#### Modernized Classes
|
|
- ✅ **AuctionInfo** - Converted to immutable `record`
|
|
- ✅ **Lot** - Converted to immutable `record` with `minutesUntilClose()` method
|
|
- ✅ **DatabaseService.java** - Uses modern Java features:
|
|
- Text blocks (`"""`) for SQL
|
|
- Record accessor methods
|
|
- Added `getImagesForLot()` method
|
|
- Added `processed_at` timestamp to images table
|
|
- Nested `ImageRecord` record
|
|
|
|
#### Preserved Components
|
|
- ✅ **NotificationService.java** - Desktop/email notifications
|
|
- ✅ **ObjectDetectionService.java** - YOLO-based object detection
|
|
- ✅ **Main.java** - Updated to use new architecture
|
|
|
|
## Database Schema
|
|
|
|
### Populated by External Scraper
|
|
- `auctions` table - Auction metadata
|
|
- `lots` table - Lot details with bidding info
|
|
|
|
### Populated by This Process
|
|
- `images` table - Downloaded images with:
|
|
- `file_path` - Local storage path
|
|
- `labels` - Detected objects (comma-separated)
|
|
- `processed_at` - Processing timestamp
|
|
|
|
## Modern Java Features Used
|
|
|
|
- **Records** - Immutable data carriers (AuctionInfo, Lot, ImageRecord)
|
|
- **Text Blocks** - Multi-line SQL queries
|
|
- **var** - Type inference throughout
|
|
- **Switch expressions** - Where applicable
|
|
- **Pattern matching** - Ready for future enhancements
|
|
|
|
## Responsibilities
|
|
|
|
### This Project
|
|
1. ✅ Image downloading from URLs in database
|
|
2. ✅ Object detection using YOLO/OpenCV
|
|
3. ✅ Bid monitoring and change detection
|
|
4. ✅ Desktop and email notifications
|
|
5. ✅ Data enrichment with image analysis
|
|
|
|
### External ARCHITECTURE-TROOSTWIJK-SCRAPER
|
|
1. 🔄 Discover auctions from Troostwijk website
|
|
2. 🔄 Scrape lot details via API
|
|
3. 🔄 Populate `auctions` and `lots` tables
|
|
4. 🔄 Share database with this process
|
|
|
|
## Usage
|
|
|
|
### Running the Monitor
|
|
```bash
|
|
# With environment variables
|
|
export DATABASE_FILE=troostwijk.db
|
|
export NOTIFICATION_CONFIG=desktop # or smtp:user:pass:email
|
|
|
|
java -jar troostwijk-monitor.jar
|
|
```
|
|
|
|
### Expected Output
|
|
```
|
|
=== Troostwijk Auction Monitor ===
|
|
|
|
✓ OpenCV loaded
|
|
Initializing monitor...
|
|
|
|
📊 Current Database State:
|
|
Total lots in database: 42
|
|
Total images processed: 0
|
|
|
|
[1/2] Processing images...
|
|
Processing pending images...
|
|
|
|
[2/2] Starting bid monitoring...
|
|
✓ Monitoring service started
|
|
|
|
✓ Monitor is running. Press Ctrl+C to stop.
|
|
|
|
NOTE: This process expects auction/lot data from the external scraper.
|
|
Make sure ARCHITECTURE-TROOSTWIJK-SCRAPER is running and populating the database.
|
|
```
|
|
|
|
## Migration Notes
|
|
|
|
1. The project now compiles successfully with Java 25
|
|
2. All scraping logic removed - rely on external scraper
|
|
3. Shared database architecture for inter-process communication
|
|
4. Clean separation of concerns
|
|
5. Modern, maintainable codebase with records and text blocks
|
|
|
|
## Next Steps
|
|
|
|
- Remove `CacheDatabase.java` if not needed
|
|
- Consider adding API endpoint for external scraper to trigger image processing
|
|
- Add metrics/logging framework
|
|
- Consider message queue (e.g., Redis, RabbitMQ) for better inter-process communication
|