Tour/auctiora
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

front-end-fix

Browse Source
This commit is contained in:
Tour
2025-12-06 05:39:59 +01:00
parent d1a149e40d
commit 528a217708
4 changed files with 51 additions and 52 deletions
Download Patch File Download Diff File Expand all files Collapse all files

91
wiki/INTEGRATION_GUIDE.md
View File

@@ -49,58 +49,58 @@ This document describes how **Troostwijk Monitor** (this Java project) integrate
The scraper and monitor use **slightly different schemas** that need to be reconciled:
| Scraper Table | Monitor Table | Integration Notes |
|---------------|---------------|-------------------|
| `auctions` | `auctions` | ✅ **Compatible** - same structure |
| `lots` | `lots` | ⚠️ **Needs mapping** - field name differences |
| `images` | `images` | ⚠️ **Partial overlap** - different purposes |
| `cache` | N/A | ❌ Monitor doesn't use cache |
| Scraper Table | Monitor Table | Integration Notes |
|---------------|---------------|-----------------------------------------------|
| `auctions` | `auctions` | ✅ **Compatible** - same structure |
| `lots` | `lots` | ⚠️ **Needs mapping** - field name differences |
| `images` | `images` | ⚠️ **Partial overlap** - different purposes |
| `cache` | N/A | ❌ Monitor doesn't use cache |
### Field Mapping: `auctions` Table
| Scraper Field | Monitor Field | Notes |
|---------------|---------------|-------|
| `auction_id` (TEXT) | `auction_id` (INTEGER) | ⚠️ **TYPE MISMATCH** - Scraper uses "A7-39813", Monitor expects INT |
| `url` | `url` | ✅ Compatible |
| `title` | `title` | ✅ Compatible |
| `location` | `location`, `city`, `country` | ⚠️ Monitor splits into 3 fields |
| `lots_count` | `lot_count` | ⚠️ Name difference |
| `first_lot_closing_time` | `closing_time` | ⚠️ Name difference |
| `scraped_at` | `discovered_at` | ⚠️ Name + type difference (TEXT vs INTEGER timestamp) |
| Scraper Field | Monitor Field | Notes |
|--------------------------|-------------------------------|---------------------------------------------------------------------|
| `auction_id` (TEXT) | `auction_id` (INTEGER) | ⚠️ **TYPE MISMATCH** - Scraper uses "A7-39813", Monitor expects INT |
| `url` | `url` | ✅ Compatible |
| `title` | `title` | ✅ Compatible |
| `location` | `location`, `city`, `country` | ⚠️ Monitor splits into 3 fields |
| `lots_count` | `lot_count` | ⚠️ Name difference |
| `first_lot_closing_time` | `closing_time` | ⚠️ Name difference |
| `scraped_at` | `discovered_at` | ⚠️ Name + type difference (TEXT vs INTEGER timestamp) |
### Field Mapping: `lots` Table
| Scraper Field | Monitor Field | Notes |
|---------------|---------------|-------|
| `lot_id` (TEXT) | `lot_id` (INTEGER) | ⚠️ **TYPE MISMATCH** - "A1-28505-5" vs INT |
| `auction_id` | `sale_id` | ⚠️ Different name |
| `url` | `url` | ✅ Compatible |
| `title` | `title` | ✅ Compatible |
| `current_bid` (TEXT) | `current_bid` (REAL) | ⚠️ **TYPE MISMATCH** - "€123.45" vs 123.45 |
| `bid_count` | N/A | ℹ️ Monitor doesn't track |
| `closing_time` | `closing_time` | ⚠️ Format difference (TEXT vs LocalDateTime) |
| `viewing_time` | N/A | ℹ️ Monitor doesn't track |
| `pickup_date` | N/A | ℹ️ Monitor doesn't track |
| `location` | N/A | ℹ️ Monitor doesn't track lot location separately |
| `description` | `description` | ✅ Compatible |
| `category` | `category` | ✅ Compatible |
| N/A | `manufacturer` | ℹ️ Monitor has additional field |
| N/A | `type` | ℹ️ Monitor has additional field |
| N/A | `year` | ℹ️ Monitor has additional field |
| N/A | `currency` | ℹ️ Monitor has additional field |
| N/A | `closing_notified` | ℹ️ Monitor tracking field |
| Scraper Field | Monitor Field | Notes |
|----------------------|----------------------|--------------------------------------------------|
| `lot_id` (TEXT) | `lot_id` (INTEGER) | ⚠️ **TYPE MISMATCH** - "A1-28505-5" vs INT |
| `auction_id` | `sale_id` | ⚠️ Different name |
| `url` | `url` | ✅ Compatible |
| `title` | `title` | ✅ Compatible |
| `current_bid` (TEXT) | `current_bid` (REAL) | ⚠️ **TYPE MISMATCH** - "€123.45" vs 123.45 |
| `bid_count` | N/A | ℹ️ Monitor doesn't track |
| `closing_time` | `closing_time` | ⚠️ Format difference (TEXT vs LocalDateTime) |
| `viewing_time` | N/A | ℹ️ Monitor doesn't track |
| `pickup_date` | N/A | ℹ️ Monitor doesn't track |
| `location` | N/A | ℹ️ Monitor doesn't track lot location separately |
| `description` | `description` | ✅ Compatible |
| `category` | `category` | ✅ Compatible |
| N/A | `manufacturer` | ℹ️ Monitor has additional field |
| N/A | `type` | ℹ️ Monitor has additional field |
| N/A | `year` | ℹ️ Monitor has additional field |
| N/A | `currency` | ℹ️ Monitor has additional field |
| N/A | `closing_notified` | ℹ️ Monitor tracking field |
### Field Mapping: `images` Table
| Scraper Field | Monitor Field | Notes |
|---------------|---------------|-------|
| `id` | `id` | ✅ Compatible |
| `lot_id` | `lot_id` | ⚠️ Type difference (TEXT vs INTEGER) |
| `url` | `url` | ✅ Compatible |
| `local_path` | `file_path` | ⚠️ Different name |
| `downloaded` (INTEGER) | N/A | ℹ️ Monitor uses `processed_at` instead |
| N/A | `labels` (TEXT) | ℹ️ Monitor adds detected objects |
| N/A | `processed_at` (INTEGER) | ℹ️ Monitor tracking field |
| Scraper Field | Monitor Field | Notes |
|------------------------|--------------------------|----------------------------------------|
| `id` | `id` | ✅ Compatible |
| `lot_id` | `lot_id` | ⚠️ Type difference (TEXT vs INTEGER) |
| `url` | `url` | ✅ Compatible |
| `local_path` | `Local_path` | ⚠️ Different name |
| `downloaded` (INTEGER) | N/A | ℹ️ Monitor uses `processed_at` instead |
| N/A | `labels` (TEXT) | ℹ️ Monitor adds detected objects |
| N/A | `processed_at` (INTEGER) | ℹ️ Monitor tracking field |
## Integration Options
@@ -252,8 +252,7 @@ CREATE TABLE IF NOT EXISTS images (
id INTEGER PRIMARY KEY AUTOINCREMENT,
lot_id TEXT, -- FK: "A1-28505-5"
url TEXT, -- Image URL from website
file_path TEXT, -- Local path after download
local_path TEXT, -- Alias for compatibility
local_path TEXT, -- Local path after download
labels TEXT, -- Detected objects (comma-separated)
downloaded INTEGER DEFAULT 0, -- 0=pending, 1=downloaded
processed_at INTEGER, -- Unix timestamp when processed
@@ -469,7 +468,7 @@ sqlite3 /mnt/okcomputer/output/cache.db "SELECT COUNT(*) FROM lots"
**Check**:
1. Scraper writes image URLs to `images` table
2. Monitor reads from `images` table with `downloaded=0`
3. Field name mapping: `local_path` vs `file_path`
3. Field name mapping: `local_path` vs `local_path`
## Next Steps