tour/auctiora
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Fix mock tests

Browse Source
This commit is contained in:
Tour
2025-12-07 02:36:00 +01:00
parent 432fcbc503
commit f561a73b01
7 changed files with 2034 additions and 751 deletions
Download Patch File Download Diff File Expand all files Collapse all files

115
README.md
View File

@@ -163,7 +163,7 @@ mvn exec:java -Dexec.mainClass="com.auction.scraper.TroostwijkScraper"
▼ ▼
┌──────────────────┐ ┌──────────────────┐
│ SQLITE DATABASE │ │ SQLITE DATABASE │
│ troostwijk.db │ │ output/cache.db │
└──────────────────┘ └──────────────────┘
│ │
┌─────────────────┼─────────────────┐ ┌─────────────────┼─────────────────┐
@@ -329,19 +329,112 @@ mvn exec:java -Dexec.mainClass="com.auction.scraper.TroostwijkScraper"
value > threshold] value > threshold]
``` ```
```mermaid
flowchart TD
subgraph P1["PHASE 1: EXTERNAL SCRAPER (Python/Playwright)"]
direction LR
A1[Listing Pages<br/>/auctions?page=N] --> A2[Extract URLs]
B1[Auction Pages<br/>/a/auction-id] --> B2[Parse __NEXT_DATA__ JSON]
C1[Lot Pages<br/>/l/lot-id] --> C2[Parse __NEXT_DATA__ JSON]
A2 --> D1[INSERT auctions to SQLite]
B2 --> D1
C2 --> D2[INSERT lots & image URLs]
D1 --> DB[(SQLite Database<br/>output/cache.db)]
D2 --> DB
end
DB --> P2_Entry
subgraph P2["PHASE 2: MONITORING & PROCESSING (Java)"]
direction TB
P2_Entry[Data Ready] --> Monitor[TroostwijkMonitor<br/>Read lots every hour]
P2_Entry --> DBService[DatabaseService<br/>Query & Import]
P2_Entry --> Adapter[ScraperDataAdapter<br/>Transform TEXT → INTEGER]
Monitor --> BM[Bid Monitoring<br/>Check API every 1h]
DBService --> IP[Image Processing<br/>Download & Analyze]
Adapter --> DataForNotify[Formatted Data]
BM --> BidUpdate{New bid?}
BidUpdate -->|Yes| UpdateDB[Update current_bid in DB]
UpdateDB --> NotifyTrigger1
IP --> Detection[Object Detection<br/>YOLO/OpenCV DNN]
Detection --> ObjectCheck{Detect objects?}
ObjectCheck -->|Vehicle| Save1[Save labels & estimate value]
ObjectCheck -->|Furniture| Save2[Save labels & estimate value]
ObjectCheck -->|Machinery| Save3[Save labels & estimate value]
Save1 --> NotifyTrigger2
Save2 --> NotifyTrigger2
Save3 --> NotifyTrigger2
CA[Closing Alerts<br/>Check < 5 min] --> TimeCheck{Time critical?}
TimeCheck -->|Yes| NotifyTrigger3
end
NotifyTrigger1 --> NS
NotifyTrigger2 --> NS
NotifyTrigger3 --> NS
subgraph P3["PHASE 3: NOTIFICATION SYSTEM"]
NS[NotificationService] --> DN[Desktop Notify<br/>Windows/macOS/Linux]
NS --> EN[Email Notify<br/>Gmail SMTP]
NS --> PL[Set Priority Level<br/>0=Normal, 1=High]
end
DN --> UI[User Interaction & Decisions]
EN --> UI
PL --> UI
subgraph UI_Details[User Decision Points / Trigger Events]
direction LR
E1["1. BID CHANGE<br/>'Nieuw bod op kavel 12345...'<br/>Actions: Place bid? Monitor? Ignore?"]
E2["2. OBJECT DETECTED<br/>'Lot contains: Vehicle...'<br/>Actions: Review? Confirm value?"]
E3["3. CLOSING ALERT<br/>'Kavel 12345 sluit binnen 5 min.'<br/>Actions: Place final bid? Let expire?"]
E4["4. VIEWING DAY QUESTIONS<br/>'Bezichtiging op [date]...'"]
E5["5. ITEM RECOGNITION CONFIRMATION<br/>'Detected: [object]...'"]
E6["6. VALUE ESTIMATE APPROVAL<br/>'Geschatte waarde: €X...'"]
E7["7. EXCEPTION HANDLING<br/>'Afwijkende sluitingstijd...'"]
end
UI --> UI_Details
%% Object Detection Sub-Flow Detail
subgraph P2_Detail["Object Detection & Value Estimation Pipeline"]
direction LR
DI[Downloaded Image] --> IPS[ImageProcessingService]
IPS --> ODS[ObjectDetectionService]
ODS --> Load[Load YOLO model]
ODS --> Run[Run inference]
ODS --> Post[Post-process detections<br/>confidence > 0.5]
Post --> ObjList["Detected Objects List<br/>(80 COCO classes)"]
ObjList --> VEL[Value Estimation Logic<br/>Future enhancement]
VEL --> Match[Match to categories]
VEL --> History[Historical price analysis]
VEL --> Condition[Condition assessment]
VEL --> Market[Market trends]
Market --> ValueEst["Estimated Value Range<br/>Confidence: 75%"]
ValueEst --> SaveToDB[Save to Database]
SaveToDB --> TriggerNotify{Value > threshold?}
end
IP -.-> P2_Detail
TriggerNotify -.-> NotifyTrigger2
```
## Integration Hooks & Timing ## Integration Hooks & Timing
| Event | Frequency | Trigger | Notification Type | User Action Required | | Event | Frequency | Trigger | Notification Type | User Action Required |
|-------|-----------|---------|-------------------|---------------------| |--------------------------------|-------------------|----------------------------|----------------------------|------------------------|
| **New auction discovered** | On scrape | Scraper finds new auction | Desktop + Email (optional) | Review auction | | **New auction discovered** | On scrape | Scraper finds new auction | Desktop + Email (optional) | Review auction |
| **Bid change detected** | Every 1 hour | Monitor detects higher bid | Desktop + Email | Place counter-bid? | | **Bid change detected** | Every 1 hour | Monitor detects higher bid | Desktop + Email | Place counter-bid? |
| **Closing soon (< 30 min)** | When detected | Time-based check | Desktop + Email | Review lot | | **Closing soon (< 30 min)** | When detected | Time-based check | Desktop + Email | Review lot |
| **Closing imminent (< 5 min)** | When detected | Time-based check | Desktop + Email (HIGH) | Final bid decision | | **Closing imminent (< 5 min)** | When detected | Time-based check | Desktop + Email (HIGH) | Final bid decision |
| **Object detected** | On image process | YOLO finds objects | Desktop + Email | Confirm identification | | **Object detected** | On image process | YOLO finds objects | Desktop + Email | Confirm identification |
| **Value estimated** | After detection | Estimation complete | Desktop + Email | Approve estimate | | **Value estimated** | After detection | Estimation complete | Desktop + Email | Approve estimate |
| **Viewing day scheduled** | From lot metadata | Scraper extracts date | Desktop + Email | Confirm attendance | | **Viewing day scheduled** | From lot metadata | Scraper extracts date | Desktop + Email | Confirm attendance |
| **Exception/Change** | On update | Scraper detects change | Desktop + Email (HIGH) | Acknowledge | | **Exception/Change** | On update | Scraper detects change | Desktop + Email (HIGH) | Acknowledge |
## Project Structure ## Project Structure

174
src/main/java/auctiora/AuctionMonitorResource.java
View File

@@ -6,9 +6,15 @@ import jakarta.ws.rs.core.MediaType;
import jakarta.ws.rs.core.Response; import jakarta.ws.rs.core.Response;
import org.jboss.logging.Logger; import org.jboss.logging.Logger;
import java.time.LocalDateTime;
import java.util.ArrayList;
import java.util.HashMap; import java.util.HashMap;
import java.util.List;
import java.util.Map; import java.util.Map;
import java.util.stream.Collectors;
import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;
import java.util.stream.Collectors;
/** /**
* REST API for Auction Monitor control and status. * REST API for Auction Monitor control and status.
* Provides endpoints for: * Provides endpoints for:
@@ -359,4 +365,170 @@ public class AuctionMonitorResource {
.build(); .build();
} }
} }
/**
* GET /api/monitor/charts/country-distribution
* Returns dynamic country distribution for charts
*/
@GET
@Path("/charts/country-distribution")
public Response getCountryDistribution() {
try {
var auctions = db.getAllAuctions();
Map<String, Long> distribution = auctions.stream()
.filter(a -> a.country() != null && !a.country().isEmpty())
.collect(Collectors.groupingBy(
AuctionInfo::country,
Collectors.counting()
));
return Response.ok(distribution).build();
} catch (Exception e) {
LOG.error("Failed to get country distribution", e);
return Response.status(Response.Status.INTERNAL_SERVER_ERROR)
.entity(Map.of("error", e.getMessage()))
.build();
}
}
/**
* GET /api/monitor/charts/category-distribution
* Returns dynamic category distribution for charts
*/
@GET
@Path("/charts/category-distribution")
public Response getCategoryDistribution() {
try {
var lots = db.getAllLots();
Map<String, Long> distribution = lots.stream()
.filter(l -> l.category() != null && !l.category().isEmpty())
.collect(Collectors.groupingBy(
l -> l.category().length() > 20 ? l.category().substring(0, 20) + "..." : l.category(),
Collectors.counting()
));
return Response.ok(distribution).build();
} catch (Exception e) {
LOG.error("Failed to get category distribution", e);
return Response.status(Response.Status.INTERNAL_SERVER_ERROR)
.entity(Map.of("error", e.getMessage()))
.build();
}
}
/**
* GET /api/monitor/charts/bidding-trend
* Returns time series data for last N hours
*/
@GET
@Path("/charts/bidding-trend")
public Response getBiddingTrend(@QueryParam("hours") @DefaultValue("24") int hours) {
try {
var lots = db.getAllLots();
Map<Integer, TrendHour> trends = new HashMap<>();
// Initialize hours
LocalDateTime now = LocalDateTime.now();
for (int i = hours - 1; i >= 0; i--) {
LocalDateTime hour = now.minusHours(i);
int hourKey = hour.getHour();
trends.put(hourKey, new TrendHour(hourKey, 0, 0));
}
// Count lots and bids per hour (mock implementation - in real app, use timestamp data)
// This is a simplified version - you'd need actual timestamps in DB
for (var lot : lots) {
if (lot.closingTime() != null) {
int hour = lot.closingTime().getHour();
TrendHour trend = trends.getOrDefault(hour, new TrendHour(hour, 0, 0));
trend.lots++;
if (lot.currentBid() > 0) trend.bids++;
}
}
return Response.ok(trends.values()).build();
} catch (Exception e) {
LOG.error("Failed to get bidding trend", e);
return Response.status(Response.Status.INTERNAL_SERVER_ERROR)
.entity(Map.of("error", e.getMessage()))
.build();
}
}
/**
* GET /api/monitor/charts/insights
* Returns intelligent insights
*/
@GET
@Path("/charts/insights")
public Response getInsights() {
try {
var lots = db.getAllLots();
var auctions = db.getAllAuctions();
List<Map<String, String>> insights = new ArrayList<>();
// Calculate insights
long criticalCount = lots.stream().filter(l -> l.minutesUntilClose() < 30).count();
if (criticalCount > 10) {
insights.add(Map.of(
"icon", "fa-exclamation-circle",
"title", criticalCount + " lots closing soon",
"description", "High urgency items require attention"
));
}
double bidRate = lots.stream().filter(l -> l.currentBid() > 0).count() * 100.0 / lots.size();
if (bidRate > 60) {
insights.add(Map.of(
"icon", "fa-chart-line",
"title", String.format("%.1f%% bid rate", bidRate),
"description", "Strong market engagement detected"
));
}
long imageCoverage = db.getImageCount() * 100 / Math.max(lots.size(), 1);
if (imageCoverage < 80) {
insights.add(Map.of(
"icon", "fa-images",
"title", imageCoverage + "% image coverage",
"description", "Consider processing more images"
));
}
// Add geographic insight
String topCountry = auctions.stream()
.collect(Collectors.groupingBy(AuctionInfo::country, Collectors.counting()))
.entrySet().stream()
.max(Map.Entry.comparingByValue())
.map(Map.Entry::getKey)
.orElse("N/A");
insights.add(Map.of(
"icon", "fa-globe",
"title", topCountry + " leading",
"description", "Top performing country"
));
return Response.ok(insights).build();
} catch (Exception e) {
LOG.error("Failed to get insights", e);
return Response.status(Response.Status.INTERNAL_SERVER_ERROR)
.entity(Map.of("error", e.getMessage()))
.build();
}
}
// Helper class for trend data
public static class TrendHour {
public int hour;
public int lots;
public int bids;
public TrendHour(int hour, int lots, int bids) {
this.hour = hour;
this.lots = lots;
this.bids = bids;
}
}
} }

1509
src/main/resources/META-INF/resources/index.html
View File

File diff suppressed because it is too large Load Diff

572
src/main/resources/META-INF/resources/praetium.html
View File

@@ -1,572 +0,0 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Auctiora - Monitoring Dashboard</title>
<script src="https://cdn.tailwindcss.com"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/plotly.js/3.0.3/plotly.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/chart.js@4.4.0/dist/chart.umd.min.js"></script>
<link href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.0.0/css/all.min.css" rel="stylesheet">
<style>
.gradient-bg {
background: linear-gradient(135deg, #1e3a8a 0%, #3b82f6 50%, #60a5fa 100%);
}
.card-hover {
transition: all 0.3s ease;
}
.card-hover:hover {
transform: translateY(-5px);
box-shadow: 0 20px 25px -5px rgba(0, 0, 0, 0.1), 0 10px 10px -5px rgba(0, 0, 0, 0.04);
}
.status-online {
animation: pulse-green 2s infinite;
}
@keyframes pulse-green {
0%, 100% { opacity: 1; }
50% { opacity: 0.5; }
}
.workflow-card {
background: linear-gradient(145deg, #ffffff 0%, #f8fafc 100%);
border-left: 4px solid #3b82f6;
}
.alert-badge {
animation: pulse-red 1.5s infinite;
}
@keyframes pulse-red {
0%, 100% { transform: scale(1); }
50% { transform: scale(1.1); }
}
.metric-card {
background: linear-gradient(145deg, #ffffff 0%, #fafbfc 100%);
}
</style>
</head>
<body class="bg-gray-50 min-h-screen">
<!-- Header -->
<header class="gradient-bg text-white py-6 shadow-lg">
<div class="container mx-auto px-4">
<div class="flex items-center justify-between">
<div>
<h1 class="text-4xl font-bold mb-2">
<i class="fas fa-cube mr-3"></i>
Auctiora Monitoring Dashboard
</h1>
<p class="text-lg opacity-90">Real-time Auction Data Processing & Monitoring</p>
</div>
<div class="text-right">
<div class="flex items-center space-x-2">
<span class="status-online inline-block w-3 h-3 bg-green-400 rounded-full"></span>
<span class="text-sm font-semibold">System Online</span>
</div>
<div class="text-xs opacity-75 mt-1" id="last-update">Last updated: --:--</div>
</div>
</div>
</div>
</header>
<!-- Main Content -->
<main class="container mx-auto px-4 py-8">
<!-- System Status Cards -->
<div class="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-5 gap-4 mb-8">
<div class="bg-white rounded-lg shadow-md p-6 card-hover metric-card">
<div class="flex items-center justify-between">
<div>
<div class="text-gray-600 text-sm font-semibold uppercase">Auctions</div>
<div class="text-3xl font-bold text-blue-600 mt-2" id="total-auctions">--</div>
</div>
<div class="p-4 rounded-full bg-blue-100">
<i class="fas fa-gavel text-2xl text-blue-600"></i>
</div>
</div>
</div>
<div class="bg-white rounded-lg shadow-md p-6 card-hover metric-card">
<div class="flex items-center justify-between">
<div>
<div class="text-gray-600 text-sm font-semibold uppercase">Total Lots</div>
<div class="text-3xl font-bold text-green-600 mt-2" id="total-lots">--</div>
</div>
<div class="p-4 rounded-full bg-green-100">
<i class="fas fa-boxes text-2xl text-green-600"></i>
</div>
</div>
</div>
<div class="bg-white rounded-lg shadow-md p-6 card-hover metric-card">
<div class="flex items-center justify-between">
<div>
<div class="text-gray-600 text-sm font-semibold uppercase">Images</div>
<div class="text-3xl font-bold text-purple-600 mt-2" id="total-images">--</div>
</div>
<div class="p-4 rounded-full bg-purple-100">
<i class="fas fa-images text-2xl text-purple-600"></i>
</div>
</div>
</div>
<div class="bg-white rounded-lg shadow-md p-6 card-hover metric-card">
<div class="flex items-center justify-between">
<div>
<div class="text-gray-600 text-sm font-semibold uppercase">Active Lots</div>
<div class="text-3xl font-bold text-yellow-600 mt-2" id="active-lots">--</div>
</div>
<div class="p-4 rounded-full bg-yellow-100">
<i class="fas fa-clock text-2xl text-yellow-600"></i>
</div>
</div>
</div>
<div class="bg-white rounded-lg shadow-md p-6 card-hover metric-card">
<div class="flex items-center justify-between">
<div>
<div class="text-gray-600 text-sm font-semibold uppercase">Closing Soon</div>
<div class="text-3xl font-bold text-red-600 mt-2" id="closing-soon">--</div>
</div>
<div class="p-4 rounded-full bg-red-100">
<i class="fas fa-bell text-2xl text-red-600"></i>
</div>
</div>
</div>
</div>
<!-- Statistics Overview -->
<div class="grid grid-cols-1 lg:grid-cols-3 gap-6 mb-8">
<div class="bg-white rounded-lg shadow-md p-6 col-span-2">
<h3 class="text-xl font-semibold mb-4 text-gray-800">
<i class="fas fa-chart-line mr-2 text-blue-600"></i>
Bidding Statistics
</h3>
<div class="grid grid-cols-3 gap-4">
<div class="text-center p-4 bg-blue-50 rounded-lg">
<div class="text-gray-600 text-sm font-semibold">Lots with Bids</div>
<div class="text-2xl font-bold text-blue-600 mt-2" id="lots-with-bids">--</div>
</div>
<div class="text-center p-4 bg-green-50 rounded-lg">
<div class="text-gray-600 text-sm font-semibold">Total Bid Value</div>
<div class="text-2xl font-bold text-green-600 mt-2" id="total-bid-value">--</div>
</div>
<div class="text-center p-4 bg-purple-50 rounded-lg">
<div class="text-gray-600 text-sm font-semibold">Average Bid</div>
<div class="text-2xl font-bold text-purple-600 mt-2" id="average-bid">--</div>
</div>
</div>
</div>
<div class="bg-white rounded-lg shadow-md p-6">
<h3 class="text-xl font-semibold mb-4 text-gray-800">
<i class="fas fa-tachometer-alt mr-2 text-green-600"></i>
Rate Limiting
</h3>
<div class="space-y-3">
<div class="flex justify-between items-center">
<span class="text-gray-600 text-sm">Total Requests</span>
<span class="font-bold text-gray-800" id="rate-total-requests">--</span>
</div>
<div class="flex justify-between items-center">
<span class="text-gray-600 text-sm">Successful</span>
<span class="font-bold text-green-600" id="rate-successful">--</span>
</div>
<div class="flex justify-between items-center">
<span class="text-gray-600 text-sm">Failed</span>
<span class="font-bold text-red-600" id="rate-failed">--</span>
</div>
<div class="flex justify-between items-center">
<span class="text-gray-600 text-sm">Avg Duration</span>
<span class="font-bold text-blue-600" id="rate-avg-duration">--</span>
</div>
</div>
</div>
</div>
<!-- Workflow Controls -->
<div class="bg-white rounded-lg shadow-md p-6 mb-8">
<h3 class="text-xl font-semibold mb-4 text-gray-800">
<i class="fas fa-play-circle mr-2 text-blue-600"></i>
Workflow Controls
</h3>
<div class="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-4 gap-4">
<button onclick="triggerWorkflow('scraper-import')"
class="workflow-card p-4 rounded-lg hover:shadow-lg transition-all">
<div class="flex items-center justify-between mb-2">
<i class="fas fa-download text-blue-600 text-xl"></i>
<span class="text-xs text-gray-500" id="status-scraper">Ready</span>
</div>
<div class="text-sm font-semibold text-gray-800">Import Scraper Data</div>
<div class="text-xs text-gray-600 mt-1">Load auction/lot data from external scraper</div>
</button>
<button onclick="triggerWorkflow('image-processing')"
class="workflow-card p-4 rounded-lg hover:shadow-lg transition-all">
<div class="flex items-center justify-between mb-2">
<i class="fas fa-image text-purple-600 text-xl"></i>
<span class="text-xs text-gray-500" id="status-images">Ready</span>
</div>
<div class="text-sm font-semibold text-gray-800">Process Images</div>
<div class="text-xs text-gray-600 mt-1">Download & analyze images with object detection</div>
</button>
<button onclick="triggerWorkflow('bid-monitoring')"
class="workflow-card p-4 rounded-lg hover:shadow-lg transition-all">
<div class="flex items-center justify-between mb-2">
<i class="fas fa-chart-line text-green-600 text-xl"></i>
<span class="text-xs text-gray-500" id="status-bids">Ready</span>
</div>
<div class="text-sm font-semibold text-gray-800">Monitor Bids</div>
<div class="text-xs text-gray-600 mt-1">Track bid changes and send notifications</div>
</button>
<button onclick="triggerWorkflow('closing-alerts')"
class="workflow-card p-4 rounded-lg hover:shadow-lg transition-all">
<div class="flex items-center justify-between mb-2">
<i class="fas fa-bell text-red-600 text-xl"></i>
<span class="text-xs text-gray-500" id="status-alerts">Ready</span>
</div>
<div class="text-sm font-semibold text-gray-800">Closing Alerts</div>
<div class="text-xs text-gray-600 mt-1">Alert for lots closing within 30 minutes</div>
</button>
</div>
</div>
<!-- Charts Section -->
<div class="grid grid-cols-1 lg:grid-cols-2 gap-8 mb-8">
<!-- Country Distribution -->
<div class="bg-white rounded-lg shadow-md p-6">
<h3 class="text-xl font-semibold mb-4 text-gray-800">
<i class="fas fa-globe mr-2 text-blue-600"></i>
Auctions by Country
</h3>
<div id="country-chart" style="height: 300px;"></div>
</div>
<!-- Category Distribution -->
<div class="bg-white rounded-lg shadow-md p-6">
<h3 class="text-xl font-semibold mb-4 text-gray-800">
<i class="fas fa-tags mr-2 text-green-600"></i>
Lots by Category
</h3>
<div id="category-chart" style="height: 300px;"></div>
</div>
</div>
<!-- Closing Soon Table -->
<div class="bg-white rounded-lg shadow-md p-6 mb-8">
<div class="flex justify-between items-center mb-6">
<h3 class="text-xl font-semibold text-gray-800">
<i class="fas fa-exclamation-triangle mr-2 text-red-600"></i>
Lots Closing Soon (< 30 min)
</h3>
<button onclick="fetchClosingSoon()" class="bg-red-600 text-white px-4 py-2 rounded-lg hover:bg-red-700 transition-colors">
<i class="fas fa-sync mr-2"></i>Refresh
</button>
</div>
<div class="overflow-x-auto">
<table class="min-w-full divide-y divide-gray-200">
<thead class="bg-gray-50">
<tr>
<th class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase">Lot ID</th>
<th class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase">Title</th>
<th class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase">Current Bid</th>
<th class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase">Closing Time</th>
<th class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase">Minutes Left</th>
<th class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase">Actions</th>
</tr>
</thead>
<tbody id="closing-soon-table" class="bg-white divide-y divide-gray-200">
<tr>
<td colspan="6" class="px-6 py-4 text-center text-gray-500">Loading...</td>
</tr>
</tbody>
</table>
</div>
</div>
<!-- Recent Activity Log -->
<div class="bg-white rounded-lg shadow-md p-6">
<h3 class="text-xl font-semibold mb-4 text-gray-800">
<i class="fas fa-history mr-2 text-purple-600"></i>
Activity Log
</h3>
<div id="activity-log" class="space-y-2 max-h-64 overflow-y-auto">
<div class="text-sm text-gray-500">Monitoring system started...</div>
</div>
</div>
</main>
<!-- Footer -->
<footer class="bg-gray-800 text-white py-6 mt-12">
<div class="container mx-auto px-4 text-center">
<p class="text-gray-300">
<i class="fas fa-info-circle mr-2"></i>
Auctiora - Auction Data Processing & Monitoring System
</p>
<p class="text-sm text-gray-400 mt-2">
Architecture: Quarkus + SQLite + REST API | Auto-refresh: 15 seconds
</p>
</div>
</footer>
<script>
// Dashboard state
let dashboardData = {
status: {},
statistics: {},
rateLimitStats: {},
closingSoon: []
};
// Initialize dashboard
document.addEventListener('DOMContentLoaded', function() {
addLog('Dashboard initialized');
fetchAllData();
// Auto-refresh every 15 seconds
setInterval(fetchAllData, 15000);
});
// Fetch all dashboard data
async function fetchAllData() {
try {
await Promise.all([
fetchStatus(),
fetchStatistics(),
fetchRateLimitStats(),
fetchClosingSoon()
]);
updateLastUpdate();
addLog('Dashboard data refreshed');
} catch (error) {
console.error('Error fetching dashboard data:', error);
addLog('Error: ' + error.message, 'error');
}
}
// Fetch system status
async function fetchStatus() {
try {
const response = await fetch('/api/monitor/status');
if (!response.ok) throw new Error('Failed to fetch status');
dashboardData.status = await response.json();
document.getElementById('total-auctions').textContent = dashboardData.status.auctions || 0;
document.getElementById('total-lots').textContent = dashboardData.status.lots || 0;
document.getElementById('total-images').textContent = dashboardData.status.images || 0;
document.getElementById('closing-soon').textContent = dashboardData.status.closingSoon || 0;
} catch (error) {
console.error('Error fetching status:', error);
}
}
// Fetch statistics
async function fetchStatistics() {
try {
const response = await fetch('/api/monitor/statistics');
if (!response.ok) throw new Error('Failed to fetch statistics');
dashboardData.statistics = await response.json();
document.getElementById('active-lots').textContent = dashboardData.statistics.activeLots || 0;
document.getElementById('lots-with-bids').textContent = dashboardData.statistics.lotsWithBids || 0;
document.getElementById('total-bid-value').textContent = dashboardData.statistics.totalBidValue || '€0.00';
document.getElementById('average-bid').textContent = dashboardData.statistics.averageBid || '€0.00';
} catch (error) {
console.error('Error fetching statistics:', error);
}
}
// Fetch rate limit stats
async function fetchRateLimitStats() {
try {
const response = await fetch('/api/monitor/rate-limit/stats');
if (!response.ok) throw new Error('Failed to fetch rate limit stats');
dashboardData.rateLimitStats = await response.json();
// Aggregate stats from all hosts
let totalRequests = 0, successfulRequests = 0, failedRequests = 0, avgDuration = 0;
const stats = dashboardData.rateLimitStats.statistics || {};
const hostCount = Object.keys(stats).length;
for (const hostStats of Object.values(stats)) {
totalRequests += hostStats.totalRequests || 0;
successfulRequests += hostStats.successfulRequests || 0;
failedRequests += hostStats.failedRequests || 0;
avgDuration += hostStats.averageDurationMs || 0;
}
document.getElementById('rate-total-requests').textContent = totalRequests;
document.getElementById('rate-successful').textContent = successfulRequests;
document.getElementById('rate-failed').textContent = failedRequests;
document.getElementById('rate-avg-duration').textContent =
hostCount > 0 ? (avgDuration / hostCount).toFixed(0) + ' ms' : '--';
} catch (error) {
console.error('Error fetching rate limit stats:', error);
}
}
// Fetch closing soon lots
async function fetchClosingSoon() {
try {
const response = await fetch('/api/monitor/lots/closing-soon?minutes=30');
if (!response.ok) throw new Error('Failed to fetch closing soon lots');
dashboardData.closingSoon = await response.json();
updateClosingSoonTable();
// Update charts (placeholder for now)
updateCharts();
} catch (error) {
console.error('Error fetching closing soon:', error);
document.getElementById('closing-soon-table').innerHTML =
'<tr><td colspan="6" class="px-6 py-4 text-center text-red-600">Error loading data</td></tr>';
}
}
// Update closing soon table
function updateClosingSoonTable() {
const tableBody = document.getElementById('closing-soon-table');
if (dashboardData.closingSoon.length === 0) {
tableBody.innerHTML = '<tr><td colspan="6" class="px-6 py-4 text-center text-gray-500">No lots closing soon</td></tr>';
return;
}
tableBody.innerHTML = '';
dashboardData.closingSoon.forEach(lot => {
const row = document.createElement('tr');
row.className = 'hover:bg-gray-50';
const minutesLeft = lot.minutesUntilClose || 0;
const badgeColor = minutesLeft < 10 ? 'bg-red-100 text-red-800' : 'bg-yellow-100 text-yellow-800';
row.innerHTML = `
<td class="px-6 py-4 whitespace-nowrap text-sm font-medium text-gray-900">${lot.lotId || '--'}</td>
<td class="px-6 py-4 text-sm text-gray-900">${lot.title || 'N/A'}</td>
<td class="px-6 py-4 whitespace-nowrap text-sm font-bold text-green-600">${lot.currency || ''}${lot.currentBid ? lot.currentBid.toFixed(2) : '0.00'}</td>
<td class="px-6 py-4 whitespace-nowrap text-sm text-gray-500">${lot.closingTime || 'N/A'}</td>
<td class="px-6 py-4 whitespace-nowrap">
<span class="inline-flex items-center px-2.5 py-0.5 rounded-full text-xs font-medium ${badgeColor}">
${minutesLeft} min
</span>
</td>
<td class="px-6 py-4 whitespace-nowrap text-sm font-medium">
<a href="${lot.url || '#'}" target="_blank" class="text-blue-600 hover:text-blue-900">
<i class="fas fa-external-link-alt"></i>
</a>
</td>
`;
tableBody.appendChild(row);
});
}
// Update charts (placeholder)
function updateCharts() {
// Country distribution pie chart
const countryData = {
values: [5, 3, 2],
labels: ['NL', 'BE', 'DE'],
type: 'pie',
marker: { colors: ['#3b82f6', '#10b981', '#f59e0b'] }
};
Plotly.newPlot('country-chart', [countryData], {
showlegend: true,
margin: { t: 20, b: 20, l: 20, r: 20 }
});
// Category distribution
const categoryData = {
values: [4, 3, 2, 1],
labels: ['Machinery', 'Material Handling', 'Power Generation', 'Furniture'],
type: 'pie',
marker: { colors: ['#3b82f6', '#10b981', '#f59e0b', '#ef4444'] }
};
Plotly.newPlot('category-chart', [categoryData], {
showlegend: true,
margin: { t: 20, b: 20, l: 20, r: 20 }
});
}
// Trigger workflow
async function triggerWorkflow(workflow) {
const statusId = 'status-' + workflow.split('-')[0];
const statusEl = document.getElementById(statusId);
statusEl.textContent = 'Running...';
statusEl.className = 'text-xs text-blue-600 font-semibold';
addLog(`Triggering workflow: ${workflow}`);
try {
const response = await fetch(`/api/monitor/trigger/${workflow}`, {
method: 'POST'
});
if (!response.ok) throw new Error('Workflow trigger failed');
const result = await response.json();
addLog(`✓ ${result.message || 'Workflow completed'}`, 'success');
statusEl.textContent = 'Complete';
statusEl.className = 'text-xs text-green-600 font-semibold';
// Refresh data after workflow
setTimeout(fetchAllData, 2000);
} catch (error) {
console.error('Error triggering workflow:', error);
addLog(`✗ Workflow failed: ${error.message}`, 'error');
statusEl.textContent = 'Failed';
statusEl.className = 'text-xs text-red-600 font-semibold';
}
// Reset status after 5 seconds
setTimeout(() => {
statusEl.textContent = 'Ready';
statusEl.className = 'text-xs text-gray-500';
}, 5000);
}
// Add log entry
function addLog(message, type = 'info') {
const logContainer = document.getElementById('activity-log');
const logEntry = document.createElement('div');
const timestamp = new Date().toLocaleTimeString();
let iconClass = 'fas fa-info-circle text-blue-600';
let textClass = 'text-gray-700';
if (type === 'success') {
iconClass = 'fas fa-check-circle text-green-600';
textClass = 'text-green-700';
} else if (type === 'error') {
iconClass = 'fas fa-exclamation-circle text-red-600';
textClass = 'text-red-700';
}
logEntry.className = `text-sm flex items-start space-x-2 ${textClass}`;
logEntry.innerHTML = `
<i class="${iconClass} mt-1"></i>
<span class="text-gray-500">${timestamp}</span>
<span>${message}</span>
`;
logContainer.insertBefore(logEntry, logContainer.firstChild);
// Keep only last 20 entries
while (logContainer.children.length > 20) {
logContainer.removeChild(logContainer.lastChild);
}
}
// Update last update timestamp
function updateLastUpdate() {
const now = new Date();
document.getElementById('last-update').textContent =
`Last updated: ${now.toLocaleTimeString()}`;
}
</script>
</body>
</html>

224
src/main/resources/META-INF/resources/status.html Normal file
View File

@@ -0,0 +1,224 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Scrape-UI 1 - Enterprise</title>
<script src="https://cdn.tailwindcss.com"></script>
<style>
.gradient-bg {
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
}
.card-hover {
transition: all 0.3s ease;
}
.card-hover:hover {
transform: translateY(-5px);
box-shadow: 0 20px 25px -5px rgba(0, 0, 0, 0.1), 0 10px 10px -5px rgba(0, 0, 0, 0.04);
}
</style>
</head>
<body class="bg-gray-50 min-h-screen">
<!-- Header -->
<header class="gradient-bg text-white py-8">
<div class="container mx-auto px-4">
<h1 class="text-4xl font-bold mb-2">Scrape-UI Enterprise</h1>
<p class="text-xl opacity-90">Powered by Quarkus + Modern Frontend</p>
</div>
</header>
<!-- Main Content -->
<main class="container mx-auto px-4 py-8">
<!-- API Status Card -->
<!-- API & Build Status Card -->
<div class="bg-white rounded-lg shadow-md p-6 mb-8 card-hover">
<h2 class="text-2xl font-bold mb-4 text-gray-800">Build & Runtime Status</h2>
<div id="api-status" class="space-y-4">
<div class="grid grid-cols-1 md:grid-cols-2 gap-4">
<!-- Build Information -->
<div class="bg-blue-50 p-4 rounded-lg">
<h3 class="font-semibold text-blue-800 mb-2">📦 Maven Build</h3>
<div class="space-y-2 text-sm">
<div class="flex justify-between">
<span class="text-gray-600">Group:</span>
<span class="font-mono font-medium" id="build-group">-</span>
</div>
<div class="flex justify-between">
<span class="text-gray-600">Artifact:</span>
<span class="font-mono font-medium" id="build-artifact">-</span>
</div>
<div class="flex justify-between">
<span class="text-gray-600">Version:</span>
<span class="font-mono font-medium px-2 py-1 bg-blue-100 rounded" id="build-version">-</span>
</div>
</div>
</div>
<!-- Runtime Information -->
<div class="bg-green-50 p-4 rounded-lg">
<h3 class="font-semibold text-green-800 mb-2">🚀 Runtime</h3>
<div class="space-y-2 text-sm">
<div class="flex justify-between">
<span class="text-gray-600">Status:</span>
<span class="px-2 py-1 rounded-full text-xs font-semibold bg-green-100 text-green-800" id="runtime-status">-</span>
</div>
<div class="flex justify-between">
<span class="text-gray-600">Java:</span>
<span class="font-mono" id="java-version">-</span>
</div>
<div class="flex justify-between">
<span class="text-gray-600">Platform:</span>
<span class="font-mono" id="runtime-os">-</span>
</div>
</div>
</div>
</div>
<!-- Timestamp & Additional Info -->
<div class="pt-4 border-t">
<div class="flex justify-between items-center">
<div>
<p class="text-sm text-gray-500">Last Updated</p>
<p class="font-medium" id="last-updated">-</p>
</div>
<button onclick="fetchStatus()" class="bg-gray-100 hover:bg-gray-200 text-gray-700 px-4 py-2 rounded-lg text-sm transition-colors">
🔄 Refresh
</button>
</div>
</div>
</div>
</div>
<!-- API Response Card -->
<div class="bg-white rounded-lg shadow-md p-6 card-hover">
<h2 class="text-2xl font-bold mb-4 text-gray-800">API Test</h2>
<button id="test-api" class="bg-blue-600 text-white px-6 py-2 rounded-lg hover:bg-blue-700 transition-colors mb-4">
Test Greeting API
</button>
<div id="api-response" class="bg-gray-100 p-4 rounded-lg">
<pre class="text-sm text-gray-700">Click the button to test the API</pre>
</div>
</div>
<!-- Features Grid -->
<div class="grid grid-cols-1 md:grid-cols-3 gap-6 mt-8">
<div class="bg-white rounded-lg shadow-md p-6 card-hover">
<h3 class="text-xl font-semibold mb-2 text-gray-800">⚡ Quarkus Backend</h3>
<p class="text-gray-600">Fast startup, low memory footprint, optimized for containers</p>
</div>
<div class="bg-white rounded-lg shadow-md p-6 card-hover">
<h3 class="text-xl font-semibold mb-2 text-gray-800">🚀 REST API</h3>
<p class="text-gray-600">RESTful endpoints with JSON responses</p>
</div>
<div class="bg-white rounded-lg shadow-md p-6 card-hover">
<h3 class="text-xl font-semibold mb-2 text-gray-800">🎨 Modern UI</h3>
<p class="text-gray-600">Responsive design with Tailwind CSS</p>
</div>
</div>
</main>
<script>
// Fetch API status on load
async function fetchStatus() {
try {
const response = await fetch('/api/status')
if (!response.ok) {
throw new Error(`HTTP ${ response.status }: ${ response.statusText }`)
}
const data = await response.json()
// Update Build Information
document.getElementById('build-group').textContent = data.groupId || 'N/A'
document.getElementById('build-artifact').textContent = data.artifactId || data.name || 'N/A'
document.getElementById('build-version').textContent = data.version || 'N/A'
// Update Runtime Information
document.getElementById('runtime-status').textContent = data.status || 'unknown'
document.getElementById('java-version').textContent = data.javaVersion || System.getProperty?.('java.version') || 'N/A'
document.getElementById('runtime-os').textContent = data.os || 'N/A'
// Update Timestamp
const timestamp = data.timestamp ? new Date(data.timestamp).toLocaleString() : 'N/A'
document.getElementById('last-updated').textContent = timestamp
// Update status badge color based on status
const statusBadge = document.getElementById('runtime-status')
if (data.status?.toLowerCase() === 'running') {
statusBadge.className = 'px-2 py-1 rounded-full text-xs font-semibold bg-green-100 text-green-800'
} else {
statusBadge.className = 'px-2 py-1 rounded-full text-xs font-semibold bg-yellow-100 text-yellow-800'
}
} catch (error) {
console.error('Error fetching status:', error)
document.getElementById('api-status').innerHTML = `
<div class="bg-red-50 border-l-4 border-red-500 p-4">
<div class="flex">
<div class="flex-shrink-0">
<svg class="h-5 w-5 text-red-400" viewBox="0 0 20 20" fill="currentColor">
<path fill-rule="evenodd" d="M10 18a8 8 0 100-16 8 8 0 000 16zM8.707 7.293a1 1 0 00-1.414 1.414L8.586 10l-1.293 1.293a1 1 0 101.414 1.414L10 11.414l1.293 1.293a1 1 0 001.414-1.414L11.414 10l1.293-1.293a1 1 0 00-1.414-1.414L10 8.586 8.707 7.293z" clip-rule="evenodd" />
</svg>
</div>
<div class="ml-3">
<p class="text-sm text-red-700">Failed to load status: ${ error.message }</p>
<button onclick="fetchStatus()" class="mt-2 text-sm text-red-700 hover:text-red-600 font-medium">
Retry →
</button>
</div>
</div>
</div>
`
}
}
// Fetch API status on load
async function fetchStatus3() {
try {
const response = await fetch('/api/status')
const data = await response.json()
document.getElementById('api-status').innerHTML = `
<p><strong>Application:</strong> ${ data.application }</p>
<p><strong>Status:</strong> <span class="text-green-600 font-semibold">${ data.status }</span></p>
<p><strong>Version:</strong> ${ data.version }</p>
<p><strong>Timestamp:</strong> ${ data.timestamp }</p>
`
} catch (error) {
document.getElementById('api-status').innerHTML = `
<p class="text-red-600">Error loading status: ${ error.message }</p>
`
}
}
// Test greeting API
document.getElementById('test-api').addEventListener('click', async () => {
try {
const response = await fetch('/api/hello')
const data = await response.json()
document.getElementById('api-response').innerHTML = `
<pre class="text-sm text-gray-700">${ JSON.stringify(data, null, 2) }</pre>
`
} catch (error) {
document.getElementById('api-response').innerHTML = `
<pre class="text-sm text-red-600">Error: ${ error.message }</pre>
`
}
})
// Auto-refresh every 30 seconds
let refreshInterval = setInterval(fetchStatus, 30000);
// Stop auto-refresh when page loses focus (optional)
document.addEventListener('visibilitychange', function() {
if (document.hidden) {
clearInterval(refreshInterval);
} else {
refreshInterval = setInterval(fetchStatus, 30000);
fetchStatus(); // Refresh immediately when returning to tab
}
});
// Load status on page load
fetchStatus()
</script>
</body>
</html>

153
wiki/EXPERT_ANALITICS.sql Normal file
View File

@@ -0,0 +1,153 @@
-- Extend 'lots' table
ALTER TABLE lots
ADD COLUMN starting_bid DECIMAL(12, 2);
ALTER TABLE lots
ADD COLUMN estimated_min DECIMAL(12, 2);
ALTER TABLE lots
ADD COLUMN estimated_max DECIMAL(12, 2);
ALTER TABLE lots
ADD COLUMN reserve_price DECIMAL(12, 2);
ALTER TABLE lots
ADD COLUMN reserve_met BOOLEAN DEFAULT FALSE;
ALTER TABLE lots
ADD COLUMN bid_increment DECIMAL(12, 2);
ALTER TABLE lots
ADD COLUMN watch_count INTEGER DEFAULT 0;
ALTER TABLE lots
ADD COLUMN view_count INTEGER DEFAULT 0;
ALTER TABLE lots
ADD COLUMN first_bid_time TEXT;
ALTER TABLE lots
ADD COLUMN last_bid_time TEXT;
ALTER TABLE lots
ADD COLUMN bid_velocity DECIMAL(5, 2);
-- bids per hour
-- New table: bid history (CRITICAL)
CREATE TABLE bid_history
(
id INTEGER PRIMARY KEY AUTOINCREMENT,
lot_id TEXT REFERENCES lots (lot_id),
bid_amount DECIMAL(12, 2) NOT NULL,
bid_time TEXT NOT NULL,
is_winning BOOLEAN DEFAULT FALSE,
is_autobid BOOLEAN DEFAULT FALSE,
bidder_id TEXT, -- anonymized
created_at TEXT DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX idx_bid_history_lot_time ON bid_history (lot_id, bid_time);
-- Extend 'lots' table
ALTER TABLE lots
ADD COLUMN condition_score DECIMAL(3, 2); -- 0.00-10.00
ALTER TABLE lots
ADD COLUMN condition_description TEXT;
ALTER TABLE lots
ADD COLUMN year_manufactured INTEGER;
ALTER TABLE lots
ADD COLUMN serial_number TEXT;
ALTER TABLE lots
ADD COLUMN originality_score DECIMAL(3, 2); -- % original parts
ALTER TABLE lots
ADD COLUMN provenance TEXT;
ALTER TABLE lots
ADD COLUMN comparable_lot_ids TEXT;
-- JSON array
-- New table: comparable sales
CREATE TABLE comparable_sales
(
id INTEGER PRIMARY KEY AUTOINCREMENT,
lot_id TEXT REFERENCES lots (lot_id),
comparable_lot_id TEXT,
similarity_score DECIMAL(3, 2), -- 0.00-1.00
price_difference_percent DECIMAL(5, 2),
created_at TEXT DEFAULT CURRENT_TIMESTAMP
);
-- New table: market indices
CREATE TABLE market_indices
(
id INTEGER PRIMARY KEY AUTOINCREMENT,
category TEXT NOT NULL,
manufacturer TEXT,
avg_price DECIMAL(12, 2),
median_price DECIMAL(12, 2),
price_change_30d DECIMAL(5, 2),
volume_change_30d DECIMAL(5, 2),
calculated_at TEXT DEFAULT CURRENT_TIMESTAMP
);
-- Extend 'auctions' table
ALTER TABLE auctions
ADD COLUMN auction_house TEXT;
ALTER TABLE auctions
ADD COLUMN auction_house_rating DECIMAL(3, 2);
ALTER TABLE auctions
ADD COLUMN buyers_premium_percent DECIMAL(5, 2);
ALTER TABLE auctions
ADD COLUMN payment_methods TEXT; -- JSON
ALTER TABLE auctions
ADD COLUMN shipping_cost_min DECIMAL(12, 2);
ALTER TABLE auctions
ADD COLUMN shipping_cost_max DECIMAL(12, 2);
ALTER TABLE auctions
ADD COLUMN seller_verified BOOLEAN DEFAULT FALSE;
-- New table: auction performance metrics
CREATE TABLE auction_metrics
(
id INTEGER PRIMARY KEY AUTOINCREMENT,
auction_id TEXT REFERENCES auctions (auction_id),
sell_through_rate DECIMAL(5, 2),
avg_hammer_vs_estimate DECIMAL(5, 2),
total_hammer_price DECIMAL(15, 2),
total_starting_price DECIMAL(15, 2),
calculated_at TEXT DEFAULT CURRENT_TIMESTAMP
);
-- New table: seasonal trends
CREATE TABLE seasonal_trends
(
id INTEGER PRIMARY KEY AUTOINCREMENT,
category TEXT NOT NULL,
month INTEGER NOT NULL,
avg_price_multiplier DECIMAL(4, 2), -- vs annual avg
volume_multiplier DECIMAL(4, 2),
PRIMARY KEY (category, month)
);
-- New table: external market data
CREATE TABLE external_market_data
(
id INTEGER PRIMARY KEY AUTOINCREMENT,
category TEXT NOT NULL,
manufacturer TEXT,
model TEXT,
dealer_avg_price DECIMAL(12, 2),
retail_avg_price DECIMAL(12, 2),
wholesale_avg_price DECIMAL(12, 2),
source TEXT,
fetched_at TEXT DEFAULT CURRENT_TIMESTAMP
);
-- New table: image analysis results
CREATE TABLE image_analysis
(
id INTEGER PRIMARY KEY AUTOINCREMENT,
image_id INTEGER REFERENCES images (id),
damage_detected BOOLEAN,
damage_severity DECIMAL(3, 2),
wear_level TEXT CHECK (wear_level IN ('EXCELLENT', 'GOOD', 'FAIR', 'POOR')),
estimated_hours_used INTEGER,
ai_confidence DECIMAL(3, 2)
);
-- New table: economic indicators
CREATE TABLE economic_indicators
(
id INTEGER PRIMARY KEY AUTOINCREMENT,
indicator_date TEXT NOT NULL,
currency TEXT NOT NULL,
exchange_rate DECIMAL(10, 4),
inflation_rate DECIMAL(5, 2),
market_volatility DECIMAL(5, 2)
);

38
wiki/EXPERT_ANALITICS_PRIORITY.md Normal file
View File

@@ -0,0 +1,38 @@
```mermaid
graph TD
A[Add bid_history table] --> B[Add watch_count + estimates]
B --> C[Create market_indices]
C --> D[Add condition + year fields]
D --> E[Build comparable matching]
E --> F[Enrich with auction house data]
F --> G[Add AI image analysis]
```
| Current Practice | New Requirement | Why |
|-----------------------|---------------------------------|---------------------------|
| Scrape once per hour | **Scrape every bid update** | Capture velocity & timing |
| Save only current bid | **Save full bid history** | Detect patterns & sniping |
| Ignore watchers | **Track watch\_count** | Predict competition |
| Skip auction metadata | **Capture house estimates** | Anchor valuations |
| No historical data | **Store sold prices** | Train prediction models |
| Basic text scraping | **Parse condition/serial/year** | Enable comparables |
```bazaar
Week 1-2: Foundation
Implement bid_history scraping (most critical)
Add watch_count, starting_bid, estimated_min/max fields
Calculate basic bid_velocity
Week 3-4: Valuation
Extract year_manufactured, manufacturer, condition_description
Create market_indices (manually or via external API)
Build comparable lot matching logic
Week 5-6: Intelligence Layer
Add auction house performance tracking
Implement undervaluation detection algorithm
Create price alert system
Week 7-8: Automation
Integrate image analysis API
Add economic indicator tracking
Refine ML-based price predictions
```