start
This commit is contained in:
191
wiki/QUICKSTART.md
Normal file
191
wiki/QUICKSTART.md
Normal file
@@ -0,0 +1,191 @@
|
||||
# Quick Start Guide
|
||||
|
||||
Get the scraper running in minutes without downloading YOLO models!
|
||||
|
||||
## Minimal Setup (No Object Detection)
|
||||
|
||||
The scraper works perfectly fine **without** YOLO object detection. You can run it immediately and add object detection later if needed.
|
||||
|
||||
### Step 1: Run the Scraper
|
||||
|
||||
```bash
|
||||
# Using Maven
|
||||
mvn clean compile exec:java -Dexec.mainClass="com.auction.scraper.TroostwijkScraper"
|
||||
```
|
||||
|
||||
Or in IntelliJ IDEA:
|
||||
1. Open `TroostwijkScraper.java`
|
||||
2. Right-click on the `main` method
|
||||
3. Select "Run 'TroostwijkScraper.main()'"
|
||||
|
||||
### What You'll See
|
||||
|
||||
```
|
||||
=== Troostwijk Auction Scraper ===
|
||||
|
||||
Initializing scraper...
|
||||
⚠️ Object detection disabled: YOLO model files not found
|
||||
Expected files:
|
||||
- models/yolov4.cfg
|
||||
- models/yolov4.weights
|
||||
- models/coco.names
|
||||
Scraper will continue without image analysis.
|
||||
|
||||
[1/3] Discovering Dutch auctions...
|
||||
✓ Found 5 auctions: [12345, 12346, 12347, 12348, 12349]
|
||||
|
||||
[2/3] Fetching lot details...
|
||||
Processing sale 12345...
|
||||
|
||||
[3/3] Starting monitoring service...
|
||||
✓ Monitoring active. Press Ctrl+C to stop.
|
||||
```
|
||||
|
||||
### Step 2: Test Desktop Notifications
|
||||
|
||||
The scraper will automatically send desktop notifications when:
|
||||
- A new bid is placed on a monitored lot
|
||||
- An auction is closing within 5 minutes
|
||||
|
||||
**No setup required** - desktop notifications work out of the box!
|
||||
|
||||
---
|
||||
|
||||
## Optional: Add Email Notifications
|
||||
|
||||
If you want email notifications in addition to desktop notifications:
|
||||
|
||||
```bash
|
||||
# Set environment variable
|
||||
export NOTIFICATION_CONFIG="smtp:your.email@gmail.com:app_password:your.email@gmail.com"
|
||||
|
||||
# Then run the scraper
|
||||
mvn exec:java -Dexec.mainClass="com.auction.scraper.TroostwijkScraper"
|
||||
```
|
||||
|
||||
**Get Gmail App Password:**
|
||||
1. Enable 2FA in Google Account
|
||||
2. Go to: Google Account → Security → 2-Step Verification → App passwords
|
||||
3. Generate password for "Mail"
|
||||
4. Use that password (not your regular Gmail password)
|
||||
|
||||
---
|
||||
|
||||
## Optional: Add Object Detection Later
|
||||
|
||||
If you want AI-powered image analysis to detect objects in auction photos:
|
||||
|
||||
### 1. Create models directory
|
||||
```bash
|
||||
mkdir models
|
||||
cd models
|
||||
```
|
||||
|
||||
### 2. Download YOLO files
|
||||
```bash
|
||||
# YOLOv4 config (small)
|
||||
curl -O https://raw.githubusercontent.com/AlexeyAB/darknet/master/cfg/yolov4.cfg
|
||||
|
||||
# YOLOv4 weights (245 MB - takes a few minutes)
|
||||
curl -LO https://github.com/AlexeyAB/darknet/releases/download/darknet_yolo_v3_optimal/yolov4.weights
|
||||
|
||||
# COCO class names
|
||||
curl -O https://raw.githubusercontent.com/AlexeyAB/darknet/master/data/coco.names
|
||||
```
|
||||
|
||||
### 3. Run again
|
||||
```bash
|
||||
mvn exec:java -Dexec.mainClass="com.auction.scraper.TroostwijkScraper"
|
||||
```
|
||||
|
||||
Now you'll see:
|
||||
```
|
||||
✓ Object detection enabled with YOLO
|
||||
```
|
||||
|
||||
The scraper will now analyze auction images and detect objects like:
|
||||
- Vehicles (cars, trucks, forklifts)
|
||||
- Equipment (machines, tools)
|
||||
- Furniture
|
||||
- Electronics
|
||||
- And 80+ other object types
|
||||
|
||||
---
|
||||
|
||||
## Features Without Object Detection
|
||||
|
||||
Even without YOLO, the scraper provides:
|
||||
|
||||
✅ **Full auction scraping** - Discovers all Dutch auctions
|
||||
✅ **Lot tracking** - Monitors bids and closing times
|
||||
✅ **Desktop notifications** - Real-time alerts
|
||||
✅ **SQLite database** - All data persisted locally
|
||||
✅ **Image downloading** - Saves all lot images
|
||||
✅ **Scheduled monitoring** - Automatic updates every hour
|
||||
|
||||
Object detection simply adds:
|
||||
- AI-powered image analysis
|
||||
- Automatic object labeling
|
||||
- Searchable image database
|
||||
|
||||
---
|
||||
|
||||
## Database Location
|
||||
|
||||
The scraper creates `troostwijk.db` in your current directory with:
|
||||
- All auction data
|
||||
- Lot details (title, description, bids, etc.)
|
||||
- Downloaded image paths
|
||||
- Object labels (if detection enabled)
|
||||
|
||||
View the database with any SQLite browser:
|
||||
```bash
|
||||
sqlite3 troostwijk.db
|
||||
.tables
|
||||
SELECT * FROM lots LIMIT 5;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Stopping the Scraper
|
||||
|
||||
Press **Ctrl+C** to stop the monitoring service.
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. ✅ **Run the scraper** without YOLO to test it
|
||||
2. ✅ **Verify desktop notifications** work
|
||||
3. ⚙️ **Optional**: Add email notifications
|
||||
4. ⚙️ **Optional**: Download YOLO models for object detection
|
||||
5. 🔧 **Customize**: Edit monitoring frequency, closing alerts, etc.
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Desktop notifications not appearing?
|
||||
- **Windows**: Check if Java has notification permissions
|
||||
- **Linux**: Ensure desktop environment is running (not headless)
|
||||
- **macOS**: Check System Preferences → Notifications
|
||||
|
||||
### OpenCV warnings?
|
||||
These are normal and can be ignored:
|
||||
```
|
||||
WARNING: A restricted method in java.lang.System has been called
|
||||
WARNING: Use --enable-native-access=ALL-UNNAMED to avoid warning
|
||||
```
|
||||
|
||||
The scraper works fine despite these warnings.
|
||||
|
||||
---
|
||||
|
||||
## Full Documentation
|
||||
|
||||
See [README.md](../README.md) for complete documentation including:
|
||||
- Email setup details
|
||||
- YOLO installation guide
|
||||
- Configuration options
|
||||
- Database schema
|
||||
- API endpoints
|
||||
Reference in New Issue
Block a user