207 lines
4.8 KiB
Markdown
207 lines
4.8 KiB
Markdown
# Auctiora Scripts
|
|
|
|
Utility scripts for managing the Auctiora auction monitoring system.
|
|
|
|
## 📦 Available Scripts
|
|
|
|
### 1. Production Data Sync
|
|
|
|
Sync production database and images from `athena.lan` to your local development environment.
|
|
|
|
#### Quick Start
|
|
|
|
**Linux/Mac (Bash)**:
|
|
```bash
|
|
# Make executable (first time only)
|
|
chmod +x scripts/sync-production-data.sh
|
|
|
|
# Sync database only
|
|
./scripts/sync-production-data.sh --db-only
|
|
|
|
# Sync everything
|
|
./scripts/sync-production-data.sh --all
|
|
|
|
# Sync images only
|
|
./scripts/sync-production-data.sh --images-only
|
|
```
|
|
|
|
## 🔧 Prerequisites
|
|
|
|
### Required
|
|
- **SSH Client**: OpenSSH or equivalent
|
|
- Windows: Built-in on Windows 10+, or install [Git Bash](https://git-scm.com/downloads)
|
|
- Linux/Mac: Pre-installed
|
|
- **SCP**: Secure copy (usually comes with SSH)
|
|
- **SSH Access**: SSH key configured for `tour@athena.lan`
|
|
|
|
### Optional
|
|
- **rsync**: For efficient incremental image sync
|
|
- Windows: Install via [WSL](https://docs.microsoft.com/en-us/windows/wsl/install) or [Cygwin](https://www.cygwin.com/)
|
|
- Linux/Mac: Usually pre-installed
|
|
- **sqlite3**: For showing database statistics
|
|
- Windows: Download from [sqlite.org](https://www.sqlite.org/download.html)
|
|
- Linux: `sudo apt install sqlite3`
|
|
- Mac: Pre-installed
|
|
|
|
## 📊 What Gets Synced
|
|
|
|
### Database (`cache.db`)
|
|
- **Size**: ~8.9 GB (as of Dec 2024)
|
|
- **Contains**:
|
|
- Auctions metadata
|
|
- Lots (kavels) with bid information
|
|
- Images metadata and URLs
|
|
- HTTP cache for scraper
|
|
- **Local Path**: `c:\mnt\okcomputer\cache.db`
|
|
|
|
### Images Directory
|
|
- **Size**: Varies (can be large)
|
|
- **Contains**:
|
|
- Downloaded lot images
|
|
- Organized by lot ID
|
|
- **Local Path**: `c:\mnt\okcomputer\images\`
|
|
|
|
## 🚀 Usage Examples
|
|
|
|
## 📁 File Locations
|
|
|
|
### Remote (Production)
|
|
```
|
|
athena.lan
|
|
├── Docker Volume: shared-auction-data
|
|
│ ├── /data/cache.db (SQLite database)
|
|
│ └── /data/images/ (Image files)
|
|
└── /tmp/ (Temporary staging area)
|
|
```
|
|
|
|
### Local (Development)
|
|
```
|
|
c:\mnt\okcomputer\
|
|
├── cache.db (SQLite database)
|
|
├── cache.db.backup-* (Automatic backups)
|
|
└── images\ (Image files)
|
|
```
|
|
|
|
## 🔒 Safety Features
|
|
|
|
### Automatic Backups
|
|
- Existing local database is automatically backed up before sync
|
|
- Backup format: `cache.db.backup-YYYYMMDD-HHMMSS`
|
|
- Keep recent backups manually or clean up old ones
|
|
|
|
### Confirmation Prompts
|
|
- PowerShell script prompts for confirmation (unless `-Force` is used)
|
|
- Shows configuration before executing
|
|
- Safe to cancel at any time
|
|
|
|
### Error Handling
|
|
- Validates SSH connection before starting
|
|
- Cleans up temporary files on remote server
|
|
- Reports clear error messages
|
|
|
|
## ⚡ Performance Tips
|
|
|
|
### Faster Image Sync with rsync
|
|
Install rsync for incremental image sync (only new/changed files):
|
|
|
|
**Windows (WSL)**:
|
|
```powershell
|
|
wsl --install
|
|
wsl -d Ubuntu
|
|
sudo apt install rsync
|
|
```
|
|
|
|
**Windows (Chocolatey)**:
|
|
```powershell
|
|
choco install rsync
|
|
```
|
|
|
|
**Benefit**: First sync downloads everything, subsequent syncs only transfer changed files.
|
|
|
|
Images can be synced separately when needed for image processing tests.
|
|
|
|
## 🐛 Troubleshooting
|
|
|
|
### SSH Connection Issues
|
|
```powershell
|
|
# Test SSH connection
|
|
ssh tour@athena.lan "echo 'Connection OK'"
|
|
|
|
# Check SSH key
|
|
ssh-add -l
|
|
```
|
|
|
|
### Permission Denied
|
|
```bash
|
|
# Add SSH key (Linux/Mac)
|
|
chmod 600 ~/.ssh/id_rsa
|
|
ssh-add ~/.ssh/id_rsa
|
|
|
|
# Windows: Use PuTTY or OpenSSH for Windows
|
|
```
|
|
|
|
### Database Locked Error
|
|
```powershell
|
|
# Make sure no other process is using the database
|
|
Get-Process | Where-Object {$_.Path -like "*java*"} | Stop-Process
|
|
|
|
# Or restart the monitor
|
|
```
|
|
|
|
### Slow Image Sync
|
|
- Use rsync instead of scp (see Performance Tips)
|
|
- Consider syncing only database for code development
|
|
- Images only needed for object detection testing
|
|
|
|
## 📝 Script Details
|
|
|
|
### sync-production-data.sh (Bash)
|
|
- **Platform**: Linux, Mac, Git Bash on Windows
|
|
- **Best for**: Unix-like environments
|
|
- **Features**: Color output, progress bars, statistics
|
|
|
|
## 🔄 Automation
|
|
|
|
### Linux/Mac Cron
|
|
```bash
|
|
# Edit crontab
|
|
crontab -e
|
|
|
|
# Add daily sync at 7 AM
|
|
0 7 * * * /path/to/auctiora/scripts/sync-production-data.sh --db-only
|
|
```
|
|
|
|
## 🆘 Support
|
|
|
|
### Getting Help
|
|
```bash
|
|
# Bash
|
|
./scripts/sync-production-data.sh --help
|
|
```
|
|
|
|
### Common Commands
|
|
```powershell
|
|
# Check database size
|
|
ls c:\mnt\okcomputer\cache.db -h
|
|
|
|
# View database contents
|
|
sqlite3 c:\mnt\okcomputer\cache.db
|
|
.tables
|
|
.schema lots
|
|
SELECT COUNT(*) FROM lots;
|
|
.quit
|
|
|
|
# Check image count
|
|
(Get-ChildItem c:\mnt\okcomputer\images -Recurse -File).Count
|
|
```
|
|
|
|
## 📚 Related Documentation
|
|
- [Database Architecture](../wiki/DATABASE_ARCHITECTURE.md)
|
|
- [Integration Flowchart](../docs/INTEGRATION_FLOWCHART.md)
|
|
- [Main README](../README.md)
|
|
|
|
---
|
|
|
|
**Last Updated**: December 2025
|
|
**Maintainer**: Auctiora Development Team
|