110 lines
2.5 KiB
Markdown
110 lines
2.5 KiB
Markdown
# Production Data Sync Setup
|
|
|
|
Quick reference for syncing production data from `athena.lan` to your local development environment.
|
|
|
|
## 🚀 One-Command Setup
|
|
|
|
### Linux/Mac
|
|
```bash
|
|
./scripts/sync-production-data.sh
|
|
```
|
|
|
|
## 📋 Complete Usage
|
|
|
|
### Bash (Linux/Mac/Git Bash)
|
|
```bash
|
|
# Database only
|
|
./scripts/sync-production-data.sh --db-only
|
|
|
|
# Everything
|
|
./scripts/sync-production-data.sh --all
|
|
|
|
# Images only
|
|
./scripts/sync-production-data.sh --images-only
|
|
```
|
|
|
|
## 🔧 What It Does
|
|
|
|
1. **Connects to athena.lan** via SSH
|
|
2. **Copies database** from Docker volume to /tmp
|
|
3. **Downloads to local** machine (c:\mnt\okcomputer\cache.db)
|
|
4. **Backs up** existing local database automatically
|
|
5. **Shows statistics** (auction count, lot count, etc.)
|
|
6. **Cleans up** temporary files on remote server
|
|
|
|
### With Images
|
|
- Also syncs the `/data/images/` directory
|
|
- Uses rsync for incremental sync (if available)
|
|
- Can be large (several GB)
|
|
|
|
## 📊 What You Get
|
|
|
|
### Database (`cache.db`)
|
|
- **~8.9 GB** of production data
|
|
- 16,000+ lots
|
|
- 536,000+ images metadata
|
|
- Full auction history
|
|
- HTTP cache from scraper
|
|
|
|
### Images (`images/`)
|
|
- Downloaded lot images
|
|
- Organized by lot ID
|
|
- Variable size (can be large)
|
|
|
|
## ⚡ Quick Workflow
|
|
|
|
### Daily Development
|
|
```powershell
|
|
# Morning: Get fresh data
|
|
.\scripts\Sync-ProductionData.sh -Force
|
|
|
|
# Develop & test
|
|
mvn quarkus:dev
|
|
|
|
# View dashboard
|
|
start http://localhost:8080
|
|
```
|
|
|
|
## 🔒 Safety Features
|
|
|
|
- ✅ **Automatic backups** before overwriting
|
|
- ✅ **Confirmation prompts** (unless `-Force`)
|
|
- ✅ **Error handling** with clear messages
|
|
- ✅ **Cleanup** of temporary files
|
|
- ✅ **Non-destructive** - production data is never modified
|
|
|
|
## 🐛 Troubleshooting
|
|
|
|
### "Permission denied" or SSH errors
|
|
```bash
|
|
# Test SSH connection
|
|
ssh tour@athena.lan "echo OK"
|
|
|
|
# If fails, check your SSH key
|
|
ssh-add -l
|
|
```
|
|
|
|
### Database already exists
|
|
- Script automatically backs up existing database
|
|
- Backup format: `cache.db.backup-YYYYMMDD-HHMMSS`
|
|
|
|
### Slow image transfer
|
|
- Install rsync for 10x faster incremental sync
|
|
- Or sync database only: `.\scripts\Sync-ProductionData.sh` (default)
|
|
|
|
## 📚 Full Documentation
|
|
|
|
See [scripts/README.md](../scripts/README.md) for:
|
|
- Prerequisites
|
|
- Performance tips
|
|
- Automation setup
|
|
- Detailed troubleshooting
|
|
|
|
## 🎯 Common Use Cases
|
|
|
|
**Quick Links**:
|
|
- [Main README](../README.md)
|
|
- [Scripts Documentation](../scripts/README.md)
|
|
- [Integration Flowchart](INTEGRATION_FLOWCHART.md)
|
|
- [Intelligence Features](INTELLIGENCE_FEATURES_SUMMARY.md)
|