diff --git a/wiki/DEPLOYMENT_GUIDE.md b/wiki/DEPLOYMENT_GUIDE.md new file mode 100644 index 0000000..5284e2b --- /dev/null +++ b/wiki/DEPLOYMENT_GUIDE.md @@ -0,0 +1,192 @@ +# Deployment Guide - App Pipeline + +Complete guide for deploying applications using the custom deployment pipeline with Gitea, Docker, and Traefik. + +## 1. Create New Application + +### 1.1 Gitea Repository + +1. Log in to Gitea: https://git.appmodel.nl +2. Create a new repository: + - **Owner**: Tour + - **Name**: `viewer` (or your app name) + +### 1.2 Generate Skeleton on Server + +On the server, run: + +```bash +apps-create viewer static-fe +``` + +This command: +- Sets up `/opt/apps/viewer` (attempts to clone from `git@git.appmodel.nl:Tour/viewer.git`) +- Generates a multi-stage Dockerfile for a Node-based static frontend +- Creates `~/infra/viewer/docker-compose.yml` with: + - Service `viewer` + - Connection to `traefik_net` + - Traefik route: `https://viewer.appmodel.nl` + +## 2. Development & Build + +On your development machine: + +```bash +# Clone the repository +git clone git@git.appmodel.nl:Tour/viewer.git +cd viewer + +# Build your app as usual +npm install +npm run build # output goes to dist/ + +# Commit and push +git add . +git commit -m "First version" +git push +``` + +**Note**: The Dockerfile expects a `dist/` directory containing static files. + +## 3. Deployment Pipeline + +### 3.1 Manual Deploy with `app-deploy` + +On the server, use the generic deploy command: + +```bash +app-deploy viewer +``` + +This command performs: +1. `cd /opt/apps/viewer` +2. `git pull --ff-only` +3. `cd /home/tour/infra/viewer` +4. `docker compose up -d --build viewer` + +Logs are saved to: `/var/log/app-deploy-viewer.log` + +### 3.2 Automatic Deployment via Gitea Hook + +Set up automatic deployment on every push: + +1. In Gitea, go to repository `Tour/viewer` +2. Navigate to **Settings → Git Hooks** +3. Select **post-receive** +4. Add the following script: + +```bash +#!/usr/bin/env bash +/usr/local/bin/app-deploy viewer +``` + +**Result**: Every `git push` to `Tour/viewer` automatically triggers a deployment. + +## 4. Traefik & DNS Configuration + +### Traefik Setup + +Traefik runs in the traefik stack on the same server and manages: +- `git.appmodel.nl` +- `auction.appmodel.nl` +- `aupi.appmodel.nl` +- ... (new apps via labels) + +### Auto-Generated Labels + +The `apps-create` command automatically adds the correct Traefik labels: + +```yaml +labels: + - "traefik.enable=true" + - "traefik.http.routers.viewer.rule=Host(`viewer.appmodel.nl`)" + - "traefik.http.routers.viewer.entrypoints=websecure" + - "traefik.http.routers.viewer.tls=true" + - "traefik.http.routers.viewer.tls.certresolver=letsencrypt" + - "traefik.http.services.viewer.loadbalancer.server.port=80" +``` + +### DNS Configuration + +Add a DNS record pointing to your server's public IP: + +``` +viewer.appmodel.nl → +``` + +Traefik + Let's Encrypt will automatically handle SSL certificate generation. + +## 5. Future Application Types + +The `apps-create` script currently supports: + +- **`static-fe`** – Node build → Nginx → static frontend + +### Planned Types + +Future app types can be added: + +```bash +# Python API +apps-create stats api-py + +# Background worker/crawler +apps-create crawler worker-py +``` + +Each type will have: +- Custom Dockerfile template +- Standard `docker-compose.yml` configuration +- Same deployment pipeline: `app-deploy ` + +## Quick Reference + +### Create and Deploy New App + +```bash +# On server: create skeleton +apps-create viewer static-fe + +# On dev machine: develop and push +git clone git@git.appmodel.nl:Tour/viewer.git +cd viewer +npm install +npm run build +git add . +git commit -m "Initial commit" +git push + +# On server: deploy (or auto-deploys via hook) +app-deploy viewer +``` + +### Existing Infrastructure + +| Service | URL | Description | +|---------|-----|-------------| +| Gitea | https://git.appmodel.nl | Git repository hosting | +| Auction | https://auction.appmodel.nl | Auction frontend | +| Aupi API | https://aupi.appmodel.nl | Auction backend API | +| Traefik | - | Reverse proxy & SSL | + +### Log Locations + +- Deployment logs: `/var/log/app-deploy-.log` +- Docker logs: `docker compose logs -f ` + +### Useful Commands + +```bash +# View deployment logs +tail -f /var/log/app-deploy-viewer.log + +# View container logs +cd ~/infra/viewer +docker compose logs -f viewer + +# Restart service +docker compose restart viewer + +# Rebuild without cache +docker compose up -d --build --no-cache viewer +``` diff --git a/wiki/DEPLOY_SERVER_SETUP.md b/wiki/DEPLOY_SERVER_SETUP.md new file mode 100644 index 0000000..e6353e2 --- /dev/null +++ b/wiki/DEPLOY_SERVER_SETUP.md @@ -0,0 +1,502 @@ +# Deploy Server Setup - Docker Build Pipeline + +Complete guide for setting up and using the Docker-based deployment pipeline on the build server. + +## Architecture Overview + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ Deployment Flow │ +├─────────────────────────────────────────────────────────────────┤ +│ │ +│ Dev Machine Gitea Server │ +│ ┌──────────┐ ┌──────┐ ┌─────────┐ │ +│ │ git │ │ Repo │ │ /opt/ │ │ +│ │ commit ├─────push────▶│Tour/ ├──hook────▶ │ apps/ │ │ +│ │ push │ │ │ │ / │ │ +│ └──────────┘ └──────┘ └────┬────┘ │ +│ │ │ +│ git pull │ +│ │ │ +│ ┌───────────▼────────┐ │ +│ │ app-deploy │ │ +│ │ (as user: git) │ │ +│ └───────────┬────────┘ │ +│ │ │ +│ docker compose up -d │ +│ --build │ +│ │ │ +│ ┌───────────▼────────┐ │ +│ │ ~/infra// │ │ +│ │ docker-compose.yml │ │ +│ └───────────┬────────┘ │ +│ │ │ +│ rebuild │ +│ │ │ +│ ┌───────────▼────────┐ │ +│ │ 🐳 Container │ │ +│ │ :latest │ │ +│ └───────────┬────────┘ │ +│ │ │ +│ traefik_net │ +│ │ │ +│ ┌───────────▼────────┐ │ +│ │ 🚦 Traefik │ │ +│ │ Reverse Proxy │ │ +│ └───────────┬────────┘ │ +│ │ │ +│ https://.appmodel.nl│ +│ │ +└─────────────────────────────────────────────────────────────────┘ +``` + +## Build Pipeline Workflow + +### Key Principles + +1. **Source of Truth**: Gitea repository `Tour/` is the authoritative source +2. **Build Server**: Server is only for building and deployment, not development +3. **Automated Deployment**: Git push triggers automatic rebuild and deployment +4. **Isolation**: Each app runs in its own Docker container +5. **Shared Networking**: All apps connect via `traefik_net` for reverse proxy + +### Pipeline Steps + +#### 1. Git Push (Developer) +```bash +git push origin main +``` + +#### 2. Post-Receive Hook (Gitea) +Located in: **Settings → Git Hooks → post-receive** + +```bash +#!/usr/bin/env bash +/usr/local/bin/app-deploy viewer +``` + +#### 3. App Deploy Script (Server) +Runs as Linux user `git`: + +```bash +app-deploy viewer +``` + +This script performs: +1. `cd /opt/apps/viewer` +2. `git pull --ff-only` from `git@git.appmodel.nl:Tour/viewer.git` +3. `cd /home/tour/infra/viewer` +4. `docker compose up -d --build viewer` + +#### 4. Docker Build (Server) +Multi-stage build process: +- **Stage 1**: Build viewer using Python + Graphviz +- **Stage 2**: Serve with Nginx + +#### 5. Traefik Routing (Server) +Traefik automatically detects the container via labels and publishes: +``` +https://viewer.appmodel.nl +``` + +#### 6. DNS Resolution (AdGuard) +AdGuard resolves `viewer.appmodel.nl` → Build server IP + +Result: Both internal and external clients use the same URL. + +## Server Directory Structure + +``` +/opt/apps/ +└── viewer/ # Git repository (pulled from Gitea) + ├── Dockerfile + ├── docker-compose.yml + ├── requirements.txt + ├── lan_architecture.py + ├── main.py + └── public/ + └── index.html + +/home/tour/infra/ +└── viewer/ + ├── docker-compose.yml # Docker Compose config (may differ from repo) + └── logs/ # Nginx logs (optional) + +/var/log/ +└── app-deploy-viewer.log # Deployment logs +``` + +## Setup Instructions + +### 1. Create Repository in Gitea + +1. Navigate to: https://git.appmodel.nl +2. Create new repository: + - **Owner**: `Tour` + - **Name**: `viewer` + - **Visibility**: Private + +### 2. Initialize Local Repository + +```bash +# On dev machine +git clone git@git.appmodel.nl:Tour/viewer.git +cd viewer + +# Add project files +git add Dockerfile docker-compose.yml requirements.txt *.py public/ +git commit -m "Initial commit: viewer" +git push origin main +``` + +### 3. Setup Server Directories + +```bash +# On server as appropriate user +sudo -u git mkdir -p /opt/apps/viewer +sudo -u git git clone git@git.appmodel.nl:Tour/viewer.git /opt/apps/viewer + +mkdir -p /home/tour/infra/viewer +cp /opt/apps/viewer/docker-compose.yml /home/tour/infra/viewer/ +``` + +### 4. Configure Gitea Post-Receive Hook + +1. Go to: https://git.appmodel.nl/Tour/viewer +2. Navigate to: **Settings → Git Hooks** +3. Select: **post-receive** +4. Add script: + +```bash +#!/usr/bin/env bash +/usr/local/bin/app-deploy viewer +``` + +5. Save and test + +### 5. Configure DNS + +Add DNS record in AdGuard: +``` +viewer.appmodel.nl → +``` + +### 6. Test Deployment + +```bash +# Manual test on server +app-deploy viewer + +# Verify container is running +cd /home/tour/infra/viewer +docker compose ps + +# Check logs +docker compose logs -f viewer + +# Test endpoint +curl -I https://viewer.appmodel.nl +``` + +## Docker Configuration + +### Dockerfile Explained + +```dockerfile +# Stage 1: Build diagrams +FROM python:3.11-slim AS builder +RUN apt-get update && apt-get install -y graphviz +WORKDIR /app +COPY requirements.txt . +RUN pip install -r requirements.txt +COPY *.py . +RUN python lan_architecture.py && python main.py + +# Stage 2: Serve with Nginx +FROM nginx:alpine +COPY --from=builder /app/*.png /usr/share/nginx/html/ +COPY public/ /usr/share/nginx/html/ +EXPOSE 80 +CMD ["nginx", "-g", "daemon off;"] +``` + +**Benefits**: +- Small final image (only Nginx + static files) +- Build tools not included in production image +- Automatic diagram generation on build + +### docker-compose.yml Configuration + +```yaml +version: '3.8' + +services: + viewer: + build: + context: . + dockerfile: Dockerfile + container_name: viewer + restart: unless-stopped + networks: + - traefik_net + labels: + - "traefik.enable=true" + - "traefik.http.routers.viewer.rule=Host(`viewer.appmodel.nl`)" + - "traefik.http.routers.viewer.entrypoints=websecure" + - "traefik.http.routers.viewer.tls=true" + - "traefik.http.routers.viewer.tls.certresolver=letsencrypt" + - "traefik.http.services.viewer.loadbalancer.server.port=80" + +networks: + traefik_net: + external: true +``` + +**Key Points**: +- Connects to external `traefik_net` network +- Traefik labels configure automatic HTTPS with Let's Encrypt +- Container restarts automatically unless stopped manually + +## Development Workflow + +### Local Development + +```bash +# Edit files locally +vim lan_architecture.py +vim public/index.html + +# Test locally (optional) +python -m venv .venv +source .venv/bin/activate # or .venv\Scripts\activate on Windows +pip install -r requirements.txt +python lan_architecture.py + +# Commit and push +git add . +git commit -m "Update viewer architecture" +git push +``` + +### Automatic Deployment + +After `git push`, the server automatically: +1. Receives post-receive hook trigger +2. Pulls latest code +3. Rebuilds Docker image +4. Restarts container +5. Updates live site + +**Timeline**: Usually completes in 30-60 seconds. + +### Manual Deployment + +If needed, manually trigger deployment: + +```bash +# On server +app-deploy viewer + +# Or manually +cd /opt/apps/viewer +git pull +cd /home/tour/infra/viewer +docker compose up -d --build viewer +``` + +## Monitoring & Troubleshooting + +### View Logs + +```bash +# Deployment logs +tail -f /var/log/app-deploy-viewer.log + +# Container logs +cd /home/tour/infra/viewer +docker compose logs -f viewer + +# Nginx access logs (if volume mounted) +tail -f /home/tour/infra/viewer/logs/access.log +``` + +### Check Container Status + +```bash +cd /home/tour/infra/viewer + +# List running containers +docker compose ps + +# Inspect container +docker compose exec viewer sh + +# Test nginx config +docker compose exec viewer nginx -t +``` + +### Debug Build Issues + +```bash +# Rebuild without cache +docker compose build --no-cache viewer + +# View build output +docker compose up --build viewer + +# Check image layers +docker image history diagram-viewer +``` + +### Common Issues + +#### 1. Build Fails - Missing Dependencies + +**Symptom**: Build fails at pip install step + +**Solution**: Verify `requirements.txt` is correct +```bash +cd /opt/apps/viewer +cat requirements.txt +``` + +#### 2. Nginx 404 Error + +**Symptom**: Container runs but shows 404 + +**Solution**: Check if files were copied correctly +```bash +docker compose exec viewer ls -la /usr/share/nginx/html/ +``` + +#### 3. Traefik Not Routing + +**Symptom**: Container runs but domain not accessible + +**Solution**: +- Verify container is on `traefik_net`: `docker network inspect traefik_net` +- Check Traefik labels: `docker inspect viewer | grep traefik` +- Verify DNS: `nslookup viewer.appmodel.nl` + +#### 4. Git Pull Fails + +**Symptom**: app-deploy fails at git pull + +**Solution**: +- Check SSH keys for git user: `sudo -u git ssh -T git@git.appmodel.nl` +- Verify remote: `cd /opt/apps/viewer && git remote -v` + +## Security Considerations + +### User Permissions + +- **git user**: Owns `/opt/apps/*` directories, runs app-deploy +- **tour user**: Owns `/home/tour/infra/*`, runs Docker Compose +- Principle: Separate concerns between git operations and container management + +### Network Isolation + +- Containers only exposed via Traefik +- No direct port exposure to host +- `traefik_net` provides isolation between services + +### Secrets Management + +For apps requiring secrets: + +```yaml +# In docker-compose.yml +services: + viewer: + environment: + - SECRET_KEY=${SECRET_KEY} + env_file: + - .env # Not committed to git +``` + +Create `.env` file on server: +```bash +echo "SECRET_KEY=your-secret-here" > /home/tour/infra/viewer/.env +chmod 600 /home/tour/infra/viewer/.env +``` + +## Scaling & Future Improvements + +### Multi-Container Apps + +For apps with multiple services: + +```yaml +services: + frontend: + build: ./frontend + labels: + - "traefik.enable=true" + - "traefik.http.routers.app-fe.rule=Host(`app.appmodel.nl`)" + + backend: + build: ./backend + labels: + - "traefik.enable=true" + - "traefik.http.routers.app-be.rule=Host(`api.app.appmodel.nl`)" + + database: + image: postgres:15 + volumes: + - pgdata:/var/lib/postgresql/data + +volumes: + pgdata: +``` + +### CI/CD Integration + +Consider adding: +- Automated tests before deployment +- Rollback mechanism +- Blue-green deployments +- Health checks before switching traffic + +### Monitoring + +Add monitoring stack: +- Prometheus for metrics +- Grafana for dashboards +- Loki for log aggregation +- Alertmanager for notifications + +## Quick Reference + +### Key Paths + +| Path | Purpose | +|------|---------| +| `/opt/apps//` | Git repository checkout | +| `/home/tour/infra//` | Docker Compose directory | +| `/var/log/app-deploy-.log` | Deployment logs | + +### Key Commands + +| Command | Purpose | +|---------|---------| +| `app-deploy ` | Deploy/redeploy application | +| `docker compose ps` | List running containers | +| `docker compose logs -f ` | Follow container logs | +| `docker compose restart ` | Restart service | +| `docker compose build --no-cache` | Force rebuild | + +### URLs + +| Service | URL | +|---------|-----| +| Gitea | https://git.appmodel.nl | +| Diagram Viewer | https://viewer.appmodel.nl | +| Traefik Dashboard | https://traefik.appmodel.nl/dashboard/ | + +## Support + +For issues or questions: +1. Check deployment logs: `/var/log/app-deploy-.log` +2. Check container logs: `docker compose logs -f` +3. Review Gitea webhook history in repository settings +4. Test manual deployment: `app-deploy ` diff --git a/wiki/Home.md b/wiki/Home.md index 82e0cf6..927fc9c 100644 --- a/wiki/Home.md +++ b/wiki/Home.md @@ -25,8 +25,8 @@ This project provides a static web interface for viewing and managing network ar ```bash # Clone repository -git clone git@git.appmodel.nl:Tour/diagram.git -cd diagram +git clone git@git.appmodel.nl:Tour/viewer.git +cd viewer # Create virtual environment python -m venv .venv @@ -47,7 +47,7 @@ python main.py ### Project Structure ``` -diagram/ +viewer/ ├── Dockerfile # Multi-stage build for production ├── docker-compose.yml # Docker Compose configuration ├── requirements.txt # Python dependencies @@ -121,11 +121,11 @@ The project uses a fully automated Docker-based deployment pipeline: ```bash # On server: Create app structure -apps-create diagram static-fe +apps-create viewer static-fe # This creates: -# - /opt/apps/diagram (git repository) -# - /home/tour/infra/diagram/docker-compose.yml +# - /opt/apps/viewer (git repository) +# - /home/tour/infra/viewer/docker-compose.yml ``` #### 2. Configure Gitea Hook @@ -134,7 +134,7 @@ In repository **Settings → Git Hooks → post-receive**: ```bash #!/usr/bin/env bash -/usr/local/bin/app-deploy diagram +/usr/local/bin/app-deploy viewer ``` #### 3. Deploy @@ -149,7 +149,7 @@ The server automatically: - Pulls latest code - Rebuilds Docker image - Restarts container -- Updates live site at https://diagram.appmodel.nl +- Updates live site at https://viewer.appmodel.nl ### Manual Deployment @@ -157,24 +157,24 @@ If needed: ```bash # On server -app-deploy diagram +app-deploy viewer # Or step-by-step -cd /opt/apps/diagram +cd /opt/apps/viewer git pull -cd /home/tour/infra/diagram -docker compose up -d --build diagram +cd /home/tour/infra/viewer +docker compose up -d --build viewer ``` ### Monitoring ```bash # View deployment logs -tail -f /var/log/app-deploy-diagram.log +tail -f /var/log/app-deploy-viewer.log # View container logs -cd /home/tour/infra/diagram -docker compose logs -f diagram +cd /home/tour/infra/viewer +docker compose logs -f viewer # Check container status docker compose ps @@ -193,7 +193,7 @@ docker compose ps │ │ │ ┌──────────────┐ │ │ │ Gitea │ (Source of Truth) │ -│ │ Tour/diagram │ │ +│ │ Tour/viewer │ │ │ └──────┬───────┘ │ │ │ git push │ │ ↓ │ @@ -201,10 +201,10 @@ docker compose ps │ │ Post-Receive │ (Auto-trigger) │ │ │ Hook │ │ │ └──────┬───────┘ │ -│ │ app-deploy diagram │ +│ │ app-deploy viewer │ │ ↓ │ │ ┌──────────────────────┐ │ -│ │ /opt/apps/diagram/ │ │ +│ │ /opt/apps/viewer/ │ │ │ │ (Git Repository) │ │ │ └──────┬───────────────┘ │ │ │ git pull │ @@ -238,7 +238,7 @@ docker compose ps │ └──────┬───────────────┘ │ │ │ HTTPS │ │ ↓ │ -│ diagram.appmodel.nl │ +│ viewer.appmodel.nl │ │ │ └─────────────────────────────────────────────┘ ``` @@ -262,7 +262,7 @@ The diagram viewer documents our home lab infrastructure: | Service | URL | Description | |---------|-----|-------------| -| Diagram Viewer | https://diagram.appmodel.nl | This application | +| Diagram Viewer | https://viewer.appmodel.nl | This application | | Gitea | https://git.appmodel.nl | Git hosting | | Auction Frontend | https://auction.appmodel.nl | Auction platform | | Aupi API | https://aupi.appmodel.nl | Backend API | @@ -328,11 +328,11 @@ pip install --upgrade diagrams **Solution**: ```bash # Check deployment logs -tail -f /var/log/app-deploy-diagram.log +tail -f /var/log/app-deploy-viewer.log # Rebuild manually with verbose output -cd /home/tour/infra/diagram -docker compose build --no-cache --progress=plain diagram +cd /home/tour/infra/viewer +docker compose build --no-cache --progress=plain viewer ``` #### Site Not Accessible @@ -340,8 +340,8 @@ docker compose build --no-cache --progress=plain diagram **Problem**: Container runs but site not reachable **Solution**: -1. Check DNS: `nslookup diagram.appmodel.nl` -2. Verify Traefik labels: `docker inspect diagram-viewer | grep traefik` +1. Check DNS: `nslookup viewer.appmodel.nl` +2. Verify Traefik labels: `docker inspect viewer-viewer | grep traefik` 3. Check Traefik logs: `docker logs traefik` 4. Test container directly: `curl http://localhost:PORT` @@ -352,9 +352,9 @@ docker compose build --no-cache --progress=plain diagram **Solution**: ```bash # Force rebuild on server -cd /home/tour/infra/diagram +cd /home/tour/infra/viewer docker compose down -docker compose up -d --build --force-recreate diagram +docker compose up -d --build --force-recreate viewer # Clear browser cache # Ctrl+Shift+R (Windows/Linux) or Cmd+Shift+R (macOS) diff --git a/wiki/Quick-Start.md b/wiki/Quick-Start.md index 9e24dae..d64922b 100644 --- a/wiki/Quick-Start.md +++ b/wiki/Quick-Start.md @@ -7,8 +7,8 @@ Get up and running with the Diagram Viewer in minutes. ### 1. Clone Repository ```bash -git clone git@git.appmodel.nl:Tour/diagram.git -cd diagram +git clone git@git.appmodel.nl:Tour/viewer.git +cd viewer ``` ### 2. Setup Environment @@ -74,7 +74,7 @@ git push origin main Simply navigate to: ``` -https://diagram.appmodel.nl +https://viewer.appmodel.nl ``` ### Manual Deployment @@ -83,13 +83,13 @@ On the server: ```bash # Trigger deployment -app-deploy diagram +app-deploy viewer # View logs -tail -f /var/log/app-deploy-diagram.log +tail -f /var/log/app-deploy-viewer.log # Check status -cd /home/tour/infra/diagram +cd /home/tour/infra/viewer docker compose ps ``` @@ -97,27 +97,27 @@ docker compose ps ```bash # Deployment logs -tail -f /var/log/app-deploy-diagram.log +tail -f /var/log/app-deploy-viewer.log # Container logs -cd /home/tour/infra/diagram -docker compose logs -f diagram +cd /home/tour/infra/viewer +docker compose logs -f viewer # Last 100 lines -docker compose logs --tail=100 diagram +docker compose logs --tail=100 viewer ``` ### Restart Service ```bash -cd /home/tour/infra/diagram +cd /home/tour/infra/viewer # Restart container -docker compose restart diagram +docker compose restart viewer # Full restart (rebuild) docker compose down -docker compose up -d --build diagram +docker compose up -d --build viewer ``` --- @@ -176,10 +176,10 @@ git push docker compose ps # Health check -docker inspect diagram-viewer | grep -A 10 Health +docker inspect viewer | grep -A 10 Health # Test endpoint -curl -I https://diagram.appmodel.nl +curl -I https://viewer.appmodel.nl ``` --- @@ -190,11 +190,11 @@ curl -I https://diagram.appmodel.nl ```bash # Check logs -tail -f /var/log/app-deploy-diagram.log +tail -f /var/log/app-deploy-viewer.log # Rebuild with verbose output -cd /home/tour/infra/diagram -docker compose build --no-cache --progress=plain diagram +cd /home/tour/infra/viewer +docker compose build --no-cache --progress=plain viewer ``` ### Site Not Loading @@ -204,10 +204,10 @@ docker compose build --no-cache --progress=plain diagram docker compose ps # Check Traefik routing -docker logs traefik | grep diagram +docker logs traefik | grep viewer # Test DNS -nslookup diagram.appmodel.nl +nslookup viewer.appmodel.nl # Test directly curl http://localhost:PORT @@ -217,8 +217,8 @@ curl http://localhost:PORT ```bash # Force rebuild -cd /home/tour/infra/diagram -docker compose up -d --build --force-recreate diagram +cd /home/tour/infra/viewer +docker compose up -d --build --force-recreate viewer # Clear browser cache # Ctrl+Shift+R (Windows/Linux) @@ -232,7 +232,7 @@ docker compose up -d --build --force-recreate diagram - Read the [Home](Home.md) page for comprehensive documentation - Review [DEPLOY_SERVER_SETUP.md](../DEPLOY_SERVER_SETUP.md) for deployment details - Check [Diagrams Library docs](https://diagrams.mingrammer.com/) for icon options -- Explore the [Mermaid editor](https://diagram.appmodel.nl) for live diagram editing +- Explore the [Mermaid editor](https://viewer.appmodel.nl) for live diagram editing --- diff --git a/wiki/WIKI_INTEGRATION.md b/wiki/WIKI_INTEGRATION.md new file mode 100644 index 0000000..47ad860 --- /dev/null +++ b/wiki/WIKI_INTEGRATION.md @@ -0,0 +1,361 @@ +# Gitea Wiki Integration Guide + +This guide explains how to integrate the local `wiki/` folder with Gitea's wiki system. + +## How Gitea Wikis Work + +Gitea stores wiki pages in a **separate Git repository** with the naming pattern: +``` +.wiki.git +``` + +For the `viewer` repository: +- Main repo: `git@git.appmodel.nl:Tour/viewer.git` +- Wiki repo: `git@git.appmodel.nl:Tour/viewer.wiki.git` + +Wiki pages are **markdown files** (`.md`) stored in the root of the wiki repository. + +## Integration Options + +### Option 1: Push Local Wiki to Gitea (Recommended) + +This approach replaces the current Gitea wiki with your local content. + +```bash +# 1. Clone the Gitea wiki repository +git clone git@git.appmodel.nl:Tour/viewer.wiki.git viewer-wiki + +# 2. Copy your wiki files to the wiki repository +cd viewer-wiki +cp ../viewer/wiki/*.md . + +# 3. Commit and push +git add . +git commit -m "Initialize wiki with documentation" +git push origin master + +# 4. View on Gitea +# Navigate to: https://git.appmodel.nl/Tour/viewer/wiki +``` + +### Option 2: Use Git Submodule (Bidirectional Sync) + +Keep the wiki as a submodule in the main repository for easy synchronization. + +```bash +# In the main viewer repository +cd /c/vibe/viewer + +# Remove local wiki folder +rm -rf wiki/ + +# Add Gitea wiki as submodule +git submodule add git@git.appmodel.nl:Tour/viewer.wiki.git wiki + +# Copy your documentation +cp ../temp-wiki-backup/*.md wiki/ + +# Commit to wiki submodule +cd wiki +git add . +git commit -m "Add documentation" +git push origin master + +# Update main repository +cd .. +git add wiki +git commit -m "Add wiki as submodule" +git push origin main +``` + +Now changes can be made either: +- **In Gitea UI**: Edit wiki pages directly +- **Locally**: Edit in `wiki/` folder, commit, and push + +To sync changes from Gitea: +```bash +cd wiki +git pull origin master +cd .. +git add wiki +git commit -m "Update wiki submodule" +git push +``` + +### Option 3: Git Subtree (Advanced) + +Similar to submodule but integrates the wiki history into the main repository. + +```bash +# Add wiki as subtree +git subtree add --prefix=wiki git@git.appmodel.nl:Tour/viewer.wiki.git master --squash + +# Push changes to wiki +git subtree push --prefix=wiki git@git.appmodel.nl:Tour/viewer.wiki.git master + +# Pull changes from wiki +git subtree pull --prefix=wiki git@git.appmodel.nl:Tour/viewer.wiki.git master --squash +``` + +### Option 4: Automated Sync Script + +Create a script to automatically sync wiki changes. + +Create `sync-wiki.sh`: +```bash +#!/bin/bash +# sync-wiki.sh - Sync local wiki with Gitea + +WIKI_REPO="git@git.appmodel.nl:Tour/viewer.wiki.git" +WIKI_DIR="wiki" +TEMP_WIKI="/tmp/gitea-wiki-$$" + +# Clone Gitea wiki +git clone "$WIKI_REPO" "$TEMP_WIKI" + +# Copy local changes to temp wiki +cp -r "$WIKI_DIR"/*.md "$TEMP_WIKI/" + +# Commit and push +cd "$TEMP_WIKI" +git add . +if git diff-staged --quiet; then + echo "No changes to sync" +else + git commit -m "Sync wiki from main repository" + git push origin master + echo "Wiki synced successfully" +fi + +# Cleanup +cd - +rm -rf "$TEMP_WIKI" +``` + +Usage: +```bash +chmod +x sync-wiki.sh +./sync-wiki.sh +``` + +## Recommended Workflow + +For your deployment pipeline, I recommend **Option 1** with a manual push: + +### Step-by-Step Setup + +1. **Create wiki repository on Gitea** (if not exists): + - Go to: https://git.appmodel.nl/Tour/viewer + - Click "Wiki" tab + - Create first page (triggers wiki repo creation) + +2. **Push local wiki content**: +```bash +# Clone wiki repository +cd /c/vibe +git clone git@git.appmodel.nl:Tour/viewer.wiki.git + +# Copy your wiki files +cd viewer.wiki +cp ../viewer/wiki/*.md . + +# Verify files +ls -la + +# Commit and push +git add . +git commit -m "Initialize wiki documentation + +Added: +- Home.md: Main documentation page +- Quick-Start.md: Quick start guide +" +git push origin master +``` + +3. **Verify on Gitea**: + - Navigate to: https://git.appmodel.nl/Tour/viewer/wiki + - You should see your wiki pages + +4. **Keep local wiki in main repo** (optional): + - Keep `wiki/` folder in main repo as "source of truth" + - When updating wiki, manually sync to Gitea wiki repo + - Or set up automated sync via CI/CD + +## Wiki Page Naming + +Gitea wikis follow these conventions: + +| Filename | URL | Purpose | +|----------|-----|---------| +| `Home.md` | `/wiki/` or `/wiki/Home` | Main wiki page | +| `Quick-Start.md` | `/wiki/Quick-Start` | Quick start guide | +| `API-Reference.md` | `/wiki/API-Reference` | API documentation | + +**Rules**: +- Use PascalCase or kebab-case for multi-word pages +- Spaces in filenames become hyphens in URLs +- First page is always `Home.md` + +## Linking Between Pages + +In your markdown files: + +```markdown + +[Quick Start Guide](Quick-Start) + + +[Get Started](Quick-Start) + + +[View Code](../) + + +[Dockerfile](../src/master/Dockerfile) + + +[Diagrams Library](https://diagrams.mingrammer.com/) +``` + +## Automation with Post-Receive Hook + +To automatically sync wiki on deployment, add to Gitea post-receive hook: + +```bash +#!/usr/bin/env bash + +# Deploy main app +/usr/local/bin/app-deploy viewer + +# Sync wiki (if wiki/ folder exists in repo) +if [ -d "/opt/apps/viewer/wiki" ]; then + cd /tmp + git clone git@git.appmodel.nl:Tour/viewer.wiki.git gitea-wiki-$$ + cp /opt/apps/viewer/wiki/*.md /tmp/gitea-wiki-$$/ + cd /tmp/gitea-wiki-$$ + git add . + if ! git diff --staged --quiet; then + git commit -m "Auto-sync from main repository" + git push origin master + fi + cd /tmp + rm -rf /tmp/gitea-wiki-$$ +fi +``` + +## Current Setup Instructions + +Based on your current setup, here's what to do: + +```bash +# 1. Navigate to your viewer project +cd /c/vibe/viewer + +# 2. Clone the wiki repository (for viewer, adjust as needed) +cd .. +git clone git@git.appmodel.nl:Tour/viewer.wiki.git + +# 3. Copy wiki files +cp viewer/wiki/*.md viewer.wiki/ + +# 4. Push to Gitea +cd viewer.wiki +git add . +git commit -m "Initialize wiki with deployment documentation" +git push origin master + +# 5. Verify +# Open: https://git.appmodel.nl/Tour/viewer/wiki +``` + +## Maintaining Both Wikis + +If you want to keep both local and Gitea wikis in sync: + +### Manual Sync (Simple) +```bash +# After editing local wiki files +cd /c/vibe/viewer/wiki +cp *.md /c/vibe/viewer.wiki/ +cd /c/vibe/viewer.wiki +git add . +git commit -m "Update documentation" +git push +``` + +### Automated Sync (Add to package.json or Makefile) +```json +{ + "scripts": { + "sync-wiki": "cp wiki/*.md ../viewer.wiki/ && cd ../viewer.wiki && git add . && git commit -m 'Sync wiki' && git push" + } +} +``` + +Or create a `Makefile`: +```makefile +.PHONY: sync-wiki + +sync-wiki: + @echo "Syncing wiki to Gitea..." + @cp wiki/*.md ../viewer.wiki/ + @cd ../viewer.wiki && git add . && git commit -m "Sync wiki from main repo" && git push + @echo "Wiki synced successfully!" +``` + +Usage: `make sync-wiki` + +## Best Practices + +1. **Single Source of Truth**: Choose one location (local or Gitea) as authoritative +2. **Version Control**: Keep wiki changes in Git history +3. **Review Process**: Use pull requests for significant documentation changes +4. **Link Checking**: Regularly verify internal links work +5. **Consistent Formatting**: Use markdown linting tools +6. **Table of Contents**: Keep `Home.md` as navigation hub + +## Troubleshooting + +### Wiki Repository Doesn't Exist + +Create the wiki first in Gitea UI: +1. Go to repository: https://git.appmodel.nl/Tour/viewer +2. Click "Wiki" tab +3. Click "Create New Page" +4. Create a dummy page (will be overwritten) +5. Now you can clone `viewer.wiki.git` + +### Permission Denied + +Ensure your SSH key is added to Gitea: +```bash +ssh -T git@git.appmodel.nl +``` + +### Merge Conflicts + +If both local and Gitea wiki are edited: +```bash +cd viewer.wiki +git pull origin master +# Resolve conflicts manually +git add . +git commit -m "Merge local and remote changes" +git push +``` + +## Summary + +**Recommended approach for your setup**: + +1. Keep `wiki/` in main repository for version control +2. Manually push to Gitea wiki repository when documentation is ready +3. Optionally add automated sync in post-receive hook +4. Edit wiki locally, sync to Gitea for visibility + +This gives you the best of both worlds: +- ✅ Documentation versioned with code +- ✅ Visible wiki in Gitea UI +- ✅ Easy to maintain and update +- ✅ Works with existing deployment pipeline diff --git a/wiki/build-pipeline.md b/wiki/build-pipeline.md new file mode 100644 index 0000000..ff2408d --- /dev/null +++ b/wiki/build-pipeline.md @@ -0,0 +1,31 @@ +# Appmodel Home Lab – Build & Deploy Pipeline + +Dit document beschrijft hoe een nieuwe applicatie in het home lab wordt aangemaakt en hoe de build- & deploy-pipeline werkt. + +## Overzicht + +- **Broncode**: Gitea (`https://git.appmodel.nl`) +- **Build & runtime**: Docker containers op netwerk `traefik_net` +- **Routing & TLS**: Traefik (`https://traefik.appmodel.nl`) +- **Automatische deploy**: Gitea `post-receive` hooks → `app-deploy ` + +## Pipeline in één diagram + +```mermaid +flowchart LR + Dev[💻 Dev machine\nVS Code / Git] -->|git push| Gitea[📚 Gitea\nTour/] + + subgraph Server[🏠 Home lab server\n192.168.1.159] + Gitea --> Hook[🔔 post-receive hook\n/app-deploy ] + + Hook --> Deploy[⚙️ app-deploy \n/git pull + docker compose up -d --build ] + + subgraph Docker[🐳 Docker / traefik_net] + AppC[🧱 App container\n.appmodel.nl] + Traefik[🚦 Traefik\nReverse Proxy] + end + end + + Deploy --> AppC + Traefik --> AppC + Client[🌐 Browser / API client] -->|https://.appmodel.nl| Traefik diff --git a/wiki/readme-pipe.md b/wiki/readme-pipe.md new file mode 100644 index 0000000..51bff8c --- /dev/null +++ b/wiki/readme-pipe.md @@ -0,0 +1,141 @@ +1. Nieuwe app aanmaken +1.1 Gitea repo + +Log in op Gitea: https://git.appmodel.nl + +Maak een nieuwe repo aan, bijvoorbeeld: + +Owner: Tour + +Name: viewer + +1.2 Skeleton genereren op de server + +Op de server: + +apps-create viewer static-fe + + +Dit doet: + +/opt/apps/viewer klaarzetten (proberen te clonen uit git@git.appmodel.nl:Tour/viewer.git) + +een multi-stage Dockerfile voor een Node-based static frontend aanmaken + +~/infra/viewer/docker-compose.yml aanmaken met: + +service viewer + +koppeling aan traefik_net + +Traefik route: https://viewer.appmodel.nl + +2. Develop & build + +Op je dev machine: + +git clone git@git.appmodel.nl:Tour/viewer.git +cd viewer +# bouw je app zoals normaal +npm install +npm run build # output in dist/ +git add . +git commit -m "First version" +git push + + +De Dockerfile verwacht een dist/ map met static files. + +3. Deploy pipeline +3.1 app-deploy + +Op de server verzorgt app-deploy de generieke deploy: + +app-deploy viewer + + +Doet: + +cd /opt/apps/viewer + +git pull --ff-only + +cd /home/tour/infra/viewer + +docker compose up -d --build viewer + +Logs komen in: + +/var/log/app-deploy-viewer.log + +3.2 Automatisch deployen via Gitea hook + +In Gitea (repo Tour/viewer): + +Ga naar Instellingen → Git Hooks + +Kies post-receive + +Gebruik: + +#!/usr/bin/env bash +/usr/local/bin/app-deploy viewer + + +Vanaf nu: + +Elke git push naar Tour/viewer triggert automatisch een deploy. + +4. Traefik & DNS + +Traefik draait in de traefik stack op dezelfde server en beheert: + +git.appmodel.nl + +auction.appmodel.nl + +aupi.appmodel.nl + +… (nieuwe apps via labels) + +Nieuwe app viewer krijgt via apps-create al de juiste labels: + +labels: + - "traefik.enable=true" + - "traefik.http.routers.viewer.rule=Host(`viewer.appmodel.nl`)" + - "traefik.http.routers.viewer.entrypoints=websecure" + - "traefik.http.routers.viewer.tls=true" + - "traefik.http.routers.viewer.tls.certresolver=letsencrypt" + - "traefik.http.services.viewer.loadbalancer.server.port=80" + + +Je moet alleen in DNS nog een record maken: + +viewer.appmodel.nl → publieke IP van de server + +Traefik + Let’s Encrypt regelen het certificaat automatisch. + +5. Nieuwe app types (toekomst) + +Het apps-create script ondersteunt nu: + +static-fe – Node build → Nginx → static frontend + +Later kun je extra types toevoegen, bijvoorbeeld: + +api-py – Python (Flask/FastAPI) API + +worker-py – background worker / crawler + +Door per type een eigen Dockerfile-sjabloon en standaard docker-compose.yml te genereren, wordt een nieuw project neerzetten: + +apps-create stats api-py +apps-create crawler worker-py + + +en blijft de pipeline (app-deploy ) identiek. + +Je kunt nu: + +apps-create viewer static-fe +app-deploy viewer \ No newline at end of file