wiki
192
DEPLOYMENT_GUIDE.md
Normal file
192
DEPLOYMENT_GUIDE.md
Normal file
@@ -0,0 +1,192 @@
|
||||
# Deployment Guide - App Pipeline
|
||||
|
||||
Complete guide for deploying applications using the custom deployment pipeline with Gitea, Docker, and Traefik.
|
||||
|
||||
## 1. Create New Application
|
||||
|
||||
### 1.1 Gitea Repository
|
||||
|
||||
1. Log in to Gitea: https://git.appmodel.nl
|
||||
2. Create a new repository:
|
||||
- **Owner**: Tour
|
||||
- **Name**: `viewer` (or your app name)
|
||||
|
||||
### 1.2 Generate Skeleton on Server
|
||||
|
||||
On the server, run:
|
||||
|
||||
```bash
|
||||
apps-create viewer static-fe
|
||||
```
|
||||
|
||||
This command:
|
||||
- Sets up `/opt/apps/viewer` (attempts to clone from `git@git.appmodel.nl:Tour/viewer.git`)
|
||||
- Generates a multi-stage Dockerfile for a Node-based static frontend
|
||||
- Creates `~/infra/viewer/docker-compose.yml` with:
|
||||
- Service `viewer`
|
||||
- Connection to `traefik_net`
|
||||
- Traefik route: `https://viewer.appmodel.nl`
|
||||
|
||||
## 2. Development & Build
|
||||
|
||||
On your development machine:
|
||||
|
||||
```bash
|
||||
# Clone the repository
|
||||
git clone git@git.appmodel.nl:Tour/viewer.git
|
||||
cd viewer
|
||||
|
||||
# Build your app as usual
|
||||
npm install
|
||||
npm run build # output goes to dist/
|
||||
|
||||
# Commit and push
|
||||
git add .
|
||||
git commit -m "First version"
|
||||
git push
|
||||
```
|
||||
|
||||
**Note**: The Dockerfile expects a `dist/` directory containing static files.
|
||||
|
||||
## 3. Deployment Pipeline
|
||||
|
||||
### 3.1 Manual Deploy with `app-deploy`
|
||||
|
||||
On the server, use the generic deploy command:
|
||||
|
||||
```bash
|
||||
app-deploy viewer
|
||||
```
|
||||
|
||||
This command performs:
|
||||
1. `cd /opt/apps/viewer`
|
||||
2. `git pull --ff-only`
|
||||
3. `cd /home/tour/infra/viewer`
|
||||
4. `docker compose up -d --build viewer`
|
||||
|
||||
Logs are saved to: `/var/log/app-deploy-viewer.log`
|
||||
|
||||
### 3.2 Automatic Deployment via Gitea Hook
|
||||
|
||||
Set up automatic deployment on every push:
|
||||
|
||||
1. In Gitea, go to repository `Tour/viewer`
|
||||
2. Navigate to **Settings → Git Hooks**
|
||||
3. Select **post-receive**
|
||||
4. Add the following script:
|
||||
|
||||
```bash
|
||||
#!/usr/bin/env bash
|
||||
/usr/local/bin/app-deploy viewer
|
||||
```
|
||||
|
||||
**Result**: Every `git push` to `Tour/viewer` automatically triggers a deployment.
|
||||
|
||||
## 4. Traefik & DNS Configuration
|
||||
|
||||
### Traefik Setup
|
||||
|
||||
Traefik runs in the traefik stack on the same server and manages:
|
||||
- `git.appmodel.nl`
|
||||
- `auction.appmodel.nl`
|
||||
- `aupi.appmodel.nl`
|
||||
- ... (new apps via labels)
|
||||
|
||||
### Auto-Generated Labels
|
||||
|
||||
The `apps-create` command automatically adds the correct Traefik labels:
|
||||
|
||||
```yaml
|
||||
labels:
|
||||
- "traefik.enable=true"
|
||||
- "traefik.http.routers.viewer.rule=Host(`viewer.appmodel.nl`)"
|
||||
- "traefik.http.routers.viewer.entrypoints=websecure"
|
||||
- "traefik.http.routers.viewer.tls=true"
|
||||
- "traefik.http.routers.viewer.tls.certresolver=letsencrypt"
|
||||
- "traefik.http.services.viewer.loadbalancer.server.port=80"
|
||||
```
|
||||
|
||||
### DNS Configuration
|
||||
|
||||
Add a DNS record pointing to your server's public IP:
|
||||
|
||||
```
|
||||
viewer.appmodel.nl → <server-public-ip>
|
||||
```
|
||||
|
||||
Traefik + Let's Encrypt will automatically handle SSL certificate generation.
|
||||
|
||||
## 5. Future Application Types
|
||||
|
||||
The `apps-create` script currently supports:
|
||||
|
||||
- **`static-fe`** – Node build → Nginx → static frontend
|
||||
|
||||
### Planned Types
|
||||
|
||||
Future app types can be added:
|
||||
|
||||
```bash
|
||||
# Python API
|
||||
apps-create stats api-py
|
||||
|
||||
# Background worker/crawler
|
||||
apps-create crawler worker-py
|
||||
```
|
||||
|
||||
Each type will have:
|
||||
- Custom Dockerfile template
|
||||
- Standard `docker-compose.yml` configuration
|
||||
- Same deployment pipeline: `app-deploy <name>`
|
||||
|
||||
## Quick Reference
|
||||
|
||||
### Create and Deploy New App
|
||||
|
||||
```bash
|
||||
# On server: create skeleton
|
||||
apps-create viewer static-fe
|
||||
|
||||
# On dev machine: develop and push
|
||||
git clone git@git.appmodel.nl:Tour/viewer.git
|
||||
cd viewer
|
||||
npm install
|
||||
npm run build
|
||||
git add .
|
||||
git commit -m "Initial commit"
|
||||
git push
|
||||
|
||||
# On server: deploy (or auto-deploys via hook)
|
||||
app-deploy viewer
|
||||
```
|
||||
|
||||
### Existing Infrastructure
|
||||
|
||||
| Service | URL | Description |
|
||||
|---------|-----|-------------|
|
||||
| Gitea | https://git.appmodel.nl | Git repository hosting |
|
||||
| Auction | https://auction.appmodel.nl | Auction frontend |
|
||||
| Aupi API | https://aupi.appmodel.nl | Auction backend API |
|
||||
| Traefik | - | Reverse proxy & SSL |
|
||||
|
||||
### Log Locations
|
||||
|
||||
- Deployment logs: `/var/log/app-deploy-<app-name>.log`
|
||||
- Docker logs: `docker compose logs -f <service-name>`
|
||||
|
||||
### Useful Commands
|
||||
|
||||
```bash
|
||||
# View deployment logs
|
||||
tail -f /var/log/app-deploy-viewer.log
|
||||
|
||||
# View container logs
|
||||
cd ~/infra/viewer
|
||||
docker compose logs -f viewer
|
||||
|
||||
# Restart service
|
||||
docker compose restart viewer
|
||||
|
||||
# Rebuild without cache
|
||||
docker compose up -d --build --no-cache viewer
|
||||
```
|
||||
502
DEPLOY_SERVER_SETUP.md
Normal file
502
DEPLOY_SERVER_SETUP.md
Normal file
@@ -0,0 +1,502 @@
|
||||
# Deploy Server Setup - Docker Build Pipeline
|
||||
|
||||
Complete guide for setting up and using the Docker-based deployment pipeline on the build server.
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ Deployment Flow │
|
||||
├─────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ Dev Machine Gitea Server │
|
||||
│ ┌──────────┐ ┌──────┐ ┌─────────┐ │
|
||||
│ │ git │ │ Repo │ │ /opt/ │ │
|
||||
│ │ commit ├─────push────▶│Tour/ ├──hook────▶ │ apps/ │ │
|
||||
│ │ push │ │<app> │ │ <app>/ │ │
|
||||
│ └──────────┘ └──────┘ └────┬────┘ │
|
||||
│ │ │
|
||||
│ git pull │
|
||||
│ │ │
|
||||
│ ┌───────────▼────────┐ │
|
||||
│ │ app-deploy <app> │ │
|
||||
│ │ (as user: git) │ │
|
||||
│ └───────────┬────────┘ │
|
||||
│ │ │
|
||||
│ docker compose up -d │
|
||||
│ --build │
|
||||
│ │ │
|
||||
│ ┌───────────▼────────┐ │
|
||||
│ │ ~/infra/<app>/ │ │
|
||||
│ │ docker-compose.yml │ │
|
||||
│ └───────────┬────────┘ │
|
||||
│ │ │
|
||||
│ rebuild │
|
||||
│ │ │
|
||||
│ ┌───────────▼────────┐ │
|
||||
│ │ 🐳 Container │ │
|
||||
│ │ <app>:latest │ │
|
||||
│ └───────────┬────────┘ │
|
||||
│ │ │
|
||||
│ traefik_net │
|
||||
│ │ │
|
||||
│ ┌───────────▼────────┐ │
|
||||
│ │ 🚦 Traefik │ │
|
||||
│ │ Reverse Proxy │ │
|
||||
│ └───────────┬────────┘ │
|
||||
│ │ │
|
||||
│ https://<app>.appmodel.nl│
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Build Pipeline Workflow
|
||||
|
||||
### Key Principles
|
||||
|
||||
1. **Source of Truth**: Gitea repository `Tour/<app>` is the authoritative source
|
||||
2. **Build Server**: Server is only for building and deployment, not development
|
||||
3. **Automated Deployment**: Git push triggers automatic rebuild and deployment
|
||||
4. **Isolation**: Each app runs in its own Docker container
|
||||
5. **Shared Networking**: All apps connect via `traefik_net` for reverse proxy
|
||||
|
||||
### Pipeline Steps
|
||||
|
||||
#### 1. Git Push (Developer)
|
||||
```bash
|
||||
git push origin main
|
||||
```
|
||||
|
||||
#### 2. Post-Receive Hook (Gitea)
|
||||
Located in: **Settings → Git Hooks → post-receive**
|
||||
|
||||
```bash
|
||||
#!/usr/bin/env bash
|
||||
/usr/local/bin/app-deploy diagram
|
||||
```
|
||||
|
||||
#### 3. App Deploy Script (Server)
|
||||
Runs as Linux user `git`:
|
||||
|
||||
```bash
|
||||
app-deploy diagram
|
||||
```
|
||||
|
||||
This script performs:
|
||||
1. `cd /opt/apps/diagram`
|
||||
2. `git pull --ff-only` from `git@git.appmodel.nl:Tour/diagram.git`
|
||||
3. `cd /home/tour/infra/diagram`
|
||||
4. `docker compose up -d --build diagram`
|
||||
|
||||
#### 4. Docker Build (Server)
|
||||
Multi-stage build process:
|
||||
- **Stage 1**: Build diagrams using Python + Graphviz
|
||||
- **Stage 2**: Serve with Nginx
|
||||
|
||||
#### 5. Traefik Routing (Server)
|
||||
Traefik automatically detects the container via labels and publishes:
|
||||
```
|
||||
https://diagram.appmodel.nl
|
||||
```
|
||||
|
||||
#### 6. DNS Resolution (AdGuard)
|
||||
AdGuard resolves `diagram.appmodel.nl` → Build server IP
|
||||
|
||||
Result: Both internal and external clients use the same URL.
|
||||
|
||||
## Server Directory Structure
|
||||
|
||||
```
|
||||
/opt/apps/
|
||||
└── diagram/ # Git repository (pulled from Gitea)
|
||||
├── Dockerfile
|
||||
├── docker-compose.yml
|
||||
├── requirements.txt
|
||||
├── lan_architecture.py
|
||||
├── main.py
|
||||
└── public/
|
||||
└── index.html
|
||||
|
||||
/home/tour/infra/
|
||||
└── diagram/
|
||||
├── docker-compose.yml # Docker Compose config (may differ from repo)
|
||||
└── logs/ # Nginx logs (optional)
|
||||
|
||||
/var/log/
|
||||
└── app-deploy-diagram.log # Deployment logs
|
||||
```
|
||||
|
||||
## Setup Instructions
|
||||
|
||||
### 1. Create Repository in Gitea
|
||||
|
||||
1. Navigate to: https://git.appmodel.nl
|
||||
2. Create new repository:
|
||||
- **Owner**: `Tour`
|
||||
- **Name**: `diagram`
|
||||
- **Visibility**: Private
|
||||
|
||||
### 2. Initialize Local Repository
|
||||
|
||||
```bash
|
||||
# On dev machine
|
||||
git clone git@git.appmodel.nl:Tour/diagram.git
|
||||
cd diagram
|
||||
|
||||
# Add project files
|
||||
git add Dockerfile docker-compose.yml requirements.txt *.py public/
|
||||
git commit -m "Initial commit: diagram viewer"
|
||||
git push origin main
|
||||
```
|
||||
|
||||
### 3. Setup Server Directories
|
||||
|
||||
```bash
|
||||
# On server as appropriate user
|
||||
sudo -u git mkdir -p /opt/apps/diagram
|
||||
sudo -u git git clone git@git.appmodel.nl:Tour/diagram.git /opt/apps/diagram
|
||||
|
||||
mkdir -p /home/tour/infra/diagram
|
||||
cp /opt/apps/diagram/docker-compose.yml /home/tour/infra/diagram/
|
||||
```
|
||||
|
||||
### 4. Configure Gitea Post-Receive Hook
|
||||
|
||||
1. Go to: https://git.appmodel.nl/Tour/diagram
|
||||
2. Navigate to: **Settings → Git Hooks**
|
||||
3. Select: **post-receive**
|
||||
4. Add script:
|
||||
|
||||
```bash
|
||||
#!/usr/bin/env bash
|
||||
/usr/local/bin/app-deploy diagram
|
||||
```
|
||||
|
||||
5. Save and test
|
||||
|
||||
### 5. Configure DNS
|
||||
|
||||
Add DNS record in AdGuard:
|
||||
```
|
||||
diagram.appmodel.nl → <server-ip>
|
||||
```
|
||||
|
||||
### 6. Test Deployment
|
||||
|
||||
```bash
|
||||
# Manual test on server
|
||||
app-deploy diagram
|
||||
|
||||
# Verify container is running
|
||||
cd /home/tour/infra/diagram
|
||||
docker compose ps
|
||||
|
||||
# Check logs
|
||||
docker compose logs -f diagram
|
||||
|
||||
# Test endpoint
|
||||
curl -I https://diagram.appmodel.nl
|
||||
```
|
||||
|
||||
## Docker Configuration
|
||||
|
||||
### Dockerfile Explained
|
||||
|
||||
```dockerfile
|
||||
# Stage 1: Build diagrams
|
||||
FROM python:3.11-slim AS builder
|
||||
RUN apt-get update && apt-get install -y graphviz
|
||||
WORKDIR /app
|
||||
COPY requirements.txt .
|
||||
RUN pip install -r requirements.txt
|
||||
COPY *.py .
|
||||
RUN python lan_architecture.py && python main.py
|
||||
|
||||
# Stage 2: Serve with Nginx
|
||||
FROM nginx:alpine
|
||||
COPY --from=builder /app/*.png /usr/share/nginx/html/
|
||||
COPY public/ /usr/share/nginx/html/
|
||||
EXPOSE 80
|
||||
CMD ["nginx", "-g", "daemon off;"]
|
||||
```
|
||||
|
||||
**Benefits**:
|
||||
- Small final image (only Nginx + static files)
|
||||
- Build tools not included in production image
|
||||
- Automatic diagram generation on build
|
||||
|
||||
### docker-compose.yml Configuration
|
||||
|
||||
```yaml
|
||||
version: '3.8'
|
||||
|
||||
services:
|
||||
diagram:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile
|
||||
container_name: diagram-viewer
|
||||
restart: unless-stopped
|
||||
networks:
|
||||
- traefik_net
|
||||
labels:
|
||||
- "traefik.enable=true"
|
||||
- "traefik.http.routers.diagram.rule=Host(`diagram.appmodel.nl`)"
|
||||
- "traefik.http.routers.diagram.entrypoints=websecure"
|
||||
- "traefik.http.routers.diagram.tls=true"
|
||||
- "traefik.http.routers.diagram.tls.certresolver=letsencrypt"
|
||||
- "traefik.http.services.diagram.loadbalancer.server.port=80"
|
||||
|
||||
networks:
|
||||
traefik_net:
|
||||
external: true
|
||||
```
|
||||
|
||||
**Key Points**:
|
||||
- Connects to external `traefik_net` network
|
||||
- Traefik labels configure automatic HTTPS with Let's Encrypt
|
||||
- Container restarts automatically unless stopped manually
|
||||
|
||||
## Development Workflow
|
||||
|
||||
### Local Development
|
||||
|
||||
```bash
|
||||
# Edit files locally
|
||||
vim lan_architecture.py
|
||||
vim public/index.html
|
||||
|
||||
# Test locally (optional)
|
||||
python -m venv .venv
|
||||
source .venv/bin/activate # or .venv\Scripts\activate on Windows
|
||||
pip install -r requirements.txt
|
||||
python lan_architecture.py
|
||||
|
||||
# Commit and push
|
||||
git add .
|
||||
git commit -m "Update diagram architecture"
|
||||
git push
|
||||
```
|
||||
|
||||
### Automatic Deployment
|
||||
|
||||
After `git push`, the server automatically:
|
||||
1. Receives post-receive hook trigger
|
||||
2. Pulls latest code
|
||||
3. Rebuilds Docker image
|
||||
4. Restarts container
|
||||
5. Updates live site
|
||||
|
||||
**Timeline**: Usually completes in 30-60 seconds.
|
||||
|
||||
### Manual Deployment
|
||||
|
||||
If needed, manually trigger deployment:
|
||||
|
||||
```bash
|
||||
# On server
|
||||
app-deploy diagram
|
||||
|
||||
# Or manually
|
||||
cd /opt/apps/diagram
|
||||
git pull
|
||||
cd /home/tour/infra/diagram
|
||||
docker compose up -d --build diagram
|
||||
```
|
||||
|
||||
## Monitoring & Troubleshooting
|
||||
|
||||
### View Logs
|
||||
|
||||
```bash
|
||||
# Deployment logs
|
||||
tail -f /var/log/app-deploy-diagram.log
|
||||
|
||||
# Container logs
|
||||
cd /home/tour/infra/diagram
|
||||
docker compose logs -f diagram
|
||||
|
||||
# Nginx access logs (if volume mounted)
|
||||
tail -f /home/tour/infra/diagram/logs/access.log
|
||||
```
|
||||
|
||||
### Check Container Status
|
||||
|
||||
```bash
|
||||
cd /home/tour/infra/diagram
|
||||
|
||||
# List running containers
|
||||
docker compose ps
|
||||
|
||||
# Inspect container
|
||||
docker compose exec diagram sh
|
||||
|
||||
# Test nginx config
|
||||
docker compose exec diagram nginx -t
|
||||
```
|
||||
|
||||
### Debug Build Issues
|
||||
|
||||
```bash
|
||||
# Rebuild without cache
|
||||
docker compose build --no-cache diagram
|
||||
|
||||
# View build output
|
||||
docker compose up --build diagram
|
||||
|
||||
# Check image layers
|
||||
docker image history diagram-viewer
|
||||
```
|
||||
|
||||
### Common Issues
|
||||
|
||||
#### 1. Build Fails - Missing Dependencies
|
||||
|
||||
**Symptom**: Build fails at pip install step
|
||||
|
||||
**Solution**: Verify `requirements.txt` is correct
|
||||
```bash
|
||||
cd /opt/apps/diagram
|
||||
cat requirements.txt
|
||||
```
|
||||
|
||||
#### 2. Nginx 404 Error
|
||||
|
||||
**Symptom**: Container runs but shows 404
|
||||
|
||||
**Solution**: Check if files were copied correctly
|
||||
```bash
|
||||
docker compose exec diagram ls -la /usr/share/nginx/html/
|
||||
```
|
||||
|
||||
#### 3. Traefik Not Routing
|
||||
|
||||
**Symptom**: Container runs but domain not accessible
|
||||
|
||||
**Solution**:
|
||||
- Verify container is on `traefik_net`: `docker network inspect traefik_net`
|
||||
- Check Traefik labels: `docker inspect diagram-viewer | grep traefik`
|
||||
- Verify DNS: `nslookup diagram.appmodel.nl`
|
||||
|
||||
#### 4. Git Pull Fails
|
||||
|
||||
**Symptom**: app-deploy fails at git pull
|
||||
|
||||
**Solution**:
|
||||
- Check SSH keys for git user: `sudo -u git ssh -T git@git.appmodel.nl`
|
||||
- Verify remote: `cd /opt/apps/diagram && git remote -v`
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### User Permissions
|
||||
|
||||
- **git user**: Owns `/opt/apps/*` directories, runs app-deploy
|
||||
- **tour user**: Owns `/home/tour/infra/*`, runs Docker Compose
|
||||
- Principle: Separate concerns between git operations and container management
|
||||
|
||||
### Network Isolation
|
||||
|
||||
- Containers only exposed via Traefik
|
||||
- No direct port exposure to host
|
||||
- `traefik_net` provides isolation between services
|
||||
|
||||
### Secrets Management
|
||||
|
||||
For apps requiring secrets:
|
||||
|
||||
```yaml
|
||||
# In docker-compose.yml
|
||||
services:
|
||||
diagram:
|
||||
environment:
|
||||
- SECRET_KEY=${SECRET_KEY}
|
||||
env_file:
|
||||
- .env # Not committed to git
|
||||
```
|
||||
|
||||
Create `.env` file on server:
|
||||
```bash
|
||||
echo "SECRET_KEY=your-secret-here" > /home/tour/infra/diagram/.env
|
||||
chmod 600 /home/tour/infra/diagram/.env
|
||||
```
|
||||
|
||||
## Scaling & Future Improvements
|
||||
|
||||
### Multi-Container Apps
|
||||
|
||||
For apps with multiple services:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
frontend:
|
||||
build: ./frontend
|
||||
labels:
|
||||
- "traefik.enable=true"
|
||||
- "traefik.http.routers.app-fe.rule=Host(`app.appmodel.nl`)"
|
||||
|
||||
backend:
|
||||
build: ./backend
|
||||
labels:
|
||||
- "traefik.enable=true"
|
||||
- "traefik.http.routers.app-be.rule=Host(`api.app.appmodel.nl`)"
|
||||
|
||||
database:
|
||||
image: postgres:15
|
||||
volumes:
|
||||
- pgdata:/var/lib/postgresql/data
|
||||
|
||||
volumes:
|
||||
pgdata:
|
||||
```
|
||||
|
||||
### CI/CD Integration
|
||||
|
||||
Consider adding:
|
||||
- Automated tests before deployment
|
||||
- Rollback mechanism
|
||||
- Blue-green deployments
|
||||
- Health checks before switching traffic
|
||||
|
||||
### Monitoring
|
||||
|
||||
Add monitoring stack:
|
||||
- Prometheus for metrics
|
||||
- Grafana for dashboards
|
||||
- Loki for log aggregation
|
||||
- Alertmanager for notifications
|
||||
|
||||
## Quick Reference
|
||||
|
||||
### Key Paths
|
||||
|
||||
| Path | Purpose |
|
||||
|------|---------|
|
||||
| `/opt/apps/<app>/` | Git repository checkout |
|
||||
| `/home/tour/infra/<app>/` | Docker Compose directory |
|
||||
| `/var/log/app-deploy-<app>.log` | Deployment logs |
|
||||
|
||||
### Key Commands
|
||||
|
||||
| Command | Purpose |
|
||||
|---------|---------|
|
||||
| `app-deploy <app>` | Deploy/redeploy application |
|
||||
| `docker compose ps` | List running containers |
|
||||
| `docker compose logs -f <app>` | Follow container logs |
|
||||
| `docker compose restart <app>` | Restart service |
|
||||
| `docker compose build --no-cache` | Force rebuild |
|
||||
|
||||
### URLs
|
||||
|
||||
| Service | URL |
|
||||
|---------|-----|
|
||||
| Gitea | https://git.appmodel.nl |
|
||||
| Diagram Viewer | https://diagram.appmodel.nl |
|
||||
| Traefik Dashboard | https://traefik.appmodel.nl/dashboard/ |
|
||||
|
||||
## Support
|
||||
|
||||
For issues or questions:
|
||||
1. Check deployment logs: `/var/log/app-deploy-<app>.log`
|
||||
2. Check container logs: `docker compose logs -f`
|
||||
3. Review Gitea webhook history in repository settings
|
||||
4. Test manual deployment: `app-deploy <app>`
|
||||
379
Home.md
379
Home.md
@@ -1 +1,378 @@
|
||||
Welkom op de wiki.
|
||||
# Diagram Viewer Wiki
|
||||
|
||||
Welcome to the **Diagram Viewer** project documentation.
|
||||
|
||||
This project provides a static web interface for viewing and managing network architecture diagrams generated with Python's `diagrams` library.
|
||||
|
||||
## 📚 Contents
|
||||
|
||||
- [Getting Started](#getting-started)
|
||||
- [Development Guide](#development-guide)
|
||||
- [Deployment Guide](#deployment-guide)
|
||||
- [Architecture Overview](#architecture-overview)
|
||||
|
||||
---
|
||||
|
||||
## Getting Started
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Python 3.8+
|
||||
- Graphviz
|
||||
- Docker (for deployment)
|
||||
|
||||
### Quick Start
|
||||
|
||||
```bash
|
||||
# Clone repository
|
||||
git clone git@git.appmodel.nl:Tour/diagram.git
|
||||
cd diagram
|
||||
|
||||
# Create virtual environment
|
||||
python -m venv .venv
|
||||
source .venv/bin/activate # Windows: .venv\Scripts\activate
|
||||
|
||||
# Install dependencies
|
||||
pip install -r requirements.txt
|
||||
|
||||
# Generate diagrams
|
||||
python lan_architecture.py
|
||||
python main.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Development Guide
|
||||
|
||||
### Project Structure
|
||||
|
||||
```
|
||||
diagram/
|
||||
├── Dockerfile # Multi-stage build for production
|
||||
├── docker-compose.yml # Docker Compose configuration
|
||||
├── requirements.txt # Python dependencies
|
||||
├── lan_architecture.py # LAN architecture diagram generator
|
||||
├── main.py # Main diagram generator
|
||||
├── public/
|
||||
│ └── index.html # Live Mermaid diagram editor
|
||||
└── wiki/ # Documentation (this wiki)
|
||||
```
|
||||
|
||||
### Adding New Diagrams
|
||||
|
||||
1. Create a new Python file (e.g., `network_diagram.py`)
|
||||
2. Import required components from `diagrams` library
|
||||
3. Define your architecture using context managers
|
||||
4. Run the script to generate PNG output
|
||||
|
||||
Example:
|
||||
```python
|
||||
from diagrams import Diagram, Cluster
|
||||
from diagrams.onprem.network import Router
|
||||
from diagrams.generic.device import Mobile
|
||||
|
||||
with Diagram("My Network", show=False, filename="my_network"):
|
||||
router = Router("Gateway")
|
||||
device = Mobile("Phone")
|
||||
device >> router
|
||||
```
|
||||
|
||||
### Testing Locally
|
||||
|
||||
```bash
|
||||
# Generate diagrams
|
||||
python lan_architecture.py
|
||||
|
||||
# View output
|
||||
ls -la *.png
|
||||
|
||||
# Test with local web server
|
||||
python -m http.server 8000 --directory public/
|
||||
# Open: http://localhost:8000
|
||||
```
|
||||
|
||||
### Available Icons
|
||||
|
||||
The `diagrams` library provides icons from multiple providers:
|
||||
- **OnPrem**: Network, compute, storage, etc.
|
||||
- **Cloud**: AWS, Azure, GCP services
|
||||
- **Generic**: Network devices, blank nodes
|
||||
- **Custom**: Use your own images
|
||||
|
||||
See: [Diagrams Documentation](https://diagrams.mingrammer.com/docs/nodes/overview)
|
||||
|
||||
---
|
||||
|
||||
## Deployment Guide
|
||||
|
||||
### Overview
|
||||
|
||||
The project uses a fully automated Docker-based deployment pipeline:
|
||||
|
||||
1. **Develop** on local machine
|
||||
2. **Commit & Push** to Gitea
|
||||
3. **Auto-Deploy** via post-receive hook
|
||||
4. **Build** multi-stage Docker image
|
||||
5. **Serve** via Traefik reverse proxy
|
||||
|
||||
### Deployment Steps
|
||||
|
||||
#### 1. Initial Setup
|
||||
|
||||
```bash
|
||||
# On server: Create app structure
|
||||
apps-create diagram static-fe
|
||||
|
||||
# This creates:
|
||||
# - /opt/apps/diagram (git repository)
|
||||
# - /home/tour/infra/diagram/docker-compose.yml
|
||||
```
|
||||
|
||||
#### 2. Configure Gitea Hook
|
||||
|
||||
In repository **Settings → Git Hooks → post-receive**:
|
||||
|
||||
```bash
|
||||
#!/usr/bin/env bash
|
||||
/usr/local/bin/app-deploy diagram
|
||||
```
|
||||
|
||||
#### 3. Deploy
|
||||
|
||||
Simply push to Gitea:
|
||||
|
||||
```bash
|
||||
git push origin main
|
||||
```
|
||||
|
||||
The server automatically:
|
||||
- Pulls latest code
|
||||
- Rebuilds Docker image
|
||||
- Restarts container
|
||||
- Updates live site at https://diagram.appmodel.nl
|
||||
|
||||
### Manual Deployment
|
||||
|
||||
If needed:
|
||||
|
||||
```bash
|
||||
# On server
|
||||
app-deploy diagram
|
||||
|
||||
# Or step-by-step
|
||||
cd /opt/apps/diagram
|
||||
git pull
|
||||
cd /home/tour/infra/diagram
|
||||
docker compose up -d --build diagram
|
||||
```
|
||||
|
||||
### Monitoring
|
||||
|
||||
```bash
|
||||
# View deployment logs
|
||||
tail -f /var/log/app-deploy-diagram.log
|
||||
|
||||
# View container logs
|
||||
cd /home/tour/infra/diagram
|
||||
docker compose logs -f diagram
|
||||
|
||||
# Check container status
|
||||
docker compose ps
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
### Infrastructure
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ Production Architecture │
|
||||
├─────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌──────────────┐ │
|
||||
│ │ Gitea │ (Source of Truth) │
|
||||
│ │ Tour/diagram │ │
|
||||
│ └──────┬───────┘ │
|
||||
│ │ git push │
|
||||
│ ↓ │
|
||||
│ ┌──────────────┐ │
|
||||
│ │ Post-Receive │ (Auto-trigger) │
|
||||
│ │ Hook │ │
|
||||
│ └──────┬───────┘ │
|
||||
│ │ app-deploy diagram │
|
||||
│ ↓ │
|
||||
│ ┌──────────────────────┐ │
|
||||
│ │ /opt/apps/diagram/ │ │
|
||||
│ │ (Git Repository) │ │
|
||||
│ └──────┬───────────────┘ │
|
||||
│ │ git pull │
|
||||
│ ↓ │
|
||||
│ ┌──────────────────────┐ │
|
||||
│ │ Docker Build Process │ │
|
||||
│ │ ┌─────────────────┐ │ │
|
||||
│ │ │ Stage 1: Build │ │ │
|
||||
│ │ │ - Python │ │ │
|
||||
│ │ │ - Graphviz │ │ │
|
||||
│ │ │ - Generate PNGs │ │ │
|
||||
│ │ └─────────────────┘ │ │
|
||||
│ │ ┌─────────────────┐ │ │
|
||||
│ │ │ Stage 2: Serve │ │ │
|
||||
│ │ │ - Nginx │ │ │
|
||||
│ │ │ - Static files │ │ │
|
||||
│ │ └─────────────────┘ │ │
|
||||
│ └──────┬───────────────┘ │
|
||||
│ │ container starts │
|
||||
│ ↓ │
|
||||
│ ┌──────────────────────┐ │
|
||||
│ │ diagram-viewer │ │
|
||||
│ │ Container :80 │ │
|
||||
│ └──────┬───────────────┘ │
|
||||
│ │ traefik_net │
|
||||
│ ↓ │
|
||||
│ ┌──────────────────────┐ │
|
||||
│ │ Traefik │ │
|
||||
│ │ Reverse Proxy │ │
|
||||
│ │ + Let's Encrypt │ │
|
||||
│ └──────┬───────────────┘ │
|
||||
│ │ HTTPS │
|
||||
│ ↓ │
|
||||
│ diagram.appmodel.nl │
|
||||
│ │
|
||||
└─────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Network Topology
|
||||
|
||||
The diagram viewer documents our home lab infrastructure:
|
||||
|
||||
- **LAN 192.168.1.0/24**: Main network
|
||||
- Core server (192.168.1.159): Traefik, Gitea, Dokku, Auction stack, MI50/Ollama
|
||||
- DNS server (192.168.1.163): AdGuard, Artifactory, CI runner
|
||||
- Home Assistant (192.168.1.193)
|
||||
- Atlas worker (192.168.1.100)
|
||||
- IoT devices
|
||||
|
||||
- **Tether 192.168.137.0/24**: Isolated subnet
|
||||
- Hermes worker (192.168.137.239)
|
||||
- Plato worker (192.168.137.163)
|
||||
|
||||
### Services
|
||||
|
||||
| Service | URL | Description |
|
||||
|---------|-----|-------------|
|
||||
| Diagram Viewer | https://diagram.appmodel.nl | This application |
|
||||
| Gitea | https://git.appmodel.nl | Git hosting |
|
||||
| Auction Frontend | https://auction.appmodel.nl | Auction platform |
|
||||
| Aupi API | https://aupi.appmodel.nl | Backend API |
|
||||
| Dokku | https://dokku.lan | PaaS platform |
|
||||
|
||||
---
|
||||
|
||||
## Additional Resources
|
||||
|
||||
### Related Documentation
|
||||
|
||||
- [DEPLOY_SERVER_SETUP.md](../DEPLOY_SERVER_SETUP.md) - Detailed deployment guide
|
||||
- [DEPLOYMENT_GUIDE.md](../DEPLOYMENT_GUIDE.md) - General deployment pipeline
|
||||
- [README.md](../README.md) - Project overview
|
||||
|
||||
### External Links
|
||||
|
||||
- [Diagrams Library](https://diagrams.mingrammer.com/)
|
||||
- [Graphviz](https://graphviz.org/)
|
||||
- [Mermaid.js](https://mermaid.js.org/) (used in live editor)
|
||||
- [Traefik Documentation](https://doc.traefik.io/traefik/)
|
||||
- [Docker Compose](https://docs.docker.com/compose/)
|
||||
|
||||
### Contributing
|
||||
|
||||
To contribute to this project:
|
||||
|
||||
1. Create a feature branch
|
||||
2. Make your changes
|
||||
3. Test locally
|
||||
4. Submit a pull request in Gitea
|
||||
5. Wait for automatic deployment after merge
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
#### Diagrams Not Generating
|
||||
|
||||
**Problem**: Python script fails to generate PNG files
|
||||
|
||||
**Solution**:
|
||||
```bash
|
||||
# Check Graphviz installation
|
||||
dot -V
|
||||
|
||||
# Reinstall if needed
|
||||
# macOS: brew install graphviz
|
||||
# Ubuntu: sudo apt-get install graphviz
|
||||
# Windows: choco install graphviz
|
||||
|
||||
# Verify Python dependencies
|
||||
pip list | grep diagrams
|
||||
pip install --upgrade diagrams
|
||||
```
|
||||
|
||||
#### Container Build Fails
|
||||
|
||||
**Problem**: Docker build fails during deployment
|
||||
|
||||
**Solution**:
|
||||
```bash
|
||||
# Check deployment logs
|
||||
tail -f /var/log/app-deploy-diagram.log
|
||||
|
||||
# Rebuild manually with verbose output
|
||||
cd /home/tour/infra/diagram
|
||||
docker compose build --no-cache --progress=plain diagram
|
||||
```
|
||||
|
||||
#### Site Not Accessible
|
||||
|
||||
**Problem**: Container runs but site not reachable
|
||||
|
||||
**Solution**:
|
||||
1. Check DNS: `nslookup diagram.appmodel.nl`
|
||||
2. Verify Traefik labels: `docker inspect diagram-viewer | grep traefik`
|
||||
3. Check Traefik logs: `docker logs traefik`
|
||||
4. Test container directly: `curl http://localhost:PORT`
|
||||
|
||||
#### Changes Not Reflected
|
||||
|
||||
**Problem**: Pushed changes but site unchanged
|
||||
|
||||
**Solution**:
|
||||
```bash
|
||||
# Force rebuild on server
|
||||
cd /home/tour/infra/diagram
|
||||
docker compose down
|
||||
docker compose up -d --build --force-recreate diagram
|
||||
|
||||
# Clear browser cache
|
||||
# Ctrl+Shift+R (Windows/Linux) or Cmd+Shift+R (macOS)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Contact & Support
|
||||
|
||||
For issues, questions, or suggestions:
|
||||
|
||||
1. Check this wiki
|
||||
2. Review [DEPLOY_SERVER_SETUP.md](../DEPLOY_SERVER_SETUP.md)
|
||||
3. Check container logs
|
||||
4. Contact the infrastructure team
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2025-12-02
|
||||
**Version**: 1.0.0
|
||||
**Maintainer**: Tour
|
||||
239
Quick-Start.md
239
Quick-Start.md
@@ -0,0 +1,239 @@
|
||||
# Quick Start Guide
|
||||
|
||||
Get up and running with the Diagram Viewer in minutes.
|
||||
|
||||
## For Developers
|
||||
|
||||
### 1. Clone Repository
|
||||
|
||||
```bash
|
||||
git clone git@git.appmodel.nl:Tour/diagram.git
|
||||
cd diagram
|
||||
```
|
||||
|
||||
### 2. Setup Environment
|
||||
|
||||
```bash
|
||||
# Create virtual environment
|
||||
python -m venv .venv
|
||||
|
||||
# Activate (choose your platform)
|
||||
source .venv/bin/activate # macOS/Linux
|
||||
.venv\Scripts\activate # Windows CMD
|
||||
source .venv/Scripts/activate # Windows Git Bash
|
||||
```
|
||||
|
||||
### 3. Install Dependencies
|
||||
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
### 4. Generate Diagrams
|
||||
|
||||
```bash
|
||||
# Generate LAN architecture diagram
|
||||
python lan_architecture.py
|
||||
|
||||
# Generate main diagram
|
||||
python main.py
|
||||
|
||||
# View generated files
|
||||
ls -la *.png
|
||||
```
|
||||
|
||||
### 5. Test Locally
|
||||
|
||||
```bash
|
||||
# Start local web server
|
||||
python -m http.server 8000 --directory public/
|
||||
|
||||
# Open browser
|
||||
# Navigate to: http://localhost:8000
|
||||
```
|
||||
|
||||
### 6. Make Changes & Deploy
|
||||
|
||||
```bash
|
||||
# Edit diagram code
|
||||
vim lan_architecture.py
|
||||
|
||||
# Commit changes
|
||||
git add .
|
||||
git commit -m "Update network diagram"
|
||||
|
||||
# Push to deploy (auto-deploys to production)
|
||||
git push origin main
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## For Operators
|
||||
|
||||
### View Live Site
|
||||
|
||||
Simply navigate to:
|
||||
```
|
||||
https://diagram.appmodel.nl
|
||||
```
|
||||
|
||||
### Manual Deployment
|
||||
|
||||
On the server:
|
||||
|
||||
```bash
|
||||
# Trigger deployment
|
||||
app-deploy diagram
|
||||
|
||||
# View logs
|
||||
tail -f /var/log/app-deploy-diagram.log
|
||||
|
||||
# Check status
|
||||
cd /home/tour/infra/diagram
|
||||
docker compose ps
|
||||
```
|
||||
|
||||
### View Logs
|
||||
|
||||
```bash
|
||||
# Deployment logs
|
||||
tail -f /var/log/app-deploy-diagram.log
|
||||
|
||||
# Container logs
|
||||
cd /home/tour/infra/diagram
|
||||
docker compose logs -f diagram
|
||||
|
||||
# Last 100 lines
|
||||
docker compose logs --tail=100 diagram
|
||||
```
|
||||
|
||||
### Restart Service
|
||||
|
||||
```bash
|
||||
cd /home/tour/infra/diagram
|
||||
|
||||
# Restart container
|
||||
docker compose restart diagram
|
||||
|
||||
# Full restart (rebuild)
|
||||
docker compose down
|
||||
docker compose up -d --build diagram
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Common Tasks
|
||||
|
||||
### Add New Diagram
|
||||
|
||||
1. Create new Python file:
|
||||
```python
|
||||
# my_diagram.py
|
||||
from diagrams import Diagram
|
||||
from diagrams.onprem.network import Router
|
||||
|
||||
with Diagram("My Network", show=False, filename="my_network"):
|
||||
Router("Gateway")
|
||||
```
|
||||
|
||||
2. Update Dockerfile if needed (add to RUN command):
|
||||
```dockerfile
|
||||
RUN python lan_architecture.py && \
|
||||
python main.py && \
|
||||
python my_diagram.py
|
||||
```
|
||||
|
||||
3. Commit and push:
|
||||
```bash
|
||||
git add my_diagram.py Dockerfile
|
||||
git commit -m "Add my network diagram"
|
||||
git push
|
||||
```
|
||||
|
||||
### Update Frontend
|
||||
|
||||
1. Edit HTML:
|
||||
```bash
|
||||
vim public/index.html
|
||||
```
|
||||
|
||||
2. Test locally:
|
||||
```bash
|
||||
python -m http.server 8000 --directory public/
|
||||
```
|
||||
|
||||
3. Deploy:
|
||||
```bash
|
||||
git add public/index.html
|
||||
git commit -m "Update frontend"
|
||||
git push
|
||||
```
|
||||
|
||||
### Check Container Health
|
||||
|
||||
```bash
|
||||
# Container status
|
||||
docker compose ps
|
||||
|
||||
# Health check
|
||||
docker inspect diagram-viewer | grep -A 10 Health
|
||||
|
||||
# Test endpoint
|
||||
curl -I https://diagram.appmodel.nl
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Build Fails
|
||||
|
||||
```bash
|
||||
# Check logs
|
||||
tail -f /var/log/app-deploy-diagram.log
|
||||
|
||||
# Rebuild with verbose output
|
||||
cd /home/tour/infra/diagram
|
||||
docker compose build --no-cache --progress=plain diagram
|
||||
```
|
||||
|
||||
### Site Not Loading
|
||||
|
||||
```bash
|
||||
# Check container
|
||||
docker compose ps
|
||||
|
||||
# Check Traefik routing
|
||||
docker logs traefik | grep diagram
|
||||
|
||||
# Test DNS
|
||||
nslookup diagram.appmodel.nl
|
||||
|
||||
# Test directly
|
||||
curl http://localhost:PORT
|
||||
```
|
||||
|
||||
### Changes Not Visible
|
||||
|
||||
```bash
|
||||
# Force rebuild
|
||||
cd /home/tour/infra/diagram
|
||||
docker compose up -d --build --force-recreate diagram
|
||||
|
||||
# Clear browser cache
|
||||
# Ctrl+Shift+R (Windows/Linux)
|
||||
# Cmd+Shift+R (macOS)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
- Read the [Home](Home.md) page for comprehensive documentation
|
||||
- Review [DEPLOY_SERVER_SETUP.md](../DEPLOY_SERVER_SETUP.md) for deployment details
|
||||
- Check [Diagrams Library docs](https://diagrams.mingrammer.com/) for icon options
|
||||
- Explore the [Mermaid editor](https://diagram.appmodel.nl) for live diagram editing
|
||||
|
||||
---
|
||||
|
||||
**Need Help?** Check the [Troubleshooting](Home.md#troubleshooting) section or review deployment logs.
|
||||
Reference in New Issue
Block a user