503 lines
14 KiB
Markdown
503 lines
14 KiB
Markdown
# Deploy Server Setup - Docker Build Pipeline
|
|
|
|
Complete guide for setting up and using the Docker-based deployment pipeline on the build server.
|
|
|
|
## Architecture Overview
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ Deployment Flow │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ │
|
|
│ Dev Machine Gitea Server │
|
|
│ ┌──────────┐ ┌──────┐ ┌─────────┐ │
|
|
│ │ git │ │ Repo │ │ /opt/ │ │
|
|
│ │ commit ├─────push────▶│Tour/ ├──hook────▶ │ apps/ │ │
|
|
│ │ push │ │<app> │ │ <app>/ │ │
|
|
│ └──────────┘ └──────┘ └────┬────┘ │
|
|
│ │ │
|
|
│ git pull │
|
|
│ │ │
|
|
│ ┌───────────▼────────┐ │
|
|
│ │ app-deploy <app> │ │
|
|
│ │ (as user: git) │ │
|
|
│ └───────────┬────────┘ │
|
|
│ │ │
|
|
│ docker compose up -d │
|
|
│ --build │
|
|
│ │ │
|
|
│ ┌───────────▼────────┐ │
|
|
│ │ ~/infra/<app>/ │ │
|
|
│ │ docker-compose.yml │ │
|
|
│ └───────────┬────────┘ │
|
|
│ │ │
|
|
│ rebuild │
|
|
│ │ │
|
|
│ ┌───────────▼────────┐ │
|
|
│ │ 🐳 Container │ │
|
|
│ │ <app>:latest │ │
|
|
│ └───────────┬────────┘ │
|
|
│ │ │
|
|
│ traefik_net │
|
|
│ │ │
|
|
│ ┌───────────▼────────┐ │
|
|
│ │ 🚦 Traefik │ │
|
|
│ │ Reverse Proxy │ │
|
|
│ └───────────┬────────┘ │
|
|
│ │ │
|
|
│ https://<app>.appmodel.nl│
|
|
│ │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
## Build Pipeline Workflow
|
|
|
|
### Key Principles
|
|
|
|
1. **Source of Truth**: Gitea repository `Tour/<app>` is the authoritative source
|
|
2. **Build Server**: Server is only for building and deployment, not development
|
|
3. **Automated Deployment**: Git push triggers automatic rebuild and deployment
|
|
4. **Isolation**: Each app runs in its own Docker container
|
|
5. **Shared Networking**: All apps connect via `traefik_net` for reverse proxy
|
|
|
|
### Pipeline Steps
|
|
|
|
#### 1. Git Push (Developer)
|
|
```bash
|
|
git push origin main
|
|
```
|
|
|
|
#### 2. Post-Receive Hook (Gitea)
|
|
Located in: **Settings → Git Hooks → post-receive**
|
|
|
|
```bash
|
|
#!/usr/bin/env bash
|
|
/usr/local/bin/app-deploy diagram
|
|
```
|
|
|
|
#### 3. App Deploy Script (Server)
|
|
Runs as Linux user `git`:
|
|
|
|
```bash
|
|
app-deploy diagram
|
|
```
|
|
|
|
This script performs:
|
|
1. `cd /opt/apps/diagram`
|
|
2. `git pull --ff-only` from `git@git.appmodel.nl:Tour/diagram.git`
|
|
3. `cd /home/tour/infra/diagram`
|
|
4. `docker compose up -d --build diagram`
|
|
|
|
#### 4. Docker Build (Server)
|
|
Multi-stage build process:
|
|
- **Stage 1**: Build diagrams using Python + Graphviz
|
|
- **Stage 2**: Serve with Nginx
|
|
|
|
#### 5. Traefik Routing (Server)
|
|
Traefik automatically detects the container via labels and publishes:
|
|
```
|
|
https://diagram.appmodel.nl
|
|
```
|
|
|
|
#### 6. DNS Resolution (AdGuard)
|
|
AdGuard resolves `diagram.appmodel.nl` → Build server IP
|
|
|
|
Result: Both internal and external clients use the same URL.
|
|
|
|
## Server Directory Structure
|
|
|
|
```
|
|
/opt/apps/
|
|
└── diagram/ # Git repository (pulled from Gitea)
|
|
├── Dockerfile
|
|
├── docker-compose.yml
|
|
├── requirements.txt
|
|
├── lan_architecture.py
|
|
├── main.py
|
|
└── public/
|
|
└── index.html
|
|
|
|
/home/tour/infra/
|
|
└── diagram/
|
|
├── docker-compose.yml # Docker Compose config (may differ from repo)
|
|
└── logs/ # Nginx logs (optional)
|
|
|
|
/var/log/
|
|
└── app-deploy-diagram.log # Deployment logs
|
|
```
|
|
|
|
## Setup Instructions
|
|
|
|
### 1. Create Repository in Gitea
|
|
|
|
1. Navigate to: https://git.appmodel.nl
|
|
2. Create new repository:
|
|
- **Owner**: `Tour`
|
|
- **Name**: `diagram`
|
|
- **Visibility**: Private
|
|
|
|
### 2. Initialize Local Repository
|
|
|
|
```bash
|
|
# On dev machine
|
|
git clone git@git.appmodel.nl:Tour/diagram.git
|
|
cd diagram
|
|
|
|
# Add project files
|
|
git add Dockerfile docker-compose.yml requirements.txt *.py public/
|
|
git commit -m "Initial commit: diagram viewer"
|
|
git push origin main
|
|
```
|
|
|
|
### 3. Setup Server Directories
|
|
|
|
```bash
|
|
# On server as appropriate user
|
|
sudo -u git mkdir -p /opt/apps/diagram
|
|
sudo -u git git clone git@git.appmodel.nl:Tour/diagram.git /opt/apps/diagram
|
|
|
|
mkdir -p /home/tour/infra/diagram
|
|
cp /opt/apps/diagram/docker-compose.yml /home/tour/infra/diagram/
|
|
```
|
|
|
|
### 4. Configure Gitea Post-Receive Hook
|
|
|
|
1. Go to: https://git.appmodel.nl/Tour/diagram
|
|
2. Navigate to: **Settings → Git Hooks**
|
|
3. Select: **post-receive**
|
|
4. Add script:
|
|
|
|
```bash
|
|
#!/usr/bin/env bash
|
|
/usr/local/bin/app-deploy diagram
|
|
```
|
|
|
|
5. Save and test
|
|
|
|
### 5. Configure DNS
|
|
|
|
Add DNS record in AdGuard:
|
|
```
|
|
diagram.appmodel.nl → <server-ip>
|
|
```
|
|
|
|
### 6. Test Deployment
|
|
|
|
```bash
|
|
# Manual test on server
|
|
app-deploy diagram
|
|
|
|
# Verify container is running
|
|
cd /home/tour/infra/diagram
|
|
docker compose ps
|
|
|
|
# Check logs
|
|
docker compose logs -f diagram
|
|
|
|
# Test endpoint
|
|
curl -I https://diagram.appmodel.nl
|
|
```
|
|
|
|
## Docker Configuration
|
|
|
|
### Dockerfile Explained
|
|
|
|
```dockerfile
|
|
# Stage 1: Build diagrams
|
|
FROM python:3.11-slim AS builder
|
|
RUN apt-get update && apt-get install -y graphviz
|
|
WORKDIR /app
|
|
COPY requirements.txt .
|
|
RUN pip install -r requirements.txt
|
|
COPY *.py .
|
|
RUN python lan_architecture.py && python main.py
|
|
|
|
# Stage 2: Serve with Nginx
|
|
FROM nginx:alpine
|
|
COPY --from=builder /app/*.png /usr/share/nginx/html/
|
|
COPY public/ /usr/share/nginx/html/
|
|
EXPOSE 80
|
|
CMD ["nginx", "-g", "daemon off;"]
|
|
```
|
|
|
|
**Benefits**:
|
|
- Small final image (only Nginx + static files)
|
|
- Build tools not included in production image
|
|
- Automatic diagram generation on build
|
|
|
|
### docker-compose.yml Configuration
|
|
|
|
```yaml
|
|
version: '3.8'
|
|
|
|
services:
|
|
diagram:
|
|
build:
|
|
context: .
|
|
dockerfile: Dockerfile
|
|
container_name: diagram-viewer
|
|
restart: unless-stopped
|
|
networks:
|
|
- traefik_net
|
|
labels:
|
|
- "traefik.enable=true"
|
|
- "traefik.http.routers.diagram.rule=Host(`diagram.appmodel.nl`)"
|
|
- "traefik.http.routers.diagram.entrypoints=websecure"
|
|
- "traefik.http.routers.diagram.tls=true"
|
|
- "traefik.http.routers.diagram.tls.certresolver=letsencrypt"
|
|
- "traefik.http.services.diagram.loadbalancer.server.port=80"
|
|
|
|
networks:
|
|
traefik_net:
|
|
external: true
|
|
```
|
|
|
|
**Key Points**:
|
|
- Connects to external `traefik_net` network
|
|
- Traefik labels configure automatic HTTPS with Let's Encrypt
|
|
- Container restarts automatically unless stopped manually
|
|
|
|
## Development Workflow
|
|
|
|
### Local Development
|
|
|
|
```bash
|
|
# Edit files locally
|
|
vim lan_architecture.py
|
|
vim public/index.html
|
|
|
|
# Test locally (optional)
|
|
python -m venv .venv
|
|
source .venv/bin/activate # or .venv\Scripts\activate on Windows
|
|
pip install -r requirements.txt
|
|
python lan_architecture.py
|
|
|
|
# Commit and push
|
|
git add .
|
|
git commit -m "Update diagram architecture"
|
|
git push
|
|
```
|
|
|
|
### Automatic Deployment
|
|
|
|
After `git push`, the server automatically:
|
|
1. Receives post-receive hook trigger
|
|
2. Pulls latest code
|
|
3. Rebuilds Docker image
|
|
4. Restarts container
|
|
5. Updates live site
|
|
|
|
**Timeline**: Usually completes in 30-60 seconds.
|
|
|
|
### Manual Deployment
|
|
|
|
If needed, manually trigger deployment:
|
|
|
|
```bash
|
|
# On server
|
|
app-deploy diagram
|
|
|
|
# Or manually
|
|
cd /opt/apps/diagram
|
|
git pull
|
|
cd /home/tour/infra/diagram
|
|
docker compose up -d --build diagram
|
|
```
|
|
|
|
## Monitoring & Troubleshooting
|
|
|
|
### View Logs
|
|
|
|
```bash
|
|
# Deployment logs
|
|
tail -f /var/log/app-deploy-diagram.log
|
|
|
|
# Container logs
|
|
cd /home/tour/infra/diagram
|
|
docker compose logs -f diagram
|
|
|
|
# Nginx access logs (if volume mounted)
|
|
tail -f /home/tour/infra/diagram/logs/access.log
|
|
```
|
|
|
|
### Check Container Status
|
|
|
|
```bash
|
|
cd /home/tour/infra/diagram
|
|
|
|
# List running containers
|
|
docker compose ps
|
|
|
|
# Inspect container
|
|
docker compose exec diagram sh
|
|
|
|
# Test nginx config
|
|
docker compose exec diagram nginx -t
|
|
```
|
|
|
|
### Debug Build Issues
|
|
|
|
```bash
|
|
# Rebuild without cache
|
|
docker compose build --no-cache diagram
|
|
|
|
# View build output
|
|
docker compose up --build diagram
|
|
|
|
# Check image layers
|
|
docker image history diagram-viewer
|
|
```
|
|
|
|
### Common Issues
|
|
|
|
#### 1. Build Fails - Missing Dependencies
|
|
|
|
**Symptom**: Build fails at pip install step
|
|
|
|
**Solution**: Verify `requirements.txt` is correct
|
|
```bash
|
|
cd /opt/apps/diagram
|
|
cat requirements.txt
|
|
```
|
|
|
|
#### 2. Nginx 404 Error
|
|
|
|
**Symptom**: Container runs but shows 404
|
|
|
|
**Solution**: Check if files were copied correctly
|
|
```bash
|
|
docker compose exec diagram ls -la /usr/share/nginx/html/
|
|
```
|
|
|
|
#### 3. Traefik Not Routing
|
|
|
|
**Symptom**: Container runs but domain not accessible
|
|
|
|
**Solution**:
|
|
- Verify container is on `traefik_net`: `docker network inspect traefik_net`
|
|
- Check Traefik labels: `docker inspect diagram-viewer | grep traefik`
|
|
- Verify DNS: `nslookup diagram.appmodel.nl`
|
|
|
|
#### 4. Git Pull Fails
|
|
|
|
**Symptom**: app-deploy fails at git pull
|
|
|
|
**Solution**:
|
|
- Check SSH keys for git user: `sudo -u git ssh -T git@git.appmodel.nl`
|
|
- Verify remote: `cd /opt/apps/diagram && git remote -v`
|
|
|
|
## Security Considerations
|
|
|
|
### User Permissions
|
|
|
|
- **git user**: Owns `/opt/apps/*` directories, runs app-deploy
|
|
- **tour user**: Owns `/home/tour/infra/*`, runs Docker Compose
|
|
- Principle: Separate concerns between git operations and container management
|
|
|
|
### Network Isolation
|
|
|
|
- Containers only exposed via Traefik
|
|
- No direct port exposure to host
|
|
- `traefik_net` provides isolation between services
|
|
|
|
### Secrets Management
|
|
|
|
For apps requiring secrets:
|
|
|
|
```yaml
|
|
# In docker-compose.yml
|
|
services:
|
|
diagram:
|
|
environment:
|
|
- SECRET_KEY=${SECRET_KEY}
|
|
env_file:
|
|
- .env # Not committed to git
|
|
```
|
|
|
|
Create `.env` file on server:
|
|
```bash
|
|
echo "SECRET_KEY=your-secret-here" > /home/tour/infra/diagram/.env
|
|
chmod 600 /home/tour/infra/diagram/.env
|
|
```
|
|
|
|
## Scaling & Future Improvements
|
|
|
|
### Multi-Container Apps
|
|
|
|
For apps with multiple services:
|
|
|
|
```yaml
|
|
services:
|
|
frontend:
|
|
build: ./frontend
|
|
labels:
|
|
- "traefik.enable=true"
|
|
- "traefik.http.routers.app-fe.rule=Host(`app.appmodel.nl`)"
|
|
|
|
backend:
|
|
build: ./backend
|
|
labels:
|
|
- "traefik.enable=true"
|
|
- "traefik.http.routers.app-be.rule=Host(`api.app.appmodel.nl`)"
|
|
|
|
database:
|
|
image: postgres:15
|
|
volumes:
|
|
- pgdata:/var/lib/postgresql/data
|
|
|
|
volumes:
|
|
pgdata:
|
|
```
|
|
|
|
### CI/CD Integration
|
|
|
|
Consider adding:
|
|
- Automated tests before deployment
|
|
- Rollback mechanism
|
|
- Blue-green deployments
|
|
- Health checks before switching traffic
|
|
|
|
### Monitoring
|
|
|
|
Add monitoring stack:
|
|
- Prometheus for metrics
|
|
- Grafana for dashboards
|
|
- Loki for log aggregation
|
|
- Alertmanager for notifications
|
|
|
|
## Quick Reference
|
|
|
|
### Key Paths
|
|
|
|
| Path | Purpose |
|
|
|------|---------|
|
|
| `/opt/apps/<app>/` | Git repository checkout |
|
|
| `/home/tour/infra/<app>/` | Docker Compose directory |
|
|
| `/var/log/app-deploy-<app>.log` | Deployment logs |
|
|
|
|
### Key Commands
|
|
|
|
| Command | Purpose |
|
|
|---------|---------|
|
|
| `app-deploy <app>` | Deploy/redeploy application |
|
|
| `docker compose ps` | List running containers |
|
|
| `docker compose logs -f <app>` | Follow container logs |
|
|
| `docker compose restart <app>` | Restart service |
|
|
| `docker compose build --no-cache` | Force rebuild |
|
|
|
|
### URLs
|
|
|
|
| Service | URL |
|
|
|---------|-----|
|
|
| Gitea | https://git.appmodel.nl |
|
|
| Diagram Viewer | https://diagram.appmodel.nl |
|
|
| Traefik Dashboard | https://traefik.appmodel.nl/dashboard/ |
|
|
|
|
## Support
|
|
|
|
For issues or questions:
|
|
1. Check deployment logs: `/var/log/app-deploy-<app>.log`
|
|
2. Check container logs: `docker compose logs -f`
|
|
3. Review Gitea webhook history in repository settings
|
|
4. Test manual deployment: `app-deploy <app>`
|