view
This commit is contained in:
502
DEPLOY_SERVER_SETUP.md
Normal file
502
DEPLOY_SERVER_SETUP.md
Normal file
@@ -0,0 +1,502 @@
|
||||
# Deploy Server Setup - Docker Build Pipeline
|
||||
|
||||
Complete guide for setting up and using the Docker-based deployment pipeline on the build server.
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ Deployment Flow │
|
||||
├─────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ Dev Machine Gitea Server │
|
||||
│ ┌──────────┐ ┌──────┐ ┌─────────┐ │
|
||||
│ │ git │ │ Repo │ │ /opt/ │ │
|
||||
│ │ commit ├─────push────▶│Tour/ ├──hook────▶ │ apps/ │ │
|
||||
│ │ push │ │<app> │ │ <app>/ │ │
|
||||
│ └──────────┘ └──────┘ └────┬────┘ │
|
||||
│ │ │
|
||||
│ git pull │
|
||||
│ │ │
|
||||
│ ┌───────────▼────────┐ │
|
||||
│ │ app-deploy <app> │ │
|
||||
│ │ (as user: git) │ │
|
||||
│ └───────────┬────────┘ │
|
||||
│ │ │
|
||||
│ docker compose up -d │
|
||||
│ --build │
|
||||
│ │ │
|
||||
│ ┌───────────▼────────┐ │
|
||||
│ │ ~/infra/<app>/ │ │
|
||||
│ │ docker-compose.yml │ │
|
||||
│ └───────────┬────────┘ │
|
||||
│ │ │
|
||||
│ rebuild │
|
||||
│ │ │
|
||||
│ ┌───────────▼────────┐ │
|
||||
│ │ 🐳 Container │ │
|
||||
│ │ <app>:latest │ │
|
||||
│ └───────────┬────────┘ │
|
||||
│ │ │
|
||||
│ traefik_net │
|
||||
│ │ │
|
||||
│ ┌───────────▼────────┐ │
|
||||
│ │ 🚦 Traefik │ │
|
||||
│ │ Reverse Proxy │ │
|
||||
│ └───────────┬────────┘ │
|
||||
│ │ │
|
||||
│ https://<app>.appmodel.nl│
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Build Pipeline Workflow
|
||||
|
||||
### Key Principles
|
||||
|
||||
1. **Source of Truth**: Gitea repository `Tour/<app>` is the authoritative source
|
||||
2. **Build Server**: Server is only for building and deployment, not development
|
||||
3. **Automated Deployment**: Git push triggers automatic rebuild and deployment
|
||||
4. **Isolation**: Each app runs in its own Docker container
|
||||
5. **Shared Networking**: All apps connect via `traefik_net` for reverse proxy
|
||||
|
||||
### Pipeline Steps
|
||||
|
||||
#### 1. Git Push (Developer)
|
||||
```bash
|
||||
git push origin main
|
||||
```
|
||||
|
||||
#### 2. Post-Receive Hook (Gitea)
|
||||
Located in: **Settings → Git Hooks → post-receive**
|
||||
|
||||
```bash
|
||||
#!/usr/bin/env bash
|
||||
/usr/local/bin/app-deploy diagram
|
||||
```
|
||||
|
||||
#### 3. App Deploy Script (Server)
|
||||
Runs as Linux user `git`:
|
||||
|
||||
```bash
|
||||
app-deploy diagram
|
||||
```
|
||||
|
||||
This script performs:
|
||||
1. `cd /opt/apps/diagram`
|
||||
2. `git pull --ff-only` from `git@git.appmodel.nl:Tour/diagram.git`
|
||||
3. `cd /home/tour/infra/diagram`
|
||||
4. `docker compose up -d --build diagram`
|
||||
|
||||
#### 4. Docker Build (Server)
|
||||
Multi-stage build process:
|
||||
- **Stage 1**: Build diagrams using Python + Graphviz
|
||||
- **Stage 2**: Serve with Nginx
|
||||
|
||||
#### 5. Traefik Routing (Server)
|
||||
Traefik automatically detects the container via labels and publishes:
|
||||
```
|
||||
https://diagram.appmodel.nl
|
||||
```
|
||||
|
||||
#### 6. DNS Resolution (AdGuard)
|
||||
AdGuard resolves `diagram.appmodel.nl` → Build server IP
|
||||
|
||||
Result: Both internal and external clients use the same URL.
|
||||
|
||||
## Server Directory Structure
|
||||
|
||||
```
|
||||
/opt/apps/
|
||||
└── diagram/ # Git repository (pulled from Gitea)
|
||||
├── Dockerfile
|
||||
├── docker-compose.yml
|
||||
├── requirements.txt
|
||||
├── lan_architecture.py
|
||||
├── main.py
|
||||
└── public/
|
||||
└── index.html
|
||||
|
||||
/home/tour/infra/
|
||||
└── diagram/
|
||||
├── docker-compose.yml # Docker Compose config (may differ from repo)
|
||||
└── logs/ # Nginx logs (optional)
|
||||
|
||||
/var/log/
|
||||
└── app-deploy-diagram.log # Deployment logs
|
||||
```
|
||||
|
||||
## Setup Instructions
|
||||
|
||||
### 1. Create Repository in Gitea
|
||||
|
||||
1. Navigate to: https://git.appmodel.nl
|
||||
2. Create new repository:
|
||||
- **Owner**: `Tour`
|
||||
- **Name**: `diagram`
|
||||
- **Visibility**: Private
|
||||
|
||||
### 2. Initialize Local Repository
|
||||
|
||||
```bash
|
||||
# On dev machine
|
||||
git clone git@git.appmodel.nl:Tour/diagram.git
|
||||
cd diagram
|
||||
|
||||
# Add project files
|
||||
git add Dockerfile docker-compose.yml requirements.txt *.py public/
|
||||
git commit -m "Initial commit: diagram viewer"
|
||||
git push origin main
|
||||
```
|
||||
|
||||
### 3. Setup Server Directories
|
||||
|
||||
```bash
|
||||
# On server as appropriate user
|
||||
sudo -u git mkdir -p /opt/apps/diagram
|
||||
sudo -u git git clone git@git.appmodel.nl:Tour/diagram.git /opt/apps/diagram
|
||||
|
||||
mkdir -p /home/tour/infra/diagram
|
||||
cp /opt/apps/diagram/docker-compose.yml /home/tour/infra/diagram/
|
||||
```
|
||||
|
||||
### 4. Configure Gitea Post-Receive Hook
|
||||
|
||||
1. Go to: https://git.appmodel.nl/Tour/diagram
|
||||
2. Navigate to: **Settings → Git Hooks**
|
||||
3. Select: **post-receive**
|
||||
4. Add script:
|
||||
|
||||
```bash
|
||||
#!/usr/bin/env bash
|
||||
/usr/local/bin/app-deploy diagram
|
||||
```
|
||||
|
||||
5. Save and test
|
||||
|
||||
### 5. Configure DNS
|
||||
|
||||
Add DNS record in AdGuard:
|
||||
```
|
||||
diagram.appmodel.nl → <server-ip>
|
||||
```
|
||||
|
||||
### 6. Test Deployment
|
||||
|
||||
```bash
|
||||
# Manual test on server
|
||||
app-deploy diagram
|
||||
|
||||
# Verify container is running
|
||||
cd /home/tour/infra/diagram
|
||||
docker compose ps
|
||||
|
||||
# Check logs
|
||||
docker compose logs -f diagram
|
||||
|
||||
# Test endpoint
|
||||
curl -I https://diagram.appmodel.nl
|
||||
```
|
||||
|
||||
## Docker Configuration
|
||||
|
||||
### Dockerfile Explained
|
||||
|
||||
```dockerfile
|
||||
# Stage 1: Build diagrams
|
||||
FROM python:3.11-slim AS builder
|
||||
RUN apt-get update && apt-get install -y graphviz
|
||||
WORKDIR /app
|
||||
COPY requirements.txt .
|
||||
RUN pip install -r requirements.txt
|
||||
COPY *.py .
|
||||
RUN python lan_architecture.py && python main.py
|
||||
|
||||
# Stage 2: Serve with Nginx
|
||||
FROM nginx:alpine
|
||||
COPY --from=builder /app/*.png /usr/share/nginx/html/
|
||||
COPY public/ /usr/share/nginx/html/
|
||||
EXPOSE 80
|
||||
CMD ["nginx", "-g", "daemon off;"]
|
||||
```
|
||||
|
||||
**Benefits**:
|
||||
- Small final image (only Nginx + static files)
|
||||
- Build tools not included in production image
|
||||
- Automatic diagram generation on build
|
||||
|
||||
### docker-compose.yml Configuration
|
||||
|
||||
```yaml
|
||||
version: '3.8'
|
||||
|
||||
services:
|
||||
diagram:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile
|
||||
container_name: diagram-viewer
|
||||
restart: unless-stopped
|
||||
networks:
|
||||
- traefik_net
|
||||
labels:
|
||||
- "traefik.enable=true"
|
||||
- "traefik.http.routers.diagram.rule=Host(`diagram.appmodel.nl`)"
|
||||
- "traefik.http.routers.diagram.entrypoints=websecure"
|
||||
- "traefik.http.routers.diagram.tls=true"
|
||||
- "traefik.http.routers.diagram.tls.certresolver=letsencrypt"
|
||||
- "traefik.http.services.diagram.loadbalancer.server.port=80"
|
||||
|
||||
networks:
|
||||
traefik_net:
|
||||
external: true
|
||||
```
|
||||
|
||||
**Key Points**:
|
||||
- Connects to external `traefik_net` network
|
||||
- Traefik labels configure automatic HTTPS with Let's Encrypt
|
||||
- Container restarts automatically unless stopped manually
|
||||
|
||||
## Development Workflow
|
||||
|
||||
### Local Development
|
||||
|
||||
```bash
|
||||
# Edit files locally
|
||||
vim lan_architecture.py
|
||||
vim public/index.html
|
||||
|
||||
# Test locally (optional)
|
||||
python -m venv .venv
|
||||
source .venv/bin/activate # or .venv\Scripts\activate on Windows
|
||||
pip install -r requirements.txt
|
||||
python lan_architecture.py
|
||||
|
||||
# Commit and push
|
||||
git add .
|
||||
git commit -m "Update diagram architecture"
|
||||
git push
|
||||
```
|
||||
|
||||
### Automatic Deployment
|
||||
|
||||
After `git push`, the server automatically:
|
||||
1. Receives post-receive hook trigger
|
||||
2. Pulls latest code
|
||||
3. Rebuilds Docker image
|
||||
4. Restarts container
|
||||
5. Updates live site
|
||||
|
||||
**Timeline**: Usually completes in 30-60 seconds.
|
||||
|
||||
### Manual Deployment
|
||||
|
||||
If needed, manually trigger deployment:
|
||||
|
||||
```bash
|
||||
# On server
|
||||
app-deploy diagram
|
||||
|
||||
# Or manually
|
||||
cd /opt/apps/diagram
|
||||
git pull
|
||||
cd /home/tour/infra/diagram
|
||||
docker compose up -d --build diagram
|
||||
```
|
||||
|
||||
## Monitoring & Troubleshooting
|
||||
|
||||
### View Logs
|
||||
|
||||
```bash
|
||||
# Deployment logs
|
||||
tail -f /var/log/app-deploy-diagram.log
|
||||
|
||||
# Container logs
|
||||
cd /home/tour/infra/diagram
|
||||
docker compose logs -f diagram
|
||||
|
||||
# Nginx access logs (if volume mounted)
|
||||
tail -f /home/tour/infra/diagram/logs/access.log
|
||||
```
|
||||
|
||||
### Check Container Status
|
||||
|
||||
```bash
|
||||
cd /home/tour/infra/diagram
|
||||
|
||||
# List running containers
|
||||
docker compose ps
|
||||
|
||||
# Inspect container
|
||||
docker compose exec diagram sh
|
||||
|
||||
# Test nginx config
|
||||
docker compose exec diagram nginx -t
|
||||
```
|
||||
|
||||
### Debug Build Issues
|
||||
|
||||
```bash
|
||||
# Rebuild without cache
|
||||
docker compose build --no-cache diagram
|
||||
|
||||
# View build output
|
||||
docker compose up --build diagram
|
||||
|
||||
# Check image layers
|
||||
docker image history diagram-viewer
|
||||
```
|
||||
|
||||
### Common Issues
|
||||
|
||||
#### 1. Build Fails - Missing Dependencies
|
||||
|
||||
**Symptom**: Build fails at pip install step
|
||||
|
||||
**Solution**: Verify `requirements.txt` is correct
|
||||
```bash
|
||||
cd /opt/apps/diagram
|
||||
cat requirements.txt
|
||||
```
|
||||
|
||||
#### 2. Nginx 404 Error
|
||||
|
||||
**Symptom**: Container runs but shows 404
|
||||
|
||||
**Solution**: Check if files were copied correctly
|
||||
```bash
|
||||
docker compose exec diagram ls -la /usr/share/nginx/html/
|
||||
```
|
||||
|
||||
#### 3. Traefik Not Routing
|
||||
|
||||
**Symptom**: Container runs but domain not accessible
|
||||
|
||||
**Solution**:
|
||||
- Verify container is on `traefik_net`: `docker network inspect traefik_net`
|
||||
- Check Traefik labels: `docker inspect diagram-viewer | grep traefik`
|
||||
- Verify DNS: `nslookup diagram.appmodel.nl`
|
||||
|
||||
#### 4. Git Pull Fails
|
||||
|
||||
**Symptom**: app-deploy fails at git pull
|
||||
|
||||
**Solution**:
|
||||
- Check SSH keys for git user: `sudo -u git ssh -T git@git.appmodel.nl`
|
||||
- Verify remote: `cd /opt/apps/diagram && git remote -v`
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### User Permissions
|
||||
|
||||
- **git user**: Owns `/opt/apps/*` directories, runs app-deploy
|
||||
- **tour user**: Owns `/home/tour/infra/*`, runs Docker Compose
|
||||
- Principle: Separate concerns between git operations and container management
|
||||
|
||||
### Network Isolation
|
||||
|
||||
- Containers only exposed via Traefik
|
||||
- No direct port exposure to host
|
||||
- `traefik_net` provides isolation between services
|
||||
|
||||
### Secrets Management
|
||||
|
||||
For apps requiring secrets:
|
||||
|
||||
```yaml
|
||||
# In docker-compose.yml
|
||||
services:
|
||||
diagram:
|
||||
environment:
|
||||
- SECRET_KEY=${SECRET_KEY}
|
||||
env_file:
|
||||
- .env # Not committed to git
|
||||
```
|
||||
|
||||
Create `.env` file on server:
|
||||
```bash
|
||||
echo "SECRET_KEY=your-secret-here" > /home/tour/infra/diagram/.env
|
||||
chmod 600 /home/tour/infra/diagram/.env
|
||||
```
|
||||
|
||||
## Scaling & Future Improvements
|
||||
|
||||
### Multi-Container Apps
|
||||
|
||||
For apps with multiple services:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
frontend:
|
||||
build: ./frontend
|
||||
labels:
|
||||
- "traefik.enable=true"
|
||||
- "traefik.http.routers.app-fe.rule=Host(`app.appmodel.nl`)"
|
||||
|
||||
backend:
|
||||
build: ./backend
|
||||
labels:
|
||||
- "traefik.enable=true"
|
||||
- "traefik.http.routers.app-be.rule=Host(`api.app.appmodel.nl`)"
|
||||
|
||||
database:
|
||||
image: postgres:15
|
||||
volumes:
|
||||
- pgdata:/var/lib/postgresql/data
|
||||
|
||||
volumes:
|
||||
pgdata:
|
||||
```
|
||||
|
||||
### CI/CD Integration
|
||||
|
||||
Consider adding:
|
||||
- Automated tests before deployment
|
||||
- Rollback mechanism
|
||||
- Blue-green deployments
|
||||
- Health checks before switching traffic
|
||||
|
||||
### Monitoring
|
||||
|
||||
Add monitoring stack:
|
||||
- Prometheus for metrics
|
||||
- Grafana for dashboards
|
||||
- Loki for log aggregation
|
||||
- Alertmanager for notifications
|
||||
|
||||
## Quick Reference
|
||||
|
||||
### Key Paths
|
||||
|
||||
| Path | Purpose |
|
||||
|------|---------|
|
||||
| `/opt/apps/<app>/` | Git repository checkout |
|
||||
| `/home/tour/infra/<app>/` | Docker Compose directory |
|
||||
| `/var/log/app-deploy-<app>.log` | Deployment logs |
|
||||
|
||||
### Key Commands
|
||||
|
||||
| Command | Purpose |
|
||||
|---------|---------|
|
||||
| `app-deploy <app>` | Deploy/redeploy application |
|
||||
| `docker compose ps` | List running containers |
|
||||
| `docker compose logs -f <app>` | Follow container logs |
|
||||
| `docker compose restart <app>` | Restart service |
|
||||
| `docker compose build --no-cache` | Force rebuild |
|
||||
|
||||
### URLs
|
||||
|
||||
| Service | URL |
|
||||
|---------|-----|
|
||||
| Gitea | https://git.appmodel.nl |
|
||||
| Diagram Viewer | https://diagram.appmodel.nl |
|
||||
| Traefik Dashboard | https://traefik.appmodel.nl/dashboard/ |
|
||||
|
||||
## Support
|
||||
|
||||
For issues or questions:
|
||||
1. Check deployment logs: `/var/log/app-deploy-<app>.log`
|
||||
2. Check container logs: `docker compose logs -f`
|
||||
3. Review Gitea webhook history in repository settings
|
||||
4. Test manual deployment: `app-deploy <app>`
|
||||
Reference in New Issue
Block a user