Compare commits

..

2 Commits

Author SHA1 Message Date
8d536151a8 view 2025-12-02 12:24:02 +01:00
bef1bd6181 Add wiki as submodule 2025-12-02 12:20:27 +01:00
3 changed files with 978 additions and 0 deletions

378
Home.md Normal file
View File

@@ -0,0 +1,378 @@
# Diagram Viewer Wiki
Welcome to the **Diagram Viewer** project documentation.
This project provides a static web interface for viewing and managing network architecture diagrams generated with Python's `diagrams` library.
## 📚 Contents
- [Getting Started](#getting-started)
- [Development Guide](#development-guide)
- [Deployment Guide](#deployment-guide)
- [Architecture Overview](#architecture-overview)
---
## Getting Started
### Prerequisites
- Python 3.8+
- Graphviz
- Docker (for deployment)
### Quick Start
```bash
# Clone repository
git clone git@git.appmodel.nl:Tour/diagram.git
cd diagram
# Create virtual environment
python -m venv .venv
source .venv/bin/activate # Windows: .venv\Scripts\activate
# Install dependencies
pip install -r requirements.txt
# Generate diagrams
python lan_architecture.py
python main.py
```
---
## Development Guide
### Project Structure
```
diagram/
├── Dockerfile # Multi-stage build for production
├── docker-compose.yml # Docker Compose configuration
├── requirements.txt # Python dependencies
├── lan_architecture.py # LAN architecture diagram generator
├── main.py # Main diagram generator
├── public/
│ └── index.html # Live Mermaid diagram editor
└── wiki/ # Documentation (this wiki)
```
### Adding New Diagrams
1. Create a new Python file (e.g., `network_diagram.py`)
2. Import required components from `diagrams` library
3. Define your architecture using context managers
4. Run the script to generate PNG output
Example:
```python
from diagrams import Diagram, Cluster
from diagrams.onprem.network import Router
from diagrams.generic.device import Mobile
with Diagram("My Network", show=False, filename="my_network"):
router = Router("Gateway")
device = Mobile("Phone")
device >> router
```
### Testing Locally
```bash
# Generate diagrams
python lan_architecture.py
# View output
ls -la *.png
# Test with local web server
python -m http.server 8000 --directory public/
# Open: http://localhost:8000
```
### Available Icons
The `diagrams` library provides icons from multiple providers:
- **OnPrem**: Network, compute, storage, etc.
- **Cloud**: AWS, Azure, GCP services
- **Generic**: Network devices, blank nodes
- **Custom**: Use your own images
See: [Diagrams Documentation](https://diagrams.mingrammer.com/docs/nodes/overview)
---
## Deployment Guide
### Overview
The project uses a fully automated Docker-based deployment pipeline:
1. **Develop** on local machine
2. **Commit & Push** to Gitea
3. **Auto-Deploy** via post-receive hook
4. **Build** multi-stage Docker image
5. **Serve** via Traefik reverse proxy
### Deployment Steps
#### 1. Initial Setup
```bash
# On server: Create app structure
apps-create diagram static-fe
# This creates:
# - /opt/apps/diagram (git repository)
# - /home/tour/infra/diagram/docker-compose.yml
```
#### 2. Configure Gitea Hook
In repository **Settings → Git Hooks → post-receive**:
```bash
#!/usr/bin/env bash
/usr/local/bin/app-deploy diagram
```
#### 3. Deploy
Simply push to Gitea:
```bash
git push origin main
```
The server automatically:
- Pulls latest code
- Rebuilds Docker image
- Restarts container
- Updates live site at https://diagram.appmodel.nl
### Manual Deployment
If needed:
```bash
# On server
app-deploy diagram
# Or step-by-step
cd /opt/apps/diagram
git pull
cd /home/tour/infra/diagram
docker compose up -d --build diagram
```
### Monitoring
```bash
# View deployment logs
tail -f /var/log/app-deploy-diagram.log
# View container logs
cd /home/tour/infra/diagram
docker compose logs -f diagram
# Check container status
docker compose ps
```
---
## Architecture Overview
### Infrastructure
```
┌─────────────────────────────────────────────┐
│ Production Architecture │
├─────────────────────────────────────────────┤
│ │
│ ┌──────────────┐ │
│ │ Gitea │ (Source of Truth) │
│ │ Tour/diagram │ │
│ └──────┬───────┘ │
│ │ git push │
│ ↓ │
│ ┌──────────────┐ │
│ │ Post-Receive │ (Auto-trigger) │
│ │ Hook │ │
│ └──────┬───────┘ │
│ │ app-deploy diagram │
│ ↓ │
│ ┌──────────────────────┐ │
│ │ /opt/apps/diagram/ │ │
│ │ (Git Repository) │ │
│ └──────┬───────────────┘ │
│ │ git pull │
│ ↓ │
│ ┌──────────────────────┐ │
│ │ Docker Build Process │ │
│ │ ┌─────────────────┐ │ │
│ │ │ Stage 1: Build │ │ │
│ │ │ - Python │ │ │
│ │ │ - Graphviz │ │ │
│ │ │ - Generate PNGs │ │ │
│ │ └─────────────────┘ │ │
│ │ ┌─────────────────┐ │ │
│ │ │ Stage 2: Serve │ │ │
│ │ │ - Nginx │ │ │
│ │ │ - Static files │ │ │
│ │ └─────────────────┘ │ │
│ └──────┬───────────────┘ │
│ │ container starts │
│ ↓ │
│ ┌──────────────────────┐ │
│ │ diagram-viewer │ │
│ │ Container :80 │ │
│ └──────┬───────────────┘ │
│ │ traefik_net │
│ ↓ │
│ ┌──────────────────────┐ │
│ │ Traefik │ │
│ │ Reverse Proxy │ │
│ │ + Let's Encrypt │ │
│ └──────┬───────────────┘ │
│ │ HTTPS │
│ ↓ │
│ diagram.appmodel.nl │
│ │
└─────────────────────────────────────────────┘
```
### Network Topology
The diagram viewer documents our home lab infrastructure:
- **LAN 192.168.1.0/24**: Main network
- Core server (192.168.1.159): Traefik, Gitea, Dokku, Auction stack, MI50/Ollama
- DNS server (192.168.1.163): AdGuard, Artifactory, CI runner
- Home Assistant (192.168.1.193)
- Atlas worker (192.168.1.100)
- IoT devices
- **Tether 192.168.137.0/24**: Isolated subnet
- Hermes worker (192.168.137.239)
- Plato worker (192.168.137.163)
### Services
| Service | URL | Description |
|---------|-----|-------------|
| Diagram Viewer | https://diagram.appmodel.nl | This application |
| Gitea | https://git.appmodel.nl | Git hosting |
| Auction Frontend | https://auction.appmodel.nl | Auction platform |
| Aupi API | https://aupi.appmodel.nl | Backend API |
| Dokku | https://dokku.lan | PaaS platform |
---
## Additional Resources
### Related Documentation
- [DEPLOY_SERVER_SETUP.md](../DEPLOY_SERVER_SETUP.md) - Detailed deployment guide
- [DEPLOYMENT_GUIDE.md](../DEPLOYMENT_GUIDE.md) - General deployment pipeline
- [README.md](../README.md) - Project overview
### External Links
- [Diagrams Library](https://diagrams.mingrammer.com/)
- [Graphviz](https://graphviz.org/)
- [Mermaid.js](https://mermaid.js.org/) (used in live editor)
- [Traefik Documentation](https://doc.traefik.io/traefik/)
- [Docker Compose](https://docs.docker.com/compose/)
### Contributing
To contribute to this project:
1. Create a feature branch
2. Make your changes
3. Test locally
4. Submit a pull request in Gitea
5. Wait for automatic deployment after merge
---
## Troubleshooting
### Common Issues
#### Diagrams Not Generating
**Problem**: Python script fails to generate PNG files
**Solution**:
```bash
# Check Graphviz installation
dot -V
# Reinstall if needed
# macOS: brew install graphviz
# Ubuntu: sudo apt-get install graphviz
# Windows: choco install graphviz
# Verify Python dependencies
pip list | grep diagrams
pip install --upgrade diagrams
```
#### Container Build Fails
**Problem**: Docker build fails during deployment
**Solution**:
```bash
# Check deployment logs
tail -f /var/log/app-deploy-diagram.log
# Rebuild manually with verbose output
cd /home/tour/infra/diagram
docker compose build --no-cache --progress=plain diagram
```
#### Site Not Accessible
**Problem**: Container runs but site not reachable
**Solution**:
1. Check DNS: `nslookup diagram.appmodel.nl`
2. Verify Traefik labels: `docker inspect diagram-viewer | grep traefik`
3. Check Traefik logs: `docker logs traefik`
4. Test container directly: `curl http://localhost:PORT`
#### Changes Not Reflected
**Problem**: Pushed changes but site unchanged
**Solution**:
```bash
# Force rebuild on server
cd /home/tour/infra/diagram
docker compose down
docker compose up -d --build --force-recreate diagram
# Clear browser cache
# Ctrl+Shift+R (Windows/Linux) or Cmd+Shift+R (macOS)
```
---
## Contact & Support
For issues, questions, or suggestions:
1. Check this wiki
2. Review [DEPLOY_SERVER_SETUP.md](../DEPLOY_SERVER_SETUP.md)
3. Check container logs
4. Contact the infrastructure team
---
**Last Updated**: 2025-12-02
**Version**: 1.0.0
**Maintainer**: Tour

239
Quick-Start.md Normal file
View File

@@ -0,0 +1,239 @@
# Quick Start Guide
Get up and running with the Diagram Viewer in minutes.
## For Developers
### 1. Clone Repository
```bash
git clone git@git.appmodel.nl:Tour/diagram.git
cd diagram
```
### 2. Setup Environment
```bash
# Create virtual environment
python -m venv .venv
# Activate (choose your platform)
source .venv/bin/activate # macOS/Linux
.venv\Scripts\activate # Windows CMD
source .venv/Scripts/activate # Windows Git Bash
```
### 3. Install Dependencies
```bash
pip install -r requirements.txt
```
### 4. Generate Diagrams
```bash
# Generate LAN architecture diagram
python lan_architecture.py
# Generate main diagram
python main.py
# View generated files
ls -la *.png
```
### 5. Test Locally
```bash
# Start local web server
python -m http.server 8000 --directory public/
# Open browser
# Navigate to: http://localhost:8000
```
### 6. Make Changes & Deploy
```bash
# Edit diagram code
vim lan_architecture.py
# Commit changes
git add .
git commit -m "Update network diagram"
# Push to deploy (auto-deploys to production)
git push origin main
```
---
## For Operators
### View Live Site
Simply navigate to:
```
https://diagram.appmodel.nl
```
### Manual Deployment
On the server:
```bash
# Trigger deployment
app-deploy diagram
# View logs
tail -f /var/log/app-deploy-diagram.log
# Check status
cd /home/tour/infra/diagram
docker compose ps
```
### View Logs
```bash
# Deployment logs
tail -f /var/log/app-deploy-diagram.log
# Container logs
cd /home/tour/infra/diagram
docker compose logs -f diagram
# Last 100 lines
docker compose logs --tail=100 diagram
```
### Restart Service
```bash
cd /home/tour/infra/diagram
# Restart container
docker compose restart diagram
# Full restart (rebuild)
docker compose down
docker compose up -d --build diagram
```
---
## Common Tasks
### Add New Diagram
1. Create new Python file:
```python
# my_diagram.py
from diagrams import Diagram
from diagrams.onprem.network import Router
with Diagram("My Network", show=False, filename="my_network"):
Router("Gateway")
```
2. Update Dockerfile if needed (add to RUN command):
```dockerfile
RUN python lan_architecture.py && \
python main.py && \
python my_diagram.py
```
3. Commit and push:
```bash
git add my_diagram.py Dockerfile
git commit -m "Add my network diagram"
git push
```
### Update Frontend
1. Edit HTML:
```bash
vim public/index.html
```
2. Test locally:
```bash
python -m http.server 8000 --directory public/
```
3. Deploy:
```bash
git add public/index.html
git commit -m "Update frontend"
git push
```
### Check Container Health
```bash
# Container status
docker compose ps
# Health check
docker inspect diagram-viewer | grep -A 10 Health
# Test endpoint
curl -I https://diagram.appmodel.nl
```
---
## Troubleshooting
### Build Fails
```bash
# Check logs
tail -f /var/log/app-deploy-diagram.log
# Rebuild with verbose output
cd /home/tour/infra/diagram
docker compose build --no-cache --progress=plain diagram
```
### Site Not Loading
```bash
# Check container
docker compose ps
# Check Traefik routing
docker logs traefik | grep diagram
# Test DNS
nslookup diagram.appmodel.nl
# Test directly
curl http://localhost:PORT
```
### Changes Not Visible
```bash
# Force rebuild
cd /home/tour/infra/diagram
docker compose up -d --build --force-recreate diagram
# Clear browser cache
# Ctrl+Shift+R (Windows/Linux)
# Cmd+Shift+R (macOS)
```
---
## Next Steps
- Read the [Home](Home.md) page for comprehensive documentation
- Review [DEPLOY_SERVER_SETUP.md](../DEPLOY_SERVER_SETUP.md) for deployment details
- Check [Diagrams Library docs](https://diagrams.mingrammer.com/) for icon options
- Explore the [Mermaid editor](https://diagram.appmodel.nl) for live diagram editing
---
**Need Help?** Check the [Troubleshooting](Home.md#troubleshooting) section or review deployment logs.

361
WIKI_INTEGRATION.md Normal file
View File

@@ -0,0 +1,361 @@
# Gitea Wiki Integration Guide
This guide explains how to integrate the local `wiki/` folder with Gitea's wiki system.
## How Gitea Wikis Work
Gitea stores wiki pages in a **separate Git repository** with the naming pattern:
```
<repository>.wiki.git
```
For the `viewer` repository:
- Main repo: `git@git.appmodel.nl:Tour/viewer.git`
- Wiki repo: `git@git.appmodel.nl:Tour/viewer.wiki.git`
Wiki pages are **markdown files** (`.md`) stored in the root of the wiki repository.
## Integration Options
### Option 1: Push Local Wiki to Gitea (Recommended)
This approach replaces the current Gitea wiki with your local content.
```bash
# 1. Clone the Gitea wiki repository
git clone git@git.appmodel.nl:Tour/viewer.wiki.git viewer-wiki
# 2. Copy your wiki files to the wiki repository
cd viewer-wiki
cp ../viewer/wiki/*.md .
# 3. Commit and push
git add .
git commit -m "Initialize wiki with documentation"
git push origin master
# 4. View on Gitea
# Navigate to: https://git.appmodel.nl/Tour/diagram/wiki
```
### Option 2: Use Git Submodule (Bidirectional Sync)
Keep the wiki as a submodule in the main repository for easy synchronization.
```bash
# In the main diagram repository
cd /c/vibe/diagram
# Remove local wiki folder
rm -rf wiki/
# Add Gitea wiki as submodule
git submodule add git@git.appmodel.nl:Tour/viewer.wiki.git wiki
# Copy your documentation
cp ../temp-wiki-backup/*.md wiki/
# Commit to wiki submodule
cd wiki
git add .
git commit -m "Add documentation"
git push origin master
# Update main repository
cd ..
git add wiki
git commit -m "Add wiki as submodule"
git push origin main
```
Now changes can be made either:
- **In Gitea UI**: Edit wiki pages directly
- **Locally**: Edit in `wiki/` folder, commit, and push
To sync changes from Gitea:
```bash
cd wiki
git pull origin master
cd ..
git add wiki
git commit -m "Update wiki submodule"
git push
```
### Option 3: Git Subtree (Advanced)
Similar to submodule but integrates the wiki history into the main repository.
```bash
# Add wiki as subtree
git subtree add --prefix=wiki git@git.appmodel.nl:Tour/diagram.wiki.git master --squash
# Push changes to wiki
git subtree push --prefix=wiki git@git.appmodel.nl:Tour/diagram.wiki.git master
# Pull changes from wiki
git subtree pull --prefix=wiki git@git.appmodel.nl:Tour/diagram.wiki.git master --squash
```
### Option 4: Automated Sync Script
Create a script to automatically sync wiki changes.
Create `sync-wiki.sh`:
```bash
#!/bin/bash
# sync-wiki.sh - Sync local wiki with Gitea
WIKI_REPO="git@git.appmodel.nl:Tour/diagram.wiki.git"
WIKI_DIR="wiki"
TEMP_WIKI="/tmp/gitea-wiki-$$"
# Clone Gitea wiki
git clone "$WIKI_REPO" "$TEMP_WIKI"
# Copy local changes to temp wiki
cp -r "$WIKI_DIR"/*.md "$TEMP_WIKI/"
# Commit and push
cd "$TEMP_WIKI"
git add .
if git diff-staged --quiet; then
echo "No changes to sync"
else
git commit -m "Sync wiki from main repository"
git push origin master
echo "Wiki synced successfully"
fi
# Cleanup
cd -
rm -rf "$TEMP_WIKI"
```
Usage:
```bash
chmod +x sync-wiki.sh
./sync-wiki.sh
```
## Recommended Workflow
For your deployment pipeline, I recommend **Option 1** with a manual push:
### Step-by-Step Setup
1. **Create wiki repository on Gitea** (if not exists):
- Go to: https://git.appmodel.nl/Tour/diagram
- Click "Wiki" tab
- Create first page (triggers wiki repo creation)
2. **Push local wiki content**:
```bash
# Clone wiki repository
cd /c/vibe
git clone git@git.appmodel.nl:Tour/diagram.wiki.git
# Copy your wiki files
cd diagram.wiki
cp ../diagram/wiki/*.md .
# Verify files
ls -la
# Commit and push
git add .
git commit -m "Initialize wiki documentation
Added:
- Home.md: Main documentation page
- Quick-Start.md: Quick start guide
"
git push origin master
```
3. **Verify on Gitea**:
- Navigate to: https://git.appmodel.nl/Tour/diagram/wiki
- You should see your wiki pages
4. **Keep local wiki in main repo** (optional):
- Keep `wiki/` folder in main repo as "source of truth"
- When updating wiki, manually sync to Gitea wiki repo
- Or set up automated sync via CI/CD
## Wiki Page Naming
Gitea wikis follow these conventions:
| Filename | URL | Purpose |
|----------|-----|---------|
| `Home.md` | `/wiki/` or `/wiki/Home` | Main wiki page |
| `Quick-Start.md` | `/wiki/Quick-Start` | Quick start guide |
| `API-Reference.md` | `/wiki/API-Reference` | API documentation |
**Rules**:
- Use PascalCase or kebab-case for multi-word pages
- Spaces in filenames become hyphens in URLs
- First page is always `Home.md`
## Linking Between Pages
In your markdown files:
```markdown
<!-- Link to another wiki page -->
[Quick Start Guide](Quick-Start)
<!-- Link with custom text -->
[Get Started](Quick-Start)
<!-- Link to main repository -->
[View Code](../)
<!-- Link to specific file -->
[Dockerfile](../src/master/Dockerfile)
<!-- External link -->
[Diagrams Library](https://diagrams.mingrammer.com/)
```
## Automation with Post-Receive Hook
To automatically sync wiki on deployment, add to Gitea post-receive hook:
```bash
#!/usr/bin/env bash
# Deploy main app
/usr/local/bin/app-deploy diagram
# Sync wiki (if wiki/ folder exists in repo)
if [ -d "/opt/apps/diagram/wiki" ]; then
cd /tmp
git clone git@git.appmodel.nl:Tour/diagram.wiki.git gitea-wiki-$$
cp /opt/apps/diagram/wiki/*.md /tmp/gitea-wiki-$$/
cd /tmp/gitea-wiki-$$
git add .
if ! git diff --staged --quiet; then
git commit -m "Auto-sync from main repository"
git push origin master
fi
cd /tmp
rm -rf /tmp/gitea-wiki-$$
fi
```
## Current Setup Instructions
Based on your current setup, here's what to do:
```bash
# 1. Navigate to your diagram project
cd /c/vibe/diagram
# 2. Clone the wiki repository (for viewer, adjust as needed)
cd ..
git clone git@git.appmodel.nl:Tour/diagram.wiki.git
# 3. Copy wiki files
cp diagram/wiki/*.md diagram.wiki/
# 4. Push to Gitea
cd diagram.wiki
git add .
git commit -m "Initialize wiki with deployment documentation"
git push origin master
# 5. Verify
# Open: https://git.appmodel.nl/Tour/diagram/wiki
```
## Maintaining Both Wikis
If you want to keep both local and Gitea wikis in sync:
### Manual Sync (Simple)
```bash
# After editing local wiki files
cd /c/vibe/diagram/wiki
cp *.md /c/vibe/diagram.wiki/
cd /c/vibe/diagram.wiki
git add .
git commit -m "Update documentation"
git push
```
### Automated Sync (Add to package.json or Makefile)
```json
{
"scripts": {
"sync-wiki": "cp wiki/*.md ../diagram.wiki/ && cd ../diagram.wiki && git add . && git commit -m 'Sync wiki' && git push"
}
}
```
Or create a `Makefile`:
```makefile
.PHONY: sync-wiki
sync-wiki:
@echo "Syncing wiki to Gitea..."
@cp wiki/*.md ../diagram.wiki/
@cd ../diagram.wiki && git add . && git commit -m "Sync wiki from main repo" && git push
@echo "Wiki synced successfully!"
```
Usage: `make sync-wiki`
## Best Practices
1. **Single Source of Truth**: Choose one location (local or Gitea) as authoritative
2. **Version Control**: Keep wiki changes in Git history
3. **Review Process**: Use pull requests for significant documentation changes
4. **Link Checking**: Regularly verify internal links work
5. **Consistent Formatting**: Use markdown linting tools
6. **Table of Contents**: Keep `Home.md` as navigation hub
## Troubleshooting
### Wiki Repository Doesn't Exist
Create the wiki first in Gitea UI:
1. Go to repository: https://git.appmodel.nl/Tour/diagram
2. Click "Wiki" tab
3. Click "Create New Page"
4. Create a dummy page (will be overwritten)
5. Now you can clone `diagram.wiki.git`
### Permission Denied
Ensure your SSH key is added to Gitea:
```bash
ssh -T git@git.appmodel.nl
```
### Merge Conflicts
If both local and Gitea wiki are edited:
```bash
cd diagram.wiki
git pull origin master
# Resolve conflicts manually
git add .
git commit -m "Merge local and remote changes"
git push
```
## Summary
**Recommended approach for your setup**:
1. Keep `wiki/` in main repository for version control
2. Manually push to Gitea wiki repository when documentation is ready
3. Optionally add automated sync in post-receive hook
4. Edit wiki locally, sync to Gitea for visibility
This gives you the best of both worlds:
- ✅ Documentation versioned with code
- ✅ Visible wiki in Gitea UI
- ✅ Easy to maintain and update
- ✅ Works with existing deployment pipeline