Files
qwen-image/AGENTS.md
mike 3b9b9e829b Summary
• Implemented global offline mode for all HuggingFace-dependent modules to eliminate unauthenticated request warnings and prevent external connection attempts during startup and runtime.

   Changes
   • Global Offline Configuration: Added HF_HUB_OFFLINE=1 and TRANSFORMERS_OFFLINE=1 to the environment variables at the top of edit_api.py, embeddings.py, watcher.py, and pose_llm/pose_llm_api.py.
   • Explicit Local Files Only: Updated all huggingface_hub.hf_hub_download calls in edit_api.py to include local_files_only=True, ensuring they never attempt to reach the Hub if models are already cached.
   • LLM Service Optimization: Updated AutoTokenizer and AutoModelForCausalLM calls in pose_llm/pose_llm_api.py to use local_files_only=True.
   • Consistency: Ensured that shared modules like embeddings.py which uses open_clip are also locked to offline mode to prevent background downloads.
2026-06-28 16:56:07 +02:00

504 lines
27 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Qwen-Image-Edit Rapid-AIO v23 — System Documentation
## Overview
A self-hosted AI image studio for generative editing, multi-view character rendering, background removal, face swapping, and group-based asset management. It runs the **Qwen 2.5 VL GGUF** model through ComfyUI and exposes a feature-rich browser frontend backed by a FastAPI service.
```
Browser (car.html SPA)
│ HTTP / fetch
FastAPI :8500 (edit_api.py)
├─── ComfyUI :8188 ← Qwen GGUF model
├─── SAM2 / rembg ← background removal
├─── insightface ← faceswap + face extraction
├─── GFPGAN ← face restoration
└─── PostgreSQL ← metadata (192.168.1.160:5433)
File system (/mnt/zim/tour-comfy/output/, wireframe/)
```
---
## Hardware
| Component | Spec |
|-----------|------|
| GPU | NVIDIA RTX A6000 (48 GB VRAM) |
| CUDA | 12.4 |
| PyTorch | 2.6.0+cu124 |
| ComfyUI | latest pinned (a6000-comfy/) |
---
## Services
### 1. ComfyUI Backend — `:8188`
- **Script**: `a6000-comfy/run_comfyui.sh`
- **Systemd**: `comfyui-backend`
- **Purpose**: Executes the Qwen GGUF model via `workflow_qwen_edit.json`
**Loaded models**
| Role | File |
|------|------|
| GGUF LLM | `Qwen-Rapid-NSFW-v23_Q8_0.gguf` |
| Text encoder | `qwen_2.5_vl_7b_fp8_scaled.safetensors` |
| VAE | `qwen_image_vae.safetensors` |
**Workflow nodes** (`workflow_qwen_edit.json`):
- `NODE_POSITIVE` — positive text prompt
- `NODE_NEGATIVE` — negative prompt
- `NODE_LATENT` — latent dimensions (width × height)
- `NODE_KSAMPLER` — sampler params (Euler/karras, seed, steps, CFG)
- `image1` — reference image (base64 uploaded to ComfyUI `/upload/image`)
- `image2` — optional wireframe pose guide (extracted from wireframe video)
---
### 2. FastAPI API — `:8500`
- **Script**: `tour-comfy/edit_api.py`
- **Systemd**: `comfyui-api`
- **DB**: PostgreSQL at `192.168.1.160:5433`, database `dv`, table `person`
#### In-Memory File Metadata Cache
To mitigate performance and network latency bottlenecks on remote/shared storage filesystems (e.g. `/mnt/zim`), the API server employs a light, thread-safe in-memory metadata cache (`_file_meta_cache`):
- **Cached Items**: File existence (`exists`) and modification timestamps (`mtime`) mapped by `filename`.
- **Cache TTL**: 5.0 seconds.
- **Idempotency & Invalidation**: Any mutation endpoints (such as crop, padding, duplication, background removal, or deletion) automatically update (`_update_cached_file_meta`) or explicitly delete (`_clear_cached_file_meta`) entries to keep cache and disk perfectly in sync.
- **Preloading Optimization**: Leverages cache queries in static indexing loops (`_sync_preloaded_images`, `_write_all_static`, `list_images`) and tracks changes to completely skip redundant static HTML rewrites if the preloaded image set is identical (`_last_preloaded_images_set`).
#### Endpoints Library
##### Core Generation & Upload
| Method | Path | Purpose |
|--------|------|---------|
| `POST` | `/upload` | Upload image; optionally add to existing group (`group_id` + `skip_poses`) |
| `POST` | `/batch` | Queue multi-prompt generation job (async) |
| `GET` | `/batch/{job_id}` | Poll batch job status / per-item progress |
| `DELETE` | `/batch/{job_id}` | Cancel active batch job |
| `POST` | `/multi-ref` | Queue a generation job with multiple reference images |
| `POST` | `/refine-prompt` | Refine a generation prompt using the external uncensored chat LLM |
| `POST` | `/edit` | Edit uploaded image |
| `POST` | `/generate-scenery` | Generate scenery background based on video frame + reference image |
| `GET` | `/scenery/library` | Return all scenery images grouped by source video reference |
##### Image Editing & Canvas Expansion (Padding & Crop)
| Method | Path | Purpose |
|--------|------|---------|
| `POST` | `/images/{filename}/crop` | Crop to pixel rect; `as_copy:true` crops a referenced copy instead of modifying in-place |
| `POST` | `/images/{filename}/autocrop` | Auto-trim transparent borders in-place |
| `POST` | `/images/{filename}/pad` | Expand image canvas with custom padding on each side, optional background fills (`transparent`, `black`, `white`), and optional Qwen-based Outpainting (`outpaint:true`) |
| `POST` | `/images/{filename}/rotate` | Rotate clockwise in 90° steps (lossless transpose, in-place) |
| `POST` | `/images/{filename}/invert-alpha` | Invert the alpha channel (recovers background removal errors) |
| `POST` | `/images/{filename}/duplicate` | Copy image into the same group with timestamp-based filename and `source_refs=[original]` |
| `POST` | `/restore-background/{filename}` | Flatten RGBA to opaque RGB via white composite background |
##### Background Removal
| Method | Path | Purpose |
|--------|------|---------|
| `POST` | `/remove-background/{filename}` | rembg BG removal, overwrites in-place; sets `has_background=false` + refreshes static data |
| `POST` | `/remove-background-sam/{filename}` | SAM2 BG removal → `.nobg.png` sidecar |
| `POST` | `/remove-background/group/{group_id}` | Queue background removal for all group members as an async background task |
| `GET` | `/sam2/check` | Return whether SAM2 background remover is available |
##### Metadata, Custom Labels & Group Management
| Method | Path | Purpose |
|--------|------|---------|
| `GET` | `/images` | List all images with DB metadata |
| `GET` | `/images/{filename}` | Serve image file |
| `GET` | `/groups` | List all groups with members |
| `GET` | `/group-names` | Return map of custom group names |
| `POST` | `/group-names/{group_id}` | Save custom group name |
| `POST` | `/groups/merge` | Merge several groups into one group |
| `POST` | `/groups/extract` | Extract a file into its own solo group |
| `GET` | `/groups/{group_id}/order` | Return order of members in a group |
| `POST` | `/groups/{group_id}/order` | Set custom sort order of group members |
| `GET` | `/similar/{filename}` | Find visually similar images using database embeddings |
| `GET` | `/names` | Map `filename → person name` |
| `POST` | `/names/{filename}` | Map image file to custom character name |
| `POST` | `/images/{filename}/tags` | Set image tags list (supports tags like `ARCHIVED`, `HIDDEN`, `SOURCE`, `VISIBLE`, `LIKE`, `DISLIKE`, `21+`) |
| `POST` | `/images/{filename}/tag-action` | Add, remove, or toggle a specific tag on an image (automatically handles mutual exclusivity of `LIKE`/`DISLIKE`) |
| `POST` | `/images/{filename}/source` | Explicitly toggle marking an image as a source asset (sets/clears `is_source` and toggles the `SOURCE` tag) |
| `POST` | `/images/{filename}/estimate-21plus` | Analyze image with the WD tagger to estimate adult/sensitive rating and apply `21+` tag |
| `POST` | `/images/bulk-move` | Move a list of filenames to a specific target filmstrip (`VISIBLE`, `HIDDEN`, `ARCHIVED`, `SOURCE`) in bulk, updating tags/columns |
| `POST` | `/images/{filename}/undress` | Queue an asynchronous batch job utilizing the `UNDRESS_PROMPT` on the given reference image |
##### Visibility, Archiving & Deletion
| Method | Path | Purpose |
|--------|------|---------|
| `POST` | `/images/{filename}/hidden` | Toggle studio visibility (soft-hide) |
| `POST` | `/images/{filename}/archive` | Soft-archive image (hidden from default view) |
| `POST` | `/images/{filename}/unarchive` | Restore soft-archived image |
| `DELETE` | `/images/{filename}` | Delete image file + DB record |
| `POST` | `/groups/{group_id}/archive` | Soft-archive an entire group |
| `POST` | `/groups/{group_id}/unarchive` | Restore an archived group |
| `DELETE` | `/groups/{group_id}` | Delete entire group files and DB records |
##### 2D Body Pose Tools (rtmlib / RTMPose)
| Method | Path | Purpose |
|--------|------|---------|
| `GET` | `/pose/check` | Report pose-estimator availability + backend (rtmlib or mediapipe) |
| `POST` | `/images/{filename}/pose` | Estimate COCO-17 body keypoints (caches descriptor for similar-search) |
| `GET` | `/pose/similar/{filename}` | Rank library images by pose similarity to this image |
| `POST` | `/pose/similar` | Rank library images by similarity to a supplied (edited) skeleton |
| `POST` | `/pose/index` | Build pose descriptors for the whole library (daemon thread) |
| `GET` | `/pose/index/status` | Poll pose-index build progress |
| `POST` | `/pose/from-wireframe` | Extract frame from a wireframe video and estimate body pose keypoints |
| `POST` | `/pose/render` | Render keypoints as OpenPose skeleton image (base64) |
| `POST` | `/generate-with-pose` | Generate image from a specific pose skeleton (avoids wireframe bleed-through) |
| `GET` | `/poses` | Load pose library from `poses.md` |
| `POST` | `/poses` | Create / update / rename a pose (`old_name` to rename) → rewrites `poses.md` |
| `DELETE` | `/poses/{name}` | Delete a pose from `poses.md` |
##### 3D Turntable & Orbit Previews
| Method | Path | Purpose |
|--------|------|---------|
| `POST` | `/orbit` | Queue an orbit preview generation. Engine `depth` (fast 2.5D card parallax) or `qwen` (turntable) |
| `GET` | `/turntable/status` | Return background turntable generation state/progress summary |
| `GET` | `/turntable/status/{group_id}` | Return turntable state for a specific group |
| `POST` | `/turntable/pause` | Pause background turntable generation |
| `POST` | `/turntable/resume` | Resume background turntable generation |
| `DELETE` | `/turntable/{group_id}` | Reset turntable cache & records for group to force re-generation |
##### Face Processing (insightface)
| Method | Path | Purpose |
|--------|------|---------|
| `POST` | `/faceswap` | insightface video faceswap |
| `GET` | `/faceswap/check` | Report available face enhancement backends |
| `POST` | `/images/{filename}/extract-face` | Extract & save face crop via insightface (background task) |
| `POST` | `/faces/similar` | Find groups with visually similar faces using embeddings |
| `POST` | `/faces/index` | Build face embedding index for all preferred faces in library |
| `GET` | `/faces/index/status` | Poll face-indexing progress status |
| `GET` | `/faces/{group_id}` | Check if face crop exists for group |
##### Videos & Wireframe Guides
| Method | Path | Purpose |
|--------|------|---------|
| `GET` | `/videos` | List all available wireframe guide videos |
| `GET` | `/wireframe/frame/{video_name}` | Extract frame at `?t=` (01) from wireframe video |
| `GET` | `/wireframe/duration/{video_name}` | Get video duration (seconds) via ffprobe |
| `POST` | `/wireframe/trim` | Trim a wireframe video to given start/end offsets |
| `POST` | `/wireframe/frame` | Extract frame at timestamp, returns base64 PNG |
| `POST` | `/generate-video` | Stitch list of output images into a short looping MP4 video |
##### Database, Config & System Diagnostics
| Method | Path | Purpose |
|--------|------|---------|
| `GET/POST` | `/config` | Read / write `config.json` |
| `GET` | `/db/inconsistencies` | Retrieve consistency report or run a check with `?run_now=true` |
| `POST` | `/db/repair` | Run repair actions (restore, import file, assign group, permanently delete) |
| `POST` | `/db/cleanup` | Delete DB records of files that are missing on disk |
| `GET` | `/health` | API health check status |
---
## Generation Pipeline (`_run_pipeline`)
```
prompt
├─ "transparent background" keywords detected?
│ YES → strip keywords from Qwen prompt (keeps `is_transparent` flag)
│ set negative: "deformed anatomy, watermark, logo"
├─ Canvas Expansion / Outpainting requested?
│ YES → if outpaint=True:
│ ├─ If source is transparent (RGBA): composite onto flat black background for maximum contrast.
│ ├─ Formulate dynamic prompt description: e.g. "replacing the black/white background areas"
│ └─ Append outpainting prompt instructions: "Naturally outpaint and extend..."
├─ Upload reference image(s) to ComfyUI /upload/image
│ image1 = source image (composited RGB if outpainting transparent images)
│ image2 = wireframe pose guide frame (optional)
├─ Patch workflow graph nodes (prompt, size, seed, sampler)
├─ POST to ComfyUI /prompt → poll /history until done
├─ Fetch PNG from ComfyUI /view
└─ is_transparent?
YES → _apply_transparency_sam2(png_bytes)
├─ SAM2 point-prompt segmentation
│ FG: 12 points (hair→shoes, lateral spread for ¾ poses)
│ BG: 7 points (corners + edge midpoints)
│ Coverage check: 5%92% (else fallback)
│ Gaussian blur radius=1 for soft edges
└─ fallback: _apply_transparency (rembg U2Net)
```
---
## Batch Worker (`_batch_worker`)
Runs in a background thread pool. One thread per batch job item.
```
For each (filename, prompt) pair:
1. Load source image from output dir
2. If wireframe_ref: extract frame at wireframe_time via OpenCV VideoCapture
3. Call _run_pipeline(pil, prompt, seed, max_area, extra_images=[wireframe_frame])
4. Save PNG to output dir (timestamped filename)
5. upsert_person(new_filename, group_id, prompt, source_refs=[original])
6. Update job progress dict (polled via /jobs/{id})
```
---
## Camera Angle Presets
Eight absolute + four relative angle prompts defined as `CAMERA_ANGLES` in car.html. Each triggers a `/batch` request with `pose=null` and the angle prompt injected.
| Name | Direction |
|------|-----------|
| Front | Straight-on frontal |
| ¾ Left / ¾ Right | 45° rotation |
| Side L / Side R | 90° profile |
| Back | Rear view |
| High / Low | Bird's-eye / worm's-eye |
| ↺ 45° / ↻ 45° | Relative +45° left / right |
| ↺ 90° / ↻ 90° | Relative +90° left / right |
---
## Background Removal
Two strategies are available:
### Configurable BG removal — `config.json` key `"bg_removal"`
Set to `"rembg"` (default) or `"sam2"`.
#### `"rembg"` mode
- Strips transparent-related keywords from the Qwen prompt entirely → Qwen renders with a natural real background
- `_apply_transparency` (rembg U2Net) post-processes the result
- Most reliable on complex Qwen-generated backgrounds (streets, shadows, cobblestones, etc.)
#### `"sam2"` mode
- Replaces transparent keywords with `"black background"` in the Qwen prompt → Qwen renders the person against a pure black background
- Negative prompt discourages any real-background content
- `_apply_transparency_sam2` (SAM2 bbox) post-processes: pure black gives maximal contrast → reliable segmentation
- Tested: produces clean full-body mask at ~74% coverage including hair, shoes, and earrings
#### Explicit `"black background"` in prompt
- Regardless of the configured method, always routes to SAM2 — the user has already set up the ideal SAM2 input
- Prompt is passed to Qwen unchanged
#### SAM2 implementation (`_apply_transparency_sam2`)
- Model: `sam2.1_hiera_base_plus.pt` at `~/.sam/sam2.1_hiera_base_plus.pt`
- Bbox `[1%, 1%, 99%, 99%]`, `multimask_output=True`, best-score mask, coverage check 5%92%, Gaussian edge blur radius=1
- Also called by the explicit `/remove-background-sam/{filename}` endpoint
---
## Face Processing (insightface)
Model: `buffalo_l`
**Faceswap** (`/faceswap`):
- Reads source face from reference image
- Swaps into each frame of a target video via OpenCV
- Optionally runs GFPGAN face restoration post-swap
**Face extraction** (`_extract_face_bg` background task):
- Triggered on `set-preferred` or explicit `extract-face` endpoint
- Detects largest face via insightface, adds padding (50% sides, 200% top headroom)
- Saves as `{group_id}_face.png` in output dir
- DB-registered for display in the studio Info tab face-book thumbnail
---
## Database Schema (`person` table)
| Column | Type | Purpose |
|--------|------|---------|
| filename | TEXT PK | Unique image filename |
| filepath | TEXT | Absolute file path |
| group_id | TEXT | Group membership |
| group_name | TEXT | Human-readable group label |
| sort_order | INTEGER | Position within group (0 = preferred) |
| name | TEXT | Person / character name |
| prompt | TEXT | Generation prompt used |
| pose | TEXT | Pose tag |
| hidden | BOOLEAN | Hidden from studio preview |
| archived | BOOLEAN | Soft-deleted |
| has_background | BOOLEAN | Whether BG was removed |
| has_clothing | BOOLEAN | Content classification flag |
| source_refs | TEXT (JSON) | Original filenames this was derived from |
| content_type | TEXT | `image` or `video` |
| faceswap_source_video | TEXT | Source video for faceswapped clips |
| face_embedding | `vector(512)` | Face recognition embedding for visually matching character faces |
| is_source | `BOOLEAN` | Whether the file is a primary reference/source asset |
| tags | TEXT (JSON) | Custom user-applied tags/labels (e.g., `['VISIBLE', 'LIKE', '21+']`) |
| pose_description | TEXT | AI-generated descriptive text of the body pose |
| pose_skeleton | TEXT | Coordinates and scoring details of the 2D COCO-17 body pose skeleton |
---
## Frontend (car.html)
Single-page application (~5000 lines). No build step — pure HTML/CSS/JS.
**Modes**:
- **Gallery** — grid of all groups; click to open Studio
- **Studio** — filmstrip + large viewer + sidebar tabs
- **Database Consistency** (`trash.html` SPA) — dedicated panel for identifying legacy/orphaned records, untracked files, and manual restore/delete of archived assets
**Sidebar tabs**:
| Tab | Purpose |
|-----|---------|
| Generate | Camera angle picker, pose selector (with inline edit/add/delete → `poses.md`), wireframe guide, auto-growing prompt override with history autocomplete, fine-tune textareas, multi-ref thumbnails (click to deselect), batch submit |
| Info | Metadata, preferred toggle, face-book thumbnail, hide/archive/delete, manual crop/expand padding (copy or in-place), rotate ±90°, No-BG / Invert-α, duplicate, Pose preview (draggable + similar-pose search) |
| Faceswap | Source face selector, target video, preview quality toggle |
| Segment | SAM2 and rembg BG removal buttons |
| Scenery | Scenery video reference for generation |
### Key Studio Features
#### Canvas Expansion & Manual Padding (Interactive Tooling)
The **Info** sidebar and dedicated toolbar provide professional, reactive manual padding controls:
- **Percent/Pixel Harmony**: Seamlessly accepts both absolute integer pixel values (e.g., `100px`) and percentage-based inputs (e.g., `10%` or `0.1`), ensuring perfect formatting and suffix preservation.
- **Quick Adjustment Presets**: Rapidly adjust margins with single-click actions for `+10%`, `+20%`, `+50%`, `+100px`, or `Reset` to clear all.
- **Uniform Locking**: Enabling the **Uniform** option synchronizes changes on any single padding field to all four sides simultaneously.
- **Intelligent Previews**: Dynamically renders real-time color-coded canvas border previews based on the selected fill type (Amber dashed for `transparent`, translucent black for `black`, and translucent white for `white`).
- **Shortcuts**: Confirm cropping or padding instantly with `Enter`, or dismiss/cancel the operation using `Escape`.
#### Reactive Layout Optimization
To prevent desyncs during image transitions, crops, canvas expansion, or 2.2x zooming:
- A `ResizeObserver` is registered on the primary studio viewer image (`lbImg`).
- Automatically triggers a layout and geometry recalculation, cleanly and immediately adjusting the background transparency checkerboard grid and padding visual overlays to match the actual visual boundaries of the contained image.
#### Multiple Filmstrips System
The filmstrip bar has been updated with five tabs that filter the group's images dynamically:
- **Group Active**: The default filmstrip containing the main/preferred assets of the group.
- **Group Variant** (`VISIBLE`): Shows all standard visible variants within the group.
- **Hidden** (`HIDDEN`): Soft-hidden variants (excluded from main cycling).
- **Source** (`SOURCE`): Displays reference images explicitly marked as source assets.
- **Archived** (`ARCHIVED`): Displays soft-archived variants.
- **Drag-and-Drop Reclassification**: Users can drag any variant thumbnail from the filmstrip and drop it directly onto any of the tab buttons to instantly update its tag/status and reassign it to that list.
#### Bulk Multi-Select & Actions
Toggling the **Multi-select** button unlocks checkbox-based selection across the filmstrip:
- Provides a live selection counter.
- Enables bulk move/reclassification buttons to dispatch batch updates to `/images/bulk-move` (e.g., `Move to Group`, `Move to Hidden`, `Move to Source`, `Move to Archived`).
#### Seamless Non-Blinking State Updates
To prevent irritating browser reload blinks or blank screen latency during intensive image editing operations:
- **Local Metadata Cloning**: Duplication, manual cropping, padding, and background removal immediately clone the original's DB metadata values locally on the client and splice the new file record into the active filmstrip list.
- **Optimistic Merge**: During periodic background server syncs (`refreshNow`), an optimistic merge algorithm compares the local client-side list and the server JSON. It preserves any freshly created local assets that haven't yet been processed/synced on the backend, maintaining perfect continuity.
#### Studio Keyboard Shortcuts Reference
| Key | Action | Description |
|-----|--------|-------------|
| `P` | Toggle Privacy Mode | Replaces the page with a discrete "AI Assist" study-chat and disguises taskbar thumbnails. |
| `C` | Toggle Checkerboard | Toggle transparent background checkerboard grid. |
| `A` | Expand/Pad Canvas | Instantly trigger the manual padding toolbar. |
| `G` / `Alt+Enter` | Trigger Generation | Clicks the main "Generate" button (when Generate tab is active). |
| `ArrowLeft` / `ArrowRight` | Nav Filmstrip | Cycle through the current list of image variants (debounced). |
| `Home` / `End` | Jump to Bounds | Instantly jump to the first or last variant in the filmstrip. |
| `Space` | Play/Pause Video | Pauses or plays a running wireframe/clip inside the viewer. |
| `H` | Toggle Hidden | Reclasses the image between standard variants and hidden list. |
| `F` | Set as Preferred | Reorders the group, placing the current image at index 0 (as preferred thumbnail). |
| `S` | Toggle Source | Mark/unmark current image as a primary reference/source asset. |
| `E` | Run 21+ Tagger | Analyze image via WD tagger and apply/remove the `21+` tag. |
| `Delete` | Soft-Archive | Move the current image into the Archive filmstrip. |
| `ArrowUp` / `ArrowDown` | Shift Order | Move/reorder the current image earlier or later in the group (debounced). |
| `2` | Move to Second | Instantly reorder current image to the 2nd position in the group. |
| `9` | Move to Last-1 | Instantly reorder current image to the second to last position. |
| `Escape` | Cancel / Dismiss | Cancel cropping, padding, close variant picker, or exit selection mode. |
**Key state variables**:
- `lbCurrentGid` — active group ID
- `lbIdx` — current filmstrip position
- `_followLatestGid` — auto-follow newly generated image
- `_followLatestFilename` — auto-follow specific new output file
- `_sbSelectedAngles` — selected camera angle names
- `_sbWireframeRef` / `_sbWireframeTime` — wireframe video + frame time
- `privacyMode` — privacy overlay toggle (key: `P`)
---
## Wireframe Pose Guide
Wireframe videos stored in `/mnt/zim/tour-comfy/wireframe/`.
The studio Generate tab shows a video picker + time scrubber. On generation:
1. Backend calls `GET /wireframe/frame/{name}?t=0.5` to extract the chosen frame
2. Frame passed as `image2` in the ComfyUI workflow (second visual reference)
3. No ControlNet installed — Qwen uses the wireframe as a layout hint via `image2` slot
---
## Pose Tools (2D body pose)
A posenet-style toolset built on a feature-detected estimator. Preferred backend is
**rtmlib** (RTMPose, ONNX via the already-installed `onnxruntime`); **mediapipe** is a
fallback. If neither is installed the endpoints return `501` and the UI shows an install hint.
- **Install (A6000 venv):** `~/comfyui/venv/bin/pip install rtmlib`. Models (~150 MB) auto-
download to `~/.cache/rtmlib` on first use.
- **CUDA libs:** `onnxruntime-gpu` needs the venv's `nvidia-*-cu12` libs on the loader path.
`a6000-comfy/start_api.sh` adds them to `LD_LIBRARY_PATH` at launch (torch finds them via
RPATH, onnxruntime does not).
**Output format:** COCO-17 keypoints (`POSE_KEYPOINT_NAMES`) + `POSE_SKELETON` edge list,
returned in image pixels with per-joint score.
**Studio overlay (`car.html`):** the **Pose** button overlays the skeleton on the viewer
(letterbox-aware canvas). Joints are **draggable** to refine/explore a pose. Overlay toolbar:
*Reset* (re-detect) and *Similar pose* (rank the library by the current — possibly edited —
skeleton). Results render as a thumbnail strip; clicking one opens that image's group.
**Similarity model (`edit_api.py`):**
- `_pose_descriptor(keypoints)` → translation/scale-invariant vector: centered on hip midpoint
(fallback shoulders), scaled by torso length, 17×(x,y) + visibility mask. `None` if < 6 joints.
- `_pose_distance(a, b)` → weighted L2 over jointly-visible joints, taking the min of the direct
and the **left-right-mirrored** comparison so mirrored poses match.
- **Pose index:** descriptors are cached in `<output>/_data/poses_index.json`. It fills
incrementally whenever `/images/{f}/pose` runs, or in bulk via `/pose/index` (daemon thread,
batched writes every 50, progress via `/pose/index/status`).
## File Layout
```
qwen-image-edit-rapid-aio-nsfw-v23/
├── tour-comfy/
│ ├── car.html # frontend SPA (Studio Interface)
│ ├── trash.html # frontend Database Consistency Panel SPA
│ ├── edit_api.py # FastAPI backend
│ ├── database.py # PostgreSQL helpers
│ ├── watcher.py # output dir auto-processor
│ ├── config.json # output_dir, comfy_url, etc.
│ └── workflow_qwen_edit.json # ComfyUI graph template
├── a6000-comfy/ # ComfyUI install (symlinked to ~/comfyui)
├── AGENTS.md # this file (System Documentation)
├── ARCHETECTURE.svg # system architecture diagram
├── backlog.md # feature backlog
└── requirements.txt # Python deps
```
---
## Configuration (`config.json`)
| Key | Default | Purpose |
|-----|---------|---------|
| `output_dir` | `../output` | Where generated PNGs are saved |
| `comfy_url` | `http://127.0.0.1:8188` | ComfyUI base URL |
| `host` | `0.0.0.0` | FastAPI bind host |
| `port` | `8500` | FastAPI bind port |
| `max_area` | `1048576` | Max pixel budget (1 MP) |
---
## Background Services
| Service | Script | Purpose |
|---------|--------|---------|
| `comfyui-backend` | `a6000-comfy/run_comfyui.sh` | Qwen GGUF inference |
| `comfyui-api` | `tour-comfy/start_api.sh` | FastAPI HTTP server |
| `watcher` | `tour-comfy/start_watcher.sh` | Monitors output dir, auto-processes dropped images |