• Implemented global offline mode for all HuggingFace-dependent modules to eliminate unauthenticated request warnings and prevent external connection attempts during startup and runtime. Changes • Global Offline Configuration: Added HF_HUB_OFFLINE=1 and TRANSFORMERS_OFFLINE=1 to the environment variables at the top of edit_api.py, embeddings.py, watcher.py, and pose_llm/pose_llm_api.py. • Explicit Local Files Only: Updated all huggingface_hub.hf_hub_download calls in edit_api.py to include local_files_only=True, ensuring they never attempt to reach the Hub if models are already cached. • LLM Service Optimization: Updated AutoTokenizer and AutoModelForCausalLM calls in pose_llm/pose_llm_api.py to use local_files_only=True. • Consistency: Ensured that shared modules like embeddings.py which uses open_clip are also locked to offline mode to prevent background downloads.
504 lines
27 KiB
Markdown
504 lines
27 KiB
Markdown
# Qwen-Image-Edit Rapid-AIO v23 — System Documentation
|
||
|
||
## Overview
|
||
|
||
A self-hosted AI image studio for generative editing, multi-view character rendering, background removal, face swapping, and group-based asset management. It runs the **Qwen 2.5 VL GGUF** model through ComfyUI and exposes a feature-rich browser frontend backed by a FastAPI service.
|
||
|
||
```
|
||
Browser (car.html SPA)
|
||
│ HTTP / fetch
|
||
▼
|
||
FastAPI :8500 (edit_api.py)
|
||
├─── ComfyUI :8188 ← Qwen GGUF model
|
||
├─── SAM2 / rembg ← background removal
|
||
├─── insightface ← faceswap + face extraction
|
||
├─── GFPGAN ← face restoration
|
||
└─── PostgreSQL ← metadata (192.168.1.160:5433)
|
||
│
|
||
File system (/mnt/zim/tour-comfy/output/, wireframe/)
|
||
```
|
||
|
||
---
|
||
|
||
## Hardware
|
||
|
||
| Component | Spec |
|
||
|-----------|------|
|
||
| GPU | NVIDIA RTX A6000 (48 GB VRAM) |
|
||
| CUDA | 12.4 |
|
||
| PyTorch | 2.6.0+cu124 |
|
||
| ComfyUI | latest pinned (a6000-comfy/) |
|
||
|
||
---
|
||
|
||
## Services
|
||
|
||
### 1. ComfyUI Backend — `:8188`
|
||
|
||
- **Script**: `a6000-comfy/run_comfyui.sh`
|
||
- **Systemd**: `comfyui-backend`
|
||
- **Purpose**: Executes the Qwen GGUF model via `workflow_qwen_edit.json`
|
||
|
||
**Loaded models**
|
||
|
||
| Role | File |
|
||
|------|------|
|
||
| GGUF LLM | `Qwen-Rapid-NSFW-v23_Q8_0.gguf` |
|
||
| Text encoder | `qwen_2.5_vl_7b_fp8_scaled.safetensors` |
|
||
| VAE | `qwen_image_vae.safetensors` |
|
||
|
||
**Workflow nodes** (`workflow_qwen_edit.json`):
|
||
- `NODE_POSITIVE` — positive text prompt
|
||
- `NODE_NEGATIVE` — negative prompt
|
||
- `NODE_LATENT` — latent dimensions (width × height)
|
||
- `NODE_KSAMPLER` — sampler params (Euler/karras, seed, steps, CFG)
|
||
- `image1` — reference image (base64 uploaded to ComfyUI `/upload/image`)
|
||
- `image2` — optional wireframe pose guide (extracted from wireframe video)
|
||
|
||
---
|
||
|
||
### 2. FastAPI API — `:8500`
|
||
|
||
- **Script**: `tour-comfy/edit_api.py`
|
||
- **Systemd**: `comfyui-api`
|
||
- **DB**: PostgreSQL at `192.168.1.160:5433`, database `dv`, table `person`
|
||
|
||
#### In-Memory File Metadata Cache
|
||
|
||
To mitigate performance and network latency bottlenecks on remote/shared storage filesystems (e.g. `/mnt/zim`), the API server employs a light, thread-safe in-memory metadata cache (`_file_meta_cache`):
|
||
- **Cached Items**: File existence (`exists`) and modification timestamps (`mtime`) mapped by `filename`.
|
||
- **Cache TTL**: 5.0 seconds.
|
||
- **Idempotency & Invalidation**: Any mutation endpoints (such as crop, padding, duplication, background removal, or deletion) automatically update (`_update_cached_file_meta`) or explicitly delete (`_clear_cached_file_meta`) entries to keep cache and disk perfectly in sync.
|
||
- **Preloading Optimization**: Leverages cache queries in static indexing loops (`_sync_preloaded_images`, `_write_all_static`, `list_images`) and tracks changes to completely skip redundant static HTML rewrites if the preloaded image set is identical (`_last_preloaded_images_set`).
|
||
|
||
#### Endpoints Library
|
||
|
||
##### Core Generation & Upload
|
||
| Method | Path | Purpose |
|
||
|--------|------|---------|
|
||
| `POST` | `/upload` | Upload image; optionally add to existing group (`group_id` + `skip_poses`) |
|
||
| `POST` | `/batch` | Queue multi-prompt generation job (async) |
|
||
| `GET` | `/batch/{job_id}` | Poll batch job status / per-item progress |
|
||
| `DELETE` | `/batch/{job_id}` | Cancel active batch job |
|
||
| `POST` | `/multi-ref` | Queue a generation job with multiple reference images |
|
||
| `POST` | `/refine-prompt` | Refine a generation prompt using the external uncensored chat LLM |
|
||
| `POST` | `/edit` | Edit uploaded image |
|
||
| `POST` | `/generate-scenery` | Generate scenery background based on video frame + reference image |
|
||
| `GET` | `/scenery/library` | Return all scenery images grouped by source video reference |
|
||
|
||
##### Image Editing & Canvas Expansion (Padding & Crop)
|
||
| Method | Path | Purpose |
|
||
|--------|------|---------|
|
||
| `POST` | `/images/{filename}/crop` | Crop to pixel rect; `as_copy:true` crops a referenced copy instead of modifying in-place |
|
||
| `POST` | `/images/{filename}/autocrop` | Auto-trim transparent borders in-place |
|
||
| `POST` | `/images/{filename}/pad` | Expand image canvas with custom padding on each side, optional background fills (`transparent`, `black`, `white`), and optional Qwen-based Outpainting (`outpaint:true`) |
|
||
| `POST` | `/images/{filename}/rotate` | Rotate clockwise in 90° steps (lossless transpose, in-place) |
|
||
| `POST` | `/images/{filename}/invert-alpha` | Invert the alpha channel (recovers background removal errors) |
|
||
| `POST` | `/images/{filename}/duplicate` | Copy image into the same group with timestamp-based filename and `source_refs=[original]` |
|
||
| `POST` | `/restore-background/{filename}` | Flatten RGBA to opaque RGB via white composite background |
|
||
|
||
##### Background Removal
|
||
| Method | Path | Purpose |
|
||
|--------|------|---------|
|
||
| `POST` | `/remove-background/{filename}` | rembg BG removal, overwrites in-place; sets `has_background=false` + refreshes static data |
|
||
| `POST` | `/remove-background-sam/{filename}` | SAM2 BG removal → `.nobg.png` sidecar |
|
||
| `POST` | `/remove-background/group/{group_id}` | Queue background removal for all group members as an async background task |
|
||
| `GET` | `/sam2/check` | Return whether SAM2 background remover is available |
|
||
|
||
##### Metadata, Custom Labels & Group Management
|
||
| Method | Path | Purpose |
|
||
|--------|------|---------|
|
||
| `GET` | `/images` | List all images with DB metadata |
|
||
| `GET` | `/images/{filename}` | Serve image file |
|
||
| `GET` | `/groups` | List all groups with members |
|
||
| `GET` | `/group-names` | Return map of custom group names |
|
||
| `POST` | `/group-names/{group_id}` | Save custom group name |
|
||
| `POST` | `/groups/merge` | Merge several groups into one group |
|
||
| `POST` | `/groups/extract` | Extract a file into its own solo group |
|
||
| `GET` | `/groups/{group_id}/order` | Return order of members in a group |
|
||
| `POST` | `/groups/{group_id}/order` | Set custom sort order of group members |
|
||
| `GET` | `/similar/{filename}` | Find visually similar images using database embeddings |
|
||
| `GET` | `/names` | Map `filename → person name` |
|
||
| `POST` | `/names/{filename}` | Map image file to custom character name |
|
||
| `POST` | `/images/{filename}/tags` | Set image tags list (supports tags like `ARCHIVED`, `HIDDEN`, `SOURCE`, `VISIBLE`, `LIKE`, `DISLIKE`, `21+`) |
|
||
| `POST` | `/images/{filename}/tag-action` | Add, remove, or toggle a specific tag on an image (automatically handles mutual exclusivity of `LIKE`/`DISLIKE`) |
|
||
| `POST` | `/images/{filename}/source` | Explicitly toggle marking an image as a source asset (sets/clears `is_source` and toggles the `SOURCE` tag) |
|
||
| `POST` | `/images/{filename}/estimate-21plus` | Analyze image with the WD tagger to estimate adult/sensitive rating and apply `21+` tag |
|
||
| `POST` | `/images/bulk-move` | Move a list of filenames to a specific target filmstrip (`VISIBLE`, `HIDDEN`, `ARCHIVED`, `SOURCE`) in bulk, updating tags/columns |
|
||
| `POST` | `/images/{filename}/undress` | Queue an asynchronous batch job utilizing the `UNDRESS_PROMPT` on the given reference image |
|
||
|
||
##### Visibility, Archiving & Deletion
|
||
| Method | Path | Purpose |
|
||
|--------|------|---------|
|
||
| `POST` | `/images/{filename}/hidden` | Toggle studio visibility (soft-hide) |
|
||
| `POST` | `/images/{filename}/archive` | Soft-archive image (hidden from default view) |
|
||
| `POST` | `/images/{filename}/unarchive` | Restore soft-archived image |
|
||
| `DELETE` | `/images/{filename}` | Delete image file + DB record |
|
||
| `POST` | `/groups/{group_id}/archive` | Soft-archive an entire group |
|
||
| `POST` | `/groups/{group_id}/unarchive` | Restore an archived group |
|
||
| `DELETE` | `/groups/{group_id}` | Delete entire group files and DB records |
|
||
|
||
##### 2D Body Pose Tools (rtmlib / RTMPose)
|
||
| Method | Path | Purpose |
|
||
|--------|------|---------|
|
||
| `GET` | `/pose/check` | Report pose-estimator availability + backend (rtmlib or mediapipe) |
|
||
| `POST` | `/images/{filename}/pose` | Estimate COCO-17 body keypoints (caches descriptor for similar-search) |
|
||
| `GET` | `/pose/similar/{filename}` | Rank library images by pose similarity to this image |
|
||
| `POST` | `/pose/similar` | Rank library images by similarity to a supplied (edited) skeleton |
|
||
| `POST` | `/pose/index` | Build pose descriptors for the whole library (daemon thread) |
|
||
| `GET` | `/pose/index/status` | Poll pose-index build progress |
|
||
| `POST` | `/pose/from-wireframe` | Extract frame from a wireframe video and estimate body pose keypoints |
|
||
| `POST` | `/pose/render` | Render keypoints as OpenPose skeleton image (base64) |
|
||
| `POST` | `/generate-with-pose` | Generate image from a specific pose skeleton (avoids wireframe bleed-through) |
|
||
| `GET` | `/poses` | Load pose library from `poses.md` |
|
||
| `POST` | `/poses` | Create / update / rename a pose (`old_name` to rename) → rewrites `poses.md` |
|
||
| `DELETE` | `/poses/{name}` | Delete a pose from `poses.md` |
|
||
|
||
##### 3D Turntable & Orbit Previews
|
||
| Method | Path | Purpose |
|
||
|--------|------|---------|
|
||
| `POST` | `/orbit` | Queue an orbit preview generation. Engine `depth` (fast 2.5D card parallax) or `qwen` (turntable) |
|
||
| `GET` | `/turntable/status` | Return background turntable generation state/progress summary |
|
||
| `GET` | `/turntable/status/{group_id}` | Return turntable state for a specific group |
|
||
| `POST` | `/turntable/pause` | Pause background turntable generation |
|
||
| `POST` | `/turntable/resume` | Resume background turntable generation |
|
||
| `DELETE` | `/turntable/{group_id}` | Reset turntable cache & records for group to force re-generation |
|
||
|
||
##### Face Processing (insightface)
|
||
| Method | Path | Purpose |
|
||
|--------|------|---------|
|
||
| `POST` | `/faceswap` | insightface video faceswap |
|
||
| `GET` | `/faceswap/check` | Report available face enhancement backends |
|
||
| `POST` | `/images/{filename}/extract-face` | Extract & save face crop via insightface (background task) |
|
||
| `POST` | `/faces/similar` | Find groups with visually similar faces using embeddings |
|
||
| `POST` | `/faces/index` | Build face embedding index for all preferred faces in library |
|
||
| `GET` | `/faces/index/status` | Poll face-indexing progress status |
|
||
| `GET` | `/faces/{group_id}` | Check if face crop exists for group |
|
||
|
||
##### Videos & Wireframe Guides
|
||
| Method | Path | Purpose |
|
||
|--------|------|---------|
|
||
| `GET` | `/videos` | List all available wireframe guide videos |
|
||
| `GET` | `/wireframe/frame/{video_name}` | Extract frame at `?t=` (0–1) from wireframe video |
|
||
| `GET` | `/wireframe/duration/{video_name}` | Get video duration (seconds) via ffprobe |
|
||
| `POST` | `/wireframe/trim` | Trim a wireframe video to given start/end offsets |
|
||
| `POST` | `/wireframe/frame` | Extract frame at timestamp, returns base64 PNG |
|
||
| `POST` | `/generate-video` | Stitch list of output images into a short looping MP4 video |
|
||
|
||
##### Database, Config & System Diagnostics
|
||
| Method | Path | Purpose |
|
||
|--------|------|---------|
|
||
| `GET/POST` | `/config` | Read / write `config.json` |
|
||
| `GET` | `/db/inconsistencies` | Retrieve consistency report or run a check with `?run_now=true` |
|
||
| `POST` | `/db/repair` | Run repair actions (restore, import file, assign group, permanently delete) |
|
||
| `POST` | `/db/cleanup` | Delete DB records of files that are missing on disk |
|
||
| `GET` | `/health` | API health check status |
|
||
|
||
---
|
||
|
||
## Generation Pipeline (`_run_pipeline`)
|
||
|
||
```
|
||
prompt
|
||
│
|
||
├─ "transparent background" keywords detected?
|
||
│ YES → strip keywords from Qwen prompt (keeps `is_transparent` flag)
|
||
│ set negative: "deformed anatomy, watermark, logo"
|
||
│
|
||
├─ Canvas Expansion / Outpainting requested?
|
||
│ YES → if outpaint=True:
|
||
│ ├─ If source is transparent (RGBA): composite onto flat black background for maximum contrast.
|
||
│ ├─ Formulate dynamic prompt description: e.g. "replacing the black/white background areas"
|
||
│ └─ Append outpainting prompt instructions: "Naturally outpaint and extend..."
|
||
│
|
||
├─ Upload reference image(s) to ComfyUI /upload/image
|
||
│ image1 = source image (composited RGB if outpainting transparent images)
|
||
│ image2 = wireframe pose guide frame (optional)
|
||
│
|
||
├─ Patch workflow graph nodes (prompt, size, seed, sampler)
|
||
├─ POST to ComfyUI /prompt → poll /history until done
|
||
├─ Fetch PNG from ComfyUI /view
|
||
│
|
||
└─ is_transparent?
|
||
YES → _apply_transparency_sam2(png_bytes)
|
||
├─ SAM2 point-prompt segmentation
|
||
│ FG: 12 points (hair→shoes, lateral spread for ¾ poses)
|
||
│ BG: 7 points (corners + edge midpoints)
|
||
│ Coverage check: 5%–92% (else fallback)
|
||
│ Gaussian blur radius=1 for soft edges
|
||
└─ fallback: _apply_transparency (rembg U2Net)
|
||
```
|
||
|
||
---
|
||
|
||
## Batch Worker (`_batch_worker`)
|
||
|
||
Runs in a background thread pool. One thread per batch job item.
|
||
|
||
```
|
||
For each (filename, prompt) pair:
|
||
1. Load source image from output dir
|
||
2. If wireframe_ref: extract frame at wireframe_time via OpenCV VideoCapture
|
||
3. Call _run_pipeline(pil, prompt, seed, max_area, extra_images=[wireframe_frame])
|
||
4. Save PNG to output dir (timestamped filename)
|
||
5. upsert_person(new_filename, group_id, prompt, source_refs=[original])
|
||
6. Update job progress dict (polled via /jobs/{id})
|
||
```
|
||
|
||
---
|
||
|
||
## Camera Angle Presets
|
||
|
||
Eight absolute + four relative angle prompts defined as `CAMERA_ANGLES` in car.html. Each triggers a `/batch` request with `pose=null` and the angle prompt injected.
|
||
|
||
| Name | Direction |
|
||
|------|-----------|
|
||
| Front | Straight-on frontal |
|
||
| ¾ Left / ¾ Right | 45° rotation |
|
||
| Side L / Side R | 90° profile |
|
||
| Back | Rear view |
|
||
| High / Low | Bird's-eye / worm's-eye |
|
||
| ↺ 45° / ↻ 45° | Relative +45° left / right |
|
||
| ↺ 90° / ↻ 90° | Relative +90° left / right |
|
||
|
||
---
|
||
|
||
## Background Removal
|
||
|
||
Two strategies are available:
|
||
|
||
### Configurable BG removal — `config.json` key `"bg_removal"`
|
||
|
||
Set to `"rembg"` (default) or `"sam2"`.
|
||
|
||
#### `"rembg"` mode
|
||
- Strips transparent-related keywords from the Qwen prompt entirely → Qwen renders with a natural real background
|
||
- `_apply_transparency` (rembg U2Net) post-processes the result
|
||
- Most reliable on complex Qwen-generated backgrounds (streets, shadows, cobblestones, etc.)
|
||
|
||
#### `"sam2"` mode
|
||
- Replaces transparent keywords with `"black background"` in the Qwen prompt → Qwen renders the person against a pure black background
|
||
- Negative prompt discourages any real-background content
|
||
- `_apply_transparency_sam2` (SAM2 bbox) post-processes: pure black gives maximal contrast → reliable segmentation
|
||
- Tested: produces clean full-body mask at ~74% coverage including hair, shoes, and earrings
|
||
|
||
#### Explicit `"black background"` in prompt
|
||
- Regardless of the configured method, always routes to SAM2 — the user has already set up the ideal SAM2 input
|
||
- Prompt is passed to Qwen unchanged
|
||
|
||
#### SAM2 implementation (`_apply_transparency_sam2`)
|
||
- Model: `sam2.1_hiera_base_plus.pt` at `~/.sam/sam2.1_hiera_base_plus.pt`
|
||
- Bbox `[1%, 1%, 99%, 99%]`, `multimask_output=True`, best-score mask, coverage check 5%–92%, Gaussian edge blur radius=1
|
||
- Also called by the explicit `/remove-background-sam/{filename}` endpoint
|
||
|
||
---
|
||
|
||
## Face Processing (insightface)
|
||
|
||
Model: `buffalo_l`
|
||
|
||
**Faceswap** (`/faceswap`):
|
||
- Reads source face from reference image
|
||
- Swaps into each frame of a target video via OpenCV
|
||
- Optionally runs GFPGAN face restoration post-swap
|
||
|
||
**Face extraction** (`_extract_face_bg` background task):
|
||
- Triggered on `set-preferred` or explicit `extract-face` endpoint
|
||
- Detects largest face via insightface, adds padding (50% sides, 200% top headroom)
|
||
- Saves as `{group_id}_face.png` in output dir
|
||
- DB-registered for display in the studio Info tab face-book thumbnail
|
||
|
||
---
|
||
|
||
## Database Schema (`person` table)
|
||
|
||
| Column | Type | Purpose |
|
||
|--------|------|---------|
|
||
| filename | TEXT PK | Unique image filename |
|
||
| filepath | TEXT | Absolute file path |
|
||
| group_id | TEXT | Group membership |
|
||
| group_name | TEXT | Human-readable group label |
|
||
| sort_order | INTEGER | Position within group (0 = preferred) |
|
||
| name | TEXT | Person / character name |
|
||
| prompt | TEXT | Generation prompt used |
|
||
| pose | TEXT | Pose tag |
|
||
| hidden | BOOLEAN | Hidden from studio preview |
|
||
| archived | BOOLEAN | Soft-deleted |
|
||
| has_background | BOOLEAN | Whether BG was removed |
|
||
| has_clothing | BOOLEAN | Content classification flag |
|
||
| source_refs | TEXT (JSON) | Original filenames this was derived from |
|
||
| content_type | TEXT | `image` or `video` |
|
||
| faceswap_source_video | TEXT | Source video for faceswapped clips |
|
||
| face_embedding | `vector(512)` | Face recognition embedding for visually matching character faces |
|
||
| is_source | `BOOLEAN` | Whether the file is a primary reference/source asset |
|
||
| tags | TEXT (JSON) | Custom user-applied tags/labels (e.g., `['VISIBLE', 'LIKE', '21+']`) |
|
||
| pose_description | TEXT | AI-generated descriptive text of the body pose |
|
||
| pose_skeleton | TEXT | Coordinates and scoring details of the 2D COCO-17 body pose skeleton |
|
||
|
||
---
|
||
|
||
## Frontend (car.html)
|
||
|
||
Single-page application (~5000 lines). No build step — pure HTML/CSS/JS.
|
||
|
||
**Modes**:
|
||
- **Gallery** — grid of all groups; click to open Studio
|
||
- **Studio** — filmstrip + large viewer + sidebar tabs
|
||
- **Database Consistency** (`trash.html` SPA) — dedicated panel for identifying legacy/orphaned records, untracked files, and manual restore/delete of archived assets
|
||
|
||
**Sidebar tabs**:
|
||
|
||
| Tab | Purpose |
|
||
|-----|---------|
|
||
| Generate | Camera angle picker, pose selector (with inline edit/add/delete → `poses.md`), wireframe guide, auto-growing prompt override with history autocomplete, fine-tune textareas, multi-ref thumbnails (click to deselect), batch submit |
|
||
| Info | Metadata, preferred toggle, face-book thumbnail, hide/archive/delete, manual crop/expand padding (copy or in-place), rotate ±90°, No-BG / Invert-α, duplicate, Pose preview (draggable + similar-pose search) |
|
||
| Faceswap | Source face selector, target video, preview quality toggle |
|
||
| Segment | SAM2 and rembg BG removal buttons |
|
||
| Scenery | Scenery video reference for generation |
|
||
|
||
### Key Studio Features
|
||
|
||
#### Canvas Expansion & Manual Padding (Interactive Tooling)
|
||
The **Info** sidebar and dedicated toolbar provide professional, reactive manual padding controls:
|
||
- **Percent/Pixel Harmony**: Seamlessly accepts both absolute integer pixel values (e.g., `100px`) and percentage-based inputs (e.g., `10%` or `0.1`), ensuring perfect formatting and suffix preservation.
|
||
- **Quick Adjustment Presets**: Rapidly adjust margins with single-click actions for `+10%`, `+20%`, `+50%`, `+100px`, or `Reset` to clear all.
|
||
- **Uniform Locking**: Enabling the **Uniform** option synchronizes changes on any single padding field to all four sides simultaneously.
|
||
- **Intelligent Previews**: Dynamically renders real-time color-coded canvas border previews based on the selected fill type (Amber dashed for `transparent`, translucent black for `black`, and translucent white for `white`).
|
||
- **Shortcuts**: Confirm cropping or padding instantly with `Enter`, or dismiss/cancel the operation using `Escape`.
|
||
|
||
#### Reactive Layout Optimization
|
||
To prevent desyncs during image transitions, crops, canvas expansion, or 2.2x zooming:
|
||
- A `ResizeObserver` is registered on the primary studio viewer image (`lbImg`).
|
||
- Automatically triggers a layout and geometry recalculation, cleanly and immediately adjusting the background transparency checkerboard grid and padding visual overlays to match the actual visual boundaries of the contained image.
|
||
|
||
#### Multiple Filmstrips System
|
||
The filmstrip bar has been updated with five tabs that filter the group's images dynamically:
|
||
- **Group Active**: The default filmstrip containing the main/preferred assets of the group.
|
||
- **Group Variant** (`VISIBLE`): Shows all standard visible variants within the group.
|
||
- **Hidden** (`HIDDEN`): Soft-hidden variants (excluded from main cycling).
|
||
- **Source** (`SOURCE`): Displays reference images explicitly marked as source assets.
|
||
- **Archived** (`ARCHIVED`): Displays soft-archived variants.
|
||
- **Drag-and-Drop Reclassification**: Users can drag any variant thumbnail from the filmstrip and drop it directly onto any of the tab buttons to instantly update its tag/status and reassign it to that list.
|
||
|
||
#### Bulk Multi-Select & Actions
|
||
Toggling the **Multi-select** button unlocks checkbox-based selection across the filmstrip:
|
||
- Provides a live selection counter.
|
||
- Enables bulk move/reclassification buttons to dispatch batch updates to `/images/bulk-move` (e.g., `Move to Group`, `Move to Hidden`, `Move to Source`, `Move to Archived`).
|
||
|
||
#### Seamless Non-Blinking State Updates
|
||
To prevent irritating browser reload blinks or blank screen latency during intensive image editing operations:
|
||
- **Local Metadata Cloning**: Duplication, manual cropping, padding, and background removal immediately clone the original's DB metadata values locally on the client and splice the new file record into the active filmstrip list.
|
||
- **Optimistic Merge**: During periodic background server syncs (`refreshNow`), an optimistic merge algorithm compares the local client-side list and the server JSON. It preserves any freshly created local assets that haven't yet been processed/synced on the backend, maintaining perfect continuity.
|
||
|
||
#### Studio Keyboard Shortcuts Reference
|
||
|
||
| Key | Action | Description |
|
||
|-----|--------|-------------|
|
||
| `P` | Toggle Privacy Mode | Replaces the page with a discrete "AI Assist" study-chat and disguises taskbar thumbnails. |
|
||
| `C` | Toggle Checkerboard | Toggle transparent background checkerboard grid. |
|
||
| `A` | Expand/Pad Canvas | Instantly trigger the manual padding toolbar. |
|
||
| `G` / `Alt+Enter` | Trigger Generation | Clicks the main "Generate" button (when Generate tab is active). |
|
||
| `ArrowLeft` / `ArrowRight` | Nav Filmstrip | Cycle through the current list of image variants (debounced). |
|
||
| `Home` / `End` | Jump to Bounds | Instantly jump to the first or last variant in the filmstrip. |
|
||
| `Space` | Play/Pause Video | Pauses or plays a running wireframe/clip inside the viewer. |
|
||
| `H` | Toggle Hidden | Reclasses the image between standard variants and hidden list. |
|
||
| `F` | Set as Preferred | Reorders the group, placing the current image at index 0 (as preferred thumbnail). |
|
||
| `S` | Toggle Source | Mark/unmark current image as a primary reference/source asset. |
|
||
| `E` | Run 21+ Tagger | Analyze image via WD tagger and apply/remove the `21+` tag. |
|
||
| `Delete` | Soft-Archive | Move the current image into the Archive filmstrip. |
|
||
| `ArrowUp` / `ArrowDown` | Shift Order | Move/reorder the current image earlier or later in the group (debounced). |
|
||
| `2` | Move to Second | Instantly reorder current image to the 2nd position in the group. |
|
||
| `9` | Move to Last-1 | Instantly reorder current image to the second to last position. |
|
||
| `Escape` | Cancel / Dismiss | Cancel cropping, padding, close variant picker, or exit selection mode. |
|
||
|
||
**Key state variables**:
|
||
- `lbCurrentGid` — active group ID
|
||
- `lbIdx` — current filmstrip position
|
||
- `_followLatestGid` — auto-follow newly generated image
|
||
- `_followLatestFilename` — auto-follow specific new output file
|
||
- `_sbSelectedAngles` — selected camera angle names
|
||
- `_sbWireframeRef` / `_sbWireframeTime` — wireframe video + frame time
|
||
- `privacyMode` — privacy overlay toggle (key: `P`)
|
||
|
||
---
|
||
|
||
## Wireframe Pose Guide
|
||
|
||
Wireframe videos stored in `/mnt/zim/tour-comfy/wireframe/`.
|
||
The studio Generate tab shows a video picker + time scrubber. On generation:
|
||
1. Backend calls `GET /wireframe/frame/{name}?t=0.5` to extract the chosen frame
|
||
2. Frame passed as `image2` in the ComfyUI workflow (second visual reference)
|
||
3. No ControlNet installed — Qwen uses the wireframe as a layout hint via `image2` slot
|
||
|
||
---
|
||
|
||
## Pose Tools (2D body pose)
|
||
|
||
A posenet-style toolset built on a feature-detected estimator. Preferred backend is
|
||
**rtmlib** (RTMPose, ONNX via the already-installed `onnxruntime`); **mediapipe** is a
|
||
fallback. If neither is installed the endpoints return `501` and the UI shows an install hint.
|
||
|
||
- **Install (A6000 venv):** `~/comfyui/venv/bin/pip install rtmlib`. Models (~150 MB) auto-
|
||
download to `~/.cache/rtmlib` on first use.
|
||
- **CUDA libs:** `onnxruntime-gpu` needs the venv's `nvidia-*-cu12` libs on the loader path.
|
||
`a6000-comfy/start_api.sh` adds them to `LD_LIBRARY_PATH` at launch (torch finds them via
|
||
RPATH, onnxruntime does not).
|
||
|
||
**Output format:** COCO-17 keypoints (`POSE_KEYPOINT_NAMES`) + `POSE_SKELETON` edge list,
|
||
returned in image pixels with per-joint score.
|
||
|
||
**Studio overlay (`car.html`):** the **Pose** button overlays the skeleton on the viewer
|
||
(letterbox-aware canvas). Joints are **draggable** to refine/explore a pose. Overlay toolbar:
|
||
*Reset* (re-detect) and *Similar pose* (rank the library by the current — possibly edited —
|
||
skeleton). Results render as a thumbnail strip; clicking one opens that image's group.
|
||
|
||
**Similarity model (`edit_api.py`):**
|
||
- `_pose_descriptor(keypoints)` → translation/scale-invariant vector: centered on hip midpoint
|
||
(fallback shoulders), scaled by torso length, 17×(x,y) + visibility mask. `None` if < 6 joints.
|
||
- `_pose_distance(a, b)` → weighted L2 over jointly-visible joints, taking the min of the direct
|
||
and the **left-right-mirrored** comparison so mirrored poses match.
|
||
- **Pose index:** descriptors are cached in `<output>/_data/poses_index.json`. It fills
|
||
incrementally whenever `/images/{f}/pose` runs, or in bulk via `/pose/index` (daemon thread,
|
||
batched writes every 50, progress via `/pose/index/status`).
|
||
|
||
## File Layout
|
||
|
||
```
|
||
qwen-image-edit-rapid-aio-nsfw-v23/
|
||
├── tour-comfy/
|
||
│ ├── car.html # frontend SPA (Studio Interface)
|
||
│ ├── trash.html # frontend Database Consistency Panel SPA
|
||
│ ├── edit_api.py # FastAPI backend
|
||
│ ├── database.py # PostgreSQL helpers
|
||
│ ├── watcher.py # output dir auto-processor
|
||
│ ├── config.json # output_dir, comfy_url, etc.
|
||
│ └── workflow_qwen_edit.json # ComfyUI graph template
|
||
├── a6000-comfy/ # ComfyUI install (symlinked to ~/comfyui)
|
||
├── AGENTS.md # this file (System Documentation)
|
||
├── ARCHETECTURE.svg # system architecture diagram
|
||
├── backlog.md # feature backlog
|
||
└── requirements.txt # Python deps
|
||
```
|
||
|
||
---
|
||
|
||
## Configuration (`config.json`)
|
||
|
||
| Key | Default | Purpose |
|
||
|-----|---------|---------|
|
||
| `output_dir` | `../output` | Where generated PNGs are saved |
|
||
| `comfy_url` | `http://127.0.0.1:8188` | ComfyUI base URL |
|
||
| `host` | `0.0.0.0` | FastAPI bind host |
|
||
| `port` | `8500` | FastAPI bind port |
|
||
| `max_area` | `1048576` | Max pixel budget (1 MP) |
|
||
|
||
---
|
||
|
||
## Background Services
|
||
|
||
| Service | Script | Purpose |
|
||
|---------|--------|---------|
|
||
| `comfyui-backend` | `a6000-comfy/run_comfyui.sh` | Qwen GGUF inference |
|
||
| `comfyui-api` | `tour-comfy/start_api.sh` | FastAPI HTTP server |
|
||
| `watcher` | `tour-comfy/start_watcher.sh` | Monitors output dir, auto-processes dropped images |
|