dphn/Dolphin3.0-Mistral-24B is the ungated mirror of the Dolphin 3.0 Mistral 24B — exactly what you asked for. It's ~48GB fp16, which needs GPU+CPU split (device_map="auto" with 32GB on GPU, ~16GB in RAM). Let me kick off the download and update the service in parallel.
This commit is contained in:
51
AGENTS.md
51
AGENTS.md
@@ -78,15 +78,27 @@ FastAPI :8500 (edit_api.py)
|
||||
| `POST` | `/images/{filename}/unarchive` | Restore archived image |
|
||||
| `POST` | `/images/{filename}/set-hidden` | Toggle studio visibility |
|
||||
| `DELETE` | `/images/{filename}` | Delete image + DB record |
|
||||
| `POST` | `/images/{filename}/crop` | Crop to pixel rect (in-place) |
|
||||
| `POST` | `/autocrop/{filename}` | Auto-trim transparent border |
|
||||
| `POST` | `/images/{filename}/crop` | Crop to pixel rect; `as_copy:true` crops a referenced copy instead of in-place |
|
||||
| `POST` | `/images/{filename}/autocrop` | Auto-trim transparent border |
|
||||
| `POST` | `/images/{filename}/rotate` | Rotate clockwise in 90° steps (lossless transpose, in-place) |
|
||||
| `POST` | `/images/{filename}/invert-alpha` | Invert the alpha channel (fixes wrong-segment BG removal) |
|
||||
| `POST` | `/images/{filename}/duplicate` | Copy into same group with `source_refs=[original]` |
|
||||
| `POST` | `/faceswap` | insightface video faceswap |
|
||||
| `POST` | `/remove-background/{filename}` | rembg BG removal → `.nobg.png` sidecar |
|
||||
| `POST` | `/remove-background/{filename}` | rembg BG removal, overwrites in-place; sets `has_background=false` + refreshes static data |
|
||||
| `POST` | `/remove-background-sam/{filename}` | SAM2 BG removal → `.nobg.png` sidecar |
|
||||
| `GET` | `/wireframe/frame/{video_name}` | Extract frame at `?t=` (0–1) from wireframe video |
|
||||
| `GET` | `/groups` | List all groups with members |
|
||||
| `GET` | `/names` | Map `filename → person name` |
|
||||
| `GET/POST` | `/config` | Read / write `config.json` |
|
||||
| `GET` | `/poses` | Load pose library from `poses.md` |
|
||||
| `POST` | `/poses` | Create / update / rename a pose (`old_name` to rename) → rewrites `poses.md` |
|
||||
| `DELETE` | `/poses/{name}` | Delete a pose from `poses.md` |
|
||||
| `GET` | `/pose/check` | Report pose-estimator availability + backend |
|
||||
| `POST` | `/images/{filename}/pose` | Estimate COCO-17 body keypoints (caches descriptor for similar-search) |
|
||||
| `GET` | `/pose/similar/{filename}` | Rank library images by pose similarity to this image |
|
||||
| `POST` | `/pose/similar` | Rank library images by similarity to a supplied (edited) skeleton |
|
||||
| `POST` | `/pose/index` | Build pose descriptors for the whole library (daemon thread) |
|
||||
| `GET` | `/pose/index/status` | Poll pose-index build progress |
|
||||
|
||||
---
|
||||
|
||||
@@ -232,8 +244,8 @@ Single-page application (~5000 lines). No build step — pure HTML/CSS/JS.
|
||||
|
||||
| Tab | Purpose |
|
||||
|-----|---------|
|
||||
| Generate | Camera angle picker, pose selector, wireframe guide, prompt override, fine-tune textarea, batch submit |
|
||||
| Info | Metadata, preferred toggle, face-book thumbnail, hide/archive/delete actions, manual crop |
|
||||
| Generate | Camera angle picker, pose selector (with inline edit/add/delete → `poses.md`), wireframe guide, auto-growing prompt override with history autocomplete, fine-tune textareas, multi-ref thumbnails (click to deselect), batch submit |
|
||||
| Info | Metadata, preferred toggle, face-book thumbnail, hide/archive/delete, manual crop (copy or in-place), rotate ±90°, No-BG / Invert-α, duplicate, Pose preview (draggable + similar-pose search) |
|
||||
| Faceswap | Source face selector, target video, preview quality toggle |
|
||||
| Segment | SAM2 and rembg BG removal buttons |
|
||||
| Scenery | Scenery video reference for generation |
|
||||
@@ -258,6 +270,35 @@ The studio Generate tab shows a video picker + time scrubber. On generation:
|
||||
|
||||
---
|
||||
|
||||
## Pose Tools (2D body pose)
|
||||
|
||||
A posenet-style toolset built on a feature-detected estimator. Preferred backend is
|
||||
**rtmlib** (RTMPose, ONNX via the already-installed `onnxruntime`); **mediapipe** is a
|
||||
fallback. If neither is installed the endpoints return `501` and the UI shows an install hint.
|
||||
|
||||
- **Install (A6000 venv):** `~/comfyui/venv/bin/pip install rtmlib`. Models (~150 MB) auto-
|
||||
download to `~/.cache/rtmlib` on first use.
|
||||
- **CUDA libs:** `onnxruntime-gpu` needs the venv's `nvidia-*-cu12` libs on the loader path.
|
||||
`a6000-comfy/start_api.sh` adds them to `LD_LIBRARY_PATH` at launch (torch finds them via
|
||||
RPATH, onnxruntime does not).
|
||||
|
||||
**Output format:** COCO-17 keypoints (`POSE_KEYPOINT_NAMES`) + `POSE_SKELETON` edge list,
|
||||
returned in image pixels with per-joint score.
|
||||
|
||||
**Studio overlay (`car.html`):** the **Pose** button overlays the skeleton on the viewer
|
||||
(letterbox-aware canvas). Joints are **draggable** to refine/explore a pose. Overlay toolbar:
|
||||
*Reset* (re-detect) and *Similar pose* (rank the library by the current — possibly edited —
|
||||
skeleton). Results render as a thumbnail strip; clicking one opens that image's group.
|
||||
|
||||
**Similarity model (`edit_api.py`):**
|
||||
- `_pose_descriptor(keypoints)` → translation/scale-invariant vector: centered on hip midpoint
|
||||
(fallback shoulders), scaled by torso length, 17×(x,y) + visibility mask. `None` if < 6 joints.
|
||||
- `_pose_distance(a, b)` → weighted L2 over jointly-visible joints, taking the min of the direct
|
||||
and the **left-right-mirrored** comparison so mirrored poses match.
|
||||
- **Pose index:** descriptors are cached in `<output>/_data/poses_index.json`. It fills
|
||||
incrementally whenever `/images/{f}/pose` runs, or in bulk via `/pose/index` (daemon thread,
|
||||
batched writes every 50, progress via `/pose/index/status`).
|
||||
|
||||
## File Layout
|
||||
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user