dphn/Dolphin3.0-Mistral-24B is the ungated mirror of the Dolphin 3.0 Mistral 24B — exactly what you asked for. It's ~48GB fp16, which needs GPU+CPU split (device_map="auto" with 32GB on GPU, ~16GB in RAM). Let me kick off the download and update the service in parallel.
This commit is contained in:
mike
51
AGENTS.md
51
AGENTS.md
@@ -78,15 +78,27 @@ FastAPI :8500 (edit_api.py)
|
||||
| `POST` | `/images/{filename}/unarchive` | Restore archived image |
|
||||
| `POST` | `/images/{filename}/set-hidden` | Toggle studio visibility |
|
||||
| `DELETE` | `/images/{filename}` | Delete image + DB record |
|
||||
| `POST` | `/images/{filename}/crop` | Crop to pixel rect (in-place) |
|
||||
| `POST` | `/autocrop/{filename}` | Auto-trim transparent border |
|
||||
| `POST` | `/images/{filename}/crop` | Crop to pixel rect; `as_copy:true` crops a referenced copy instead of in-place |
|
||||
| `POST` | `/images/{filename}/autocrop` | Auto-trim transparent border |
|
||||
| `POST` | `/images/{filename}/rotate` | Rotate clockwise in 90° steps (lossless transpose, in-place) |
|
||||
| `POST` | `/images/{filename}/invert-alpha` | Invert the alpha channel (fixes wrong-segment BG removal) |
|
||||
| `POST` | `/images/{filename}/duplicate` | Copy into same group with `source_refs=[original]` |
|
||||
| `POST` | `/faceswap` | insightface video faceswap |
|
||||
| `POST` | `/remove-background/{filename}` | rembg BG removal → `.nobg.png` sidecar |
|
||||
| `POST` | `/remove-background/{filename}` | rembg BG removal, overwrites in-place; sets `has_background=false` + refreshes static data |
|
||||
| `POST` | `/remove-background-sam/{filename}` | SAM2 BG removal → `.nobg.png` sidecar |
|
||||
| `GET` | `/wireframe/frame/{video_name}` | Extract frame at `?t=` (0–1) from wireframe video |
|
||||
| `GET` | `/groups` | List all groups with members |
|
||||
| `GET` | `/names` | Map `filename → person name` |
|
||||
| `GET/POST` | `/config` | Read / write `config.json` |
|
||||
| `GET` | `/poses` | Load pose library from `poses.md` |
|
||||
| `POST` | `/poses` | Create / update / rename a pose (`old_name` to rename) → rewrites `poses.md` |
|
||||
| `DELETE` | `/poses/{name}` | Delete a pose from `poses.md` |
|
||||
| `GET` | `/pose/check` | Report pose-estimator availability + backend |
|
||||
| `POST` | `/images/{filename}/pose` | Estimate COCO-17 body keypoints (caches descriptor for similar-search) |
|
||||
| `GET` | `/pose/similar/{filename}` | Rank library images by pose similarity to this image |
|
||||
| `POST` | `/pose/similar` | Rank library images by similarity to a supplied (edited) skeleton |
|
||||
| `POST` | `/pose/index` | Build pose descriptors for the whole library (daemon thread) |
|
||||
| `GET` | `/pose/index/status` | Poll pose-index build progress |
|
||||
|
||||
---
|
||||
|
||||
@@ -232,8 +244,8 @@ Single-page application (~5000 lines). No build step — pure HTML/CSS/JS.
|
||||
|
||||
| Tab | Purpose |
|
||||
|-----|---------|
|
||||
| Generate | Camera angle picker, pose selector, wireframe guide, prompt override, fine-tune textarea, batch submit |
|
||||
| Info | Metadata, preferred toggle, face-book thumbnail, hide/archive/delete actions, manual crop |
|
||||
| Generate | Camera angle picker, pose selector (with inline edit/add/delete → `poses.md`), wireframe guide, auto-growing prompt override with history autocomplete, fine-tune textareas, multi-ref thumbnails (click to deselect), batch submit |
|
||||
| Info | Metadata, preferred toggle, face-book thumbnail, hide/archive/delete, manual crop (copy or in-place), rotate ±90°, No-BG / Invert-α, duplicate, Pose preview (draggable + similar-pose search) |
|
||||
| Faceswap | Source face selector, target video, preview quality toggle |
|
||||
| Segment | SAM2 and rembg BG removal buttons |
|
||||
| Scenery | Scenery video reference for generation |
|
||||
@@ -258,6 +270,35 @@ The studio Generate tab shows a video picker + time scrubber. On generation:
|
||||
|
||||
---
|
||||
|
||||
## Pose Tools (2D body pose)
|
||||
|
||||
A posenet-style toolset built on a feature-detected estimator. Preferred backend is
|
||||
**rtmlib** (RTMPose, ONNX via the already-installed `onnxruntime`); **mediapipe** is a
|
||||
fallback. If neither is installed the endpoints return `501` and the UI shows an install hint.
|
||||
|
||||
- **Install (A6000 venv):** `~/comfyui/venv/bin/pip install rtmlib`. Models (~150 MB) auto-
|
||||
download to `~/.cache/rtmlib` on first use.
|
||||
- **CUDA libs:** `onnxruntime-gpu` needs the venv's `nvidia-*-cu12` libs on the loader path.
|
||||
`a6000-comfy/start_api.sh` adds them to `LD_LIBRARY_PATH` at launch (torch finds them via
|
||||
RPATH, onnxruntime does not).
|
||||
|
||||
**Output format:** COCO-17 keypoints (`POSE_KEYPOINT_NAMES`) + `POSE_SKELETON` edge list,
|
||||
returned in image pixels with per-joint score.
|
||||
|
||||
**Studio overlay (`car.html`):** the **Pose** button overlays the skeleton on the viewer
|
||||
(letterbox-aware canvas). Joints are **draggable** to refine/explore a pose. Overlay toolbar:
|
||||
*Reset* (re-detect) and *Similar pose* (rank the library by the current — possibly edited —
|
||||
skeleton). Results render as a thumbnail strip; clicking one opens that image's group.
|
||||
|
||||
**Similarity model (`edit_api.py`):**
|
||||
- `_pose_descriptor(keypoints)` → translation/scale-invariant vector: centered on hip midpoint
|
||||
(fallback shoulders), scaled by torso length, 17×(x,y) + visibility mask. `None` if < 6 joints.
|
||||
- `_pose_distance(a, b)` → weighted L2 over jointly-visible joints, taking the min of the direct
|
||||
and the **left-right-mirrored** comparison so mirrored poses match.
|
||||
- **Pose index:** descriptors are cached in `<output>/_data/poses_index.json`. It fills
|
||||
incrementally whenever `/images/{f}/pose` runs, or in bulk via `/pose/index` (daemon thread,
|
||||
batched writes every 50, progress via `/pose/index/status`).
|
||||
|
||||
## File Layout
|
||||
|
||||
```
|
||||
|
||||
@@ -6,6 +6,12 @@ source "$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )/env.sh"
|
||||
source "$VENV/bin/activate"
|
||||
cd "$API_DIR"
|
||||
|
||||
# onnxruntime-gpu (used by the body-pose estimator) needs the CUDA runtime libs that
|
||||
# ship inside the venv's nvidia-*-cu12 pip packages on the loader path. torch finds
|
||||
# them via RPATH, but onnxruntime does not — so put them on LD_LIBRARY_PATH here.
|
||||
NV_LIBS=$(find "$VENV"/lib/python*/site-packages/nvidia -maxdepth 2 -name lib -type d 2>/dev/null | paste -sd:)
|
||||
[ -n "$NV_LIBS" ] && export LD_LIBRARY_PATH="${NV_LIBS}${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
|
||||
|
||||
export COMFY_URL="http://127.0.0.1:8188"
|
||||
export HOST="0.0.0.0"
|
||||
export PORT="8500"
|
||||
|
||||
37
backlog.md
37
backlog.md
@@ -1,3 +1,27 @@
|
||||
# Backlog
|
||||
|
||||
## Shipped (recent)
|
||||
|
||||
Studio / Generate UX:
|
||||
- ✅ Crop reaches the bottom edge (toolbar moved to top); choose copy-vs-replace ("Save as copy")
|
||||
- ✅ Paste-to-group image shows immediately (no refresh needed)
|
||||
- ✅ Generated images store the prompt used (and pose name) in the DB
|
||||
- ✅ Edit / add / delete poses from the Generate tab (persists to `poses.md`)
|
||||
- ✅ flag beta poses in ui — beta badge + editable beta flag
|
||||
- ✅ Custom prompt field auto-grows; keeps `list="promptHistory"` autocomplete
|
||||
- ✅ Remove-BG persists `has_background` + refreshes static data (survives reload)
|
||||
- ✅ Invert-α button (fixes wrong-segment background removal)
|
||||
- ✅ zien van de geselecteerde images in de prompt — multi-ref thumbs visible; click to deselect
|
||||
- ✅ Landing-page group merge syncs immediately (synchronous static write)
|
||||
- ✅ Rotate ±90° in the studio
|
||||
|
||||
Pose tools (rtmlib / RTMPose, 2D):
|
||||
- ✅ Pose skeleton preview overlay in studio
|
||||
- ✅ Draggable joints to refine/explore a pose
|
||||
- ✅ Find similar poses across the library (translation/scale/mirror-invariant; `poses_index.json`)
|
||||
|
||||
## Open
|
||||
|
||||
fix queen.mp
|
||||
moet gegenereerd worden door de htmlbuilder, niet via de webserver
|
||||
images.json
|
||||
@@ -7,22 +31,21 @@ groups.json
|
||||
config
|
||||
videos
|
||||
|
||||
flag beta poses in ui
|
||||
|
||||
extract a frame from a clip (scenery)
|
||||
faceswap defaults nothing enabled, see jobs even when refresh
|
||||
|
||||
pose bestaat gewoon uit meerdere delen, camera, scenery + addition
|
||||
|
||||
focus op head + sam2.hyria seg + mesh
|
||||
focus op head + sam2.hyria seg + mesh (3D / mesh — pose editor is currently 2D only)
|
||||
|
||||
zien van de geselecteerde images in de prompt in studio view
|
||||
sam2 saifrail, connected pixels from mid? als invertor?
|
||||
poses assignment bij de camera positie generaties
|
||||
|
||||
poses: hyper realistic, lighting, detailled skin etc.. see last generations for better results.
|
||||
|
||||
check 7b uncensored for automation
|
||||
|
||||
|
||||
create disk 128gb 70b dolphin uncencored op server.
|
||||
|
||||
rating based pose, thumbs up/down find good/bad poses easier
|
||||
(pairs well with the new pose index — could weight similar-pose results by rating)
|
||||
|
||||
when refresh page, we lose track of current jobs running.
|
||||
|
||||
@@ -1,64 +0,0 @@
|
||||
#!/bin/bash
|
||||
# One-time (idempotent) host setup for the Qwen-Image-Edit service.
|
||||
# Runs as the service user (NO sudo). Safe to re-run: existing pieces are skipped.
|
||||
#
|
||||
# Builds, under the project BASE (the parent of this api/ dir):
|
||||
# venv/ torch 2.3.1+rocm5.7 + ComfyUI deps (gfx906 / ROCm 5.7)
|
||||
# ComfyUI/ pinned to v0.3.77 + ComfyUI-GGUF custom node
|
||||
# ComfyUI/models/{unet,text_encoders,vae}/ the v23 Q8 GGUF + encoder + vae
|
||||
set -e
|
||||
|
||||
source "$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )/env.sh"
|
||||
GGUF_NODE="$COMFY/custom_nodes/ComfyUI-GGUF"
|
||||
COMFY_TAG="v0.3.77" # newest tag that runs on torch 2.3.1 (no comfy_kitchen)
|
||||
|
||||
echo "[bootstrap] BASE=$BASE VENV=$VENV"
|
||||
|
||||
# --- ComfyUI (pinned) -------------------------------------------------------
|
||||
if [ ! -d "$COMFY/.git" ]; then
|
||||
echo "[bootstrap] cloning ComfyUI @ $COMFY_TAG ..."
|
||||
git clone --depth 1 --branch "$COMFY_TAG" \
|
||||
https://github.com/comfyanonymous/ComfyUI.git "$COMFY"
|
||||
fi
|
||||
|
||||
# --- venv + python deps -----------------------------------------------------
|
||||
if [ ! -d "$VENV" ]; then
|
||||
echo "[bootstrap] creating venv at $VENV ..."
|
||||
python3 -m venv "$VENV"
|
||||
fi
|
||||
source "$VENV/bin/activate"
|
||||
python -m pip install --upgrade pip wheel
|
||||
echo "[bootstrap] installing torch (rocm5.7) ..."
|
||||
pip install torch==2.3.1+rocm5.7 torchvision==0.18.1+rocm5.7 \
|
||||
--index-url https://download.pytorch.org/whl/rocm5.7
|
||||
echo "[bootstrap] installing ComfyUI requirements ..."
|
||||
pip install -r "$COMFY/requirements.txt"
|
||||
|
||||
# --- ComfyUI-GGUF custom node ----------------------------------------------
|
||||
if [ ! -d "$GGUF_NODE" ]; then
|
||||
echo "[bootstrap] cloning ComfyUI-GGUF ..."
|
||||
git clone --depth 1 https://github.com/city96/ComfyUI-GGUF.git "$GGUF_NODE"
|
||||
fi
|
||||
pip install -r "$GGUF_NODE/requirements.txt" || pip install gguf
|
||||
|
||||
# --- API deps ---------------------------------------------------------------
|
||||
pip install fastapi "uvicorn[standard]" websocket-client python-multipart pillow requests
|
||||
|
||||
# --- models (resume-safe; skipped if already complete) ----------------------
|
||||
M="$COMFY/models"
|
||||
mkdir -p "$M/unet" "$M/text_encoders" "$M/vae"
|
||||
dl() { # url dest
|
||||
if [ -s "$2" ]; then echo "[bootstrap] have $(basename "$2")"; return; fi
|
||||
echo "[bootstrap] downloading $(basename "$2") ..."
|
||||
wget -c -q -O "$2" "$1"
|
||||
}
|
||||
dl "https://huggingface.co/Novice25/Qwen-Image-Edit-Rapid-AIO-GGUF/resolve/main/v23/Qwen-Rapid-NSFW-v23_Q8_0.gguf" \
|
||||
"$M/unet/Qwen-Rapid-NSFW-v23_Q8_0.gguf"
|
||||
dl "https://huggingface.co/Comfy-Org/Qwen-Image_ComfyUI/resolve/main/split_files/text_encoders/qwen_2.5_vl_7b_fp8_scaled.safetensors" \
|
||||
"$M/text_encoders/qwen_2.5_vl_7b_fp8_scaled.safetensors"
|
||||
dl "https://huggingface.co/Comfy-Org/Qwen-Image_ComfyUI/resolve/main/split_files/vae/qwen_image_vae.safetensors" \
|
||||
"$M/vae/qwen_image_vae.safetensors"
|
||||
|
||||
echo "[bootstrap] verifying torch + GPU ..."
|
||||
python -c "import torch; print('torch', torch.__version__, 'cuda', torch.cuda.is_available(), torch.cuda.get_device_name(0) if torch.cuda.is_available() else 'NO GPU')"
|
||||
echo "[bootstrap] BOOTSTRAP_DONE"
|
||||
@@ -1459,6 +1459,13 @@
|
||||
font-size: 9px; font-weight: 700; border-radius: 50%; width: 15px; height: 15px;
|
||||
display: flex; align-items: center; justify-content: center; line-height: 1;
|
||||
}
|
||||
.sb-gen-refs .sb-gen-ref-x {
|
||||
position: absolute; top: -4px; right: -4px; background: #dc2626; color: #fff;
|
||||
font-size: 11px; font-weight: 700; border-radius: 50%; width: 15px; height: 15px;
|
||||
display: none; align-items: center; justify-content: center; line-height: 1;
|
||||
}
|
||||
.sb-gen-refs .sb-gen-ref:hover .sb-gen-ref-x { display: flex; }
|
||||
.sb-gen-refs .sb-gen-ref:hover img { border-color: #dc2626; }
|
||||
#studioAngleBar {
|
||||
position: absolute; bottom: 8px; left: 50%; transform: translateX(-50%);
|
||||
display: flex; gap: 4px; opacity: 0; transition: opacity 0.2s;
|
||||
@@ -1490,7 +1497,9 @@
|
||||
}
|
||||
.sb-template-card:hover { border-color: #7c3aed; transform: scale(1.02); }
|
||||
.sb-template-card.selected { border-color: #22c55e; }
|
||||
.sb-template-card video { width:100%; height:100%; object-fit:cover; pointer-events:none; }
|
||||
.sb-template-card video,
|
||||
.sb-template-card img { width:100%; height:100%; object-fit:cover; pointer-events:none; }
|
||||
.sb-template-card video { position:absolute; inset:0; }
|
||||
.sb-template-label {
|
||||
position: absolute; bottom: 0; left: 0; right: 0;
|
||||
background: linear-gradient(transparent,rgba(0,0,0,0.82));
|
||||
@@ -1845,11 +1854,13 @@
|
||||
<div class="sb-actions" style="margin-bottom:8px">
|
||||
<button class="sb-btn" id="lbExtract" style="display:none" onclick="extractCurrentImage()">Extract</button>
|
||||
<button class="sb-btn" id="lbNoBgBtn" onclick="lbRemoveBg()">No BG</button>
|
||||
<button class="sb-btn" id="lbInvertAlphaBtn" onclick="lbInvertAlpha()" title="Invert transparency — fixes when the wrong segment was kept">Invert α</button>
|
||||
<button class="sb-btn" id="lbCropBtn" style="display:none" onclick="lbAutoCrop()" title="Crop away transparent border">Crop</button>
|
||||
<button class="sb-btn" id="lbDuplicateBtn" onclick="lbDuplicate()" title="Duplicate image into same group">Duplicate</button>
|
||||
<button class="sb-btn" id="lbManualCropBtn" onclick="startManualCrop()" title="Drag to crop image">Crop…</button>
|
||||
<button class="sb-btn" id="lbRotateLeftBtn" onclick="lbRotate(-90)" title="Rotate 90° counter-clockwise">⟲ 90°</button>
|
||||
<button class="sb-btn" id="lbRotateRightBtn" onclick="lbRotate(90)" title="Rotate 90° clockwise">⟳ 90°</button>
|
||||
<button class="sb-btn" id="lbPoseBtn" onclick="lbTogglePose()" title="Preview body pose skeleton">Pose</button>
|
||||
<a class="sb-btn" id="lbDownloadBtn" download style="text-decoration:none">Download</a>
|
||||
<button class="sb-btn" id="lbArchiveBtn" onclick="lbArchive()" style="color:#f59e0b" title="Move to archive (recoverable)">Archive</button>
|
||||
<button class="sb-btn danger" id="lbDeleteBtn" onclick="lbDeleteArm()" title="Click once to arm, then again to confirm">Delete</button>
|
||||
@@ -2330,6 +2341,13 @@
|
||||
document.getElementById('studio').classList.remove('open');
|
||||
document.body.style.overflow = '';
|
||||
stopSlideshow();
|
||||
// Release every streaming/decoding video so closing the studio frees memory
|
||||
// and stops background network streams (template clips + scenery preview).
|
||||
_tplStopAll();
|
||||
['sceneVideoEl', 'lbVideo', 'cltVideoPreview'].forEach(id => {
|
||||
const v = document.getElementById(id);
|
||||
if (v) { v.pause(); v.removeAttribute('src'); v.load(); }
|
||||
});
|
||||
}
|
||||
|
||||
function toggleSlideshow() {
|
||||
@@ -2391,6 +2409,8 @@
|
||||
|
||||
function updateStudio() {
|
||||
const fname = lbNames[lbIdx];
|
||||
if (document.getElementById('poseCanvas')) _removePoseOverlay(); // skeleton is per-image
|
||||
document.getElementById('poseResults')?.remove();
|
||||
_fsModelFilename = fname; // keep faceswap/scenery/segment in sync
|
||||
const isVid = isVideo(fname) || fileContentType[fname] === 'video';
|
||||
const lbImgEl = document.getElementById('lbImg');
|
||||
@@ -2442,6 +2462,11 @@
|
||||
if (faceswapBtn)faceswapBtn.style.display = !isVid ? '' : 'none';
|
||||
const cropBtn = document.getElementById('lbCropBtn');
|
||||
if (cropBtn) cropBtn.style.display = (!isVid && fileHasBg[fname] === false) ? '' : 'none';
|
||||
// Invert α is only meaningful on a transparent image (after background removal).
|
||||
const invAlphaBtn = document.getElementById('lbInvertAlphaBtn');
|
||||
if (invAlphaBtn) invAlphaBtn.style.display = (!isVid && fileHasBg[fname] === false) ? '' : 'none';
|
||||
const poseBtn = document.getElementById('lbPoseBtn');
|
||||
if (poseBtn) poseBtn.style.display = isVid ? 'none' : '';
|
||||
const archiveBtn = document.getElementById('lbArchiveBtn');
|
||||
if (archiveBtn) archiveBtn.textContent = fileArchived[fname] ? 'Restore' : 'Archive';
|
||||
|
||||
@@ -2767,7 +2792,6 @@
|
||||
showToast('Rotated', 'success');
|
||||
lbUrls[lbIdx] = IMAGE_FOLDER + fname + '?t=' + Date.now();
|
||||
updateStudio();
|
||||
refreshNow();
|
||||
} else {
|
||||
showToast('Rotate failed: ' + await r.text(), 'error');
|
||||
}
|
||||
@@ -2775,6 +2799,239 @@
|
||||
btns.forEach(b => { if (b) b.disabled = false; });
|
||||
}
|
||||
|
||||
// --- body-pose skeleton preview (interactive) ---
|
||||
const POSE_MIN_SCORE = 0.3;
|
||||
const POSE_KP_COLORS = ['#ef4444','#f97316','#eab308','#22c55e','#06b6d4','#3b82f6','#a855f7'];
|
||||
|
||||
function _removePoseOverlay() {
|
||||
document.getElementById('poseCanvas')?.remove();
|
||||
document.getElementById('poseToolbar')?.remove();
|
||||
window._poseState = null;
|
||||
const btn = document.getElementById('lbPoseBtn');
|
||||
if (btn) btn.classList.remove('primary');
|
||||
}
|
||||
|
||||
async function lbTogglePose() {
|
||||
if (document.getElementById('poseCanvas')) { _removePoseOverlay(); return; }
|
||||
const fname = lbNames[lbIdx];
|
||||
if (!fname || !/\.(png|jpg|jpeg|webp)$/i.test(fname)) { showToast('Select an image', 'info'); return; }
|
||||
const btn = document.getElementById('lbPoseBtn');
|
||||
if (btn) { btn.disabled = true; btn.textContent = '…'; }
|
||||
try {
|
||||
const r = await fetch(`${API}/images/${encodeURIComponent(fname)}/pose`, { method: 'POST' });
|
||||
if (r.status === 501) {
|
||||
showToast('Pose estimator not installed on the server (pip install rtmlib onnxruntime)', 'info', 6000);
|
||||
} else if (r.ok) {
|
||||
const d = await r.json();
|
||||
if (!d.people || !d.people.length) showToast('No person detected', 'info');
|
||||
else { _initPoseOverlay(d, fname); showToast(`Pose (${d.backend}) — drag joints to edit`, 'success'); }
|
||||
} else {
|
||||
showToast('Pose failed: ' + await r.text(), 'error');
|
||||
}
|
||||
} catch (e) { showToast('Pose failed: ' + e, 'error'); }
|
||||
if (btn) { btn.disabled = false; btn.textContent = 'Pose'; }
|
||||
}
|
||||
|
||||
function _initPoseOverlay(data, fname) {
|
||||
_removePoseOverlay();
|
||||
const viewer = document.getElementById('studioViewer');
|
||||
const img = document.getElementById('lbImg');
|
||||
if (!viewer || !img) return;
|
||||
const canvas = document.createElement('canvas');
|
||||
canvas.id = 'poseCanvas';
|
||||
canvas.style.cssText = 'position:absolute;top:0;left:0;width:100%;height:100%;cursor:grab;z-index:90;';
|
||||
viewer.appendChild(canvas);
|
||||
canvas.width = viewer.clientWidth;
|
||||
canvas.height = viewer.clientHeight;
|
||||
|
||||
// object-fit:contain letterbox mapping image px ↔ canvas px.
|
||||
const iW = data.width, iH = data.height;
|
||||
const scale = Math.min(canvas.width / iW, canvas.height / iH);
|
||||
const st = window._poseState = {
|
||||
fname, canvas, ctx: canvas.getContext('2d'),
|
||||
people: data.people.map(p => p.map(k => k.slice())), // deep copy (editable)
|
||||
skeleton: data.skeleton, width: iW, height: iH,
|
||||
scale, offX: (canvas.width - iW * scale) / 2, offY: (canvas.height - iH * scale) / 2,
|
||||
drag: null,
|
||||
};
|
||||
|
||||
// Floating toolbar (top-left so it doesn't fight the crop bar on the right).
|
||||
const bar = document.createElement('div');
|
||||
bar.id = 'poseToolbar';
|
||||
bar.style.cssText = 'position:absolute;top:8px;left:8px;display:flex;gap:6px;z-index:102;';
|
||||
bar.innerHTML =
|
||||
'<button class="sb-btn" onclick="lbPoseReset()" title="Re-detect the original pose">Reset</button>'
|
||||
+ '<button class="sb-btn primary" onclick="lbFindSimilarPose()" title="Find library images in this pose">Similar pose</button>';
|
||||
viewer.appendChild(bar);
|
||||
|
||||
canvas.addEventListener('mousedown', _posePointerDown);
|
||||
canvas.addEventListener('mousemove', _posePointerMove);
|
||||
window.addEventListener('mouseup', _posePointerUp);
|
||||
_drawPose();
|
||||
const btn = document.getElementById('lbPoseBtn');
|
||||
if (btn) btn.classList.add('primary');
|
||||
}
|
||||
|
||||
function _drawPose() {
|
||||
const st = window._poseState; if (!st) return;
|
||||
const { ctx, canvas, scale, offX, offY } = st;
|
||||
const cx = x => offX + x * scale, cy = y => offY + y * scale;
|
||||
ctx.clearRect(0, 0, canvas.width, canvas.height);
|
||||
st.people.forEach((kpts, pi) => {
|
||||
const color = POSE_KP_COLORS[pi % POSE_KP_COLORS.length];
|
||||
ctx.lineWidth = 3; ctx.strokeStyle = color;
|
||||
st.skeleton.forEach(([a, b]) => {
|
||||
const pa = kpts[a], pb = kpts[b];
|
||||
if (!pa || !pb || pa[2] < POSE_MIN_SCORE || pb[2] < POSE_MIN_SCORE) return;
|
||||
ctx.beginPath(); ctx.moveTo(cx(pa[0]), cy(pa[1])); ctx.lineTo(cx(pb[0]), cy(pb[1])); ctx.stroke();
|
||||
});
|
||||
kpts.forEach((p, ki) => {
|
||||
if (!p || p[2] < POSE_MIN_SCORE) return;
|
||||
const hot = st.drag && st.drag.pi === pi && st.drag.ki === ki;
|
||||
ctx.beginPath(); ctx.arc(cx(p[0]), cy(p[1]), hot ? 7 : 5, 0, Math.PI * 2);
|
||||
ctx.fillStyle = '#fff'; ctx.fill();
|
||||
ctx.lineWidth = 2; ctx.strokeStyle = color; ctx.stroke();
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
function _poseHitTest(mx, my) {
|
||||
const st = window._poseState; if (!st) return null;
|
||||
const cx = x => st.offX + x * st.scale, cy = y => st.offY + y * st.scale;
|
||||
let best = null, bestD = 14 * 14; // ~14px grab radius
|
||||
st.people.forEach((kpts, pi) => kpts.forEach((p, ki) => {
|
||||
if (!p || p[2] < POSE_MIN_SCORE) return;
|
||||
const dx = mx - cx(p[0]), dy = my - cy(p[1]), d = dx * dx + dy * dy;
|
||||
if (d < bestD) { bestD = d; best = { pi, ki }; }
|
||||
}));
|
||||
return best;
|
||||
}
|
||||
|
||||
function _poseEvtXY(e) {
|
||||
const r = window._poseState.canvas.getBoundingClientRect();
|
||||
return [e.clientX - r.left, e.clientY - r.top];
|
||||
}
|
||||
function _posePointerDown(e) {
|
||||
const st = window._poseState; if (!st) return;
|
||||
const [mx, my] = _poseEvtXY(e);
|
||||
const hit = _poseHitTest(mx, my);
|
||||
if (hit) { st.drag = hit; st.canvas.style.cursor = 'grabbing'; _drawPose(); e.preventDefault(); }
|
||||
}
|
||||
function _posePointerMove(e) {
|
||||
const st = window._poseState; if (!st) return;
|
||||
if (!st.drag) {
|
||||
const [mx, my] = _poseEvtXY(e);
|
||||
st.canvas.style.cursor = _poseHitTest(mx, my) ? 'grab' : 'default';
|
||||
return;
|
||||
}
|
||||
const [mx, my] = _poseEvtXY(e);
|
||||
const p = st.people[st.drag.pi][st.drag.ki];
|
||||
p[0] = Math.max(0, Math.min(st.width, (mx - st.offX) / st.scale));
|
||||
p[1] = Math.max(0, Math.min(st.height, (my - st.offY) / st.scale));
|
||||
if (p[2] < POSE_MIN_SCORE) p[2] = 1.0; // dragging a hidden joint makes it visible
|
||||
_drawPose();
|
||||
}
|
||||
function _posePointerUp() {
|
||||
const st = window._poseState; if (!st || !st.drag) return;
|
||||
st.drag = null; st.canvas.style.cursor = 'grab'; _drawPose();
|
||||
}
|
||||
|
||||
async function lbPoseReset() {
|
||||
const st = window._poseState; if (!st) return;
|
||||
const fname = st.fname;
|
||||
try {
|
||||
const r = await fetch(`${API}/images/${encodeURIComponent(fname)}/pose`, { method: 'POST' });
|
||||
if (r.ok) { const d = await r.json(); if (d.people?.length) _initPoseOverlay(d, fname); }
|
||||
} catch (e) { showToast('Reset failed: ' + e, 'error'); }
|
||||
}
|
||||
|
||||
async function lbFindSimilarPose() {
|
||||
const st = window._poseState; if (!st) return;
|
||||
// Use the (possibly edited) primary skeleton.
|
||||
const kpts = st.people[0];
|
||||
const btn = event?.target;
|
||||
if (btn) btn.disabled = true;
|
||||
try {
|
||||
const r = await fetch(`${API}/pose/similar`, {
|
||||
method: 'POST', headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ keypoints: kpts, width: st.width, height: st.height, limit: 12 }),
|
||||
});
|
||||
if (r.ok) {
|
||||
const d = await r.json();
|
||||
_renderPoseResults(d.similar || []);
|
||||
} else if (r.status === 400) {
|
||||
showToast('Pose too sparse — drag more joints into place', 'info');
|
||||
} else {
|
||||
showToast('Similar failed: ' + await r.text(), 'error');
|
||||
}
|
||||
} catch (e) { showToast('Similar failed: ' + e, 'error'); }
|
||||
if (btn) btn.disabled = false;
|
||||
}
|
||||
|
||||
function _renderPoseResults(items) {
|
||||
document.getElementById('poseResults')?.remove();
|
||||
const viewer = document.getElementById('studioViewer');
|
||||
if (!viewer) return;
|
||||
const panel = document.createElement('div');
|
||||
panel.id = 'poseResults';
|
||||
panel.style.cssText = 'position:absolute;left:0;right:0;bottom:0;z-index:103;background:rgba(0,0,0,0.85);'
|
||||
+ 'padding:8px;display:flex;gap:8px;overflow-x:auto;align-items:center;';
|
||||
if (!items.length) {
|
||||
panel.innerHTML = '<span style="color:#aaa;font-size:12px;flex:1">No similar poses found — try building the pose index.</span>'
|
||||
+ '<button class="sb-btn" onclick="lbBuildPoseIndex()">Build index</button>'
|
||||
+ '<button class="sb-btn" onclick="document.getElementById(\'poseResults\')?.remove()">Close</button>';
|
||||
} else {
|
||||
panel.innerHTML = '<span style="color:#aaa;font-size:11px;flex-shrink:0">Similar poses:</span>'
|
||||
+ items.map(it => {
|
||||
const u = IMAGE_FOLDER + it.filename + '?t=' + Date.now();
|
||||
return `<div title="dist ${it.distance}" style="flex-shrink:0;cursor:pointer;text-align:center"
|
||||
onclick="openPoseResult('${(it.group_id||'').replace(/'/g,"\\'")}','${it.filename.replace(/'/g,"\\'")}')">
|
||||
<img src="${u}" loading="lazy" style="width:64px;height:64px;object-fit:cover;border-radius:5px;border:1px solid #444" onerror="this.style.opacity='0.3'">
|
||||
<div style="font-size:9px;color:#777">${it.distance}</div></div>`;
|
||||
}).join('')
|
||||
+ '<button class="sb-btn" style="flex-shrink:0" onclick="document.getElementById(\'poseResults\')?.remove()">Close</button>';
|
||||
}
|
||||
viewer.appendChild(panel);
|
||||
}
|
||||
|
||||
function openPoseResult(gid, fname) {
|
||||
_removePoseOverlay();
|
||||
document.getElementById('poseResults')?.remove();
|
||||
if (gid && groupData.has(gid)) {
|
||||
const i = groupData.get(gid).names.indexOf(fname);
|
||||
openStudio(gid, i >= 0 ? i : 0); // openStudio sets lbUrls/lbNames/lbIdx + updateStudio
|
||||
} else {
|
||||
const i = lbNames.indexOf(fname);
|
||||
if (i >= 0) { lbIdx = i; updateStudio(); }
|
||||
else showToast('Open its group from the gallery: ' + fname, 'info', 5000);
|
||||
}
|
||||
}
|
||||
|
||||
async function lbBuildPoseIndex() {
|
||||
try {
|
||||
const r = await fetch(`${API}/pose/index`, { method: 'POST' });
|
||||
if (r.status === 501) { showToast('Pose estimator not installed', 'info'); return; }
|
||||
if (!r.ok) { showToast('Index failed: ' + await r.text(), 'error'); return; }
|
||||
showToast('Building pose index…', 'info');
|
||||
_pollPoseIndex();
|
||||
} catch (e) { showToast('Index failed: ' + e, 'error'); }
|
||||
}
|
||||
|
||||
async function _pollPoseIndex() {
|
||||
try {
|
||||
const r = await fetch(`${API}/pose/index/status`);
|
||||
if (r.ok) {
|
||||
const s = await r.json();
|
||||
if (s.running) {
|
||||
showToast(`Pose index: ${s.done}/${s.total}…`, 'info', 2500);
|
||||
setTimeout(_pollPoseIndex, 2000);
|
||||
} else {
|
||||
showToast(`Pose index ready (${s.indexed} images)`, 'success');
|
||||
}
|
||||
}
|
||||
} catch (e) { /* stop polling */ }
|
||||
}
|
||||
|
||||
function startManualCrop() {
|
||||
const fname = lbNames[lbIdx];
|
||||
if (!fname || !/\.(png|jpg|jpeg|webp)$/i.test(fname)) { showToast('Select an image to crop', 'info'); return; }
|
||||
@@ -2930,23 +3187,54 @@
|
||||
|
||||
async function lbRemoveBg() {
|
||||
const fname = lbNames[lbIdx];
|
||||
showToast(`Removing background for ${fname}...`);
|
||||
const btn = document.getElementById('lbNoBgBtn');
|
||||
if (btn) { btn.disabled = true; btn.textContent = 'Removing…'; }
|
||||
showToast('Removing background…');
|
||||
try {
|
||||
const r = await fetch(`${API}/remove-background/${fname}`, { method: 'POST' });
|
||||
if (r.ok) {
|
||||
showToast(`Background removed for ${fname}`, 'success');
|
||||
// Refresh current image in lightbox by adding timestamp
|
||||
const img = document.getElementById('lbImg');
|
||||
img.src = img.src.split('?')[0] + '?t=' + Date.now();
|
||||
refreshNow();
|
||||
showToast('Background removed', 'success');
|
||||
fileHasBg[fname] = false;
|
||||
_bustStudioImage(fname); // cache-busts viewer + calls updateStudio()
|
||||
} else {
|
||||
showToast(`Failed to remove background for ${fname}`, 'error');
|
||||
}
|
||||
} catch (e) {
|
||||
showToast(`Failed to remove background: ${e}`, 'error');
|
||||
} finally {
|
||||
if (btn) { btn.disabled = false; btn.textContent = 'No BG'; }
|
||||
}
|
||||
}
|
||||
|
||||
async function lbInvertAlpha() {
|
||||
const fname = lbNames[lbIdx];
|
||||
if (!fname || !/\.(png|webp)$/i.test(fname)) { showToast('Invert needs a transparent PNG', 'info'); return; }
|
||||
const btn = document.getElementById('lbInvertAlphaBtn');
|
||||
if (btn) btn.disabled = true;
|
||||
try {
|
||||
const r = await fetch(`${API}/images/${encodeURIComponent(fname)}/invert-alpha`, { method: 'POST' });
|
||||
if (r.ok) {
|
||||
showToast('Transparency inverted', 'success');
|
||||
fileHasBg[fname] = false;
|
||||
_bustStudioImage(fname);
|
||||
refreshNow();
|
||||
} else {
|
||||
showToast('Invert failed: ' + await r.text(), 'error');
|
||||
}
|
||||
} catch (e) { showToast('Invert failed: ' + e, 'error'); }
|
||||
if (btn) btn.disabled = false;
|
||||
}
|
||||
|
||||
// Force the studio image (filmstrip + main viewer) to reload from disk.
|
||||
function _bustStudioImage(fname) {
|
||||
const t = Date.now();
|
||||
const i = lbNames.indexOf(fname);
|
||||
if (i >= 0) lbUrls[i] = IMAGE_FOLDER + fname + '?t=' + t;
|
||||
const img = document.getElementById('lbImg');
|
||||
if (img && lbNames[lbIdx] === fname) img.src = IMAGE_FOLDER + fname + '?t=' + t;
|
||||
updateStudio();
|
||||
}
|
||||
|
||||
async function deleteGroup(el, event) {
|
||||
if (event) event.stopPropagation();
|
||||
const gid = el.closest('.image-card').dataset.group;
|
||||
@@ -3018,8 +3306,9 @@
|
||||
showToast(`Error uploading ${fileName}: ${e}`, 'error');
|
||||
}
|
||||
}
|
||||
// Trigger refresh
|
||||
refreshNow();
|
||||
// The source image is registered by a background task slightly after /upload
|
||||
// returns; one debounced refresh picks it up without piling up concurrent loads.
|
||||
setTimeout(refreshNow, 800);
|
||||
}
|
||||
|
||||
// --- clipboard paste support ---
|
||||
@@ -3131,6 +3420,8 @@
|
||||
}
|
||||
|
||||
async function loadImages() {
|
||||
if (loadImages._inFlight) return;
|
||||
loadImages._inFlight = true;
|
||||
const gallery = document.getElementById('gallery');
|
||||
const statusDot = document.getElementById('statusDot');
|
||||
statusDot.classList.add('updating');
|
||||
@@ -3204,6 +3495,7 @@
|
||||
</div>`;
|
||||
}
|
||||
statusDot.classList.remove('updating');
|
||||
loadImages._inFlight = false;
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -3311,14 +3603,27 @@
|
||||
if (fresh) {
|
||||
const curName = lbNames[lbIdx];
|
||||
const newIdx = fresh.names.indexOf(curName);
|
||||
// Snapshot ref filenames before overwriting lbNames
|
||||
const oldRefNames = _sbRefIndices.map(i => lbNames[i]).filter(Boolean);
|
||||
lbUrls = fresh.urls;
|
||||
lbNames = fresh.names;
|
||||
lbIdx = newIdx >= 0 ? newIdx : Math.min(lbIdx, fresh.names.length - 1);
|
||||
// Keep lbIdx stable when current image not yet in fresh data
|
||||
// (e.g. duplicate just created — server JSON may lag by the debounce window).
|
||||
if (newIdx >= 0) {
|
||||
lbIdx = newIdx;
|
||||
} else if (lbIdx >= fresh.names.length) {
|
||||
lbIdx = fresh.names.length - 1;
|
||||
}
|
||||
// Re-map multi-ref selections to new positions by filename
|
||||
_sbRefIndices = oldRefNames
|
||||
.map(n => fresh.names.indexOf(n))
|
||||
.filter(i => i >= 0);
|
||||
updateStudio();
|
||||
}
|
||||
}
|
||||
|
||||
statusDot.classList.remove('updating');
|
||||
loadImages._inFlight = false;
|
||||
}
|
||||
|
||||
// Discover images by trying common patterns (fallback for file:// protocol)
|
||||
@@ -3365,8 +3670,13 @@
|
||||
if (!_isStudioOpen()) refreshNow();
|
||||
}
|
||||
|
||||
// Debounced refresh — coalesces multiple rapid calls into one loadImages() run.
|
||||
// Direct callers that need a response (pollJob completion, etc.) still use
|
||||
// loadImages() directly; everything else should go through refreshNow().
|
||||
let _refreshTimer = null;
|
||||
function refreshNow() {
|
||||
loadImages();
|
||||
clearTimeout(_refreshTimer);
|
||||
_refreshTimer = setTimeout(loadImages, 80);
|
||||
}
|
||||
|
||||
const API = 'http://127.0.0.1:8500';
|
||||
@@ -3711,7 +4021,7 @@
|
||||
const data = groupData.get(lbCurrentGid);
|
||||
if (!data) return;
|
||||
const filename = data.names[lbIdx];
|
||||
const btn = document.querySelector('#lightbox .lb-btn[onclick="tagCurrentImage()"]');
|
||||
const btn = document.querySelector('button[onclick="tagCurrentImage()"]');
|
||||
if (btn) btn.textContent = '…';
|
||||
try {
|
||||
const r = await fetch(`${API}/tag`, {
|
||||
@@ -3732,7 +4042,7 @@
|
||||
showToast(`Tag failed: ${err}`, 'info');
|
||||
}
|
||||
} catch (e) { showToast('API not reachable', 'info'); }
|
||||
if (btn) btn.textContent = 'Tag';
|
||||
if (btn) btn.textContent = 'Re-tag';
|
||||
}
|
||||
|
||||
async function extractCurrentImage() {
|
||||
@@ -3840,8 +4150,15 @@
|
||||
}
|
||||
if (!document.getElementById('studio').classList.contains('open')) return;
|
||||
if (tag === 'INPUT' || tag === 'TEXTAREA') return;
|
||||
if (e.key === 'ArrowLeft') { lbNav(-1); e.preventDefault(); }
|
||||
if (e.key === 'ArrowRight') { lbNav(1); e.preventDefault(); }
|
||||
// Throttle filmstrip navigation — ignore if last nav was < 80 ms ago.
|
||||
if (e.key === 'ArrowLeft' || e.key === 'ArrowRight') {
|
||||
const now = Date.now();
|
||||
if (now - (document._lbNavLast || 0) >= 80) {
|
||||
document._lbNavLast = now;
|
||||
lbNav(e.key === 'ArrowLeft' ? -1 : 1);
|
||||
}
|
||||
e.preventDefault();
|
||||
}
|
||||
if (e.key === 'Home') {
|
||||
lbIdx = 0;
|
||||
updateStudio();
|
||||
@@ -3859,7 +4176,10 @@
|
||||
e.preventDefault();
|
||||
}
|
||||
}
|
||||
if ((e.key === 'h' || e.key === 'H') && tag !== 'INPUT') { lbToggleHidden(); }
|
||||
// Guard async actions against concurrent executions via a simple inflight flag.
|
||||
if ((e.key === 'h' || e.key === 'H') && tag !== 'INPUT') {
|
||||
if (!document._lbHideInFlight) { document._lbHideInFlight = true; lbToggleHidden().finally(() => { document._lbHideInFlight = false; }); }
|
||||
}
|
||||
if (e.key === 'F' || e.key === 'f') {
|
||||
lbSetPreferred();
|
||||
e.preventDefault();
|
||||
@@ -3868,12 +4188,11 @@
|
||||
lbArchive();
|
||||
e.preventDefault();
|
||||
}
|
||||
if (e.key === 'ArrowUp') {
|
||||
lbMoveInGroup(-1);
|
||||
e.preventDefault();
|
||||
}
|
||||
if (e.key === 'ArrowDown') {
|
||||
lbMoveInGroup(1);
|
||||
// ArrowUp/Down move images within the group — debounce rapid keypresses.
|
||||
if (e.key === 'ArrowUp' || e.key === 'ArrowDown') {
|
||||
clearTimeout(document._lbMoveTimer);
|
||||
const d = e.key === 'ArrowUp' ? -1 : 1;
|
||||
document._lbMoveTimer = setTimeout(() => lbMoveInGroup(d), 120);
|
||||
e.preventDefault();
|
||||
}
|
||||
});
|
||||
@@ -3955,7 +4274,7 @@
|
||||
|
||||
async function setAsPreferred(filename, btn) {
|
||||
try {
|
||||
const r = await fetch(`/images/${encodeURIComponent(filename)}/set-preferred`, {method:'POST'});
|
||||
const r = await fetch(`${API}/images/${encodeURIComponent(filename)}/set-preferred`, {method:'POST'});
|
||||
if (!r.ok) { console.error('set-preferred failed', r.status); return; }
|
||||
const data = await r.json();
|
||||
// Update sort orders locally
|
||||
@@ -4008,6 +4327,7 @@
|
||||
}
|
||||
|
||||
function switchSidebarTab(tab, initial) {
|
||||
_tplStopAll(); // release any hovered template video before re-rendering panels
|
||||
_activeSidebarTab = tab;
|
||||
localStorage.setItem('studioSidebarTab', tab);
|
||||
document.querySelectorAll('#studioSidebar .sb-tab').forEach(t =>
|
||||
@@ -4143,9 +4463,9 @@
|
||||
const fn = lbNames[idx] || '';
|
||||
const u = lbUrls[idx] || '';
|
||||
const thumb = isVideo(fn) ? posterFor(u) : u;
|
||||
return `<div class="sb-gen-ref" title="${escHtml(fn)}"><img src="${thumb}" loading="lazy" onerror="this.style.opacity='0.3'"><div class="sb-gen-ref-badge">${pos+1}</div></div>`;
|
||||
return `<div class="sb-gen-ref" title="Click to remove · ${escHtml(fn)}" style="cursor:pointer" onclick="sbRemoveRef(${idx})"><img src="${thumb}" loading="lazy" onerror="this.style.opacity='0.3'"><div class="sb-gen-ref-badge">${pos+1}</div><div class="sb-gen-ref-x">×</div></div>`;
|
||||
}).join('');
|
||||
html += `<div class="sb-label" style="margin-bottom:4px">References <span style="color:#555;font-weight:400;font-size:9px">· Shift+Click filmstrip to change</span></div>
|
||||
html += `<div class="sb-label" style="margin-bottom:4px">References <span style="color:#555;font-weight:400;font-size:9px">· Shift+Click filmstrip or click a thumb to remove</span></div>
|
||||
<div class="sb-gen-refs">${refThumbs}</div>`;
|
||||
// With 2–3 refs, let the user choose how the prompt is applied.
|
||||
if (_sbRefIndices.length >= 2) {
|
||||
@@ -4326,6 +4646,15 @@
|
||||
renderSidebarGenerate();
|
||||
}
|
||||
|
||||
// Remove a reference by clicking its thumbnail above the custom prompt — no page
|
||||
// refresh needed (previously refs could only be cleared via Shift+Click filmstrip).
|
||||
function sbRemoveRef(idx) {
|
||||
const pos = _sbRefIndices.indexOf(idx);
|
||||
if (pos >= 0) _sbRefIndices.splice(pos, 1);
|
||||
updateStudio();
|
||||
if (_activeSidebarTab === 'generate') renderSidebarGenerate();
|
||||
}
|
||||
|
||||
function sbSelectWireframe(val) {
|
||||
_sbWireframeRef = val;
|
||||
renderSidebarGenerate();
|
||||
@@ -4543,6 +4872,53 @@
|
||||
}
|
||||
}
|
||||
|
||||
// ---- template video cards (faceswap / scenery) ----
|
||||
// The wireframe library is large (dozens of files, >1 GB total) and lives on a
|
||||
// network mount. Rendering every card as an autoplaying <video> spun up that many
|
||||
// simultaneous decoders + range-streams, which OOM-crashed the renderer. Instead
|
||||
// each card shows a static poster frame (a small server-rendered PNG) and only
|
||||
// mounts a single real <video> while hovered — at most one decoder alive at a time.
|
||||
|
||||
// HTML for one template card. `extraHTML` lets faceswap add its trim button.
|
||||
function _tplCardHTML(name, selected, onclickExpr, extraHTML = '') {
|
||||
const stem = name.replace(/\.[^.]+$/, '');
|
||||
const sel = selected ? ' selected' : '';
|
||||
const nSafe = name.replace(/'/g, "\\'");
|
||||
return `<div class="sb-template-card${sel}" onclick="${onclickExpr}"
|
||||
onmouseenter="_tplPlay(this,'${nSafe}')" onmouseleave="_tplStop(this)">
|
||||
<img src="${API}/wireframe/frame/${encodeURIComponent(name)}?t=0" loading="lazy" alt="">
|
||||
<div class="sb-template-label">${escHtml(stem)}</div>${extraHTML}
|
||||
</div>`;
|
||||
}
|
||||
|
||||
// Mount + play a single <video> over the poster while hovered.
|
||||
function _tplPlay(card, name) {
|
||||
if (card._vid) return;
|
||||
const v = document.createElement('video');
|
||||
v.src = `${API}/wireframe/${encodeURIComponent(name)}`;
|
||||
v.muted = true; v.loop = true; v.playsInline = true; v.preload = 'auto';
|
||||
card.appendChild(v);
|
||||
card._vid = v;
|
||||
v.play().catch(() => {});
|
||||
}
|
||||
|
||||
// Tear the video down completely so the decoder + network stream are released.
|
||||
function _tplStop(card) {
|
||||
const v = card && card._vid;
|
||||
if (!v) return;
|
||||
v.pause();
|
||||
v.removeAttribute('src');
|
||||
v.load(); // aborts the in-flight stream and frees the decoder
|
||||
v.remove();
|
||||
card._vid = null;
|
||||
}
|
||||
|
||||
// Defensive: stop every active template video in the sidebar (called on tab switch
|
||||
// so a hovered clip can't keep streaming under a freshly-rendered panel).
|
||||
function _tplStopAll() {
|
||||
document.querySelectorAll('#studioSidebar .sb-template-card').forEach(_tplStop);
|
||||
}
|
||||
|
||||
// ---- faceswap sidebar tab ----
|
||||
|
||||
async function loadTemplateVideos() {
|
||||
@@ -4612,17 +4988,10 @@
|
||||
} else {
|
||||
html += '<div class="sb-template-grid">';
|
||||
availableVideos.forEach(v => {
|
||||
const stem = v.replace(/\.[^.]+$/, '');
|
||||
const vSafe = v.replace(/'/g, "\\'");
|
||||
const sel = _fsSelectedVideo === v ? ' selected' : '';
|
||||
html += `<div class="sb-template-card${sel}" onclick="sbSelectTemplate('${vSafe}')"
|
||||
onmouseenter="const v=this.querySelector('video');v&&v.play().catch(()=>{})" onmouseleave="const v=this.querySelector('video');v&&v.pause()">
|
||||
<video src="${API}/wireframe/${encodeURIComponent(v)}" muted loop playsinline
|
||||
preload="metadata" style="pointer-events:none;width:100%;height:100%;object-fit:cover"></video>
|
||||
<div class="sb-template-label">${escHtml(stem)}</div>
|
||||
<button class="sb-template-trim-btn" title="Trim video"
|
||||
onclick="event.stopPropagation();openTrimPanel('${vSafe}')">✂</button>
|
||||
</div>`;
|
||||
const trimBtn = `<button class="sb-template-trim-btn" title="Trim video"
|
||||
onclick="event.stopPropagation();openTrimPanel('${vSafe}')">✂</button>`;
|
||||
html += _tplCardHTML(v, _fsSelectedVideo === v, `sbSelectTemplate('${vSafe}')`, trimBtn);
|
||||
});
|
||||
html += '</div>';
|
||||
}
|
||||
@@ -4658,9 +5027,7 @@
|
||||
</div>`;
|
||||
|
||||
panel.innerHTML = html;
|
||||
requestAnimationFrame(() => {
|
||||
panel.querySelectorAll('.sb-template-card video').forEach(v => v.play().catch(()=>{}));
|
||||
});
|
||||
// Cards show static poster frames; videos mount only on hover (_tplPlay).
|
||||
// Check FaceFusion availability
|
||||
fetch(`${API}/faceswap/check`).then(r => r.json()).then(s => {
|
||||
const hairCb = document.getElementById('sbFsHair');
|
||||
@@ -4863,15 +5230,8 @@
|
||||
if (availableVideos.length) {
|
||||
html += '<div class="sb-template-grid">';
|
||||
availableVideos.forEach(v => {
|
||||
const stem = v.replace(/\.[^.]+$/, '');
|
||||
const vSafe = v.replace(/'/g, "\\'");
|
||||
const sel = _sceneVideo === v ? ' selected' : '';
|
||||
html += `<div class="sb-template-card${sel}" onclick="sceneSelectVideo('${vSafe}')"
|
||||
onmouseenter="const v=this.querySelector('video');v&&v.play().catch(()=>{})" onmouseleave="const v=this.querySelector('video');v&&v.pause()">
|
||||
<video src="${API}/wireframe/${encodeURIComponent(v)}" muted loop playsinline
|
||||
preload="metadata" style="pointer-events:none;width:100%;height:100%;object-fit:cover"></video>
|
||||
<div class="sb-template-label">${escHtml(stem)}</div>
|
||||
</div>`;
|
||||
html += _tplCardHTML(v, _sceneVideo === v, `sceneSelectVideo('${vSafe}')`);
|
||||
});
|
||||
html += '</div>';
|
||||
}
|
||||
@@ -4913,9 +5273,7 @@
|
||||
if (scrubber) { scrubber.max = _sceneDuration; scrubber.step = Math.max(0.05, _sceneDuration / 500); }
|
||||
}
|
||||
}
|
||||
requestAnimationFrame(() => {
|
||||
panel.querySelectorAll('.sb-template-card video').forEach(v => v.play().catch(()=>{}));
|
||||
});
|
||||
// Cards show static poster frames; videos mount only on hover (_tplPlay).
|
||||
}
|
||||
|
||||
async function sceneSelectVideo(v) {
|
||||
|
||||
@@ -1,13 +1,13 @@
|
||||
{
|
||||
"api_url": "http://127.0.0.1:8500/edit",
|
||||
"prompt": "high quality. realistic. detailed, female nude. realistic, high quality. realistic. detailed. female nude. realistic",
|
||||
"prompt": "high quality. hyper realistic. detailed, detailed skin, detailed female nude. realistic, high quality. realistic. detailed. female nude. realistic",
|
||||
"base_prompts": [
|
||||
"Head-on full-nude-body three-quarter female portrait, realistic, transparent background",
|
||||
"Head-on straight-on full-nude-body female portrait, realistic, transparent background",
|
||||
"Head-on straight-on full-body female portrait, realistic, no background",
|
||||
"Head-on full-nude-body three-quarter female portrait, realistic, black void background",
|
||||
"Head-on straight-on full-nude-body female portrait, realistic, black void background",
|
||||
"Head-on straight-on full-body female portrait, realistic, black void background",
|
||||
"high quality, full-nude-body, female, masterpiece, realistic, photo, looking at viewer",
|
||||
"high quality, full-nude-body, female, masterpiece, realistic, photo, detailed skin, professional lighting, transparent background",
|
||||
"high quality, full-nude-body, female, masterpiece, realistic, photo, cinematic lighting, dramatic shadows, sharp focus, transparent background"
|
||||
"high quality, full-nude-body, female, masterpiece, realistic, photo, detailed skin, professional lighting, black void background",
|
||||
"high quality, full-nude-body, female, masterpiece, realistic, photo, cinematic lighting, dramatic shadows, sharp focus, black void background"
|
||||
],
|
||||
"seed": -1,
|
||||
"max_area": 655360,
|
||||
|
||||
@@ -993,6 +993,8 @@ def _move_to_trash(filepath: str):
|
||||
# --- static data files -------------------------------------------------------
|
||||
|
||||
_static_write_lock = threading.Lock()
|
||||
_invalidate_timer: "threading.Timer | None" = None
|
||||
_invalidate_timer_lock = threading.Lock()
|
||||
|
||||
|
||||
def _write_json(path: str, data) -> None:
|
||||
@@ -1084,8 +1086,17 @@ def _write_all_static() -> None:
|
||||
|
||||
|
||||
def _invalidate_static() -> None:
|
||||
"""Spawn a daemon thread to regenerate all static data files (non-blocking)."""
|
||||
threading.Thread(target=_write_all_static, daemon=True).start()
|
||||
"""Coalesce rapid invalidation calls — restarts a 0.3 s debounce timer each time.
|
||||
At most one _write_all_static() runs per quiet window, preventing thread floods
|
||||
during batch jobs that call this after every single image."""
|
||||
global _invalidate_timer
|
||||
with _invalidate_timer_lock:
|
||||
if _invalidate_timer is not None:
|
||||
_invalidate_timer.cancel()
|
||||
t = threading.Timer(0.3, _write_all_static)
|
||||
t.daemon = True
|
||||
t.start()
|
||||
_invalidate_timer = t
|
||||
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
@@ -1253,6 +1264,9 @@ def _multi_ref_worker(job_id: str, filenames: list[str], prompts: list[str], pos
|
||||
print(f"DB error in multi-ref: {db_err}")
|
||||
|
||||
jobs[job_id]["done"] += 1
|
||||
# Regenerate static JSON so the frontend's polling picks up the new
|
||||
# image immediately (progressive refresh, matching _batch_worker).
|
||||
_invalidate_static()
|
||||
except Exception as e:
|
||||
print(f"Error in multi-ref for prompt '{prompt}': {e}")
|
||||
jobs[job_id]["failed"] += 1
|
||||
@@ -1734,7 +1748,9 @@ def merge_groups(req: MergeRequest):
|
||||
except Exception as db_err:
|
||||
print(f"Database error in merge: {db_err}")
|
||||
|
||||
_invalidate_static()
|
||||
# Write synchronously: the frontend reloads images.json immediately after this
|
||||
# returns, so an async rebuild would race and show the pre-merge grouping.
|
||||
_write_all_static()
|
||||
return {"group_id": gid, "files": req.filenames}
|
||||
|
||||
|
||||
@@ -1750,7 +1766,7 @@ def extract_from_group(req: ExtractRequest):
|
||||
except Exception as db_err:
|
||||
print(f"Database error in extract: {db_err}")
|
||||
|
||||
_invalidate_static()
|
||||
_write_all_static()
|
||||
return {"filename": req.filename}
|
||||
|
||||
|
||||
@@ -1928,6 +1944,10 @@ def _process_upload(file_path: str, filename: str, prompts: list[str], name: str
|
||||
clip_description=clip_desc, tags=tags, embedding=embedding,
|
||||
group_id=group_id, sort_order=0, has_clothing=has_clothing,
|
||||
)
|
||||
# Surface the new group with its base image right away — the pose/base-prompt
|
||||
# generation below can take a while, and the user shouldn't wait for it to
|
||||
# see the group land on the gallery.
|
||||
_invalidate_static()
|
||||
|
||||
# 4. Crop if needed
|
||||
cropped_pil = _crop_to_bbox(pil)
|
||||
@@ -2172,6 +2192,28 @@ def remove_background(filename: str):
|
||||
with open(path, "wb") as f:
|
||||
f.write(transparent_png)
|
||||
|
||||
# Persist the state + refresh static data so the flag (and No-BG/Crop buttons)
|
||||
# survive a page reload instead of reverting to has_background=True.
|
||||
database.upsert_person(filename, has_background=False)
|
||||
_invalidate_static()
|
||||
return {"status": "success", "filename": filename, "has_background": False}
|
||||
|
||||
|
||||
@app.post("/images/{filename}/invert-alpha")
|
||||
def invert_alpha(filename: str):
|
||||
"""Invert the alpha channel in place — recovers cases where background removal
|
||||
kept the background and dropped the subject (the wrong segment)."""
|
||||
import numpy as np
|
||||
person = database.get_person(filename)
|
||||
if not person or not person[5] or not os.path.exists(person[5]):
|
||||
raise HTTPException(404, "Image file not found")
|
||||
path = person[5]
|
||||
img = Image.open(path).convert("RGBA")
|
||||
arr = np.array(img)
|
||||
arr[:, :, 3] = 255 - arr[:, :, 3]
|
||||
Image.fromarray(arr, "RGBA").save(path, format="PNG")
|
||||
database.upsert_person(filename, has_background=False)
|
||||
_invalidate_static()
|
||||
return {"status": "success", "filename": filename}
|
||||
|
||||
|
||||
@@ -2747,6 +2789,358 @@ def sam2_check():
|
||||
return {"sam2": predictor is not False and predictor is not None}
|
||||
|
||||
|
||||
# --- 2D body-pose preview -----------------------------------------------------
|
||||
# Estimates COCO-17 keypoints from the model image so the UI can overlay a
|
||||
# posenet-style skeleton. Estimator is feature-detected: rtmlib (ONNX, reuses the
|
||||
# already-installed onnxruntime) is preferred, mediapipe is a fallback. If neither
|
||||
# is installed the endpoints report unavailable instead of erroring the request.
|
||||
|
||||
_pose_estimator = None # cached (callable, backend_name) or False if unavailable
|
||||
_pose_lock = threading.Lock()
|
||||
|
||||
# COCO-17 keypoint names (the order rtmlib's Body model returns).
|
||||
POSE_KEYPOINT_NAMES = [
|
||||
"nose", "left_eye", "right_eye", "left_ear", "right_ear",
|
||||
"left_shoulder", "right_shoulder", "left_elbow", "right_elbow",
|
||||
"left_wrist", "right_wrist", "left_hip", "right_hip",
|
||||
"left_knee", "right_knee", "left_ankle", "right_ankle",
|
||||
]
|
||||
# Bone connections (index pairs into COCO-17) for drawing the skeleton.
|
||||
POSE_SKELETON = [
|
||||
(5, 7), (7, 9), (6, 8), (8, 10), # arms
|
||||
(11, 13), (13, 15), (12, 14), (14, 16), # legs
|
||||
(5, 6), (11, 12), (5, 11), (6, 12), # torso
|
||||
(0, 1), (0, 2), (1, 3), (2, 4), (0, 5), (0, 6), # head/neck
|
||||
]
|
||||
# mediapipe Pose (33 landmarks) → COCO-17 index map.
|
||||
_MP_TO_COCO = [0, 2, 5, 7, 8, 11, 12, 13, 14, 15, 16, 23, 24, 25, 26, 27, 28]
|
||||
|
||||
|
||||
def _load_pose_estimator():
|
||||
global _pose_estimator
|
||||
if _pose_estimator is not None:
|
||||
return _pose_estimator
|
||||
with _pose_lock:
|
||||
if _pose_estimator is not None:
|
||||
return _pose_estimator
|
||||
# Preferred: rtmlib (RTMPose, ONNX) — returns COCO-17 directly.
|
||||
try:
|
||||
from rtmlib import Body
|
||||
import numpy as np
|
||||
model = Body(mode="balanced", backend="onnxruntime", device="cpu")
|
||||
|
||||
def _infer_rtm(pil):
|
||||
bgr = np.array(pil.convert("RGB"))[:, :, ::-1]
|
||||
kpts, scores = model(bgr) # (N,17,2), (N,17)
|
||||
people = []
|
||||
for person_kpts, person_scores in zip(kpts, scores):
|
||||
people.append([[float(x), float(y), float(s)]
|
||||
for (x, y), s in zip(person_kpts, person_scores)])
|
||||
return people
|
||||
|
||||
_pose_estimator = (_infer_rtm, "rtmlib")
|
||||
print("[pose] using rtmlib (RTMPose)")
|
||||
return _pose_estimator
|
||||
except Exception as e:
|
||||
print(f"[pose] rtmlib unavailable: {e}")
|
||||
|
||||
# Fallback: mediapipe Pose (single person, normalized landmarks).
|
||||
try:
|
||||
import mediapipe as mp
|
||||
import numpy as np
|
||||
mp_pose = mp.solutions.pose.Pose(static_image_mode=True, model_complexity=2)
|
||||
|
||||
def _infer_mp(pil):
|
||||
rgb = np.array(pil.convert("RGB"))
|
||||
h, w = rgb.shape[:2]
|
||||
res = mp_pose.process(rgb)
|
||||
if not res.pose_landmarks:
|
||||
return []
|
||||
lm = res.pose_landmarks.landmark
|
||||
kpts = []
|
||||
for mp_idx in _MP_TO_COCO:
|
||||
p = lm[mp_idx]
|
||||
kpts.append([float(p.x * w), float(p.y * h), float(p.visibility)])
|
||||
return [kpts]
|
||||
|
||||
_pose_estimator = (_infer_mp, "mediapipe")
|
||||
print("[pose] using mediapipe Pose")
|
||||
return _pose_estimator
|
||||
except Exception as e:
|
||||
print(f"[pose] mediapipe unavailable: {e}")
|
||||
|
||||
_pose_estimator = False
|
||||
return _pose_estimator
|
||||
|
||||
|
||||
# --- pose similarity (descriptor + index) -------------------------------------
|
||||
# Pose descriptors are normalized (translation + scale invariant) COCO-17 vectors,
|
||||
# cached in <output>/_data/poses_index.json so we can rank library images by pose.
|
||||
|
||||
_POSE_MIN_SCORE = 0.3
|
||||
# Left/right keypoint pairs for the mirror-invariant distance.
|
||||
_POSE_MIRROR = [0, 2, 1, 4, 3, 6, 5, 8, 7, 10, 9, 12, 11, 14, 13, 16, 15]
|
||||
_pose_index_status = {"running": False, "done": 0, "total": 0}
|
||||
|
||||
|
||||
def _pose_descriptor(keypoints):
|
||||
"""Normalize one person's COCO-17 keypoints into a translation/scale-invariant
|
||||
descriptor: {"vec": [34 floats], "vis": [17 ints]}. Returns None if too sparse."""
|
||||
vis = [1 if (kp[2] >= _POSE_MIN_SCORE) else 0 for kp in keypoints]
|
||||
if sum(vis) < 6:
|
||||
return None
|
||||
|
||||
def _mid(a, b):
|
||||
if vis[a] and vis[b]:
|
||||
return ((keypoints[a][0] + keypoints[b][0]) / 2.0,
|
||||
(keypoints[a][1] + keypoints[b][1]) / 2.0)
|
||||
return None
|
||||
|
||||
hip = _mid(11, 12)
|
||||
sho = _mid(5, 6)
|
||||
center = hip or sho
|
||||
if center is None:
|
||||
return None
|
||||
# Scale by torso length; fall back to keypoint spread if torso isn't visible.
|
||||
if hip and sho:
|
||||
scale = ((hip[0] - sho[0]) ** 2 + (hip[1] - sho[1]) ** 2) ** 0.5
|
||||
else:
|
||||
xs = [keypoints[i][0] for i in range(17) if vis[i]]
|
||||
ys = [keypoints[i][1] for i in range(17) if vis[i]]
|
||||
scale = max(max(xs) - min(xs), max(ys) - min(ys))
|
||||
if not scale or scale < 1e-3:
|
||||
return None
|
||||
|
||||
vec = []
|
||||
for i in range(17):
|
||||
if vis[i]:
|
||||
vec.append((keypoints[i][0] - center[0]) / scale)
|
||||
vec.append((keypoints[i][1] - center[1]) / scale)
|
||||
else:
|
||||
vec.extend([0.0, 0.0])
|
||||
return {"vec": vec, "vis": vis}
|
||||
|
||||
|
||||
def _pose_distance(a, b):
|
||||
"""Weighted L2 between two descriptors over jointly-visible joints, taking the
|
||||
min of the direct and left-right-mirrored comparison. Lower = more similar."""
|
||||
def _dist(av, avis, bv, bvis, mirror):
|
||||
total, n = 0.0, 0
|
||||
for i in range(17):
|
||||
j = _POSE_MIRROR[i] if mirror else i
|
||||
if not (avis[i] and bvis[j]):
|
||||
continue
|
||||
bx = bv[j * 2] * (-1 if mirror else 1) # flip x when mirrored
|
||||
by = bv[j * 2 + 1]
|
||||
dx = av[i * 2] - bx
|
||||
dy = av[i * 2 + 1] - by
|
||||
total += dx * dx + dy * dy
|
||||
n += 1
|
||||
return (total / n) ** 0.5 if n >= 4 else float("inf")
|
||||
direct = _dist(a["vec"], a["vis"], b["vec"], b["vis"], False)
|
||||
mirror = _dist(a["vec"], a["vis"], b["vec"], b["vis"], True)
|
||||
return min(direct, mirror)
|
||||
|
||||
|
||||
def _best_person(people):
|
||||
"""Pick the largest-bbox person from an estimator result (most prominent subject)."""
|
||||
best, best_area = None, -1.0
|
||||
for kpts in people:
|
||||
xs = [k[0] for k in kpts if k[2] >= _POSE_MIN_SCORE]
|
||||
ys = [k[1] for k in kpts if k[2] >= _POSE_MIN_SCORE]
|
||||
if len(xs) < 2:
|
||||
continue
|
||||
area = (max(xs) - min(xs)) * (max(ys) - min(ys))
|
||||
if area > best_area:
|
||||
best, best_area = kpts, area
|
||||
return best
|
||||
|
||||
|
||||
def _pose_index_path():
|
||||
return os.path.join(_load_output_dir(), "_data", "poses_index.json")
|
||||
|
||||
|
||||
def _load_pose_index():
|
||||
try:
|
||||
with open(_pose_index_path(), "r") as f:
|
||||
return json.load(f)
|
||||
except Exception:
|
||||
return {}
|
||||
|
||||
|
||||
_pose_index_lock = threading.Lock()
|
||||
|
||||
|
||||
def _save_pose_index_entry(filename, desc):
|
||||
with _pose_index_lock:
|
||||
idx = _load_pose_index()
|
||||
idx[filename] = desc
|
||||
os.makedirs(os.path.dirname(_pose_index_path()), exist_ok=True)
|
||||
_write_json(_pose_index_path(), idx)
|
||||
|
||||
|
||||
@app.get("/pose/check")
|
||||
def pose_check():
|
||||
"""Report whether a body-pose estimator is available (and which backend)."""
|
||||
est = _load_pose_estimator()
|
||||
if not est:
|
||||
return {"available": False,
|
||||
"hint": "pip install rtmlib onnxruntime (or: pip install mediapipe)"}
|
||||
return {"available": True, "backend": est[1]}
|
||||
|
||||
|
||||
@app.post("/images/{filename}/pose")
|
||||
def estimate_pose(filename: str):
|
||||
"""Estimate COCO-17 body keypoints for an image. Returns pixel-space keypoints
|
||||
plus the skeleton edge list so the frontend can overlay a posenet-style preview."""
|
||||
person = database.get_person(filename)
|
||||
if not person or not person[5] or not os.path.exists(person[5]):
|
||||
raise HTTPException(404, "Image file not found")
|
||||
est = _load_pose_estimator()
|
||||
if not est:
|
||||
raise HTTPException(501, "No pose estimator installed. Try: pip install rtmlib onnxruntime")
|
||||
infer, backend = est
|
||||
pil = Image.open(person[5]).convert("RGB")
|
||||
try:
|
||||
people = infer(pil)
|
||||
except Exception as e:
|
||||
raise HTTPException(500, f"Pose estimation failed: {e}")
|
||||
# Cache the descriptor so "find similar pose" can rank this image later.
|
||||
best = _best_person(people)
|
||||
if best is not None:
|
||||
desc = _pose_descriptor(best)
|
||||
if desc is not None:
|
||||
try:
|
||||
_save_pose_index_entry(filename, desc)
|
||||
except Exception as e:
|
||||
print(f"[pose] index save failed for {filename}: {e}")
|
||||
return {
|
||||
"status": "success",
|
||||
"backend": backend,
|
||||
"width": pil.width,
|
||||
"height": pil.height,
|
||||
"names": POSE_KEYPOINT_NAMES,
|
||||
"skeleton": POSE_SKELETON,
|
||||
"people": people,
|
||||
}
|
||||
|
||||
|
||||
def _build_pose_index_task():
|
||||
try:
|
||||
est = _load_pose_estimator()
|
||||
if not est:
|
||||
return
|
||||
infer, _ = est
|
||||
output_dir = _load_output_dir()
|
||||
with _pose_index_lock:
|
||||
idx = _load_pose_index()
|
||||
persons = database.list_persons()
|
||||
todo = [p[0] for p in persons
|
||||
if p[0] not in idx
|
||||
and (p[12] or "image") != "video"
|
||||
and p[0].lower().endswith(('.png', '.jpg', '.jpeg', '.webp'))]
|
||||
_pose_index_status.update(running=True, done=0, total=len(todo))
|
||||
print(f"[pose] index build: {len(todo)} images to process")
|
||||
dirty = 0
|
||||
for fn in todo:
|
||||
try:
|
||||
fpath = os.path.join(output_dir, fn)
|
||||
if os.path.exists(fpath):
|
||||
best = _best_person(infer(Image.open(fpath).convert("RGB")))
|
||||
desc = _pose_descriptor(best) if best is not None else None
|
||||
if desc is not None:
|
||||
idx[fn] = desc
|
||||
dirty += 1
|
||||
except Exception as e:
|
||||
print(f"[pose] index error for {fn}: {e}")
|
||||
_pose_index_status["done"] += 1
|
||||
# Batch-flush every 50 to avoid O(n^2) full-file rewrites.
|
||||
if dirty >= 50:
|
||||
with _pose_index_lock:
|
||||
_write_json(_pose_index_path(), idx)
|
||||
dirty = 0
|
||||
print(f"[pose] index progress: {_pose_index_status['done']}/{len(todo)}")
|
||||
with _pose_index_lock:
|
||||
_write_json(_pose_index_path(), idx)
|
||||
print(f"[pose] index build complete: {len(idx)} entries")
|
||||
except Exception as e:
|
||||
print(f"[pose] index build failed: {e}")
|
||||
finally:
|
||||
_pose_index_status["running"] = False
|
||||
|
||||
|
||||
@app.post("/pose/index")
|
||||
def build_pose_index():
|
||||
"""Compute pose descriptors for all library images lacking one (daemon thread)."""
|
||||
if not _load_pose_estimator():
|
||||
raise HTTPException(501, "No pose estimator installed. Try: pip install rtmlib onnxruntime")
|
||||
if _pose_index_status.get("running"):
|
||||
return {"status": "already_running", **_pose_index_status}
|
||||
threading.Thread(target=_build_pose_index_task, daemon=True).start()
|
||||
return {"status": "started"}
|
||||
|
||||
|
||||
@app.get("/pose/index/status")
|
||||
def pose_index_status():
|
||||
idx = _load_pose_index()
|
||||
return {**_pose_index_status, "indexed": len(idx)}
|
||||
|
||||
|
||||
def _rank_similar_poses(query_desc, limit, exclude=None):
|
||||
idx = _load_pose_index()
|
||||
scored = []
|
||||
for fn, desc in idx.items():
|
||||
if fn == exclude or not desc or "vec" not in desc:
|
||||
continue
|
||||
d = _pose_distance(query_desc, desc)
|
||||
if d != float("inf"):
|
||||
scored.append((d, fn))
|
||||
scored.sort(key=lambda x: x[0])
|
||||
groups = get_groups() if scored else {}
|
||||
return [{"filename": fn, "group_id": groups.get(fn), "distance": round(d, 4)}
|
||||
for d, fn in scored[:limit]]
|
||||
|
||||
|
||||
@app.get("/pose/similar/{filename}")
|
||||
def similar_pose(filename: str, limit: int = 12):
|
||||
"""Rank library images by pose similarity to the given image."""
|
||||
idx = _load_pose_index()
|
||||
query = idx.get(filename)
|
||||
if query is None:
|
||||
# Compute on demand (also caches it).
|
||||
person = database.get_person(filename)
|
||||
if not person or not person[5] or not os.path.exists(person[5]):
|
||||
raise HTTPException(404, "Image file not found")
|
||||
est = _load_pose_estimator()
|
||||
if not est:
|
||||
raise HTTPException(501, "No pose estimator installed.")
|
||||
best = _best_person(est[0](Image.open(person[5]).convert("RGB")))
|
||||
query = _pose_descriptor(best) if best is not None else None
|
||||
if query is None:
|
||||
raise HTTPException(404, "No detectable pose in this image")
|
||||
try:
|
||||
_save_pose_index_entry(filename, query)
|
||||
except Exception:
|
||||
pass
|
||||
return {"filename": filename, "similar": _rank_similar_poses(query, limit, exclude=filename)}
|
||||
|
||||
|
||||
class PoseSimilarRequest(BaseModel):
|
||||
keypoints: list[list[float]] # [[x,y,score], ...17] in image pixels
|
||||
width: int = 0
|
||||
height: int = 0
|
||||
limit: int = 12
|
||||
|
||||
|
||||
@app.post("/pose/similar")
|
||||
def similar_pose_from_keypoints(req: PoseSimilarRequest):
|
||||
"""Rank library images by similarity to a supplied (e.g. hand-edited) skeleton."""
|
||||
query = _pose_descriptor(req.keypoints)
|
||||
if query is None:
|
||||
raise HTTPException(400, "Supplied pose is too sparse to match")
|
||||
return {"similar": _rank_similar_poses(query, req.limit)}
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import uvicorn
|
||||
uvicorn.run(app, host=os.environ.get("HOST", "0.0.0.0"),
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,18 +0,0 @@
|
||||
[Unit]
|
||||
Description=Qwen-Image-Edit FastAPI Service
|
||||
After=comfyui-backend.service
|
||||
Requires=comfyui-backend.service
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=__USER__
|
||||
Group=__GROUP__
|
||||
WorkingDirectory=__BASE__/api
|
||||
ExecStart=/bin/bash __BASE__/api/start_api.sh
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
StandardOutput=journal
|
||||
StandardError=journal
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -1,17 +0,0 @@
|
||||
[Unit]
|
||||
Description=ComfyUI Backend for Qwen-Image-Edit
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=__USER__
|
||||
Group=__GROUP__
|
||||
WorkingDirectory=__BASE__
|
||||
ExecStart=/bin/bash __BASE__/api/run_comfyui.sh
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
StandardOutput=journal
|
||||
StandardError=journal
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
Reference in New Issue
Block a user