aa

tour-comfy/README.md

@@ -0,0 +1,72 @@
+# Qwen-Image-Edit Rapid-AIO v23 — headless edit API (tour / MI50)
+
+Runs Phr00t's **Rapid-AIO NSFW v23** (the same tune as the gradio app) as a
+**Q8 GGUF** on the AMD Instinct **MI50 32GB** (`tour`, gfx906, ROCm 5.7,
+torch 2.3.1), served by ComfyUI behind a thin FastAPI throughput endpoint:
+**image + prompt in → edited PNG out**.
+
+## Layout on `tour`
+```
+/media/tour/APPS/comfyui/
+  venv/                       torch 2.3.1+rocm5.7 + ComfyUI deps
+  ComfyUI/                    backend + ComfyUI-GGUF custom node
+    models/unet/Qwen-Rapid-NSFW-v23_Q8_0.gguf          (21.8 GB)
+    models/text_encoders/qwen_2.5_vl_7b_fp8_scaled.safetensors (9.4 GB)
+    models/vae/qwen_image_vae.safetensors              (0.25 GB)
+  api/                        edit_api.py + workflow + scripts
+```
+
+## Run
+```bash
+# 1) backend (terminal 1)
+/media/tour/APPS/comfyui/api/run_comfyui.sh
+# 2) API (terminal 2)
+/media/tour/APPS/comfyui/api/start_api.sh
+```
+
+## Use
+```bash
+curl -s -X POST http://192.168.1.160:8500/edit \
+  -F image=@car.png \
+  -F 'prompt=put the car on a beach at sunset' \
+  -F seed=-1 -F steps=4 \
+  -o out.png
+```
+Form fields: `prompt` (required), `seed` (-1 = random), `steps` (4),
+`cfg` (1.0), `sampler_name` (euler_ancestral), `scheduler` (beta),
+`max_area` (px, 0 = server default ~1MP). Response headers expose
+`X-Seed`, `X-Width`, `X-Height`.
+
+Settings match Phr00t's v23 recommendation: **euler_ancestral / beta, 4 steps,
+CFG 1**. Output size tracks the input aspect ratio, scaled to `MAX_AREA`.
+
+## Performance (MI50, gfx906, 4 steps)
+Compute-bound — the 20.8GB unet stays resident; only the text encoder swaps.
+| Output budget | Latency |
+|---------------|---------|
+| 0.59 MP       | ~110 s  |
+| 0.79 MP (default) | ~140 s |
+| 1.0 MP        | ~180 s  |
+
+## Gotchas (why it's built this way)
+- **ComfyUI pinned to `v0.3.77`.** `master` imports `comfy_kitchen`, which needs
+  `torch.library.custom_op` (torch ≥ 2.4). gfx906/ROCm-5.7 tops out at the proven
+  `torch 2.3.1+rocm5.7`, so newer ComfyUI won't import. v0.3.77 is the newest tag
+  that still has `TextEncodeQwenImageEditPlus` without that dependency.
+- **`--use-split-cross-attention`** (in `run_comfyui.sh`) is required: the default
+  pytorch attention path OOMs a 20B edit model on 32GB. Split attention chunks the
+  attention matmul so it fits.
+- The audio custom-node import errors in the ComfyUI log (`libcudart.so.13`) are
+  harmless — a CUDA `torchaudio` got pulled in; image editing doesn't use it.
+
+## Manage (systemd)
+```bash
+sudo bash /media/tour/APPS/comfyui/api/deploy.sh  # (re)install and start services
+sudo bash /media/tour/APPS/comfyui/api/stop.sh    # stop and disable services
+```
+
+### Logs
+```bash
+journalctl -u comfyui-backend -f
+journalctl -u comfyui-api -f
+```

tour-comfy/deploy.sh

@@ -0,0 +1,30 @@
+#!/bin/bash
+# Deploy systemd services for Qwen-Image-Edit
+set -e
+
+# Must be run with sudo to copy to /etc/systemd/system
+if [[ $EUID -ne 0 ]]; then
+   echo "This script must be run as root (use sudo)"
+   exit 1
+fi
+
+SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
+SYSTEMD_DIR="$SCRIPT_DIR/systemd"
+
+echo "Copying service files..."
+cp "$SYSTEMD_DIR/comfyui-backend.service" /etc/systemd/system/
+cp "$SYSTEMD_DIR/comfyui-api.service" /etc/systemd/system/
+
+echo "Reloading systemd daemon..."
+systemctl daemon-reload
+
+echo "Enabling services..."
+systemctl enable comfyui-backend.service
+systemctl enable comfyui-api.service
+
+echo "Starting services..."
+systemctl restart comfyui-backend.service
+systemctl restart comfyui-api.service
+
+echo "Deployment complete."
+echo "Check status with: systemctl status comfyui-backend comfyui-api"

tour-comfy/edit_api.py

@@ -0,0 +1,191 @@
+"""
+edit_api.py — headless throughput API for Qwen-Image-Edit Rapid-AIO (v23 Q8 GGUF)
+running on top of a local ComfyUI server.
+
+Flow per request:  image + prompt  ->  upload to ComfyUI  ->  inject into the
+workflow graph  ->  queue  ->  poll until done  ->  return the edited PNG.
+
+Run ComfyUI first (run_comfyui.sh), then this service (start_api.sh).
+"""
+
+import io
+import os
+import json
+import time
+import uuid
+import random
+import copy
+
+import requests
+from PIL import Image
+from fastapi import FastAPI, UploadFile, File, Form, HTTPException
+from fastapi.responses import Response
+
+# --- config -----------------------------------------------------------------
+COMFY = os.environ.get("COMFY_URL", "http://127.0.0.1:8188").rstrip("/")
+WORKFLOW_PATH = os.environ.get(
+    "WORKFLOW_PATH",
+    os.path.join(os.path.dirname(os.path.abspath(__file__)), "workflow_qwen_edit.json"),
+)
+# Default target pixel area for the output latent. The MI50 is not fast, so we
+# cap at ~1MP by default; raise via MAX_AREA env if you want bigger output.
+MAX_AREA = int(os.environ.get("MAX_AREA", str(1024 * 1024)))
+GEN_TIMEOUT = int(os.environ.get("GEN_TIMEOUT", "600"))  # seconds per request
+
+# Node ids in workflow_qwen_edit.json (kept stable on purpose).
+NODE_LOADIMAGE = "4"
+NODE_POSITIVE = "5"
+NODE_LATENT = "7"
+NODE_KSAMPLER = "8"
+NODE_SAVE = "10"
+
+MAX_SEED = 2**32 - 1
+
+with open(WORKFLOW_PATH, "r", encoding="utf-8") as f:
+    BASE_WORKFLOW = json.load(f)
+
+app = FastAPI(title="Qwen-Image-Edit Rapid-AIO API", version="1.0")
+
+
+# --- helpers ----------------------------------------------------------------
+def _round16(x: int) -> int:
+    return max(16, int(round(x / 16.0)) * 16)
+
+
+def _target_size(w: int, h: int, max_area: int) -> tuple[int, int]:
+    """Scale (w, h) to ~max_area preserving aspect, rounded to /16."""
+    scale = (max_area / float(w * h)) ** 0.5
+    return _round16(w * scale), _round16(h * scale)
+
+
+def _comfy_upload(img_bytes: bytes, filename: str) -> str:
+    """Upload an image to ComfyUI's input dir; return the stored name."""
+    r = requests.post(
+        f"{COMFY}/upload/image",
+        files={"image": (filename, img_bytes, "image/png")},
+        data={"overwrite": "true", "type": "input"},
+        timeout=60,
+    )
+    r.raise_for_status()
+    j = r.json()
+    name = j["name"]
+    sub = j.get("subfolder", "")
+    return f"{sub}/{name}" if sub else name
+
+
+def _comfy_queue(graph: dict, client_id: str) -> str:
+    r = requests.post(
+        f"{COMFY}/prompt",
+        json={"prompt": graph, "client_id": client_id},
+        timeout=60,
+    )
+    if r.status_code != 200:
+        raise HTTPException(502, f"ComfyUI rejected workflow: {r.text}")
+    return r.json()["prompt_id"]
+
+
+def _comfy_wait(prompt_id: str, deadline: float) -> dict:
+    """Poll /history until the prompt finishes; return its outputs dict."""
+    while time.time() < deadline:
+        r = requests.get(f"{COMFY}/history/{prompt_id}", timeout=30)
+        if r.status_code == 200:
+            hist = r.json()
+            if prompt_id in hist:
+                entry = hist[prompt_id]
+                status = entry.get("status", {})
+                if status.get("status_str") == "error":
+                    raise HTTPException(500, f"ComfyUI execution error: {json.dumps(status)}")
+                outputs = entry.get("outputs", {})
+                if outputs:
+                    return outputs
+        time.sleep(0.5)
+    raise HTTPException(504, f"Generation timed out after {GEN_TIMEOUT}s")
+
+
+def _comfy_fetch_image(outputs: dict) -> bytes:
+    node_out = outputs.get(NODE_SAVE) or next(
+        (v for v in outputs.values() if "images" in v), None
+    )
+    if not node_out or not node_out.get("images"):
+        raise HTTPException(500, "No output image produced")
+    img = node_out["images"][0]
+    r = requests.get(
+        f"{COMFY}/view",
+        params={
+            "filename": img["filename"],
+            "subfolder": img.get("subfolder", ""),
+            "type": img.get("type", "output"),
+        },
+        timeout=60,
+    )
+    r.raise_for_status()
+    return r.content
+
+
+# --- routes -----------------------------------------------------------------
+@app.get("/health")
+def health():
+    try:
+        requests.get(f"{COMFY}/system_stats", timeout=5).raise_for_status()
+        return {"status": "ok", "comfy": COMFY}
+    except Exception as e:
+        raise HTTPException(503, f"ComfyUI unreachable at {COMFY}: {e}")
+
+
+@app.post("/edit")
+async def edit(
+    image: UploadFile = File(...),
+    prompt: str = Form(...),
+    seed: int = Form(-1),
+    steps: int = Form(4),
+    cfg: float = Form(1.0),
+    sampler_name: str = Form("euler_ancestral"),
+    scheduler: str = Form("beta"),
+    max_area: int = Form(0),
+):
+    raw = await image.read()
+    try:
+        pil = Image.open(io.BytesIO(raw)).convert("RGB")
+    except Exception as e:
+        raise HTTPException(400, f"Invalid image: {e}")
+
+    area = max_area if max_area > 0 else MAX_AREA
+    w, h = _target_size(pil.width, pil.height, area)
+
+    buf = io.BytesIO()
+    pil.save(buf, format="PNG")
+    stored = _comfy_upload(buf.getvalue(), f"in_{uuid.uuid4().hex[:8]}.png")
+
+    if seed is None or seed < 0:
+        seed = random.randint(0, MAX_SEED)
+
+    graph = copy.deepcopy(BASE_WORKFLOW)
+    graph[NODE_LOADIMAGE]["inputs"]["image"] = stored
+    graph[NODE_POSITIVE]["inputs"]["prompt"] = prompt
+    graph[NODE_LATENT]["inputs"]["width"] = w
+    graph[NODE_LATENT]["inputs"]["height"] = h
+    ks = graph[NODE_KSAMPLER]["inputs"]
+    ks.update(seed=seed, steps=steps, cfg=cfg,
+              sampler_name=sampler_name, scheduler=scheduler)
+
+    client_id = uuid.uuid4().hex
+    prompt_id = _comfy_queue(graph, client_id)
+    outputs = _comfy_wait(prompt_id, time.time() + GEN_TIMEOUT)
+    png = _comfy_fetch_image(outputs)
+
+    return Response(
+        content=png,
+        media_type="image/png",
+        headers={
+            "X-Seed": str(seed),
+            "X-Width": str(w),
+            "X-Height": str(h),
+            "X-Prompt-Id": prompt_id,
+        },
+    )
+
+
+if __name__ == "__main__":
+    import uvicorn
+    uvicorn.run(app, host=os.environ.get("HOST", "0.0.0.0"),
+                port=int(os.environ.get("PORT", "8500")))

tour-comfy/run_comfyui.sh

@@ -0,0 +1,19 @@
+#!/bin/bash
+# Launch the ComfyUI backend (headless) for the Qwen-Image-Edit API.
+# gfx906 (MI50) has no flash-attention, so use the pytorch cross-attention path.
+set -e
+BASE=/media/tour/APPS/comfyui
+cd "$BASE/ComfyUI"
+source "$BASE/venv/bin/activate"
+
+# MI50 / Vega20 is happiest in fp16; avoid bf16 emulation.
+export PYTORCH_HIP_ALLOC_CONF=expandable_segments:True
+export HSA_ENABLE_SDMA=0
+
+# Split cross-attention chunks the attention matmul -> much lower peak VRAM,
+# which is what lets the 20B Q8 edit model + reference-image sequence fit in 32GB.
+exec python main.py \
+    --listen 127.0.0.1 \
+    --port 8188 \
+    --use-split-cross-attention \
+    "$@"

tour-comfy/start_api.sh

@@ -0,0 +1,16 @@
+#!/bin/bash
+# Launch the FastAPI edit service (talks to the local ComfyUI on :8188).
+set -e
+BASE=/media/tour/APPS/comfyui
+source "$BASE/venv/bin/activate"
+cd "$BASE/api"
+
+export COMFY_URL="http://127.0.0.1:8188"
+export HOST="0.0.0.0"
+export PORT="8500"
+# Output pixel budget. MI50 is compute-bound on this 20B model:
+#   ~0.59MP -> ~110s   ~0.79MP -> ~140s   ~1.0MP -> ~180s  (4 steps)
+# 0.79MP is a sane speed/quality default; raise for bigger output.
+export MAX_AREA="${MAX_AREA:-786432}"
+
+exec python edit_api.py

tour-comfy/stop.sh

@@ -0,0 +1,18 @@
+#!/bin/bash
+# Stop and disable systemd services for Qwen-Image-Edit
+set -e
+
+if [[ $EUID -ne 0 ]]; then
+   echo "This script must be run as root (use sudo)"
+   exit 1
+fi
+
+echo "Stopping services..."
+systemctl stop comfyui-api.service
+systemctl stop comfyui-backend.service
+
+echo "Disabling services..."
+systemctl disable comfyui-api.service
+systemctl disable comfyui-backend.service
+
+echo "Services stopped and disabled."

tour-comfy/systemd/comfyui-api.service

@@ -0,0 +1,17 @@
+[Unit]
+Description=Qwen-Image-Edit FastAPI Service
+After=comfyui-backend.service
+
+[Service]
+Type=simple
+User=tour
+Group=tour
+WorkingDirectory=/media/tour/APPS/comfyui/api
+ExecStart=/bin/bash /media/tour/APPS/comfyui/api/start_api.sh
+Restart=always
+RestartSec=5
+StandardOutput=journal
+StandardError=journal
+
+[Install]
+WantedBy=multi-user.target

tour-comfy/systemd/comfyui-backend.service

@@ -0,0 +1,17 @@
+[Unit]
+Description=ComfyUI Backend for Qwen-Image-Edit
+After=network.target
+
+[Service]
+Type=simple
+User=tour
+Group=tour
+WorkingDirectory=/media/tour/APPS/comfyui
+ExecStart=/bin/bash /media/tour/APPS/comfyui/api/run_comfyui.sh
+Restart=always
+RestartSec=5
+StandardOutput=journal
+StandardError=journal
+
+[Install]
+WantedBy=multi-user.target