ci/qwen-image
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

dphn/Dolphin3.0-Mistral-24B is the ungated mirror of the Dolphin 3.0 Mistral 24B — exactly what you asked for. It's ~48GB fp16, which needs GPU+CPU split (device_map="auto" with 32GB on GPU, ~16GB in RAM). Let me kick off the download and update the service in parallel.

Browse Source
This commit is contained in:
mike
2026-06-24 03:30:33 +02:00
parent dad8b244f7
commit 42f924566e
6 changed files with 342 additions and 1692 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
tour-comfy/pose_llm_api.py
View File

@@ -53,7 +53,7 @@ _MISTRAL_TEMPLATE = (
MODEL_ID = os.environ.get("MODEL_ID", "dphn/Dolphin3.0-Mistral-24B")
# For 24B fp16 (~48GB): split across 32GB VRAM + CPU RAM via device_map="auto".
# Override with MAX_GPU_MEM / MAX_CPU_MEM env vars if needed.
MAX_GPU_MEM = os.environ.get("MAX_GPU_MEM", "30GiB") # leave ~2GB headroom
MAX_GPU_MEM = os.environ.get("MAX_GPU_MEM", "27GiB") # 32GiB total - 4GiB sys - 1GiB headroom
MAX_CPU_MEM = os.environ.get("MAX_CPU_MEM", "100GiB") # 113GB available
state: dict = {}