feat: GPU-Split, Phase-Timer, llama.cpp-Update-Fixes
GPU-Setup: - Coder → GPU 1 (device=1), Judge → GPU 2 (device=2), kein --tensor-split - --chat-template-kwargs deprecated → --reasoning on (neue llama.cpp-API) - Smoke-Test-Timeout 180s → 300s (neue fitting-Phase beim Start) Fortschrittsanzeige: - Phase-Timer [MM:SS] in Statuszeile während LLM-Inference (sendAndWait) - currentActivity in Fix-Phase zeigt konkreten Blocker statt generischem Text Dokumentation: - README: GPU-Tabelle, VRAM-Abschätzung, --reasoning on, Timer-Beispiele - BEDIENUNGSANLEITUNG: Ladezeit, Fortschrittsanzeige mit Timer-Beispielen Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
42d29354ad
commit
3f7ffe60c2
5 changed files with 122 additions and 72 deletions
|
|
@ -22,7 +22,7 @@ fi
|
|||
|
||||
echo "[*] Starte llama.cpp-Server für Judge ..."
|
||||
docker run -d \
|
||||
--gpus '"device=1,2"' \
|
||||
--gpus '"device=2"' \
|
||||
--name "$CONTAINER_NAME" \
|
||||
--restart unless-stopped \
|
||||
-e HF_HOME="/hf_home" \
|
||||
|
|
@ -34,7 +34,7 @@ docker run -d \
|
|||
-c 262144 \
|
||||
-n 16384 \
|
||||
--jinja \
|
||||
--chat-template-kwargs '{"enable_thinking":true}' \
|
||||
--reasoning on \
|
||||
--no-context-shift \
|
||||
--temp 0.7 \
|
||||
--top-p 0.80 \
|
||||
|
|
@ -42,7 +42,6 @@ docker run -d \
|
|||
--min-p 0.01 \
|
||||
--repeat-penalty 1.05 \
|
||||
--main-gpu 0 \
|
||||
--tensor-split 0.5,0.5 \
|
||||
-ngl 999 \
|
||||
-fa on \
|
||||
--kv-unified \
|
||||
|
|
@ -57,7 +56,7 @@ docker run -d \
|
|||
|
||||
echo "[*] Warte auf Modell-Bereitschaft (Completion-Check, max. 180 s) ..."
|
||||
MODEL_READY=0
|
||||
for i in {1..90}; do
|
||||
for i in {1..150}; do
|
||||
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" --max-time 10 \
|
||||
-X POST "http://localhost:${HOST_PORT}/v1/chat/completions" \
|
||||
-H "Content-Type: application/json" \
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue