docs(config): clarify profile names and make examples copy-paste safe
Rename the size-numbered profiles so the digits read as parameter count, not Qwen version: qwen35 -> qwen35b (llama.cpp.config.example, build_archive smoke test, SECURITY_AND_OPERATIONS.md) and qwen27 -> qwen27b (llama.cpp.config). Drop the misleading `--profile qwen35` from the README/man/install examples: that profile only exists in the .example file, so pasted commands failed against the real config. Primary examples now omit --profile (using the [default] section, which always resolves), with one example plus a note showing how to select a real [model.<name>] profile. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
53b6849c27
commit
d5a69e8bf8
7 changed files with 41 additions and 25 deletions
31
README.md
31
README.md
|
|
@ -76,41 +76,50 @@ Auflösungsreihenfolge (niedrigste zu höchster Priorität):
|
|||
## Verwendung
|
||||
|
||||
```bash
|
||||
# Server starten
|
||||
llamacppctl --start --config llama.cpp.config --profile qwen35
|
||||
# Server starten (nutzt die [default]-Sektion der Config)
|
||||
llamacppctl --start --config llama.cpp.config
|
||||
|
||||
# Nur den geplanten docker-run-Befehl anzeigen, nichts ausführen
|
||||
llamacppctl --start --config llama.cpp.config --profile qwen35 --dry-run
|
||||
llamacppctl --start --config llama.cpp.config --dry-run
|
||||
|
||||
# Effektive Konfiguration als JSON ausgeben
|
||||
llamacppctl --print-effective-config --config llama.cpp.config --profile qwen35 --start
|
||||
llamacppctl --print-effective-config --config llama.cpp.config --start
|
||||
|
||||
# Status prüfen
|
||||
llamacppctl --check --config llama.cpp.config --profile qwen35
|
||||
llamacppctl --check --config llama.cpp.config
|
||||
|
||||
# Server stoppen und Container entfernen
|
||||
llamacppctl --stop --config llama.cpp.config --profile qwen35
|
||||
llamacppctl --stop --config llama.cpp.config
|
||||
|
||||
# Ein bestimmtes Modell-Profil starten (siehe Hinweis unter dem Block)
|
||||
llamacppctl --start --config llama.cpp.config --profile qwopus
|
||||
|
||||
# Modell wechseln (stop, neu konfigurieren, restart) unter Lock-Schutz
|
||||
llamacppctl --change --config llama.cpp.config --profile deepseek
|
||||
llamacppctl --change --config llama.cpp.config --profile carnice
|
||||
|
||||
# Direkten Chat-Request an einen laufenden Server senden
|
||||
llamacppctl --chat --config llama.cpp.config --profile qwen35 \
|
||||
llamacppctl --chat --config llama.cpp.config \
|
||||
-s "Du antwortest kurz." -p "Was ist 2+2?"
|
||||
|
||||
# Chat mit Live-Streaming und großem Antwortbudget (lange Texte)
|
||||
llamacppctl --chat --config llama.cpp.config --profile qwen35 \
|
||||
llamacppctl --chat --config llama.cpp.config \
|
||||
--stream --max-tokens 8000 -p "Schreibe eine kurze Rede zum Jahreswechsel."
|
||||
|
||||
# System-Prompt aus einer lokalen Datei, User-Prompt als Literal
|
||||
llamacppctl --start --config llama.cpp.config --profile qwen35 \
|
||||
llamacppctl --start --config llama.cpp.config \
|
||||
--system-file ./prompts/coding.md -p "Refaktoriere diese Funktion."
|
||||
|
||||
# System-Prompt von einer HTTPS-URL (öffentliches, nicht-privates Ziel)
|
||||
llamacppctl --chat --config llama.cpp.config --profile qwen35 \
|
||||
llamacppctl --chat --config llama.cpp.config \
|
||||
--system-url https://example.com/prompts/system.txt -p "Hallo"
|
||||
```
|
||||
|
||||
Ohne `--profile` gilt die `[default]`-Sektion. Mit `--profile <name>` wählst du
|
||||
ein `[model.<name>]`-Profil; in `llama.cpp.config` sind das z. B. `carnice`,
|
||||
`qwen27b`, `qwopus`, in der Vorlage `llama.cpp.config.example` `qwen35b` und
|
||||
`deepseek`. Die Ziffern (z. B. `35b`) meinen die Modellgröße in Milliarden
|
||||
Parametern, nicht die Qwen-Version.
|
||||
|
||||
Vollständige Optionsliste: `llamacppctl --help` oder die Manpage
|
||||
(`man/llamacppctl.1`, siehe unten).
|
||||
|
||||
|
|
|
|||
|
|
@ -335,7 +335,7 @@ def smoke_test_install(output_path: Path) -> None:
|
|||
str(llamacppctl_bin),
|
||||
"--print-effective-config",
|
||||
"--config", str(config_example),
|
||||
"--profile", "qwen35",
|
||||
"--profile", "qwen35b",
|
||||
"--start",
|
||||
"--dry-run",
|
||||
],
|
||||
|
|
|
|||
|
|
@ -48,6 +48,6 @@ cd llamacppctl
|
|||
python3 -m venv .venv && source .venv/bin/activate
|
||||
pip install .
|
||||
cp llama.cpp.config.example llama.cpp.config # anpassen
|
||||
llamacppctl --print-effective-config --config llama.cpp.config --profile qwen35 --start
|
||||
llamacppctl --print-effective-config --config llama.cpp.config --start
|
||||
|
||||
Da build_archive.py selbst mit im Archiv liegt, kannst du das Archiv auf deinem eigenen Rechner jederzeit neu bauen und erneut verifizieren, ganz ohne auf mich angewiesen zu sein.
|
||||
|
|
|
|||
|
|
@ -263,8 +263,8 @@ gpu_device = 0
|
|||
container_name = llama_cpp_server
|
||||
host_port = 8001
|
||||
|
||||
[model.qwen35]
|
||||
container_name = llama_cpp_qwen35
|
||||
[model.qwen35b]
|
||||
container_name = llama_cpp_qwen35b
|
||||
gpu_device = 1
|
||||
host_port = 8002
|
||||
|
||||
|
|
|
|||
|
|
@ -66,7 +66,7 @@ chat_temperature =
|
|||
[model.carnice]
|
||||
model_path = models/qwen3/Carnice-Qwen3.6-MoE-35B-A3B-Q4_K_M.gguf
|
||||
|
||||
[model.qwen27]
|
||||
[model.qwen27b]
|
||||
model_path = models/qwen3/Qwen3.6-27B-Uncensored-HauhauCS-Aggressive-IQ4_XS.gguf
|
||||
|
||||
[model.qwopus]
|
||||
|
|
|
|||
|
|
@ -63,10 +63,10 @@ chat_temperature =
|
|||
|
||||
# Example second model profile, pinned to the second GPU (e.g. RTX 3090 #2)
|
||||
# with its own port and container name so it can run alongside [default].
|
||||
[model.qwen35]
|
||||
[model.qwen35b]
|
||||
model_path = qwen3/Qwen3-35B-A3B-Q4_K_M.gguf
|
||||
model_alias = qwen35
|
||||
container_name = llama_cpp_qwen35
|
||||
model_alias = qwen35b
|
||||
container_name = llama_cpp_qwen35b
|
||||
host_port = 8002
|
||||
gpu_device = 1
|
||||
|
||||
|
|
|
|||
|
|
@ -273,32 +273,39 @@ HTTP request failed or server did not become ready in time.
|
|||
Lock busy (a concurrent \fB\-\-change\fR is already running for the same \fBcontainer_name\fR).
|
||||
.SH EXAMPLES
|
||||
.PP
|
||||
Start a profile and show the effective configuration first:
|
||||
Start the default profile and show the effective configuration first:
|
||||
.RS
|
||||
.nf
|
||||
llamacppctl \-\-print\-effective\-config \-\-config llama.cpp.config \-\-profile qwen35 \-\-start
|
||||
llamacppctl \-\-start \-\-config llama.cpp.config \-\-profile qwen35
|
||||
llamacppctl \-\-print\-effective\-config \-\-config llama.cpp.config \-\-start
|
||||
llamacppctl \-\-start \-\-config llama.cpp.config
|
||||
.fi
|
||||
.RE
|
||||
.PP
|
||||
Preview the docker run command without executing it:
|
||||
.RS
|
||||
.nf
|
||||
llamacppctl \-\-start \-\-config llama.cpp.config \-\-profile qwen35 \-\-dry\-run
|
||||
llamacppctl \-\-start \-\-config llama.cpp.config \-\-dry\-run
|
||||
.fi
|
||||
.RE
|
||||
.PP
|
||||
Check status including a live chat completion:
|
||||
.RS
|
||||
.nf
|
||||
llamacppctl \-\-check \-\-config llama.cpp.config \-\-profile qwen35 \-p "ping"
|
||||
llamacppctl \-\-check \-\-config llama.cpp.config \-p "ping"
|
||||
.fi
|
||||
.RE
|
||||
.PP
|
||||
Select a named model profile from a \fB[model.<name>]\fR section:
|
||||
.RS
|
||||
.nf
|
||||
llamacppctl \-\-start \-\-config llama.cpp.config \-\-profile qwopus
|
||||
.fi
|
||||
.RE
|
||||
.PP
|
||||
Switch models under lock protection:
|
||||
.RS
|
||||
.nf
|
||||
llamacppctl \-\-change \-\-config llama.cpp.config \-\-profile deepseek
|
||||
llamacppctl \-\-change \-\-config llama.cpp.config \-\-profile carnice
|
||||
.fi
|
||||
.RE
|
||||
.SH AUTHOR
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue