docs(config): clarify profile names and make examples copy-paste safe
Rename the size-numbered profiles so the digits read as parameter count, not Qwen version: qwen35 -> qwen35b (llama.cpp.config.example, build_archive smoke test, SECURITY_AND_OPERATIONS.md) and qwen27 -> qwen27b (llama.cpp.config). Drop the misleading `--profile qwen35` from the README/man/install examples: that profile only exists in the .example file, so pasted commands failed against the real config. Primary examples now omit --profile (using the [default] section, which always resolves), with one example plus a note showing how to select a real [model.<name>] profile. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
53b6849c27
commit
d5a69e8bf8
7 changed files with 41 additions and 25 deletions
31
README.md
31
README.md
|
|
@ -76,41 +76,50 @@ Auflösungsreihenfolge (niedrigste zu höchster Priorität):
|
||||||
## Verwendung
|
## Verwendung
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Server starten
|
# Server starten (nutzt die [default]-Sektion der Config)
|
||||||
llamacppctl --start --config llama.cpp.config --profile qwen35
|
llamacppctl --start --config llama.cpp.config
|
||||||
|
|
||||||
# Nur den geplanten docker-run-Befehl anzeigen, nichts ausführen
|
# Nur den geplanten docker-run-Befehl anzeigen, nichts ausführen
|
||||||
llamacppctl --start --config llama.cpp.config --profile qwen35 --dry-run
|
llamacppctl --start --config llama.cpp.config --dry-run
|
||||||
|
|
||||||
# Effektive Konfiguration als JSON ausgeben
|
# Effektive Konfiguration als JSON ausgeben
|
||||||
llamacppctl --print-effective-config --config llama.cpp.config --profile qwen35 --start
|
llamacppctl --print-effective-config --config llama.cpp.config --start
|
||||||
|
|
||||||
# Status prüfen
|
# Status prüfen
|
||||||
llamacppctl --check --config llama.cpp.config --profile qwen35
|
llamacppctl --check --config llama.cpp.config
|
||||||
|
|
||||||
# Server stoppen und Container entfernen
|
# Server stoppen und Container entfernen
|
||||||
llamacppctl --stop --config llama.cpp.config --profile qwen35
|
llamacppctl --stop --config llama.cpp.config
|
||||||
|
|
||||||
|
# Ein bestimmtes Modell-Profil starten (siehe Hinweis unter dem Block)
|
||||||
|
llamacppctl --start --config llama.cpp.config --profile qwopus
|
||||||
|
|
||||||
# Modell wechseln (stop, neu konfigurieren, restart) unter Lock-Schutz
|
# Modell wechseln (stop, neu konfigurieren, restart) unter Lock-Schutz
|
||||||
llamacppctl --change --config llama.cpp.config --profile deepseek
|
llamacppctl --change --config llama.cpp.config --profile carnice
|
||||||
|
|
||||||
# Direkten Chat-Request an einen laufenden Server senden
|
# Direkten Chat-Request an einen laufenden Server senden
|
||||||
llamacppctl --chat --config llama.cpp.config --profile qwen35 \
|
llamacppctl --chat --config llama.cpp.config \
|
||||||
-s "Du antwortest kurz." -p "Was ist 2+2?"
|
-s "Du antwortest kurz." -p "Was ist 2+2?"
|
||||||
|
|
||||||
# Chat mit Live-Streaming und großem Antwortbudget (lange Texte)
|
# Chat mit Live-Streaming und großem Antwortbudget (lange Texte)
|
||||||
llamacppctl --chat --config llama.cpp.config --profile qwen35 \
|
llamacppctl --chat --config llama.cpp.config \
|
||||||
--stream --max-tokens 8000 -p "Schreibe eine kurze Rede zum Jahreswechsel."
|
--stream --max-tokens 8000 -p "Schreibe eine kurze Rede zum Jahreswechsel."
|
||||||
|
|
||||||
# System-Prompt aus einer lokalen Datei, User-Prompt als Literal
|
# System-Prompt aus einer lokalen Datei, User-Prompt als Literal
|
||||||
llamacppctl --start --config llama.cpp.config --profile qwen35 \
|
llamacppctl --start --config llama.cpp.config \
|
||||||
--system-file ./prompts/coding.md -p "Refaktoriere diese Funktion."
|
--system-file ./prompts/coding.md -p "Refaktoriere diese Funktion."
|
||||||
|
|
||||||
# System-Prompt von einer HTTPS-URL (öffentliches, nicht-privates Ziel)
|
# System-Prompt von einer HTTPS-URL (öffentliches, nicht-privates Ziel)
|
||||||
llamacppctl --chat --config llama.cpp.config --profile qwen35 \
|
llamacppctl --chat --config llama.cpp.config \
|
||||||
--system-url https://example.com/prompts/system.txt -p "Hallo"
|
--system-url https://example.com/prompts/system.txt -p "Hallo"
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Ohne `--profile` gilt die `[default]`-Sektion. Mit `--profile <name>` wählst du
|
||||||
|
ein `[model.<name>]`-Profil; in `llama.cpp.config` sind das z. B. `carnice`,
|
||||||
|
`qwen27b`, `qwopus`, in der Vorlage `llama.cpp.config.example` `qwen35b` und
|
||||||
|
`deepseek`. Die Ziffern (z. B. `35b`) meinen die Modellgröße in Milliarden
|
||||||
|
Parametern, nicht die Qwen-Version.
|
||||||
|
|
||||||
Vollständige Optionsliste: `llamacppctl --help` oder die Manpage
|
Vollständige Optionsliste: `llamacppctl --help` oder die Manpage
|
||||||
(`man/llamacppctl.1`, siehe unten).
|
(`man/llamacppctl.1`, siehe unten).
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -335,7 +335,7 @@ def smoke_test_install(output_path: Path) -> None:
|
||||||
str(llamacppctl_bin),
|
str(llamacppctl_bin),
|
||||||
"--print-effective-config",
|
"--print-effective-config",
|
||||||
"--config", str(config_example),
|
"--config", str(config_example),
|
||||||
"--profile", "qwen35",
|
"--profile", "qwen35b",
|
||||||
"--start",
|
"--start",
|
||||||
"--dry-run",
|
"--dry-run",
|
||||||
],
|
],
|
||||||
|
|
|
||||||
|
|
@ -48,6 +48,6 @@ cd llamacppctl
|
||||||
python3 -m venv .venv && source .venv/bin/activate
|
python3 -m venv .venv && source .venv/bin/activate
|
||||||
pip install .
|
pip install .
|
||||||
cp llama.cpp.config.example llama.cpp.config # anpassen
|
cp llama.cpp.config.example llama.cpp.config # anpassen
|
||||||
llamacppctl --print-effective-config --config llama.cpp.config --profile qwen35 --start
|
llamacppctl --print-effective-config --config llama.cpp.config --start
|
||||||
|
|
||||||
Da build_archive.py selbst mit im Archiv liegt, kannst du das Archiv auf deinem eigenen Rechner jederzeit neu bauen und erneut verifizieren, ganz ohne auf mich angewiesen zu sein.
|
Da build_archive.py selbst mit im Archiv liegt, kannst du das Archiv auf deinem eigenen Rechner jederzeit neu bauen und erneut verifizieren, ganz ohne auf mich angewiesen zu sein.
|
||||||
|
|
|
||||||
|
|
@ -263,8 +263,8 @@ gpu_device = 0
|
||||||
container_name = llama_cpp_server
|
container_name = llama_cpp_server
|
||||||
host_port = 8001
|
host_port = 8001
|
||||||
|
|
||||||
[model.qwen35]
|
[model.qwen35b]
|
||||||
container_name = llama_cpp_qwen35
|
container_name = llama_cpp_qwen35b
|
||||||
gpu_device = 1
|
gpu_device = 1
|
||||||
host_port = 8002
|
host_port = 8002
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -66,7 +66,7 @@ chat_temperature =
|
||||||
[model.carnice]
|
[model.carnice]
|
||||||
model_path = models/qwen3/Carnice-Qwen3.6-MoE-35B-A3B-Q4_K_M.gguf
|
model_path = models/qwen3/Carnice-Qwen3.6-MoE-35B-A3B-Q4_K_M.gguf
|
||||||
|
|
||||||
[model.qwen27]
|
[model.qwen27b]
|
||||||
model_path = models/qwen3/Qwen3.6-27B-Uncensored-HauhauCS-Aggressive-IQ4_XS.gguf
|
model_path = models/qwen3/Qwen3.6-27B-Uncensored-HauhauCS-Aggressive-IQ4_XS.gguf
|
||||||
|
|
||||||
[model.qwopus]
|
[model.qwopus]
|
||||||
|
|
|
||||||
|
|
@ -63,10 +63,10 @@ chat_temperature =
|
||||||
|
|
||||||
# Example second model profile, pinned to the second GPU (e.g. RTX 3090 #2)
|
# Example second model profile, pinned to the second GPU (e.g. RTX 3090 #2)
|
||||||
# with its own port and container name so it can run alongside [default].
|
# with its own port and container name so it can run alongside [default].
|
||||||
[model.qwen35]
|
[model.qwen35b]
|
||||||
model_path = qwen3/Qwen3-35B-A3B-Q4_K_M.gguf
|
model_path = qwen3/Qwen3-35B-A3B-Q4_K_M.gguf
|
||||||
model_alias = qwen35
|
model_alias = qwen35b
|
||||||
container_name = llama_cpp_qwen35
|
container_name = llama_cpp_qwen35b
|
||||||
host_port = 8002
|
host_port = 8002
|
||||||
gpu_device = 1
|
gpu_device = 1
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -273,32 +273,39 @@ HTTP request failed or server did not become ready in time.
|
||||||
Lock busy (a concurrent \fB\-\-change\fR is already running for the same \fBcontainer_name\fR).
|
Lock busy (a concurrent \fB\-\-change\fR is already running for the same \fBcontainer_name\fR).
|
||||||
.SH EXAMPLES
|
.SH EXAMPLES
|
||||||
.PP
|
.PP
|
||||||
Start a profile and show the effective configuration first:
|
Start the default profile and show the effective configuration first:
|
||||||
.RS
|
.RS
|
||||||
.nf
|
.nf
|
||||||
llamacppctl \-\-print\-effective\-config \-\-config llama.cpp.config \-\-profile qwen35 \-\-start
|
llamacppctl \-\-print\-effective\-config \-\-config llama.cpp.config \-\-start
|
||||||
llamacppctl \-\-start \-\-config llama.cpp.config \-\-profile qwen35
|
llamacppctl \-\-start \-\-config llama.cpp.config
|
||||||
.fi
|
.fi
|
||||||
.RE
|
.RE
|
||||||
.PP
|
.PP
|
||||||
Preview the docker run command without executing it:
|
Preview the docker run command without executing it:
|
||||||
.RS
|
.RS
|
||||||
.nf
|
.nf
|
||||||
llamacppctl \-\-start \-\-config llama.cpp.config \-\-profile qwen35 \-\-dry\-run
|
llamacppctl \-\-start \-\-config llama.cpp.config \-\-dry\-run
|
||||||
.fi
|
.fi
|
||||||
.RE
|
.RE
|
||||||
.PP
|
.PP
|
||||||
Check status including a live chat completion:
|
Check status including a live chat completion:
|
||||||
.RS
|
.RS
|
||||||
.nf
|
.nf
|
||||||
llamacppctl \-\-check \-\-config llama.cpp.config \-\-profile qwen35 \-p "ping"
|
llamacppctl \-\-check \-\-config llama.cpp.config \-p "ping"
|
||||||
|
.fi
|
||||||
|
.RE
|
||||||
|
.PP
|
||||||
|
Select a named model profile from a \fB[model.<name>]\fR section:
|
||||||
|
.RS
|
||||||
|
.nf
|
||||||
|
llamacppctl \-\-start \-\-config llama.cpp.config \-\-profile qwopus
|
||||||
.fi
|
.fi
|
||||||
.RE
|
.RE
|
||||||
.PP
|
.PP
|
||||||
Switch models under lock protection:
|
Switch models under lock protection:
|
||||||
.RS
|
.RS
|
||||||
.nf
|
.nf
|
||||||
llamacppctl \-\-change \-\-config llama.cpp.config \-\-profile deepseek
|
llamacppctl \-\-change \-\-config llama.cpp.config \-\-profile carnice
|
||||||
.fi
|
.fi
|
||||||
.RE
|
.RE
|
||||||
.SH AUTHOR
|
.SH AUTHOR
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue