dschlueter
3158d16f9b
Steuert einen llama.cpp-Server als Docker-Container: --start/--check/--stop/
--change/--chat, INI-Konfiguration (builtin defaults -> [default] ->
[model.<profile>] -> CLI), SSRF-gehärtete Prompt-Eingabe (Datei/HTTPS-URL),
File-Locking für --start/--change und ein OpenAI-kompatibler HTTP-Layer.
Enthält u. a.:
- Env-Var-Expansion in hf_home (hf_home = ${HF_HOME})
- konfigurierbares Chat-Antwortbudget (max_tokens/chat_temperature,
CLI: --max-tokens/--chat-temp); temperature defer an Server-Default
- DNS-Pinning gegen DNS-Rebinding bei URL-Quellen
- dry-run als nebenwirkungsfreie Vorschau (kein Lock/Removal/Modell-Check)
- 98 Tests (pytest)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
292 lines
9.9 KiB
Groff
292 lines
9.9 KiB
Groff
.TH LLAMACPPCTL 1 "2026-07-05" "llamacppctl 0.1.0" "User Commands"
|
|
.SH NAME
|
|
llamacppctl \- start, check, stop, switch, or prompt a llama.cpp server running as a Docker container
|
|
.SH SYNOPSIS
|
|
.B llamacppctl
|
|
\fB\-\-start\fR|\fB\-\-check\fR|\fB\-\-stop\fR|\fB\-\-change\fR|\fB\-\-chat\fR
|
|
[\fIOPTIONS\fR]
|
|
.SH DESCRIPTION
|
|
.B llamacppctl
|
|
manages a
|
|
.B llama.cpp
|
|
OpenAI-compatible inference server running inside a Docker container. It
|
|
replaces a collection of shell scripts with a single Python CLI that shares
|
|
one configuration model, one locking mechanism, and one security-hardened
|
|
prompt input layer for system and user prompts.
|
|
.SH ACTIONS
|
|
Exactly one action is required.
|
|
.TP
|
|
.B \-\-start
|
|
Start the llama.cpp server container. Removes any pre-existing container of
|
|
the same name first. Waits for readiness via real chat completions, not just
|
|
an open port.
|
|
.TP
|
|
.B \-\-check
|
|
Report Docker container status, HTTP health/model endpoint reachability, and
|
|
(if a user prompt is supplied) whether a live chat completion succeeds.
|
|
.TP
|
|
.B \-\-stop
|
|
Stop and remove the container. No-op if it does not exist.
|
|
.TP
|
|
.B \-\-change
|
|
Stop, reconfigure, and restart the server under an exclusive file lock keyed
|
|
on \fBcontainer_name\fR, preventing concurrent modification of the same
|
|
container.
|
|
.TP
|
|
.B \-\-chat
|
|
Send a single chat completion request to an already-running server and print
|
|
the response.
|
|
.SH CONFIGURATION MODEL
|
|
Configuration is a standard INI file (default path: \fIllama.cpp.config\fR,
|
|
override with \fB\-\-config\fR). Resolution order, lowest to highest
|
|
priority:
|
|
.IP 1. 4
|
|
Built-in defaults
|
|
.IP 2. 4
|
|
\fB[default]\fR section
|
|
.IP 3. 4
|
|
\fB[model.<profile>]\fR section, selected via \fB\-\-profile\fR
|
|
.IP 4. 4
|
|
CLI overrides (\fB\-\-image\fR, \fB\-\-host\-port\fR, ...)
|
|
.PP
|
|
\fB[prompt.<name>]\fR sections, selected via \fB\-\-system\-prompt\-profile\fR,
|
|
provide only a \fIfallback\fR system prompt; an explicit
|
|
\fB\-s\fR/\fB\-\-system\-file\fR/\fB\-\-system\-url\fR always wins.
|
|
.PP
|
|
\fBcontainer_name\fR is mandatory and must be non-empty after resolution. It
|
|
is the single identity anchor for Docker naming, \fB\-\-stop\fR/\fB\-\-check\fR
|
|
targeting, and the derived lock file path
|
|
(\fI/tmp/llamacppctl.<container_name>.lock\fR). Profiles intended to run
|
|
concurrently must each use a distinct \fBcontainer_name\fR.
|
|
.SH OPTIONS
|
|
.SS General
|
|
.TP
|
|
.B \-\-config \fIPATH\fR
|
|
Path to the INI configuration file. Default: \fIllama.cpp.config\fR.
|
|
.TP
|
|
.B \-\-profile \fINAME\fR
|
|
Select a \fB[model.<NAME>]\fR section.
|
|
.TP
|
|
.B \-\-system\-prompt\-profile \fINAME\fR
|
|
Select a \fB[prompt.<NAME>]\fR section for a fallback system prompt.
|
|
.SS Prompt sources (system)
|
|
Mutually exclusive.
|
|
.TP
|
|
.B \-s\fR, \fB\-\-system \fITEXT\fR
|
|
Literal system prompt text.
|
|
.TP
|
|
.B \-\-system\-file \fIPATH\fR
|
|
Read the system prompt from a local file.
|
|
.TP
|
|
.B \-\-system\-url \fIURL\fR
|
|
Read the system prompt from a remote HTTPS URL.
|
|
.SS Prompt sources (user)
|
|
Mutually exclusive.
|
|
.TP
|
|
.B \-p\fR, \fB\-\-prompt \fITEXT\fR
|
|
Literal user prompt text.
|
|
.TP
|
|
.B \-\-prompt\-file \fIPATH\fR
|
|
Read the user prompt from a local file.
|
|
.TP
|
|
.B \-\-prompt\-url \fIURL\fR
|
|
Read the user prompt from a remote HTTPS URL.
|
|
.SS Docker / server overrides
|
|
.TP
|
|
.B \-\-image \fIIMAGE\fR
|
|
Docker image, e.g. \fIghcr.io/ggml-org/llama.cpp:server-cuda\fR.
|
|
.TP
|
|
.B \-\-hf\-home \fIPATH\fR
|
|
Host path mounted read-only into the container as model storage. Environment
|
|
variables (e.g. \fB${HF_HOME}\fR) and \fB~\fR are expanded, so the config may
|
|
contain \fBhf_home = ${HF_HOME}\fR.
|
|
.TP
|
|
.B \-\-model\-path \fIPATH\fR
|
|
Model path, relative to \fB\-\-hf\-home\fR unless absolute.
|
|
.TP
|
|
.B \-\-container\-name \fINAME\fR
|
|
Docker container name. Must be unique per concurrently-running profile.
|
|
.TP
|
|
.B \-\-host\-port \fIPORT\fR\fR, \fB\-\-container\-port \fIPORT\fR
|
|
Host / container-internal TCP port (1-65535).
|
|
.TP
|
|
.B \-\-model\-alias \fINAME\fR
|
|
Model alias exposed via the OpenAI-compatible API.
|
|
.TP
|
|
.B \-\-gpu \fIID\fR
|
|
GPU device id passed to \fB\-\-gpus device=<ID>\fR.
|
|
.TP
|
|
.B \-\-restart\-policy \fIPOLICY\fR
|
|
Docker \fB\-\-restart\fR policy.
|
|
.SS llama.cpp runtime parameters
|
|
.B \-\-ctx\-size\fR, \fB\-\-n\-predict\fR, \fB\-\-temp\fR, \fB\-\-top\-p\fR,
|
|
\fB\-\-top\-k\fR, \fB\-\-min\-p\fR, \fB\-\-repeat\-penalty\fR,
|
|
\fB\-\-main\-gpu\fR, \fB\-\-ngl\fR, \fB\-\-batch\-size\fR,
|
|
\fB\-\-ubatch\-size\fR, \fB\-\-parallel\fR, \fB\-\-cache\-type\-k\fR,
|
|
\fB\-\-cache\-type\-v\fR, \fB\-\-reasoning {on,off}\fR.
|
|
.PP
|
|
Boolean pairs (unset by default, meaning "use config/default value"):
|
|
\fB\-\-jinja\fR/\fB\-\-no\-jinja\fR, \fB\-\-fa\fR/\fB\-\-no\-fa\fR,
|
|
\fB\-\-kv\-unified\fR/\fB\-\-no\-kv\-unified\fR,
|
|
\fB\-\-cont\-batching\fR/\fB\-\-no\-cont\-batching\fR,
|
|
\fB\-\-no\-context\-shift\fR/\fB\-\-context\-shift\fR.
|
|
.SS Chat request parameters
|
|
Applied to the reply produced by \fB\-\-chat\fR and the post-start reply of
|
|
\fB\-\-start\fR (they do not affect the container itself). Config keys:
|
|
\fBmax_tokens\fR, \fBchat_temperature\fR.
|
|
.TP
|
|
.B \-\-max\-tokens \fIN\fR
|
|
Maximum tokens for the chat reply. Default 2048; raise it for reasoning models
|
|
or long-form output (they spend many tokens "thinking" before the visible
|
|
answer, so a small budget yields empty content). Must be > 0.
|
|
.TP
|
|
.B \-\-chat\-temp \fIFLOAT\fR
|
|
Temperature for the chat request. If unset, the request omits temperature and
|
|
the server's configured \fB\-\-temp\fR applies (single source of truth).
|
|
.SS Security / behavior flags (prompt input)
|
|
.TP
|
|
.B \-\-allow\-insecure\-http
|
|
Permit \fBhttp://\fR URLs for \fB\-\-system\-url\fR/\fB\-\-prompt\-url\fR (default: HTTPS only).
|
|
.TP
|
|
.B \-\-allow\-private\-url
|
|
Disable private/loopback/link-local/reserved/metadata IP blocking for URL
|
|
sources. Cannot be combined with \fB\-\-url\-allow\-host\fR.
|
|
.TP
|
|
.B \-\-allow\-ip\-host
|
|
Permit IP-literal hostnames in URLs (default: rejected).
|
|
.TP
|
|
.B \-\-follow\-redirects
|
|
Follow HTTP redirects for URL sources (default: rejected). Each hop is
|
|
re-validated against the same SSRF rules. Bounded by an internal max-redirect
|
|
count.
|
|
.TP
|
|
.B \-\-allow\-html\-input
|
|
Accept \fBtext/html\fR responses in addition to the default text-like
|
|
content-type allowlist.
|
|
.TP
|
|
.B \-\-allow\-symlinks\fR / \fB\-\-no\-allow\-symlinks
|
|
Allow or reject symlinked files for \fB\-\-system\-file\fR/\fB\-\-prompt\-file\fR.
|
|
Default: rejected (opt in with \fB\-\-allow\-symlinks\fR).
|
|
.TP
|
|
.B \-\-max\-input\-bytes \fIN\fR
|
|
Maximum size in bytes for any resolved prompt input (file or URL). Default:
|
|
1048576 (1 MiB). Must be > 0.
|
|
.TP
|
|
.B \-\-url\-allow\-host \fIHOST\fR
|
|
Restrict URL sources to an explicit host allowlist. Repeatable.
|
|
.TP
|
|
.B \-\-connect\-timeout \fISECONDS\fR\fR, \fB\-\-read\-timeout \fISECONDS\fR
|
|
Network timeouts for URL sources. Must be > 0. Defaults: 3.0 / 10.0.
|
|
.SS Behavior / diagnostics
|
|
.TP
|
|
.B \-\-timeout \fISECONDS\fR
|
|
Readiness timeout for \fB\-\-start\fR/\fB\-\-change\fR. Default from config
|
|
(300s).
|
|
.TP
|
|
.B \-\-poll\-interval \fISECONDS\fR
|
|
Readiness poll interval. Default from config (2s).
|
|
.TP
|
|
.B \-\-lock\-file \fIPATH\fR
|
|
Override the lock file path used by \fB\-\-start\fR/\fB\-\-change\fR. Default:
|
|
derived from \fBcontainer_name\fR.
|
|
.TP
|
|
.B \-\-logs
|
|
Print container log tail on readiness failure.
|
|
.TP
|
|
.B \-\-log\-lines \fIN\fR
|
|
Number of log lines to print. Default: 100.
|
|
.TP
|
|
.B \-\-dry\-run
|
|
Print the fully constructed \fBdocker run\fR command and exit without
|
|
executing it. Usable with \fB\-\-start\fR/\fB\-\-change\fR.
|
|
.TP
|
|
.B \-\-force
|
|
Force stop/change despite inconsistencies.
|
|
.TP
|
|
.B \-\-print\-effective\-config
|
|
Print the fully merged configuration as JSON and exit (unless combined with
|
|
an action that also runs).
|
|
.SH SECURITY NOTES
|
|
All Docker interaction goes through \fBsubprocess.run()\fR with argument
|
|
lists, never shell strings — there is no shell-injection surface via
|
|
configuration values or CLI arguments.
|
|
.PP
|
|
The remote URL prompt source (\fB\-\-system\-url\fR/\fB\-\-prompt\-url\fR) is
|
|
SSRF-hardened by default: HTTPS-only, no IP-literal hosts, no embedded
|
|
credentials, DNS resolution and IP classification for every candidate
|
|
address (blocking private/loopback/link-local/multicast/reserved/
|
|
unspecified ranges and the cloud metadata address
|
|
\fI169.254.169.254\fR), no redirects unless explicitly enabled (with
|
|
per-hop re-validation), a content-type allowlist, and a hard size limit
|
|
enforced both via \fBContent-Length\fR and while streaming the response body.
|
|
See \fIdocs/SECURITY_AND_OPERATIONS.md\fR in the source distribution for the
|
|
full threat model.
|
|
.PP
|
|
\fBcontainer_name\fR is mandatory and must be unique per concurrently-running
|
|
profile; reusing a name across profiles that run at the same time will cause
|
|
one to replace the other's container on the next \fB\-\-start\fR.
|
|
.SH FILES
|
|
.TP
|
|
.I llama.cpp.config
|
|
Default configuration file (INI format). See \fIllama.cpp.config.example\fR
|
|
in the source distribution.
|
|
.TP
|
|
.I /tmp/llamacppctl.<container_name>.lock
|
|
Default lock file used by \fB\-\-change\fR, derived from \fBcontainer_name\fR
|
|
unless \fB\-\-lock\-file\fR is set.
|
|
.SH EXIT STATUS
|
|
.TP
|
|
.B 0
|
|
Success.
|
|
.TP
|
|
.B 1
|
|
General error (Docker unavailable, unexpected exception, prompt input error, configuration error).
|
|
.TP
|
|
.B 2
|
|
CLI argument / semantic validation error.
|
|
.TP
|
|
.B 3
|
|
Model path not found on disk.
|
|
.TP
|
|
.B 4
|
|
Docker command failed.
|
|
.TP
|
|
.B 5
|
|
HTTP request failed or server did not become ready in time.
|
|
.TP
|
|
.B 6
|
|
Lock busy (a concurrent \fB\-\-change\fR is already running for the same \fBcontainer_name\fR).
|
|
.SH EXAMPLES
|
|
.PP
|
|
Start a profile and show the effective configuration first:
|
|
.RS
|
|
.nf
|
|
llamacppctl \-\-print\-effective\-config \-\-config llama.cpp.config \-\-profile qwen35 \-\-start
|
|
llamacppctl \-\-start \-\-config llama.cpp.config \-\-profile qwen35
|
|
.fi
|
|
.RE
|
|
.PP
|
|
Preview the docker run command without executing it:
|
|
.RS
|
|
.nf
|
|
llamacppctl \-\-start \-\-config llama.cpp.config \-\-profile qwen35 \-\-dry\-run
|
|
.fi
|
|
.RE
|
|
.PP
|
|
Check status including a live chat completion:
|
|
.RS
|
|
.nf
|
|
llamacppctl \-\-check \-\-config llama.cpp.config \-\-profile qwen35 \-p "ping"
|
|
.fi
|
|
.RE
|
|
.PP
|
|
Switch models under lock protection:
|
|
.RS
|
|
.nf
|
|
llamacppctl \-\-change \-\-config llama.cpp.config \-\-profile deepseek
|
|
.fi
|
|
.RE
|
|
.SH AUTHOR
|
|
Dieter Schlueter
|
|
.SH SEE ALSO
|
|
.BR docker (1)
|