llamacppctl/man/llamacppctl.1

.TH LLAMACPPCTL 1 "2026-07-05" "llamacppctl 0.1.0" "User Commands"
.SH NAME
llamacppctl \- start, check, stop, switch, or prompt a llama.cpp server running as a Docker container
.SH SYNOPSIS
.B llamacppctl
\fB\-\-start\fR|\fB\-\-check\fR|\fB\-\-stop\fR|\fB\-\-change\fR|\fB\-\-chat\fR
[\fIOPTIONS\fR]
.SH DESCRIPTION
.B llamacppctl
manages a
.B llama.cpp
OpenAI-compatible inference server running inside a Docker container. It
replaces a collection of shell scripts with a single Python CLI that shares
one configuration model, one locking mechanism, and one security-hardened
prompt input layer for system and user prompts.
.SH ACTIONS
Exactly one action is required.
.TP
.B \-\-start
Start the llama.cpp server container. Removes any pre-existing container of
the same name first. Waits for readiness via real chat completions, not just
an open port.
.TP
.B \-\-check
Report Docker container status, HTTP health/model endpoint reachability, and
(if a user prompt is supplied) whether a live chat completion succeeds. Exits
0 when the container is running and reachable, otherwise 5 (scriptable).
.TP
.B \-\-stop
Stop and remove the container. No-op if it does not exist.
.TP
.B \-\-change
Stop, reconfigure, and restart the server under an exclusive file lock keyed
on \fBcontainer_name\fR, preventing concurrent modification of the same
container.
.TP
.B \-\-chat
Send a single chat completion request to an already-running server and print
the response.
.SH CONFIGURATION MODEL
Configuration is a standard INI file (default path: \fIllama.cpp.config\fR,
override with \fB\-\-config\fR). Resolution order, lowest to highest
priority:
.IP 1. 4
Built-in defaults
.IP 2. 4
\fB[default]\fR section
.IP 3. 4
\fB[model.<profile>]\fR section, selected via \fB\-\-profile\fR
.IP 4. 4
CLI overrides (\fB\-\-image\fR, \fB\-\-host\-port\fR, ...)
.PP
\fB[prompt.<name>]\fR sections, selected via \fB\-\-system\-prompt\-profile\fR,
provide only a \fIfallback\fR system prompt; an explicit
\fB\-s\fR/\fB\-\-system\-file\fR/\fB\-\-system\-url\fR always wins.
.PP
\fBcontainer_name\fR is mandatory and must be non-empty after resolution. It
is the single identity anchor for Docker naming, \fB\-\-stop\fR/\fB\-\-check\fR
targeting, and the derived lock file path
(\fI/tmp/llamacppctl.<container_name>.lock\fR). Profiles intended to run
concurrently must each use a distinct \fBcontainer_name\fR.
.SH OPTIONS
.SS General
.TP
.B \-\-config \fIPATH\fR
Path to the INI configuration file. Default: \fIllama.cpp.config\fR.
.TP
.B \-\-profile \fINAME\fR
Select a \fB[model.<NAME>]\fR section.
.TP
.B \-\-system\-prompt\-profile \fINAME\fR
Select a \fB[prompt.<NAME>]\fR section for a fallback system prompt.
.SS Prompt sources (system)
Mutually exclusive.
.TP
.B \-s\fR, \fB\-\-system \fITEXT\fR
Literal system prompt text.
.TP
.B \-\-system\-file \fIPATH\fR
Read the system prompt from a local file.
.TP
.B \-\-system\-url \fIURL\fR
Read the system prompt from a remote HTTPS URL.
.SS Prompt sources (user)
Mutually exclusive.
.TP
.B \-p\fR, \fB\-\-prompt \fITEXT\fR
Literal user prompt text.
.TP
.B \-\-prompt\-file \fIPATH\fR
Read the user prompt from a local file.
.TP
.B \-\-prompt\-url \fIURL\fR
Read the user prompt from a remote HTTPS URL.
.SS Docker / server overrides
.TP
.B \-\-image \fIIMAGE\fR
Docker image, e.g. \fIghcr.io/ggml-org/llama.cpp:server-cuda\fR.
.TP
.B \-\-hf\-home \fIPATH\fR
Host path mounted read-only into the container as model storage. Environment
variables (e.g. \fB${HF_HOME}\fR) and \fB~\fR are expanded, so the config may
contain \fBhf_home = ${HF_HOME}\fR.
.TP
.B \-\-model\-path \fIPATH\fR
Model path, relative to \fB\-\-hf\-home\fR unless absolute.
.TP
.B \-\-container\-name \fINAME\fR
Docker container name. Must be unique per concurrently-running profile.
.TP
.B \-\-host\-port \fIPORT\fR\fR, \fB\-\-container\-port \fIPORT\fR
Host / container-internal TCP port (1-65535).
.TP
.B \-\-model\-alias \fINAME\fR
Model alias exposed via the OpenAI-compatible API.
.TP
.B \-\-gpu \fIID\fR
GPU device id passed to \fB\-\-gpus device=<ID>\fR.
.TP
.B \-\-restart\-policy \fIPOLICY\fR
Docker \fB\-\-restart\fR policy.
.TP
.B \-\-expose\fR / \fB\-\-no\-expose
Publish the port on all host interfaces (LAN-reachable) or on loopback only.
Default: loopback only, so the unauthenticated API is not exposed to the
network. Config key: \fBexpose\fR.
.TP
.B \-\-api\-key \fIKEY\fR
Require an API key on the server (\fB\-\-api\-key\fR) and send it as a Bearer
token on every request. Config key: \fBapi_key\fR. Recommended together with
\fB\-\-expose\fR.
.SS llama.cpp runtime parameters
.B \-\-ctx\-size\fR, \fB\-\-n\-predict\fR, \fB\-\-temp\fR, \fB\-\-top\-p\fR,
\fB\-\-top\-k\fR, \fB\-\-min\-p\fR, \fB\-\-repeat\-penalty\fR,
\fB\-\-main\-gpu\fR, \fB\-\-ngl\fR, \fB\-\-batch\-size\fR,
\fB\-\-ubatch\-size\fR, \fB\-\-parallel\fR, \fB\-\-cache\-type\-k\fR,
\fB\-\-cache\-type\-v\fR, \fB\-\-reasoning {on,off}\fR.
.PP
Boolean pairs (unset by default, meaning "use config/default value"):
\fB\-\-jinja\fR/\fB\-\-no\-jinja\fR, \fB\-\-fa\fR/\fB\-\-no\-fa\fR,
\fB\-\-kv\-unified\fR/\fB\-\-no\-kv\-unified\fR,
\fB\-\-cont\-batching\fR/\fB\-\-no\-cont\-batching\fR,
\fB\-\-no\-context\-shift\fR/\fB\-\-context\-shift\fR.
.SS Chat request parameters
Applied to the reply produced by \fB\-\-chat\fR and the post-start reply of
\fB\-\-start\fR (they do not affect the container itself). Config keys:
\fBmax_tokens\fR, \fBchat_temperature\fR.
.TP
.B \-\-max\-tokens \fIN\fR
Maximum tokens for the chat reply. Default 2048; raise it for reasoning models
or long-form output (they spend many tokens "thinking" before the visible
answer, so a small budget yields empty content). Must be > 0.
.TP
.B \-\-chat\-temp \fIFLOAT\fR
Temperature for the chat request. If unset, the request omits temperature and
the server's configured \fB\-\-temp\fR applies (single source of truth).
.TP
.B \-\-stream
Stream the \fB\-\-chat\fR reply token-by-token to stdout as it is generated,
instead of waiting for the full response.
.SS Security / behavior flags (prompt input)
.TP
.B \-\-allow\-insecure\-http
Permit \fBhttp://\fR URLs for \fB\-\-system\-url\fR/\fB\-\-prompt\-url\fR (default: HTTPS only).
.TP
.B \-\-allow\-private\-url
Disable private/loopback/link-local/reserved/metadata IP blocking for URL
sources. Cannot be combined with \fB\-\-url\-allow\-host\fR.
.TP
.B \-\-allow\-ip\-host
Permit IP-literal hostnames in URLs (default: rejected).
.TP
.B \-\-follow\-redirects
Follow HTTP redirects for URL sources (default: rejected). Each hop is
re-validated against the same SSRF rules. Bounded by an internal max-redirect
count.
.TP
.B \-\-allow\-html\-input
Accept \fBtext/html\fR responses in addition to the default text-like
content-type allowlist.
.TP
.B \-\-allow\-symlinks\fR / \fB\-\-no\-allow\-symlinks
Allow or reject symlinked files for \fB\-\-system\-file\fR/\fB\-\-prompt\-file\fR.
Default: rejected (opt in with \fB\-\-allow\-symlinks\fR).
.TP
.B \-\-max\-input\-bytes \fIN\fR
Maximum size in bytes for any resolved prompt input (file or URL). Default:
1048576 (1 MiB). Must be > 0.
.TP
.B \-\-url\-allow\-host \fIHOST\fR
Restrict URL sources to an explicit host allowlist. Repeatable.
.TP
.B \-\-connect\-timeout \fISECONDS\fR\fR, \fB\-\-read\-timeout \fISECONDS\fR
Network timeouts for URL sources. Must be > 0. Defaults: 3.0 / 10.0.
.SS Behavior / diagnostics
.TP
.B \-\-timeout \fISECONDS\fR
Readiness timeout for \fB\-\-start\fR/\fB\-\-change\fR. Default from config
(300s).
.TP
.B \-\-poll\-interval \fISECONDS\fR
Readiness poll interval. Default from config (2s).
.TP
.B \-\-lock\-file \fIPATH\fR
Override the lock file path used by \fB\-\-start\fR/\fB\-\-change\fR. Default:
derived from \fBcontainer_name\fR.
.TP
.B \-\-logs
Print container log tail on readiness failure.
.TP
.B \-\-log\-lines \fIN\fR
Number of log lines to print. Default: 100.
.TP
.B \-\-dry\-run
Print the fully constructed \fBdocker run\fR command and exit without
executing it. Usable with \fB\-\-start\fR/\fB\-\-change\fR.
.TP
.B \-\-force
Force stop/change despite inconsistencies.
.TP
.B \-\-print\-effective\-config
Print the fully merged configuration as JSON and exit (unless combined with
an action that also runs).
.SH SECURITY NOTES
All Docker interaction goes through \fBsubprocess.run()\fR with argument
lists, never shell strings — there is no shell-injection surface via
configuration values or CLI arguments.
.PP
The remote URL prompt source (\fB\-\-system\-url\fR/\fB\-\-prompt\-url\fR) is
SSRF-hardened by default: HTTPS-only, no IP-literal hosts, no embedded
credentials, DNS resolution and IP classification for every candidate
address (blocking private/loopback/link-local/multicast/reserved/
unspecified ranges and the cloud metadata address
\fI169.254.169.254\fR), no redirects unless explicitly enabled (with
per-hop re-validation), a content-type allowlist, and a hard size limit
enforced both via \fBContent-Length\fR and while streaming the response body.
See \fIdocs/SECURITY_AND_OPERATIONS.md\fR in the source distribution for the
full threat model.
.PP
\fBcontainer_name\fR is mandatory and must be unique per concurrently-running
profile; reusing a name across profiles that run at the same time will cause
one to replace the other's container on the next \fB\-\-start\fR.
.SH FILES
.TP
.I llama.cpp.config
Default configuration file (INI format). See \fIllama.cpp.config.example\fR
in the source distribution.
.TP
.I /tmp/llamacppctl.<container_name>.lock
Default lock file used by \fB\-\-change\fR, derived from \fBcontainer_name\fR
unless \fB\-\-lock\-file\fR is set.
.SH EXIT STATUS
.TP
.B 0
Success.
.TP
.B 1
General error (Docker unavailable, unexpected exception, prompt input error, configuration error).
.TP
.B 2
CLI argument / semantic validation error.
.TP
.B 3
Model path not found on disk.
.TP
.B 4
Docker command failed.
.TP
.B 5
HTTP request failed or server did not become ready in time.
.TP
.B 6
Lock busy (a concurrent \fB\-\-change\fR is already running for the same \fBcontainer_name\fR).
.SH EXAMPLES
.PP
Start the default profile and show the effective configuration first:
.RS
.nf
llamacppctl \-\-print\-effective\-config \-\-config llama.cpp.config \-\-start
llamacppctl \-\-start \-\-config llama.cpp.config
.fi
.RE
.PP
Preview the docker run command without executing it:
.RS
.nf
llamacppctl \-\-start \-\-config llama.cpp.config \-\-dry\-run
.fi
.RE
.PP
Check status including a live chat completion:
.RS
.nf
llamacppctl \-\-check \-\-config llama.cpp.config \-p "ping"
.fi
.RE
.PP
Select a named model profile from a \fB[model.<name>]\fR section:
.RS
.nf
llamacppctl \-\-start \-\-config llama.cpp.config \-\-profile qwopus
.fi
.RE
.PP
Switch models under lock protection:
.RS
.nf
llamacppctl \-\-change \-\-config llama.cpp.config \-\-profile carnice
.fi
.RE
.SH AUTHOR
Dieter Schlueter
.SH SEE ALSO
.BR docker (1)