Vision-capable GGUFs need a separate projector (mmproj) that maps image
embeddings into the text model's space. Add `mmproj` and `mmproj_offload`
as config keys and CLI overrides, and pass them through to llama-server.
The projector path resolves under hf_home exactly like model_path, so it
is covered by the existing read-only mount. validate_model_path() now also
checks the projector, which means --change rejects a missing one *before*
it removes the running container.
--no-mmproj-offload is suppressed when no projector is configured, since
llama.cpp rejects the flag on its own.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The chat request previously used a hard-coded 30 s timeout and ignored
--read-timeout entirely (that flag only bounded URL prompt fetching), so
slow reasoning models were cut off mid-generation unless one remembered
to pass --stream (which used a separate hard-coded 600 s).
Resolve `stream`, `read_timeout` and `connect_timeout` from
[default]/[model.<profile>] into PromptConfig and wire the (connect, read)
timeout into both the streaming and non-streaming chat calls. CLI
--stream/--read-timeout/--connect-timeout still override; the two timeout
flags default to None so a config value can win, with the URL-fetch
fallbacks (3 s/10 s) preserved. Default chat read_timeout is now 600 s.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
build_archive.py had no lint or test coverage, which is how the obsolete
requirements/pyproject mirror check slipped through unnoticed. Close that gap:
- Add tests/test_build_archive.py: a network-free smoke test that runs the
manifest check, dependency parsing, tarball build, and re-verification, and
asserts the moved docs/ files, LICENSE, and package sources are packaged.
- Lint build_archive.py in both the local gate (scripts/check.sh) and the CI
workflow; fix the one issue this surfaced (unused variable py_bin).
- List the new test in the REQUIRED_FILES manifest.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Back the shipped py.typed promise with an enforced type check and a linter:
- Add ruff + mypy (+ types-requests) to the dev extras and dev requirements,
with [tool.ruff]/[tool.mypy] config in pyproject.toml (mypy checks the
package, not the tests).
- Add a lint job to the Forgejo workflow running ruff check + mypy.
- Fix the issues this surfaced: type FileLock.fd as TextIO | None, add a
targeted type: ignore for the intentional socket.getaddrinfo monkeypatch,
and drop an unused import.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The ghcr.io llama.cpp image bakes in a HEALTHCHECK that curls port 8080
(the llama.cpp default). When the server runs on a different --port (here
8000), that check always fails and Docker reports the container as
"unhealthy" even though it serves fine. Override the healthcheck in the
docker run command to target the configured container_port/health_endpoint,
with a 300s start-period so large-context model loads don't flap.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
llama.cpp sends the streaming response as text/event-stream WITHOUT a charset;
requests then does not decode as UTF-8, so iter_lines(decode_unicode=True)
mangled multibyte characters (German Umlaute) into double-encoded garbage
(e.g. "schön" -> "schön"). The non-streaming path via resp.json() was fine.
Set resp.encoding = "utf-8" before iter_lines. Verified live: streamed bytes for
"schön" are now c3 b6 (correct UTF-8), file detected as UTF-8.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sieben Verbesserungen; die Dateien überschneiden sich thematisch, daher ein
Commit (jeder Commit bleibt grün: 115 Tests).
- #1 --check ist scriptbar: Exit 0 wenn Container läuft und erreichbar,
sonst 5 (check_exit_code / CheckResult).
- #2 --force implementiert: Bypass eines belegten Locks mit Warnung
(_container_lock) und stop_container(force=…) schluckt Inkonsistenzen.
- #3 stille Trunkierung behoben: chat_completion_text liefert ChatReply
(content + finish_reason); bei finish_reason=length Hinweis auf stderr,
--chat gibt Exit 1 bei leerem Content zurück.
- #4 keine vermeidbare Downtime: --change validiert den Modellpfad VOR dem
Entfernen des laufenden Containers.
- #5 Netzwerk dicht: Port-Publish standardmäßig nur auf 127.0.0.1
(--expose/expose für alle Interfaces), optionaler --api-key/api_key
(Server --api-key + Bearer-Token auf allen Requests).
- #6 --stream: Chat-Reply token-weise via SSE auf stdout (stream_chat).
- #7 tests/test_actions.py: Orchestrierungs-Ebene (dry-run-Nebenwirkungen,
Lock, validate-before-remove, chat/stream/exit-codes).
Doku aktualisiert (Manpage, README, llama.cpp.config.example).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Steuert einen llama.cpp-Server als Docker-Container: --start/--check/--stop/
--change/--chat, INI-Konfiguration (builtin defaults -> [default] ->
[model.<profile>] -> CLI), SSRF-gehärtete Prompt-Eingabe (Datei/HTTPS-URL),
File-Locking für --start/--change und ein OpenAI-kompatibler HTTP-Layer.
Enthält u. a.:
- Env-Var-Expansion in hf_home (hf_home = ${HF_HOME})
- konfigurierbares Chat-Antwortbudget (max_tokens/chat_temperature,
CLI: --max-tokens/--chat-temp); temperature defer an Server-Default
- DNS-Pinning gegen DNS-Rebinding bei URL-Quellen
- dry-run als nebenwirkungsfreie Vorschau (kein Lock/Removal/Modell-Check)
- 98 Tests (pytest)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
