classify_refusal() separates refusal (head), preamble, appended disclaimer
(tail) and moralising insert (inline). Only the first three are meaningful, and
only where a prompt forbids them: a preamble on a coding answer is normal, so it
no longer counts as evasion.
Two sources of false positives had to be removed first, both found by checking
the detector's hits against the archive rather than trusting them:
* llamacppctl's own truncation warning on stderr had leaked into an archived
output and was read as a model disclaimer. It also inflated that run's word
count, so count_words() strips it too.
* An AI character saying "Bitte beachten Sie:" inside a dystopian story is
plot, not distancing. Quoted speech is removed before markers are matched.
run_prompt_suite.sh gains a CASES filter so a single prompt can be re-run.
Measured result, recorded in KI_TOOLS_PROFILES.md: across seven prose prompts
neither the abliterated model nor the aligned base model evaded once. On
literary prose the abliteration buys nothing measurable.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The four existing prose prompts never made the aligned base model refuse, so
the abliteration comparison had no difference to measure. These three pick
subjects where an aligned model plausibly balks — an executioner's workday told
without remorse, an unreliable narrator who never concedes his fraud, a relapse
rendered without a warning — while keeping the literary demands of the domain.
Each forbids preamble and closing framing explicitly. That turns an evasion into
a measurable instruction violation rather than a matter of taste. None asks for
operative content: prosa_08 rules out substances, quantities and procurement.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
KI_TOOLS_PROFILES.md records what each local GGUF is actually for, based on
the model cards: ornith is a purpose-built agentic coding model, Carnice targets
agent runtimes, Qwopus is reasoning plus vision, and the two HauhauCS models are
abliterated — an axis about refusals, not literary quality. It also maps which
models the local mmproj.gguf fits (the Qwen3.6-35B-A3B based ones, not ornith).
Capability and benchmark claims are attributed to their authors, not asserted.
EVAL_RUBRIC.md separates what a script can measure from what needs reading, and
warns that a passing test suite only proves the code satisfies its own tests.
CHANGELOG covers the mmproj feature, the --print-effective-config fix, and the
new tooling.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
eval_prompt_tests.py measures the objective half of docs/EVAL_RUBRIC.md over
the manual test archive: word count against the target stated in each prompt,
truncation suspicion, and — for the coding domain — it writes the generated
module and tests to a temp dir and actually runs pytest against them.
Deriving the module's filename is the delicate part: a name taken from a test's
`import sqlite3` would shadow the stdlib and fail the run for a reason the model
is not responsible for. Names now come from the last *.py mention before the
block, then from `from X import`, and anything in sys.stdlib_module_names is
rejected. A module that no test imports is reported as such, since that is a
finding about test quality rather than a guess the runner got wrong.
run_prompt_suite.sh drives one prompt domain against a running profile and
stores the outputs under the archive's naming convention. Both scripts join the
ruff gate.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
argparse requires exactly one action, so the documented diagnostic form
`--print-effective-config --config … --start` never took the early-return
branch in main.run(): it printed the resolved config and then executed a
real do_start(), silently replacing a running container with the [default]
model. README, installation guide and manual all recommended that form.
Make it an action in the mutually exclusive group. It can no longer be
combined with --start/--check/--stop/--change/--chat (argparse error,
exit 2), and it skips the docker_available() check, so it now really is
the offline config check the docs promise. --dry-run remains a modifier
and still requires a reachable daemon.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Vision-capable GGUFs need a separate projector (mmproj) that maps image
embeddings into the text model's space. Add `mmproj` and `mmproj_offload`
as config keys and CLI overrides, and pass them through to llama-server.
The projector path resolves under hf_home exactly like model_path, so it
is covered by the existing read-only mount. validate_model_path() now also
checks the projector, which means --change rejects a missing one *before*
it removes the running container.
--no-mmproj-offload is suppressed when no projector is configured, since
llama.cpp rejects the flag on its own.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add coding_05_rng_uniform.md (cryptographically strong uniform float in
[0.0, 1.0), 53-bit mantissa, bias-free, with distribution tests) and
document running the coding prompts against the ornith35b profile, which
ships coding-tuned sampling plus config-backed stream + read_timeout.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The chat request previously used a hard-coded 30 s timeout and ignored
--read-timeout entirely (that flag only bounded URL prompt fetching), so
slow reasoning models were cut off mid-generation unless one remembered
to pass --stream (which used a separate hard-coded 600 s).
Resolve `stream`, `read_timeout` and `connect_timeout` from
[default]/[model.<profile>] into PromptConfig and wire the (connect, read)
timeout into both the streaming and non-streaming chat calls. CLI
--stream/--read-timeout/--connect-timeout still override; the two timeout
flags default to None so a config value can win, with the URL-fetch
fallbacks (3 s/10 s) preserved. Default chat read_timeout is now 600 s.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Multi-model evaluation runs surfaced recurring coding defects; tighten the
coding system prompt to target them directly:
- Every emitted test must actually pass against the emitted code (mentally
run each test before output) -- several models shipped tests that fail.
- Concrete benchmark size ceiling: an O(n^2) reference must finish well under
a second (at most a few thousand elements), not tens/hundreds of thousands.
- No invented attributes/properties, with the concrete recurring example that
sqlite3.Connection has no `.closed` attribute (crashed two models).
- A `with` block does not necessarily close a resource: sqlite3's connection
context manager only manages the transaction, not close(); state the actual
semantics instead of asserting auto-close.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two more correctness guards, prompted by verified defects in evaluation runs:
- Test fixtures/inputs must actually satisfy the preconditions the same test
assumes (e.g. a record expected to be "valid" must satisfy the schema the
test uses) -- a generated JSONL test asserted the opposite of what it tested.
- Benchmarks must pick input sizes at which even an intentionally inefficient
O(n^2) reference finishes in seconds, and must not present made-up runtimes
as fact -- a generated benchmark hung for minutes and printed fabricated
numbers.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Based on evaluation runs against the default model:
- prose + speeches: when a length/duration is given, hold it strictly with a
maximum ±5% deviation (the model systematically ran ~20-25% short).
- coding: test and production code must share the same contract (same
exception type for the same failure), and forbid unfounded claims about
language/framework behaviour (e.g. that a context manager closes the
connection) -- both were real defects found by actually running the output.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add example_system_prompts/ (prose, speeches, coding) and matching
example_user_prompts/ (four user prompts per domain, one file each so they
run directly via --prompt-file), plus a README mapping the pairs and the run
command against the default model.
Also refine the three system prompts: separate narrative register from
character speech and add a length default (prose); add a duration→word-count
rule, spoken-language guidance, a no-invented-evidence rule, and a salutation
note (speeches); fix the mangled numbered list and add a language default,
a no-invented-APIs rule, and clearer output ordering (coding).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
build_archive.py had no lint or test coverage, which is how the obsolete
requirements/pyproject mirror check slipped through unnoticed. Close that gap:
- Add tests/test_build_archive.py: a network-free smoke test that runs the
manifest check, dependency parsing, tarball build, and re-verification, and
asserts the moved docs/ files, LICENSE, and package sources are packaged.
- Lint build_archive.py in both the local gate (scripts/check.sh) and the CI
workflow; fix the one issue this surfaced (unused variable py_bin).
- List the new test in the REQUIRED_FILES manifest.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Keep only README, LICENSE, and CHANGELOG as prose at the repo root
(tooling/convention) and move the remaining manuals under docs/ next to
SECURITY_AND_OPERATIONS.md.
- git mv the two files into docs/ (history preserved).
- Update all cross-references: README doc-index links, the internal
SECURITY_AND_OPERATIONS link and the §12 pointer list in
BEDIENUNGSANLEITUNG, and the repo inventory in SECURITY_AND_OPERATIONS §1.
- build_archive.py: point REQUIRED_FILES at docs/ and drop the now-redundant
INCLUDE_FILES entries (the docs/ dir is included wholesale).
- build_archive.py: drop the obsolete requirements.txt<->pyproject dependency
mirror check, which broke once requirements.txt was reduced to `.`
(pyproject is the single source of truth). Verified by building and
re-opening the archive.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Improve orientation across the doc set without duplicating content:
- README: add a "Dokumentation" table mapping each doc (README,
BEDIENUNGSANLEITUNG, INSTALL_FROM_ARCHIVE, SECURITY_AND_OPERATIONS, man
page, CHANGELOG) to who it is for and what it covers.
- SECURITY_AND_OPERATIONS §1: extend the module map with a repo-level
inventory (tests, config template, packaging, build/CI/hook scripts,
docs) so "which file does what" is answered beyond the src/ modules.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Clean up documentation redundancy:
- Delete docs/Archiv_fertig_-_Check_und_Installation.md: an AI hand-off
transcript (first-person, stale test count, malformed markdown) whose
useful content is already in INSTALL_FROM_ARCHIVE.md.
- Delete docs/How_to_use.md after moving its one unique asset -- the
old-script -> new-command migration table -- into the README; drop it
from the build_archive manifest.
- Fix INSTALL_FROM_ARCHIVE.md drift: list LICENSE/CHANGELOG.md in the
archive allowlist and use `python -m pytest`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Forgejo Actions is not enabled on the instance, so run the same checks
locally before pushing:
- scripts/check.sh runs ruff, mypy, and pytest (mirrors the CI workflow).
- .githooks/pre-push invokes it; enable per clone with
`git config core.hooksPath .githooks`. Bypass with `git push --no-verify`.
- Fix the pytest invocation in the CI workflow (and document it): use
`python -m pytest` so the repo root is on sys.path, otherwise the test
modules fail to `import tests.*`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Add CHANGELOG.md (Keep a Changelog format) with a 0.1.0 entry summarizing
the initial release plus the recent healthcheck fix, image pin, and
profile-naming changes.
- Add LICENSE and CHANGELOG.md to the build_archive.py manifest so the
distributable tarball actually contains the license and history.
- Expand the README license section to the full MIT + copyright line.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
requirements.txt / requirements-dev.txt used to duplicate the dependency
pins from pyproject.toml by hand, which can silently drift. Point them at
the package itself (`.` and `-e .[dev]`) so the pins live in exactly one
place; `pip install -r requirements*.txt` keeps working.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Back the shipped py.typed promise with an enforced type check and a linter:
- Add ruff + mypy (+ types-requests) to the dev extras and dev requirements,
with [tool.ruff]/[tool.mypy] config in pyproject.toml (mypy checks the
package, not the tests).
- Add a lint job to the Forgejo workflow running ruff check + mypy.
- Fix the issues this surfaced: type FileLock.fd as TextIO | None, add a
targeted type: ignore for the intentional socket.getaddrinfo monkeypatch,
and drop an unused import.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The :server-cuda tag is a moving target, so a fresh pull could silently
change server behaviour (flags, the baked-in healthcheck, ...). Pin the
default image to the current digest for reproducibility; overriding `image`
in the config or via --image still works. A comment documents how to bump
the pin.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Public-repo hygiene now that the project is hosted on Forgejo:
- LICENSE: add the MIT text that pyproject.toml already declares.
- .forgejo/workflows/ci.yml: run the pytest suite on push/PR against
Python 3.10 and 3.12, so regressions (e.g. the healthcheck port bug)
get caught automatically.
- Untrack llama.cpp.config and gitignore it: it is a machine-specific
runtime config that may later hold an api_key. The tracked template
remains llama.cpp.config.example (copy it to get started).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rename the size-numbered profiles so the digits read as parameter count,
not Qwen version: qwen35 -> qwen35b (llama.cpp.config.example, build_archive
smoke test, SECURITY_AND_OPERATIONS.md) and qwen27 -> qwen27b
(llama.cpp.config).
Drop the misleading `--profile qwen35` from the README/man/install examples:
that profile only exists in the .example file, so pasted commands failed
against the real config. Primary examples now omit --profile (using the
[default] section, which always resolves), with one example plus a note
showing how to select a real [model.<name>] profile.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The ghcr.io llama.cpp image bakes in a HEALTHCHECK that curls port 8080
(the llama.cpp default). When the server runs on a different --port (here
8000), that check always fails and Docker reports the container as
"unhealthy" even though it serves fine. Override the healthcheck in the
docker run command to target the configured container_port/health_endpoint,
with a 300s start-period so large-context model loads don't flap.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the stale example profiles (qwen35->8002, deepseek->8003, pointing at
non-existent paths) with profiles for the actually installed models. They now
override ONLY model_path and inherit host_port=8001, container_name and alias
from [default] -> one model at a time on the standard port; --start/--change
swaps it. Distinct port/container is only needed for concurrent operation.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
llama.cpp sends the streaming response as text/event-stream WITHOUT a charset;
requests then does not decode as UTF-8, so iter_lines(decode_unicode=True)
mangled multibyte characters (German Umlaute) into double-encoded garbage
(e.g. "schön" -> "schön"). The non-streaming path via resp.json() was fine.
Set resp.encoding = "utf-8" before iter_lines. Verified live: streamed bytes for
"schön" are now c3 b6 (correct UTF-8), file detected as UTF-8.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Vollständige deutschsprachige Bedienungsanleitung (Installation, Konfiguration,
alle fünf Aktionen, Prompt-Quellen, Antwort-/Reasoning-Steuerung, Netzwerk/
Sicherheit, GPU/Kontext, Exit-Codes, Fehlersuche, Tests) sowie eine Anleitung
zur Installation aus dem .tar.gz-Archiv.
build_archive.py: beide Anleitungen in INCLUDE_FILES + REQUIRED_FILES (werden
gepackt und verifiziert).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Die docs/ waren gegenüber den neuen Features veraltet.
- SECURITY_AND_OPERATIONS.md: Netzwerk-Exposition (Loopback-Default,
--expose/--api-key inkl. 401-Verhalten der Endpunkte), DNS-Pinning,
--start/--change unter Lock + --force, --change validate-before-remove,
Chat-Parameter (max_tokens/chat_temperature/--stream), ${ENV}-Expansion,
--check-Exit-Codes, smoke.sh, ChatReply; Fehler-Tabelle erweitert
(401, Trunkierung, Docker-health vs. HTTP-OK).
- How_to_use.md: Abschnitt zu den neuen Optionen.
- Archiv-Report: Testzahl (130) und Dateiliste aktualisiert.
- build_archive.py: scripts/ in INCLUDE_DIRS (sonst fehlt smoke.sh im Archiv),
test_actions.py/How_to_use.md/smoke.sh in REQUIRED_FILES aufgenommen.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
llama.cpp exempts /health and /v1/models from --api-key; only
/v1/chat/completions is protected. The smoke test probed /v1/models and thus
saw 200 without a key. Probe /v1/chat/completions instead (no key -> 401,
key -> 200). Verified live end-to-end: 9 PASS, 0 FAIL.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sieben Verbesserungen; die Dateien überschneiden sich thematisch, daher ein
Commit (jeder Commit bleibt grün: 115 Tests).
- #1 --check ist scriptbar: Exit 0 wenn Container läuft und erreichbar,
sonst 5 (check_exit_code / CheckResult).
- #2 --force implementiert: Bypass eines belegten Locks mit Warnung
(_container_lock) und stop_container(force=…) schluckt Inkonsistenzen.
- #3 stille Trunkierung behoben: chat_completion_text liefert ChatReply
(content + finish_reason); bei finish_reason=length Hinweis auf stderr,
--chat gibt Exit 1 bei leerem Content zurück.
- #4 keine vermeidbare Downtime: --change validiert den Modellpfad VOR dem
Entfernen des laufenden Containers.
- #5 Netzwerk dicht: Port-Publish standardmäßig nur auf 127.0.0.1
(--expose/expose für alle Interfaces), optionaler --api-key/api_key
(Server --api-key + Bearer-Token auf allen Requests).
- #6 --stream: Chat-Reply token-weise via SSE auf stdout (stream_chat).
- #7 tests/test_actions.py: Orchestrierungs-Ebene (dry-run-Nebenwirkungen,
Lock, validate-before-remove, chat/stream/exit-codes).
Doku aktualisiert (Manpage, README, llama.cpp.config.example).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Steuert einen llama.cpp-Server als Docker-Container: --start/--check/--stop/
--change/--chat, INI-Konfiguration (builtin defaults -> [default] ->
[model.<profile>] -> CLI), SSRF-gehärtete Prompt-Eingabe (Datei/HTTPS-URL),
File-Locking für --start/--change und ein OpenAI-kompatibler HTTP-Layer.
Enthält u. a.:
- Env-Var-Expansion in hf_home (hf_home = ${HF_HOME})
- konfigurierbares Chat-Antwortbudget (max_tokens/chat_temperature,
CLI: --max-tokens/--chat-temp); temperature defer an Server-Default
- DNS-Pinning gegen DNS-Rebinding bei URL-Quellen
- dry-run als nebenwirkungsfreie Vorschau (kein Lock/Removal/Modell-Check)
- 98 Tests (pytest)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
