feat(llm): lokales llama.cpp-Modell (va_llm) als zentrale, sprachoptimierte KI
- scripts/llm-server/: start/stop/status fuer llama.cpp-Server (Port 8001, GPU 1, Modell Qwen3.6-35B-A3B-Uncensored, Alias va_llm) - alles per ENV ueberschreibbar - Defaults auf den lokalen Server umgestellt (config.py, .example-Configs, .env.example) - Provider local-openai-compatible sprachoptimiert: Reasoning aus (chat_template_kwargs.enable_thinking=false) + knapper Sprach-System-Prompt, optional max_tokens/temperature - Antwort ~9x schneller, kurze vorlesbare Texte - Makefile-Targets llm-up/llm-down/llm-status - Doku (README, BEDIENUNGSANLEITUNG) auf llama.cpp statt Ollama aktualisiert Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
cd7aef852d
commit
28c729f1d4
11 changed files with 323 additions and 31 deletions
|
|
@ -55,7 +55,13 @@ STT_REGISTRY = {
|
|||
LLM_REGISTRY = {
|
||||
"openrouter": lambda s: OpenRouterLLMProvider(s.openrouter_api_key, s.openrouter_llm_model),
|
||||
"local-openai-compatible": lambda s: LocalOpenAICompatibleLLM(
|
||||
s.local_llm_base_url, s.local_llm_api_key, s.local_llm_model
|
||||
s.local_llm_base_url,
|
||||
s.local_llm_api_key,
|
||||
s.local_llm_model,
|
||||
system_prompt=s.local_llm_system_prompt,
|
||||
disable_reasoning=s.local_llm_disable_reasoning,
|
||||
max_tokens=s.local_llm_max_tokens,
|
||||
temperature=s.local_llm_temperature,
|
||||
),
|
||||
}
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue