dschlueter/my_voice_assistant_v2
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

feat(llm): lokales llama.cpp-Modell (va_llm) als zentrale, sprachoptimierte KI

Browse source 
- scripts/llm-server/: start/stop/status fuer llama.cpp-Server (Port 8001, GPU 1,
  Modell Qwen3.6-35B-A3B-Uncensored, Alias va_llm) - alles per ENV ueberschreibbar
- Defaults auf den lokalen Server umgestellt (config.py, .example-Configs, .env.example)
- Provider local-openai-compatible sprachoptimiert: Reasoning aus
  (chat_template_kwargs.enable_thinking=false) + knapper Sprach-System-Prompt,
  optional max_tokens/temperature - Antwort ~9x schneller, kurze vorlesbare Texte
- Makefile-Targets llm-up/llm-down/llm-status
- Doku (README, BEDIENUNGSANLEITUNG) auf llama.cpp statt Ollama aktualisiert

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Dieter Schlüter 2026-06-18 02:57:57 +02:00
commit 28c729f1d4
11 changed files with 323 additions and 31 deletions
Download patch file Download diff file Expand all files Collapse all files

14
app/config.py
View file

@ -117,9 +117,19 @@ class Settings(BaseSettings):
default_stt_provider: str = "openrouter"
default_llm_provider: str = "local-openai-compatible"
default_tts_provider: str = "openrouter"
local_llm_base_url: str = "http://127.0.0.1:11434/v1"
# Lokaler llama.cpp-Server (OpenAI-kompatibel), siehe scripts/llm-server/.
local_llm_base_url: str = "http://127.0.0.1:8001/v1"
local_llm_api_key: str = "dummy"
local_llm_model: str = "llama3.1"
local_llm_model: str = "va_llm" # = --alias des llama.cpp-Servers
# Sprach-Assistent: knappe, vorlesbare Antworten + Reasoning aus = deutlich schneller.
local_llm_system_prompt: str = (
"Du bist ein gesprochener Sprachassistent. Antworte kurz und natuerlich "
"(in der Regel 1-3 Saetze), in reinem Fliesstext ohne Markdown, ohne "
"Aufzaehlungen, ohne Emojis. Formuliere so, wie man es laut vorliest."
)
local_llm_disable_reasoning: bool = True # Qwen3 /no_think: spart die Denkphase
local_llm_max_tokens: int = 0 # 0 = serverseitiges Limit (-n)
local_llm_temperature: float = 0.3
faster_whisper_model: str = "base" # tiny|base|small|medium|large-v3
faster_whisper_device: str = "auto" # auto|cpu|cuda
faster_whisper_compute_type: str = "default" # default|int8|float16|int8_float16