feat: Geräte-TTS, native Stimmen, Favicon, Auth-Gate, UI-Fixes
Web-UI / TTS:
- Geräte-TTS ("📱 Gerät"): Antwort wird on-device vorgelesen (Web Speech
API), Server sendet nur Text (text_only) -> spart Bandbreite/Kosten.
Mobil-Default, geräte-lokale Speicherung, iOS-Autoplay-Freischaltung.
- Vorlese-Symbol (🔊) je Bubble: Hybrid-Replay (Assistent-PCM gecacht,
Eingabe via /api/speak); SVG-Icon mit kontrastreicher Farbe.
- Kombiniertes Sprachmenü (Flex + feste Sprachen) statt separatem Modus-Menü.
- "Neues Gespräch"-Button (frische Session gegen Sprach-Trägheit).
- Dark-Mode: lesbare <option>-Popups (Kontrast-Fix).
- Favicon (SVG + PNG-Fallbacks) aus mund.png.
TTS-Backend:
- Sprache wird an alle TTS-Provider durchgereicht; Piper-Stimme folgt der
Sprache; Chatterbox mehrsprachig + cross-lingual.
- Native Referenz-Stimmen je Sprache (config/voices/<lang>.wav, FLEURS CC-BY),
loudness-normalisiert.
LLM-Sprache:
- Antwort folgt zuverlässig der gewählten Sprache (verstärkte Anweisung +
Erinnerung an der letzten Nutzer-Nachricht gegen History-Trägheit).
Admin / Auth:
- Wörterbuch: alle 8 Sprachen, Zeilen editierbar, alphabetische Sortierung.
- Web-UI hinter Auth-Gate (Redirect auf SSO_LOGIN_URL / 401); Favicons offen.
- Log-Tab: Hinweis, wenn der systemd-Dienst nicht aktiv ist.
- Einstellungen: Hinweis "pro Nutzer überschreibbar" bei Sprache/Modus/Qualität.
Doku (BEDIENUNGSANLEITUNG.md): Geräte-TTS §6.5.0, Fix/Flex §6.6, native
Stimmen §6.5.3, llama.cpp<->Ollama-Wechsel §4.7, Auth/SSO §7.4.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
dc8ac80505
commit
878bf785dd
51 changed files with 985 additions and 202 deletions
7
.gitignore
vendored
7
.gitignore
vendored
|
|
@ -9,6 +9,8 @@ data/
|
||||||
|
|
||||||
# Generierte Audio-Ausgaben (z. B. chat_client.py)
|
# Generierte Audio-Ausgaben (z. B. chat_client.py)
|
||||||
*.wav
|
*.wav
|
||||||
|
# Ausnahme: native Referenz-Stimmen je Sprache (versioniert, Feature-Asset)
|
||||||
|
!config/voices/*.wav
|
||||||
|
|
||||||
# Python
|
# Python
|
||||||
__pycache__/
|
__pycache__/
|
||||||
|
|
@ -25,3 +27,8 @@ venv/
|
||||||
*.bak
|
*.bak
|
||||||
*.patch
|
*.patch
|
||||||
*.orig
|
*.orig
|
||||||
|
|
||||||
|
# Dateien
|
||||||
|
3
|
||||||
|
TODO.md
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -570,45 +570,71 @@ VA_PROFILE=local-dev make run # alles lokal (STT/TTS in-process, LLM via Ollam
|
||||||
|
|
||||||
### 4.7 Zwischen llama.cpp und Ollama wechseln
|
### 4.7 Zwischen llama.cpp und Ollama wechseln
|
||||||
|
|
||||||
Beide Server können nicht gleichzeitig auf demselben GPU-Speicher laufen. Vor dem
|
Beide nutzen denselben Gateway-Provider `local-openai-compatible` (OpenAI-kompatible
|
||||||
Wechsel muss der jeweils andere Backend-Prozess beendet werden.
|
API). Der Wechsel besteht daher aus drei Dingen: **(A)** den jeweils anderen
|
||||||
|
Backend-Prozess stoppen (beide teilen sich den GPU-Speicher), **(B)** das gewünschte
|
||||||
|
Backend starten, **(C)** die drei `LOCAL_LLM_*`-Zeilen in `.env` umstellen und das
|
||||||
|
Gateway neu starten. `DEFAULT_LLM_PROVIDER` bleibt unverändert.
|
||||||
|
|
||||||
**Merkhilfe:**
|
**Merkhilfe:**
|
||||||
- llama.cpp = Docker-Container `va_llm` → stoppen mit `make llm-down`
|
- llama.cpp = Docker-Container `va_llm` → `make llm-up` / `make llm-down` (Port 8001)
|
||||||
- Ollama = systemd-Dienst → stoppen mit `sudo systemctl stop ollama`
|
- Ollama = systemd-Dienst → `sudo systemctl start/stop ollama` (Port 11434)
|
||||||
|
|
||||||
#### Von Ollama → llama.cpp wechseln
|
> **Wichtig:** `.env`-Änderungen werden erst bei einem **Gateway-Neustart** wirksam
|
||||||
|
> (uvicorn `--reload` lädt nur bei Code-Änderungen neu, nicht bei `.env`). Als Dienst:
|
||||||
```bash
|
> `systemctl --user restart voice-assistant.service`.
|
||||||
# 1) Ollama stoppen
|
|
||||||
sudo systemctl stop ollama
|
|
||||||
# falls manuell gestartet (ollama serve im Vordergrund):
|
|
||||||
pkill -f "ollama serve" 2>/dev/null || true
|
|
||||||
|
|
||||||
# 2) llama.cpp starten und warten
|
|
||||||
make llm-up
|
|
||||||
make llm-status # warten bis „Modell bereit" + HTTP OK erscheint
|
|
||||||
|
|
||||||
# 3) Gateway starten
|
|
||||||
VA_PROFILE=hybrid make run
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Von llama.cpp → Ollama wechseln
|
#### Von llama.cpp → Ollama wechseln
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# 1) llama.cpp stoppen
|
# 1) llama.cpp stoppen (GPU freigeben)
|
||||||
make llm-down
|
make llm-down # alternativ: docker rm -f va_llm
|
||||||
# alternativ direkt:
|
|
||||||
docker rm -f va_llm
|
|
||||||
|
|
||||||
# 2) Ollama starten
|
# 2) Ollama starten und Modell sicherstellen
|
||||||
sudo systemctl start ollama
|
sudo systemctl start ollama
|
||||||
ollama ps # prüfen ob Modell aktiv (oder leer — wird beim ersten Request geladen)
|
ollama list # exakten Modellnamen ablesen (z. B. gemma4:12b)
|
||||||
|
ollama pull gemma4:12b # nur falls noch nicht vorhanden
|
||||||
|
|
||||||
# 3) Gateway starten
|
# 3) .env umstellen:
|
||||||
VA_PROFILE=hybrid make run
|
# LOCAL_LLM_BASE_URL=http://127.0.0.1:11434/v1
|
||||||
|
# LOCAL_LLM_API_KEY=ollama
|
||||||
|
# LOCAL_LLM_MODEL=gemma4:12b # exakter Name aus 'ollama list'
|
||||||
|
|
||||||
|
# 4) Gateway neu starten
|
||||||
|
VA_PROFILE=hybrid make run # oder: systemctl --user restart voice-assistant.service
|
||||||
```
|
```
|
||||||
|
|
||||||
|
#### Von Ollama → llama.cpp wechseln
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1) Ollama stoppen (GPU freigeben)
|
||||||
|
sudo systemctl stop ollama
|
||||||
|
pkill -f "ollama serve" 2>/dev/null || true # falls manuell im Vordergrund gestartet
|
||||||
|
|
||||||
|
# 2) llama.cpp starten und warten
|
||||||
|
make llm-up
|
||||||
|
make llm-status # warten bis „Modell bereit" + HTTP OK
|
||||||
|
|
||||||
|
# 3) .env umstellen:
|
||||||
|
# LOCAL_LLM_BASE_URL=http://127.0.0.1:8001/v1
|
||||||
|
# LOCAL_LLM_API_KEY=dummy
|
||||||
|
# LOCAL_LLM_MODEL=va_llm
|
||||||
|
|
||||||
|
# 4) Gateway neu starten
|
||||||
|
VA_PROFILE=hybrid make run # oder: systemctl --user restart voice-assistant.service
|
||||||
|
```
|
||||||
|
|
||||||
|
**Prüfen** (egal welche Richtung):
|
||||||
|
```bash
|
||||||
|
curl http://127.0.0.1:11434/v1/models # Ollama (bzw. :8001 für llama.cpp)
|
||||||
|
# danach im Admin → Status den LLM-Provider/das Modell kontrollieren oder kurz testen
|
||||||
|
```
|
||||||
|
|
||||||
|
> **Reasoning/Latenz:** Das Gateway sendet `enable_thinking:false`; **Ollama ignoriert**
|
||||||
|
> das. Modelle mit eingebautem „Thinking" (z. B. `gemma4:12b`) liefern die Antwort sauber
|
||||||
|
> im `content`, denken aber intern mit → höhere Latenz. Für reinen Smalltalk ggf. ein
|
||||||
|
> kleineres/nicht-reasonendes Modell wählen.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### 4.8 Alles stoppen
|
### 4.8 Alles stoppen
|
||||||
|
|
@ -692,7 +718,7 @@ https://va.beispiel.de/ ← remote über Reverse-Proxy (alles, inkl. Mikr
|
||||||
|
|
||||||
```
|
```
|
||||||
┌─────────────────────────────────────────────────────────┐
|
┌─────────────────────────────────────────────────────────┐
|
||||||
│ Voice Assistant [☀️/🌙] Angemeldet als … │
|
│ Voice Assistant [🇩🇪 ▾] [Qualität ▾] [⚙️] [🌙] │
|
||||||
├─────────────────────────────────────────────────────────┤
|
├─────────────────────────────────────────────────────────┤
|
||||||
│ │
|
│ │
|
||||||
│ Nachrichtenverlauf │
|
│ Nachrichtenverlauf │
|
||||||
|
|
@ -700,7 +726,7 @@ https://va.beispiel.de/ ← remote über Reverse-Proxy (alles, inkl. Mikr
|
||||||
│ (Assistent: graue Blase links) │
|
│ (Assistent: graue Blase links) │
|
||||||
│ │
|
│ │
|
||||||
├─────────────────────────────┬───────────────────────────┤
|
├─────────────────────────────┬───────────────────────────┤
|
||||||
│ Texteingabe … [Senden] │ [🎤] [Stimme ▾] │
|
│ Texteingabe … [Senden] │ [🎤] │
|
||||||
└─────────────────────────────┴───────────────────────────┘
|
└─────────────────────────────┴───────────────────────────┘
|
||||||
Statuszeile: „denkt …" / „verarbeite Sprache …" / leer
|
Statuszeile: „denkt …" / „verarbeite Sprache …" / leer
|
||||||
```
|
```
|
||||||
|
|
@ -709,7 +735,8 @@ https://va.beispiel.de/ ← remote über Reverse-Proxy (alles, inkl. Mikr
|
||||||
|---------|----------|
|
|---------|----------|
|
||||||
| **Texteingabe + Senden** | Text tippen, dann Enter oder „Senden" |
|
| **Texteingabe + Senden** | Text tippen, dann Enter oder „Senden" |
|
||||||
| **🎤 Mikrofon-Button** | **Idle (grün 🎤):** Tippen → Aufnahme startet · **Aufnahme (rot pulsierend 🎤):** Tippen → Aufnahme stoppt und wird gesendet · **KI antwortet (amber ⏹):** Tippen → Antwort sofort unterbrechen (Barge-in) |
|
| **🎤 Mikrofon-Button** | **Idle (grün 🎤):** Tippen → Aufnahme startet · **Aufnahme (rot pulsierend 🎤):** Tippen → Aufnahme stoppt und wird gesendet · **KI antwortet (amber ⏹):** Tippen → Antwort sofort unterbrechen (Barge-in) |
|
||||||
| **Stimme ▾** | TTS-Provider wählen: leer = Server-Default, `chatterbox` = neuronale Stimme, `openrouter` = Cloud-TTS |
|
| **Sprache ▾** | Antwortsprache wählen (→ § 6.6): `🔄 Flex` = folgt automatisch der gesprochenen Sprache · feste Sprache (🇩🇪/🇬🇧/…) = Eingabe wird in diese Sprache übersetzt. Die vorlesende Stimme folgt der Auswahl. |
|
||||||
|
| **Qualität ▾** | Vorlese-Quelle wählen: `📱 Gerät` = das Handy/der Browser liest selbst vor (Web Speech API, kein Server-Audio → spart Daten/Kosten) · `Schnell` = piper (lokal) · `Hoch` = chatterbox (neuronal) · `Cloud` = openrouter |
|
||||||
| **☀️ / 🌙** | Tag-/Nacht-Modus; folgt sonst automatisch dem Betriebssystem |
|
| **☀️ / 🌙** | Tag-/Nacht-Modus; folgt sonst automatisch dem Betriebssystem |
|
||||||
| **⚙️** (Admin) | Öffnet das Admin-Panel — nur für Admin-Nutzer sichtbar (→ § 7.5) |
|
| **⚙️** (Admin) | Öffnet das Admin-Panel — nur für Admin-Nutzer sichtbar (→ § 7.5) |
|
||||||
| **Angemeldet als …** | SSO-Identität; „Gast" wenn AUTH deaktiviert oder kein SSO-Cookie |
|
| **Angemeldet als …** | SSO-Identität; „Gast" wenn AUTH deaktiviert oder kein SSO-Cookie |
|
||||||
|
|
@ -957,6 +984,27 @@ LOCAL_LLM_SYSTEM_PROMPT=
|
||||||
|
|
||||||
### 6.5 TTS-Einstellungen (Sprachsynthese)
|
### 6.5 TTS-Einstellungen (Sprachsynthese)
|
||||||
|
|
||||||
|
Die Vorlese-Quelle wählt der Nutzer im Kopfzeilen-Menü **Qualität** (→ § 5.1.2). Es gibt
|
||||||
|
vier Optionen: das **Gerät** selbst (§ 6.5.0) oder einen der drei Server-Provider
|
||||||
|
**piper** (§ 6.5.2), **chatterbox** (§ 6.5.3) bzw. **OpenRouter** (§ 6.5.1).
|
||||||
|
|
||||||
|
#### 6.5.0 Geräte-TTS (Web Speech API) — „📱 Gerät"
|
||||||
|
|
||||||
|
Wählt der Nutzer **„📱 Gerät"**, liest das Endgerät die Antwort **selbst** vor (iPhone:
|
||||||
|
Safari/Siri-Stimmen, Android: System-TTS). Der Server erzeugt und überträgt dann **kein
|
||||||
|
Audio** — er schickt nur den Text. Das spart Mobilfunk-Daten und TTS-Rechenzeit/Kosten.
|
||||||
|
|
||||||
|
- **Vorteile:** keine Audio-Bytes, geringere Latenz, funktioniert auch ohne laufenden
|
||||||
|
TTS-Dienst.
|
||||||
|
- **Grenzen:** Stimme/Qualität hängen vom Gerät ab; deine eigenen Stimmen (Klon,
|
||||||
|
FLEURS-Referenzen) und der Aussprache-Normalizer/das Wörterbuch (§ 6.5.4) greifen **nicht**.
|
||||||
|
- **Verhalten:** Auf Mobilgeräten ist „Gerät" voreingestellt (solange nichts gewählt wurde).
|
||||||
|
Die Wahl ist **geräte-lokal** gespeichert (kein Server-Pref), da On-Device-Stimmen pro
|
||||||
|
Gerät verschieden sind. Browser ohne Web Speech API blenden die Option aus.
|
||||||
|
- **Technik:** Das Frontend sendet `text_only:true`; die Antwortsprache je Bubble steuert
|
||||||
|
`SpeechSynthesisUtterance.lang`. iOS erlaubt Sprachausgabe erst nach einer Nutzergeste —
|
||||||
|
das Frontend schaltet sie beim ersten Tippen/Senden frei.
|
||||||
|
|
||||||
#### 6.5.1 Cloud-TTS (OpenRouter) — Stimmen wählen
|
#### 6.5.1 Cloud-TTS (OpenRouter) — Stimmen wählen
|
||||||
|
|
||||||
Das Gateway reicht den Stimmennamen unverändert an OpenRouter weiter. Welche
|
Das Gateway reicht den Stimmennamen unverändert an OpenRouter weiter. Welche
|
||||||
|
|
@ -1050,10 +1098,22 @@ Konfiguration in `.env`:
|
||||||
```bash
|
```bash
|
||||||
CHATTERBOX_BASE_URL=http://127.0.0.1:9999
|
CHATTERBOX_BASE_URL=http://127.0.0.1:9999
|
||||||
CHATTERBOX_VOICE=/pfad/zu/referenz_stimme.wav # leer = Standardstimme
|
CHATTERBOX_VOICE=/pfad/zu/referenz_stimme.wav # leer = Standardstimme
|
||||||
CHATTERBOX_LANG=de
|
CHATTERBOX_LANG=de # Fallback-Sprache (Gesprächssprache gewinnt)
|
||||||
CHATTERBOX_SPEED=1.0
|
CHATTERBOX_SPEED=1.0
|
||||||
|
CHATTERBOX_VOICES_DIR=config/voices # native Referenz-Stimmen je Sprache
|
||||||
```
|
```
|
||||||
|
|
||||||
|
**Mehrsprachig + native Stimme je Sprache.** Chatterbox ist mehrsprachig (de, en, fr,
|
||||||
|
es, it, nl, ru, zh u. a.) und klont **cross-lingual**: Die Antwortsprache (→ § 6.6)
|
||||||
|
wird automatisch an den Dienst übergeben, und die passende Referenz-Stimme wird aus
|
||||||
|
`CHATTERBOX_VOICES_DIR` nach Konvention `<lang>.wav` gewählt (z. B. `fr.wav`, `zh.wav`).
|
||||||
|
So spricht jede Sprache mit einer muttersprachlichen Stimme statt deutsch-akzentuiert.
|
||||||
|
|
||||||
|
- Reihenfolge der Stimm-Auswahl: explizit angefragte `voice` (WAV-Pfad) → `config/voices/<lang>.wav` → `CHATTERBOX_VOICE` (persönlicher Klon) → Standardstimme des Dienstes.
|
||||||
|
- Deutsch nutzt bewusst keine Datei in `config/voices/`, sondern `CHATTERBOX_VOICE`.
|
||||||
|
- Die mitgelieferten Referenz-Clips stammen aus dem FLEURS-Datensatz (CC-BY 4.0) —
|
||||||
|
Quelle/Lizenz/Austausch siehe `config/voices/README.md`.
|
||||||
|
|
||||||
#### 6.5.4 Aussprache verbessern (TTS-Normalizer)
|
#### 6.5.4 Aussprache verbessern (TTS-Normalizer)
|
||||||
|
|
||||||
Vor dem TTS läuft ein Normalizer, der Ausspracheprobleme des Phonemizers behebt:
|
Vor dem TTS läuft ein Normalizer, der Ausspracheprobleme des Phonemizers behebt:
|
||||||
|
|
@ -1232,20 +1292,49 @@ print(result) # → 'Chluteur kommt.'
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### 6.6 Sprache wechseln
|
### 6.6 Sprache wechseln (Fix / Flex)
|
||||||
|
|
||||||
|
Die Antwortsprache wird in der Web-Oberfläche über **ein einziges Dropdown** („Sprache")
|
||||||
|
gesteuert. Es kennt zwei Arten von Werten:
|
||||||
|
|
||||||
|
| Auswahl | Bedeutung | Whisper (STT) | LLM-Antwort + Stimme |
|
||||||
|
|---------|-----------|---------------|----------------------|
|
||||||
|
| **🔄 Flex** | keine feste Sprache — folgt automatisch der gesprochenen | erkennt die Sprache, **übersetzt nicht** | in der **erkannten** Sprache (blaue Blase zeigt das Original) |
|
||||||
|
| **🇩🇪 / 🇬🇧 / … (feste Sprache)** | „Fix": System bleibt bei dieser Sprache | bekommt die feste Sprache → **übersetzt** die Eingabe | immer in der **festen** Sprache, egal worin gefragt wurde |
|
||||||
|
|
||||||
|
**Beispiele** (Eingabe auf Französisch gesprochen):
|
||||||
|
|
||||||
|
- **Flex** → blaue Blase: französischer Originaltext · Antwort + Stimme: Französisch.
|
||||||
|
- **🇩🇪 DE** → blaue Blase: deutsche Übersetzung · Antwort + Stimme: Deutsch.
|
||||||
|
|
||||||
|
Die vorlesende Piper-Stimme folgt immer der Antwortsprache automatisch (z. B. `thorsten`
|
||||||
|
für Deutsch, `siwis` für Französisch — Zuordnung → `LANG_TO_PIPER_VOICE`). Bei Text-Chat
|
||||||
|
im Flex-Modus (keine Audio-Erkennung möglich) bekommt das LLM **keine** Sprachvorgabe und
|
||||||
|
antwortet von selbst in der Sprache der Eingabe; als Fallback gilt `DEFAULT_LANGUAGE`.
|
||||||
|
|
||||||
|
> **Hinweis:** Es gibt kein separates „Fix/Flex"-Menü mehr — eine konkrete Sprache zu
|
||||||
|
> wählen *ist* der Fix-Modus, „🔄 Flex" der flexible. Intern bleiben zwei Felder
|
||||||
|
> erhalten: `language` (ISO-Code) und `language_mode` (`fix` | `flex`).
|
||||||
|
|
||||||
|
**Konfiguration außerhalb der Web-UI:**
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
DEFAULT_LANGUAGE=de # global in .env
|
DEFAULT_LANGUAGE=de # global in .env (Fallback-/Standardsprache)
|
||||||
|
DEFAULT_LANGUAGE_MODE=fix # global: fix | flex (Admin: Tab „⚙ Einstellungen")
|
||||||
```
|
```
|
||||||
|
|
||||||
Pro Nutzer:
|
Pro Nutzer (dauerhaft):
|
||||||
```bash
|
```bash
|
||||||
curl -X PUT $URL/api/me/prefs \
|
# Feste Sprache (Fix):
|
||||||
-H "Authorization: Bearer $TOKEN" \
|
curl -X PUT $URL/api/me/prefs -H "Authorization: Bearer $TOKEN" \
|
||||||
-H 'Content-Type: application/json' -d '{"language":"en"}'
|
-H 'Content-Type: application/json' -d '{"language":"en","language_mode":"fix"}'
|
||||||
|
|
||||||
|
# Flex (Sprache folgt automatisch):
|
||||||
|
curl -X PUT $URL/api/me/prefs -H "Authorization: Bearer $TOKEN" \
|
||||||
|
-H 'Content-Type: application/json' -d '{"language_mode":"flex"}'
|
||||||
```
|
```
|
||||||
|
|
||||||
Pro Aufruf: `{"text":"…","language":"en"}` im Body.
|
Pro Aufruf: `{"text":"…","language":"en","language_mode":"fix"}` im Body.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
@ -1564,6 +1653,7 @@ TRUSTED_AUTH_COOKIE_CLAIM=user
|
||||||
TRUSTED_PROXY_IPS=192.168.179.10
|
TRUSTED_PROXY_IPS=192.168.179.10
|
||||||
ADMIN_USERS=atoor,dieterschlueter,dschlueter
|
ADMIN_USERS=atoor,dieterschlueter,dschlueter
|
||||||
SSO_LOGOUT_URL=https://linix.de/yunohost/sso/?action=logout
|
SSO_LOGOUT_URL=https://linix.de/yunohost/sso/?action=logout
|
||||||
|
SSO_LOGIN_URL=https://linix.de/yunohost/sso/ # unauth. Seitenaufrufe -> hierhin umleiten
|
||||||
|
|
||||||
# Optional: Signaturprüfung des JWT-Cookies
|
# Optional: Signaturprüfung des JWT-Cookies
|
||||||
TRUSTED_AUTH_JWT_SECRET=<hs256-secret aus /etc/yunohost/.ssowat_cookie_secret>
|
TRUSTED_AUTH_JWT_SECRET=<hs256-secret aus /etc/yunohost/.ssowat_cookie_secret>
|
||||||
|
|
@ -1574,6 +1664,17 @@ TRUSTED_AUTH_HEADER=X-Remote-User
|
||||||
|
|
||||||
Vollständige Anleitung: [deploy/README.md](deploy/README.md).
|
Vollständige Anleitung: [deploy/README.md](deploy/README.md).
|
||||||
|
|
||||||
|
**Zugriffsschutz der Web-UI (mehrstufig):**
|
||||||
|
1. **YunoHost SSOwat (primär):** Die App-Berechtigung darf die Gruppe „visitors" nicht
|
||||||
|
enthalten → unauthentifizierte Besucher werden zum Portal umgeleitet, bevor sie das
|
||||||
|
Gateway erreichen: `yunohost user permission update <APP>.main --remove visitors --add all_users`.
|
||||||
|
2. **Gateway, API/WS:** Anfragen über den Proxy ohne gültiges `yunohost.portal`-Cookie
|
||||||
|
bekommen 401 (auch bei `AUTH_ENABLED=false` — das betrifft nur den LAN-Direktzugriff).
|
||||||
|
3. **Gateway, statische Seite:** Unauthentifizierte Seitenaufrufe werden auf `SSO_LOGIN_URL`
|
||||||
|
umgeleitet (bzw. 401, falls nicht gesetzt) — Defense-in-Depth, falls SSOwat umgangen wird.
|
||||||
|
4. **Port:** Das Gateway sollte nicht offen im Netz lauschen (`HOST=127.0.0.1` bzw. Firewall
|
||||||
|
auf die Proxy-IP), damit der SSO-Weg nicht per Direktzugriff umgangen werden kann.
|
||||||
|
|
||||||
### 7.5 Admin-Web-Panel
|
### 7.5 Admin-Web-Panel
|
||||||
|
|
||||||
> 🔧 Admin — erreichbar über den **⚙️-Button** im Web-Interface (nur für Admin-Nutzer sichtbar)
|
> 🔧 Admin — erreichbar über den **⚙️-Button** im Web-Interface (nur für Admin-Nutzer sichtbar)
|
||||||
|
|
@ -2073,13 +2174,15 @@ Wichtige Body-Felder für `/api/chat` und `/api/speak`:
|
||||||
| Feld | Typ | Bedeutung |
|
| Feld | Typ | Bedeutung |
|
||||||
|------|-----|-----------|
|
|------|-----|-----------|
|
||||||
| `text` | string | Eingabe-Text (Pflicht) |
|
| `text` | string | Eingabe-Text (Pflicht) |
|
||||||
| `language` | string | Sprache, z. B. `de`, `en` |
|
| `language` | string | Sprache, z. B. `de`, `en` (im Fix-Modus maßgeblich) |
|
||||||
|
| `language_mode` | string | `fix` (feste Sprache, Eingabe wird übersetzt) oder `flex` (folgt der erkannten Sprache) — → § 6.6 |
|
||||||
| `stt_provider` | string | Provider für diese Anfrage |
|
| `stt_provider` | string | Provider für diese Anfrage |
|
||||||
| `llm_provider` | string | Provider für diese Anfrage |
|
| `llm_provider` | string | Provider für diese Anfrage |
|
||||||
| `tts_provider` | string | Provider für diese Anfrage |
|
| `tts_provider` | string | Provider für diese Anfrage |
|
||||||
| `voice` | string | TTS-Stimme für diese Anfrage |
|
| `voice` | string | TTS-Stimme für diese Anfrage |
|
||||||
| `stream` | bool | LLM-Token-Streaming (nur WebSocket) |
|
| `stream` | bool | LLM-Token-Streaming (nur WebSocket) |
|
||||||
| `audio_stream` | bool | Satzweises Audio-Streaming (nur WebSocket) |
|
| `audio_stream` | bool | Satzweises Audio-Streaming (nur WebSocket) |
|
||||||
|
| `text_only` | bool | Kein Server-Audio erzeugen/senden (nur Text) — fürs Geräte-TTS (→ § 6.5.0) |
|
||||||
|
|
||||||
## B.3 Sessions und Routing
|
## B.3 Sessions und Routing
|
||||||
|
|
||||||
|
|
@ -2095,7 +2198,7 @@ Body-Felder: `input_endpoint`, `output_endpoint`, `stt_provider`, `llm_provider`
|
||||||
| Methode | Pfad | Beschreibung |
|
| Methode | Pfad | Beschreibung |
|
||||||
|---------|------|--------------|
|
|---------|------|--------------|
|
||||||
| `GET` | `/api/me` | Aktueller Nutzer + Präferenzen |
|
| `GET` | `/api/me` | Aktueller Nutzer + Präferenzen |
|
||||||
| `PUT` | `/api/me/prefs` | Dauerhafte Routing-Präferenzen setzen |
|
| `PUT` | `/api/me/prefs` | Dauerhafte Routing-Präferenzen setzen (Merge; Felder u. a. `language`, `language_mode`, `tts_provider` — → § 6.6) |
|
||||||
| `GET` | `/api/me/memories` | Alle Langzeit-Erinnerungen |
|
| `GET` | `/api/me/memories` | Alle Langzeit-Erinnerungen |
|
||||||
| `POST` | `/api/me/memories` | Erinnerung hinzufügen |
|
| `POST` | `/api/me/memories` | Erinnerung hinzufügen |
|
||||||
| `DELETE` | `/api/me/memories/{id}` | Erinnerung löschen |
|
| `DELETE` | `/api/me/memories/{id}` | Erinnerung löschen |
|
||||||
|
|
@ -2190,7 +2293,9 @@ in `app/dependencies.py` + Implementierung in `app/providers/`. → [Architektur
|
||||||
| Fallback-Ketten | § 9.1, Anhang A.3 |
|
| Fallback-Ketten | § 9.1, Anhang A.3 |
|
||||||
| faster-whisper | § 6.3, Anhang C |
|
| faster-whisper | § 6.3, Anhang C |
|
||||||
| Fehlerbehebung | § 13 |
|
| Fehlerbehebung | § 13 |
|
||||||
|
| Fix / Flex (Sprachmodus) | § 6.6 |
|
||||||
| Gedächtnis (Sitzung) | § 8.1 |
|
| Gedächtnis (Sitzung) | § 8.1 |
|
||||||
|
| Geräte-TTS (Web Speech API) | § 6.5.0 |
|
||||||
| Hybrid-Profil | § 3.2 |
|
| Hybrid-Profil | § 3.2 |
|
||||||
| Installation | § 2 |
|
| Installation | § 2 |
|
||||||
| Konfigurationsebenen / Priorität | § 6.1 |
|
| Konfigurationsebenen / Priorität | § 6.1 |
|
||||||
|
|
@ -2212,8 +2317,10 @@ in `app/dependencies.py` + Implementierung in `app/providers/`. → [Architektur
|
||||||
| Remote-Zugang / HTTPS / SSO | § 11 |
|
| Remote-Zugang / HTTPS / SSO | § 11 |
|
||||||
| Sachregister | Anhang D |
|
| Sachregister | Anhang D |
|
||||||
| Sitzungsgedächtnis | § 8.1 |
|
| Sitzungsgedächtnis | § 8.1 |
|
||||||
|
| Sprache wechseln (Fix/Flex) | § 6.6 |
|
||||||
| Sprech-Loop | § 5.2 |
|
| Sprech-Loop | § 5.2 |
|
||||||
| Stimmen (TTS) | § 6.5.1, § 6.5.2 |
|
| Stimmen (TTS) | § 6.5.1, § 6.5.2 |
|
||||||
|
| Stimme folgt Sprache (Flex) | § 6.6 |
|
||||||
| STT-Einstellungen | § 6.3 |
|
| STT-Einstellungen | § 6.3 |
|
||||||
| Streaming (Audio/Token/VAD) | § 6.8, Anhang B.6 |
|
| Streaming (Audio/Token/VAD) | § 6.8, Anhang B.6 |
|
||||||
| Tests | § 12 |
|
| Tests | § 12 |
|
||||||
|
|
|
||||||
|
|
@ -16,7 +16,7 @@ router = APIRouter()
|
||||||
|
|
||||||
_CONFIG_DIR = Path(__file__).resolve().parents[2] / "config"
|
_CONFIG_DIR = Path(__file__).resolve().parents[2] / "config"
|
||||||
_ALLOWED_SECTIONS = {"abbreviations", "units", "terms"}
|
_ALLOWED_SECTIONS = {"abbreviations", "units", "terms"}
|
||||||
_ALLOWED_LANGS = {"de", "en"}
|
_ALLOWED_LANGS = {"de", "en", "fr", "es", "it", "nl", "ru", "zh"}
|
||||||
|
|
||||||
|
|
||||||
class PronunciationEntry(BaseModel):
|
class PronunciationEntry(BaseModel):
|
||||||
|
|
@ -231,6 +231,10 @@ async def add_pronunciation(lang: str, entry: PronunciationEntry):
|
||||||
raise HTTPException(status_code=422, detail="key und value duerfen nicht leer sein.")
|
raise HTTPException(status_code=422, detail="key und value duerfen nicht leer sein.")
|
||||||
data = _read_pronunciation(lang)
|
data = _read_pronunciation(lang)
|
||||||
data[entry.section][entry.key.strip()] = entry.value.strip()
|
data[entry.section][entry.key.strip()] = entry.value.strip()
|
||||||
|
# Nach jedem Einfügen: Sektion alphabetisch aufsteigend nach Schlüssel sortieren.
|
||||||
|
data[entry.section] = dict(
|
||||||
|
sorted(data[entry.section].items(), key=lambda kv: kv[0].lower())
|
||||||
|
)
|
||||||
_write_pronunciation(lang, data)
|
_write_pronunciation(lang, data)
|
||||||
return {"section": entry.section, "key": entry.key.strip(), "value": entry.value.strip()}
|
return {"section": entry.section, "key": entry.key.strip(), "value": entry.value.strip()}
|
||||||
|
|
||||||
|
|
@ -314,6 +318,26 @@ async def admin_log_ws(websocket: WebSocket, key: str | None = None):
|
||||||
return
|
return
|
||||||
|
|
||||||
await websocket.accept()
|
await websocket.accept()
|
||||||
|
|
||||||
|
# Hinweis, falls die laufende Instanz NICHT der systemd-Dienst ist (z. B. manueller
|
||||||
|
# `uvicorn --reload`-Start). Dann hat das Journal dieser Unit keine aktuellen Zeilen,
|
||||||
|
# und der Log-Tab bliebe sonst kommentarlos leer.
|
||||||
|
try:
|
||||||
|
check = await asyncio.create_subprocess_exec(
|
||||||
|
"systemctl", "--user", "is-active", "voice-assistant.service",
|
||||||
|
stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.DEVNULL,
|
||||||
|
)
|
||||||
|
out, _ = await check.communicate()
|
||||||
|
if out.decode().strip() != "active":
|
||||||
|
await websocket.send_text(
|
||||||
|
"⚠ Der systemd-Dienst 'voice-assistant.service' ist nicht aktiv — "
|
||||||
|
"die laufende Instanz wurde vermutlich manuell gestartet (uvicorn --reload). "
|
||||||
|
"Live-Logs erscheinen hier nur, wenn das Gateway als Dienst läuft "
|
||||||
|
"(systemctl --user start voice-assistant.service). Manuelle Starts loggen ins Terminal."
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
proc = await asyncio.create_subprocess_exec(
|
proc = await asyncio.create_subprocess_exec(
|
||||||
"journalctl", "--user", "-f", "-u", "voice-assistant.service",
|
"journalctl", "--user", "-f", "-u", "voice-assistant.service",
|
||||||
"-n", "100", "--no-pager", "-o", "short",
|
"-n", "100", "--no-pager", "-o", "short",
|
||||||
|
|
|
||||||
|
|
@ -12,6 +12,7 @@ from app.dependencies import (
|
||||||
build_orchestrator,
|
build_orchestrator,
|
||||||
resolve_output_endpoint,
|
resolve_output_endpoint,
|
||||||
get_store,
|
get_store,
|
||||||
|
piper_voice_for_language,
|
||||||
)
|
)
|
||||||
from app.core.memory_extractor import maybe_schedule_extraction
|
from app.core.memory_extractor import maybe_schedule_extraction
|
||||||
from app.quota import enforce_quota, record_usage, QuotaExceededError
|
from app.quota import enforce_quota, record_usage, QuotaExceededError
|
||||||
|
|
@ -48,13 +49,11 @@ async def chat(
|
||||||
"input_endpoint": payload.input_endpoint,
|
"input_endpoint": payload.input_endpoint,
|
||||||
"output_endpoint": payload.output_endpoint,
|
"output_endpoint": payload.output_endpoint,
|
||||||
"language": payload.language,
|
"language": payload.language,
|
||||||
|
"language_mode": payload.language_mode,
|
||||||
"stt_provider": payload.stt_provider,
|
"stt_provider": payload.stt_provider,
|
||||||
"llm_provider": payload.llm_provider,
|
"llm_provider": payload.llm_provider,
|
||||||
"tts_provider": payload.tts_provider,
|
"tts_provider": payload.tts_provider,
|
||||||
}
|
}
|
||||||
# None -> der gewaehlte TTS-Provider nimmt seinen eigenen Default.
|
|
||||||
voice = payload.voice
|
|
||||||
|
|
||||||
store = get_store()
|
store = get_store()
|
||||||
|
|
||||||
try:
|
try:
|
||||||
|
|
@ -96,6 +95,9 @@ async def chat(
|
||||||
except QuotaExceededError as exc:
|
except QuotaExceededError as exc:
|
||||||
raise HTTPException(status_code=429, detail=str(exc))
|
raise HTTPException(status_code=429, detail=str(exc))
|
||||||
|
|
||||||
|
# Explizit angeforderte Stimme gewinnt; sonst folgt sie der Routensprache.
|
||||||
|
voice = payload.voice or piper_voice_for_language(route.tts_provider, route.language)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
trace, audio = await orchestrator.chat_text(
|
trace, audio = await orchestrator.chat_text(
|
||||||
payload.text,
|
payload.text,
|
||||||
|
|
@ -103,6 +105,8 @@ async def chat(
|
||||||
voice=voice,
|
voice=voice,
|
||||||
output=output,
|
output=output,
|
||||||
history=llm_context,
|
history=llm_context,
|
||||||
|
language_mode=route.language_mode,
|
||||||
|
text_only=bool(payload.text_only),
|
||||||
)
|
)
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
raise HTTPException(status_code=502, detail=str(exc))
|
raise HTTPException(status_code=502, detail=str(exc))
|
||||||
|
|
|
||||||
|
|
@ -23,9 +23,10 @@ async def get_me(user: User = Depends(require_user)):
|
||||||
|
|
||||||
@router.put("/me/prefs")
|
@router.put("/me/prefs")
|
||||||
async def set_my_prefs(payload: UserPrefs, user: User = Depends(require_user)):
|
async def set_my_prefs(payload: UserPrefs, user: User = Depends(require_user)):
|
||||||
"""Setzt die dauerhaften Routing-Praeferenzen des Nutzers (nur gesetzte Felder)."""
|
"""Setzt dauerhafter Routing-Präferenzen des Nutzers. Merge mit bestehenden Prefs."""
|
||||||
prefs = {k: v for k, v in payload.model_dump().items() if v is not None}
|
new = {k: v for k, v in payload.model_dump().items() if v is not None}
|
||||||
updated = get_store().set_user_prefs(user.id, prefs)
|
merged = {**user.prefs, **new}
|
||||||
|
updated = get_store().set_user_prefs(user.id, merged)
|
||||||
return {"user_id": updated.id, "prefs": updated.prefs}
|
return {"user_id": updated.id, "prefs": updated.prefs}
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -11,6 +11,7 @@ from app.dependencies import (
|
||||||
build_orchestrator,
|
build_orchestrator,
|
||||||
resolve_output_endpoint,
|
resolve_output_endpoint,
|
||||||
get_store,
|
get_store,
|
||||||
|
piper_voice_for_language,
|
||||||
)
|
)
|
||||||
from app.quota import enforce_quota, record_usage, QuotaExceededError
|
from app.quota import enforce_quota, record_usage, QuotaExceededError
|
||||||
from app.schemas import SpeakRequest
|
from app.schemas import SpeakRequest
|
||||||
|
|
@ -32,11 +33,10 @@ async def speak(
|
||||||
"language": payload.language,
|
"language": payload.language,
|
||||||
"tts_provider": payload.tts_provider,
|
"tts_provider": payload.tts_provider,
|
||||||
}
|
}
|
||||||
# None -> der gewaehlte TTS-Provider nimmt seinen eigenen Default.
|
|
||||||
voice = payload.voice
|
|
||||||
|
|
||||||
try:
|
try:
|
||||||
route = resolve_route(user, session_id, overrides)
|
route = resolve_route(user, session_id, overrides)
|
||||||
|
# Explizit angefragte Stimme gewinnt; sonst folgt sie der Sprache (Piper-Parität zu /chat).
|
||||||
|
voice = payload.voice or piper_voice_for_language(route.tts_provider, route.language)
|
||||||
orchestrator = build_orchestrator(route)
|
orchestrator = build_orchestrator(route)
|
||||||
output = await resolve_output_endpoint(route)
|
output = await resolve_output_endpoint(route)
|
||||||
except SessionOwnershipError as exc:
|
except SessionOwnershipError as exc:
|
||||||
|
|
|
||||||
|
|
@ -26,6 +26,7 @@ from app.dependencies import (
|
||||||
resolve_route,
|
resolve_route,
|
||||||
build_orchestrator,
|
build_orchestrator,
|
||||||
resolve_output_endpoint,
|
resolve_output_endpoint,
|
||||||
|
piper_voice_for_language,
|
||||||
)
|
)
|
||||||
from app.store import ANONYMOUS_USER_ID, SessionOwnershipError
|
from app.store import ANONYMOUS_USER_ID, SessionOwnershipError
|
||||||
from app.audio.vad import EnergyVAD
|
from app.audio.vad import EnergyVAD
|
||||||
|
|
@ -40,6 +41,7 @@ _OVERRIDE_KEYS = (
|
||||||
"input_endpoint",
|
"input_endpoint",
|
||||||
"output_endpoint",
|
"output_endpoint",
|
||||||
"language",
|
"language",
|
||||||
|
"language_mode",
|
||||||
"stt_provider",
|
"stt_provider",
|
||||||
"llm_provider",
|
"llm_provider",
|
||||||
"tts_provider",
|
"tts_provider",
|
||||||
|
|
@ -62,7 +64,10 @@ async def _resolve(user, session_id, options):
|
||||||
return route, orchestrator, output
|
return route, orchestrator, output
|
||||||
|
|
||||||
|
|
||||||
async def _run_turn(websocket, store, user, session_id, route, orchestrator, output, text, options):
|
async def _run_turn(
|
||||||
|
websocket, store, user, session_id, route, orchestrator, output, text, options,
|
||||||
|
detected_language: str | None = None, effective_voice: str | None = None,
|
||||||
|
):
|
||||||
"""Faehrt einen Antwort-Turn und streamt die Events an den Client."""
|
"""Faehrt einen Antwort-Turn und streamt die Events an den Client."""
|
||||||
conversation = (
|
conversation = (
|
||||||
store.get_recent_messages(session_id, settings.history_max_messages)
|
store.get_recent_messages(session_id, settings.history_max_messages)
|
||||||
|
|
@ -108,13 +113,24 @@ async def _run_turn(websocket, store, user, session_id, route, orchestrator, out
|
||||||
|
|
||||||
await websocket.send_json({"type": "ack", "route": route.as_dict()})
|
await websocket.send_json({"type": "ack", "route": route.as_dict()})
|
||||||
|
|
||||||
# Ohne explizite Stimme None lassen -> jeder TTS-Provider nimmt SEINEN Default
|
# Stimme: explizit angeforderte gewinnt, sonst die vom Caller (Sprach-Turn)
|
||||||
# (OpenRouter: OPENROUTER_TTS_VOICE, piper: PIPER_VOICE).
|
# vorberechnete sprachpassende Stimme, sonst folgt sie der Routensprache
|
||||||
voice = options.get("voice")
|
# (greift v. a. beim Text-Chat ohne Spracherkennung).
|
||||||
|
explicit_voice = options.get("voice")
|
||||||
|
if explicit_voice:
|
||||||
|
voice = explicit_voice
|
||||||
|
elif effective_voice is not None:
|
||||||
|
voice = effective_voice
|
||||||
|
else:
|
||||||
|
voice = piper_voice_for_language(route.tts_provider, route.language)
|
||||||
stream = bool(options.get("stream"))
|
stream = bool(options.get("stream"))
|
||||||
|
# text_only: Geräte-TTS (Web Speech API) spricht selbst -> kein Server-Audio erzeugen/senden.
|
||||||
|
text_only = bool(options.get("text_only"))
|
||||||
# audio_stream: explizite Anfrage gewinnt, sonst der serverseitige Default (Admin).
|
# audio_stream: explizite Anfrage gewinnt, sonst der serverseitige Default (Admin).
|
||||||
audio_stream = bool(options["audio_stream"]) if "audio_stream" in options \
|
audio_stream = False if text_only else (
|
||||||
|
bool(options["audio_stream"]) if "audio_stream" in options
|
||||||
else settings.audio_stream_default
|
else settings.audio_stream_default
|
||||||
|
)
|
||||||
|
|
||||||
on_token = None
|
on_token = None
|
||||||
if stream:
|
if stream:
|
||||||
|
|
@ -141,6 +157,9 @@ async def _run_turn(websocket, store, user, session_id, route, orchestrator, out
|
||||||
history=llm_context,
|
history=llm_context,
|
||||||
on_token=on_token,
|
on_token=on_token,
|
||||||
on_audio=on_audio,
|
on_audio=on_audio,
|
||||||
|
language_mode=route.language_mode,
|
||||||
|
detected_language=detected_language,
|
||||||
|
text_only=text_only,
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
trace, audio = await orchestrator.chat_text(
|
trace, audio = await orchestrator.chat_text(
|
||||||
|
|
@ -149,6 +168,9 @@ async def _run_turn(websocket, store, user, session_id, route, orchestrator, out
|
||||||
voice=voice,
|
voice=voice,
|
||||||
output=output,
|
output=output,
|
||||||
history=llm_context,
|
history=llm_context,
|
||||||
|
language_mode=route.language_mode,
|
||||||
|
detected_language=detected_language,
|
||||||
|
text_only=text_only,
|
||||||
)
|
)
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
await websocket.send_json({"type": "error", "status": 502, "detail": str(exc)})
|
await websocket.send_json({"type": "error", "status": 502, "detail": str(exc)})
|
||||||
|
|
@ -164,7 +186,8 @@ async def _run_turn(websocket, store, user, session_id, route, orchestrator, out
|
||||||
await websocket.send_json(
|
await websocket.send_json(
|
||||||
{"type": "semantic", "text": trace.semantic_response, "spoken": trace.spoken_response}
|
{"type": "semantic", "text": trace.semantic_response, "spoken": trace.spoken_response}
|
||||||
)
|
)
|
||||||
if not audio_stream:
|
# Im text_only-Modus kommt kein Audio (Gerät spricht selbst).
|
||||||
|
if not audio_stream and not text_only:
|
||||||
await websocket.send_bytes(audio)
|
await websocket.send_bytes(audio)
|
||||||
await websocket.send_json({"type": "done", "audio_format": "pcm", "sample_rate": 24000})
|
await websocket.send_json({"type": "done", "audio_format": "pcm", "sample_rate": 24000})
|
||||||
|
|
||||||
|
|
@ -202,19 +225,42 @@ async def _voice_turn(websocket, store, user, session_id, audio, fmt, options):
|
||||||
except RoutingError as exc:
|
except RoutingError as exc:
|
||||||
await websocket.send_json({"type": "error", "status": 422, "detail": str(exc)})
|
await websocket.send_json({"type": "error", "status": 422, "detail": str(exc)})
|
||||||
return
|
return
|
||||||
|
|
||||||
try:
|
try:
|
||||||
transcript = await orchestrator.stt.transcribe(audio, fmt=fmt, language=route.language)
|
if route.language_mode == "flex":
|
||||||
|
# Flex: Sprache auto-erkennen, kein Übersetzen durch Whisper.
|
||||||
|
transcript, detected_language = await orchestrator.stt.transcribe_detect(
|
||||||
|
audio, fmt=fmt, language=None
|
||||||
|
)
|
||||||
|
# Stimme folgt der erkannten Sprache (Fallback: Routensprache).
|
||||||
|
effective_voice = piper_voice_for_language(
|
||||||
|
route.tts_provider, detected_language or route.language
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
# Fix: konfigurierte Sprache an Whisper → Whisper übersetzt automatisch.
|
||||||
|
transcript = await orchestrator.stt.transcribe(audio, fmt=fmt, language=route.language)
|
||||||
|
detected_language = None
|
||||||
|
# Stimme folgt der konfigurierten Sprache.
|
||||||
|
effective_voice = piper_voice_for_language(route.tts_provider, route.language)
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
await websocket.send_json({"type": "error", "status": 502, "detail": str(exc)})
|
await websocket.send_json({"type": "error", "status": 502, "detail": str(exc)})
|
||||||
return
|
return
|
||||||
await websocket.send_json({"type": "transcript", "text": transcript})
|
|
||||||
|
await websocket.send_json({
|
||||||
|
"type": "transcript",
|
||||||
|
"text": transcript,
|
||||||
|
"detected_language": detected_language,
|
||||||
|
})
|
||||||
if not transcript or not transcript.strip():
|
if not transcript or not transcript.strip():
|
||||||
await websocket.send_json({
|
await websocket.send_json({
|
||||||
"type": "error",
|
"type": "error",
|
||||||
"detail": "Keine Sprache erkannt — bitte erneut sprechen.",
|
"detail": "Keine Sprache erkannt — bitte erneut sprechen.",
|
||||||
})
|
})
|
||||||
return
|
return
|
||||||
await _run_turn(websocket, store, user, session_id, route, orchestrator, output, transcript, options)
|
await _run_turn(
|
||||||
|
websocket, store, user, session_id, route, orchestrator, output, transcript, options,
|
||||||
|
detected_language=detected_language, effective_voice=effective_voice,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
@router.websocket("/ws/chat")
|
@router.websocket("/ws/chat")
|
||||||
|
|
|
||||||
|
|
@ -112,6 +112,7 @@ class Settings(BaseSettings):
|
||||||
openrouter_tts_voice: str = "alloy"
|
openrouter_tts_voice: str = "alloy"
|
||||||
openrouter_llm_model: str = "openai/gpt-4.1-mini"
|
openrouter_llm_model: str = "openai/gpt-4.1-mini"
|
||||||
default_language: str = "de"
|
default_language: str = "de"
|
||||||
|
default_language_mode: str = "fix"
|
||||||
default_input_endpoint: str = "local-default"
|
default_input_endpoint: str = "local-default"
|
||||||
default_output_endpoint: str = "local-default"
|
default_output_endpoint: str = "local-default"
|
||||||
default_stt_provider: str = "openrouter"
|
default_stt_provider: str = "openrouter"
|
||||||
|
|
@ -144,6 +145,9 @@ class Settings(BaseSettings):
|
||||||
chatterbox_lang: str = "de"
|
chatterbox_lang: str = "de"
|
||||||
chatterbox_speed: float = 1.0
|
chatterbox_speed: float = 1.0
|
||||||
chatterbox_timeout: int = 180
|
chatterbox_timeout: int = 180
|
||||||
|
# Verzeichnis mit nativen Referenz-WAVs je Sprache (Konvention <lang>.wav, z. B. fr.wav).
|
||||||
|
# Greift cross-lingual: pro Sprache eine muttersprachliche Stimme. Leer -> nur chatterbox_voice.
|
||||||
|
chatterbox_voices_dir: str = str(BASE_DIR / "config" / "voices")
|
||||||
db_path: str = str(BASE_DIR / "data" / "voice-assistant.db")
|
db_path: str = str(BASE_DIR / "data" / "voice-assistant.db")
|
||||||
admin_api_key: str = ""
|
admin_api_key: str = ""
|
||||||
auth_enabled: bool = True
|
auth_enabled: bool = True
|
||||||
|
|
@ -161,6 +165,9 @@ class Settings(BaseSettings):
|
||||||
trusted_proxy_ips: str = "" # kommasepariert; IP(s) des Reverse-Proxys
|
trusted_proxy_ips: str = "" # kommasepariert; IP(s) des Reverse-Proxys
|
||||||
admin_users: str = "" # kommaseparierte SSO-Usernamen mit Admin-Rechten
|
admin_users: str = "" # kommaseparierte SSO-Usernamen mit Admin-Rechten
|
||||||
sso_logout_url: str = "" # Logout-Link fuers Frontend (SSO-Portal)
|
sso_logout_url: str = "" # Logout-Link fuers Frontend (SSO-Portal)
|
||||||
|
# Login-/Portal-URL: unauthentifizierte Seitenaufrufe werden hierhin umgeleitet
|
||||||
|
# (Defense-in-Depth zusaetzlich zu SSOwat). Leer -> stattdessen HTTP 401.
|
||||||
|
sso_login_url: str = ""
|
||||||
history_max_messages: int = 10
|
history_max_messages: int = 10
|
||||||
# Automatische Erinnerungs-Extraktion: das LLM destilliert dauerhafte Fakten
|
# Automatische Erinnerungs-Extraktion: das LLM destilliert dauerhafte Fakten
|
||||||
# aus dem Gespraech und legt sie als Nutzer-Erinnerungen ab (best-effort,
|
# aus dem Gespraech und legt sie als Nutzer-Erinnerungen ab (best-effort,
|
||||||
|
|
|
||||||
|
|
@ -78,7 +78,7 @@ class Orchestrator:
|
||||||
spoken, language=language, level=self.normalize_level
|
spoken, language=language, level=self.normalize_level
|
||||||
)
|
)
|
||||||
with _stage("tts"):
|
with _stage("tts"):
|
||||||
audio = await self.tts.synthesize(normalized, voice=voice)
|
audio = await self.tts.synthesize(normalized, voice=voice, language=language)
|
||||||
await self._emit_to_output(audio, output)
|
await self._emit_to_output(audio, output)
|
||||||
return audio
|
return audio
|
||||||
|
|
||||||
|
|
@ -89,35 +89,48 @@ class Orchestrator:
|
||||||
voice: str | None = None,
|
voice: str | None = None,
|
||||||
output=None,
|
output=None,
|
||||||
history: list[dict] | None = None,
|
history: list[dict] | None = None,
|
||||||
|
language_mode: str = "fix",
|
||||||
|
detected_language: str | None = None,
|
||||||
|
text_only: bool = False,
|
||||||
):
|
):
|
||||||
trace = PipelineTrace()
|
trace = PipelineTrace()
|
||||||
|
|
||||||
trace.raw_transcript = text
|
trace.raw_transcript = text
|
||||||
trace.cleaned_transcript = await self.input_cleaner.run(text or "")
|
trace.cleaned_transcript = await self.input_cleaner.run(text or "")
|
||||||
|
|
||||||
|
# Flex: die erkannte Sprache bestimmt die Ausgabe. Gibt es keine (z. B. Text-Chat
|
||||||
|
# ohne Audio), bleibt sie None -> das LLM antwortet von selbst in der Eingabesprache.
|
||||||
|
# Fix: immer die konfigurierte Sprache (Whisper hat die Eingabe übersetzt).
|
||||||
|
effective_language = detected_language if language_mode == "flex" else language
|
||||||
|
|
||||||
with _stage("llm"):
|
with _stage("llm"):
|
||||||
trace.semantic_response = await self.llm.complete(
|
trace.semantic_response = await self.llm.complete(
|
||||||
trace.cleaned_transcript or "",
|
trace.cleaned_transcript or "",
|
||||||
history=history,
|
history=history,
|
||||||
language=language,
|
language=effective_language,
|
||||||
)
|
)
|
||||||
if not trace.semantic_response:
|
if not trace.semantic_response:
|
||||||
raise RuntimeError("LLM returned an empty response")
|
raise RuntimeError("LLM returned an empty response")
|
||||||
|
|
||||||
trace.spoken_response = await self.spoken_adapter.run(
|
trace.spoken_response = await self.spoken_adapter.run(
|
||||||
trace.semantic_response,
|
trace.semantic_response,
|
||||||
language=language,
|
language=effective_language,
|
||||||
)
|
)
|
||||||
trace.tts_ready_text = await self.tts_normalizer.run(
|
trace.tts_ready_text = await self.tts_normalizer.run(
|
||||||
trace.spoken_response,
|
trace.spoken_response,
|
||||||
language=language,
|
language=effective_language,
|
||||||
level=self.normalize_level,
|
level=self.normalize_level,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# text_only: Geräte-TTS (Web Speech API) übernimmt das Sprechen -> kein Server-Audio.
|
||||||
|
if text_only:
|
||||||
|
return trace, b""
|
||||||
|
|
||||||
with _stage("tts"):
|
with _stage("tts"):
|
||||||
audio = await self.tts.synthesize(
|
audio = await self.tts.synthesize(
|
||||||
trace.tts_ready_text,
|
trace.tts_ready_text,
|
||||||
voice=voice,
|
voice=voice,
|
||||||
|
language=effective_language,
|
||||||
)
|
)
|
||||||
await self._emit_to_output(audio, output)
|
await self._emit_to_output(audio, output)
|
||||||
return trace, audio
|
return trace, audio
|
||||||
|
|
@ -131,6 +144,9 @@ class Orchestrator:
|
||||||
history: list[dict] | None = None,
|
history: list[dict] | None = None,
|
||||||
on_token=None,
|
on_token=None,
|
||||||
on_audio=None,
|
on_audio=None,
|
||||||
|
language_mode: str = "fix",
|
||||||
|
detected_language: str | None = None,
|
||||||
|
text_only: bool = False,
|
||||||
):
|
):
|
||||||
"""Wie chat_text, aber gestreamt.
|
"""Wie chat_text, aber gestreamt.
|
||||||
|
|
||||||
|
|
@ -142,24 +158,29 @@ class Orchestrator:
|
||||||
trace.raw_transcript = text
|
trace.raw_transcript = text
|
||||||
trace.cleaned_transcript = await self.input_cleaner.run(text or "")
|
trace.cleaned_transcript = await self.input_cleaner.run(text or "")
|
||||||
|
|
||||||
chunker = SentenceChunker() if on_audio else None
|
# Siehe chat_text: Flex folgt der erkannten Sprache (None -> LLM spiegelt Eingabe),
|
||||||
|
# Fix nutzt die konfigurierte Sprache.
|
||||||
|
effective_language = detected_language if language_mode == "flex" else language
|
||||||
|
|
||||||
|
# text_only: kein Server-Audio (Geräte-TTS spricht selbst) -> kein Chunked-TTS.
|
||||||
|
chunker = SentenceChunker() if (on_audio and not text_only) else None
|
||||||
audio_parts: list[bytes] = []
|
audio_parts: list[bytes] = []
|
||||||
|
|
||||||
async def _emit_sentence(sentence: str) -> None:
|
async def _emit_sentence(sentence: str) -> None:
|
||||||
spoken = await self.spoken_adapter.run(sentence, language=language)
|
spoken = await self.spoken_adapter.run(sentence, language=effective_language)
|
||||||
ready = await self.tts_normalizer.run(
|
ready = await self.tts_normalizer.run(
|
||||||
spoken, language=language, level=self.normalize_level
|
spoken, language=effective_language, level=self.normalize_level
|
||||||
)
|
)
|
||||||
if not ready.strip():
|
if not ready.strip():
|
||||||
return
|
return
|
||||||
chunk = await self.tts.synthesize(ready, voice=voice)
|
chunk = await self.tts.synthesize(ready, voice=voice, language=effective_language)
|
||||||
audio_parts.append(chunk)
|
audio_parts.append(chunk)
|
||||||
await on_audio(chunk)
|
await on_audio(chunk)
|
||||||
|
|
||||||
parts: list[str] = []
|
parts: list[str] = []
|
||||||
stream_fn = getattr(self.llm, "stream", None)
|
stream_fn = getattr(self.llm, "stream", None)
|
||||||
if stream_fn is not None:
|
if stream_fn is not None:
|
||||||
async for delta in stream_fn(trace.cleaned_transcript or "", history=history, language=language):
|
async for delta in stream_fn(trace.cleaned_transcript or "", history=history, language=effective_language):
|
||||||
parts.append(delta)
|
parts.append(delta)
|
||||||
if on_token:
|
if on_token:
|
||||||
await on_token(delta)
|
await on_token(delta)
|
||||||
|
|
@ -167,8 +188,7 @@ class Orchestrator:
|
||||||
for sentence in chunker.feed(delta):
|
for sentence in chunker.feed(delta):
|
||||||
await _emit_sentence(sentence)
|
await _emit_sentence(sentence)
|
||||||
else:
|
else:
|
||||||
# Provider ohne Streaming -> komplette Antwort als ein Token.
|
result = await self.llm.complete(trace.cleaned_transcript or "", history=history, language=effective_language)
|
||||||
result = await self.llm.complete(trace.cleaned_transcript or "", history=history, language=language)
|
|
||||||
parts.append(result)
|
parts.append(result)
|
||||||
if on_token:
|
if on_token:
|
||||||
await on_token(result)
|
await on_token(result)
|
||||||
|
|
@ -182,21 +202,26 @@ class Orchestrator:
|
||||||
|
|
||||||
trace.spoken_response = await self.spoken_adapter.run(
|
trace.spoken_response = await self.spoken_adapter.run(
|
||||||
trace.semantic_response,
|
trace.semantic_response,
|
||||||
language=language,
|
language=effective_language,
|
||||||
)
|
)
|
||||||
trace.tts_ready_text = await self.tts_normalizer.run(
|
trace.tts_ready_text = await self.tts_normalizer.run(
|
||||||
trace.spoken_response,
|
trace.spoken_response,
|
||||||
language=language,
|
language=effective_language,
|
||||||
level=self.normalize_level,
|
level=self.normalize_level,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
if text_only:
|
||||||
|
return trace, b""
|
||||||
|
|
||||||
if chunker:
|
if chunker:
|
||||||
tail = chunker.flush()
|
tail = chunker.flush()
|
||||||
if tail:
|
if tail:
|
||||||
await _emit_sentence(tail)
|
await _emit_sentence(tail)
|
||||||
audio = b"".join(audio_parts)
|
audio = b"".join(audio_parts)
|
||||||
else:
|
else:
|
||||||
audio = await self.tts.synthesize(trace.tts_ready_text, voice=voice)
|
audio = await self.tts.synthesize(
|
||||||
|
trace.tts_ready_text, voice=voice, language=effective_language
|
||||||
|
)
|
||||||
|
|
||||||
await self._emit_to_output(audio, output)
|
await self._emit_to_output(audio, output)
|
||||||
return trace, audio
|
return trace, audio
|
||||||
|
|
|
||||||
|
|
@ -77,6 +77,7 @@ TTS_REGISTRY = {
|
||||||
s.chatterbox_speed,
|
s.chatterbox_speed,
|
||||||
s.tts_sample_rate,
|
s.tts_sample_rate,
|
||||||
s.chatterbox_timeout,
|
s.chatterbox_timeout,
|
||||||
|
voices_dir=s.chatterbox_voices_dir,
|
||||||
),
|
),
|
||||||
"piper": lambda s: PiperTTSProvider(
|
"piper": lambda s: PiperTTSProvider(
|
||||||
s.piper_bin, s.piper_voices_dir, s.piper_voice, s.tts_sample_rate
|
s.piper_bin, s.piper_voices_dir, s.piper_voice, s.tts_sample_rate
|
||||||
|
|
@ -148,8 +149,35 @@ ROUTE_KEYS = (
|
||||||
"llm_provider",
|
"llm_provider",
|
||||||
"tts_provider",
|
"tts_provider",
|
||||||
"language",
|
"language",
|
||||||
|
"language_mode",
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# Stimm-Auswahl nach Sprache: (effektive) Sprache → beste Piper-Stimme.
|
||||||
|
# Gilt in BEIDEN Modi — die gesprochene Stimme folgt immer der Antwortsprache.
|
||||||
|
LANG_TO_PIPER_VOICE: dict[str, str] = {
|
||||||
|
"de": "de_DE-thorsten-high",
|
||||||
|
"en": "en_US-lessac-high",
|
||||||
|
"fr": "fr_FR-siwis-medium",
|
||||||
|
"es": "es_ES-sharvard-medium",
|
||||||
|
"it": "it_IT-paola-medium",
|
||||||
|
"nl": "nl_NL-mls-medium",
|
||||||
|
"ru": "ru_RU-irina-medium",
|
||||||
|
"zh": "zh_CN-huayan-medium",
|
||||||
|
"cmn": "zh_CN-huayan-medium",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def piper_voice_for_language(tts_provider: str, language: str | None) -> str | None:
|
||||||
|
"""Liefert die zur Sprache passende Piper-Stimme (sonst None → Provider-Default).
|
||||||
|
|
||||||
|
Nur für Piper sinnvoll: Cloud-TTS (OpenRouter) und Chatterbox haben eigene
|
||||||
|
Stimmnamen (z. B. "Zephyr") und steuern die Sprache anders. Bei None nimmt der
|
||||||
|
jeweilige Provider seine konfigurierte Default-Stimme.
|
||||||
|
"""
|
||||||
|
if tts_provider == "piper" and language:
|
||||||
|
return LANG_TO_PIPER_VOICE.get(language)
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
class ResolvedRoute:
|
class ResolvedRoute:
|
||||||
|
|
@ -159,6 +187,7 @@ class ResolvedRoute:
|
||||||
llm_provider: str
|
llm_provider: str
|
||||||
tts_provider: str
|
tts_provider: str
|
||||||
language: str
|
language: str
|
||||||
|
language_mode: str = "fix"
|
||||||
|
|
||||||
def as_dict(self) -> dict:
|
def as_dict(self) -> dict:
|
||||||
return {
|
return {
|
||||||
|
|
@ -168,6 +197,7 @@ class ResolvedRoute:
|
||||||
"llm_provider": self.llm_provider,
|
"llm_provider": self.llm_provider,
|
||||||
"tts_provider": self.tts_provider,
|
"tts_provider": self.tts_provider,
|
||||||
"language": self.language,
|
"language": self.language,
|
||||||
|
"language_mode": self.language_mode,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -208,6 +238,7 @@ def resolve_route(
|
||||||
"llm_provider": cfg.default_llm_provider,
|
"llm_provider": cfg.default_llm_provider,
|
||||||
"tts_provider": cfg.default_tts_provider,
|
"tts_provider": cfg.default_tts_provider,
|
||||||
"language": cfg.default_language,
|
"language": cfg.default_language,
|
||||||
|
"language_mode": cfg.default_language_mode,
|
||||||
}
|
}
|
||||||
|
|
||||||
user_prefs = user.prefs if user is not None else {}
|
user_prefs = user.prefs if user is not None else {}
|
||||||
|
|
|
||||||
29
app/main.py
29
app/main.py
|
|
@ -5,7 +5,10 @@ from pathlib import Path
|
||||||
|
|
||||||
from fastapi import FastAPI, Request
|
from fastapi import FastAPI, Request
|
||||||
from fastapi.staticfiles import StaticFiles
|
from fastapi.staticfiles import StaticFiles
|
||||||
|
from starlette.responses import RedirectResponse, JSONResponse
|
||||||
|
|
||||||
|
from app.config import settings
|
||||||
|
from app.auth import authenticate, _bearer_token
|
||||||
from app.core.warmup import warmup_local_models
|
from app.core.warmup import warmup_local_models
|
||||||
from app.metrics import metrics
|
from app.metrics import metrics
|
||||||
from app.api.health import router as health_router
|
from app.api.health import router as health_router
|
||||||
|
|
@ -48,6 +51,32 @@ async def record_metrics(request: Request, call_next):
|
||||||
return response
|
return response
|
||||||
|
|
||||||
|
|
||||||
|
# Pfade mit eigener Auth, Health-Checks und Favicons: nicht gaten.
|
||||||
|
_PUBLIC_PREFIXES = ("/api", "/ws", "/health", "/favicon", "/apple-touch-icon")
|
||||||
|
|
||||||
|
|
||||||
|
@app.middleware("http")
|
||||||
|
async def gate_web_ui(request: Request, call_next):
|
||||||
|
"""Schuetzt die statische Web-UI: unauthentifizierte Seitenaufrufe -> SSO-Login (sonst 401).
|
||||||
|
|
||||||
|
Defense-in-Depth zusaetzlich zu SSOwat: Selbst wenn der Reverse-Proxy einen
|
||||||
|
unauthentifizierten Request durchliesse (oder jemand den Port direkt trifft),
|
||||||
|
bekommt er die Seite nicht ausgeliefert. API/WS haben ihre eigene Auth (require_user
|
||||||
|
-> 401/JSON), Health-Checks bleiben offen. Bei abgeschalteter Auth (LAN-Dev) liefert
|
||||||
|
authenticate() einen anonymen Nutzer -> die Seite wird wie bisher ausgeliefert.
|
||||||
|
"""
|
||||||
|
path = request.url.path
|
||||||
|
if not path.startswith(_PUBLIC_PREFIXES):
|
||||||
|
client_host = request.client.host if request.client else ""
|
||||||
|
token = _bearer_token(request.headers.get("authorization"))
|
||||||
|
if authenticate(request.headers, client_host, token) is None:
|
||||||
|
login = settings.sso_login_url.strip()
|
||||||
|
if login:
|
||||||
|
return RedirectResponse(login, status_code=302)
|
||||||
|
return JSONResponse({"detail": "Authentication required"}, status_code=401)
|
||||||
|
return await call_next(request)
|
||||||
|
|
||||||
|
|
||||||
app.include_router(health_router)
|
app.include_router(health_router)
|
||||||
app.include_router(chat_router, prefix="/api")
|
app.include_router(chat_router, prefix="/api")
|
||||||
app.include_router(transcribe_router, prefix="/api")
|
app.include_router(transcribe_router, prefix="/api")
|
||||||
|
|
|
||||||
|
|
@ -35,6 +35,19 @@ class FallbackSTTProvider(_Chain):
|
||||||
self._on_error(name)
|
self._on_error(name)
|
||||||
raise last_exc
|
raise last_exc
|
||||||
|
|
||||||
|
async def transcribe_detect(self, audio_bytes, fmt, language=None) -> tuple[str, str | None]:
|
||||||
|
last_exc = None
|
||||||
|
for index, (name, provider) in enumerate(self.entries):
|
||||||
|
try:
|
||||||
|
result = await provider.transcribe_detect(audio_bytes, fmt, language=language)
|
||||||
|
if index > 0:
|
||||||
|
self._on_fallback()
|
||||||
|
return result
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
last_exc = exc
|
||||||
|
self._on_error(name)
|
||||||
|
raise last_exc
|
||||||
|
|
||||||
|
|
||||||
class FallbackLLMProvider(_Chain):
|
class FallbackLLMProvider(_Chain):
|
||||||
async def complete(self, text, history=None, session_id=None, language=None) -> str:
|
async def complete(self, text, history=None, session_id=None, language=None) -> str:
|
||||||
|
|
@ -75,11 +88,13 @@ class FallbackLLMProvider(_Chain):
|
||||||
|
|
||||||
|
|
||||||
class FallbackTTSProvider(_Chain):
|
class FallbackTTSProvider(_Chain):
|
||||||
async def synthesize(self, text, voice=None, audio_format="pcm") -> bytes:
|
async def synthesize(self, text, voice=None, audio_format="pcm", language=None) -> bytes:
|
||||||
last_exc = None
|
last_exc = None
|
||||||
for index, (name, provider) in enumerate(self.entries):
|
for index, (name, provider) in enumerate(self.entries):
|
||||||
try:
|
try:
|
||||||
result = await provider.synthesize(text, voice=voice, audio_format=audio_format)
|
result = await provider.synthesize(
|
||||||
|
text, voice=voice, audio_format=audio_format, language=language
|
||||||
|
)
|
||||||
if index > 0:
|
if index > 0:
|
||||||
self._on_fallback()
|
self._on_fallback()
|
||||||
return result
|
return result
|
||||||
|
|
|
||||||
|
|
@ -16,11 +16,28 @@ _LANG_NAMES: dict[str, str] = {
|
||||||
|
|
||||||
|
|
||||||
def lang_instruction(language: str | None) -> str | None:
|
def lang_instruction(language: str | None) -> str | None:
|
||||||
"""'Respond in X.' instruction for the given ISO language code, or None."""
|
"""Sprach-Anweisung für den ISO-Code, oder None.
|
||||||
|
|
||||||
|
Bewusst nachdrücklich formuliert: Eine lange anderssprachige History (z. B. der
|
||||||
|
Nutzer hat zwischendurch englisch gesprochen) soll die gewünschte Sprache NICHT
|
||||||
|
überstimmen. Wird zusätzlich nah an die Generierung gesetzt (siehe with_lang_reminder).
|
||||||
|
"""
|
||||||
if not language:
|
if not language:
|
||||||
return None
|
return None
|
||||||
name = _LANG_NAMES.get(language, language)
|
name = _LANG_NAMES.get(language, language)
|
||||||
return f"Respond in {name}."
|
return f"Respond ONLY in {name}, regardless of the language used earlier in this conversation."
|
||||||
|
|
||||||
|
|
||||||
|
def with_lang_reminder(text: str, language: str | None) -> str:
|
||||||
|
"""Hängt eine knappe Sprach-Erinnerung an die letzte Nutzer-Nachricht.
|
||||||
|
|
||||||
|
Direkt vor der Generierung platziert wirkt sie stärker als der (weit zurückliegende)
|
||||||
|
System-Prompt, wenn die bisherige Unterhaltung in einer anderen Sprache lief.
|
||||||
|
"""
|
||||||
|
if not language:
|
||||||
|
return text
|
||||||
|
name = _LANG_NAMES.get(language, language)
|
||||||
|
return f"{text}\n\n[Bitte ausschließlich auf {name} antworten.]"
|
||||||
|
|
||||||
|
|
||||||
def sse_delta(line: str) -> str | None:
|
def sse_delta(line: str) -> str | None:
|
||||||
|
|
|
||||||
|
|
@ -2,7 +2,12 @@ from collections.abc import AsyncIterator
|
||||||
|
|
||||||
import httpx
|
import httpx
|
||||||
|
|
||||||
from app.providers.llm.base import LLMProvider, lang_instruction, sse_delta
|
from app.providers.llm.base import (
|
||||||
|
LLMProvider,
|
||||||
|
lang_instruction,
|
||||||
|
with_lang_reminder,
|
||||||
|
sse_delta,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class LocalOpenAICompatibleLLM(LLMProvider):
|
class LocalOpenAICompatibleLLM(LLMProvider):
|
||||||
|
|
@ -50,7 +55,8 @@ class LocalOpenAICompatibleLLM(LLMProvider):
|
||||||
if system_parts:
|
if system_parts:
|
||||||
messages.append({"role": "system", "content": "\n\n".join(system_parts)})
|
messages.append({"role": "system", "content": "\n\n".join(system_parts)})
|
||||||
messages.extend(rest)
|
messages.extend(rest)
|
||||||
messages.append({"role": "user", "content": text})
|
# Sprach-Erinnerung direkt an der letzten Nutzer-Nachricht (schlägt History-Trägheit).
|
||||||
|
messages.append({"role": "user", "content": with_lang_reminder(text, language)})
|
||||||
return messages
|
return messages
|
||||||
|
|
||||||
def _payload(
|
def _payload(
|
||||||
|
|
|
||||||
|
|
@ -2,7 +2,12 @@ from collections.abc import AsyncIterator
|
||||||
|
|
||||||
import httpx
|
import httpx
|
||||||
|
|
||||||
from app.providers.llm.base import LLMProvider, lang_instruction, sse_delta
|
from app.providers.llm.base import (
|
||||||
|
LLMProvider,
|
||||||
|
lang_instruction,
|
||||||
|
with_lang_reminder,
|
||||||
|
sse_delta,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
SYSTEM_PROMPT = """
|
SYSTEM_PROMPT = """
|
||||||
|
|
@ -69,7 +74,8 @@ class OpenRouterLLMProvider(LLMProvider):
|
||||||
messages = [{"role": "system", "content": system_content}]
|
messages = [{"role": "system", "content": system_content}]
|
||||||
if history:
|
if history:
|
||||||
messages.extend(history)
|
messages.extend(history)
|
||||||
messages.append({"role": "user", "content": text.strip()})
|
# Sprach-Erinnerung direkt an der letzten Nutzer-Nachricht (schlägt History-Trägheit).
|
||||||
|
messages.append({"role": "user", "content": with_lang_reminder(text.strip(), language)})
|
||||||
return messages
|
return messages
|
||||||
|
|
||||||
async def complete(
|
async def complete(
|
||||||
|
|
|
||||||
|
|
@ -1,5 +1,17 @@
|
||||||
from abc import ABC, abstractmethod
|
from abc import ABC, abstractmethod
|
||||||
|
|
||||||
|
|
||||||
class STTProvider(ABC):
|
class STTProvider(ABC):
|
||||||
@abstractmethod
|
@abstractmethod
|
||||||
async def transcribe(self, audio_bytes: bytes, fmt: str, language: str | None = None) -> str: ...
|
async def transcribe(self, audio_bytes: bytes, fmt: str, language: str | None = None) -> str: ...
|
||||||
|
|
||||||
|
async def transcribe_detect(
|
||||||
|
self, audio_bytes: bytes, fmt: str, language: str | None = None
|
||||||
|
) -> tuple[str, str | None]:
|
||||||
|
"""Transkribiert und liefert (text, detected_language).
|
||||||
|
|
||||||
|
Standardimplementierung delegiert an transcribe(); detected_language=None.
|
||||||
|
Provider mit Spracherkennung überschreiben diese Methode.
|
||||||
|
"""
|
||||||
|
text = await self.transcribe(audio_bytes, fmt, language=language)
|
||||||
|
return text, None
|
||||||
|
|
|
||||||
|
|
@ -38,12 +38,24 @@ class FasterWhisperProvider(STTProvider):
|
||||||
self.device = device or settings.faster_whisper_device
|
self.device = device or settings.faster_whisper_device
|
||||||
self.compute_type = compute_type or settings.faster_whisper_compute_type
|
self.compute_type = compute_type or settings.faster_whisper_compute_type
|
||||||
|
|
||||||
def _transcribe_sync(self, audio_bytes: bytes, language: str | None) -> str:
|
def _transcribe_sync(
|
||||||
|
self, audio_bytes: bytes, language: str | None
|
||||||
|
) -> tuple[str, str | None]:
|
||||||
model = _load_model(self.model_size, self.device, self.compute_type)
|
model = _load_model(self.model_size, self.device, self.compute_type)
|
||||||
segments, _info = model.transcribe(io.BytesIO(audio_bytes), language=language)
|
segments, info = model.transcribe(io.BytesIO(audio_bytes), language=language)
|
||||||
return "".join(segment.text for segment in segments).strip()
|
text = "".join(segment.text for segment in segments).strip()
|
||||||
|
detected = getattr(info, "language", None)
|
||||||
|
return text, detected
|
||||||
|
|
||||||
async def transcribe(self, audio_bytes: bytes, fmt: str, language: str | None = None) -> str:
|
async def transcribe(self, audio_bytes: bytes, fmt: str, language: str | None = None) -> str:
|
||||||
|
if not audio_bytes:
|
||||||
|
raise ValueError("STT input audio is empty")
|
||||||
|
text, _ = await asyncio.to_thread(self._transcribe_sync, audio_bytes, language)
|
||||||
|
return text
|
||||||
|
|
||||||
|
async def transcribe_detect(
|
||||||
|
self, audio_bytes: bytes, fmt: str, language: str | None = None
|
||||||
|
) -> tuple[str, str | None]:
|
||||||
if not audio_bytes:
|
if not audio_bytes:
|
||||||
raise ValueError("STT input audio is empty")
|
raise ValueError("STT input audio is empty")
|
||||||
return await asyncio.to_thread(self._transcribe_sync, audio_bytes, language)
|
return await asyncio.to_thread(self._transcribe_sync, audio_bytes, language)
|
||||||
|
|
|
||||||
|
|
@ -52,4 +52,48 @@ class OpenRouterSTTProvider(STTProvider):
|
||||||
except httpx.HTTPError as exc:
|
except httpx.HTTPError as exc:
|
||||||
raise RuntimeError(f"OpenRouter STT transport error: {exc}") from exc
|
raise RuntimeError(f"OpenRouter STT transport error: {exc}") from exc
|
||||||
|
|
||||||
return response.json().get("text", "")
|
data = response.json()
|
||||||
|
return data.get("text", "")
|
||||||
|
|
||||||
|
async def transcribe_detect(
|
||||||
|
self, audio_bytes: bytes, fmt: str, language: str | None = None
|
||||||
|
) -> tuple[str, str | None]:
|
||||||
|
if not self.api_key:
|
||||||
|
raise ValueError("OPENROUTER_API_KEY is empty")
|
||||||
|
if not self.model:
|
||||||
|
raise ValueError("OPENROUTER_STT_MODEL is empty")
|
||||||
|
if not audio_bytes:
|
||||||
|
raise ValueError("STT input audio is empty")
|
||||||
|
|
||||||
|
payload = {
|
||||||
|
"model": self.model,
|
||||||
|
"input_audio": {
|
||||||
|
"data": base64.b64encode(audio_bytes).decode("utf-8"),
|
||||||
|
"format": fmt,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
if language:
|
||||||
|
payload["language"] = language
|
||||||
|
|
||||||
|
timeout = httpx.Timeout(connect=10.0, read=120.0, write=30.0, pool=10.0)
|
||||||
|
async with httpx.AsyncClient(timeout=timeout) as client:
|
||||||
|
try:
|
||||||
|
response = await client.post(
|
||||||
|
"https://openrouter.ai/api/v1/audio/transcriptions",
|
||||||
|
headers={
|
||||||
|
"Authorization": f"Bearer {self.api_key}",
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
},
|
||||||
|
json=payload,
|
||||||
|
)
|
||||||
|
response.raise_for_status()
|
||||||
|
except httpx.HTTPStatusError as exc:
|
||||||
|
raise RuntimeError(
|
||||||
|
f"OpenRouter STT error {exc.response.status_code}: {exc.response.text}"
|
||||||
|
) from exc
|
||||||
|
except httpx.TimeoutException as exc:
|
||||||
|
raise RuntimeError("OpenRouter STT timeout") from exc
|
||||||
|
except httpx.HTTPError as exc:
|
||||||
|
raise RuntimeError(f"OpenRouter STT transport error: {exc}") from exc
|
||||||
|
data = response.json()
|
||||||
|
return data.get("text", ""), data.get("language")
|
||||||
|
|
|
||||||
|
|
@ -2,4 +2,10 @@ from abc import ABC, abstractmethod
|
||||||
|
|
||||||
class TTSProvider(ABC):
|
class TTSProvider(ABC):
|
||||||
@abstractmethod
|
@abstractmethod
|
||||||
async def synthesize(self, text: str, voice: str | None = None, audio_format: str = "pcm") -> bytes: ...
|
async def synthesize(
|
||||||
|
self,
|
||||||
|
text: str,
|
||||||
|
voice: str | None = None,
|
||||||
|
audio_format: str = "pcm",
|
||||||
|
language: str | None = None,
|
||||||
|
) -> bytes: ...
|
||||||
|
|
|
||||||
|
|
@ -15,6 +15,7 @@ import asyncio
|
||||||
import io
|
import io
|
||||||
import time
|
import time
|
||||||
import wave
|
import wave
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
import httpx
|
import httpx
|
||||||
|
|
||||||
|
|
@ -31,6 +32,7 @@ class ChatterboxTTSProvider(TTSProvider):
|
||||||
speed: float = 1.0,
|
speed: float = 1.0,
|
||||||
target_rate: int = 24000,
|
target_rate: int = 24000,
|
||||||
timeout: int = 180,
|
timeout: int = 180,
|
||||||
|
voices_dir: str = "",
|
||||||
):
|
):
|
||||||
self.base_url = base_url.rstrip("/")
|
self.base_url = base_url.rstrip("/")
|
||||||
self.voice = (voice or "").strip() # Pfad zu Referenz-WAV (Voice-Cloning) oder leer
|
self.voice = (voice or "").strip() # Pfad zu Referenz-WAV (Voice-Cloning) oder leer
|
||||||
|
|
@ -38,32 +40,51 @@ class ChatterboxTTSProvider(TTSProvider):
|
||||||
self.speed = speed
|
self.speed = speed
|
||||||
self.target_rate = int(target_rate)
|
self.target_rate = int(target_rate)
|
||||||
self.timeout = timeout
|
self.timeout = timeout
|
||||||
|
# Absolut auflösen: der Chatterbox-Dienst liest den Pfad mit eigenem CWD.
|
||||||
|
self.voices_dir = Path(voices_dir).expanduser().resolve() if voices_dir else None
|
||||||
|
|
||||||
def _ref_voice(self, voice: str | None) -> str | None:
|
def _lang_ref(self, lang: str) -> str | None:
|
||||||
# Eine angefragte Stimme gilt nur, wenn sie wie ein WAV-Pfad aussieht
|
"""Native Referenz-WAV der Sprache (Konvention <voices_dir>/<lang>.wav), falls vorhanden."""
|
||||||
# (Cloud-Stimmennamen wie "Zephyr"/"alloy" ignorieren -> Default nehmen).
|
if not self.voices_dir:
|
||||||
|
return None
|
||||||
|
path = self.voices_dir / f"{lang}.wav"
|
||||||
|
return str(path) if path.exists() else None
|
||||||
|
|
||||||
|
def _ref_voice(self, voice: str | None, lang: str) -> str | None:
|
||||||
|
# 1. Explizit angefragte Stimme nur, wenn sie wie ein WAV-Pfad aussieht
|
||||||
|
# (Cloud-Stimmennamen wie "Zephyr"/"alloy" ignorieren).
|
||||||
cand = (voice or "").strip()
|
cand = (voice or "").strip()
|
||||||
if cand.endswith(".wav"):
|
if cand.endswith(".wav"):
|
||||||
return cand
|
return cand
|
||||||
return self.voice or None
|
# 2. Native Stimme der jeweiligen Sprache, sonst die konfigurierte Default-Stimme.
|
||||||
|
return self._lang_ref(lang) or self.voice or None
|
||||||
|
|
||||||
|
# Chatterbox ist mehrsprachig und klont die Stimme cross-lingual: die Referenz-WAV
|
||||||
|
# liefert das Timbre, `lang` die Aussprache. So spricht die Klon-Stimme jede Sprache.
|
||||||
|
_LANG_ALIASES = {"cmn": "zh"}
|
||||||
|
|
||||||
async def synthesize(
|
async def synthesize(
|
||||||
self,
|
self,
|
||||||
text: str,
|
text: str,
|
||||||
voice: str | None = None,
|
voice: str | None = None,
|
||||||
audio_format: str = "pcm",
|
audio_format: str = "pcm",
|
||||||
|
language: str | None = None,
|
||||||
) -> bytes:
|
) -> bytes:
|
||||||
if not text or not text.strip():
|
if not text or not text.strip():
|
||||||
raise ValueError("TTS input text is empty")
|
raise ValueError("TTS input text is empty")
|
||||||
|
|
||||||
|
# Gesprächssprache bevorzugen; ohne Angabe der konfigurierte Default.
|
||||||
|
lang = (language or self.lang or "de").strip()
|
||||||
|
lang = self._LANG_ALIASES.get(lang, lang)
|
||||||
|
|
||||||
payload = {
|
payload = {
|
||||||
"text": text.strip(),
|
"text": text.strip(),
|
||||||
"lang": self.lang,
|
"lang": lang,
|
||||||
"speed": self.speed,
|
"speed": self.speed,
|
||||||
"keep_audio": True,
|
"keep_audio": True,
|
||||||
"no_playback": True,
|
"no_playback": True,
|
||||||
}
|
}
|
||||||
ref = self._ref_voice(voice)
|
ref = self._ref_voice(voice, lang)
|
||||||
if ref:
|
if ref:
|
||||||
payload["voice"] = ref
|
payload["voice"] = ref
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -21,6 +21,7 @@ class OpenRouterTTSProvider(TTSProvider):
|
||||||
text: str,
|
text: str,
|
||||||
voice: str | None = None,
|
voice: str | None = None,
|
||||||
audio_format: str = "pcm",
|
audio_format: str = "pcm",
|
||||||
|
language: str | None = None, # Cloud-TTS steuert Sprache über Stimme/Modell
|
||||||
) -> bytes:
|
) -> bytes:
|
||||||
if not self.api_key:
|
if not self.api_key:
|
||||||
raise ValueError("OPENROUTER_API_KEY is empty")
|
raise ValueError("OPENROUTER_API_KEY is empty")
|
||||||
|
|
|
||||||
|
|
@ -109,6 +109,7 @@ class PiperTTSProvider(TTSProvider):
|
||||||
text: str,
|
text: str,
|
||||||
voice: str | None = None,
|
voice: str | None = None,
|
||||||
audio_format: str = "pcm",
|
audio_format: str = "pcm",
|
||||||
|
language: str | None = None, # ignoriert: die Piper-Stimme kodiert die Sprache
|
||||||
) -> bytes:
|
) -> bytes:
|
||||||
if not text or not text.strip():
|
if not text or not text.strip():
|
||||||
raise ValueError("TTS input text is empty")
|
raise ValueError("TTS input text is empty")
|
||||||
|
|
|
||||||
|
|
@ -18,8 +18,9 @@ from app.config import Settings, settings as _base
|
||||||
RUNTIME_SETTABLE: dict[str, tuple[str, str, str]] = {
|
RUNTIME_SETTABLE: dict[str, tuple[str, str, str]] = {
|
||||||
"default_stt_provider": ("STT-Provider (Standard)", "str", "openrouter | faster-whisper"),
|
"default_stt_provider": ("STT-Provider (Standard)", "str", "openrouter | faster-whisper"),
|
||||||
"default_llm_provider": ("LLM-Provider (Standard)", "str", "openrouter | local-openai-compatible"),
|
"default_llm_provider": ("LLM-Provider (Standard)", "str", "openrouter | local-openai-compatible"),
|
||||||
"default_tts_provider": ("TTS-Provider (Standard)", "str", "openrouter | piper | chatterbox"),
|
"default_tts_provider": ("TTS-Provider (Standard)", "str", "openrouter | piper | chatterbox — Standard, pro Nutzer überschreibbar"),
|
||||||
"default_language": ("Sprache (Standard)", "str", "de | en | …"),
|
"default_language": ("Sprache (Standard)", "str", "de | en | … — Standard, pro Nutzer überschreibbar"),
|
||||||
|
"default_language_mode": ("Sprachmodus (Standard)", "str", "fix | flex — Standard, pro Nutzer überschreibbar"),
|
||||||
"openrouter_llm_model": ("LLM-Modell (OpenRouter)", "str", "z.B. google/gemini-3.1-flash-lite"),
|
"openrouter_llm_model": ("LLM-Modell (OpenRouter)", "str", "z.B. google/gemini-3.1-flash-lite"),
|
||||||
"openrouter_tts_model": ("TTS-Modell (OpenRouter)", "str", "z.B. google/gemini-3.1-flash-tts-preview"),
|
"openrouter_tts_model": ("TTS-Modell (OpenRouter)", "str", "z.B. google/gemini-3.1-flash-tts-preview"),
|
||||||
"openrouter_tts_voice": ("TTS-Stimme (OpenRouter)", "str", "z.B. Zephyr, Puck, Kore"),
|
"openrouter_tts_voice": ("TTS-Stimme (OpenRouter)", "str", "z.B. Zephyr, Puck, Kore"),
|
||||||
|
|
|
||||||
|
|
@ -46,10 +46,12 @@ class ChatRequest(BaseModel):
|
||||||
input_endpoint: str | None = None
|
input_endpoint: str | None = None
|
||||||
output_endpoint: str | None = None
|
output_endpoint: str | None = None
|
||||||
language: str | None = None
|
language: str | None = None
|
||||||
|
language_mode: Literal["fix", "flex"] | None = None
|
||||||
voice: str | None = None
|
voice: str | None = None
|
||||||
stt_provider: str | None = None
|
stt_provider: str | None = None
|
||||||
llm_provider: str | None = None
|
llm_provider: str | None = None
|
||||||
tts_provider: str | None = None
|
tts_provider: str | None = None
|
||||||
|
text_only: bool | None = None # True -> kein Server-Audio (Geräte-TTS spricht selbst)
|
||||||
|
|
||||||
|
|
||||||
class SessionRouteRequest(BaseModel):
|
class SessionRouteRequest(BaseModel):
|
||||||
|
|
@ -91,6 +93,7 @@ class UserPrefs(BaseModel):
|
||||||
llm_provider: str | None = None
|
llm_provider: str | None = None
|
||||||
tts_provider: str | None = None
|
tts_provider: str | None = None
|
||||||
language: str | None = None
|
language: str | None = None
|
||||||
|
language_mode: Literal["fix", "flex"] | None = None
|
||||||
|
|
||||||
|
|
||||||
class MemoryCreate(BaseModel):
|
class MemoryCreate(BaseModel):
|
||||||
|
|
|
||||||
333
app/web/app.js
333
app/web/app.js
|
|
@ -8,11 +8,121 @@ const promptEl = $("#prompt");
|
||||||
const formEl = $("#prompt-form");
|
const formEl = $("#prompt-form");
|
||||||
const micBtn = $("#mic");
|
const micBtn = $("#mic");
|
||||||
const ttsSel = $("#tts");
|
const ttsSel = $("#tts");
|
||||||
|
const langSel = $("#lang-sel");
|
||||||
|
|
||||||
// Gewählter TTS-Provider (leer = Default/piper) als Request-Override.
|
// "Flex" ist im Sprachmenü nur ein weiterer Wert: keine feste Sprache, sondern
|
||||||
|
// automatische Erkennung. Eine konkrete Sprache = Fix-Modus (Eingabe wird übersetzt).
|
||||||
|
// Beide internen Felder (language, language_mode) werden aus dieser einen Auswahl abgeleitet.
|
||||||
|
function langOverrides() {
|
||||||
|
const sel = langSel && langSel.value;
|
||||||
|
if (!sel || sel === "flex") return { language_mode: "flex" };
|
||||||
|
return { language: sel, language_mode: "fix" };
|
||||||
|
}
|
||||||
|
|
||||||
|
// Nutzer-Präferenzen als Request-Overrides für jeden Turn.
|
||||||
function overrides() {
|
function overrides() {
|
||||||
const t = ttsSel && ttsSel.value;
|
const out = { ...langOverrides() };
|
||||||
return t ? { tts_provider: t } : {};
|
if (isDeviceMode()) {
|
||||||
|
out.text_only = true; // kein Server-Audio -> Gerät spricht selbst
|
||||||
|
} else if (ttsSel && ttsSel.value) {
|
||||||
|
out.tts_provider = ttsSel.value;
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function savePrefs(obj) {
|
||||||
|
try {
|
||||||
|
await fetch("/api/me/prefs", {
|
||||||
|
method: "PUT",
|
||||||
|
headers: { "Content-Type": "application/json" },
|
||||||
|
body: JSON.stringify(obj),
|
||||||
|
});
|
||||||
|
} catch (e) { console.error("savePrefs", e); }
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------- Geräte-TTS (Web Speech API) ----------
|
||||||
|
// "Gerät" liest die Antwort lokal vor -> der Server schickt kein Audio (spart Bytes/Kosten).
|
||||||
|
const TTS_SUPPORTED = typeof window !== "undefined" && "speechSynthesis" in window;
|
||||||
|
const LANG_BCP47 = {
|
||||||
|
de: "de-DE", en: "en-US", fr: "fr-FR", es: "es-ES",
|
||||||
|
it: "it-IT", nl: "nl-NL", ru: "ru-RU", zh: "zh-CN",
|
||||||
|
};
|
||||||
|
|
||||||
|
function isMobile() {
|
||||||
|
const ua = navigator.userAgent || "";
|
||||||
|
return /Android|iPhone|iPad|iPod/i.test(ua) ||
|
||||||
|
(navigator.maxTouchPoints > 1 && /Macintosh/.test(ua)); // iPad ab iOS 13
|
||||||
|
}
|
||||||
|
|
||||||
|
function isDeviceMode() { return !!ttsSel && ttsSel.value === "device"; }
|
||||||
|
|
||||||
|
let _voices = [];
|
||||||
|
function loadVoices() { _voices = TTS_SUPPORTED ? speechSynthesis.getVoices() : []; }
|
||||||
|
if (TTS_SUPPORTED) {
|
||||||
|
loadVoices();
|
||||||
|
speechSynthesis.addEventListener("voiceschanged", loadVoices);
|
||||||
|
}
|
||||||
|
|
||||||
|
function pickVoice(bcp) {
|
||||||
|
if (!_voices.length) loadVoices();
|
||||||
|
const lc = bcp.toLowerCase();
|
||||||
|
const two = lc.slice(0, 2);
|
||||||
|
return _voices.find((v) => v.lang && v.lang.toLowerCase() === lc) ||
|
||||||
|
_voices.find((v) => v.lang && v.lang.toLowerCase().startsWith(two)) || null;
|
||||||
|
}
|
||||||
|
|
||||||
|
function _makeUtterance(text, lang) {
|
||||||
|
const u = new SpeechSynthesisUtterance(text);
|
||||||
|
u.lang = LANG_BCP47[lang] || lang || "de-DE";
|
||||||
|
const v = pickVoice(u.lang);
|
||||||
|
if (v) u.voice = v;
|
||||||
|
return u;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Vorlesen ohne Button-Status (automatisches Vorlesen der Antwort).
|
||||||
|
function deviceSpeak(text, lang) {
|
||||||
|
if (!TTS_SUPPORTED || !text) return;
|
||||||
|
speechSynthesis.cancel();
|
||||||
|
speechSynthesis.speak(_makeUtterance(text, lang));
|
||||||
|
}
|
||||||
|
|
||||||
|
// Vorlesen mit Button-Status (Replay): Icon -> ⏹ während des Sprechens, danach zurück.
|
||||||
|
function deviceSpeakBtn(text, lang, btn) {
|
||||||
|
if (!TTS_SUPPORTED || !text) { setReplayPlaying(btn, false); return; }
|
||||||
|
speechSynthesis.cancel();
|
||||||
|
const u = _makeUtterance(text, lang);
|
||||||
|
u.onend = u.onerror = () => setReplayPlaying(btn, false);
|
||||||
|
setReplayPlaying(btn, true);
|
||||||
|
speechSynthesis.speak(u);
|
||||||
|
}
|
||||||
|
|
||||||
|
// iOS: speechSynthesis darf nur nach einer Nutzergeste starten -> beim ersten Tap freischalten.
|
||||||
|
let _ttsUnlocked = false;
|
||||||
|
function unlockTTS() {
|
||||||
|
if (_ttsUnlocked || !TTS_SUPPORTED) return;
|
||||||
|
try { speechSynthesis.speak(new SpeechSynthesisUtterance("")); _ttsUnlocked = true; } catch (e) {}
|
||||||
|
}
|
||||||
|
|
||||||
|
const PLAYBACK_KEY = "va-playback"; // geräte-lokal: "device" wenn Geräte-TTS gewählt
|
||||||
|
|
||||||
|
// Setzt das Qualität-Dropdown beim Laden. Vorrang:
|
||||||
|
// 1. lokale Wahl (localStorage), 2. Server-Pref (tts_provider), 3. Mobil-Default = Gerät.
|
||||||
|
function applyPlaybackChoice(serverProvider) {
|
||||||
|
if (!ttsSel) return;
|
||||||
|
// "Gerät"-Option ausblenden, wenn der Browser keine Web Speech API hat.
|
||||||
|
if (!TTS_SUPPORTED) {
|
||||||
|
const opt = ttsSel.querySelector('option[value="device"]');
|
||||||
|
if (opt) opt.remove();
|
||||||
|
}
|
||||||
|
const local = localStorage.getItem(PLAYBACK_KEY);
|
||||||
|
if (local === "device" && TTS_SUPPORTED) {
|
||||||
|
ttsSel.value = "device";
|
||||||
|
} else if (serverProvider) {
|
||||||
|
ttsSel.value = serverProvider;
|
||||||
|
} else if (TTS_SUPPORTED && isMobile() && local === null) {
|
||||||
|
ttsSel.value = "device"; // Mobil-Default, solange nichts gewählt wurde
|
||||||
|
localStorage.setItem(PLAYBACK_KEY, "device");
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
let busy = false;
|
let busy = false;
|
||||||
|
|
@ -23,6 +133,8 @@ let activeSources = []; // laufende AudioBufferSourceNodes (stoppbar per Barge-
|
||||||
// /api/me abgeleitet; bis dahin null -> ensureSession() wartet auf loadMe().
|
// /api/me abgeleitet; bis dahin null -> ensureSession() wartet auf loadMe().
|
||||||
let sessionId = null;
|
let sessionId = null;
|
||||||
let mePromise = null;
|
let mePromise = null;
|
||||||
|
let currentUserId = null;
|
||||||
|
const sessionKey = (uid) => "va-session-" + uid;
|
||||||
|
|
||||||
// ---------- Identitaet / Menue ----------
|
// ---------- Identitaet / Menue ----------
|
||||||
function loadMe() {
|
function loadMe() {
|
||||||
|
|
@ -34,7 +146,9 @@ function loadMe() {
|
||||||
return null;
|
return null;
|
||||||
}
|
}
|
||||||
const me = await res.json();
|
const me = await res.json();
|
||||||
sessionId = "web-" + me.user_id;
|
currentUserId = me.user_id;
|
||||||
|
// Zuletzt aktive (ggf. frische) Session bevorzugen, sonst die Basis-Session.
|
||||||
|
sessionId = localStorage.getItem(sessionKey(me.user_id)) || ("web-" + me.user_id);
|
||||||
$("#identity").textContent = "Angemeldet als " + (me.display_name || me.external_id || "Gast");
|
$("#identity").textContent = "Angemeldet als " + (me.display_name || me.external_id || "Gast");
|
||||||
if (me.sso_logout_url) {
|
if (me.sso_logout_url) {
|
||||||
const logout = $("#logout");
|
const logout = $("#logout");
|
||||||
|
|
@ -44,6 +158,15 @@ function loadMe() {
|
||||||
if (me.is_admin) {
|
if (me.is_admin) {
|
||||||
$("#admin-toggle").classList.remove("hidden");
|
$("#admin-toggle").classList.remove("hidden");
|
||||||
}
|
}
|
||||||
|
// Gespeicherte Nutzer-Präferenzen in die Dropdowns laden.
|
||||||
|
const prefs = me.prefs || {};
|
||||||
|
if (langSel) {
|
||||||
|
// Flex-Modus -> "flex", sonst die feste Sprache (Fallback: Flex).
|
||||||
|
langSel.value = prefs.language_mode === "flex"
|
||||||
|
? "flex"
|
||||||
|
: (prefs.language || "flex");
|
||||||
|
}
|
||||||
|
applyPlaybackChoice(prefs.tts_provider);
|
||||||
return me;
|
return me;
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
$("#identity").textContent = "Verbindung fehlgeschlagen";
|
$("#identity").textContent = "Verbindung fehlgeschlagen";
|
||||||
|
|
@ -76,19 +199,98 @@ const BUBBLE = {
|
||||||
emergency: "mx-auto max-w-[90%] rounded-lg bg-red-100 px-3 py-1.5 text-sm font-semibold text-red-800 dark:bg-red-900/40 dark:text-red-200",
|
emergency: "mx-auto max-w-[90%] rounded-lg bg-red-100 px-3 py-1.5 text-sm font-semibold text-red-800 dark:bg-red-900/40 dark:text-red-200",
|
||||||
};
|
};
|
||||||
|
|
||||||
function addMessage(role, text) {
|
// Lautsprecher-/Stop-Icons als SVG (currentColor → per CSS einfärbbar, anders als Emoji).
|
||||||
|
const ICON_SPEAKER =
|
||||||
|
'<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" ' +
|
||||||
|
'class="w-4 h-4"><path stroke-linecap="round" stroke-linejoin="round" ' +
|
||||||
|
'd="M19.114 5.636a9 9 0 0 1 0 12.728M16.463 8.288a5.25 5.25 0 0 1 0 7.424M6.75 8.25l4.72-4.72a.75.75 0 0 1 1.28.53v15.88a.75.75 0 0 1-1.28.53l-4.72-4.72H4.51c-.88 0-1.704-.507-1.938-1.354A9.009 9.009 0 0 1 2.25 12c0-.83.112-1.633.322-2.396C2.806 8.756 3.63 8.25 4.51 8.25H6.75Z" /></svg>';
|
||||||
|
const ICON_STOP =
|
||||||
|
'<svg viewBox="0 0 24 24" fill="currentColor" class="w-4 h-4">' +
|
||||||
|
'<rect x="6" y="6" width="12" height="12" rx="2" /></svg>';
|
||||||
|
|
||||||
|
// meta.lang: Sprache, in der diese Bubble vorgelesen werden soll (für Replay).
|
||||||
|
function addMessage(role, text, meta = {}) {
|
||||||
const div = document.createElement("div");
|
const div = document.createElement("div");
|
||||||
div.className = BUBBLE[role] || BUBBLE.assistant;
|
div.className = BUBBLE[role] || BUBBLE.assistant;
|
||||||
div.textContent = text;
|
const span = document.createElement("span");
|
||||||
|
span.className = "bubble-text";
|
||||||
|
span.textContent = text;
|
||||||
|
div.appendChild(span);
|
||||||
|
div._textEl = span;
|
||||||
|
// Vorlese-Symbol nur an Nutzer-/Assistenten-Bubbles (nicht System/Notfall).
|
||||||
|
if (role === "user" || role === "assistant") {
|
||||||
|
div.classList.add("relative", "pr-7");
|
||||||
|
div._replay = { lang: meta.lang || null, chunks: null }; // chunks: gecachtes PCM (Assistent)
|
||||||
|
const btn = document.createElement("button");
|
||||||
|
btn.type = "button";
|
||||||
|
btn.innerHTML = ICON_SPEAKER;
|
||||||
|
btn.title = "Vorlesen";
|
||||||
|
btn.setAttribute("aria-label", "Vorlesen");
|
||||||
|
// Kontrastreiche Farbe je Bubble: weiß auf Blau; im Dark-Mode hell auf Dunkelgrau.
|
||||||
|
const replayColor = role === "user"
|
||||||
|
? "text-white/80 hover:text-white"
|
||||||
|
: "text-slate-500 hover:text-slate-700 dark:text-slate-300 dark:hover:text-white";
|
||||||
|
btn.className = "replay-btn absolute top-1.5 right-1.5 transition-colors " + replayColor;
|
||||||
|
btn.addEventListener("click", () => replayBubble(div));
|
||||||
|
div._replayBtn = btn;
|
||||||
|
div.appendChild(btn);
|
||||||
|
}
|
||||||
messagesEl.appendChild(div);
|
messagesEl.appendChild(div);
|
||||||
scrollToBottom();
|
scrollToBottom();
|
||||||
return div;
|
return div;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ---------- Vorlesen / Replay ----------
|
||||||
|
function setReplayPlaying(btn, on) {
|
||||||
|
if (!btn) return;
|
||||||
|
btn.innerHTML = on ? ICON_STOP : ICON_SPEAKER;
|
||||||
|
btn.dataset.playing = on ? "1" : "";
|
||||||
|
}
|
||||||
|
|
||||||
|
function startReplay(chunks, btn) {
|
||||||
|
stopAudio(); // Barge-in: laufendes Audio (auch andere Replays) stoppen
|
||||||
|
setReplayPlaying(btn, true);
|
||||||
|
playPcm(chunks, 24000, () => setReplayPlaying(btn, false));
|
||||||
|
}
|
||||||
|
|
||||||
|
async function speakBubble(text, lang, btn) {
|
||||||
|
try {
|
||||||
|
const body = { text };
|
||||||
|
if (lang) body.language = lang;
|
||||||
|
if (ttsSel && ttsSel.value) body.tts_provider = ttsSel.value;
|
||||||
|
const r = await fetch("/api/speak", {
|
||||||
|
method: "POST",
|
||||||
|
headers: { "Content-Type": "application/json" },
|
||||||
|
body: JSON.stringify(body),
|
||||||
|
});
|
||||||
|
if (!r.ok) throw new Error("HTTP " + r.status);
|
||||||
|
startReplay([await r.arrayBuffer()], btn);
|
||||||
|
} catch (e) {
|
||||||
|
setReplayPlaying(btn, false);
|
||||||
|
statusEl.textContent = "Vorlesen fehlgeschlagen";
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function replayBubble(div) {
|
||||||
|
const btn = div._replayBtn;
|
||||||
|
if (btn && btn.dataset.playing) { stopAudio(); setReplayPlaying(btn, false); return; } // Toggle
|
||||||
|
const info = div._replay || {};
|
||||||
|
const text = (div._textEl && div._textEl.textContent) || "";
|
||||||
|
if (info.chunks && info.chunks.length) {
|
||||||
|
startReplay(info.chunks, btn); // Assistent: gecachtes PCM, sofort & gratis
|
||||||
|
} else if (isDeviceMode()) {
|
||||||
|
stopAudio();
|
||||||
|
deviceSpeakBtn(text, info.lang, btn); // Geräte-TTS: lokal vorlesen, gratis
|
||||||
|
} else {
|
||||||
|
speakBubble(text, info.lang, btn); // Server-TTS via /api/speak
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// ---------- Barge-in-Hilfsfunktionen ----------
|
// ---------- Barge-in-Hilfsfunktionen ----------
|
||||||
function stopAudio() {
|
function stopAudio() {
|
||||||
activeSources.forEach((s) => { try { s.stop(); } catch (e) {} });
|
activeSources.forEach((s) => { try { s.stop(); } catch (e) {} });
|
||||||
activeSources = [];
|
activeSources = [];
|
||||||
|
if (TTS_SUPPORTED) { try { speechSynthesis.cancel(); } catch (e) {} } // Geräte-TTS stoppen
|
||||||
}
|
}
|
||||||
|
|
||||||
function setMicIdle() {
|
function setMicIdle() {
|
||||||
|
|
@ -113,8 +315,8 @@ function setMicBusy() {
|
||||||
|
|
||||||
// ---------- Audio-Wiedergabe (PCM s16le) ----------
|
// ---------- Audio-Wiedergabe (PCM s16le) ----------
|
||||||
let audioCtx = null;
|
let audioCtx = null;
|
||||||
function playPcm(chunks, sampleRate) {
|
function playPcm(chunks, sampleRate, onended) {
|
||||||
if (!chunks.length) return;
|
if (!chunks.length) { if (onended) onended(); return; }
|
||||||
const total = chunks.reduce((n, c) => n + c.byteLength, 0);
|
const total = chunks.reduce((n, c) => n + c.byteLength, 0);
|
||||||
const merged = new Uint8Array(total);
|
const merged = new Uint8Array(total);
|
||||||
let off = 0;
|
let off = 0;
|
||||||
|
|
@ -122,6 +324,7 @@ function playPcm(chunks, sampleRate) {
|
||||||
const view = new DataView(merged.buffer);
|
const view = new DataView(merged.buffer);
|
||||||
const n = Math.floor(merged.byteLength / 2);
|
const n = Math.floor(merged.byteLength / 2);
|
||||||
audioCtx = audioCtx || new (window.AudioContext || window.webkitAudioContext)();
|
audioCtx = audioCtx || new (window.AudioContext || window.webkitAudioContext)();
|
||||||
|
if (audioCtx.state === "suspended") audioCtx.resume();
|
||||||
const buf = audioCtx.createBuffer(1, n, sampleRate || 24000);
|
const buf = audioCtx.createBuffer(1, n, sampleRate || 24000);
|
||||||
const ch = buf.getChannelData(0);
|
const ch = buf.getChannelData(0);
|
||||||
for (let i = 0; i < n; i++) ch[i] = view.getInt16(i * 2, true) / 32768;
|
for (let i = 0; i < n; i++) ch[i] = view.getInt16(i * 2, true) / 32768;
|
||||||
|
|
@ -129,7 +332,10 @@ function playPcm(chunks, sampleRate) {
|
||||||
src.buffer = buf;
|
src.buffer = buf;
|
||||||
src.connect(audioCtx.destination);
|
src.connect(audioCtx.destination);
|
||||||
activeSources.push(src);
|
activeSources.push(src);
|
||||||
src.onended = () => { activeSources = activeSources.filter((s) => s !== src); };
|
src.onended = () => {
|
||||||
|
activeSources = activeSources.filter((s) => s !== src);
|
||||||
|
if (onended) onended();
|
||||||
|
};
|
||||||
src.start();
|
src.start();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -148,6 +354,10 @@ function runTurn(path, onopen) {
|
||||||
const pcm = [];
|
const pcm = [];
|
||||||
let answerEl = null;
|
let answerEl = null;
|
||||||
let sampleRate = 24000;
|
let sampleRate = 24000;
|
||||||
|
// Sprache dieses Turns (für den Replay je Bubble): Flex folgt erkannter Sprache.
|
||||||
|
let routeLang = null, routeMode = null, detectedLang = null;
|
||||||
|
const turnLang = () =>
|
||||||
|
(routeMode === "flex" ? (detectedLang || routeLang) : routeLang) || null;
|
||||||
|
|
||||||
ws.onopen = () => onopen(ws);
|
ws.onopen = () => onopen(ws);
|
||||||
ws.onerror = () => { statusEl.textContent = "Verbindungsfehler"; resolve(); };
|
ws.onerror = () => { statusEl.textContent = "Verbindungsfehler"; resolve(); };
|
||||||
|
|
@ -158,17 +368,25 @@ function runTurn(path, onopen) {
|
||||||
let msg;
|
let msg;
|
||||||
try { msg = JSON.parse(event.data); } catch { return; }
|
try { msg = JSON.parse(event.data); } catch { return; }
|
||||||
switch (msg.type) {
|
switch (msg.type) {
|
||||||
|
case "ack":
|
||||||
|
routeLang = (msg.route && msg.route.language) || null;
|
||||||
|
routeMode = (msg.route && msg.route.language_mode) || null;
|
||||||
|
break;
|
||||||
case "transcript":
|
case "transcript":
|
||||||
if (msg.text) addMessage("user", msg.text);
|
detectedLang = msg.detected_language || null;
|
||||||
|
if (msg.text) addMessage("user", msg.text, { lang: turnLang() });
|
||||||
break;
|
break;
|
||||||
case "token":
|
case "token":
|
||||||
if (!answerEl) answerEl = addMessage("assistant", "");
|
if (!answerEl) answerEl = addMessage("assistant", "", { lang: turnLang() });
|
||||||
answerEl.textContent += msg.text || "";
|
answerEl._textEl.textContent += msg.text || "";
|
||||||
scrollToBottom();
|
scrollToBottom();
|
||||||
break;
|
break;
|
||||||
case "semantic":
|
case "semantic":
|
||||||
if (!answerEl) answerEl = addMessage("assistant", "");
|
if (!answerEl) answerEl = addMessage("assistant", "", { lang: turnLang() });
|
||||||
if (msg.text) answerEl.textContent = msg.text;
|
if (msg.text) answerEl._textEl.textContent = msg.text;
|
||||||
|
// Sprache final festhalten (für Replay) und im Geräte-Modus lokal vorlesen.
|
||||||
|
if (answerEl._replay) answerEl._replay.lang = turnLang();
|
||||||
|
if (isDeviceMode() && msg.text) deviceSpeak(msg.text, turnLang());
|
||||||
scrollToBottom();
|
scrollToBottom();
|
||||||
break;
|
break;
|
||||||
case "emergency":
|
case "emergency":
|
||||||
|
|
@ -176,7 +394,11 @@ function runTurn(path, onopen) {
|
||||||
break;
|
break;
|
||||||
case "done":
|
case "done":
|
||||||
sampleRate = msg.sample_rate || 24000;
|
sampleRate = msg.sample_rate || 24000;
|
||||||
playPcm(pcm, sampleRate);
|
// Geräte-Modus: kein Server-Audio -> nichts abspielen/cachen (Gerät hat schon gesprochen).
|
||||||
|
if (!isDeviceMode()) {
|
||||||
|
if (answerEl && answerEl._replay) answerEl._replay.chunks = pcm.slice(); // Replay-Cache
|
||||||
|
playPcm(pcm, sampleRate);
|
||||||
|
}
|
||||||
statusEl.textContent = "";
|
statusEl.textContent = "";
|
||||||
scrollToBottom();
|
scrollToBottom();
|
||||||
ws.close();
|
ws.close();
|
||||||
|
|
@ -200,10 +422,13 @@ async function sendText(text) {
|
||||||
busy = true;
|
busy = true;
|
||||||
setMicBusy();
|
setMicBusy();
|
||||||
await ensureSession();
|
await ensureSession();
|
||||||
addMessage("user", text);
|
// Bei fester Sprache wird der eigene Text in dieser Sprache vorgelesen; bei Flex offen.
|
||||||
|
const lo = langOverrides();
|
||||||
|
addMessage("user", text, { lang: lo.language_mode === "fix" ? lo.language : null });
|
||||||
statusEl.textContent = "denkt …";
|
statusEl.textContent = "denkt …";
|
||||||
|
const audioStream = !isDeviceMode(); // Geräte-Modus: kein Server-Audio anfordern
|
||||||
await runTurn("/ws/chat", (ws) => {
|
await runTurn("/ws/chat", (ws) => {
|
||||||
ws.send(JSON.stringify({ text, stream: true, audio_stream: true, ...overrides() }));
|
ws.send(JSON.stringify({ text, stream: true, audio_stream: audioStream, ...overrides() }));
|
||||||
});
|
});
|
||||||
busy = false;
|
busy = false;
|
||||||
setMicIdle();
|
setMicIdle();
|
||||||
|
|
@ -211,6 +436,7 @@ async function sendText(text) {
|
||||||
|
|
||||||
formEl.addEventListener("submit", (e) => {
|
formEl.addEventListener("submit", (e) => {
|
||||||
e.preventDefault();
|
e.preventDefault();
|
||||||
|
unlockTTS(); // iOS: Geräte-TTS bei der Nutzergeste freischalten
|
||||||
const text = promptEl.value;
|
const text = promptEl.value;
|
||||||
promptEl.value = "";
|
promptEl.value = "";
|
||||||
sendText(text);
|
sendText(text);
|
||||||
|
|
@ -272,8 +498,9 @@ async function sendVoice(bytes, fmt = "webm") {
|
||||||
setMicBusy();
|
setMicBusy();
|
||||||
await ensureSession();
|
await ensureSession();
|
||||||
statusEl.textContent = "verarbeite Sprache …";
|
statusEl.textContent = "verarbeite Sprache …";
|
||||||
|
const audioStream = !isDeviceMode(); // Geräte-Modus: kein Server-Audio anfordern
|
||||||
await runTurn("/ws/voice", (ws) => {
|
await runTurn("/ws/voice", (ws) => {
|
||||||
ws.send(JSON.stringify({ type: "start", format: fmt, stream: true, audio_stream: true, ...overrides() }));
|
ws.send(JSON.stringify({ type: "start", format: fmt, stream: true, audio_stream: audioStream, ...overrides() }));
|
||||||
ws.send(bytes);
|
ws.send(bytes);
|
||||||
ws.send(JSON.stringify({ type: "end" }));
|
ws.send(JSON.stringify({ type: "end" }));
|
||||||
});
|
});
|
||||||
|
|
@ -282,6 +509,7 @@ async function sendVoice(bytes, fmt = "webm") {
|
||||||
}
|
}
|
||||||
|
|
||||||
micBtn.addEventListener("click", () => {
|
micBtn.addEventListener("click", () => {
|
||||||
|
unlockTTS(); // iOS: Geräte-TTS bei der Nutzergeste freischalten
|
||||||
if (busy) {
|
if (busy) {
|
||||||
// Barge-in: laufende KI-Antwort sofort unterbrechen
|
// Barge-in: laufende KI-Antwort sofort unterbrechen
|
||||||
if (activeWs) activeWs.send(JSON.stringify({ type: "interrupt" }));
|
if (activeWs) activeWs.send(JSON.stringify({ type: "interrupt" }));
|
||||||
|
|
@ -297,6 +525,23 @@ micBtn.addEventListener("click", () => {
|
||||||
|
|
||||||
$("#admin-toggle").addEventListener("click", openAdminPanel);
|
$("#admin-toggle").addEventListener("click", openAdminPanel);
|
||||||
|
|
||||||
|
// ---------- Neues Gespräch ----------
|
||||||
|
// Startet eine frische Session (neuer session_id) -> der bisherige Verlauf wird nicht
|
||||||
|
// mehr als LLM-Kontext genutzt. Behebt "Sprache springt nicht um", wenn die History
|
||||||
|
// in einer anderen Sprache dominiert. Nicht-destruktiv: alte Session bleibt in der DB.
|
||||||
|
function newConversation() {
|
||||||
|
const base = currentUserId || "anon";
|
||||||
|
sessionId = "web-" + base + "-" + Date.now();
|
||||||
|
if (currentUserId) localStorage.setItem(sessionKey(currentUserId), sessionId);
|
||||||
|
stopAudio();
|
||||||
|
messagesEl.innerHTML = "";
|
||||||
|
statusEl.textContent = "Neues Gespräch begonnen";
|
||||||
|
setTimeout(() => {
|
||||||
|
if (statusEl.textContent === "Neues Gespräch begonnen") statusEl.textContent = "";
|
||||||
|
}, 2500);
|
||||||
|
}
|
||||||
|
$("#new-chat").addEventListener("click", newConversation);
|
||||||
|
|
||||||
// Tag/Nacht: Umschalter (manuell) + automatisch dem System folgen, solange nichts gewählt.
|
// Tag/Nacht: Umschalter (manuell) + automatisch dem System folgen, solange nichts gewählt.
|
||||||
const themeBtn = $("#theme-toggle");
|
const themeBtn = $("#theme-toggle");
|
||||||
const setThemeIcon = () =>
|
const setThemeIcon = () =>
|
||||||
|
|
@ -321,6 +566,21 @@ if (window.visualViewport) {
|
||||||
window.addEventListener("resize", scrollToBottom);
|
window.addEventListener("resize", scrollToBottom);
|
||||||
promptEl.addEventListener("focus", () => setTimeout(scrollToBottom, 300));
|
promptEl.addEventListener("focus", () => setTimeout(scrollToBottom, 300));
|
||||||
|
|
||||||
|
// Dropdown-Änderungen dauerhaft als Nutzer-Präferenz speichern.
|
||||||
|
if (langSel) langSel.addEventListener("change", () => savePrefs(langOverrides()));
|
||||||
|
if (ttsSel) {
|
||||||
|
ttsSel.addEventListener("change", () => {
|
||||||
|
unlockTTS(); // Auswahl ist eine Nutzergeste -> Geräte-TTS gleich freischalten
|
||||||
|
if (ttsSel.value === "device") {
|
||||||
|
// Geräte-TTS ist geräte-lokal (nicht server-seitig) -> nur lokal merken.
|
||||||
|
localStorage.setItem(PLAYBACK_KEY, "device");
|
||||||
|
} else {
|
||||||
|
localStorage.removeItem(PLAYBACK_KEY);
|
||||||
|
savePrefs({ tts_provider: ttsSel.value || null }); // echter Provider -> Server-Pref
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
loadMe();
|
loadMe();
|
||||||
|
|
||||||
// ═══════════════════════════════════════════════════════════════
|
// ═══════════════════════════════════════════════════════════════
|
||||||
|
|
@ -854,13 +1114,14 @@ function buildWortSection(title, section, entries, lang) {
|
||||||
wrap.className = "bg-white dark:bg-slate-800 rounded-xl border border-slate-200 dark:border-slate-700 p-4";
|
wrap.className = "bg-white dark:bg-slate-800 rounded-xl border border-slate-200 dark:border-slate-700 p-4";
|
||||||
|
|
||||||
const rows = Object.entries(entries).map(([k, v]) => `
|
const rows = Object.entries(entries).map(([k, v]) => `
|
||||||
<tr class="hover:bg-slate-50 dark:hover:bg-slate-700/40 transition-colors group">
|
<tr class="row-edit cursor-pointer hover:bg-slate-50 dark:hover:bg-slate-700/40 transition-colors group"
|
||||||
|
data-key="${escHtml(k)}" data-val="${escHtml(v)}" title="Zum Bearbeiten anklicken">
|
||||||
<td class="px-3 py-2 text-xs font-mono text-slate-700 dark:text-slate-300">${escHtml(k)}</td>
|
<td class="px-3 py-2 text-xs font-mono text-slate-700 dark:text-slate-300">${escHtml(k)}</td>
|
||||||
<td class="px-3 py-2 text-xs text-slate-400">→</td>
|
<td class="px-3 py-2 text-xs text-slate-400">→</td>
|
||||||
<td class="px-3 py-2 text-xs text-slate-700 dark:text-slate-300 font-medium">${escHtml(v)}</td>
|
<td class="px-3 py-2 text-xs text-slate-700 dark:text-slate-300 font-medium">${escHtml(v)}</td>
|
||||||
<td class="px-3 py-2 text-right">
|
<td class="px-3 py-2 text-right">
|
||||||
<button class="del-btn text-slate-300 dark:text-slate-600 hover:text-red-500 dark:hover:text-red-400 transition-colors text-xs opacity-0 group-hover:opacity-100"
|
<button class="del-btn text-slate-300 dark:text-slate-600 hover:text-red-500 dark:hover:text-red-400 transition-colors text-xs opacity-0 group-hover:opacity-100"
|
||||||
data-key="${escHtml(k)}">✕</button>
|
data-key="${escHtml(k)}" title="Löschen">✕</button>
|
||||||
</td>
|
</td>
|
||||||
</tr>`).join("");
|
</tr>`).join("");
|
||||||
|
|
||||||
|
|
@ -885,8 +1146,24 @@ function buildWortSection(title, section, entries, lang) {
|
||||||
</form>
|
</form>
|
||||||
`;
|
`;
|
||||||
|
|
||||||
|
const form = wrap.querySelector(".add-form");
|
||||||
|
const [keyInp, valInp] = form.querySelectorAll("input");
|
||||||
|
const submitBtn = form.querySelector('button[type="submit"]');
|
||||||
|
|
||||||
|
// Zeile anklicken -> Begriffspaar in die Editier-Felder laden, Button wird zu „Speichern".
|
||||||
|
wrap.querySelectorAll("tr.row-edit").forEach((tr) =>
|
||||||
|
tr.addEventListener("click", () => {
|
||||||
|
keyInp.value = tr.dataset.key;
|
||||||
|
valInp.value = tr.dataset.val;
|
||||||
|
form.dataset.editKey = tr.dataset.key; // Originalschlüssel merken (für Umbenennen)
|
||||||
|
submitBtn.textContent = "Speichern";
|
||||||
|
valInp.focus();
|
||||||
|
})
|
||||||
|
);
|
||||||
|
|
||||||
wrap.querySelectorAll(".del-btn").forEach((btn) =>
|
wrap.querySelectorAll(".del-btn").forEach((btn) =>
|
||||||
btn.addEventListener("click", async () => {
|
btn.addEventListener("click", async (e) => {
|
||||||
|
e.stopPropagation(); // nicht zugleich die Zeile zum Bearbeiten laden
|
||||||
const key = btn.dataset.key;
|
const key = btn.dataset.key;
|
||||||
if (!confirm(`Eintrag „${key}" löschen?`)) return;
|
if (!confirm(`Eintrag „${key}" löschen?`)) return;
|
||||||
const res = await adminFetch(
|
const res = await adminFetch(
|
||||||
|
|
@ -896,14 +1173,22 @@ function buildWortSection(title, section, entries, lang) {
|
||||||
})
|
})
|
||||||
);
|
);
|
||||||
|
|
||||||
wrap.querySelector(".add-form").addEventListener("submit", async (e) => {
|
form.addEventListener("submit", async (e) => {
|
||||||
e.preventDefault();
|
e.preventDefault();
|
||||||
const [keyInp, valInp] = e.target.querySelectorAll("input");
|
|
||||||
const key = keyInp.value.trim();
|
const key = keyInp.value.trim();
|
||||||
const value = valInp.value.trim();
|
const value = valInp.value.trim();
|
||||||
if (!key || !value) return;
|
if (!key || !value) return;
|
||||||
|
const editKey = form.dataset.editKey || "";
|
||||||
|
// Hinzufügen ODER Aktualisieren (gleicher Schlüssel wird überschrieben; Backend sortiert).
|
||||||
const res = await adminFetch(`/api/admin/pronunciation/${lang}`, "POST", { section, key, value });
|
const res = await adminFetch(`/api/admin/pronunciation/${lang}`, "POST", { section, key, value });
|
||||||
if (res !== null) { keyInp.value = ""; valInp.value = ""; loadWoerterbuch(); }
|
if (res === null) return;
|
||||||
|
// Umbenannt -> alten Schlüssel entfernen (neuer wurde oben bereits angelegt).
|
||||||
|
if (editKey && editKey !== key) {
|
||||||
|
await adminFetch(
|
||||||
|
`/api/admin/pronunciation/${lang}/${section}/${encodeURIComponent(editKey)}`, "DELETE"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
loadWoerterbuch(); // neu laden -> sortierte Liste, Formular zurück auf „+ Hinzufügen"
|
||||||
});
|
});
|
||||||
|
|
||||||
return wrap;
|
return wrap;
|
||||||
|
|
|
||||||
BIN
app/web/apple-touch-icon.png
Normal file
BIN
app/web/apple-touch-icon.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 22 KiB |
BIN
app/web/favicon-32.png
Normal file
BIN
app/web/favicon-32.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 2.4 KiB |
3
app/web/favicon.svg
Normal file
3
app/web/favicon.svg
Normal file
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 44 KiB |
|
|
@ -4,8 +4,17 @@
|
||||||
<meta charset="utf-8" />
|
<meta charset="utf-8" />
|
||||||
<meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover" />
|
<meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover" />
|
||||||
<title>Voice Assistant</title>
|
<title>Voice Assistant</title>
|
||||||
|
<link rel="icon" type="image/svg+xml" href="/favicon.svg?v=1" />
|
||||||
|
<link rel="icon" type="image/png" sizes="32x32" href="/favicon-32.png?v=1" />
|
||||||
|
<link rel="apple-touch-icon" href="/apple-touch-icon.png?v=1" />
|
||||||
<script src="https://cdn.tailwindcss.com"></script>
|
<script src="https://cdn.tailwindcss.com"></script>
|
||||||
<script>tailwind.config = { darkMode: "class" };</script>
|
<script>tailwind.config = { darkMode: "class" };</script>
|
||||||
|
<style>
|
||||||
|
/* Dropdown-Popup (<option>) hat sonst Browser-Default-Weiß -> im Dark-Mode
|
||||||
|
helle Schrift auf hellem Grund = unlesbar. Farben explizit setzen. */
|
||||||
|
select option { background-color: #ffffff; color: #1e293b; } /* slate-800 */
|
||||||
|
.dark select option { background-color: #1e293b; color: #f1f5f9; } /* slate-100 */
|
||||||
|
</style>
|
||||||
<script>
|
<script>
|
||||||
(function () {
|
(function () {
|
||||||
try {
|
try {
|
||||||
|
|
@ -137,6 +146,12 @@
|
||||||
class="text-sm rounded-lg border border-slate-300 dark:border-slate-600 bg-transparent px-2 py-1.5">
|
class="text-sm rounded-lg border border-slate-300 dark:border-slate-600 bg-transparent px-2 py-1.5">
|
||||||
<option value="de">Deutsch (de)</option>
|
<option value="de">Deutsch (de)</option>
|
||||||
<option value="en">Englisch (en)</option>
|
<option value="en">Englisch (en)</option>
|
||||||
|
<option value="fr">Französisch (fr)</option>
|
||||||
|
<option value="es">Spanisch (es)</option>
|
||||||
|
<option value="it">Italienisch (it)</option>
|
||||||
|
<option value="nl">Niederländisch (nl)</option>
|
||||||
|
<option value="ru">Russisch (ru)</option>
|
||||||
|
<option value="zh">Chinesisch (zh)</option>
|
||||||
</select>
|
</select>
|
||||||
<button id="load-words"
|
<button id="load-words"
|
||||||
class="rounded-lg border border-slate-300 dark:border-slate-600 text-sm px-3 py-1.5 hover:bg-slate-100 dark:hover:bg-slate-700 transition-colors">Aktualisieren</button>
|
class="rounded-lg border border-slate-300 dark:border-slate-600 text-sm px-3 py-1.5 hover:bg-slate-100 dark:hover:bg-slate-700 transition-colors">Aktualisieren</button>
|
||||||
|
|
@ -185,14 +200,31 @@
|
||||||
|
|
||||||
|
|
||||||
<!-- ═══════════════════ CHAT INTERFACE ═══════════════════ -->
|
<!-- ═══════════════════ CHAT INTERFACE ═══════════════════ -->
|
||||||
<header class="shrink-0 h-14 flex items-center gap-3 px-4 border-b border-slate-200 dark:border-slate-700 bg-white/80 dark:bg-slate-800/80 backdrop-blur">
|
<header class="shrink-0 h-14 flex items-center gap-2 px-3 border-b border-slate-200 dark:border-slate-700 bg-white/80 dark:bg-slate-800/80 backdrop-blur">
|
||||||
<h1 class="flex-1 truncate font-semibold text-base">Voice Assistant</h1>
|
<h1 class="flex-1 truncate font-semibold text-base">Voice Assistant</h1>
|
||||||
<select id="tts" title="Stimme"
|
<select id="lang-sel" title="Sprache — feste Sprache (Eingabe wird übersetzt) oder 🔄 Flex (folgt automatisch der gesprochenen Sprache)"
|
||||||
class="text-sm rounded-lg border border-slate-300 dark:border-slate-600 bg-transparent px-2 py-1.5">
|
class="text-sm rounded-lg border border-slate-300 dark:border-slate-600 bg-transparent px-2 py-1.5">
|
||||||
<option value="">Schnell</option>
|
<option value="flex">🔄 Flex</option>
|
||||||
<option value="chatterbox">Hohe Qualität</option>
|
<option value="de">🇩🇪 DE</option>
|
||||||
|
<option value="en">🇬🇧 EN</option>
|
||||||
|
<option value="fr">🇫🇷 FR</option>
|
||||||
|
<option value="es">🇪🇸 ES</option>
|
||||||
|
<option value="it">🇮🇹 IT</option>
|
||||||
|
<option value="nl">🇳🇱 NL</option>
|
||||||
|
<option value="ru">🇷🇺 RU</option>
|
||||||
|
<option value="zh">🇨🇳 ZH</option>
|
||||||
|
</select>
|
||||||
|
<select id="tts" title="Sprachqualität"
|
||||||
|
class="text-sm rounded-lg border border-slate-300 dark:border-slate-600 bg-transparent px-2 py-1.5">
|
||||||
|
<option value="device">📱 Gerät</option>
|
||||||
|
<option value="piper">Schnell</option>
|
||||||
|
<option value="chatterbox">Hoch</option>
|
||||||
<option value="openrouter">Cloud</option>
|
<option value="openrouter">Cloud</option>
|
||||||
</select>
|
</select>
|
||||||
|
<button id="new-chat" title="Neues Gespräch (Verlauf zurücksetzen)"
|
||||||
|
class="h-9 w-9 grid place-items-center rounded-lg border border-slate-300 dark:border-slate-600 hover:bg-slate-100 dark:hover:bg-slate-700 transition-colors">
|
||||||
|
<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.7" class="w-5 h-5"><path stroke-linecap="round" stroke-linejoin="round" d="M16.862 4.487l1.687-1.688a1.875 1.875 0 1 1 2.652 2.652L10.582 16.07a4.5 4.5 0 0 1-1.897 1.13L6 18l.8-2.685a4.5 4.5 0 0 1 1.13-1.897l8.932-8.931Zm0 0L19.5 7.125M18 14v4.75A2.25 2.25 0 0 1 15.75 21H5.25A2.25 2.25 0 0 1 3 18.75V8.25A2.25 2.25 0 0 1 5.25 6H10"/></svg>
|
||||||
|
</button>
|
||||||
<button id="admin-toggle" title="Admin"
|
<button id="admin-toggle" title="Admin"
|
||||||
class="hidden h-9 w-9 grid place-items-center rounded-lg border border-slate-300 dark:border-slate-600 hover:bg-slate-100 dark:hover:bg-slate-700 transition-colors">⚙️</button>
|
class="hidden h-9 w-9 grid place-items-center rounded-lg border border-slate-300 dark:border-slate-600 hover:bg-slate-100 dark:hover:bg-slate-700 transition-colors">⚙️</button>
|
||||||
<button id="theme-toggle" title="Tag/Nacht"
|
<button id="theme-toggle" title="Tag/Nacht"
|
||||||
|
|
@ -218,6 +250,6 @@
|
||||||
<div id="status" class="w-full max-w-3xl mx-auto mt-1 min-h-[1rem] text-xs text-slate-500 dark:text-slate-400"></div>
|
<div id="status" class="w-full max-w-3xl mx-auto mt-1 min-h-[1rem] text-xs text-slate-500 dark:text-slate-400"></div>
|
||||||
</footer>
|
</footer>
|
||||||
|
|
||||||
<script src="/app.js?v=19"></script>
|
<script src="/app.js?v=27"></script>
|
||||||
</body>
|
</body>
|
||||||
</html>
|
</html>
|
||||||
|
|
|
||||||
|
|
@ -1,34 +1,17 @@
|
||||||
# Aussprache-Lexikon (Deutsch) für die TTS-Normalisierung vor Piper.
|
abbreviations: {}
|
||||||
#
|
units: {}
|
||||||
# Dies ist das GRUNDGERÜST – Inhalte werden später schrittweise ergänzt.
|
|
||||||
# Einträge hier ERWEITERN/ÜBERSCHREIBEN die eingebauten Defaults des Normalizers.
|
|
||||||
# Wirkt nur bei lokalem TTS (piper, Stufe "full"); Cloud-TTS bleibt unangetastet.
|
|
||||||
#
|
|
||||||
# Format:
|
|
||||||
# abbreviations: ganze Token werden wortgrenzen-sicher ersetzt (Punkt mitschreiben!)
|
|
||||||
# units: werden nur DIREKT NACH EINER ZAHL ersetzt (z. B. "10 kg")
|
|
||||||
# terms: Fachwörter / Eigennamen / englische Begriffe
|
|
||||||
#
|
|
||||||
# Beispiele (auskommentiert – bei Bedarf aktivieren/ergänzen):
|
|
||||||
|
|
||||||
abbreviations:
|
|
||||||
# "ggf.": "gegebenenfalls"
|
|
||||||
# "inkl.": "inklusive"
|
|
||||||
|
|
||||||
units:
|
|
||||||
# "kWh": "Kilowattstunden"
|
|
||||||
|
|
||||||
# Aussprache-Korrekturen: espeak-ng (in Piper) verkürzt bei einigen Wortformen
|
|
||||||
# fälschlich einen langen Vokal. Umschreibung mit "h" -> langer Vokal.
|
|
||||||
# Alle Einträge mit `espeak-ng -v de -x "<wort>"` verifiziert (Y:/E:/o: = lang).
|
|
||||||
# Matching ist case-insensitiv (deckt auch Satzanfänge ab).
|
|
||||||
terms:
|
terms:
|
||||||
"strömt": "ströhmt"
|
bläst: blähst
|
||||||
"strömte": "ströhmte"
|
büßt: bühßt
|
||||||
"löst": "löhst"
|
Chat: Tschätt
|
||||||
"tönt": "töhnt"
|
Dornröschen: Dornröhß-chen
|
||||||
"bläst": "blähst"
|
Glycerin: Glüzeriehn
|
||||||
"büßt": "bühßt"
|
grüßt: grühßt
|
||||||
"grüßt": "grühßt"
|
Iljitsch: Illjitsch
|
||||||
"Mond": "Mohnd"
|
löst: löhst
|
||||||
# "Linux": "Linuks" # Beispiel für Eigennamen/IT-Begriffe
|
Mond: Mohnd
|
||||||
|
Piotr: Pjottr
|
||||||
|
strömt: ströhmt
|
||||||
|
strömte: ströhmte
|
||||||
|
Tchaikovsky: Tschai'kowski
|
||||||
|
tönt: töhnt
|
||||||
|
|
|
||||||
|
|
@ -1,17 +1,5 @@
|
||||||
# Pronunciation lexicon (English) for TTS normalization before piper.
|
abbreviations: {}
|
||||||
# Grundgerüst – entries here extend/override the normalizer's built-in defaults.
|
units: {}
|
||||||
# Only applied for local TTS (piper, level "full").
|
|
||||||
|
|
||||||
abbreviations:
|
|
||||||
# "approx.": "approximately"
|
|
||||||
|
|
||||||
units:
|
|
||||||
# "kWh": "kilowatt hours"
|
|
||||||
|
|
||||||
terms:
|
terms:
|
||||||
# "SQL": "sequel"
|
Dieter: Deeter
|
||||||
# Name des Systembetreibers: phonetische Anpassung für englische Aussprache.
|
Schlüter: Shleeter
|
||||||
# "ie" in EN → /aɪ/ ("die"); "Deeter" ergibt /ˈdiːtər/.
|
|
||||||
# "Schlueter" = eingedeutschte Schreibung, EN-espeak: /ˈʃluːtər/.
|
|
||||||
"Dieter": "Deeter"
|
|
||||||
"Schlüter": "Schlueter"
|
|
||||||
|
|
|
||||||
|
|
@ -1,16 +1,5 @@
|
||||||
# Lexique de prononciation (Français) pour la normalisation TTS avant Piper.
|
abbreviations: {}
|
||||||
# Format identique à pronunciation.de.yaml.
|
units: {}
|
||||||
# Wirkt nur bei lokalem TTS (piper, Stufe "full").
|
|
||||||
|
|
||||||
abbreviations:
|
|
||||||
# "etc.": "et cetera"
|
|
||||||
|
|
||||||
units:
|
|
||||||
# "km": "kilomètres"
|
|
||||||
|
|
||||||
terms:
|
terms:
|
||||||
# Nom du responsable système — approximation phonétique pour espeak-ng fr.
|
Dieter: Diter
|
||||||
# FR: "ch"=/ʃ/, "u"=/y/ (= Umlaut ü!), "eur"=/œʁ/ → /ʃlytœʁ/ ≈ allemand /ʃlyːtɐ/.
|
Schlüter: Chluteur
|
||||||
"Schlüter": "Chluteur"
|
|
||||||
# "Dieter" → FR espeak donne /djɛtɛʁ/, acceptable; décommenter si trop éloigné.
|
|
||||||
# "Dieter": "Diéter"
|
|
||||||
|
|
|
||||||
37
config/voices/README.md
Normal file
37
config/voices/README.md
Normal file
|
|
@ -0,0 +1,37 @@
|
||||||
|
# Native Referenz-Stimmen (Chatterbox cross-lingual)
|
||||||
|
|
||||||
|
Je Sprache eine kurze Referenz-WAV (~6–8 s). Der Chatterbox-TTS-Provider wählt
|
||||||
|
sie automatisch nach der Antwortsprache aus (Konvention `<lang>.wav`, siehe
|
||||||
|
`CHATTERBOX_VOICES_DIR`). Das Timbre der Referenz wird cross-lingual auf die
|
||||||
|
jeweilige Sprache übertragen. Fehlt eine Datei, greift `CHATTERBOX_VOICE`
|
||||||
|
(persönlicher Klon) bzw. die Standardstimme des Dienstes.
|
||||||
|
|
||||||
|
Deutsch (`de`) hat bewusst **keine** Datei hier — dafür gilt der persönliche
|
||||||
|
Klon aus `CHATTERBOX_VOICE`.
|
||||||
|
|
||||||
|
## Quelle & Lizenz
|
||||||
|
|
||||||
|
Die Clips stammen aus dem **FLEURS**-Datensatz (Google), Split `validation`:
|
||||||
|
|
||||||
|
| Datei | FLEURS-Config | Sprache |
|
||||||
|
|-------|---------------|---------|
|
||||||
|
| en.wav | en_us | Englisch (US) |
|
||||||
|
| fr.wav | fr_fr | Französisch |
|
||||||
|
| es.wav | es_419 | Spanisch (Lateinamerika) |
|
||||||
|
| it.wav | it_it | Italienisch |
|
||||||
|
| nl.wav | nl_nl | Niederländisch |
|
||||||
|
| ru.wav | ru_ru | Russisch |
|
||||||
|
| zh.wav | cmn_hans_cn | Mandarin-Chinesisch |
|
||||||
|
|
||||||
|
Die Clips wurden auf einheitliche Lautheit normalisiert (EBU R128, `loudnorm`
|
||||||
|
I=-16 LUFS, TP=-1.5 dB) — die FLEURS-Originalpegel waren stark uneinheitlich
|
||||||
|
(z. B. ru/en/nl deutlich zu leise). Nur Pegelanpassung, kein inhaltlicher Eingriff.
|
||||||
|
|
||||||
|
**Lizenz: CC-BY 4.0** — Namensnennung erforderlich.
|
||||||
|
|
||||||
|
> FLEURS: Conneau et al., "FLEURS: Few-shot Learning Evaluation of Universal
|
||||||
|
> Representations of Speech" (Google Research). Datensatz: `google/fleurs`
|
||||||
|
> auf Hugging Face. Lizenz: Creative Commons Attribution 4.0 (CC-BY 4.0).
|
||||||
|
|
||||||
|
Zum Austauschen einfach eine andere `<lang>.wav` ablegen (Pfad muss für den
|
||||||
|
Chatterbox-Dienst lesbar sein; der Provider löst ihn absolut auf).
|
||||||
BIN
config/voices/en.wav
Normal file
BIN
config/voices/en.wav
Normal file
Binary file not shown.
BIN
config/voices/es.wav
Normal file
BIN
config/voices/es.wav
Normal file
Binary file not shown.
BIN
config/voices/fr.wav
Normal file
BIN
config/voices/fr.wav
Normal file
Binary file not shown.
BIN
config/voices/it.wav
Normal file
BIN
config/voices/it.wav
Normal file
Binary file not shown.
BIN
config/voices/nl.wav
Normal file
BIN
config/voices/nl.wav
Normal file
Binary file not shown.
BIN
config/voices/ru.wav
Normal file
BIN
config/voices/ru.wav
Normal file
Binary file not shown.
BIN
config/voices/zh.wav
Normal file
BIN
config/voices/zh.wav
Normal file
Binary file not shown.
|
|
@ -70,7 +70,7 @@ def test_user_prefs_applied_to_route(monkeypatch):
|
||||||
_enable_auth(monkeypatch)
|
_enable_auth(monkeypatch)
|
||||||
|
|
||||||
class StubTTS:
|
class StubTTS:
|
||||||
async def synthesize(self, text, voice=None, audio_format="pcm"):
|
async def synthesize(self, text, voice=None, audio_format="pcm", language=None):
|
||||||
return b""
|
return b""
|
||||||
monkeypatch.setitem(deps.TTS_REGISTRY, "stub-tts", lambda s: StubTTS())
|
monkeypatch.setitem(deps.TTS_REGISTRY, "stub-tts", lambda s: StubTTS())
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -77,6 +77,15 @@ def test_empty_text_raises(monkeypatch):
|
||||||
|
|
||||||
def test_ref_voice_selection():
|
def test_ref_voice_selection():
|
||||||
p = ChatterboxTTSProvider("http://x:9999", voice="/default.wav")
|
p = ChatterboxTTSProvider("http://x:9999", voice="/default.wav")
|
||||||
assert p._ref_voice(None) == "/default.wav"
|
assert p._ref_voice(None, "de") == "/default.wav"
|
||||||
assert p._ref_voice("Zephyr") == "/default.wav" # Cloud-Name -> Default
|
assert p._ref_voice("Zephyr", "de") == "/default.wav" # Cloud-Name -> Default
|
||||||
assert p._ref_voice("/custom.wav") == "/custom.wav" # Pfad -> uebernommen
|
assert p._ref_voice("/custom.wav", "de") == "/custom.wav" # Pfad -> uebernommen
|
||||||
|
|
||||||
|
|
||||||
|
def test_ref_voice_per_language(tmp_path):
|
||||||
|
# Native Sprach-Referenz gewinnt über die Default-Stimme, fehlt sie -> Default.
|
||||||
|
(tmp_path / "fr.wav").write_bytes(b"RIFF")
|
||||||
|
p = ChatterboxTTSProvider("http://x:9999", voice="/default.wav", voices_dir=str(tmp_path))
|
||||||
|
assert p._ref_voice(None, "fr") == str(tmp_path / "fr.wav") # native fr-Stimme
|
||||||
|
assert p._ref_voice(None, "de") == "/default.wav" # keine de.wav -> Default
|
||||||
|
assert p._ref_voice("/custom.wav", "fr") == "/custom.wav" # expliziter Pfad gewinnt
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@ client = TestClient(app)
|
||||||
def _stub_tts(monkeypatch, name="stub-tts", audio=b""):
|
def _stub_tts(monkeypatch, name="stub-tts", audio=b""):
|
||||||
"""Registriert einen deterministischen TTS-Stub (kein Netz, kein lokales Binary)."""
|
"""Registriert einen deterministischen TTS-Stub (kein Netz, kein lokales Binary)."""
|
||||||
class StubTTS:
|
class StubTTS:
|
||||||
async def synthesize(self, text, voice=None, audio_format="pcm"):
|
async def synthesize(self, text, voice=None, audio_format="pcm", language=None):
|
||||||
return audio
|
return audio
|
||||||
monkeypatch.setitem(deps.TTS_REGISTRY, name, lambda s: StubTTS())
|
monkeypatch.setitem(deps.TTS_REGISTRY, name, lambda s: StubTTS())
|
||||||
return name
|
return name
|
||||||
|
|
@ -63,7 +63,7 @@ def test_chat_per_request_override_and_loopback(monkeypatch):
|
||||||
return "Mir geht es gut, danke."
|
return "Mir geht es gut, danke."
|
||||||
|
|
||||||
class StubTTS:
|
class StubTTS:
|
||||||
async def synthesize(self, text, voice=None, audio_format="pcm"):
|
async def synthesize(self, text, voice=None, audio_format="pcm", language=None):
|
||||||
return b"AUDIO"
|
return b"AUDIO"
|
||||||
|
|
||||||
monkeypatch.setitem(deps.LLM_REGISTRY, "stub", lambda s: StubLLM())
|
monkeypatch.setitem(deps.LLM_REGISTRY, "stub", lambda s: StubLLM())
|
||||||
|
|
|
||||||
|
|
@ -72,6 +72,37 @@ def test_admin_users_endpoint_gated(forward_auth):
|
||||||
|
|
||||||
|
|
||||||
def test_static_index_served():
|
def test_static_index_served():
|
||||||
|
# Auth aus (LAN-Dev) -> anonymer Nutzer -> Seite wird ausgeliefert.
|
||||||
r = client.get("/")
|
r = client.get("/")
|
||||||
assert r.status_code == 200
|
assert r.status_code == 200
|
||||||
assert "Voice Assistant" in r.text
|
assert "Voice Assistant" in r.text
|
||||||
|
|
||||||
|
|
||||||
|
def test_static_index_gated_without_auth(monkeypatch):
|
||||||
|
# Auth an, keine Identitaet -> Seite ist gesperrt (kein Login-URL -> 401).
|
||||||
|
monkeypatch.setattr(settings, "auth_enabled", True)
|
||||||
|
monkeypatch.setattr(settings, "trusted_auth_header", "")
|
||||||
|
monkeypatch.setattr(settings, "trusted_auth_cookie", "")
|
||||||
|
monkeypatch.setattr(settings, "sso_login_url", "")
|
||||||
|
r = client.get("/", follow_redirects=False)
|
||||||
|
assert r.status_code == 401
|
||||||
|
|
||||||
|
|
||||||
|
def test_static_index_redirects_to_sso(monkeypatch):
|
||||||
|
# Auth an, keine Identitaet, Login-URL gesetzt -> Redirect zum SSO-Portal.
|
||||||
|
monkeypatch.setattr(settings, "auth_enabled", True)
|
||||||
|
monkeypatch.setattr(settings, "trusted_auth_header", "")
|
||||||
|
monkeypatch.setattr(settings, "trusted_auth_cookie", "")
|
||||||
|
monkeypatch.setattr(settings, "sso_login_url", "https://linix.de/yunohost/sso/")
|
||||||
|
r = client.get("/", follow_redirects=False)
|
||||||
|
assert r.status_code == 302
|
||||||
|
assert r.headers["location"].startswith("https://linix.de/yunohost/sso/")
|
||||||
|
|
||||||
|
|
||||||
|
def test_health_open_without_auth(monkeypatch):
|
||||||
|
# Health-Check bleibt auch bei aktiver Auth ohne Identitaet erreichbar.
|
||||||
|
monkeypatch.setattr(settings, "auth_enabled", True)
|
||||||
|
monkeypatch.setattr(settings, "trusted_auth_header", "")
|
||||||
|
monkeypatch.setattr(settings, "trusted_auth_cookie", "")
|
||||||
|
r = client.get("/health")
|
||||||
|
assert r.status_code == 200
|
||||||
|
|
|
||||||
|
|
@ -13,7 +13,7 @@ def _install_stubs(monkeypatch, captured):
|
||||||
return f"Antwort auf: {text}"
|
return f"Antwort auf: {text}"
|
||||||
|
|
||||||
class StubTTS:
|
class StubTTS:
|
||||||
async def synthesize(self, text, voice=None, audio_format="pcm"):
|
async def synthesize(self, text, voice=None, audio_format="pcm", language=None):
|
||||||
return b"AUDIO"
|
return b"AUDIO"
|
||||||
|
|
||||||
monkeypatch.setitem(deps.LLM_REGISTRY, "mem", lambda s: MemLLM())
|
monkeypatch.setitem(deps.LLM_REGISTRY, "mem", lambda s: MemLLM())
|
||||||
|
|
|
||||||
|
|
@ -14,7 +14,7 @@ def _stub_chat(monkeypatch, answer="ok"):
|
||||||
return answer
|
return answer
|
||||||
|
|
||||||
class StubTTS:
|
class StubTTS:
|
||||||
async def synthesize(self, text, voice=None, audio_format="pcm"):
|
async def synthesize(self, text, voice=None, audio_format="pcm", language=None):
|
||||||
return b"A"
|
return b"A"
|
||||||
|
|
||||||
monkeypatch.setitem(deps.LLM_REGISTRY, "l", lambda s: StubLLM())
|
monkeypatch.setitem(deps.LLM_REGISTRY, "l", lambda s: StubLLM())
|
||||||
|
|
|
||||||
|
|
@ -51,7 +51,7 @@ def _install_slow_stream(monkeypatch):
|
||||||
yield f"t{i} "
|
yield f"t{i} "
|
||||||
|
|
||||||
class StubTTS:
|
class StubTTS:
|
||||||
async def synthesize(self, text, voice=None, audio_format="pcm"):
|
async def synthesize(self, text, voice=None, audio_format="pcm", language=None):
|
||||||
return b"A"
|
return b"A"
|
||||||
|
|
||||||
monkeypatch.setitem(deps.LLM_REGISTRY, "slow", lambda s: SlowLLM())
|
monkeypatch.setitem(deps.LLM_REGISTRY, "slow", lambda s: SlowLLM())
|
||||||
|
|
@ -93,7 +93,7 @@ def _install_voice_stubs(monkeypatch):
|
||||||
return "antwort"
|
return "antwort"
|
||||||
|
|
||||||
class TTS:
|
class TTS:
|
||||||
async def synthesize(self, text, voice=None, audio_format="pcm"):
|
async def synthesize(self, text, voice=None, audio_format="pcm", language=None):
|
||||||
return b"A"
|
return b"A"
|
||||||
|
|
||||||
monkeypatch.setitem(deps.STT_REGISTRY, "s", lambda x: STT())
|
monkeypatch.setitem(deps.STT_REGISTRY, "s", lambda x: STT())
|
||||||
|
|
|
||||||
|
|
@ -82,7 +82,7 @@ def test_config_llm_fallback_applied(monkeypatch):
|
||||||
return "rescued"
|
return "rescued"
|
||||||
|
|
||||||
class StubTTS:
|
class StubTTS:
|
||||||
async def synthesize(self, text, voice=None, audio_format="pcm"):
|
async def synthesize(self, text, voice=None, audio_format="pcm", language=None):
|
||||||
return b"A"
|
return b"A"
|
||||||
|
|
||||||
monkeypatch.setitem(deps.LLM_REGISTRY, "bad", lambda s: BadLLM())
|
monkeypatch.setitem(deps.LLM_REGISTRY, "bad", lambda s: BadLLM())
|
||||||
|
|
@ -106,7 +106,7 @@ def test_metrics_endpoint_records_requests_and_stages(monkeypatch):
|
||||||
return "hi"
|
return "hi"
|
||||||
|
|
||||||
class StubTTS:
|
class StubTTS:
|
||||||
async def synthesize(self, text, voice=None, audio_format="pcm"):
|
async def synthesize(self, text, voice=None, audio_format="pcm", language=None):
|
||||||
return b"A"
|
return b"A"
|
||||||
|
|
||||||
monkeypatch.setitem(deps.LLM_REGISTRY, "l", lambda s: StubLLM())
|
monkeypatch.setitem(deps.LLM_REGISTRY, "l", lambda s: StubLLM())
|
||||||
|
|
|
||||||
|
|
@ -59,7 +59,7 @@ def _install_streaming(monkeypatch, tokens):
|
||||||
yield tok
|
yield tok
|
||||||
|
|
||||||
class StubTTS:
|
class StubTTS:
|
||||||
async def synthesize(self, text, voice=None, audio_format="pcm"):
|
async def synthesize(self, text, voice=None, audio_format="pcm", language=None):
|
||||||
return b"AUD"
|
return b"AUD"
|
||||||
|
|
||||||
monkeypatch.setitem(deps.LLM_REGISTRY, "stream", lambda s: StreamLLM())
|
monkeypatch.setitem(deps.LLM_REGISTRY, "stream", lambda s: StreamLLM())
|
||||||
|
|
@ -99,7 +99,7 @@ def test_ws_stream_fallback_for_nonstreaming_llm(monkeypatch):
|
||||||
return "komplett"
|
return "komplett"
|
||||||
|
|
||||||
class StubTTS:
|
class StubTTS:
|
||||||
async def synthesize(self, text, voice=None, audio_format="pcm"):
|
async def synthesize(self, text, voice=None, audio_format="pcm", language=None):
|
||||||
return b"X"
|
return b"X"
|
||||||
|
|
||||||
monkeypatch.setitem(deps.LLM_REGISTRY, "oc", lambda s: OnlyComplete())
|
monkeypatch.setitem(deps.LLM_REGISTRY, "oc", lambda s: OnlyComplete())
|
||||||
|
|
@ -126,7 +126,7 @@ def test_ws_audio_stream_sends_chunks_per_sentence(monkeypatch):
|
||||||
yield tok
|
yield tok
|
||||||
|
|
||||||
class CountTTS:
|
class CountTTS:
|
||||||
async def synthesize(self, text, voice=None, audio_format="pcm"):
|
async def synthesize(self, text, voice=None, audio_format="pcm", language=None):
|
||||||
tts_calls.append(text)
|
tts_calls.append(text)
|
||||||
return b"A" * len(tts_calls)
|
return b"A" * len(tts_calls)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@ def test_warmup_loads_piper(monkeypatch):
|
||||||
calls = {}
|
calls = {}
|
||||||
|
|
||||||
class FakePiperTTSProvider: # Name muss zur Typpruefung im warmup passen
|
class FakePiperTTSProvider: # Name muss zur Typpruefung im warmup passen
|
||||||
async def synthesize(self, text, voice=None, audio_format="pcm"):
|
async def synthesize(self, text, voice=None, audio_format="pcm", language=None):
|
||||||
calls["tts"] = text
|
calls["tts"] = text
|
||||||
return b"AUDIO"
|
return b"AUDIO"
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -16,7 +16,7 @@ def _install_stubs(monkeypatch, captured):
|
||||||
return f"Echo: {text}"
|
return f"Echo: {text}"
|
||||||
|
|
||||||
class StubTTS:
|
class StubTTS:
|
||||||
async def synthesize(self, text, voice=None, audio_format="pcm"):
|
async def synthesize(self, text, voice=None, audio_format="pcm", language=None):
|
||||||
return b"WSAUDIO"
|
return b"WSAUDIO"
|
||||||
|
|
||||||
monkeypatch.setitem(deps.LLM_REGISTRY, "mem", lambda s: MemLLM())
|
monkeypatch.setitem(deps.LLM_REGISTRY, "mem", lambda s: MemLLM())
|
||||||
|
|
@ -78,7 +78,7 @@ def _install_voice_stubs(monkeypatch):
|
||||||
return f"Antwort zu {text}"
|
return f"Antwort zu {text}"
|
||||||
|
|
||||||
class StubTTS:
|
class StubTTS:
|
||||||
async def synthesize(self, text, voice=None, audio_format="pcm"):
|
async def synthesize(self, text, voice=None, audio_format="pcm", language=None):
|
||||||
return b"VOICEAUD"
|
return b"VOICEAUD"
|
||||||
|
|
||||||
monkeypatch.setitem(deps.STT_REGISTRY, "ss", lambda s: StubSTT())
|
monkeypatch.setitem(deps.STT_REGISTRY, "ss", lambda s: StubSTT())
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue