Initial commit: Voice Assistant Gateway mit Konfig-/Routing-Fundament
- FastAPI-Gateway mit REST-Endpunkten (chat/speak/transcribe/devices/sessions/config) - Geschichtete Konfiguration mit Profilen (local-dev/hybrid/cloud) via TOML + ENV - Registry-Pattern + einheitliche Route-Aufloesung (Default->Profil->ENV->Session->Request) - Device Router (strikt, Singleton) und Output-Lifecycle im Orchestrator - OpenRouter-Adapter (STT multipart/LLM/TTS) + lokale Provider-Stubs - Regelbasierte Pipeline (Cleaner/Spoken-Adapter/TTS-Normalizer) - 22 automatisierte Tests; Doku: README, BEDIENUNGSANLEITUNG, Architektur - Secrets ausschliesslich ueber Umgebung; .env und lokale config gitignored Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Dieter Schlüter
commit
293ed257db
72 changed files with 2612 additions and 0 deletions
31
.env.example
Normal file
31
.env.example
Normal file
|
|
@ -0,0 +1,31 @@
|
||||||
|
# Port bei Bedarf anpassen
|
||||||
|
APP_ENV=dev
|
||||||
|
HOST=0.0.0.0
|
||||||
|
PORT=8080
|
||||||
|
LOG_LEVEL=info
|
||||||
|
|
||||||
|
# Secret nur ueber die Umgebung setzen (nicht hier eintragen), z. B. export in ~/.bashrc
|
||||||
|
OPENROUTER_API_KEY=
|
||||||
|
|
||||||
|
# --- Zentrale Konfiguration / Profile -------------------------------------
|
||||||
|
# Aktives Profil aus config/voice-assistant.toml waehlen: local-dev | hybrid | cloud
|
||||||
|
# (leer lassen = nur Defaults/ENV). Eigener Pfad via VA_CONFIG_FILE.
|
||||||
|
VA_PROFILE=
|
||||||
|
# VA_CONFIG_FILE=config/voice-assistant.toml
|
||||||
|
|
||||||
|
# Hinweis zur Praezedenz: ENV gewinnt ueber die TOML-Datei. Die DEFAULT_*_PROVIDER-
|
||||||
|
# Zeilen unten ueberschreiben daher ein gesetztes VA_PROFILE. Wer profilbasiert
|
||||||
|
# umschalten will, sollte sie auskommentiert lassen.
|
||||||
|
OPENROUTER_STT_MODEL=openai/whisper-large-v3
|
||||||
|
OPENROUTER_TTS_MODEL=openai/gpt-4o-mini-tts
|
||||||
|
OPENROUTER_TTS_VOICE=alloy
|
||||||
|
OPENROUTER_LLM_MODEL=openai/gpt-4.1-mini
|
||||||
|
DEFAULT_LANGUAGE=de
|
||||||
|
DEFAULT_INPUT_ENDPOINT=local-default
|
||||||
|
DEFAULT_OUTPUT_ENDPOINT=local-default
|
||||||
|
# DEFAULT_STT_PROVIDER=openrouter
|
||||||
|
# DEFAULT_LLM_PROVIDER=local-openai-compatible
|
||||||
|
# DEFAULT_TTS_PROVIDER=openrouter
|
||||||
|
LOCAL_LLM_BASE_URL=http://127.0.0.1:11434/v1
|
||||||
|
LOCAL_LLM_API_KEY=dummy
|
||||||
|
LOCAL_LLM_MODEL=llama3.1
|
||||||
21
.gitignore
vendored
Normal file
21
.gitignore
vendored
Normal file
|
|
@ -0,0 +1,21 @@
|
||||||
|
# Secrets / lokale Konfiguration
|
||||||
|
.env
|
||||||
|
|
||||||
|
# Lokale/instanzspezifische Konfiguration (nur die *.example.toml wird versioniert)
|
||||||
|
config/voice-assistant.toml
|
||||||
|
|
||||||
|
# Python
|
||||||
|
__pycache__/
|
||||||
|
*.py[cod]
|
||||||
|
*.egg-info/
|
||||||
|
.venv/
|
||||||
|
venv/
|
||||||
|
.pytest_cache/
|
||||||
|
|
||||||
|
# Lokale Tool-/Editor-Konfiguration
|
||||||
|
.claude/
|
||||||
|
|
||||||
|
# Editor-/Backup-Reste
|
||||||
|
*.bak
|
||||||
|
*.patch
|
||||||
|
*.orig
|
||||||
223
BEDIENUNGSANLEITUNG.md
Normal file
223
BEDIENUNGSANLEITUNG.md
Normal file
|
|
@ -0,0 +1,223 @@
|
||||||
|
# Bedienungsanleitung — Voice Assistant Gateway
|
||||||
|
|
||||||
|
Diese Anleitung führt Schritt für Schritt durch Installation, Start, Konfiguration
|
||||||
|
und Fehlerbehebung. Technische Hintergründe stehen im
|
||||||
|
[Architektur-Dokument](Docs/voice-assistant-architecture.md), eine kompakte
|
||||||
|
Übersicht im [README](README.md).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Voraussetzungen
|
||||||
|
|
||||||
|
- **Python 3.11 oder neuer** (`python3 --version`)
|
||||||
|
- Ein **OpenRouter-API-Key** — nur nötig, wenn ein Profil entfernte KI nutzt
|
||||||
|
(`hybrid`, `cloud`). Für rein lokalen Betrieb (`local-dev`) nicht erforderlich.
|
||||||
|
- Optional: Docker, falls im Container betrieben.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Installation
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd voice-assistant-scaffold
|
||||||
|
python3 -m venv .venv
|
||||||
|
source .venv/bin/activate
|
||||||
|
pip install -U pip
|
||||||
|
pip install -e .[test]
|
||||||
|
```
|
||||||
|
|
||||||
|
Danach die zentrale Konfigurationsdatei anlegen:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cp config/voice-assistant.example.toml config/voice-assistant.toml
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. API-Key hinterlegen (für Cloud/Hybrid)
|
||||||
|
|
||||||
|
Der Schlüssel wird **aus der Umgebung** gelesen und gehört **nicht** in eine Datei.
|
||||||
|
Dauerhaft am besten in `~/.bashrc`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
echo 'export OPENROUTER_API_KEY=sk-or-v1-DEIN_KEY' >> ~/.bashrc
|
||||||
|
chmod 600 ~/.bashrc
|
||||||
|
source ~/.bashrc
|
||||||
|
```
|
||||||
|
|
||||||
|
Prüfen, ob er ankommt:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
echo ${OPENROUTER_API_KEY:0:8} # zeigt nur den Anfang
|
||||||
|
```
|
||||||
|
|
||||||
|
> **Sicherheit:** Den Key niemals in `.env` oder `config/*.toml` schreiben. Wird ein
|
||||||
|
> Key versehentlich öffentlich, im OpenRouter-Dashboard löschen (= widerrufen) und
|
||||||
|
> neu erzeugen.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Betriebsart (Profil) wählen
|
||||||
|
|
||||||
|
Profile bestimmen, welche KI-Module genutzt werden:
|
||||||
|
|
||||||
|
| Profil | Bedeutung | Key nötig? |
|
||||||
|
|-------------|--------------------------------------------|------------|
|
||||||
|
| `local-dev` | alles lokal (eigene KI/Hardware) | nein |
|
||||||
|
| `hybrid` | STT/TTS über Cloud, Haupt-LLM lokal | ja |
|
||||||
|
| `cloud` | alles über OpenRouter (Standardbetrieb) | ja |
|
||||||
|
|
||||||
|
Profil **einmalig** für einen Start:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
VA_PROFILE=cloud make run
|
||||||
|
```
|
||||||
|
|
||||||
|
Profil **dauerhaft** — in `.env` eintragen:
|
||||||
|
|
||||||
|
```
|
||||||
|
VA_PROFILE=cloud
|
||||||
|
```
|
||||||
|
|
||||||
|
> Hinweis: Stehen in `.env` noch `DEFAULT_STT_PROVIDER` / `DEFAULT_LLM_PROVIDER` /
|
||||||
|
> `DEFAULT_TTS_PROVIDER`, überschreiben diese das Profil. Für profilbasiertes
|
||||||
|
> Umschalten sollten sie auskommentiert sein.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Starten und Stoppen
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make run
|
||||||
|
```
|
||||||
|
|
||||||
|
Standard-Adresse: `http://localhost:8080` (Port änderbar, siehe Abschnitt 8).
|
||||||
|
Beenden mit **Strg + C**.
|
||||||
|
|
||||||
|
Schnelltest in einem zweiten Terminal:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://localhost:8080/health
|
||||||
|
# {"status":"ok"}
|
||||||
|
|
||||||
|
curl http://localhost:8080/api/config
|
||||||
|
# zeigt aktives Profil und die aufgelöste Standard-Route
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Tägliche Bedienung — typische Aufgaben
|
||||||
|
|
||||||
|
### a) Text sprechen lassen (`/api/speak`)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST http://localhost:8080/api/speak \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d '{"text":"Guten Morgen, wie geht es Ihnen?"}' \
|
||||||
|
--output antwort.pcm
|
||||||
|
```
|
||||||
|
|
||||||
|
### b) Chatten (Text rein, gesprochene Antwort raus) (`/api/chat`)
|
||||||
|
|
||||||
|
Nur den Trace als JSON ansehen (ohne Audio):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST "http://localhost:8080/api/chat?debug=true" \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d '{"text":"Wie wird das Wetter morgen?"}'
|
||||||
|
```
|
||||||
|
|
||||||
|
Komfortabler mit dem mitgelieferten Client (spielt die Antwort ab):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python chat_client.py "Erzähl mir einen guten Morgen-Spruch"
|
||||||
|
```
|
||||||
|
|
||||||
|
> `chat_client.py` erwartet den Dienst auf Port **8003** — bei Bedarf im Skript
|
||||||
|
> `GATEWAY_URL` anpassen oder den Dienst mit `PORT=8003 make run` starten.
|
||||||
|
|
||||||
|
### c) Audio transkribieren (`/api/transcribe`)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST http://localhost:8080/api/transcribe \
|
||||||
|
-F "file=@aufnahme.wav" -F "language=de"
|
||||||
|
```
|
||||||
|
|
||||||
|
### d) Gerät oder Provider einmalig umstellen (pro Aufruf)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST http://localhost:8080/api/speak \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d '{"text":"Test","tts_provider":"piper","output_endpoint":"loopback"}'
|
||||||
|
```
|
||||||
|
|
||||||
|
### e) Präferenzen für eine Session festlegen
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# einmal setzen
|
||||||
|
curl -X POST http://localhost:8080/api/sessions/oma-anna/route \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d '{"llm_provider":"openrouter","language":"de"}'
|
||||||
|
|
||||||
|
# danach mit dieser Session nutzen
|
||||||
|
curl -X POST "http://localhost:8080/api/chat?session_id=oma-anna&debug=true" \
|
||||||
|
-H 'Content-Type: application/json' -d '{"text":"Hallo!"}'
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. Verfügbare Geräte und Bausteine ansehen
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://localhost:8080/api/devices # Audio-Endpunkte mit Fähigkeiten
|
||||||
|
curl http://localhost:8080/api/config # Profil, Route, Provider, Endpunkte
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. Port ändern
|
||||||
|
|
||||||
|
```bash
|
||||||
|
PORT=8003 make run # einmalig
|
||||||
|
sed -i 's/^PORT=.*/PORT=8003/' .env # dauerhaft
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. Mit Docker betreiben
|
||||||
|
|
||||||
|
```bash
|
||||||
|
export OPENROUTER_API_KEY=sk-or-v1-...
|
||||||
|
docker compose up --build
|
||||||
|
```
|
||||||
|
|
||||||
|
Der Key wird aus der Shell in den Container durchgereicht; fehlt er, bricht der
|
||||||
|
Start mit klarer Meldung ab.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 10. Fehlerbehebung
|
||||||
|
|
||||||
|
| Symptom | Ursache | Lösung |
|
||||||
|
|---|---|---|
|
||||||
|
| `OPENROUTER_API_KEY is empty` / 401 | Key nicht in der Umgebung | `export OPENROUTER_API_KEY=…`, neues Terminal / `source ~/.bashrc` |
|
||||||
|
| HTTP **422** „Unbekannter …-Provider/Endpunkt" | Tippfehler in `*_provider` / `*_endpoint` | gültige Werte via `GET /api/config` prüfen |
|
||||||
|
| `VA_PROFILE` wirkt nicht | `DEFAULT_*_PROVIDER` in `.env` überschreibt es | diese Zeilen in `.env` auskommentieren |
|
||||||
|
| LLM-Timeout / Connection refused (lokal) | lokaler LLM-Server (Port 11434) läuft nicht | LLM-Server starten oder Profil `cloud` wählen |
|
||||||
|
| `Address already in use` | Port belegt | anderen `PORT` setzen (Abschnitt 8) |
|
||||||
|
| `chat_client.py` bekommt keine Antwort | Client nutzt Port 8003 | Dienst mit `PORT=8003` starten oder `GATEWAY_URL` anpassen |
|
||||||
|
| Profil greift nicht / Standardwerte | `config/voice-assistant.toml` fehlt | Datei aus `*.example.toml` kopieren (Abschnitt 2) |
|
||||||
|
|
||||||
|
Logs erscheinen im Terminal, in dem `make run` läuft. Für mehr Details
|
||||||
|
`LOG_LEVEL=debug` in `.env` setzen.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 11. Tests ausführen
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make test
|
||||||
|
```
|
||||||
|
|
||||||
|
Alle Tests sollten grün sein. Schlägt etwas fehl, gibt die Ausgabe den genauen
|
||||||
|
Testnamen und die Ursache an.
|
||||||
7
Dockerfile
Normal file
7
Dockerfile
Normal file
|
|
@ -0,0 +1,7 @@
|
||||||
|
FROM python:3.12-slim
|
||||||
|
WORKDIR /app
|
||||||
|
COPY pyproject.toml README.md ./
|
||||||
|
COPY app ./app
|
||||||
|
RUN pip install --no-cache-dir -U pip && pip install --no-cache-dir .[test]
|
||||||
|
EXPOSE 8080
|
||||||
|
CMD ["sh", "-c", "uvicorn app.main:app --host ${HOST:-0.0.0.0} --port ${PORT:-8080}"]
|
||||||
220
Docs/voice-assistant-architecture.md
Normal file
220
Docs/voice-assistant-architecture.md
Normal file
|
|
@ -0,0 +1,220 @@
|
||||||
|
# Architektur: Modularer Voice-Assistent
|
||||||
|
|
||||||
|
> Stand: aktueller Implementierungsstand des Gateways. Dieses Dokument beschreibt
|
||||||
|
> das Konzept, den umgesetzten Stand und die geplanten nächsten Schritte.
|
||||||
|
|
||||||
|
## 1. Ziel & Leitidee
|
||||||
|
|
||||||
|
Ein modularer, **cloud-first, aber hybrid betreibbarer** Sprachassistent für
|
||||||
|
Senioren — ein „digitaler Vertrauter", erreichbar von zuhause und unterwegs.
|
||||||
|
|
||||||
|
Der Hauptbetrieb läuft in der Cloud / auf einem vHost (Senioren sollen keinen
|
||||||
|
teuren KI-Rechner zuhause brauchen). Gleichzeitig muss jede Achse frei wählbar
|
||||||
|
bleiben, je nach Installation und Entwicklungs-/Testbedarf:
|
||||||
|
|
||||||
|
- **Hardware** — Audio-Eingabe und -Ausgabe (lokal, Bluetooth, Handy, Netzwerk)
|
||||||
|
- **Betrieb** — lokal, Cloud oder hybrid
|
||||||
|
- **Software** — lokale oder entfernte KI (STT / LLM / TTS), ganz oder teilweise
|
||||||
|
|
||||||
|
Designziele: geringe Latenz, robuste Fallbacks, gute Debugbarkeit, klare Trennung
|
||||||
|
von **semantischer** und **gesprochener** Antwort, und vor allem **Austauschbarkeit**:
|
||||||
|
einzelne Module gegen andere tauschen, ohne das Programm umzuschreiben.
|
||||||
|
|
||||||
|
## 2. Architekturprinzip — fünf Ebenen
|
||||||
|
|
||||||
|
1. **Audio Endpoints** – konkrete Quellen/Ziele (lokal, Bluetooth, Handy, WebRTC)
|
||||||
|
2. **Device Router** – wählt passende Input-/Output-Endpunkte
|
||||||
|
3. **Speech Pipeline** – STT, Input-Cleaner, Dialog-LLM, Spoken-Response-Adapter, TTS-Normalizer, TTS
|
||||||
|
4. **Transport Router** – lokale vs. entfernte Ausführung einzelner Module
|
||||||
|
5. **Orchestrator** – Session, Turn-Taking, Fallbacks, Metrik
|
||||||
|
|
||||||
|
Jede Ebene kommuniziert über wohldefinierte Interfaces (ABCs + Pydantic-Schemas),
|
||||||
|
nicht über konkrete Bibliotheken.
|
||||||
|
|
||||||
|
## 3. Konfigurations- & Routing-Modell (umgesetzt)
|
||||||
|
|
||||||
|
Das Herzstück der Austauschbarkeit. Jede Achse ist auf mehreren Ebenen
|
||||||
|
einstellbar; höhere Ebene gewinnt:
|
||||||
|
|
||||||
|
```
|
||||||
|
eingebaute Defaults < config/voice-assistant.toml (inkl. aktivem Profil)
|
||||||
|
< ENV / .env < Session-Route < Request
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3.1 Zentrale Config + Profile
|
||||||
|
|
||||||
|
Eine geschichtete zentrale Datei (`config/voice-assistant.toml`, Vorlage
|
||||||
|
`config/voice-assistant.example.toml`), gelesen über die stdlib (`tomllib`).
|
||||||
|
**Profile** bündeln Betriebsarten und werden per ENV `VA_PROFILE` aktiviert:
|
||||||
|
|
||||||
|
| Profil | STT | LLM | TTS |
|
||||||
|
|-------------|----------------|--------------------------|------------|
|
||||||
|
| `local-dev` | faster-whisper | local-openai-compatible | piper |
|
||||||
|
| `hybrid` | openrouter | local-openai-compatible | openrouter |
|
||||||
|
| `cloud` | openrouter | openrouter | openrouter |
|
||||||
|
|
||||||
|
Umgesetzt in `app/config.py`: `load_profile_config()` merged `[defaults]` +
|
||||||
|
`[profiles.<aktiv>]`; `TomlProfileSource` hängt diese Werte als Settings-Quelle
|
||||||
|
**unter** ENV ein (`settings_customise_sources`). `VA_PROFILE`/`VA_CONFIG_FILE`
|
||||||
|
werden aus echter Umgebung **oder** `.env` gelesen.
|
||||||
|
|
||||||
|
> **Secrets gehören nie in die Config-Datei** — nur in die Umgebung
|
||||||
|
> (z. B. `OPENROUTER_API_KEY`). Die TOML-Datei darf versioniert/geteilt werden.
|
||||||
|
|
||||||
|
### 3.2 Einheitliche Route-Auflösung
|
||||||
|
|
||||||
|
`app/dependencies.py` löst pro Aufruf eine `ResolvedRoute` auf
|
||||||
|
(`resolve_route(session_id, overrides)`): Defaults < Session-Route < Request.
|
||||||
|
Die Route umfasst `input_endpoint`, `output_endpoint`, `stt_provider`,
|
||||||
|
`llm_provider`, `tts_provider`, `language`.
|
||||||
|
|
||||||
|
### 3.3 Registry-Pattern (Provider austauschbar)
|
||||||
|
|
||||||
|
`STT_REGISTRY` / `LLM_REGISTRY` / `TTS_REGISTRY` bilden `name -> factory(settings)`.
|
||||||
|
Ein neuer Provider = ein Eintrag, ohne Kern-Code zu ändern. Unbekannter Name →
|
||||||
|
`UnknownComponentError` → HTTP 422.
|
||||||
|
|
||||||
|
### 3.4 Device Router
|
||||||
|
|
||||||
|
`app/audio/router.py` wählt Endpunkte per `id`- oder `kind`-Match. Ein angefragter,
|
||||||
|
aber unbekannter Endpunkt wirft `UnknownEndpointError` (→ 422) — **kein** stiller
|
||||||
|
Default-Fallback. Ohne Wunsch greift der als `default` markierte Endpunkt. Der
|
||||||
|
Router ist ein Singleton (stabiler Zustand, z. B. `LoopbackOutput`).
|
||||||
|
|
||||||
|
## 4. Datenmodelle & Interfaces
|
||||||
|
|
||||||
|
Schemas in `app/schemas.py`, Interfaces als ABCs in den jeweiligen `base.py`.
|
||||||
|
|
||||||
|
```python
|
||||||
|
class EndpointCapabilities(BaseModel):
|
||||||
|
id: str; kind: str
|
||||||
|
direction: Literal["input", "output"]
|
||||||
|
sample_rate: int = 16000; channels: int = 1
|
||||||
|
latency_class: Literal["low", "medium", "high"] = "medium"
|
||||||
|
supports_aec: bool = False; supports_barge_in: bool = False
|
||||||
|
networked: bool = False; bluetooth: bool = False
|
||||||
|
mobile: bool = False; default: bool = False
|
||||||
|
|
||||||
|
class AudioChunk(BaseModel):
|
||||||
|
data: bytes; sample_rate: int = 16000; channels: int = 1
|
||||||
|
format: str = "wav"; timestamp_ms: int = 0
|
||||||
|
|
||||||
|
class PipelineTrace(BaseModel):
|
||||||
|
raw_transcript: str | None = None
|
||||||
|
cleaned_transcript: str | None = None
|
||||||
|
semantic_response: str | None = None
|
||||||
|
spoken_response: str | None = None
|
||||||
|
tts_ready_text: str | None = None
|
||||||
|
```
|
||||||
|
|
||||||
|
Provider-Interfaces:
|
||||||
|
|
||||||
|
```python
|
||||||
|
class STTProvider(ABC):
|
||||||
|
async def transcribe(self, audio_bytes: bytes, fmt: str, language: str | None = None) -> str: ...
|
||||||
|
|
||||||
|
class LLMProvider(ABC):
|
||||||
|
async def complete(self, text: str, session_id: str | None = None) -> str: ...
|
||||||
|
|
||||||
|
class TTSProvider(ABC):
|
||||||
|
async def synthesize(self, text: str, voice: str | None = None, audio_format: str = "pcm") -> bytes: ...
|
||||||
|
```
|
||||||
|
|
||||||
|
Audio-Endpunkt-Interfaces: `AudioInputEndpoint` (`capabilities/open/read_chunk/close`),
|
||||||
|
`AudioOutputEndpoint` (`capabilities/open/write_chunk/flush/close`) — alle async.
|
||||||
|
|
||||||
|
## 5. Speech Pipeline
|
||||||
|
|
||||||
|
Trennung von **semantischer** und **gesprochener** Antwort ist zentral: eine
|
||||||
|
inhaltlich gute Antwort ist nicht automatisch gut hörbar.
|
||||||
|
|
||||||
|
Stufen: `raw_transcript → cleaned_transcript → semantic_response → spoken_response → tts_ready_text`.
|
||||||
|
|
||||||
|
- **Input Cleaner** (`pipeline/input_cleaner.py`) – konservative Bereinigung des STT-Texts (Füllwörter, Whitespace). Verändert die Nutzerintention nicht.
|
||||||
|
- **Dialog-LLM** – semantische Antwort; Persona/Sicherheitsregeln im System-Prompt (`providers/llm/openrouter.py`).
|
||||||
|
- **Spoken Response Adapter** (`pipeline/spoken_response_adapter.py`) – macht die Antwort sprechbar/seniorengerecht (Markdown raus, Listen → Sätze, Uhrzeiten erhalten).
|
||||||
|
- **TTS Normalizer** (`pipeline/tts_normalizer.py`) – Zahlen/Abkürzungen/Einheiten verbal ausformulieren, sprachabhängig (de/en).
|
||||||
|
|
||||||
|
Empfehlung: nicht jede Zwischenstufe braucht ein großes LLM — Cleaner und
|
||||||
|
Normalizer überwiegend regelbasiert (so heute umgesetzt), Adapter promptbasiert.
|
||||||
|
|
||||||
|
## 6. Orchestrator
|
||||||
|
|
||||||
|
`app/core/orchestrator.py` verbindet Provider, Pipeline und Output-Endpunkt.
|
||||||
|
|
||||||
|
- `chat_text(text, language, voice, output)` → `(trace, audio_bytes)`
|
||||||
|
- `speak_only(text, voice, language, output)` → `audio_bytes`
|
||||||
|
- `transcribe_only(audio_bytes, fmt, language, input)` → `trace`
|
||||||
|
|
||||||
|
Das synthetisierte Audio wird **zusätzlich** durch den gewählten Output-Endpunkt
|
||||||
|
geschrieben (`open → write_chunk → flush → close`) und **gleichzeitig** als
|
||||||
|
HTTP-Stream zurückgegeben (additiv). Bei lokalen Geräten ist `write_chunk` heute
|
||||||
|
ein No-op; `LoopbackOutput` sammelt die Chunks (testbar ohne Hardware).
|
||||||
|
|
||||||
|
## 7. FastAPI-Endpunkte (umgesetzt)
|
||||||
|
|
||||||
|
| Methode & Pfad | Zweck |
|
||||||
|
|--------------------------------------|-------|
|
||||||
|
| `GET /health` | Liveness |
|
||||||
|
| `POST /api/chat` | Text rein → Audio raus (`?debug=true` → JSON-Trace) |
|
||||||
|
| `POST /api/speak` | Text rein → TTS-Audio raus |
|
||||||
|
| `POST /api/transcribe` | Audio-Upload → Transkript |
|
||||||
|
| `GET /api/devices` | verfügbare Audio-Endpunkte + Capabilities |
|
||||||
|
| `POST /api/sessions/{id}/route` | bevorzugte Geräte/Provider/Sprache je Session |
|
||||||
|
| `GET /api/config` | aktives Profil + aufgelöste Route (ohne Secrets) |
|
||||||
|
|
||||||
|
Endpunkt-/Provider-Auswahl ist über **Request-Body** (pro Aufruf), **Session**
|
||||||
|
(`?session_id=…`) und **Defaults/Profil** steuerbar. Verwendete Route erscheint als
|
||||||
|
`X-*`-Header bzw. im `?debug`-JSON.
|
||||||
|
|
||||||
|
## 8. Stand der Implementierung
|
||||||
|
|
||||||
|
**Umgesetzt:** FastAPI-Gateway, alle o. g. REST-Endpunkte; OpenRouter-Adapter für
|
||||||
|
STT (multipart), LLM und TTS; lokaler OpenAI-kompatibler LLM-Adapter; regelbasierte
|
||||||
|
Pipeline; geschichtete Config + Profile; Registry + einheitliche Route-Auflösung;
|
||||||
|
Device Router (strikt, Singleton); Output-Lifecycle; 22 automatisierte Tests.
|
||||||
|
|
||||||
|
**Platzhalter (Gerüst):** Audio-Endpunkte (`local-default`, `bluetooth`,
|
||||||
|
`mobile-ws`, `mobile-webrtc`) liefern leere Chunks — nur Auswahl/Lifecycle sind
|
||||||
|
verdrahtet, kein echtes Hardware-I/O. Lokale Provider `faster-whisper`, `piper`,
|
||||||
|
`chatterbox` sind Stubs. `transport_router.py` (Ebene 4) existiert, ist aber noch
|
||||||
|
nicht aktiv (lokal/remote trägt vorerst der Provider-Name).
|
||||||
|
|
||||||
|
## 9. Roadmap / bewusste nächste Schritte
|
||||||
|
|
||||||
|
Reihenfolge der Weiterentwicklung:
|
||||||
|
|
||||||
|
1. **(erledigt)** Konfig- & Routing-Fundament: Profile, Device Router, Registry, Pro-Request-Override.
|
||||||
|
2. **Cloud-Fundament:** Authentifizierung + Mehrbenutzer + persistenter Session-/Profil-Store (heute ist `SessionManager` in-memory → nicht skalierend, ohne Auth).
|
||||||
|
3. **Konversationsgedächtnis:** Verlauf + Langzeit-Präferenzen (heute ist `llm.complete` zustandslos).
|
||||||
|
4. **Echtzeit:** Streaming-STT/TTS, WebSocket/WebRTC-Endpunkte real, Barge-in, Turn-Manager.
|
||||||
|
5. **Resilienz:** Fallback-Policy (remote KI fällt aus → lokaler/alternativer Provider), Metriken/Tracing.
|
||||||
|
6. **Betrieb:** Kosten-/Quota-Kontrolle pro Nutzer; Notfall-/Eskalationskonzept (Senioren-Kontext).
|
||||||
|
7. **TransportRouter** als eigene lokal/remote-Achse aktivieren.
|
||||||
|
|
||||||
|
**Datenschutz (querschnittlich, ab sofort mitdenken):** Senioren-Sprachdaten sind
|
||||||
|
hochsensibel (oft gesundheitsbezogen → DSGVO Art. 9). EU-Datenresidenz,
|
||||||
|
Verschlüsselung at-rest/in-transit, Löschkonzept, Einwilligung — „privacy by design".
|
||||||
|
|
||||||
|
## 10. Verzeichnisstruktur (Ist)
|
||||||
|
|
||||||
|
```text
|
||||||
|
voice-assistant-scaffold/
|
||||||
|
├── app/
|
||||||
|
│ ├── main.py # FastAPI-App + Router-Registrierung
|
||||||
|
│ ├── config.py # Settings, TOML-Profile, Präzedenz
|
||||||
|
│ ├── dependencies.py # Registries, ResolvedRoute, resolve_route, Singleton-Router
|
||||||
|
│ ├── errors.py # RoutingError -> HTTP 422
|
||||||
|
│ ├── schemas.py # Pydantic-Modelle
|
||||||
|
│ ├── api/ # health, chat, speak, transcribe, devices, sessions, config
|
||||||
|
│ ├── core/ # orchestrator, session_manager
|
||||||
|
│ ├── audio/ # router, transport_router, endpoints/input|output/*
|
||||||
|
│ ├── pipeline/ # input_cleaner, spoken_response_adapter, tts_normalizer
|
||||||
|
│ └── providers/ # stt/ llm/ tts/ (openrouter + lokale Stubs)
|
||||||
|
├── config/ # voice-assistant.example.toml (+ lokale .toml, gitignored)
|
||||||
|
├── deploy/ # systemd unit + env-Beispiel
|
||||||
|
├── tests/ # config-profile, routing, audio-router, e2e
|
||||||
|
├── Docs/ # dieses Dokument
|
||||||
|
├── Dockerfile, docker-compose.yml, Makefile, pyproject.toml
|
||||||
|
└── README.md, BEDIENUNGSANLEITUNG.md
|
||||||
|
```
|
||||||
28
Makefile
Normal file
28
Makefile
Normal file
|
|
@ -0,0 +1,28 @@
|
||||||
|
ifneq (,$(wildcard ./.env))
|
||||||
|
include .env
|
||||||
|
export
|
||||||
|
endif
|
||||||
|
|
||||||
|
PORT ?= 8080
|
||||||
|
HOST ?= 0.0.0.0
|
||||||
|
|
||||||
|
.PHONY: ensure-env install run test docker-build
|
||||||
|
|
||||||
|
ensure-env:
|
||||||
|
@if [ ! -f .env ] && [ -f .env.example ]; then \
|
||||||
|
cp .env.example .env; \
|
||||||
|
echo "Created .env from .env.example"; \
|
||||||
|
fi
|
||||||
|
|
||||||
|
install: ensure-env
|
||||||
|
python3 -m venv .venv
|
||||||
|
. .venv/bin/activate && pip install -U pip && pip install -e .[test]
|
||||||
|
|
||||||
|
run: ensure-env
|
||||||
|
. .venv/bin/activate && uvicorn app.main:app --host $(HOST) --port $(PORT) --reload
|
||||||
|
|
||||||
|
test: ensure-env
|
||||||
|
. .venv/bin/activate && pytest tests/
|
||||||
|
|
||||||
|
docker-build:
|
||||||
|
docker build -t voice-assistant-gateway .
|
||||||
130
README.md
Normal file
130
README.md
Normal file
|
|
@ -0,0 +1,130 @@
|
||||||
|
# Voice Assistant Gateway
|
||||||
|
|
||||||
|
Modulares FastAPI-Gateway für einen **seniorengerechten Sprachassistenten** —
|
||||||
|
cloud-first, aber hybrid/lokal betreibbar, mit austauschbaren Audio-Endpunkten und
|
||||||
|
STT-/LLM-/TTS-Providern.
|
||||||
|
|
||||||
|
Jede Achse — **Hardware** (Audio In/Out), **Betrieb** (lokal/cloud) und **Software**
|
||||||
|
(lokale/remote KI) — ist frei konfigurierbar, ohne Code zu ändern. Konzept und
|
||||||
|
Details: [`Docs/voice-assistant-architecture.md`](Docs/voice-assistant-architecture.md).
|
||||||
|
Praktische Bedienung: [`BEDIENUNGSANLEITUNG.md`](BEDIENUNGSANLEITUNG.md).
|
||||||
|
|
||||||
|
## Features
|
||||||
|
|
||||||
|
- **Pipeline mit getrennter Semantik/Sprache:** STT → Input-Cleaner → LLM → Spoken-Adapter → TTS-Normalizer → TTS
|
||||||
|
- **Provider austauschbar** über Registry (OpenRouter remote; faster-whisper/piper/chatterbox als lokale Stubs)
|
||||||
|
- **Geschichtete Konfiguration** mit Profilen (`local-dev` / `hybrid` / `cloud`)
|
||||||
|
- **Routing auf jeder Ebene:** Default → Profil → ENV → Session → Request
|
||||||
|
- **REST-API** für Chat, Transkription, Sprachausgabe, Geräte, Sessions, Config
|
||||||
|
- **Ohne Secrets im Code** — API-Keys nur über die Umgebung
|
||||||
|
|
||||||
|
## Schnellstart
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 -m venv .venv
|
||||||
|
source .venv/bin/activate
|
||||||
|
pip install -U pip
|
||||||
|
pip install -e .[test]
|
||||||
|
|
||||||
|
cp config/voice-assistant.example.toml config/voice-assistant.toml
|
||||||
|
export OPENROUTER_API_KEY=sk-or-v1-... # nur für Cloud-/Hybrid-Profile nötig
|
||||||
|
make run
|
||||||
|
```
|
||||||
|
|
||||||
|
Fehlt `.env`, wird sie beim ersten `make run` aus `.env.example` erzeugt.
|
||||||
|
Die App läuft dann auf `http://localhost:8080` (bzw. dem in `.env` gesetzten `PORT`).
|
||||||
|
|
||||||
|
Kurztest:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://localhost:8080/health
|
||||||
|
curl http://localhost:8080/api/config
|
||||||
|
```
|
||||||
|
|
||||||
|
## Konfiguration & Profile
|
||||||
|
|
||||||
|
Höhere Ebene gewinnt:
|
||||||
|
|
||||||
|
```
|
||||||
|
eingebaute Defaults < config/voice-assistant.toml (inkl. aktivem Profil)
|
||||||
|
< ENV / .env < Session-Route < Request
|
||||||
|
```
|
||||||
|
|
||||||
|
**Profile** umschalten per Umgebungsvariable (oder dauerhaft in `.env`):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
VA_PROFILE=local-dev make run # alles lokal (faster-whisper / lokales LLM / piper)
|
||||||
|
VA_PROFILE=hybrid make run # STT/TTS remote, LLM lokal
|
||||||
|
VA_PROFILE=cloud make run # alles über OpenRouter
|
||||||
|
```
|
||||||
|
|
||||||
|
> Secrets gehören **nicht** in `config/*.toml` — nur in die Umgebung
|
||||||
|
> (`export OPENROUTER_API_KEY=…`). Gesetzte `DEFAULT_*_PROVIDER`-Werte in `.env`
|
||||||
|
> überschreiben ein `VA_PROFILE`.
|
||||||
|
|
||||||
|
**Pro Session:** `POST /api/sessions/{id}/route` (`input_endpoint`, `output_endpoint`,
|
||||||
|
`stt_provider`, `llm_provider`, `tts_provider`, `language`), dann Aufrufe mit `?session_id=…`.
|
||||||
|
|
||||||
|
**Pro Request:** dieselben Felder im Body von `/api/chat` bzw. `/api/speak`.
|
||||||
|
|
||||||
|
Aktive Konfiguration prüfen: `curl http://localhost:8080/api/config`.
|
||||||
|
|
||||||
|
## API-Überblick
|
||||||
|
|
||||||
|
| Methode & Pfad | Zweck |
|
||||||
|
|---------------------------------|-------|
|
||||||
|
| `GET /health` | Liveness-Check |
|
||||||
|
| `POST /api/chat` | Text rein → Audio raus (`?debug=true` → JSON-Trace) |
|
||||||
|
| `POST /api/speak` | Text rein → TTS-Audio raus |
|
||||||
|
| `POST /api/transcribe` | Audio-Upload → Transkript |
|
||||||
|
| `GET /api/devices` | verfügbare Audio-Endpunkte + Capabilities |
|
||||||
|
| `POST /api/sessions/{id}/route` | Geräte/Provider/Sprache je Session setzen |
|
||||||
|
| `GET /api/config` | aktives Profil + aufgelöste Route (ohne Secrets) |
|
||||||
|
|
||||||
|
Beispiel (Sprachausgabe an den Test-Loopback, lokaler TTS-Stub):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST http://localhost:8080/api/speak \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d '{"text":"Guten Morgen!","tts_provider":"piper","output_endpoint":"loopback"}'
|
||||||
|
```
|
||||||
|
|
||||||
|
Unbekannter Endpunkt/Provider → `HTTP 422` mit Klartext-Hinweis.
|
||||||
|
|
||||||
|
## Tests
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make test # oder: pytest -q
|
||||||
|
```
|
||||||
|
|
||||||
|
Abgedeckt: Config-Profile & Präzedenz, Route-Auflösung, Device Router,
|
||||||
|
End-to-End (Loopback, 422-Fälle, Session-/Request-Override, `/api/config`).
|
||||||
|
|
||||||
|
## Port ändern
|
||||||
|
|
||||||
|
```bash
|
||||||
|
PORT=8003 make run # einmalig
|
||||||
|
sed -i 's/^PORT=.*/PORT=8003/' .env # dauerhaft
|
||||||
|
PORT=8003 docker compose up # mit Docker
|
||||||
|
```
|
||||||
|
|
||||||
|
## Deployment
|
||||||
|
|
||||||
|
- **Docker:** `docker compose up --build` (reicht `OPENROUTER_API_KEY` aus der Shell durch)
|
||||||
|
- **systemd:** Vorlagen unter `deploy/` (`voice-assistant.service`, `voice-assistant.env.example`)
|
||||||
|
|
||||||
|
## Projektstruktur (Kurzform)
|
||||||
|
|
||||||
|
```text
|
||||||
|
app/ Gateway: config, dependencies, api/, core/, audio/, pipeline/, providers/
|
||||||
|
config/ voice-assistant.example.toml (lokale .toml ist gitignored)
|
||||||
|
deploy/ systemd-Unit + env-Beispiel
|
||||||
|
tests/ Pytest-Suite
|
||||||
|
Docs/ Architektur-Dokument
|
||||||
|
```
|
||||||
|
|
||||||
|
## Lizenz / Status
|
||||||
|
|
||||||
|
Frühes, aktiv entwickeltes Projektgerüst. Audio-Hardware-/Streaming-Anbindung,
|
||||||
|
Authentifizierung, Persistenz und Gedächtnis sind als nächste Schritte vorgesehen
|
||||||
|
(siehe Roadmap im Architektur-Dokument).
|
||||||
0
app/__init__.py
Normal file
0
app/__init__.py
Normal file
0
app/api/__init__.py
Normal file
0
app/api/__init__.py
Normal file
92
app/api/chat.py
Normal file
92
app/api/chat.py
Normal file
|
|
@ -0,0 +1,92 @@
|
||||||
|
from io import BytesIO
|
||||||
|
|
||||||
|
from fastapi import APIRouter, HTTPException, Query
|
||||||
|
from fastapi.responses import JSONResponse, StreamingResponse
|
||||||
|
|
||||||
|
from app.config import settings
|
||||||
|
from app.errors import RoutingError
|
||||||
|
from app.dependencies import (
|
||||||
|
resolve_route,
|
||||||
|
build_orchestrator,
|
||||||
|
resolve_output_endpoint,
|
||||||
|
)
|
||||||
|
from app.schemas import ChatRequest
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
def _route_headers(route) -> dict:
|
||||||
|
return {
|
||||||
|
"X-Input-Endpoint": route.input_endpoint,
|
||||||
|
"X-Output-Endpoint": route.output_endpoint,
|
||||||
|
"X-STT-Provider": route.stt_provider,
|
||||||
|
"X-LLM-Provider": route.llm_provider,
|
||||||
|
"X-TTS-Provider": route.tts_provider,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/chat")
|
||||||
|
async def chat(
|
||||||
|
payload: ChatRequest,
|
||||||
|
debug: bool = Query(
|
||||||
|
default=False,
|
||||||
|
description="Return JSON trace instead of audio response",
|
||||||
|
),
|
||||||
|
session_id: str | None = Query(
|
||||||
|
default=None,
|
||||||
|
description="Optional session id to apply a stored route",
|
||||||
|
),
|
||||||
|
):
|
||||||
|
overrides = {
|
||||||
|
"input_endpoint": payload.input_endpoint,
|
||||||
|
"output_endpoint": payload.output_endpoint,
|
||||||
|
"language": payload.language,
|
||||||
|
"stt_provider": payload.stt_provider,
|
||||||
|
"llm_provider": payload.llm_provider,
|
||||||
|
"tts_provider": payload.tts_provider,
|
||||||
|
}
|
||||||
|
route = resolve_route(session_id, overrides)
|
||||||
|
voice = payload.voice or settings.openrouter_tts_voice
|
||||||
|
|
||||||
|
try:
|
||||||
|
orchestrator = build_orchestrator(route)
|
||||||
|
output = await resolve_output_endpoint(route)
|
||||||
|
except RoutingError as exc:
|
||||||
|
raise HTTPException(status_code=422, detail=str(exc))
|
||||||
|
|
||||||
|
try:
|
||||||
|
trace, audio = await orchestrator.chat_text(
|
||||||
|
payload.text,
|
||||||
|
language=route.language,
|
||||||
|
voice=voice,
|
||||||
|
output=output,
|
||||||
|
)
|
||||||
|
|
||||||
|
if debug:
|
||||||
|
return JSONResponse(
|
||||||
|
content={
|
||||||
|
"ok": True,
|
||||||
|
"voice": voice,
|
||||||
|
"route": route.as_dict(),
|
||||||
|
"trace": {
|
||||||
|
"raw_transcript": trace.raw_transcript,
|
||||||
|
"cleaned_transcript": trace.cleaned_transcript,
|
||||||
|
"semantic_response": trace.semantic_response,
|
||||||
|
"spoken_response": trace.spoken_response,
|
||||||
|
"tts_ready_text": trace.tts_ready_text,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
headers = {
|
||||||
|
"Content-Language": route.language,
|
||||||
|
"X-Audio-Format": "pcm",
|
||||||
|
"X-Audio-Sample-Rate": "24000",
|
||||||
|
"X-Audio-Channels": "1",
|
||||||
|
"X-Audio-Sample-Width": "16",
|
||||||
|
**_route_headers(route),
|
||||||
|
}
|
||||||
|
return StreamingResponse(BytesIO(audio), media_type="audio/pcm", headers=headers)
|
||||||
|
|
||||||
|
except Exception as exc:
|
||||||
|
raise HTTPException(status_code=502, detail=str(exc))
|
||||||
37
app/api/config.py
Normal file
37
app/api/config.py
Normal file
|
|
@ -0,0 +1,37 @@
|
||||||
|
from fastapi import APIRouter
|
||||||
|
|
||||||
|
from app.config import settings, active_profile
|
||||||
|
from app.dependencies import (
|
||||||
|
resolve_route,
|
||||||
|
get_audio_router,
|
||||||
|
STT_REGISTRY,
|
||||||
|
LLM_REGISTRY,
|
||||||
|
TTS_REGISTRY,
|
||||||
|
)
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/config")
|
||||||
|
async def get_config():
|
||||||
|
"""Zeigt aktives Profil, aufgeloeste Default-Route und verfuegbare Bausteine.
|
||||||
|
|
||||||
|
Bewusst OHNE Secrets - API-Keys werden nur als 'gesetzt/nicht gesetzt' gemeldet.
|
||||||
|
"""
|
||||||
|
route = resolve_route()
|
||||||
|
audio_router = get_audio_router()
|
||||||
|
return {
|
||||||
|
"profile": active_profile(),
|
||||||
|
"app_env": settings.app_env,
|
||||||
|
"default_route": route.as_dict(),
|
||||||
|
"available": {
|
||||||
|
"stt_providers": sorted(STT_REGISTRY),
|
||||||
|
"llm_providers": sorted(LLM_REGISTRY),
|
||||||
|
"tts_providers": sorted(TTS_REGISTRY),
|
||||||
|
"input_endpoints": [c.model_dump() for c in await audio_router.list_inputs()],
|
||||||
|
"output_endpoints": [c.model_dump() for c in await audio_router.list_outputs()],
|
||||||
|
},
|
||||||
|
"secrets": {
|
||||||
|
"openrouter_api_key_set": bool(settings.openrouter_api_key.strip()),
|
||||||
|
},
|
||||||
|
}
|
||||||
12
app/api/devices.py
Normal file
12
app/api/devices.py
Normal file
|
|
@ -0,0 +1,12 @@
|
||||||
|
from fastapi import APIRouter
|
||||||
|
from app.dependencies import get_audio_router
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
@router.get("/devices")
|
||||||
|
async def list_devices():
|
||||||
|
audio_router = get_audio_router()
|
||||||
|
return {
|
||||||
|
"inputs": [item.model_dump() for item in await audio_router.list_inputs()],
|
||||||
|
"outputs": [item.model_dump() for item in await audio_router.list_outputs()],
|
||||||
|
}
|
||||||
7
app/api/health.py
Normal file
7
app/api/health.py
Normal file
|
|
@ -0,0 +1,7 @@
|
||||||
|
from fastapi import APIRouter
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
@router.get("/health")
|
||||||
|
async def health():
|
||||||
|
return {"status": "ok"}
|
||||||
10
app/api/sessions.py
Normal file
10
app/api/sessions.py
Normal file
|
|
@ -0,0 +1,10 @@
|
||||||
|
from fastapi import APIRouter
|
||||||
|
from app.schemas import SessionRouteRequest
|
||||||
|
from app.dependencies import session_manager
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
@router.post("/sessions/{session_id}/route")
|
||||||
|
async def set_session_route(session_id: str, payload: SessionRouteRequest):
|
||||||
|
session = session_manager.update(session_id, payload.model_dump())
|
||||||
|
return {"session_id": session_id, "route": session}
|
||||||
60
app/api/speak.py
Normal file
60
app/api/speak.py
Normal file
|
|
@ -0,0 +1,60 @@
|
||||||
|
from io import BytesIO
|
||||||
|
|
||||||
|
from fastapi import APIRouter, HTTPException, Query
|
||||||
|
from fastapi.responses import StreamingResponse
|
||||||
|
|
||||||
|
from app.config import settings
|
||||||
|
from app.errors import RoutingError
|
||||||
|
from app.dependencies import (
|
||||||
|
resolve_route,
|
||||||
|
build_orchestrator,
|
||||||
|
resolve_output_endpoint,
|
||||||
|
)
|
||||||
|
from app.schemas import SpeakRequest
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/speak")
|
||||||
|
async def speak(
|
||||||
|
payload: SpeakRequest,
|
||||||
|
session_id: str | None = Query(
|
||||||
|
default=None,
|
||||||
|
description="Optional session id to apply a stored route",
|
||||||
|
),
|
||||||
|
):
|
||||||
|
overrides = {
|
||||||
|
"output_endpoint": payload.output_endpoint,
|
||||||
|
"language": payload.language,
|
||||||
|
"tts_provider": payload.tts_provider,
|
||||||
|
}
|
||||||
|
route = resolve_route(session_id, overrides)
|
||||||
|
voice = payload.voice or settings.openrouter_tts_voice
|
||||||
|
|
||||||
|
try:
|
||||||
|
orchestrator = build_orchestrator(route)
|
||||||
|
output = await resolve_output_endpoint(route)
|
||||||
|
except RoutingError as exc:
|
||||||
|
raise HTTPException(status_code=422, detail=str(exc))
|
||||||
|
|
||||||
|
try:
|
||||||
|
audio = await orchestrator.speak_only(
|
||||||
|
payload.text,
|
||||||
|
voice=voice,
|
||||||
|
language=route.language,
|
||||||
|
output=output,
|
||||||
|
)
|
||||||
|
|
||||||
|
headers = {
|
||||||
|
"Content-Language": route.language,
|
||||||
|
"X-Audio-Format": "pcm",
|
||||||
|
"X-Audio-Sample-Rate": "24000",
|
||||||
|
"X-Audio-Channels": "1",
|
||||||
|
"X-Audio-Sample-Width": "16",
|
||||||
|
"X-Output-Endpoint": route.output_endpoint,
|
||||||
|
"X-TTS-Provider": route.tts_provider,
|
||||||
|
}
|
||||||
|
return StreamingResponse(BytesIO(audio), media_type="audio/pcm", headers=headers)
|
||||||
|
|
||||||
|
except Exception as exc:
|
||||||
|
raise HTTPException(status_code=502, detail=str(exc))
|
||||||
50
app/api/transcribe.py
Normal file
50
app/api/transcribe.py
Normal file
|
|
@ -0,0 +1,50 @@
|
||||||
|
from fastapi import APIRouter, File, Form, HTTPException, Query, UploadFile
|
||||||
|
|
||||||
|
from app.errors import RoutingError
|
||||||
|
from app.dependencies import (
|
||||||
|
resolve_route,
|
||||||
|
build_orchestrator,
|
||||||
|
resolve_input_endpoint,
|
||||||
|
)
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/transcribe")
|
||||||
|
async def transcribe(
|
||||||
|
file: UploadFile = File(...),
|
||||||
|
language: str | None = Form(default=None),
|
||||||
|
input_endpoint: str | None = Form(default=None),
|
||||||
|
stt_provider: str | None = Form(default=None),
|
||||||
|
session_id: str | None = Query(
|
||||||
|
default=None,
|
||||||
|
description="Optional session id to apply a stored route",
|
||||||
|
),
|
||||||
|
):
|
||||||
|
overrides = {
|
||||||
|
"input_endpoint": input_endpoint,
|
||||||
|
"language": language,
|
||||||
|
"stt_provider": stt_provider,
|
||||||
|
}
|
||||||
|
route = resolve_route(session_id, overrides)
|
||||||
|
|
||||||
|
try:
|
||||||
|
orchestrator = build_orchestrator(route)
|
||||||
|
source = await resolve_input_endpoint(route)
|
||||||
|
except RoutingError as exc:
|
||||||
|
raise HTTPException(status_code=422, detail=str(exc))
|
||||||
|
|
||||||
|
content = await file.read()
|
||||||
|
suffix = (file.filename or "audio.wav").rsplit(".", 1)[-1].lower()
|
||||||
|
|
||||||
|
try:
|
||||||
|
trace = await orchestrator.transcribe_only(
|
||||||
|
content,
|
||||||
|
fmt=suffix,
|
||||||
|
language=route.language,
|
||||||
|
input=source,
|
||||||
|
)
|
||||||
|
except Exception as exc:
|
||||||
|
raise HTTPException(status_code=502, detail=str(exc))
|
||||||
|
|
||||||
|
return {"route": route.as_dict(), "trace": trace.model_dump()}
|
||||||
0
app/audio/__init__.py
Normal file
0
app/audio/__init__.py
Normal file
0
app/audio/endpoints/__init__.py
Normal file
0
app/audio/endpoints/__init__.py
Normal file
0
app/audio/endpoints/input/__init__.py
Normal file
0
app/audio/endpoints/input/__init__.py
Normal file
17
app/audio/endpoints/input/base.py
Normal file
17
app/audio/endpoints/input/base.py
Normal file
|
|
@ -0,0 +1,17 @@
|
||||||
|
from abc import ABC, abstractmethod
|
||||||
|
from app.schemas import AudioChunk, EndpointCapabilities
|
||||||
|
|
||||||
|
class AudioInputEndpoint(ABC):
|
||||||
|
endpoint_id: str
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
async def capabilities(self) -> EndpointCapabilities: ...
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
async def open(self) -> None: ...
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
async def read_chunk(self) -> AudioChunk: ...
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
async def close(self) -> None: ...
|
||||||
23
app/audio/endpoints/input/bluetooth.py
Normal file
23
app/audio/endpoints/input/bluetooth.py
Normal file
|
|
@ -0,0 +1,23 @@
|
||||||
|
from app.audio.endpoints.input.base import AudioInputEndpoint
|
||||||
|
from app.schemas import AudioChunk, EndpointCapabilities
|
||||||
|
|
||||||
|
class BluetoothInput(AudioInputEndpoint):
|
||||||
|
endpoint_id = "bt-headset-01"
|
||||||
|
|
||||||
|
async def capabilities(self) -> EndpointCapabilities:
|
||||||
|
return EndpointCapabilities(
|
||||||
|
id=self.endpoint_id,
|
||||||
|
kind="bluetooth",
|
||||||
|
direction="input",
|
||||||
|
latency_class="medium",
|
||||||
|
bluetooth=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
async def open(self) -> None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def read_chunk(self) -> AudioChunk:
|
||||||
|
return AudioChunk(data=b"", format="wav", timestamp_ms=0)
|
||||||
|
|
||||||
|
async def close(self) -> None:
|
||||||
|
return None
|
||||||
24
app/audio/endpoints/input/local_default.py
Normal file
24
app/audio/endpoints/input/local_default.py
Normal file
|
|
@ -0,0 +1,24 @@
|
||||||
|
from app.audio.endpoints.input.base import AudioInputEndpoint
|
||||||
|
from app.schemas import AudioChunk, EndpointCapabilities
|
||||||
|
|
||||||
|
class LocalDefaultInput(AudioInputEndpoint):
|
||||||
|
endpoint_id = "local-default-mic"
|
||||||
|
|
||||||
|
async def capabilities(self) -> EndpointCapabilities:
|
||||||
|
return EndpointCapabilities(
|
||||||
|
id=self.endpoint_id,
|
||||||
|
kind="local-default",
|
||||||
|
direction="input",
|
||||||
|
latency_class="low",
|
||||||
|
supports_barge_in=True,
|
||||||
|
default=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
async def open(self) -> None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def read_chunk(self) -> AudioChunk:
|
||||||
|
return AudioChunk(data=b"", format="wav", timestamp_ms=0)
|
||||||
|
|
||||||
|
async def close(self) -> None:
|
||||||
|
return None
|
||||||
25
app/audio/endpoints/input/mobile_webrtc.py
Normal file
25
app/audio/endpoints/input/mobile_webrtc.py
Normal file
|
|
@ -0,0 +1,25 @@
|
||||||
|
from app.audio.endpoints.input.base import AudioInputEndpoint
|
||||||
|
from app.schemas import AudioChunk, EndpointCapabilities
|
||||||
|
|
||||||
|
class MobileWebRTCInput(AudioInputEndpoint):
|
||||||
|
endpoint_id = "mobile-webrtc-client"
|
||||||
|
|
||||||
|
async def capabilities(self) -> EndpointCapabilities:
|
||||||
|
return EndpointCapabilities(
|
||||||
|
id=self.endpoint_id,
|
||||||
|
kind="mobile-webrtc",
|
||||||
|
direction="input",
|
||||||
|
latency_class="low",
|
||||||
|
networked=True,
|
||||||
|
mobile=True,
|
||||||
|
supports_barge_in=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
async def open(self) -> None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def read_chunk(self) -> AudioChunk:
|
||||||
|
return AudioChunk(data=b"", format="wav", timestamp_ms=0)
|
||||||
|
|
||||||
|
async def close(self) -> None:
|
||||||
|
return None
|
||||||
24
app/audio/endpoints/input/mobile_ws.py
Normal file
24
app/audio/endpoints/input/mobile_ws.py
Normal file
|
|
@ -0,0 +1,24 @@
|
||||||
|
from app.audio.endpoints.input.base import AudioInputEndpoint
|
||||||
|
from app.schemas import AudioChunk, EndpointCapabilities
|
||||||
|
|
||||||
|
class MobileWebSocketInput(AudioInputEndpoint):
|
||||||
|
endpoint_id = "mobile-ws-client"
|
||||||
|
|
||||||
|
async def capabilities(self) -> EndpointCapabilities:
|
||||||
|
return EndpointCapabilities(
|
||||||
|
id=self.endpoint_id,
|
||||||
|
kind="mobile-ws",
|
||||||
|
direction="input",
|
||||||
|
latency_class="medium",
|
||||||
|
networked=True,
|
||||||
|
mobile=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
async def open(self) -> None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def read_chunk(self) -> AudioChunk:
|
||||||
|
return AudioChunk(data=b"", format="wav", timestamp_ms=0)
|
||||||
|
|
||||||
|
async def close(self) -> None:
|
||||||
|
return None
|
||||||
0
app/audio/endpoints/output/__init__.py
Normal file
0
app/audio/endpoints/output/__init__.py
Normal file
20
app/audio/endpoints/output/base.py
Normal file
20
app/audio/endpoints/output/base.py
Normal file
|
|
@ -0,0 +1,20 @@
|
||||||
|
from abc import ABC, abstractmethod
|
||||||
|
from app.schemas import AudioChunk, EndpointCapabilities
|
||||||
|
|
||||||
|
class AudioOutputEndpoint(ABC):
|
||||||
|
endpoint_id: str
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
async def capabilities(self) -> EndpointCapabilities: ...
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
async def open(self) -> None: ...
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
async def write_chunk(self, chunk: AudioChunk) -> None: ...
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
async def flush(self) -> None: ...
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
async def close(self) -> None: ...
|
||||||
26
app/audio/endpoints/output/bluetooth.py
Normal file
26
app/audio/endpoints/output/bluetooth.py
Normal file
|
|
@ -0,0 +1,26 @@
|
||||||
|
from app.audio.endpoints.output.base import AudioOutputEndpoint
|
||||||
|
from app.schemas import AudioChunk, EndpointCapabilities
|
||||||
|
|
||||||
|
class BluetoothOutput(AudioOutputEndpoint):
|
||||||
|
endpoint_id = "bt-speaker-01"
|
||||||
|
|
||||||
|
async def capabilities(self) -> EndpointCapabilities:
|
||||||
|
return EndpointCapabilities(
|
||||||
|
id=self.endpoint_id,
|
||||||
|
kind="bluetooth",
|
||||||
|
direction="output",
|
||||||
|
latency_class="medium",
|
||||||
|
bluetooth=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
async def open(self) -> None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def write_chunk(self, chunk: AudioChunk) -> None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def flush(self) -> None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def close(self) -> None:
|
||||||
|
return None
|
||||||
26
app/audio/endpoints/output/local_default.py
Normal file
26
app/audio/endpoints/output/local_default.py
Normal file
|
|
@ -0,0 +1,26 @@
|
||||||
|
from app.audio.endpoints.output.base import AudioOutputEndpoint
|
||||||
|
from app.schemas import AudioChunk, EndpointCapabilities
|
||||||
|
|
||||||
|
class LocalDefaultOutput(AudioOutputEndpoint):
|
||||||
|
endpoint_id = "local-default-speaker"
|
||||||
|
|
||||||
|
async def capabilities(self) -> EndpointCapabilities:
|
||||||
|
return EndpointCapabilities(
|
||||||
|
id=self.endpoint_id,
|
||||||
|
kind="local-default",
|
||||||
|
direction="output",
|
||||||
|
latency_class="low",
|
||||||
|
default=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
async def open(self) -> None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def write_chunk(self, chunk: AudioChunk) -> None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def flush(self) -> None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def close(self) -> None:
|
||||||
|
return None
|
||||||
28
app/audio/endpoints/output/loopback.py
Normal file
28
app/audio/endpoints/output/loopback.py
Normal file
|
|
@ -0,0 +1,28 @@
|
||||||
|
from app.audio.endpoints.output.base import AudioOutputEndpoint
|
||||||
|
from app.schemas import AudioChunk, EndpointCapabilities
|
||||||
|
|
||||||
|
class LoopbackOutput(AudioOutputEndpoint):
|
||||||
|
endpoint_id = "loopback-output"
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
self.chunks = []
|
||||||
|
|
||||||
|
async def capabilities(self) -> EndpointCapabilities:
|
||||||
|
return EndpointCapabilities(
|
||||||
|
id=self.endpoint_id,
|
||||||
|
kind="loopback",
|
||||||
|
direction="output",
|
||||||
|
latency_class="low",
|
||||||
|
)
|
||||||
|
|
||||||
|
async def open(self) -> None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def write_chunk(self, chunk: AudioChunk) -> None:
|
||||||
|
self.chunks.append(chunk)
|
||||||
|
|
||||||
|
async def flush(self) -> None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def close(self) -> None:
|
||||||
|
return None
|
||||||
27
app/audio/endpoints/output/mobile_webrtc.py
Normal file
27
app/audio/endpoints/output/mobile_webrtc.py
Normal file
|
|
@ -0,0 +1,27 @@
|
||||||
|
from app.audio.endpoints.output.base import AudioOutputEndpoint
|
||||||
|
from app.schemas import AudioChunk, EndpointCapabilities
|
||||||
|
|
||||||
|
class MobileWebRTCOutput(AudioOutputEndpoint):
|
||||||
|
endpoint_id = "mobile-webrtc-client"
|
||||||
|
|
||||||
|
async def capabilities(self) -> EndpointCapabilities:
|
||||||
|
return EndpointCapabilities(
|
||||||
|
id=self.endpoint_id,
|
||||||
|
kind="mobile-webrtc",
|
||||||
|
direction="output",
|
||||||
|
latency_class="low",
|
||||||
|
networked=True,
|
||||||
|
mobile=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
async def open(self) -> None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def write_chunk(self, chunk: AudioChunk) -> None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def flush(self) -> None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def close(self) -> None:
|
||||||
|
return None
|
||||||
27
app/audio/endpoints/output/mobile_ws.py
Normal file
27
app/audio/endpoints/output/mobile_ws.py
Normal file
|
|
@ -0,0 +1,27 @@
|
||||||
|
from app.audio.endpoints.output.base import AudioOutputEndpoint
|
||||||
|
from app.schemas import AudioChunk, EndpointCapabilities
|
||||||
|
|
||||||
|
class MobileWebSocketOutput(AudioOutputEndpoint):
|
||||||
|
endpoint_id = "mobile-ws-client"
|
||||||
|
|
||||||
|
async def capabilities(self) -> EndpointCapabilities:
|
||||||
|
return EndpointCapabilities(
|
||||||
|
id=self.endpoint_id,
|
||||||
|
kind="mobile-ws",
|
||||||
|
direction="output",
|
||||||
|
latency_class="medium",
|
||||||
|
networked=True,
|
||||||
|
mobile=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
async def open(self) -> None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def write_chunk(self, chunk: AudioChunk) -> None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def flush(self) -> None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def close(self) -> None:
|
||||||
|
return None
|
||||||
38
app/audio/router.py
Normal file
38
app/audio/router.py
Normal file
|
|
@ -0,0 +1,38 @@
|
||||||
|
from app.errors import UnknownEndpointError
|
||||||
|
|
||||||
|
|
||||||
|
class AudioRouter:
|
||||||
|
def __init__(self, inputs, outputs):
|
||||||
|
self.inputs = inputs
|
||||||
|
self.outputs = outputs
|
||||||
|
|
||||||
|
async def list_inputs(self):
|
||||||
|
return [await endpoint.capabilities() for endpoint in self.inputs]
|
||||||
|
|
||||||
|
async def list_outputs(self):
|
||||||
|
return [await endpoint.capabilities() for endpoint in self.outputs]
|
||||||
|
|
||||||
|
async def select_input(self, preferred: str | None = None):
|
||||||
|
return await self._select(self.inputs, preferred, direction="input")
|
||||||
|
|
||||||
|
async def select_output(self, preferred: str | None = None):
|
||||||
|
return await self._select(self.outputs, preferred, direction="output")
|
||||||
|
|
||||||
|
async def _select(self, endpoints, preferred: str | None, direction: str):
|
||||||
|
# Capabilities einmal sammeln (Basis fuer spaetere capability-basierte Auswahl).
|
||||||
|
pairs = [(endpoint, await endpoint.capabilities()) for endpoint in endpoints]
|
||||||
|
|
||||||
|
if preferred:
|
||||||
|
for endpoint, caps in pairs:
|
||||||
|
if caps.id == preferred or caps.kind == preferred:
|
||||||
|
return endpoint
|
||||||
|
# Angefragter Endpunkt existiert nicht -> KEIN stiller Default-Fallback.
|
||||||
|
available = sorted({caps.id for _, caps in pairs} | {caps.kind for _, caps in pairs})
|
||||||
|
raise UnknownEndpointError(
|
||||||
|
f"Unbekannter {direction}-Endpunkt {preferred!r}. Verfuegbar: {available}"
|
||||||
|
)
|
||||||
|
|
||||||
|
for endpoint, caps in pairs:
|
||||||
|
if caps.default:
|
||||||
|
return endpoint
|
||||||
|
raise RuntimeError(f"Kein {direction}-Standardendpunkt verfuegbar")
|
||||||
11
app/audio/transport_router.py
Normal file
11
app/audio/transport_router.py
Normal file
|
|
@ -0,0 +1,11 @@
|
||||||
|
class TransportRouter:
|
||||||
|
def __init__(self, local_registry: dict, remote_registry: dict):
|
||||||
|
self.local_registry = local_registry
|
||||||
|
self.remote_registry = remote_registry
|
||||||
|
|
||||||
|
def resolve(self, module_type: str, provider_name: str):
|
||||||
|
if provider_name in self.local_registry.get(module_type, {}):
|
||||||
|
return self.local_registry[module_type][provider_name]
|
||||||
|
if provider_name in self.remote_registry.get(module_type, {}):
|
||||||
|
return self.remote_registry[module_type][provider_name]
|
||||||
|
raise KeyError(f"Unknown provider: {module_type}/{provider_name}")
|
||||||
146
app/config.py
Normal file
146
app/config.py
Normal file
|
|
@ -0,0 +1,146 @@
|
||||||
|
import os
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
try:
|
||||||
|
import tomllib # Python >= 3.11 (stdlib)
|
||||||
|
except ModuleNotFoundError: # pragma: no cover - Fallback fuer aeltere Interpreter
|
||||||
|
import tomli as tomllib # type: ignore
|
||||||
|
|
||||||
|
from pydantic.fields import FieldInfo
|
||||||
|
from pydantic_settings import (
|
||||||
|
BaseSettings,
|
||||||
|
PydanticBaseSettingsSource,
|
||||||
|
SettingsConfigDict,
|
||||||
|
)
|
||||||
|
|
||||||
|
BASE_DIR = Path(__file__).resolve().parent.parent
|
||||||
|
ENV_FILE = BASE_DIR / ".env"
|
||||||
|
DEFAULT_CONFIG_FILE = BASE_DIR / "config" / "voice-assistant.toml"
|
||||||
|
|
||||||
|
|
||||||
|
def _setting_lookup(key: str) -> str | None:
|
||||||
|
"""Liest einen Steuer-Schluessel: echte Umgebung zuerst, dann die .env-Datei.
|
||||||
|
|
||||||
|
Noetig fuer VA_PROFILE/VA_CONFIG_FILE, weil diese gebraucht werden, BEVOR
|
||||||
|
pydantic-settings die .env laedt - und .env-Werte sonst nicht in os.environ stehen.
|
||||||
|
"""
|
||||||
|
value = os.getenv(key)
|
||||||
|
if value is not None:
|
||||||
|
return value
|
||||||
|
try:
|
||||||
|
from dotenv import dotenv_values
|
||||||
|
except ModuleNotFoundError: # pragma: no cover
|
||||||
|
return None
|
||||||
|
if ENV_FILE.is_file():
|
||||||
|
return dotenv_values(ENV_FILE).get(key)
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _config_file_path() -> Path:
|
||||||
|
return Path(_setting_lookup("VA_CONFIG_FILE") or str(DEFAULT_CONFIG_FILE))
|
||||||
|
|
||||||
|
|
||||||
|
def active_profile() -> str | None:
|
||||||
|
"""Name des aktiven Profils (VA_PROFILE) aus Umgebung oder .env, falls gesetzt."""
|
||||||
|
profile = _setting_lookup("VA_PROFILE")
|
||||||
|
return profile.strip() or None if profile else None
|
||||||
|
|
||||||
|
|
||||||
|
def load_profile_config() -> dict:
|
||||||
|
"""Liest die zentrale TOML-Config und merged [defaults] + [profiles.<VA_PROFILE>].
|
||||||
|
|
||||||
|
- Fehlt die Datei, gilt ein leeres dict (nur ENV/Defaults greifen) - kein Fehler,
|
||||||
|
damit reine Cloud-Deployments ohne Datei (nur ENV) funktionieren.
|
||||||
|
- Ein gesetztes, aber unbekanntes VA_PROFILE ist ein Konfigurationsfehler.
|
||||||
|
"""
|
||||||
|
path = _config_file_path()
|
||||||
|
if not path.is_file():
|
||||||
|
return {}
|
||||||
|
|
||||||
|
with path.open("rb") as handle:
|
||||||
|
data = tomllib.load(handle)
|
||||||
|
|
||||||
|
merged: dict = dict(data.get("defaults", {}))
|
||||||
|
|
||||||
|
profile = active_profile()
|
||||||
|
if profile:
|
||||||
|
profiles = data.get("profiles", {})
|
||||||
|
if profile not in profiles:
|
||||||
|
raise ValueError(
|
||||||
|
f"Unbekanntes VA_PROFILE {profile!r}. "
|
||||||
|
f"Verfuegbar: {sorted(profiles)}"
|
||||||
|
)
|
||||||
|
merged.update(profiles[profile])
|
||||||
|
|
||||||
|
return merged
|
||||||
|
|
||||||
|
|
||||||
|
class TomlProfileSource(PydanticBaseSettingsSource):
|
||||||
|
"""Settings-Quelle aus der zentralen TOML-Config (inkl. aktivem Profil).
|
||||||
|
|
||||||
|
Liegt in der Praezedenz unter ENV/.env, aber ueber den eingebauten Defaults.
|
||||||
|
Es werden nur Schluessel durchgereicht, die auch als Settings-Feld existieren.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, settings_cls):
|
||||||
|
super().__init__(settings_cls)
|
||||||
|
raw = load_profile_config()
|
||||||
|
known = set(settings_cls.model_fields)
|
||||||
|
self._values = {
|
||||||
|
key.lower(): value
|
||||||
|
for key, value in raw.items()
|
||||||
|
if key.lower() in known
|
||||||
|
}
|
||||||
|
|
||||||
|
def get_field_value(self, field: FieldInfo, field_name: str):
|
||||||
|
if field_name in self._values:
|
||||||
|
return self._values[field_name], field_name, False
|
||||||
|
return None, field_name, False
|
||||||
|
|
||||||
|
def __call__(self) -> dict:
|
||||||
|
return dict(self._values)
|
||||||
|
|
||||||
|
|
||||||
|
class Settings(BaseSettings):
|
||||||
|
app_env: str = "dev"
|
||||||
|
host: str = "0.0.0.0"
|
||||||
|
port: int = 8080
|
||||||
|
log_level: str = "info"
|
||||||
|
openrouter_api_key: str = ""
|
||||||
|
openrouter_stt_model: str = "openai/whisper-large-v3"
|
||||||
|
openrouter_tts_model: str = "openai/gpt-4o-mini-tts"
|
||||||
|
openrouter_tts_voice: str = "alloy"
|
||||||
|
openrouter_llm_model: str = "openai/gpt-4.1-mini"
|
||||||
|
default_language: str = "de"
|
||||||
|
default_input_endpoint: str = "local-default"
|
||||||
|
default_output_endpoint: str = "local-default"
|
||||||
|
default_stt_provider: str = "openrouter"
|
||||||
|
default_llm_provider: str = "local-openai-compatible"
|
||||||
|
default_tts_provider: str = "openrouter"
|
||||||
|
local_llm_base_url: str = "http://127.0.0.1:11434/v1"
|
||||||
|
local_llm_api_key: str = "dummy"
|
||||||
|
local_llm_model: str = "llama3.1"
|
||||||
|
model_config = SettingsConfigDict(
|
||||||
|
env_file=ENV_FILE, case_sensitive=False, extra="ignore"
|
||||||
|
)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def settings_customise_sources(
|
||||||
|
cls,
|
||||||
|
settings_cls,
|
||||||
|
init_settings,
|
||||||
|
env_settings,
|
||||||
|
dotenv_settings,
|
||||||
|
file_secret_settings,
|
||||||
|
):
|
||||||
|
# Praezedenz (frueher = hoeher): init > ENV > .env > TOML/Profil > Defaults
|
||||||
|
return (
|
||||||
|
init_settings,
|
||||||
|
env_settings,
|
||||||
|
dotenv_settings,
|
||||||
|
TomlProfileSource(settings_cls),
|
||||||
|
file_secret_settings,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
settings = Settings()
|
||||||
0
app/core/__init__.py
Normal file
0
app/core/__init__.py
Normal file
105
app/core/orchestrator.py
Normal file
105
app/core/orchestrator.py
Normal file
|
|
@ -0,0 +1,105 @@
|
||||||
|
from app.schemas import AudioChunk, PipelineTrace
|
||||||
|
|
||||||
|
# Festes Ausgabeformat der TTS-Stufe (s16le PCM, 24 kHz, mono).
|
||||||
|
TTS_AUDIO_FORMAT = "pcm"
|
||||||
|
TTS_SAMPLE_RATE = 24000
|
||||||
|
TTS_CHANNELS = 1
|
||||||
|
|
||||||
|
|
||||||
|
class Orchestrator:
|
||||||
|
def __init__(self, stt, llm, tts, input_cleaner, spoken_adapter, tts_normalizer):
|
||||||
|
self.stt = stt
|
||||||
|
self.llm = llm
|
||||||
|
self.tts = tts
|
||||||
|
self.input_cleaner = input_cleaner
|
||||||
|
self.spoken_adapter = spoken_adapter
|
||||||
|
self.tts_normalizer = tts_normalizer
|
||||||
|
|
||||||
|
async def _emit_to_output(self, audio: bytes, output) -> None:
|
||||||
|
"""Schreibt das synthetisierte Audio durch den gewaehlten Output-Endpunkt.
|
||||||
|
|
||||||
|
Der HTTP-Stream bleibt davon unberuehrt (additiv). Bei lokalen Geraeten
|
||||||
|
ist write_chunk heute ein no-op; LoopbackOutput sammelt die Chunks.
|
||||||
|
"""
|
||||||
|
if output is None:
|
||||||
|
return
|
||||||
|
chunk = AudioChunk(
|
||||||
|
data=audio,
|
||||||
|
sample_rate=TTS_SAMPLE_RATE,
|
||||||
|
channels=TTS_CHANNELS,
|
||||||
|
format=TTS_AUDIO_FORMAT,
|
||||||
|
)
|
||||||
|
await output.open()
|
||||||
|
try:
|
||||||
|
await output.write_chunk(chunk)
|
||||||
|
await output.flush()
|
||||||
|
finally:
|
||||||
|
await output.close()
|
||||||
|
|
||||||
|
async def transcribe_only(
|
||||||
|
self,
|
||||||
|
audio_bytes: bytes,
|
||||||
|
fmt: str,
|
||||||
|
language: str | None = None,
|
||||||
|
input=None,
|
||||||
|
):
|
||||||
|
trace = PipelineTrace()
|
||||||
|
# input dient hier nur der Validierung/Metadaten; das Audio kommt per Upload.
|
||||||
|
if input is not None:
|
||||||
|
await input.capabilities()
|
||||||
|
trace.raw_transcript = await self.stt.transcribe(
|
||||||
|
audio_bytes,
|
||||||
|
fmt=fmt,
|
||||||
|
language=language,
|
||||||
|
)
|
||||||
|
trace.cleaned_transcript = await self.input_cleaner.run(
|
||||||
|
trace.raw_transcript or ""
|
||||||
|
)
|
||||||
|
return trace
|
||||||
|
|
||||||
|
async def speak_only(
|
||||||
|
self,
|
||||||
|
text: str,
|
||||||
|
voice: str | None = None,
|
||||||
|
language: str | None = None,
|
||||||
|
output=None,
|
||||||
|
):
|
||||||
|
spoken = await self.spoken_adapter.run(text, language=language)
|
||||||
|
normalized = await self.tts_normalizer.run(spoken, language=language)
|
||||||
|
audio = await self.tts.synthesize(normalized, voice=voice)
|
||||||
|
await self._emit_to_output(audio, output)
|
||||||
|
return audio
|
||||||
|
|
||||||
|
async def chat_text(
|
||||||
|
self,
|
||||||
|
text: str,
|
||||||
|
language: str | None = None,
|
||||||
|
voice: str | None = None,
|
||||||
|
output=None,
|
||||||
|
):
|
||||||
|
trace = PipelineTrace()
|
||||||
|
|
||||||
|
trace.raw_transcript = text
|
||||||
|
trace.cleaned_transcript = await self.input_cleaner.run(text or "")
|
||||||
|
|
||||||
|
trace.semantic_response = await self.llm.complete(
|
||||||
|
trace.cleaned_transcript or ""
|
||||||
|
)
|
||||||
|
if not trace.semantic_response:
|
||||||
|
raise RuntimeError("LLM returned an empty response")
|
||||||
|
|
||||||
|
trace.spoken_response = await self.spoken_adapter.run(
|
||||||
|
trace.semantic_response,
|
||||||
|
language=language,
|
||||||
|
)
|
||||||
|
trace.tts_ready_text = await self.tts_normalizer.run(
|
||||||
|
trace.spoken_response,
|
||||||
|
language=language,
|
||||||
|
)
|
||||||
|
|
||||||
|
audio = await self.tts.synthesize(
|
||||||
|
trace.tts_ready_text,
|
||||||
|
voice=voice,
|
||||||
|
)
|
||||||
|
await self._emit_to_output(audio, output)
|
||||||
|
return trace, audio
|
||||||
11
app/core/session_manager.py
Normal file
11
app/core/session_manager.py
Normal file
|
|
@ -0,0 +1,11 @@
|
||||||
|
class SessionManager:
|
||||||
|
def __init__(self):
|
||||||
|
self._sessions = {}
|
||||||
|
|
||||||
|
def get(self, session_id: str) -> dict:
|
||||||
|
return self._sessions.setdefault(session_id, {})
|
||||||
|
|
||||||
|
def update(self, session_id: str, values: dict) -> dict:
|
||||||
|
session = self.get(session_id)
|
||||||
|
session.update({k: v for k, v in values.items() if v is not None})
|
||||||
|
return session
|
||||||
187
app/dependencies.py
Normal file
187
app/dependencies.py
Normal file
|
|
@ -0,0 +1,187 @@
|
||||||
|
from dataclasses import dataclass
|
||||||
|
|
||||||
|
from app.config import Settings, settings
|
||||||
|
from app.errors import UnknownComponentError
|
||||||
|
from app.audio.router import AudioRouter
|
||||||
|
from app.audio.endpoints.input.local_default import LocalDefaultInput
|
||||||
|
from app.audio.endpoints.input.bluetooth import BluetoothInput
|
||||||
|
from app.audio.endpoints.input.mobile_ws import MobileWebSocketInput
|
||||||
|
from app.audio.endpoints.input.mobile_webrtc import MobileWebRTCInput
|
||||||
|
from app.audio.endpoints.output.local_default import LocalDefaultOutput
|
||||||
|
from app.audio.endpoints.output.bluetooth import BluetoothOutput
|
||||||
|
from app.audio.endpoints.output.mobile_ws import MobileWebSocketOutput
|
||||||
|
from app.audio.endpoints.output.mobile_webrtc import MobileWebRTCOutput
|
||||||
|
from app.audio.endpoints.output.loopback import LoopbackOutput
|
||||||
|
from app.providers.stt.openrouter import OpenRouterSTTProvider
|
||||||
|
from app.providers.stt.faster_whisper import FasterWhisperProvider
|
||||||
|
from app.providers.llm.local_openai_compatible import LocalOpenAICompatibleLLM
|
||||||
|
from app.providers.llm.openrouter import OpenRouterLLMProvider
|
||||||
|
from app.providers.tts.openrouter import OpenRouterTTSProvider
|
||||||
|
from app.providers.tts.chatterbox import ChatterboxTTSProvider
|
||||||
|
from app.providers.tts.piper import PiperTTSProvider
|
||||||
|
from app.pipeline.input_cleaner import InputCleaner
|
||||||
|
from app.pipeline.spoken_response_adapter import SpokenResponseAdapter
|
||||||
|
from app.pipeline.tts_normalizer import TTSNormalizer
|
||||||
|
from app.core.orchestrator import Orchestrator
|
||||||
|
from app.core.session_manager import SessionManager
|
||||||
|
|
||||||
|
session_manager = SessionManager()
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Provider-Registries: Modul austauschbar via Name, ohne Kern-Code zu aendern.
|
||||||
|
# Ein neuer Provider = ein Eintrag. Unbekannter Name -> UnknownComponentError.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
STT_REGISTRY = {
|
||||||
|
"openrouter": lambda s: OpenRouterSTTProvider(s.openrouter_api_key, s.openrouter_stt_model),
|
||||||
|
"faster-whisper": lambda s: FasterWhisperProvider(),
|
||||||
|
}
|
||||||
|
|
||||||
|
LLM_REGISTRY = {
|
||||||
|
"openrouter": lambda s: OpenRouterLLMProvider(s.openrouter_api_key, s.openrouter_llm_model),
|
||||||
|
"local-openai-compatible": lambda s: LocalOpenAICompatibleLLM(
|
||||||
|
s.local_llm_base_url, s.local_llm_api_key, s.local_llm_model
|
||||||
|
),
|
||||||
|
}
|
||||||
|
|
||||||
|
TTS_REGISTRY = {
|
||||||
|
"openrouter": lambda s: OpenRouterTTSProvider(
|
||||||
|
s.openrouter_api_key, s.openrouter_tts_model, s.openrouter_tts_voice
|
||||||
|
),
|
||||||
|
"chatterbox": lambda s: ChatterboxTTSProvider(),
|
||||||
|
"piper": lambda s: PiperTTSProvider(),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _from_registry(registry: dict, name: str, kind: str, cfg: Settings):
|
||||||
|
try:
|
||||||
|
factory = registry[name]
|
||||||
|
except KeyError as exc:
|
||||||
|
raise UnknownComponentError(
|
||||||
|
f"Unbekannter {kind}-Provider {name!r}. Verfuegbar: {sorted(registry)}"
|
||||||
|
) from exc
|
||||||
|
return factory(cfg)
|
||||||
|
|
||||||
|
|
||||||
|
def get_stt_provider(name: str | None = None, cfg: Settings = settings):
|
||||||
|
return _from_registry(STT_REGISTRY, name or cfg.default_stt_provider, "STT", cfg)
|
||||||
|
|
||||||
|
|
||||||
|
def get_llm_provider(name: str | None = None, cfg: Settings = settings):
|
||||||
|
return _from_registry(LLM_REGISTRY, name or cfg.default_llm_provider, "LLM", cfg)
|
||||||
|
|
||||||
|
|
||||||
|
def get_tts_provider(name: str | None = None, cfg: Settings = settings):
|
||||||
|
return _from_registry(TTS_REGISTRY, name or cfg.default_tts_provider, "TTS", cfg)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Audio-Router: Modul-Singleton, damit zustandsbehaftete Endpunkte
|
||||||
|
# (z. B. LoopbackOutput.chunks) ueber Requests hinweg stabil bleiben.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
_audio_router: AudioRouter | None = None
|
||||||
|
|
||||||
|
|
||||||
|
def get_audio_router() -> AudioRouter:
|
||||||
|
global _audio_router
|
||||||
|
if _audio_router is None:
|
||||||
|
_audio_router = AudioRouter(
|
||||||
|
inputs=[
|
||||||
|
LocalDefaultInput(),
|
||||||
|
BluetoothInput(),
|
||||||
|
MobileWebSocketInput(),
|
||||||
|
MobileWebRTCInput(),
|
||||||
|
],
|
||||||
|
outputs=[
|
||||||
|
LocalDefaultOutput(),
|
||||||
|
BluetoothOutput(),
|
||||||
|
MobileWebSocketOutput(),
|
||||||
|
MobileWebRTCOutput(),
|
||||||
|
LoopbackOutput(),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
return _audio_router
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Session-Routing und einheitliche Route-Aufloesung ueber alle Achsen.
|
||||||
|
# Praezedenz: Settings-Defaults < Session-Route < Request-Overrides.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
ROUTE_KEYS = (
|
||||||
|
"input_endpoint",
|
||||||
|
"output_endpoint",
|
||||||
|
"stt_provider",
|
||||||
|
"llm_provider",
|
||||||
|
"tts_provider",
|
||||||
|
"language",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class ResolvedRoute:
|
||||||
|
input_endpoint: str
|
||||||
|
output_endpoint: str
|
||||||
|
stt_provider: str
|
||||||
|
llm_provider: str
|
||||||
|
tts_provider: str
|
||||||
|
language: str
|
||||||
|
|
||||||
|
def as_dict(self) -> dict:
|
||||||
|
return {
|
||||||
|
"input_endpoint": self.input_endpoint,
|
||||||
|
"output_endpoint": self.output_endpoint,
|
||||||
|
"stt_provider": self.stt_provider,
|
||||||
|
"llm_provider": self.llm_provider,
|
||||||
|
"tts_provider": self.tts_provider,
|
||||||
|
"language": self.language,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def get_session_route(session_id: str | None) -> dict:
|
||||||
|
"""Liefert die gespeicherte Route einer Session (leeres dict ohne session_id)."""
|
||||||
|
return session_manager.get(session_id) if session_id else {}
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_route(
|
||||||
|
session_id: str | None = None,
|
||||||
|
overrides: dict | None = None,
|
||||||
|
cfg: Settings = settings,
|
||||||
|
) -> ResolvedRoute:
|
||||||
|
"""Loest die effektive Route aus Defaults, Session und Request-Overrides auf."""
|
||||||
|
resolved = {
|
||||||
|
"input_endpoint": cfg.default_input_endpoint,
|
||||||
|
"output_endpoint": cfg.default_output_endpoint,
|
||||||
|
"stt_provider": cfg.default_stt_provider,
|
||||||
|
"llm_provider": cfg.default_llm_provider,
|
||||||
|
"tts_provider": cfg.default_tts_provider,
|
||||||
|
"language": cfg.default_language,
|
||||||
|
}
|
||||||
|
|
||||||
|
session_route = get_session_route(session_id)
|
||||||
|
request_overrides = overrides or {}
|
||||||
|
|
||||||
|
for layer in (session_route, request_overrides):
|
||||||
|
for key in ROUTE_KEYS:
|
||||||
|
value = layer.get(key)
|
||||||
|
if value is not None:
|
||||||
|
resolved[key] = value
|
||||||
|
|
||||||
|
return ResolvedRoute(**resolved)
|
||||||
|
|
||||||
|
|
||||||
|
def build_orchestrator(route: ResolvedRoute, cfg: Settings = settings) -> Orchestrator:
|
||||||
|
return Orchestrator(
|
||||||
|
stt=get_stt_provider(route.stt_provider, cfg),
|
||||||
|
llm=get_llm_provider(route.llm_provider, cfg),
|
||||||
|
tts=get_tts_provider(route.tts_provider, cfg),
|
||||||
|
input_cleaner=InputCleaner(),
|
||||||
|
spoken_adapter=SpokenResponseAdapter(),
|
||||||
|
tts_normalizer=TTSNormalizer(),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def resolve_output_endpoint(route: ResolvedRoute):
|
||||||
|
return await get_audio_router().select_output(route.output_endpoint)
|
||||||
|
|
||||||
|
|
||||||
|
async def resolve_input_endpoint(route: ResolvedRoute):
|
||||||
|
return await get_audio_router().select_input(route.input_endpoint)
|
||||||
13
app/errors.py
Normal file
13
app/errors.py
Normal file
|
|
@ -0,0 +1,13 @@
|
||||||
|
class RoutingError(Exception):
|
||||||
|
"""Basis fuer Fehler bei der Routing-/Komponentenauswahl.
|
||||||
|
|
||||||
|
Wird in der API-Schicht zu HTTP 422 uebersetzt (Client-Konfigurationsfehler).
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
class UnknownComponentError(RoutingError):
|
||||||
|
"""Unbekannter Provider-Name fuer STT, LLM oder TTS."""
|
||||||
|
|
||||||
|
|
||||||
|
class UnknownEndpointError(RoutingError):
|
||||||
|
"""Ein angefragter Audio-Endpunkt (input/output) existiert nicht."""
|
||||||
17
app/main.py
Normal file
17
app/main.py
Normal file
|
|
@ -0,0 +1,17 @@
|
||||||
|
from fastapi import FastAPI
|
||||||
|
from app.api.health import router as health_router
|
||||||
|
from app.api.chat import router as chat_router
|
||||||
|
from app.api.transcribe import router as transcribe_router
|
||||||
|
from app.api.speak import router as speak_router
|
||||||
|
from app.api.devices import router as devices_router
|
||||||
|
from app.api.sessions import router as sessions_router
|
||||||
|
from app.api.config import router as config_router
|
||||||
|
|
||||||
|
app = FastAPI(title="Voice Assistant Gateway")
|
||||||
|
app.include_router(health_router)
|
||||||
|
app.include_router(chat_router, prefix="/api")
|
||||||
|
app.include_router(transcribe_router, prefix="/api")
|
||||||
|
app.include_router(speak_router, prefix="/api")
|
||||||
|
app.include_router(devices_router, prefix="/api")
|
||||||
|
app.include_router(sessions_router, prefix="/api")
|
||||||
|
app.include_router(config_router, prefix="/api")
|
||||||
0
app/pipeline/__init__.py
Normal file
0
app/pipeline/__init__.py
Normal file
4
app/pipeline/input_cleaner.py
Normal file
4
app/pipeline/input_cleaner.py
Normal file
|
|
@ -0,0 +1,4 @@
|
||||||
|
class InputCleaner:
|
||||||
|
async def run(self, text: str) -> str:
|
||||||
|
cleaned = " ".join(text.strip().split())
|
||||||
|
return cleaned.replace(" äh ", " ").replace(" hm ", " ")
|
||||||
37
app/pipeline/spoken_response_adapter.py
Normal file
37
app/pipeline/spoken_response_adapter.py
Normal file
|
|
@ -0,0 +1,37 @@
|
||||||
|
import re
|
||||||
|
|
||||||
|
|
||||||
|
class SpokenResponseAdapter:
|
||||||
|
async def run(self, text: str, language: str = "de") -> str:
|
||||||
|
if not text:
|
||||||
|
return ""
|
||||||
|
|
||||||
|
text = text.strip()
|
||||||
|
|
||||||
|
# Markdown / Formatierung entfernen
|
||||||
|
text = re.sub(r"```[\s\S]*?```", " ", text) # code blocks
|
||||||
|
text = re.sub(r"`([^`]*)`", r"\1", text) # inline code
|
||||||
|
text = re.sub(r"\[([^\]]+)\]\([^)]+\)", r"\1", text) # markdown links
|
||||||
|
text = re.sub(r"[*_~#>]+", " ", text) # markdown symbols
|
||||||
|
|
||||||
|
# Listen entschärfen
|
||||||
|
text = re.sub(r"(?m)^\s*[-•]\s+", "", text)
|
||||||
|
text = re.sub(r"(?m)^\s*\d+\.\s+", "", text)
|
||||||
|
|
||||||
|
# Mehrfache Leerzeichen / Zeilenumbrüche glätten
|
||||||
|
text = re.sub(r"\s+", " ", text).strip()
|
||||||
|
|
||||||
|
# Für Voice natürlicher machen: Doppelpunkte/Semikolons etwas beruhigen,
|
||||||
|
# aber Uhrzeiten/Verhältnisse (10:30) nicht zerstören -> nur am Wortende ersetzen.
|
||||||
|
text = re.sub(r"[:;](?=\s|$)", ",", text)
|
||||||
|
|
||||||
|
# Klammern meist nicht gut für TTS
|
||||||
|
text = text.replace("(", ", ")
|
||||||
|
text = text.replace(")", " ")
|
||||||
|
|
||||||
|
# Abschlusspunktion sicherstellen
|
||||||
|
if text and not text.endswith((".", "!", "?")):
|
||||||
|
text += "."
|
||||||
|
|
||||||
|
return text
|
||||||
|
|
||||||
59
app/pipeline/tts_normalizer.py
Normal file
59
app/pipeline/tts_normalizer.py
Normal file
|
|
@ -0,0 +1,59 @@
|
||||||
|
import re
|
||||||
|
|
||||||
|
|
||||||
|
class TTSNormalizer:
|
||||||
|
async def run(self, text: str, language: str = "de") -> str:
|
||||||
|
if not text:
|
||||||
|
return ""
|
||||||
|
|
||||||
|
normalized = text
|
||||||
|
|
||||||
|
if language == "de":
|
||||||
|
replacements = {
|
||||||
|
"24/7": "vierundzwanzig sieben",
|
||||||
|
"&": " und ",
|
||||||
|
"%": " Prozent",
|
||||||
|
"€": " Euro",
|
||||||
|
"$": " Dollar",
|
||||||
|
"km/h": " Kilometer pro Stunde",
|
||||||
|
"z.B.": "zum Beispiel",
|
||||||
|
"bzw.": "beziehungsweise",
|
||||||
|
"u.a.": "unter anderem",
|
||||||
|
"ca.": "circa",
|
||||||
|
}
|
||||||
|
else:
|
||||||
|
replacements = {
|
||||||
|
"24/7": "twenty four seven",
|
||||||
|
"&": " and ",
|
||||||
|
"%": " percent",
|
||||||
|
"€": " euros",
|
||||||
|
"$": " dollars",
|
||||||
|
"km/h": " kilometers per hour",
|
||||||
|
"e.g.": "for example",
|
||||||
|
"i.e.": "that is",
|
||||||
|
}
|
||||||
|
|
||||||
|
for old, new in replacements.items():
|
||||||
|
normalized = normalized.replace(old, new)
|
||||||
|
|
||||||
|
# Slashes zwischen Wörtern/Zahlen sprachfreundlicher machen
|
||||||
|
normalized = re.sub(r"(\w)/(\w)", r"\1 oder \2", normalized)
|
||||||
|
|
||||||
|
# Datums-/Versions-/Bereichsstriche etwas entschärfen
|
||||||
|
normalized = normalized.replace("–", " bis ")
|
||||||
|
normalized = normalized.replace("—", ", ")
|
||||||
|
normalized = normalized.replace(" - ", ", ")
|
||||||
|
|
||||||
|
# URLs und E-Mails nicht roh vorlesen
|
||||||
|
normalized = re.sub(r"https?://\S+", "Link", normalized)
|
||||||
|
normalized = re.sub(r"\b[\w\.-]+@[\w\.-]+\.\w+\b", "E-Mail-Adresse", normalized)
|
||||||
|
|
||||||
|
# Mehrfache Leerzeichen glätten
|
||||||
|
normalized = re.sub(r"\s+", " ", normalized).strip()
|
||||||
|
|
||||||
|
if normalized and not normalized.endswith((".", "!", "?")):
|
||||||
|
normalized += "."
|
||||||
|
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
|
||||||
0
app/providers/__init__.py
Normal file
0
app/providers/__init__.py
Normal file
0
app/providers/llm/__init__.py
Normal file
0
app/providers/llm/__init__.py
Normal file
5
app/providers/llm/base.py
Normal file
5
app/providers/llm/base.py
Normal file
|
|
@ -0,0 +1,5 @@
|
||||||
|
from abc import ABC, abstractmethod
|
||||||
|
|
||||||
|
class LLMProvider(ABC):
|
||||||
|
@abstractmethod
|
||||||
|
async def complete(self, text: str, session_id: str | None = None) -> str: ...
|
||||||
24
app/providers/llm/local_openai_compatible.py
Normal file
24
app/providers/llm/local_openai_compatible.py
Normal file
|
|
@ -0,0 +1,24 @@
|
||||||
|
import httpx
|
||||||
|
from app.providers.llm.base import LLMProvider
|
||||||
|
|
||||||
|
class LocalOpenAICompatibleLLM(LLMProvider):
|
||||||
|
def __init__(self, base_url: str, api_key: str, model: str):
|
||||||
|
self.base_url = base_url.rstrip("/")
|
||||||
|
self.api_key = api_key
|
||||||
|
self.model = model
|
||||||
|
|
||||||
|
async def complete(self, text: str, session_id: str | None = None) -> str:
|
||||||
|
payload = {
|
||||||
|
"model": self.model,
|
||||||
|
"messages": [{"role": "user", "content": text}],
|
||||||
|
"temperature": 0.3,
|
||||||
|
}
|
||||||
|
async with httpx.AsyncClient(timeout=120) as client:
|
||||||
|
response = await client.post(
|
||||||
|
f"{self.base_url}/chat/completions",
|
||||||
|
headers={"Authorization": f"Bearer {self.api_key}"},
|
||||||
|
json=payload,
|
||||||
|
)
|
||||||
|
response.raise_for_status()
|
||||||
|
data = response.json()
|
||||||
|
return data["choices"][0]["message"]["content"]
|
||||||
101
app/providers/llm/openrouter.py
Normal file
101
app/providers/llm/openrouter.py
Normal file
|
|
@ -0,0 +1,101 @@
|
||||||
|
import httpx
|
||||||
|
|
||||||
|
from app.providers.llm.base import LLMProvider
|
||||||
|
|
||||||
|
|
||||||
|
SYSTEM_PROMPT = """
|
||||||
|
You are a voice assistant for spoken conversations with older adults.
|
||||||
|
|
||||||
|
Speak naturally, clearly, and calmly.
|
||||||
|
Use short, simple sentences.
|
||||||
|
Prefer plain everyday language over technical wording.
|
||||||
|
Answer in the same language as the user, unless the user asks to switch languages.
|
||||||
|
|
||||||
|
Important response rules:
|
||||||
|
- Output plain text only.
|
||||||
|
- No markdown.
|
||||||
|
- No bullet points.
|
||||||
|
- No numbered lists.
|
||||||
|
- No tables.
|
||||||
|
- No code.
|
||||||
|
- No emojis.
|
||||||
|
- No URLs unless the user explicitly asks for one.
|
||||||
|
- Do not use asterisks, hashtags, or formatting symbols.
|
||||||
|
- Do not write headings.
|
||||||
|
- Do not use long disclaimers.
|
||||||
|
|
||||||
|
Voice style rules:
|
||||||
|
- Sound helpful, warm, and patient.
|
||||||
|
- Keep answers brief by default: 1 to 3 short sentences.
|
||||||
|
- If more detail is needed, explain step by step in natural spoken sentences.
|
||||||
|
- Ask at most one follow-up question at a time.
|
||||||
|
- If the answer contains several items, present them as natural speech, not as a list.
|
||||||
|
- Use wording that sounds good when spoken aloud.
|
||||||
|
- Avoid abbreviations when possible.
|
||||||
|
- Avoid symbols when words are better.
|
||||||
|
- Prefer complete spoken forms for dates, times, and numbers when useful.
|
||||||
|
|
||||||
|
Safety and honesty rules:
|
||||||
|
- If you are unsure, say so briefly and clearly.
|
||||||
|
- Do not invent facts.
|
||||||
|
- If current real-world information is needed and unavailable, say that clearly.
|
||||||
|
|
||||||
|
Always optimize your answer for listening, not for reading.
|
||||||
|
""".strip()
|
||||||
|
|
||||||
|
|
||||||
|
class OpenRouterLLMProvider(LLMProvider):
|
||||||
|
def __init__(self, api_key: str, model: str):
|
||||||
|
self.api_key = (api_key or "").strip()
|
||||||
|
self.model = (model or "").strip()
|
||||||
|
|
||||||
|
async def complete(self, text: str, session_id: str | None = None) -> str:
|
||||||
|
if not self.api_key:
|
||||||
|
raise ValueError("OPENROUTER_API_KEY is empty")
|
||||||
|
if not self.model:
|
||||||
|
raise ValueError("OPENROUTER_LLM_MODEL is empty")
|
||||||
|
if not text or not text.strip():
|
||||||
|
raise ValueError("LLM input text is empty")
|
||||||
|
|
||||||
|
payload = {
|
||||||
|
"model": self.model,
|
||||||
|
"messages": [
|
||||||
|
{"role": "system", "content": SYSTEM_PROMPT},
|
||||||
|
{"role": "user", "content": text.strip()},
|
||||||
|
],
|
||||||
|
}
|
||||||
|
|
||||||
|
timeout = httpx.Timeout(connect=10.0, read=120.0, write=30.0, pool=10.0)
|
||||||
|
|
||||||
|
async with httpx.AsyncClient(timeout=timeout) as client:
|
||||||
|
try:
|
||||||
|
response = await client.post(
|
||||||
|
"https://openrouter.ai/api/v1/chat/completions",
|
||||||
|
headers={
|
||||||
|
"Authorization": f"Bearer {self.api_key}",
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
},
|
||||||
|
json=payload,
|
||||||
|
)
|
||||||
|
response.raise_for_status()
|
||||||
|
except httpx.HTTPStatusError as exc:
|
||||||
|
raise RuntimeError(
|
||||||
|
f"OpenRouter LLM error {exc.response.status_code}: {exc.response.text}"
|
||||||
|
) from exc
|
||||||
|
except httpx.TimeoutException as exc:
|
||||||
|
raise RuntimeError("OpenRouter LLM timeout") from exc
|
||||||
|
except httpx.HTTPError as exc:
|
||||||
|
raise RuntimeError(f"OpenRouter LLM transport error: {exc}") from exc
|
||||||
|
|
||||||
|
data = response.json()
|
||||||
|
|
||||||
|
try:
|
||||||
|
content = data["choices"][0]["message"]["content"]
|
||||||
|
except (KeyError, IndexError, TypeError) as exc:
|
||||||
|
raise RuntimeError(f"Unexpected OpenRouter LLM response: {data}") from exc
|
||||||
|
|
||||||
|
if not content or not str(content).strip():
|
||||||
|
raise RuntimeError("OpenRouter LLM returned empty content")
|
||||||
|
|
||||||
|
return str(content).strip()
|
||||||
|
|
||||||
0
app/providers/stt/__init__.py
Normal file
0
app/providers/stt/__init__.py
Normal file
5
app/providers/stt/base.py
Normal file
5
app/providers/stt/base.py
Normal file
|
|
@ -0,0 +1,5 @@
|
||||||
|
from abc import ABC, abstractmethod
|
||||||
|
|
||||||
|
class STTProvider(ABC):
|
||||||
|
@abstractmethod
|
||||||
|
async def transcribe(self, audio_bytes: bytes, fmt: str, language: str | None = None) -> str: ...
|
||||||
5
app/providers/stt/faster_whisper.py
Normal file
5
app/providers/stt/faster_whisper.py
Normal file
|
|
@ -0,0 +1,5 @@
|
||||||
|
from app.providers.stt.base import STTProvider
|
||||||
|
|
||||||
|
class FasterWhisperProvider(STTProvider):
|
||||||
|
async def transcribe(self, audio_bytes: bytes, fmt: str, language: str | None = None) -> str:
|
||||||
|
return "[local transcription placeholder]"
|
||||||
46
app/providers/stt/openrouter.py
Normal file
46
app/providers/stt/openrouter.py
Normal file
|
|
@ -0,0 +1,46 @@
|
||||||
|
import httpx
|
||||||
|
|
||||||
|
from app.providers.stt.base import STTProvider
|
||||||
|
|
||||||
|
|
||||||
|
class OpenRouterSTTProvider(STTProvider):
|
||||||
|
def __init__(self, api_key: str, model: str):
|
||||||
|
self.api_key = (api_key or "").strip()
|
||||||
|
self.model = (model or "").strip()
|
||||||
|
|
||||||
|
async def transcribe(self, audio_bytes: bytes, fmt: str, language: str | None = None) -> str:
|
||||||
|
if not self.api_key:
|
||||||
|
raise ValueError("OPENROUTER_API_KEY is empty")
|
||||||
|
if not self.model:
|
||||||
|
raise ValueError("OPENROUTER_STT_MODEL is empty")
|
||||||
|
if not audio_bytes:
|
||||||
|
raise ValueError("STT input audio is empty")
|
||||||
|
|
||||||
|
# OpenAI-kompatibler /audio/transcriptions-Endpunkt erwartet multipart/form-data
|
||||||
|
# mit binärem file-Feld, nicht JSON mit base64.
|
||||||
|
files = {"file": (f"audio.{fmt}", audio_bytes, f"audio/{fmt}")}
|
||||||
|
data: dict[str, str] = {"model": self.model}
|
||||||
|
if language:
|
||||||
|
data["language"] = language
|
||||||
|
|
||||||
|
timeout = httpx.Timeout(connect=10.0, read=120.0, write=30.0, pool=10.0)
|
||||||
|
|
||||||
|
async with httpx.AsyncClient(timeout=timeout) as client:
|
||||||
|
try:
|
||||||
|
response = await client.post(
|
||||||
|
"https://openrouter.ai/api/v1/audio/transcriptions",
|
||||||
|
headers={"Authorization": f"Bearer {self.api_key}"},
|
||||||
|
files=files,
|
||||||
|
data=data,
|
||||||
|
)
|
||||||
|
response.raise_for_status()
|
||||||
|
except httpx.HTTPStatusError as exc:
|
||||||
|
raise RuntimeError(
|
||||||
|
f"OpenRouter STT error {exc.response.status_code}: {exc.response.text}"
|
||||||
|
) from exc
|
||||||
|
except httpx.TimeoutException as exc:
|
||||||
|
raise RuntimeError("OpenRouter STT timeout") from exc
|
||||||
|
except httpx.HTTPError as exc:
|
||||||
|
raise RuntimeError(f"OpenRouter STT transport error: {exc}") from exc
|
||||||
|
|
||||||
|
return response.json().get("text", "")
|
||||||
0
app/providers/tts/__init__.py
Normal file
0
app/providers/tts/__init__.py
Normal file
5
app/providers/tts/base.py
Normal file
5
app/providers/tts/base.py
Normal file
|
|
@ -0,0 +1,5 @@
|
||||||
|
from abc import ABC, abstractmethod
|
||||||
|
|
||||||
|
class TTSProvider(ABC):
|
||||||
|
@abstractmethod
|
||||||
|
async def synthesize(self, text: str, voice: str | None = None, audio_format: str = "pcm") -> bytes: ...
|
||||||
5
app/providers/tts/chatterbox.py
Normal file
5
app/providers/tts/chatterbox.py
Normal file
|
|
@ -0,0 +1,5 @@
|
||||||
|
from app.providers.tts.base import TTSProvider
|
||||||
|
|
||||||
|
class ChatterboxTTSProvider(TTSProvider):
|
||||||
|
async def synthesize(self, text: str, voice: str | None = None, audio_format: str = "pcm") -> bytes:
|
||||||
|
return b""
|
||||||
62
app/providers/tts/openrouter.py
Normal file
62
app/providers/tts/openrouter.py
Normal file
|
|
@ -0,0 +1,62 @@
|
||||||
|
import httpx
|
||||||
|
|
||||||
|
from app.providers.tts.base import TTSProvider
|
||||||
|
|
||||||
|
|
||||||
|
class OpenRouterTTSProvider(TTSProvider):
|
||||||
|
def __init__(self, api_key: str, model: str, voice: str):
|
||||||
|
self.api_key = (api_key or "").strip()
|
||||||
|
self.model = (model or "").strip()
|
||||||
|
self.voice = (voice or "").strip()
|
||||||
|
|
||||||
|
async def synthesize(
|
||||||
|
self,
|
||||||
|
text: str,
|
||||||
|
voice: str | None = None,
|
||||||
|
audio_format: str = "pcm",
|
||||||
|
) -> bytes:
|
||||||
|
if not self.api_key:
|
||||||
|
raise ValueError("OPENROUTER_API_KEY is empty")
|
||||||
|
if not self.model:
|
||||||
|
raise ValueError("OPENROUTER_TTS_MODEL is empty")
|
||||||
|
if not text or not text.strip():
|
||||||
|
raise ValueError("TTS input text is empty")
|
||||||
|
|
||||||
|
effective_voice = (voice or self.voice).strip()
|
||||||
|
if not effective_voice:
|
||||||
|
raise ValueError("TTS voice is required for OpenRouter TTS")
|
||||||
|
|
||||||
|
payload = {
|
||||||
|
"model": self.model,
|
||||||
|
"input": text.strip(),
|
||||||
|
"voice": effective_voice,
|
||||||
|
"response_format": audio_format,
|
||||||
|
}
|
||||||
|
|
||||||
|
timeout = httpx.Timeout(connect=10.0, read=120.0, write=30.0, pool=10.0)
|
||||||
|
|
||||||
|
async with httpx.AsyncClient(timeout=timeout) as client:
|
||||||
|
try:
|
||||||
|
response = await client.post(
|
||||||
|
"https://openrouter.ai/api/v1/audio/speech",
|
||||||
|
headers={
|
||||||
|
"Authorization": f"Bearer {self.api_key}",
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
},
|
||||||
|
json=payload,
|
||||||
|
)
|
||||||
|
response.raise_for_status()
|
||||||
|
except httpx.HTTPStatusError as exc:
|
||||||
|
raise RuntimeError(
|
||||||
|
f"OpenRouter TTS error {exc.response.status_code}: {exc.response.text}"
|
||||||
|
) from exc
|
||||||
|
except httpx.TimeoutException as exc:
|
||||||
|
raise RuntimeError("OpenRouter TTS timeout") from exc
|
||||||
|
except httpx.HTTPError as exc:
|
||||||
|
raise RuntimeError(f"OpenRouter TTS transport error: {exc}") from exc
|
||||||
|
|
||||||
|
if not response.content:
|
||||||
|
raise RuntimeError("OpenRouter TTS returned empty audio content")
|
||||||
|
|
||||||
|
return response.content
|
||||||
|
|
||||||
5
app/providers/tts/piper.py
Normal file
5
app/providers/tts/piper.py
Normal file
|
|
@ -0,0 +1,5 @@
|
||||||
|
from app.providers.tts.base import TTSProvider
|
||||||
|
|
||||||
|
class PiperTTSProvider(TTSProvider):
|
||||||
|
async def synthesize(self, text: str, voice: str | None = None, audio_format: str = "pcm") -> bytes:
|
||||||
|
return b""
|
||||||
71
app/schemas.py
Normal file
71
app/schemas.py
Normal file
|
|
@ -0,0 +1,71 @@
|
||||||
|
from typing import Literal
|
||||||
|
from pydantic import BaseModel, Field
|
||||||
|
|
||||||
|
|
||||||
|
class EndpointCapabilities(BaseModel):
|
||||||
|
id: str
|
||||||
|
kind: str
|
||||||
|
direction: Literal["input", "output"]
|
||||||
|
sample_rate: int = 16000
|
||||||
|
channels: int = 1
|
||||||
|
latency_class: Literal["low", "medium", "high"] = "medium"
|
||||||
|
supports_aec: bool = False
|
||||||
|
supports_barge_in: bool = False
|
||||||
|
networked: bool = False
|
||||||
|
bluetooth: bool = False
|
||||||
|
mobile: bool = False
|
||||||
|
default: bool = False
|
||||||
|
|
||||||
|
|
||||||
|
class AudioChunk(BaseModel):
|
||||||
|
data: bytes
|
||||||
|
sample_rate: int = 16000
|
||||||
|
channels: int = 1
|
||||||
|
format: str = "wav"
|
||||||
|
timestamp_ms: int = 0
|
||||||
|
|
||||||
|
|
||||||
|
class PipelineTrace(BaseModel):
|
||||||
|
raw_transcript: str | None = None
|
||||||
|
cleaned_transcript: str | None = None
|
||||||
|
semantic_response: str | None = None
|
||||||
|
spoken_response: str | None = None
|
||||||
|
tts_ready_text: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class SpeakRequest(BaseModel):
|
||||||
|
text: str = Field(min_length=1)
|
||||||
|
voice: str | None = None
|
||||||
|
language: str | None = None
|
||||||
|
output_endpoint: str | None = None
|
||||||
|
tts_provider: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class ChatRequest(BaseModel):
|
||||||
|
text: str = Field(min_length=1)
|
||||||
|
input_endpoint: str | None = None
|
||||||
|
output_endpoint: str | None = None
|
||||||
|
language: str | None = None
|
||||||
|
voice: str | None = None
|
||||||
|
stt_provider: str | None = None
|
||||||
|
llm_provider: str | None = None
|
||||||
|
tts_provider: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class SessionRouteRequest(BaseModel):
|
||||||
|
input_endpoint: str | None = None
|
||||||
|
output_endpoint: str | None = None
|
||||||
|
stt_provider: str | None = None
|
||||||
|
llm_provider: str | None = None
|
||||||
|
tts_provider: str | None = None
|
||||||
|
language: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class RouteInfo(BaseModel):
|
||||||
|
input_endpoint: str
|
||||||
|
output_endpoint: str
|
||||||
|
stt_provider: str
|
||||||
|
llm_provider: str
|
||||||
|
tts_provider: str
|
||||||
|
language: str
|
||||||
|
|
||||||
0
app/utils/__init__.py
Normal file
0
app/utils/__init__.py
Normal file
76
chat_client.py
Normal file
76
chat_client.py
Normal file
|
|
@ -0,0 +1,76 @@
|
||||||
|
import io
|
||||||
|
import wave
|
||||||
|
import requests
|
||||||
|
import soundfile as sf
|
||||||
|
import numpy as np
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
|
||||||
|
GATEWAY_URL = "http://localhost:8003"
|
||||||
|
CHAT_ENDPOINT = f"{GATEWAY_URL}/api/chat"
|
||||||
|
|
||||||
|
# feste Annahmen für Gemini 3.1 Flash TTS über OpenRouter
|
||||||
|
SAMPLE_RATE = 24000
|
||||||
|
CHANNELS = 1
|
||||||
|
SAMPLE_WIDTH = 2 # 16-bit PCM
|
||||||
|
|
||||||
|
|
||||||
|
def pcm_to_wav(pcm_bytes: bytes, wav_path: str) -> None:
|
||||||
|
"""Rohes s16le-PCM in eine WAV-Datei schreiben."""
|
||||||
|
with wave.open(wav_path, "wb") as wf:
|
||||||
|
wf.setnchannels(CHANNELS)
|
||||||
|
wf.setsampwidth(SAMPLE_WIDTH)
|
||||||
|
wf.setframerate(SAMPLE_RATE)
|
||||||
|
wf.writeframes(pcm_bytes)
|
||||||
|
|
||||||
|
|
||||||
|
def play_wav(wav_path: str) -> None:
|
||||||
|
"""WAV-Datei abspielen (ffplay oder aplay/mpv, je nach System)."""
|
||||||
|
for cmd in (
|
||||||
|
["ffplay", "-nodisp", "-autoexit", wav_path],
|
||||||
|
["aplay", wav_path],
|
||||||
|
["mpv", wav_path],
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
subprocess.run(cmd, check=True)
|
||||||
|
return
|
||||||
|
except (FileNotFoundError, subprocess.CalledProcessError):
|
||||||
|
continue
|
||||||
|
print(f"Konnte keine geeignete Player-CLI finden für {wav_path}", file=sys.stderr)
|
||||||
|
|
||||||
|
|
||||||
|
def chat_and_play(text: str, language: str = "de") -> None:
|
||||||
|
payload = {"text": text, "language": language}
|
||||||
|
|
||||||
|
resp = requests.post(
|
||||||
|
CHAT_ENDPOINT,
|
||||||
|
json=payload,
|
||||||
|
stream=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
if not resp.ok:
|
||||||
|
print("HTTP", resp.status_code)
|
||||||
|
print(resp.text)
|
||||||
|
return
|
||||||
|
|
||||||
|
pcm_bytes = b"".join(resp.iter_content(chunk_size=8192))
|
||||||
|
|
||||||
|
# Hinweis: Den Text-Trace (Transkript/Antwort) liefert /api/chat nur im JSON,
|
||||||
|
# wenn man ?debug=true anhängt - nicht als Header im Audio-Stream.
|
||||||
|
print("Audio-Format:", resp.headers.get("X-Audio-Format"))
|
||||||
|
print("Sample-Rate:", resp.headers.get("X-Audio-Sample-Rate"))
|
||||||
|
|
||||||
|
wav_path = "chat_reply.wav"
|
||||||
|
pcm_to_wav(pcm_bytes, wav_path)
|
||||||
|
print(f"WAV gespeichert unter {wav_path}")
|
||||||
|
play_wav(wav_path)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
if len(sys.argv) > 1:
|
||||||
|
user_text = " ".join(sys.argv[1:])
|
||||||
|
else:
|
||||||
|
user_text = "Wie wird das Wetter morgen in Bünde?"
|
||||||
|
|
||||||
|
chat_and_play(user_text, language="de")
|
||||||
|
|
||||||
44
config/voice-assistant.example.toml
Normal file
44
config/voice-assistant.example.toml
Normal file
|
|
@ -0,0 +1,44 @@
|
||||||
|
# Zentrale Konfiguration des Voice-Assistant-Gateways.
|
||||||
|
#
|
||||||
|
# WICHTIG: Secrets (API-Keys) gehoeren NICHT in diese Datei -> ausschliesslich
|
||||||
|
# ueber Umgebungsvariablen (z. B. OPENROUTER_API_KEY).
|
||||||
|
#
|
||||||
|
# Praezedenz (hoeher gewinnt):
|
||||||
|
# eingebaute Defaults < diese TOML-Datei < ENV/.env < Session-Route < Request
|
||||||
|
#
|
||||||
|
# Aktives Profil waehlen via ENV: VA_PROFILE=local-dev | hybrid | cloud
|
||||||
|
# Eigenen Pfad setzen via ENV: VA_CONFIG_FILE=/pfad/zu/voice-assistant.toml
|
||||||
|
#
|
||||||
|
# Diese Datei nach config/voice-assistant.toml kopieren und anpassen.
|
||||||
|
|
||||||
|
# Basiswerte, die fuer alle Profile gelten (von Profilen ueberschreibbar).
|
||||||
|
[defaults]
|
||||||
|
default_language = "de"
|
||||||
|
default_input_endpoint = "local-default"
|
||||||
|
default_output_endpoint = "local-default"
|
||||||
|
|
||||||
|
openrouter_stt_model = "openai/whisper-large-v3"
|
||||||
|
openrouter_tts_model = "openai/gpt-4o-mini-tts"
|
||||||
|
openrouter_tts_voice = "alloy"
|
||||||
|
openrouter_llm_model = "openai/gpt-4.1-mini"
|
||||||
|
|
||||||
|
local_llm_base_url = "http://127.0.0.1:11434/v1"
|
||||||
|
local_llm_model = "llama3.1"
|
||||||
|
|
||||||
|
# Reines lokales Setup (eigene Hardware/KI) - z. B. fuer Entwicklung/Offline-Test.
|
||||||
|
[profiles.local-dev]
|
||||||
|
default_stt_provider = "faster-whisper"
|
||||||
|
default_llm_provider = "local-openai-compatible"
|
||||||
|
default_tts_provider = "piper"
|
||||||
|
|
||||||
|
# Hybrid: STT/TTS remote, Haupt-LLM lokal.
|
||||||
|
[profiles.hybrid]
|
||||||
|
default_stt_provider = "openrouter"
|
||||||
|
default_llm_provider = "local-openai-compatible"
|
||||||
|
default_tts_provider = "openrouter"
|
||||||
|
|
||||||
|
# Voll-Cloud: alle KI-Module remote (Standard fuer den produktiven vHost-Betrieb).
|
||||||
|
[profiles.cloud]
|
||||||
|
default_stt_provider = "openrouter"
|
||||||
|
default_llm_provider = "openrouter"
|
||||||
|
default_tts_provider = "openrouter"
|
||||||
3
deploy/voice-assistant.env.example
Normal file
3
deploy/voice-assistant.env.example
Normal file
|
|
@ -0,0 +1,3 @@
|
||||||
|
HOST=0.0.0.0
|
||||||
|
PORT=8080
|
||||||
|
OPENROUTER_API_KEY=
|
||||||
15
deploy/voice-assistant.service
Normal file
15
deploy/voice-assistant.service
Normal file
|
|
@ -0,0 +1,15 @@
|
||||||
|
[Unit]
|
||||||
|
Description=Voice Assistant Gateway
|
||||||
|
After=network.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=simple
|
||||||
|
User=voice
|
||||||
|
WorkingDirectory=/opt/voice-assistant
|
||||||
|
EnvironmentFile=/etc/voice-assistant/voice-assistant.env
|
||||||
|
ExecStart=/opt/voice-assistant/.venv/bin/uvicorn app.main:app --host ${HOST} --port ${PORT}
|
||||||
|
Restart=always
|
||||||
|
RestartSec=2
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
15
docker-compose.yml
Normal file
15
docker-compose.yml
Normal file
|
|
@ -0,0 +1,15 @@
|
||||||
|
services:
|
||||||
|
voice-assistant:
|
||||||
|
build: .
|
||||||
|
ports:
|
||||||
|
- "${PORT:-8080}:${PORT:-8080}"
|
||||||
|
env_file:
|
||||||
|
- .env
|
||||||
|
environment:
|
||||||
|
HOST: "${HOST:-0.0.0.0}"
|
||||||
|
PORT: "${PORT:-8080}"
|
||||||
|
# Secret aus der Shell-Umgebung durchreichen (nicht aus .env), z. B. export in ~/.bashrc
|
||||||
|
OPENROUTER_API_KEY: "${OPENROUTER_API_KEY:?OPENROUTER_API_KEY ist nicht gesetzt}"
|
||||||
|
command: >
|
||||||
|
sh -c 'uvicorn app.main:app --host "$${HOST}" --port "$${PORT}"'
|
||||||
|
restart: unless-stopped
|
||||||
28
pyproject.toml
Normal file
28
pyproject.toml
Normal file
|
|
@ -0,0 +1,28 @@
|
||||||
|
[project]
|
||||||
|
name = "voice-assistant-gateway"
|
||||||
|
version = "0.1.0"
|
||||||
|
description = "Modular voice assistant gateway with pluggable audio endpoints and provider adapters"
|
||||||
|
readme = "README.md"
|
||||||
|
requires-python = ">=3.11"
|
||||||
|
dependencies = [
|
||||||
|
"fastapi>=0.116.0",
|
||||||
|
"uvicorn[standard]>=0.35.0",
|
||||||
|
"httpx>=0.28.0",
|
||||||
|
"pydantic>=2.11.0",
|
||||||
|
"pydantic-settings>=2.10.0",
|
||||||
|
"python-multipart>=0.0.20"
|
||||||
|
]
|
||||||
|
|
||||||
|
[project.optional-dependencies]
|
||||||
|
test = [
|
||||||
|
"pytest>=8.0"
|
||||||
|
]
|
||||||
|
|
||||||
|
[build-system]
|
||||||
|
requires = ["setuptools>=68", "wheel"]
|
||||||
|
build-backend = "setuptools.build_meta"
|
||||||
|
|
||||||
|
[tool.setuptools.packages.find]
|
||||||
|
where = ["."]
|
||||||
|
include = ["app*"]
|
||||||
|
exclude = ["deploy*", "tests*"]
|
||||||
22
tests/conftest.py
Normal file
22
tests/conftest.py
Normal file
|
|
@ -0,0 +1,22 @@
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
import app.dependencies as deps
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def reset_state():
|
||||||
|
"""Isoliert den Singleton-Audio-Router (Loopback-Buffer) und Sessions je Test."""
|
||||||
|
deps._audio_router = None
|
||||||
|
deps.session_manager._sessions.clear()
|
||||||
|
yield
|
||||||
|
deps._audio_router = None
|
||||||
|
deps.session_manager._sessions.clear()
|
||||||
|
|
||||||
|
|
||||||
|
def loopback_output():
|
||||||
|
"""Liefert den LoopbackOutput aus dem aktuellen Singleton-Router."""
|
||||||
|
router = deps.get_audio_router()
|
||||||
|
for endpoint in router.outputs:
|
||||||
|
if type(endpoint).__name__ == "LoopbackOutput":
|
||||||
|
return endpoint
|
||||||
|
raise AssertionError("LoopbackOutput nicht gefunden")
|
||||||
43
tests/test_audio_router.py
Normal file
43
tests/test_audio_router.py
Normal file
|
|
@ -0,0 +1,43 @@
|
||||||
|
import asyncio
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from app.audio.router import AudioRouter
|
||||||
|
from app.audio.endpoints.input.local_default import LocalDefaultInput
|
||||||
|
from app.audio.endpoints.input.bluetooth import BluetoothInput
|
||||||
|
from app.audio.endpoints.output.local_default import LocalDefaultOutput
|
||||||
|
from app.audio.endpoints.output.loopback import LoopbackOutput
|
||||||
|
from app.errors import UnknownEndpointError
|
||||||
|
|
||||||
|
|
||||||
|
def make_router():
|
||||||
|
return AudioRouter(
|
||||||
|
inputs=[LocalDefaultInput(), BluetoothInput()],
|
||||||
|
outputs=[LocalDefaultOutput(), LoopbackOutput()],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_select_default_output():
|
||||||
|
router = make_router()
|
||||||
|
endpoint = asyncio.run(router.select_output())
|
||||||
|
caps = asyncio.run(endpoint.capabilities())
|
||||||
|
assert caps.default is True
|
||||||
|
assert caps.kind == "local-default"
|
||||||
|
|
||||||
|
|
||||||
|
def test_select_output_by_kind():
|
||||||
|
router = make_router()
|
||||||
|
endpoint = asyncio.run(router.select_output("loopback"))
|
||||||
|
assert asyncio.run(endpoint.capabilities()).kind == "loopback"
|
||||||
|
|
||||||
|
|
||||||
|
def test_select_input_by_id():
|
||||||
|
router = make_router()
|
||||||
|
endpoint = asyncio.run(router.select_input("local-default-mic"))
|
||||||
|
assert asyncio.run(endpoint.capabilities()).id == "local-default-mic"
|
||||||
|
|
||||||
|
|
||||||
|
def test_unknown_endpoint_raises():
|
||||||
|
router = make_router()
|
||||||
|
with pytest.raises(UnknownEndpointError):
|
||||||
|
asyncio.run(router.select_output("does-not-exist"))
|
||||||
7
tests/test_basic_layout.py
Normal file
7
tests/test_basic_layout.py
Normal file
|
|
@ -0,0 +1,7 @@
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
def test_project_files_exist():
|
||||||
|
root = Path(__file__).resolve().parents[1]
|
||||||
|
assert (root / "app" / "main.py").exists()
|
||||||
|
assert (root / "pyproject.toml").exists()
|
||||||
|
assert (root / "docker-compose.yml").exists()
|
||||||
60
tests/test_config_profiles.py
Normal file
60
tests/test_config_profiles.py
Normal file
|
|
@ -0,0 +1,60 @@
|
||||||
|
import pytest
|
||||||
|
from pydantic_settings import SettingsConfigDict
|
||||||
|
|
||||||
|
from app import config as cfg
|
||||||
|
|
||||||
|
EXAMPLE_TOML = str(cfg.BASE_DIR / "config" / "voice-assistant.example.toml")
|
||||||
|
|
||||||
|
|
||||||
|
class IsolatedSettings(cfg.Settings):
|
||||||
|
# .env ausblenden, damit nur TOML/Defaults/ENV-Monkeypatch zaehlen.
|
||||||
|
model_config = SettingsConfigDict(env_file=None, case_sensitive=False, extra="ignore")
|
||||||
|
|
||||||
|
|
||||||
|
def _clear_provider_env(monkeypatch):
|
||||||
|
for name in ("DEFAULT_STT_PROVIDER", "DEFAULT_LLM_PROVIDER", "DEFAULT_TTS_PROVIDER"):
|
||||||
|
monkeypatch.delenv(name, raising=False)
|
||||||
|
monkeypatch.setenv("VA_CONFIG_FILE", EXAMPLE_TOML)
|
||||||
|
|
||||||
|
|
||||||
|
def test_profile_local_dev(monkeypatch):
|
||||||
|
_clear_provider_env(monkeypatch)
|
||||||
|
monkeypatch.setenv("VA_PROFILE", "local-dev")
|
||||||
|
s = IsolatedSettings()
|
||||||
|
assert s.default_stt_provider == "faster-whisper"
|
||||||
|
assert s.default_llm_provider == "local-openai-compatible"
|
||||||
|
assert s.default_tts_provider == "piper"
|
||||||
|
|
||||||
|
|
||||||
|
def test_profile_cloud(monkeypatch):
|
||||||
|
_clear_provider_env(monkeypatch)
|
||||||
|
monkeypatch.setenv("VA_PROFILE", "cloud")
|
||||||
|
s = IsolatedSettings()
|
||||||
|
assert s.default_stt_provider == "openrouter"
|
||||||
|
assert s.default_llm_provider == "openrouter"
|
||||||
|
assert s.default_tts_provider == "openrouter"
|
||||||
|
|
||||||
|
|
||||||
|
def test_env_overrides_toml(monkeypatch):
|
||||||
|
_clear_provider_env(monkeypatch)
|
||||||
|
monkeypatch.setenv("VA_PROFILE", "local-dev")
|
||||||
|
monkeypatch.setenv("DEFAULT_LLM_PROVIDER", "openrouter") # ENV gewinnt ueber TOML
|
||||||
|
s = IsolatedSettings()
|
||||||
|
assert s.default_llm_provider == "openrouter"
|
||||||
|
assert s.default_tts_provider == "piper" # vom Profil, nicht ueberschrieben
|
||||||
|
|
||||||
|
|
||||||
|
def test_unknown_profile_raises(monkeypatch):
|
||||||
|
_clear_provider_env(monkeypatch)
|
||||||
|
monkeypatch.setenv("VA_PROFILE", "gibtsnicht")
|
||||||
|
with pytest.raises(ValueError):
|
||||||
|
IsolatedSettings()
|
||||||
|
|
||||||
|
|
||||||
|
def test_missing_config_file_falls_back_to_defaults(monkeypatch):
|
||||||
|
for name in ("DEFAULT_STT_PROVIDER", "DEFAULT_LLM_PROVIDER", "DEFAULT_TTS_PROVIDER"):
|
||||||
|
monkeypatch.delenv(name, raising=False)
|
||||||
|
monkeypatch.setenv("VA_CONFIG_FILE", "/nonexistent/voice-assistant.toml")
|
||||||
|
monkeypatch.delenv("VA_PROFILE", raising=False)
|
||||||
|
s = IsolatedSettings()
|
||||||
|
assert s.default_stt_provider == "openrouter" # eingebauter Field-Default
|
||||||
96
tests/test_endpoints_e2e.py
Normal file
96
tests/test_endpoints_e2e.py
Normal file
|
|
@ -0,0 +1,96 @@
|
||||||
|
import json
|
||||||
|
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
import app.dependencies as deps
|
||||||
|
from app.main import app
|
||||||
|
from tests.conftest import loopback_output
|
||||||
|
|
||||||
|
client = TestClient(app)
|
||||||
|
|
||||||
|
|
||||||
|
def test_speak_loopback_collects_chunks():
|
||||||
|
# piper-Stub liefert b"" -> kein Netzcall; Loopback sammelt den Chunk.
|
||||||
|
resp = client.post(
|
||||||
|
"/api/speak",
|
||||||
|
json={"text": "Hallo Welt", "tts_provider": "piper", "output_endpoint": "loopback"},
|
||||||
|
)
|
||||||
|
assert resp.status_code == 200
|
||||||
|
assert resp.headers["X-Output-Endpoint"] == "loopback"
|
||||||
|
assert resp.headers["X-TTS-Provider"] == "piper"
|
||||||
|
assert len(loopback_output().chunks) == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_unknown_endpoint_returns_422():
|
||||||
|
resp = client.post(
|
||||||
|
"/api/speak",
|
||||||
|
json={"text": "x", "tts_provider": "piper", "output_endpoint": "gibtsnicht"},
|
||||||
|
)
|
||||||
|
assert resp.status_code == 422
|
||||||
|
assert "gibtsnicht" in resp.json()["detail"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_unknown_provider_returns_422():
|
||||||
|
resp = client.post("/api/speak", json={"text": "x", "tts_provider": "gibtsnicht"})
|
||||||
|
assert resp.status_code == 422
|
||||||
|
|
||||||
|
|
||||||
|
def test_session_route_applies():
|
||||||
|
client.post(
|
||||||
|
"/api/sessions/s1/route",
|
||||||
|
json={"tts_provider": "piper", "output_endpoint": "loopback"},
|
||||||
|
)
|
||||||
|
resp = client.post("/api/speak?session_id=s1", json={"text": "hallo"})
|
||||||
|
assert resp.status_code == 200
|
||||||
|
assert resp.headers["X-Output-Endpoint"] == "loopback"
|
||||||
|
assert resp.headers["X-TTS-Provider"] == "piper"
|
||||||
|
|
||||||
|
|
||||||
|
def test_chat_per_request_override_and_loopback(monkeypatch):
|
||||||
|
class StubLLM:
|
||||||
|
async def complete(self, text, session_id=None):
|
||||||
|
return "Mir geht es gut, danke."
|
||||||
|
|
||||||
|
class StubTTS:
|
||||||
|
async def synthesize(self, text, voice=None, audio_format="pcm"):
|
||||||
|
return b"AUDIO"
|
||||||
|
|
||||||
|
monkeypatch.setitem(deps.LLM_REGISTRY, "stub", lambda s: StubLLM())
|
||||||
|
monkeypatch.setitem(deps.TTS_REGISTRY, "stub", lambda s: StubTTS())
|
||||||
|
|
||||||
|
resp = client.post(
|
||||||
|
"/api/chat?debug=true",
|
||||||
|
json={
|
||||||
|
"text": "Wie geht es dir?",
|
||||||
|
"llm_provider": "stub",
|
||||||
|
"tts_provider": "stub",
|
||||||
|
"output_endpoint": "loopback",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
assert resp.status_code == 200
|
||||||
|
body = resp.json()
|
||||||
|
assert body["route"]["llm_provider"] == "stub"
|
||||||
|
assert body["route"]["output_endpoint"] == "loopback"
|
||||||
|
assert body["trace"]["semantic_response"] == "Mir geht es gut, danke."
|
||||||
|
assert loopback_output().chunks[0].data == b"AUDIO"
|
||||||
|
|
||||||
|
|
||||||
|
def test_transcribe_local_provider():
|
||||||
|
files = {"file": ("a.wav", b"RIFFdata", "audio/wav")}
|
||||||
|
data = {"stt_provider": "faster-whisper"}
|
||||||
|
resp = client.post("/api/transcribe", data=data, files=files)
|
||||||
|
assert resp.status_code == 200
|
||||||
|
body = resp.json()
|
||||||
|
assert body["route"]["stt_provider"] == "faster-whisper"
|
||||||
|
assert body["trace"]["raw_transcript"] == "[local transcription placeholder]"
|
||||||
|
|
||||||
|
|
||||||
|
def test_config_endpoint_exposes_no_secrets():
|
||||||
|
resp = client.get("/api/config")
|
||||||
|
assert resp.status_code == 200
|
||||||
|
body = resp.json()
|
||||||
|
assert "piper" in body["available"]["tts_providers"]
|
||||||
|
assert "loopback" in {e["kind"] for e in body["available"]["output_endpoints"]}
|
||||||
|
# Keine echten Secrets im Body.
|
||||||
|
assert "sk-or-" not in json.dumps(body)
|
||||||
|
assert set(body["secrets"].keys()) == {"openrouter_api_key_set"}
|
||||||
46
tests/test_routing.py
Normal file
46
tests/test_routing.py
Normal file
|
|
@ -0,0 +1,46 @@
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from app.config import Settings
|
||||||
|
from app.errors import UnknownComponentError
|
||||||
|
from app.dependencies import (
|
||||||
|
resolve_route,
|
||||||
|
get_llm_provider,
|
||||||
|
get_tts_provider,
|
||||||
|
session_manager,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_default_route_from_settings():
|
||||||
|
cfg = Settings()
|
||||||
|
route = resolve_route(cfg=cfg)
|
||||||
|
assert route.stt_provider == cfg.default_stt_provider
|
||||||
|
assert route.llm_provider == cfg.default_llm_provider
|
||||||
|
assert route.input_endpoint == cfg.default_input_endpoint
|
||||||
|
assert route.language == cfg.default_language
|
||||||
|
|
||||||
|
|
||||||
|
def test_request_overrides_win():
|
||||||
|
route = resolve_route(overrides={"llm_provider": "openrouter", "output_endpoint": "loopback"})
|
||||||
|
assert route.llm_provider == "openrouter"
|
||||||
|
assert route.output_endpoint == "loopback"
|
||||||
|
|
||||||
|
|
||||||
|
def test_session_then_request_precedence():
|
||||||
|
session_manager.update("s_test", {"tts_provider": "piper", "language": "en"})
|
||||||
|
route = resolve_route("s_test")
|
||||||
|
assert route.tts_provider == "piper"
|
||||||
|
assert route.language == "en"
|
||||||
|
|
||||||
|
# Request schlaegt Session.
|
||||||
|
route2 = resolve_route("s_test", {"tts_provider": "chatterbox"})
|
||||||
|
assert route2.tts_provider == "chatterbox"
|
||||||
|
assert route2.language == "en"
|
||||||
|
|
||||||
|
|
||||||
|
def test_registry_unknown_provider_raises():
|
||||||
|
with pytest.raises(UnknownComponentError):
|
||||||
|
get_llm_provider("does-not-exist")
|
||||||
|
|
||||||
|
|
||||||
|
def test_registry_known_provider():
|
||||||
|
assert type(get_tts_provider("piper")).__name__ == "PiperTTSProvider"
|
||||||
Loading…
Add table
Add a link
Reference in a new issue