Initial commit: Voice Assistant Gateway mit Konfig-/Routing-Fundament
- FastAPI-Gateway mit REST-Endpunkten (chat/speak/transcribe/devices/sessions/config) - Geschichtete Konfiguration mit Profilen (local-dev/hybrid/cloud) via TOML + ENV - Registry-Pattern + einheitliche Route-Aufloesung (Default->Profil->ENV->Session->Request) - Device Router (strikt, Singleton) und Output-Lifecycle im Orchestrator - OpenRouter-Adapter (STT multipart/LLM/TTS) + lokale Provider-Stubs - Regelbasierte Pipeline (Cleaner/Spoken-Adapter/TTS-Normalizer) - 22 automatisierte Tests; Doku: README, BEDIENUNGSANLEITUNG, Architektur - Secrets ausschliesslich ueber Umgebung; .env und lokale config gitignored Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Dieter Schlüter
commit
293ed257db
72 changed files with 2612 additions and 0 deletions
31
.env.example
Normal file
31
.env.example
Normal file
|
|
@ -0,0 +1,31 @@
|
||||||
|
# Port bei Bedarf anpassen
|
||||||
|
APP_ENV=dev
|
||||||
|
HOST=0.0.0.0
|
||||||
|
PORT=8080
|
||||||
|
LOG_LEVEL=info
|
||||||
|
|
||||||
|
# Secret nur ueber die Umgebung setzen (nicht hier eintragen), z. B. export in ~/.bashrc
|
||||||
|
OPENROUTER_API_KEY=
|
||||||
|
|
||||||
|
# --- Zentrale Konfiguration / Profile -------------------------------------
|
||||||
|
# Aktives Profil aus config/voice-assistant.toml waehlen: local-dev | hybrid | cloud
|
||||||
|
# (leer lassen = nur Defaults/ENV). Eigener Pfad via VA_CONFIG_FILE.
|
||||||
|
VA_PROFILE=
|
||||||
|
# VA_CONFIG_FILE=config/voice-assistant.toml
|
||||||
|
|
||||||
|
# Hinweis zur Praezedenz: ENV gewinnt ueber die TOML-Datei. Die DEFAULT_*_PROVIDER-
|
||||||
|
# Zeilen unten ueberschreiben daher ein gesetztes VA_PROFILE. Wer profilbasiert
|
||||||
|
# umschalten will, sollte sie auskommentiert lassen.
|
||||||
|
OPENROUTER_STT_MODEL=openai/whisper-large-v3
|
||||||
|
OPENROUTER_TTS_MODEL=openai/gpt-4o-mini-tts
|
||||||
|
OPENROUTER_TTS_VOICE=alloy
|
||||||
|
OPENROUTER_LLM_MODEL=openai/gpt-4.1-mini
|
||||||
|
DEFAULT_LANGUAGE=de
|
||||||
|
DEFAULT_INPUT_ENDPOINT=local-default
|
||||||
|
DEFAULT_OUTPUT_ENDPOINT=local-default
|
||||||
|
# DEFAULT_STT_PROVIDER=openrouter
|
||||||
|
# DEFAULT_LLM_PROVIDER=local-openai-compatible
|
||||||
|
# DEFAULT_TTS_PROVIDER=openrouter
|
||||||
|
LOCAL_LLM_BASE_URL=http://127.0.0.1:11434/v1
|
||||||
|
LOCAL_LLM_API_KEY=dummy
|
||||||
|
LOCAL_LLM_MODEL=llama3.1
|
||||||
21
.gitignore
vendored
Normal file
21
.gitignore
vendored
Normal file
|
|
@ -0,0 +1,21 @@
|
||||||
|
# Secrets / lokale Konfiguration
|
||||||
|
.env
|
||||||
|
|
||||||
|
# Lokale/instanzspezifische Konfiguration (nur die *.example.toml wird versioniert)
|
||||||
|
config/voice-assistant.toml
|
||||||
|
|
||||||
|
# Python
|
||||||
|
__pycache__/
|
||||||
|
*.py[cod]
|
||||||
|
*.egg-info/
|
||||||
|
.venv/
|
||||||
|
venv/
|
||||||
|
.pytest_cache/
|
||||||
|
|
||||||
|
# Lokale Tool-/Editor-Konfiguration
|
||||||
|
.claude/
|
||||||
|
|
||||||
|
# Editor-/Backup-Reste
|
||||||
|
*.bak
|
||||||
|
*.patch
|
||||||
|
*.orig
|
||||||
223
BEDIENUNGSANLEITUNG.md
Normal file
223
BEDIENUNGSANLEITUNG.md
Normal file
|
|
@ -0,0 +1,223 @@
|
||||||
|
# Bedienungsanleitung — Voice Assistant Gateway
|
||||||
|
|
||||||
|
Diese Anleitung führt Schritt für Schritt durch Installation, Start, Konfiguration
|
||||||
|
und Fehlerbehebung. Technische Hintergründe stehen im
|
||||||
|
[Architektur-Dokument](Docs/voice-assistant-architecture.md), eine kompakte
|
||||||
|
Übersicht im [README](README.md).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Voraussetzungen
|
||||||
|
|
||||||
|
- **Python 3.11 oder neuer** (`python3 --version`)
|
||||||
|
- Ein **OpenRouter-API-Key** — nur nötig, wenn ein Profil entfernte KI nutzt
|
||||||
|
(`hybrid`, `cloud`). Für rein lokalen Betrieb (`local-dev`) nicht erforderlich.
|
||||||
|
- Optional: Docker, falls im Container betrieben.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Installation
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd voice-assistant-scaffold
|
||||||
|
python3 -m venv .venv
|
||||||
|
source .venv/bin/activate
|
||||||
|
pip install -U pip
|
||||||
|
pip install -e .[test]
|
||||||
|
```
|
||||||
|
|
||||||
|
Danach die zentrale Konfigurationsdatei anlegen:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cp config/voice-assistant.example.toml config/voice-assistant.toml
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. API-Key hinterlegen (für Cloud/Hybrid)
|
||||||
|
|
||||||
|
Der Schlüssel wird **aus der Umgebung** gelesen und gehört **nicht** in eine Datei.
|
||||||
|
Dauerhaft am besten in `~/.bashrc`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
echo 'export OPENROUTER_API_KEY=sk-or-v1-DEIN_KEY' >> ~/.bashrc
|
||||||
|
chmod 600 ~/.bashrc
|
||||||
|
source ~/.bashrc
|
||||||
|
```
|
||||||
|
|
||||||
|
Prüfen, ob er ankommt:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
echo ${OPENROUTER_API_KEY:0:8} # zeigt nur den Anfang
|
||||||
|
```
|
||||||
|
|
||||||
|
> **Sicherheit:** Den Key niemals in `.env` oder `config/*.toml` schreiben. Wird ein
|
||||||
|
> Key versehentlich öffentlich, im OpenRouter-Dashboard löschen (= widerrufen) und
|
||||||
|
> neu erzeugen.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Betriebsart (Profil) wählen
|
||||||
|
|
||||||
|
Profile bestimmen, welche KI-Module genutzt werden:
|
||||||
|
|
||||||
|
| Profil | Bedeutung | Key nötig? |
|
||||||
|
|-------------|--------------------------------------------|------------|
|
||||||
|
| `local-dev` | alles lokal (eigene KI/Hardware) | nein |
|
||||||
|
| `hybrid` | STT/TTS über Cloud, Haupt-LLM lokal | ja |
|
||||||
|
| `cloud` | alles über OpenRouter (Standardbetrieb) | ja |
|
||||||
|
|
||||||
|
Profil **einmalig** für einen Start:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
VA_PROFILE=cloud make run
|
||||||
|
```
|
||||||
|
|
||||||
|
Profil **dauerhaft** — in `.env` eintragen:
|
||||||
|
|
||||||
|
```
|
||||||
|
VA_PROFILE=cloud
|
||||||
|
```
|
||||||
|
|
||||||
|
> Hinweis: Stehen in `.env` noch `DEFAULT_STT_PROVIDER` / `DEFAULT_LLM_PROVIDER` /
|
||||||
|
> `DEFAULT_TTS_PROVIDER`, überschreiben diese das Profil. Für profilbasiertes
|
||||||
|
> Umschalten sollten sie auskommentiert sein.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Starten und Stoppen
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make run
|
||||||
|
```
|
||||||
|
|
||||||
|
Standard-Adresse: `http://localhost:8080` (Port änderbar, siehe Abschnitt 8).
|
||||||
|
Beenden mit **Strg + C**.
|
||||||
|
|
||||||
|
Schnelltest in einem zweiten Terminal:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://localhost:8080/health
|
||||||
|
# {"status":"ok"}
|
||||||
|
|
||||||
|
curl http://localhost:8080/api/config
|
||||||
|
# zeigt aktives Profil und die aufgelöste Standard-Route
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Tägliche Bedienung — typische Aufgaben
|
||||||
|
|
||||||
|
### a) Text sprechen lassen (`/api/speak`)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST http://localhost:8080/api/speak \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d '{"text":"Guten Morgen, wie geht es Ihnen?"}' \
|
||||||
|
--output antwort.pcm
|
||||||
|
```
|
||||||
|
|
||||||
|
### b) Chatten (Text rein, gesprochene Antwort raus) (`/api/chat`)
|
||||||
|
|
||||||
|
Nur den Trace als JSON ansehen (ohne Audio):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST "http://localhost:8080/api/chat?debug=true" \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d '{"text":"Wie wird das Wetter morgen?"}'
|
||||||
|
```
|
||||||
|
|
||||||
|
Komfortabler mit dem mitgelieferten Client (spielt die Antwort ab):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python chat_client.py "Erzähl mir einen guten Morgen-Spruch"
|
||||||
|
```
|
||||||
|
|
||||||
|
> `chat_client.py` erwartet den Dienst auf Port **8003** — bei Bedarf im Skript
|
||||||
|
> `GATEWAY_URL` anpassen oder den Dienst mit `PORT=8003 make run` starten.
|
||||||
|
|
||||||
|
### c) Audio transkribieren (`/api/transcribe`)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST http://localhost:8080/api/transcribe \
|
||||||
|
-F "file=@aufnahme.wav" -F "language=de"
|
||||||
|
```
|
||||||
|
|
||||||
|
### d) Gerät oder Provider einmalig umstellen (pro Aufruf)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST http://localhost:8080/api/speak \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d '{"text":"Test","tts_provider":"piper","output_endpoint":"loopback"}'
|
||||||
|
```
|
||||||
|
|
||||||
|
### e) Präferenzen für eine Session festlegen
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# einmal setzen
|
||||||
|
curl -X POST http://localhost:8080/api/sessions/oma-anna/route \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d '{"llm_provider":"openrouter","language":"de"}'
|
||||||
|
|
||||||
|
# danach mit dieser Session nutzen
|
||||||
|
curl -X POST "http://localhost:8080/api/chat?session_id=oma-anna&debug=true" \
|
||||||
|
-H 'Content-Type: application/json' -d '{"text":"Hallo!"}'
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. Verfügbare Geräte und Bausteine ansehen
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://localhost:8080/api/devices # Audio-Endpunkte mit Fähigkeiten
|
||||||
|
curl http://localhost:8080/api/config # Profil, Route, Provider, Endpunkte
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. Port ändern
|
||||||
|
|
||||||
|
```bash
|
||||||
|
PORT=8003 make run # einmalig
|
||||||
|
sed -i 's/^PORT=.*/PORT=8003/' .env # dauerhaft
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. Mit Docker betreiben
|
||||||
|
|
||||||
|
```bash
|
||||||
|
export OPENROUTER_API_KEY=sk-or-v1-...
|
||||||
|
docker compose up --build
|
||||||
|
```
|
||||||
|
|
||||||
|
Der Key wird aus der Shell in den Container durchgereicht; fehlt er, bricht der
|
||||||
|
Start mit klarer Meldung ab.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 10. Fehlerbehebung
|
||||||
|
|
||||||
|
| Symptom | Ursache | Lösung |
|
||||||
|
|---|---|---|
|
||||||
|
| `OPENROUTER_API_KEY is empty` / 401 | Key nicht in der Umgebung | `export OPENROUTER_API_KEY=…`, neues Terminal / `source ~/.bashrc` |
|
||||||
|
| HTTP **422** „Unbekannter …-Provider/Endpunkt" | Tippfehler in `*_provider` / `*_endpoint` | gültige Werte via `GET /api/config` prüfen |
|
||||||
|
| `VA_PROFILE` wirkt nicht | `DEFAULT_*_PROVIDER` in `.env` überschreibt es | diese Zeilen in `.env` auskommentieren |
|
||||||
|
| LLM-Timeout / Connection refused (lokal) | lokaler LLM-Server (Port 11434) läuft nicht | LLM-Server starten oder Profil `cloud` wählen |
|
||||||
|
| `Address already in use` | Port belegt | anderen `PORT` setzen (Abschnitt 8) |
|
||||||
|
| `chat_client.py` bekommt keine Antwort | Client nutzt Port 8003 | Dienst mit `PORT=8003` starten oder `GATEWAY_URL` anpassen |
|
||||||
|
| Profil greift nicht / Standardwerte | `config/voice-assistant.toml` fehlt | Datei aus `*.example.toml` kopieren (Abschnitt 2) |
|
||||||
|
|
||||||
|
Logs erscheinen im Terminal, in dem `make run` läuft. Für mehr Details
|
||||||
|
`LOG_LEVEL=debug` in `.env` setzen.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 11. Tests ausführen
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make test
|
||||||
|
```
|
||||||
|
|
||||||
|
Alle Tests sollten grün sein. Schlägt etwas fehl, gibt die Ausgabe den genauen
|
||||||
|
Testnamen und die Ursache an.
|
||||||
7
Dockerfile
Normal file
7
Dockerfile
Normal file
|
|
@ -0,0 +1,7 @@
|
||||||
|
FROM python:3.12-slim
|
||||||
|
WORKDIR /app
|
||||||
|
COPY pyproject.toml README.md ./
|
||||||
|
COPY app ./app
|
||||||
|
RUN pip install --no-cache-dir -U pip && pip install --no-cache-dir .[test]
|
||||||
|
EXPOSE 8080
|
||||||
|
CMD ["sh", "-c", "uvicorn app.main:app --host ${HOST:-0.0.0.0} --port ${PORT:-8080}"]
|
||||||
220
Docs/voice-assistant-architecture.md
Normal file
220
Docs/voice-assistant-architecture.md
Normal file
|
|
@ -0,0 +1,220 @@
|
||||||
|
# Architektur: Modularer Voice-Assistent
|
||||||
|
|
||||||
|
> Stand: aktueller Implementierungsstand des Gateways. Dieses Dokument beschreibt
|
||||||
|
> das Konzept, den umgesetzten Stand und die geplanten nächsten Schritte.
|
||||||
|
|
||||||
|
## 1. Ziel & Leitidee
|
||||||
|
|
||||||
|
Ein modularer, **cloud-first, aber hybrid betreibbarer** Sprachassistent für
|
||||||
|
Senioren — ein „digitaler Vertrauter", erreichbar von zuhause und unterwegs.
|
||||||
|
|
||||||
|
Der Hauptbetrieb läuft in der Cloud / auf einem vHost (Senioren sollen keinen
|
||||||
|
teuren KI-Rechner zuhause brauchen). Gleichzeitig muss jede Achse frei wählbar
|
||||||
|
bleiben, je nach Installation und Entwicklungs-/Testbedarf:
|
||||||
|
|
||||||
|
- **Hardware** — Audio-Eingabe und -Ausgabe (lokal, Bluetooth, Handy, Netzwerk)
|
||||||
|
- **Betrieb** — lokal, Cloud oder hybrid
|
||||||
|
- **Software** — lokale oder entfernte KI (STT / LLM / TTS), ganz oder teilweise
|
||||||
|
|
||||||
|
Designziele: geringe Latenz, robuste Fallbacks, gute Debugbarkeit, klare Trennung
|
||||||
|
von **semantischer** und **gesprochener** Antwort, und vor allem **Austauschbarkeit**:
|
||||||
|
einzelne Module gegen andere tauschen, ohne das Programm umzuschreiben.
|
||||||
|
|
||||||
|
## 2. Architekturprinzip — fünf Ebenen
|
||||||
|
|
||||||
|
1. **Audio Endpoints** – konkrete Quellen/Ziele (lokal, Bluetooth, Handy, WebRTC)
|
||||||
|
2. **Device Router** – wählt passende Input-/Output-Endpunkte
|
||||||
|
3. **Speech Pipeline** – STT, Input-Cleaner, Dialog-LLM, Spoken-Response-Adapter, TTS-Normalizer, TTS
|
||||||
|
4. **Transport Router** – lokale vs. entfernte Ausführung einzelner Module
|
||||||
|
5. **Orchestrator** – Session, Turn-Taking, Fallbacks, Metrik
|
||||||
|
|
||||||
|
Jede Ebene kommuniziert über wohldefinierte Interfaces (ABCs + Pydantic-Schemas),
|
||||||
|
nicht über konkrete Bibliotheken.
|
||||||
|
|
||||||
|
## 3. Konfigurations- & Routing-Modell (umgesetzt)
|
||||||
|
|
||||||
|
Das Herzstück der Austauschbarkeit. Jede Achse ist auf mehreren Ebenen
|
||||||
|
einstellbar; höhere Ebene gewinnt:
|
||||||
|
|
||||||
|
```
|
||||||
|
eingebaute Defaults < config/voice-assistant.toml (inkl. aktivem Profil)
|
||||||
|
< ENV / .env < Session-Route < Request
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3.1 Zentrale Config + Profile
|
||||||
|
|
||||||
|
Eine geschichtete zentrale Datei (`config/voice-assistant.toml`, Vorlage
|
||||||
|
`config/voice-assistant.example.toml`), gelesen über die stdlib (`tomllib`).
|
||||||
|
**Profile** bündeln Betriebsarten und werden per ENV `VA_PROFILE` aktiviert:
|
||||||
|
|
||||||
|
| Profil | STT | LLM | TTS |
|
||||||
|
|-------------|----------------|--------------------------|------------|
|
||||||
|
| `local-dev` | faster-whisper | local-openai-compatible | piper |
|
||||||
|
| `hybrid` | openrouter | local-openai-compatible | openrouter |
|
||||||
|
| `cloud` | openrouter | openrouter | openrouter |
|
||||||
|
|
||||||
|
Umgesetzt in `app/config.py`: `load_profile_config()` merged `[defaults]` +
|
||||||
|
`[profiles.<aktiv>]`; `TomlProfileSource` hängt diese Werte als Settings-Quelle
|
||||||
|
**unter** ENV ein (`settings_customise_sources`). `VA_PROFILE`/`VA_CONFIG_FILE`
|
||||||
|
werden aus echter Umgebung **oder** `.env` gelesen.
|
||||||
|
|
||||||
|
> **Secrets gehören nie in die Config-Datei** — nur in die Umgebung
|
||||||
|
> (z. B. `OPENROUTER_API_KEY`). Die TOML-Datei darf versioniert/geteilt werden.
|
||||||
|
|
||||||
|
### 3.2 Einheitliche Route-Auflösung
|
||||||
|
|
||||||
|
`app/dependencies.py` löst pro Aufruf eine `ResolvedRoute` auf
|
||||||
|
(`resolve_route(session_id, overrides)`): Defaults < Session-Route < Request.
|
||||||
|
Die Route umfasst `input_endpoint`, `output_endpoint`, `stt_provider`,
|
||||||
|
`llm_provider`, `tts_provider`, `language`.
|
||||||
|
|
||||||
|
### 3.3 Registry-Pattern (Provider austauschbar)
|
||||||
|
|
||||||
|
`STT_REGISTRY` / `LLM_REGISTRY` / `TTS_REGISTRY` bilden `name -> factory(settings)`.
|
||||||
|
Ein neuer Provider = ein Eintrag, ohne Kern-Code zu ändern. Unbekannter Name →
|
||||||
|
`UnknownComponentError` → HTTP 422.
|
||||||
|
|
||||||
|
### 3.4 Device Router
|
||||||
|
|
||||||
|
`app/audio/router.py` wählt Endpunkte per `id`- oder `kind`-Match. Ein angefragter,
|
||||||
|
aber unbekannter Endpunkt wirft `UnknownEndpointError` (→ 422) — **kein** stiller
|
||||||
|
Default-Fallback. Ohne Wunsch greift der als `default` markierte Endpunkt. Der
|
||||||
|
Router ist ein Singleton (stabiler Zustand, z. B. `LoopbackOutput`).
|
||||||
|
|
||||||
|
## 4. Datenmodelle & Interfaces
|
||||||
|
|
||||||
|
Schemas in `app/schemas.py`, Interfaces als ABCs in den jeweiligen `base.py`.
|
||||||
|
|
||||||
|
```python
|
||||||
|
class EndpointCapabilities(BaseModel):
|
||||||
|
id: str; kind: str
|
||||||
|
direction: Literal["input", "output"]
|
||||||
|
sample_rate: int = 16000; channels: int = 1
|
||||||
|
latency_class: Literal["low", "medium", "high"] = "medium"
|
||||||
|
supports_aec: bool = False; supports_barge_in: bool = False
|
||||||
|
networked: bool = False; bluetooth: bool = False
|
||||||
|
mobile: bool = False; default: bool = False
|
||||||
|
|
||||||
|
class AudioChunk(BaseModel):
|
||||||
|
data: bytes; sample_rate: int = 16000; channels: int = 1
|
||||||
|
format: str = "wav"; timestamp_ms: int = 0
|
||||||
|
|
||||||
|
class PipelineTrace(BaseModel):
|
||||||
|
raw_transcript: str | None = None
|
||||||
|
cleaned_transcript: str | None = None
|
||||||
|
semantic_response: str | None = None
|
||||||
|
spoken_response: str | None = None
|
||||||
|
tts_ready_text: str | None = None
|
||||||
|
```
|
||||||
|
|
||||||
|
Provider-Interfaces:
|
||||||
|
|
||||||
|
```python
|
||||||
|
class STTProvider(ABC):
|
||||||
|
async def transcribe(self, audio_bytes: bytes, fmt: str, language: str | None = None) -> str: ...
|
||||||
|
|
||||||
|
class LLMProvider(ABC):
|
||||||
|
async def complete(self, text: str, session_id: str | None = None) -> str: ...
|
||||||
|
|
||||||
|
class TTSProvider(ABC):
|
||||||
|
async def synthesize(self, text: str, voice: str | None = None, audio_format: str = "pcm") -> bytes: ...
|
||||||
|
```
|
||||||
|
|
||||||
|
Audio-Endpunkt-Interfaces: `AudioInputEndpoint` (`capabilities/open/read_chunk/close`),
|
||||||
|
`AudioOutputEndpoint` (`capabilities/open/write_chunk/flush/close`) — alle async.
|
||||||
|
|
||||||
|
## 5. Speech Pipeline
|
||||||
|
|
||||||
|
Trennung von **semantischer** und **gesprochener** Antwort ist zentral: eine
|
||||||
|
inhaltlich gute Antwort ist nicht automatisch gut hörbar.
|
||||||
|
|
||||||
|
Stufen: `raw_transcript → cleaned_transcript → semantic_response → spoken_response → tts_ready_text`.
|
||||||
|
|
||||||
|
- **Input Cleaner** (`pipeline/input_cleaner.py`) – konservative Bereinigung des STT-Texts (Füllwörter, Whitespace). Verändert die Nutzerintention nicht.
|
||||||
|
- **Dialog-LLM** – semantische Antwort; Persona/Sicherheitsregeln im System-Prompt (`providers/llm/openrouter.py`).
|
||||||
|
- **Spoken Response Adapter** (`pipeline/spoken_response_adapter.py`) – macht die Antwort sprechbar/seniorengerecht (Markdown raus, Listen → Sätze, Uhrzeiten erhalten).
|
||||||
|
- **TTS Normalizer** (`pipeline/tts_normalizer.py`) – Zahlen/Abkürzungen/Einheiten verbal ausformulieren, sprachabhängig (de/en).
|
||||||
|
|
||||||
|
Empfehlung: nicht jede Zwischenstufe braucht ein großes LLM — Cleaner und
|
||||||
|
Normalizer überwiegend regelbasiert (so heute umgesetzt), Adapter promptbasiert.
|
||||||
|
|
||||||
|
## 6. Orchestrator
|
||||||
|
|
||||||
|
`app/core/orchestrator.py` verbindet Provider, Pipeline und Output-Endpunkt.
|
||||||
|
|
||||||
|
- `chat_text(text, language, voice, output)` → `(trace, audio_bytes)`
|
||||||
|
- `speak_only(text, voice, language, output)` → `audio_bytes`
|
||||||
|
- `transcribe_only(audio_bytes, fmt, language, input)` → `trace`
|
||||||
|
|
||||||
|
Das synthetisierte Audio wird **zusätzlich** durch den gewählten Output-Endpunkt
|
||||||
|
geschrieben (`open → write_chunk → flush → close`) und **gleichzeitig** als
|
||||||
|
HTTP-Stream zurückgegeben (additiv). Bei lokalen Geräten ist `write_chunk` heute
|
||||||
|
ein No-op; `LoopbackOutput` sammelt die Chunks (testbar ohne Hardware).
|
||||||
|
|
||||||
|
## 7. FastAPI-Endpunkte (umgesetzt)
|
||||||
|
|
||||||
|
| Methode & Pfad | Zweck |
|
||||||
|
|--------------------------------------|-------|
|
||||||
|
| `GET /health` | Liveness |
|
||||||
|
| `POST /api/chat` | Text rein → Audio raus (`?debug=true` → JSON-Trace) |
|
||||||
|
| `POST /api/speak` | Text rein → TTS-Audio raus |
|
||||||
|
| `POST /api/transcribe` | Audio-Upload → Transkript |
|
||||||
|
| `GET /api/devices` | verfügbare Audio-Endpunkte + Capabilities |
|
||||||
|
| `POST /api/sessions/{id}/route` | bevorzugte Geräte/Provider/Sprache je Session |
|
||||||
|
| `GET /api/config` | aktives Profil + aufgelöste Route (ohne Secrets) |
|
||||||
|
|
||||||
|
Endpunkt-/Provider-Auswahl ist über **Request-Body** (pro Aufruf), **Session**
|
||||||
|
(`?session_id=…`) und **Defaults/Profil** steuerbar. Verwendete Route erscheint als
|
||||||
|
`X-*`-Header bzw. im `?debug`-JSON.
|
||||||
|
|
||||||
|
## 8. Stand der Implementierung
|
||||||
|
|
||||||
|
**Umgesetzt:** FastAPI-Gateway, alle o. g. REST-Endpunkte; OpenRouter-Adapter für
|
||||||
|
STT (multipart), LLM und TTS; lokaler OpenAI-kompatibler LLM-Adapter; regelbasierte
|
||||||
|
Pipeline; geschichtete Config + Profile; Registry + einheitliche Route-Auflösung;
|
||||||
|
Device Router (strikt, Singleton); Output-Lifecycle; 22 automatisierte Tests.
|
||||||
|
|
||||||
|
**Platzhalter (Gerüst):** Audio-Endpunkte (`local-default`, `bluetooth`,
|
||||||
|
`mobile-ws`, `mobile-webrtc`) liefern leere Chunks — nur Auswahl/Lifecycle sind
|
||||||
|
verdrahtet, kein echtes Hardware-I/O. Lokale Provider `faster-whisper`, `piper`,
|
||||||
|
`chatterbox` sind Stubs. `transport_router.py` (Ebene 4) existiert, ist aber noch
|
||||||
|
nicht aktiv (lokal/remote trägt vorerst der Provider-Name).
|
||||||
|
|
||||||
|
## 9. Roadmap / bewusste nächste Schritte
|
||||||
|
|
||||||
|
Reihenfolge der Weiterentwicklung:
|
||||||
|
|
||||||
|
1. **(erledigt)** Konfig- & Routing-Fundament: Profile, Device Router, Registry, Pro-Request-Override.
|
||||||
|
2. **Cloud-Fundament:** Authentifizierung + Mehrbenutzer + persistenter Session-/Profil-Store (heute ist `SessionManager` in-memory → nicht skalierend, ohne Auth).
|
||||||
|
3. **Konversationsgedächtnis:** Verlauf + Langzeit-Präferenzen (heute ist `llm.complete` zustandslos).
|
||||||
|
4. **Echtzeit:** Streaming-STT/TTS, WebSocket/WebRTC-Endpunkte real, Barge-in, Turn-Manager.
|
||||||
|
5. **Resilienz:** Fallback-Policy (remote KI fällt aus → lokaler/alternativer Provider), Metriken/Tracing.
|
||||||
|
6. **Betrieb:** Kosten-/Quota-Kontrolle pro Nutzer; Notfall-/Eskalationskonzept (Senioren-Kontext).
|
||||||
|
7. **TransportRouter** als eigene lokal/remote-Achse aktivieren.
|
||||||
|
|
||||||
|
**Datenschutz (querschnittlich, ab sofort mitdenken):** Senioren-Sprachdaten sind
|
||||||
|
hochsensibel (oft gesundheitsbezogen → DSGVO Art. 9). EU-Datenresidenz,
|
||||||
|
Verschlüsselung at-rest/in-transit, Löschkonzept, Einwilligung — „privacy by design".
|
||||||
|
|
||||||
|
## 10. Verzeichnisstruktur (Ist)
|
||||||
|
|
||||||
|
```text
|
||||||
|
voice-assistant-scaffold/
|
||||||
|
├── app/
|
||||||
|
│ ├── main.py # FastAPI-App + Router-Registrierung
|
||||||
|
│ ├── config.py # Settings, TOML-Profile, Präzedenz
|
||||||
|
│ ├── dependencies.py # Registries, ResolvedRoute, resolve_route, Singleton-Router
|
||||||
|
│ ├── errors.py # RoutingError -> HTTP 422
|
||||||
|
│ ├── schemas.py # Pydantic-Modelle
|
||||||
|
│ ├── api/ # health, chat, speak, transcribe, devices, sessions, config
|
||||||
|
│ ├── core/ # orchestrator, session_manager
|
||||||
|
│ ├── audio/ # router, transport_router, endpoints/input|output/*
|
||||||
|
│ ├── pipeline/ # input_cleaner, spoken_response_adapter, tts_normalizer
|
||||||
|
│ └── providers/ # stt/ llm/ tts/ (openrouter + lokale Stubs)
|
||||||
|
├── config/ # voice-assistant.example.toml (+ lokale .toml, gitignored)
|
||||||
|
├── deploy/ # systemd unit + env-Beispiel
|
||||||
|
├── tests/ # config-profile, routing, audio-router, e2e
|
||||||
|
├── Docs/ # dieses Dokument
|
||||||
|
├── Dockerfile, docker-compose.yml, Makefile, pyproject.toml
|
||||||
|
└── README.md, BEDIENUNGSANLEITUNG.md
|
||||||
|
```
|
||||||
28
Makefile
Normal file
28
Makefile
Normal file
|
|
@ -0,0 +1,28 @@
|
||||||
|
ifneq (,$(wildcard ./.env))
|
||||||
|
include .env
|
||||||
|
export
|
||||||
|
endif
|
||||||
|
|
||||||
|
PORT ?= 8080
|
||||||
|
HOST ?= 0.0.0.0
|
||||||
|
|
||||||
|
.PHONY: ensure-env install run test docker-build
|
||||||
|
|
||||||
|
ensure-env:
|
||||||
|
@if [ ! -f .env ] && [ -f .env.example ]; then \
|
||||||
|
cp .env.example .env; \
|
||||||
|
echo "Created .env from .env.example"; \
|
||||||
|
fi
|
||||||
|
|
||||||
|
install: ensure-env
|
||||||
|
python3 -m venv .venv
|
||||||
|
. .venv/bin/activate && pip install -U pip && pip install -e .[test]
|
||||||
|
|
||||||
|
run: ensure-env
|
||||||
|
. .venv/bin/activate && uvicorn app.main:app --host $(HOST) --port $(PORT) --reload
|
||||||
|
|
||||||
|
test: ensure-env
|
||||||
|
. .venv/bin/activate && pytest tests/
|
||||||
|
|
||||||
|
docker-build:
|
||||||
|
docker build -t voice-assistant-gateway .
|
||||||
130
README.md
Normal file
130
README.md
Normal file
|
|
@ -0,0 +1,130 @@
|
||||||
|
# Voice Assistant Gateway
|
||||||
|
|
||||||
|
Modulares FastAPI-Gateway für einen **seniorengerechten Sprachassistenten** —
|
||||||
|
cloud-first, aber hybrid/lokal betreibbar, mit austauschbaren Audio-Endpunkten und
|
||||||
|
STT-/LLM-/TTS-Providern.
|
||||||
|
|
||||||
|
Jede Achse — **Hardware** (Audio In/Out), **Betrieb** (lokal/cloud) und **Software**
|
||||||
|
(lokale/remote KI) — ist frei konfigurierbar, ohne Code zu ändern. Konzept und
|
||||||
|
Details: [`Docs/voice-assistant-architecture.md`](Docs/voice-assistant-architecture.md).
|
||||||
|
Praktische Bedienung: [`BEDIENUNGSANLEITUNG.md`](BEDIENUNGSANLEITUNG.md).
|
||||||
|
|
||||||
|
## Features
|
||||||
|
|
||||||
|
- **Pipeline mit getrennter Semantik/Sprache:** STT → Input-Cleaner → LLM → Spoken-Adapter → TTS-Normalizer → TTS
|
||||||
|
- **Provider austauschbar** über Registry (OpenRouter remote; faster-whisper/piper/chatterbox als lokale Stubs)
|
||||||
|
- **Geschichtete Konfiguration** mit Profilen (`local-dev` / `hybrid` / `cloud`)
|
||||||
|
- **Routing auf jeder Ebene:** Default → Profil → ENV → Session → Request
|
||||||
|
- **REST-API** für Chat, Transkription, Sprachausgabe, Geräte, Sessions, Config
|
||||||
|
- **Ohne Secrets im Code** — API-Keys nur über die Umgebung
|
||||||
|
|
||||||
|
## Schnellstart
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 -m venv .venv
|
||||||
|
source .venv/bin/activate
|
||||||
|
pip install -U pip
|
||||||
|
pip install -e .[test]
|
||||||
|
|
||||||
|
cp config/voice-assistant.example.toml config/voice-assistant.toml
|
||||||
|
export OPENROUTER_API_KEY=sk-or-v1-... # nur für Cloud-/Hybrid-Profile nötig
|
||||||
|
make run
|
||||||
|
```
|
||||||
|
|
||||||
|
Fehlt `.env`, wird sie beim ersten `make run` aus `.env.example` erzeugt.
|
||||||
|
Die App läuft dann auf `http://localhost:8080` (bzw. dem in `.env` gesetzten `PORT`).
|
||||||
|
|
||||||
|
Kurztest:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://localhost:8080/health
|
||||||
|
curl http://localhost:8080/api/config
|
||||||
|
```
|
||||||
|
|
||||||
|
## Konfiguration & Profile
|
||||||
|
|
||||||
|
Höhere Ebene gewinnt:
|
||||||
|
|
||||||
|
```
|
||||||
|
eingebaute Defaults < config/voice-assistant.toml (inkl. aktivem Profil)
|
||||||
|
< ENV / .env < Session-Route < Request
|
||||||
|
```
|
||||||
|
|
||||||
|
**Profile** umschalten per Umgebungsvariable (oder dauerhaft in `.env`):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
VA_PROFILE=local-dev make run # alles lokal (faster-whisper / lokales LLM / piper)
|
||||||
|
VA_PROFILE=hybrid make run # STT/TTS remote, LLM lokal
|
||||||
|
VA_PROFILE=cloud make run # alles über OpenRouter
|
||||||
|
```
|
||||||
|
|
||||||
|
> Secrets gehören **nicht** in `config/*.toml` — nur in die Umgebung
|
||||||
|
> (`export OPENROUTER_API_KEY=…`). Gesetzte `DEFAULT_*_PROVIDER`-Werte in `.env`
|
||||||
|
> überschreiben ein `VA_PROFILE`.
|
||||||
|
|
||||||
|
**Pro Session:** `POST /api/sessions/{id}/route` (`input_endpoint`, `output_endpoint`,
|
||||||
|
`stt_provider`, `llm_provider`, `tts_provider`, `language`), dann Aufrufe mit `?session_id=…`.
|
||||||
|
|
||||||
|
**Pro Request:** dieselben Felder im Body von `/api/chat` bzw. `/api/speak`.
|
||||||
|
|
||||||
|
Aktive Konfiguration prüfen: `curl http://localhost:8080/api/config`.
|
||||||
|
|
||||||
|
## API-Überblick
|
||||||
|
|
||||||
|
| Methode & Pfad | Zweck |
|
||||||
|
|---------------------------------|-------|
|
||||||
|
| `GET /health` | Liveness-Check |
|
||||||
|
| `POST /api/chat` | Text rein → Audio raus (`?debug=true` → JSON-Trace) |
|
||||||
|
| `POST /api/speak` | Text rein → TTS-Audio raus |
|
||||||
|
| `POST /api/transcribe` | Audio-Upload → Transkript |
|
||||||
|
| `GET /api/devices` | verfügbare Audio-Endpunkte + Capabilities |
|
||||||
|
| `POST /api/sessions/{id}/route` | Geräte/Provider/Sprache je Session setzen |
|
||||||
|
| `GET /api/config` | aktives Profil + aufgelöste Route (ohne Secrets) |
|
||||||
|
|
||||||
|
Beispiel (Sprachausgabe an den Test-Loopback, lokaler TTS-Stub):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST http://localhost:8080/api/speak \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d '{"text":"Guten Morgen!","tts_provider":"piper","output_endpoint":"loopback"}'
|
||||||
|
```
|
||||||
|
|
||||||
|
Unbekannter Endpunkt/Provider → `HTTP 422` mit Klartext-Hinweis.
|
||||||
|
|
||||||
|
## Tests
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make test # oder: pytest -q
|
||||||
|
```
|
||||||
|
|
||||||
|
Abgedeckt: Config-Profile & Präzedenz, Route-Auflösung, Device Router,
|
||||||
|
End-to-End (Loopback, 422-Fälle, Session-/Request-Override, `/api/config`).
|
||||||
|
|
||||||
|
## Port ändern
|
||||||
|
|
||||||
|
```bash
|
||||||
|
PORT=8003 make run # einmalig
|
||||||
|
sed -i 's/^PORT=.*/PORT=8003/' .env # dauerhaft
|
||||||
|
PORT=8003 docker compose up # mit Docker
|
||||||
|
```
|
||||||
|
|
||||||
|
## Deployment
|
||||||
|
|
||||||
|
- **Docker:** `docker compose up --build` (reicht `OPENROUTER_API_KEY` aus der Shell durch)
|
||||||
|
- **systemd:** Vorlagen unter `deploy/` (`voice-assistant.service`, `voice-assistant.env.example`)
|
||||||
|
|
||||||
|
## Projektstruktur (Kurzform)
|
||||||
|
|
||||||
|
```text
|
||||||
|
app/ Gateway: config, dependencies, api/, core/, audio/, pipeline/, providers/
|
||||||
|
config/ voice-assistant.example.toml (lokale .toml ist gitignored)
|
||||||
|
deploy/ systemd-Unit + env-Beispiel
|
||||||
|
tests/ Pytest-Suite
|
||||||
|
Docs/ Architektur-Dokument
|
||||||
|
```
|
||||||
|
|
||||||
|
## Lizenz / Status
|
||||||
|
|
||||||
|
Frühes, aktiv entwickeltes Projektgerüst. Audio-Hardware-/Streaming-Anbindung,
|
||||||
|
Authentifizierung, Persistenz und Gedächtnis sind als nächste Schritte vorgesehen
|
||||||
|
(siehe Roadmap im Architektur-Dokument).
|
||||||
0
app/__init__.py
Normal file
0
app/__init__.py
Normal file
0
app/api/__init__.py
Normal file
0
app/api/__init__.py
Normal file
92
app/api/chat.py
Normal file
92
app/api/chat.py
Normal file
|
|
@ -0,0 +1,92 @@
|
||||||
|
from io import BytesIO
|
||||||
|
|
||||||
|
from fastapi import APIRouter, HTTPException, Query
|
||||||
|
from fastapi.responses import JSONResponse, StreamingResponse
|
||||||
|
|
||||||
|
from app.config import settings
|
||||||
|
from app.errors import RoutingError
|
||||||
|
from app.dependencies import (
|
||||||
|
resolve_route,
|
||||||
|
build_orchestrator,
|
||||||
|
resolve_output_endpoint,
|
||||||
|
)
|
||||||
|
from app.schemas import ChatRequest
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
def _route_headers(route) -> dict:
|
||||||
|
return {
|
||||||
|
"X-Input-Endpoint": route.input_endpoint,
|
||||||
|
"X-Output-Endpoint": route.output_endpoint,
|
||||||
|
"X-STT-Provider": route.stt_provider,
|
||||||
|
"X-LLM-Provider": route.llm_provider,
|
||||||
|
"X-TTS-Provider": route.tts_provider,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/chat")
|
||||||
|
async def chat(
|
||||||
|
payload: ChatRequest,
|
||||||
|
debug: bool = Query(
|
||||||
|
default=False,
|
||||||
|
description="Return JSON trace instead of audio response",
|
||||||
|
),
|
||||||
|
session_id: str | None = Query(
|
||||||
|
default=None,
|
||||||
|
description="Optional session id to apply a stored route",
|
||||||
|
),
|
||||||
|
):
|
||||||
|
overrides = {
|
||||||
|
"input_endpoint": payload.input_endpoint,
|
||||||
|
"output_endpoint": payload.output_endpoint,
|
||||||
|
"language": payload.language,
|
||||||
|
"stt_provider": payload.stt_provider,
|
||||||
|
"llm_provider": payload.llm_provider,
|
||||||
|
"tts_provider": payload.tts_provider,
|
||||||
|
}
|
||||||
|
route = resolve_route(session_id, overrides)
|
||||||
|
voice = payload.voice or settings.openrouter_tts_voice
|
||||||
|
|
||||||
|
try:
|
||||||
|
orchestrator = build_orchestrator(route)
|
||||||
|
output = await resolve_output_endpoint(route)
|
||||||
|
except RoutingError as exc:
|
||||||
|
raise HTTPException(status_code=422, detail=str(exc))
|
||||||
|
|
||||||
|
try:
|
||||||
|
trace, audio = await orchestrator.chat_text(
|
||||||
|
payload.text,
|
||||||
|
language=route.language,
|
||||||
|
voice=voice,
|
||||||
|
output=output,
|
||||||
|
)
|
||||||
|
|
||||||
|
if debug:
|
||||||
|
return JSONResponse(
|
||||||
|
content={
|
||||||
|
"ok": True,
|
||||||
|
"voice": voice,
|
||||||
|
"route": route.as_dict(),
|
||||||
|
"trace": {
|
||||||
|
"raw_transcript": trace.raw_transcript,
|
||||||
|
"cleaned_transcript": trace.cleaned_transcript,
|
||||||
|
"semantic_response": trace.semantic_response,
|
||||||
|
"spoken_response": trace.spoken_response,
|
||||||
|
"tts_ready_text": trace.tts_ready_text,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
headers = {
|
||||||
|
"Content-Language": route.language,
|
||||||
|
"X-Audio-Format": "pcm",
|
||||||
|
"X-Audio-Sample-Rate": "24000",
|
||||||
|
"X-Audio-Channels": "1",
|
||||||
|
"X-Audio-Sample-Width": "16",
|
||||||
|
**_route_headers(route),
|
||||||
|
}
|
||||||
|
return StreamingResponse(BytesIO(audio), media_type="audio/pcm", headers=headers)
|
||||||
|
|
||||||
|
except Exception as exc:
|
||||||
|
raise HTTPException(status_code=502, detail=str(exc))
|
||||||
37
app/api/config.py
Normal file
37
app/api/config.py
Normal file
|
|
@ -0,0 +1,37 @@
|
||||||
|
from fastapi import APIRouter
|
||||||
|
|
||||||
|
from app.config import settings, active_profile
|
||||||
|
from app.dependencies import (
|
||||||
|
resolve_route,
|
||||||
|
get_audio_router,
|
||||||
|
STT_REGISTRY,
|
||||||
|
LLM_REGISTRY,
|
||||||
|
TTS_REGISTRY,
|
||||||
|
)
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/config")
|
||||||
|
async def get_config():
|
||||||
|
"""Zeigt aktives Profil, aufgeloeste Default-Route und verfuegbare Bausteine.
|
||||||
|
|
||||||
|
Bewusst OHNE Secrets - API-Keys werden nur als 'gesetzt/nicht gesetzt' gemeldet.
|
||||||
|
"""
|
||||||
|
route = resolve_route()
|
||||||
|
audio_router = get_audio_router()
|
||||||
|
return {
|
||||||
|
"profile": active_profile(),
|
||||||
|
"app_env": settings.app_env,
|
||||||
|
"default_route": route.as_dict(),
|
||||||
|
"available": {
|
||||||
|
"stt_providers": sorted(STT_REGISTRY),
|
||||||
|
"llm_providers": sorted(LLM_REGISTRY),
|
||||||
|
"tts_providers": sorted(TTS_REGISTRY),
|
||||||
|
"input_endpoints": [c.model_dump() for c in await audio_router.list_inputs()],
|
||||||
|
"output_endpoints": [c.model_dump() for c in await audio_router.list_outputs()],
|
||||||
|
},
|
||||||
|
"secrets": {
|
||||||
|
"openrouter_api_key_set": bool(settings.openrouter_api_key.strip()),
|
||||||
|
},
|
||||||
|
}
|
||||||
12
app/api/devices.py
Normal file
12
app/api/devices.py
Normal file
|
|
@ -0,0 +1,12 @@
|
||||||
|
from fastapi import APIRouter
|
||||||
|
from app.dependencies import get_audio_router
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
@router.get("/devices")
|
||||||
|
async def list_devices():
|
||||||
|
audio_router = get_audio_router()
|
||||||
|
return {
|
||||||
|
"inputs": [item.model_dump() for item in await audio_router.list_inputs()],
|
||||||
|
"outputs": [item.model_dump() for item in await audio_router.list_outputs()],
|
||||||
|
}
|
||||||
7
app/api/health.py
Normal file
7
app/api/health.py
Normal file
|
|
@ -0,0 +1,7 @@
|
||||||
|
from fastapi import APIRouter
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
@router.get("/health")
|
||||||
|
async def health():
|
||||||
|
return {"status": "ok"}
|
||||||
10
app/api/sessions.py
Normal file
10
app/api/sessions.py
Normal file
|
|
@ -0,0 +1,10 @@
|
||||||
|
from fastapi import APIRouter
|
||||||
|
from app.schemas import SessionRouteRequest
|
||||||
|
from app.dependencies import session_manager
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
@router.post("/sessions/{session_id}/route")
|
||||||
|
async def set_session_route(session_id: str, payload: SessionRouteRequest):
|
||||||
|
session = session_manager.update(session_id, payload.model_dump())
|
||||||
|
return {"session_id": session_id, "route": session}
|
||||||
60
app/api/speak.py
Normal file
60
app/api/speak.py
Normal file
|
|
@ -0,0 +1,60 @@
|
||||||
|
from io import BytesIO
|
||||||
|
|
||||||
|
from fastapi import APIRouter, HTTPException, Query
|
||||||
|
from fastapi.responses import StreamingResponse
|
||||||
|
|
||||||
|
from app.config import settings
|
||||||
|
from app.errors import RoutingError
|
||||||
|
from app.dependencies import (
|
||||||
|
resolve_route,
|
||||||
|
build_orchestrator,
|
||||||
|
resolve_output_endpoint,
|
||||||
|
)
|
||||||
|
from app.schemas import SpeakRequest
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/speak")
|
||||||
|
async def speak(
|
||||||
|
payload: SpeakRequest,
|
||||||
|
session_id: str | None = Query(
|
||||||
|
default=None,
|
||||||
|
description="Optional session id to apply a stored route",
|
||||||
|
),
|
||||||
|
):
|
||||||
|
overrides = {
|
||||||
|
"output_endpoint": payload.output_endpoint,
|
||||||
|
"language": payload.language,
|
||||||
|
"tts_provider": payload.tts_provider,
|
||||||
|
}
|
||||||
|
route = resolve_route(session_id, overrides)
|
||||||
|
voice = payload.voice or settings.openrouter_tts_voice
|
||||||
|
|
||||||
|
try:
|
||||||
|
orchestrator = build_orchestrator(route)
|
||||||
|
output = await resolve_output_endpoint(route)
|
||||||
|
except RoutingError as exc:
|
||||||
|
raise HTTPException(status_code=422, detail=str(exc))
|
||||||
|
|
||||||
|
try:
|
||||||
|
audio = await orchestrator.speak_only(
|
||||||
|
payload.text,
|
||||||
|
voice=voice,
|
||||||
|
language=route.language,
|
||||||
|
output=output,
|
||||||
|
)
|
||||||
|
|
||||||
|
headers = {
|
||||||
|
"Content-Language": route.language,
|
||||||
|
"X-Audio-Format": "pcm",
|
||||||
|
"X-Audio-Sample-Rate": "24000",
|
||||||
|
"X-Audio-Channels": "1",
|
||||||
|
"X-Audio-Sample-Width": "16",
|
||||||
|
"X-Output-Endpoint": route.output_endpoint,
|
||||||
|
"X-TTS-Provider": route.tts_provider,
|
||||||
|
}
|
||||||
|
return StreamingResponse(BytesIO(audio), media_type="audio/pcm", headers=headers)
|
||||||
|
|
||||||
|
except Exception as exc:
|
||||||
|
raise HTTPException(status_code=502, detail=str(exc))
|
||||||
50
app/api/transcribe.py
Normal file
50
app/api/transcribe.py
Normal file
|
|
@ -0,0 +1,50 @@
|
||||||
|
from fastapi import APIRouter, File, Form, HTTPException, Query, UploadFile
|
||||||
|
|
||||||
|
from app.errors import RoutingError
|
||||||
|
from app.dependencies import (
|
||||||
|
resolve_route,
|
||||||
|
build_orchestrator,
|
||||||
|
resolve_input_endpoint,
|
||||||
|
)
|
||||||
|
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/transcribe")
|
||||||
|
async def transcribe(
|
||||||
|
file: UploadFile = File(...),
|
||||||
|
language: str | None = Form(default=None),
|
||||||
|
input_endpoint: str | None = Form(default=None),
|
||||||
|
stt_provider: str | None = Form(default=None),
|
||||||
|
session_id: str | None = Query(
|
||||||
|
default=None,
|
||||||
|
description="Optional session id to apply a stored route",
|
||||||
|
),
|
||||||
|
):
|
||||||
|
overrides = {
|
||||||
|
"input_endpoint": input_endpoint,
|
||||||
|
"language": language,
|
||||||
|
"stt_provider": stt_provider,
|
||||||
|
}
|
||||||
|
route = resolve_route(session_id, overrides)
|
||||||
|
|
||||||
|
try:
|
||||||
|
orchestrator = build_orchestrator(route)
|
||||||
|
source = await resolve_input_endpoint(route)
|
||||||
|
except RoutingError as exc:
|
||||||
|
raise HTTPException(status_code=422, detail=str(exc))
|
||||||
|
|
||||||
|
content = await file.read()
|
||||||
|
suffix = (file.filename or "audio.wav").rsplit(".", 1)[-1].lower()
|
||||||
|
|
||||||
|
try:
|
||||||
|
trace = await orchestrator.transcribe_only(
|
||||||
|
content,
|
||||||
|
fmt=suffix,
|
||||||
|
language=route.language,
|
||||||
|
input=source,
|
||||||
|
)
|
||||||
|
except Exception as exc:
|
||||||
|
raise HTTPException(status_code=502, detail=str(exc))
|
||||||
|
|
||||||
|
return {"route": route.as_dict(), "trace": trace.model_dump()}
|
||||||
0
app/audio/__init__.py
Normal file
0
app/audio/__init__.py
Normal file
0
app/audio/endpoints/__init__.py
Normal file
0
app/audio/endpoints/__init__.py
Normal file
0
app/audio/endpoints/input/__init__.py
Normal file
0
app/audio/endpoints/input/__init__.py
Normal file
17
app/audio/endpoints/input/base.py
Normal file
17
app/audio/endpoints/input/base.py
Normal file
|
|
@ -0,0 +1,17 @@
|
||||||
|
from abc import ABC, abstractmethod
|
||||||
|
from app.schemas import AudioChunk, EndpointCapabilities
|
||||||
|
|
||||||
|
class AudioInputEndpoint(ABC):
|
||||||
|
endpoint_id: str
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
async def capabilities(self) -> EndpointCapabilities: ...
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
async def open(self) -> None: ...
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
async def read_chunk(self) -> AudioChunk: ...
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
async def close(self) -> None: ...
|
||||||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Reference in a new issue