feat: Token-Level-LLM-Streaming über WebSocket (#4 Ausbau)

- LLMProvider.stream: Basis-Default (Fallback über complete) + SSE-Streaming
  fuer OpenRouter und lokalen OpenAI-kompatiblen Provider; gemeinsamer Parser sse_delta
- Orchestrator.chat_stream: LLM-Token live via on_token-Callback, danach
  Spoken-Adapter/Normalizer/TTS/Output; Fallback fuer Provider ohne stream
- WS /ws/chat: opt-in {"stream":true} -> ack -> token* -> semantic -> audio -> done
- Tests: 43 gruen (+5: SSE-Parsing, Default-Fallback, WS-Token-Flow)
- Doku aktualisiert; .gitignore: *.wav (generierte Audio-Ausgaben)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Dieter Schlüter 2026-06-17 04:37:37 +02:00
commit 379e002460
10 changed files with 302 additions and 24 deletions

View file

@ -95,14 +95,28 @@ async def ws_chat(
await websocket.send_json({"type": "ack", "route": route.as_dict()})
voice = msg.get("voice") or settings.openrouter_tts_voice
stream = bool(msg.get("stream"))
try:
trace, audio = await orchestrator.chat_text(
text,
language=route.language,
voice=voice,
output=output,
history=llm_context,
)
if stream:
async def on_token(delta):
await websocket.send_json({"type": "token", "text": delta})
trace, audio = await orchestrator.chat_stream(
text,
language=route.language,
voice=voice,
output=output,
history=llm_context,
on_token=on_token,
)
else:
trace, audio = await orchestrator.chat_text(
text,
language=route.language,
voice=voice,
output=output,
history=llm_context,
)
except Exception as exc:
await websocket.send_json({"type": "error", "status": 502, "detail": str(exc)})
continue