feat: Token-Level-LLM-Streaming über WebSocket (#4 Ausbau)
- LLMProvider.stream: Basis-Default (Fallback über complete) + SSE-Streaming
fuer OpenRouter und lokalen OpenAI-kompatiblen Provider; gemeinsamer Parser sse_delta
- Orchestrator.chat_stream: LLM-Token live via on_token-Callback, danach
Spoken-Adapter/Normalizer/TTS/Output; Fallback fuer Provider ohne stream
- WS /ws/chat: opt-in {"stream":true} -> ack -> token* -> semantic -> audio -> done
- Tests: 43 gruen (+5: SSE-Parsing, Default-Fallback, WS-Token-Flow)
- Doku aktualisiert; .gitignore: *.wav (generierte Audio-Ausgaben)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
531b57e08d
commit
379e002460
10 changed files with 302 additions and 24 deletions
|
|
@ -95,14 +95,28 @@ async def ws_chat(
|
|||
await websocket.send_json({"type": "ack", "route": route.as_dict()})
|
||||
|
||||
voice = msg.get("voice") or settings.openrouter_tts_voice
|
||||
stream = bool(msg.get("stream"))
|
||||
try:
|
||||
trace, audio = await orchestrator.chat_text(
|
||||
text,
|
||||
language=route.language,
|
||||
voice=voice,
|
||||
output=output,
|
||||
history=llm_context,
|
||||
)
|
||||
if stream:
|
||||
async def on_token(delta):
|
||||
await websocket.send_json({"type": "token", "text": delta})
|
||||
|
||||
trace, audio = await orchestrator.chat_stream(
|
||||
text,
|
||||
language=route.language,
|
||||
voice=voice,
|
||||
output=output,
|
||||
history=llm_context,
|
||||
on_token=on_token,
|
||||
)
|
||||
else:
|
||||
trace, audio = await orchestrator.chat_text(
|
||||
text,
|
||||
language=route.language,
|
||||
voice=voice,
|
||||
output=output,
|
||||
history=llm_context,
|
||||
)
|
||||
except Exception as exc:
|
||||
await websocket.send_json({"type": "error", "status": 502, "detail": str(exc)})
|
||||
continue
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue