feat(safety): zweistufige Notfall-Erkennung mit LLM-Klassifikation
- app/safety/llm_classifier.py: LLM klassifiziert Aeusserungen (medical/self_harm/ help/none) mit Konfidenz-Schwelle, eigener JSON-Prompt, Reasoning aus, temp 0 - emergency.py: Eskalation in _escalate() gebuendelt (Log/Metrik/Webhook, source-Label); schedule_llm_emergency_check() faehrt Stufe 2 als Hintergrund-Task NUR wenn die Stichwort-Heuristik nichts fand -> faengt verpasste Formulierungen ohne Mehrlatenz - /ws/voice schickt bei LLM-Treffer ein nachgelagertes emergency-Event (source=llm) - Konfig: EMERGENCY_LLM_ENABLED/_PROVIDER/_MIN_CONFIDENCE - Tests: Parsing, Konfidenz-Schwelle, Skip-bei-Keyword-Treffer, Eskalation+Callback, disabled, kaputtes JSON - Doku: README + Architektur-Roadmap (Punkt 6) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
aa64ccf585
commit
855fc71a1e
9 changed files with 341 additions and 25 deletions
|
|
@ -31,7 +31,7 @@ from app.store import SessionOwnershipError
|
|||
from app.audio.vad import EnergyVAD
|
||||
from app.core.memory_extractor import maybe_schedule_extraction
|
||||
from app.quota import enforce_quota, record_usage, QuotaExceededError
|
||||
from app.safety.emergency import handle_emergency
|
||||
from app.safety.emergency import handle_emergency, schedule_llm_emergency_check
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
|
@ -80,6 +80,20 @@ async def _run_turn(websocket, store, user, session_id, route, orchestrator, out
|
|||
|
||||
# Notfall-Erkennung zuerst (immer eskalieren, auch bei Quota-Limit).
|
||||
emergency = handle_emergency(user, text, store)
|
||||
|
||||
async def _on_llm_emergency(category):
|
||||
try:
|
||||
await websocket.send_json(
|
||||
{"type": "emergency", "category": category, "source": "llm"}
|
||||
)
|
||||
except Exception: # Socket evtl. geschlossen -> ignorieren
|
||||
pass
|
||||
|
||||
# Stufe 2: LLM-Klassifikation als Hintergrund-Task (nur wenn Stichwoerter nichts fanden).
|
||||
schedule_llm_emergency_check(
|
||||
user, text, store, emergency, on_emergency=_on_llm_emergency
|
||||
)
|
||||
|
||||
if emergency:
|
||||
await websocket.send_json({"type": "emergency", "category": emergency["category"]})
|
||||
else:
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue