dschlueter
703a695bed
Frische-/Web-Such-Funktion: Das zentrale Modell entscheidet selbst via web_search-Tool, ob es tagesaktuelle Fakten braucht, holt sie über perplexity/sonar und formuliert die Antwort in Persona (Augment). - SonarTool (app/tools/web_search.py): Fakten via perplexity/sonar, Citations als Metadaten, honest-punt-Sentinel bei Fehler/Timeout. - ToolCallingLLM (app/providers/llm/tool_calling.py): agentischer Loop als LLMProvider; complete() + gestreamtes stream() mit SSE-Tool-Assembler; Persona- + Trigger- + Vorrang-Prompt (Tool-Ergebnis schlaegt Gedaechtnis). - Verdrahtung: Registry-Eintrag openrouter-tools; web_search_enabled (global an, pro Nutzer/Profil abschaltbar) via Route-Layering; build_orchestrator waehlt tool-faehig vs. plain, Fallback-Kette erhalten. - Filler: ephemerer Beruhigungssatz beim Tool-Start (sofort angezeigt UND gesprochen als Satz null), nie in semantic_response/History; on_tool_start defensiv durch die stream()-Kette gefaedelt (kein Bruch bestehender Provider). - Modellwechsel: Standard auf mistralai/mistral-small-3.2-24b-instruct (tool-faehig; im Eval einziger Recall-Gate-Passer). 2501 ist tool-unfaehig. - Eval-Harness (eval/tool_calling/): Datensatz + Runner zur Modellauswahl. Doc: Docs/weg2-tool-calling.md. Tests: 290 gruen. Offen (Schritt 5): Koreferenz-Vorstufe (nl-Pronomen) + Metrik-Zaehler. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
105 lines
3.9 KiB
Python
105 lines
3.9 KiB
Python
"""Fallback-Ketten: versuchen mehrere Provider der Reihe nach.
|
|
|
|
Faellt der primaere Provider aus (Timeout/Fehler), wird transparent der naechste
|
|
versucht. Erfolgreicher Fallback und Provider-Fehler werden als Metrik erfasst.
|
|
"""
|
|
|
|
from collections.abc import AsyncIterator
|
|
|
|
from app.metrics import metrics
|
|
|
|
|
|
class _Chain:
|
|
def __init__(self, module: str, entries: list[tuple[str, object]]):
|
|
self.module = module
|
|
self.entries = entries # [(provider_name, provider), ...]
|
|
|
|
def _on_error(self, name: str) -> None:
|
|
metrics.inc("provider_error_total", {"module": self.module, "provider": name})
|
|
|
|
def _on_fallback(self) -> None:
|
|
metrics.inc("provider_fallback_total", {"module": self.module})
|
|
|
|
|
|
class FallbackSTTProvider(_Chain):
|
|
async def transcribe(self, audio_bytes, fmt, language=None) -> str:
|
|
last_exc = None
|
|
for index, (name, provider) in enumerate(self.entries):
|
|
try:
|
|
result = await provider.transcribe(audio_bytes, fmt, language=language)
|
|
if index > 0:
|
|
self._on_fallback()
|
|
return result
|
|
except Exception as exc: # noqa: BLE001 - bewusst breit fuer Resilienz
|
|
last_exc = exc
|
|
self._on_error(name)
|
|
raise last_exc
|
|
|
|
async def transcribe_detect(self, audio_bytes, fmt, language=None) -> tuple[str, str | None]:
|
|
last_exc = None
|
|
for index, (name, provider) in enumerate(self.entries):
|
|
try:
|
|
result = await provider.transcribe_detect(audio_bytes, fmt, language=language)
|
|
if index > 0:
|
|
self._on_fallback()
|
|
return result
|
|
except Exception as exc: # noqa: BLE001
|
|
last_exc = exc
|
|
self._on_error(name)
|
|
raise last_exc
|
|
|
|
|
|
class FallbackLLMProvider(_Chain):
|
|
async def complete(self, text, history=None, session_id=None, language=None) -> str:
|
|
last_exc = None
|
|
for index, (name, provider) in enumerate(self.entries):
|
|
try:
|
|
result = await provider.complete(
|
|
text, history=history, session_id=session_id, language=language
|
|
)
|
|
if index > 0:
|
|
self._on_fallback()
|
|
return result
|
|
except Exception as exc: # noqa: BLE001
|
|
last_exc = exc
|
|
self._on_error(name)
|
|
raise last_exc
|
|
|
|
async def stream(self, text, history=None, session_id=None, language=None,
|
|
**kwargs) -> AsyncIterator[str]:
|
|
last_exc = None
|
|
for index, (name, provider) in enumerate(self.entries):
|
|
produced = False
|
|
try:
|
|
async for delta in provider.stream(
|
|
text, history=history, session_id=session_id, language=language, **kwargs
|
|
):
|
|
produced = True
|
|
yield delta
|
|
if index > 0:
|
|
self._on_fallback()
|
|
return
|
|
except Exception as exc: # noqa: BLE001
|
|
last_exc = exc
|
|
self._on_error(name)
|
|
if produced:
|
|
# Schon Token gesendet -> kein Fallback mehr moeglich.
|
|
raise
|
|
raise last_exc
|
|
|
|
|
|
class FallbackTTSProvider(_Chain):
|
|
async def synthesize(self, text, voice=None, audio_format="pcm", language=None) -> bytes:
|
|
last_exc = None
|
|
for index, (name, provider) in enumerate(self.entries):
|
|
try:
|
|
result = await provider.synthesize(
|
|
text, voice=voice, audio_format=audio_format, language=language
|
|
)
|
|
if index > 0:
|
|
self._on_fallback()
|
|
return result
|
|
except Exception as exc: # noqa: BLE001
|
|
last_exc = exc
|
|
self._on_error(name)
|
|
raise last_exc
|