feat: KI-gestützte Inhaltsanalyse via OpenRouter (hash-gegated)
Optionale semantische Prüfung von Text und Bildern (inkl. OCR) auf problematische Inhalte ohne Link-Signal: Pornografie, Propaganda, diffamierende/strafbare Texte, versteckter Spam, themenfremde Werbung, widersprüchliche Aussagen. Bewertet die thematische Passung zum deklarierten site_context — eingestreute Heimat-Begriffe täuschen die Erkennung nicht. Kern: - scanner/ai_analyzer.py: run_ai_analysis + score_ai_findings. Hash-Gate über data/ai_ledger.json → unveränderte Inhalte = Cache- Treffer = kein API-Call. Nur neue/geänderte Inhalte kosten etwas. - Modell-Kette mit zweifacher Eskalation (Stufe 1+2 free, Stufe 3 günstig bezahlt); eskaliert bei Fehler ODER Timeout (attempt_timeout). Erfolgreiches Modell wird im Ledger vermerkt. - KI-Funde sind auf GELB gedeckelt — ROT bleibt harten Integritäts- Signalen vorbehalten. Graceful degradation: ohne Key/bei Fehler wird übersprungen, Scan läuft unverändert weiter. Integration: - baseline.py: load/save_ai_ledger, dismiss_ai_entries. - config.py: ai_analysis-Block + ai_* Scoring-Schlüssel. - __main__.py: Einhängung in cmd_scan/cmd_check, ai-dismiss-Subcommand, approve quittiert zugehörige Funde, status-Anzeige, diff-only + report. - alerter.py + __main__.py: beanstandete Dateien erscheinen mit URL, Begründung und Quittier-Fingerprint in E-Mail UND Markdown-Report. - plain.py: laienverständliche KI-Sätze. API-Key nur aus Umgebungsvariable (OPENROUTER_API_KEY). Audio/Video als abschaltbare Hooks vorbereitet (default aus). Modelle live gegen OpenRouter verifiziert; Demo auf bredelar.info zeigte korrekte Erkennung (echter Inhalt clean, eingeschleuster Casino-Spam mit Heimat-Begriffen als hidden_spam erkannt). Tests: 227 grün (+26 für ai_analyzer, +1 für E-Mail-KI-Abschnitt). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
0aaf53135a
commit
cae3dbb985
11 changed files with 1301 additions and 7 deletions
|
|
@ -233,6 +233,55 @@ class BaselineManager:
|
|||
)
|
||||
logger.info("Asset hashes saved: %d entries", len(hashes))
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# AI verdict ledger (cache keyed by content fingerprint)
|
||||
# ------------------------------------------------------------------
|
||||
#
|
||||
# Unlike the baseline, the ledger is a *cache*: it records the AI verdict
|
||||
# for every fingerprint already analysed — including "clean" ones — so that
|
||||
# unchanged content is never re-sent to the API. It lives in data_dir
|
||||
# (not baseline_dir): it is operational state, not an approved reference.
|
||||
|
||||
@property
|
||||
def _ai_ledger_path(self) -> Path:
|
||||
return self.data_dir / "ai_ledger.json"
|
||||
|
||||
def load_ai_ledger(self) -> dict:
|
||||
"""Load the AI verdict ledger. Returns {'entries': {}} if none/corrupt."""
|
||||
if not self._ai_ledger_path.exists():
|
||||
return {"entries": {}}
|
||||
try:
|
||||
data = json.loads(self._ai_ledger_path.read_text(encoding="utf-8"))
|
||||
except (json.JSONDecodeError, ValueError):
|
||||
return {"entries": {}}
|
||||
data.setdefault("entries", {})
|
||||
return data
|
||||
|
||||
def save_ai_ledger(self, ledger: dict) -> None:
|
||||
"""Persist the AI verdict ledger."""
|
||||
self.data_dir.mkdir(parents=True, exist_ok=True)
|
||||
self._ai_ledger_path.write_text(
|
||||
json.dumps(ledger, indent=2, ensure_ascii=False), encoding="utf-8"
|
||||
)
|
||||
|
||||
def dismiss_ai_entries(self, fingerprints: list[str] | None = None) -> int:
|
||||
"""
|
||||
Mark ledger entries as dismissed (false-positive acknowledgement).
|
||||
None = dismiss all currently flagged entries. Returns count dismissed.
|
||||
"""
|
||||
ledger = self.load_ai_ledger()
|
||||
entries = ledger.get("entries", {})
|
||||
count = 0
|
||||
for fp, entry in entries.items():
|
||||
if fingerprints is not None and fp not in fingerprints:
|
||||
continue
|
||||
if entry.get("category", "clean") != "clean" and not entry.get("dismissed"):
|
||||
entry["dismissed"] = True
|
||||
count += 1
|
||||
if count:
|
||||
self.save_ai_ledger(ledger)
|
||||
return count
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Run-state (drives auto-scheduling of the weekly checks)
|
||||
# ------------------------------------------------------------------
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue