Docs: close the gaps an audit turned up

Audited README, BEDIENUNGSANLEITUNG and CLAUDE.md against what the repo and the running stack actually do. Findings, all fixed: - CLAUDE.md claimed the repo holds "only deployment config (docker-compose.yml, .env, .gitignore)". It has not been that for a while: services/ is our own source (TTS/STT servers + systemd units), scripts/ is tooling, and prompts/podcast/ + config/content_core.yaml are overlays bind-mounted over the image. Those overlays are the fragile part, so the section now says so and points at smoke_test.sh. - CLAUDE.md still described the image as a moving tag and argued against patching app code "because it would rot against pull_policy: always" — both obsolete since the digest pin. - README listed neither yt-dlp (needed by add_video_source.py) nor smoke_test.sh in the verification step, and section 2.3 still told you to `docker compose pull` as if the tag moved. - BEDIENUNGSANLEITUNG had nothing at all about updates — the very thing a user has to do periodically. New section 8 covers check_updates / update_stack / smoke_test and, importantly, why updating is not automatic (DB migration is one-way without a backup; the local adaptations can break silently without crashing). - Renumbered the ad-hoc "3a. Updates" in README into a real section 4, and the BEDIENUNGSANLEITUNG sections after the new 8 accordingly. Verified mechanically: every scripts//services//config//prompts/ path named in the docs exists, and all internal and cross-document anchors resolve (0 dead links — one was already broken before this change: README pointed at BEDIENUNGSANLEITUNG.md#stimmklonung instead of #5-stimmklonung). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pin images by digest; add update, smoke-test and status scripts
2026-07-11 14:17:06 +02:00 · 2026-07-11 14:12:31 +02:00 · 2026-07-11 13:49:42 +02:00 · 2026-07-11 13:20:14 +02:00 · 2026-07-11 13:14:25 +02:00 · 2026-07-11 13:02:32 +02:00
17 changed files with 1920 additions and 66 deletions
--- a/.gitignore
+++ b/.gitignore
@ -3,3 +3,13 @@ surreal_data/
 notebook_data/
 services/logs/
 services/voices/
 # Python
 __pycache__/
 *.pyc
 # Artefakt versehentlicher Shell-Redirects
 /3
 # Backups vor Image-Updates (scripts/update_stack.sh)
 backups/
--- a/BEDIENUNGSANLEITUNG.md
+++ b/BEDIENUNGSANLEITUNG.md
@ -15,7 +15,10 @@ siehe [`CLAUDE.md`](CLAUDE.md).
 5. [Stimmklonung](#5-stimmklonung)
 6. [Modelle und Provider verwalten](#6-modelle-und-provider-verwalten)
 7. [Dienste starten/stoppen](#7-dienste-startenstoppen)
-8. [Troubleshooting](#8-troubleshooting)
+8. [Updates einspielen](#8-updates-einspielen)
 9. [Suche über Chunks (Search/Ask)](#9-suche-über-chunks-searchask)
 10. [Die API in eigenen Programmen nutzen](#10-die-api-in-eigenen-programmen-nutzen)
 11. [Troubleshooting](#11-troubleshooting)
 ---
@ -116,6 +119,36 @@ mehreren Sprechern) erzeugen — Text via Sprachmodell, Audio via TTS (hier: lok
 Alle Episode-/Sprecherprofile sind bereits auf lokale Modelle (`qwen3.5:27b` für Text,
 Chatterbox für Sprache) umgestellt — keine zusätzlichen Kosten.
 ### Episoden löschen räumt nur halb auf
 **Der Mülleimer-Button löscht nicht alles.** Er entfernt die finale MP3 und den
 Datenbankeintrag — der Ordner der Episode bleibt aber liegen, samt Einzelclips:
 ```
 notebook_data/podcasts/episodes/<uuid>/
 ├── clips/            ← 1 MP3 pro Dialogsegment   BLEIBT LIEGEN
 ├── outline.json                                  BLEIBT LIEGEN
 ├── transcript.json                               BLEIBT LIEGEN
 └── audio/<uuid>.mp3  ← nur diese wird gelöscht
 ```
 Das ist eine Lücke im Open-Notebook-Code selbst (der Löschpfad ruft `unlink()` nur auf die eine
 MP3-Datei auf), nicht an dieser Installation einstellbar. Ebenso hinterlässt jeder
 fehlgeschlagene oder per Retry wiederholte Lauf einen Ordner, der nie aufgeräumt wird.
 Zum Aufräumen dient:
 ```bash
 ./scripts/prune_podcast_data.py          # zeigt nur an, was wegfiele (Trockenlauf)
 ./scripts/prune_podcast_data.py --yes    # löscht wirklich
 ```
 Es entfernt (a) Ordner ohne zugehörige Episode und (b) die `clips/` fertiger Episoden — die
 Einzelclips sind nach dem Zusammenbau reines Zwischenprodukt, die Episode bleibt vollständig
 abspielbar. Schutzmechanismen: bei einem laufenden Podcast-Job bricht es ab, und Ordner, die
 jünger als 60 Minuten sind, bleiben unangetastet (`--min-age`). Mit `--keep-clips` werden nur
 die verwaisten Ordner gelöscht.
 ---
 ## 5. Stimmklonung
@ -134,8 +167,7 @@ geklonten echten Personen, siehe [`CLAUDE.md`](CLAUDE.md)).
 2. Datei nach `services/voices/<name>.wav` legen (z. B. `services/voices/elena.wav`).
 3. TTS-Server neu starten (Stimmen werden nur beim Start eingelesen):
   ```bash
-   pkill -f "python tts_server.py"
+   systemctl --user restart open-notebook-tts
   ./scripts/start_services.sh
   ```
 4. Prüfen: `curl http://127.0.0.1:8901/health` sollte `<name>` in der `voices`-Liste zeigen.
 5. In einem Sprecherprofil (Podcast) den betreffenden Sprecher auf `voice_id: "<name>"` setzen,
@ -205,12 +237,17 @@ Weitere OpenRouter-Modelle lassen sich jederzeit über `POST /api/models` regist
 docker compose up -d
 docker compose down
-# TTS/STT (lokale Hintergrundprozesse, nicht in Docker)
+# TTS/STT (auf dem Host, nicht in Docker — systemd-User-Dienste)
-./scripts/start_services.sh
+systemctl --user status  open-notebook-tts open-notebook-stt
-pkill -f "python tts_server.py"
+systemctl --user restart open-notebook-tts open-notebook-stt
-pkill -f "python stt_server.py"
+systemctl --user stop    open-notebook-tts open-notebook-stt
 journalctl --user -u open-notebook-tts -f     # Logs
 ```
 TTS/STT starten nach einem Reboot **automatisch** mit. Nur bei der Ersteinrichtung (oder nach
 Änderungen an `services/systemd/*.service`) einmal `./scripts/start_services.sh` ausführen — das
 installiert die Unit-Dateien und aktiviert den Autostart.
 Ollama selbst läuft als systemd-Dienst und muss normalerweise nicht manuell verwaltet werden:
 ```bash
@ -220,7 +257,159 @@ ollama ps        # aktuell geladene Modelle + GPU/CPU-Verteilung
 ---
-## 8. Troubleshooting
+## 8. Updates einspielen
 Open Notebook läuft hier auf einer **festgenagelten Version** (Digest-Pin in
 `docker-compose.yml`). Weder ein Start noch ein Reboot tauscht die Anwendung unbemerkt aus — Sie
 bekommen immer genau den Stand, der zuletzt geprüft wurde.
 **Beim Sitzungsstart** (ändert nichts, dauert Sekunden):
 ```bash
 ./scripts/check_updates.sh
 ```
 Ausgabe entweder „Du laeufst auf dem aktuellen Release. Nichts zu tun." oder ein Hinweis auf ein
 neues Image.
 **Update einspielen:**
 ```bash
 ./scripts/update_stack.sh
 ```
 Das Skript sichert zuerst Datenbank und Notebook-Daten nach `backups/<Zeitstempel>/`, stellt dann
 auf das neue Image um, startet den Stack und fährt einen **Rauchtest** (API, Sprachausgabe,
 Spracherkennung, YouTube-Import, alle lokalen Anpassungen). Schlägt der fehl, wird **automatisch
 zurückgerollt** — Daten und Konfiguration landen wieder auf dem alten Stand.
 Läuft alles, committen Sie den neuen Stand:
 ```bash
 git add docker-compose.yml && git commit -m "Update auf <Version>"
 ```
 **Warum nicht einfach immer die neueste Version?** Ein Update verändert das Datenbankschema, und
 eine ältere Version kann eine bereits migrierte Datenbank in der Regel nicht mehr lesen — ohne
 Sicherung wäre ein misslungenes Update also eine Einbahnstraße. Außerdem hängen die Anpassungen
 dieser Installation (Podcast-Vorlagen, YouTube-Sprachen, TTS-Einstellungen) an Interna der
 Anwendung und können durch ein Update **still** kaputtgehen, ohne dass etwas abstürzt. Genau
 deshalb der Rauchtest.
 Ihn können Sie auch jederzeit einzeln laufen lassen, etwa wenn sich etwas seltsam verhält:
 ```bash
 ./scripts/smoke_test.sh
 ```
 ---
 ## 9. Suche über Chunks (Search/Ask)
 Jedes Dokument wird beim Hochladen automatisch in feste, überlappende Textabschnitte
 ("Chunks", ~400 Tokens) zerlegt und eingebettet — **nicht** kapitelweise, sondern rein
 größenbasiert (siehe Abschnitt 2). Um gezielt in diesen Chunks zu suchen, gibt es zwei
 API-Endpunkte (in der UI vermutlich unter einem eigenen Menüpunkt wie „Search"/„Ask" mit
 Lupen-Symbol, getrennt von „Mit Notebook chatten" — die genaue Beschriftung war ohne
 Browser-Zugriff nicht zu verifizieren):
 ### Einfache Suche — schnell, liefert Roh-Textstellen
 ```bash
 curl -s -X POST http://127.0.0.1:5055/api/search \
  -H "Content-Type: application/json" \
  -d '{"query": "Suchbegriff", "type": "vector", "limit": 5}'
 ```
 Liefert die ähnlichsten Chunks per Embedding-Vergleich samt Ähnlichkeits-Score (0–1), ohne
 KI-Interpretation — am schnellsten, aber du musst die Antwort selbst zusammensetzen.
 ### „Ask" — durchsuchen und direkt beantworten lassen
 Die smartere Variante: Die KI zerlegt die Frage selbst in mehrere gezielte Suchanfragen,
 durchsucht die Chunks für jede getrennt und fasst alles zu einer einzigen, mit
 Quellenverweisen (`[source:...]`) belegten Antwort zusammen. Getestet mit „Wie kam Wallenstein
 ums Leben?" — Ergebnis war eine vollständige, korrekt zitierte Antwort in ca. 3 Minuten
 (mehrere LLM-Aufrufe hintereinander: Suchstrategie → Einzelantworten → Zusammenfassung).
 ```bash
 MODEL_ID=$(curl -s http://127.0.0.1:5055/api/models/defaults | python3 -c "import json,sys; print(json.load(sys.stdin)['default_chat_model'])")
 curl -N -X POST http://127.0.0.1:5055/api/search/ask \
  -H "Content-Type: application/json" \
  -d "{\"question\": \"Deine Frage\", \"strategy_model\": \"$MODEL_ID\", \"answer_model\": \"$MODEL_ID\", \"final_answer_model\": \"$MODEL_ID\"}"
 ```
 Antwort kommt als Server-Sent-Events-Stream (`data: {...}`-Zeilen, kein einzelnes JSON-Objekt);
 das letzte `"type": "complete"`-Event enthält die finale Antwort im Feld `final_answer`.
 **Wann was nutzen:** Für gezielte Detailfragen über lange/viele Quellen ist „Ask" meist
 treffsicherer und dabei noch schneller als der „vollständiger Inhalt"-Chat-Modus (Abschnitt 3),
 weil nur die relevanten Chunks statt des ganzen Dokuments verarbeitet werden.
 ---
 ## 10. Die API in eigenen Programmen nutzen
 Die REST-API ist eine ganz normale JSON-über-HTTP-API **ohne Authentifizierung** — kein API-Key
 nötig, da nur `127.0.0.1` gebunden (jeder lokale Prozess auf diesem Rechner hat Zugriff, aber
 niemand aus dem Netzwerk). Funktioniert mit jeder Sprache, die HTTP kann.
 **Bester Startpunkt:** `http://127.0.0.1:5055/docs` — interaktive Swagger-UI mit allen
 Endpunkten, Schemas und einer „Try it out"-Funktion zum direkten Ausprobieren im Browser.
 Rohes OpenAPI-Schema: `http://127.0.0.1:5055/openapi.json`.
 ### Mit curl
 ```bash
 # Notebooks auflisten
 curl -s http://127.0.0.1:5055/api/notebooks | python3 -m json.tool
 # Neues Notebook anlegen
 curl -s -X POST http://127.0.0.1:5055/api/notebooks \
  -H "Content-Type: application/json" \
  -d '{"name": "Mein Notebook", "description": "..."}'
 # Text-Quelle hinzufügen (multipart, nicht JSON)
 curl -s -X POST http://127.0.0.1:5055/api/sources \
  -F "type=text" -F "notebook_id=notebook:xxx" -F "title=..." \
  -F "content=..." -F "embed=true"
 # Chat-Session erstellen + Nachricht senden
 curl -s -X POST http://127.0.0.1:5055/api/chat/sessions \
  -H "Content-Type: application/json" -d '{"notebook_id": "notebook:xxx"}'
 curl -s -X POST http://127.0.0.1:5055/api/chat/execute \
  -H "Content-Type: application/json" \
  -d '{"session_id": "chat_session:xxx", "message": "...", "context": {}}'
 ```
 Weitere Beispiele (Quellen-Insights, Podcast-Generierung, Modell-/Provider-Verwaltung,
 Chunk-Suche) stehen bereits in den jeweiligen Abschnitten dieser Anleitung.
 ### Aus Python
 ```python
 import requests
 BASE = "http://127.0.0.1:5055/api"
 notebooks = requests.get(f"{BASE}/notebooks").json()
 session = requests.post(f"{BASE}/chat/sessions",
                         json={"notebook_id": notebooks[0]["id"]}).json()
 answer = requests.post(f"{BASE}/chat/execute", json={
    "session_id": session["id"],
    "message": "Wovon handelt das Buch?",
    "context": {},
 }).json()
 print(answer["messages"][-1]["content"])
 ```
 Genauso mit JavaScript (`fetch`), Go, oder jeder anderen Sprache — nichts Open-Notebook-
 Spezifisches nötig, kein SDK erforderlich.
 ---
 ## 11. Troubleshooting
 ### „Chat gibt keine Antwort" (leere Antwort, kein Fehler)
@ -261,8 +450,8 @@ Such-/Ask-Funktion oder „nur Erkenntnisse" nach einer Zusammenfassung nutzen.
 docker compose exec open_notebook curl -sf http://host.docker.internal:8901/health
 ```
-Bei Timeout: `ufw`-Regeln fehlen (siehe README 2.4) oder die Server laufen nicht
+Bei Timeout: `ufw`-Regeln fehlen (siehe README 2.4) oder die Dienste laufen nicht
-(`./scripts/start_services.sh`).
+(`systemctl --user status open-notebook-tts open-notebook-stt`).
 ### Podcast-Profil zeigt „Einrichtung erforderlich"
@ -272,6 +461,114 @@ Das Episode- oder Sprecherprofil verweist auf ein Modell ohne konfiguriertes Cre
 `PUT /api/speaker-profiles/{id}` (`voice_model`) auf registrierte lokale/OpenRouter-Modelle
 umstellen — siehe `scripts/setup_models.sh` für die IDs der Standardmodelle.
 ### Podcast wird auf Englisch statt Deutsch erzeugt (oder auf Englisch mit deutschem Akzent vorgelesen)
 Das ist behoben: Die Podcast-Prompt-Vorlagen (`prompts/podcast/outline.jinja` und
 `transcript.jinja` in diesem Projekt) wurden um eine explizite Sprachanweisung ergänzt, die das
 `language`-Feld des Episode-Profils auswertet — sie werden per Bind-Mount aus `docker-compose.yml`
 in den Container eingehängt. Setzt du am Episode-Profil `language: "de"`, kommt der Podcast auf
 Deutsch. (Vorher hatte das `language`-Feld keine Wirkung, weil die mitgelieferten Vorlagen es
 ignorierten — daher englischer Text, der von der deutsch-fixierten TTS mit Akzent vorgelesen wurde.)
 Falls doch wieder Englisch erscheint, prüfe das `language`-Feld:
 ```bash
 curl -s http://127.0.0.1:5055/api/episode-profiles | python3 -c \
  "import json,sys; [print(p['name'], p['language']) for p in json.load(sys.stdin)]"
 ```
 Fehlt `de`/`de-DE`, per `PUT /api/episode-profiles/{id}` (vollständiger Body) ergänzen. Zusätzlich
 sind Briefing und Sprecherbeschreibungen der mitgelieferten Profile auf Deutsch übersetzt (hilft
 gegen Sprach-Drift, wenn der Quellinhalt englisch ist). **Wichtig:** Änderst du die
 Vorlagen-Dateien selbst, ist danach ein Container-Neustart nötig (`docker compose up -d
 --force-recreate open_notebook`), damit der Bind-Mount die neue Version übernimmt.
 ### Podcast-Erzeugung schlägt mit „Invalid json output" fehl
 Das Textmodell hat ungültiges JSON geliefert — passierte bei `qwen3.5:27b` (einem „denkenden"
 Modell), wenn der gesamte Notebook-Inhalt (bei einem ganzen Buch ~70.000 Tokens) in jeden
 Transkript-Abschnitt gepackt wird: Der Denk-Block frisst dann das Antwort-Budget auf und das JSON
 wird abgeschnitten. Behoben durch `/no_think` als erste Zeile beider Podcast-Vorlagen (schaltet
 den Denkmodus ab, ~3× schneller, volles Budget fürs JSON). Falls es dennoch auftritt: kürzeren
 Quellinhalt verwenden (nicht das ganze Buch) oder die Generierung erneut starten.
 ### Video ohne Untertitel hinzufügen (`scripts/add_video_source.py`)
 Open Notebook liest bei Videos **nur vorhandene Untertitel**. Gibt es keine, bleibt die Quelle
 leer. Für diesen Fall gibt es das Skript:
 ```bash
 ./scripts/add_video_source.py https://www.youtube.com/watch?v=...
 ./scripts/add_video_source.py <url> --notebook notebook:abc --language de
 ./scripts/add_video_source.py <url> --force-audio     # Untertitel ignorieren
 ```
 Es wählt den Weg selbst:
 - **Untertitel vorhanden** → die URL wird normal als Link-Quelle übergeben (schnell, kein
  Download).
 - **keine Untertitel** → `yt-dlp` lädt nur die Tonspur, die wird hochgeladen und von Open
  Notebook mit dem lokalen Whisper-Server transkribiert und eingebettet. Rechnen Sie mit etwa
  1–2 Minuten pro 20 Minuten Video (läuft auf GPU 2). Die hochgeladene Audiodatei wird danach
  automatisch wieder gelöscht.
 Das funktioniert für alles, was `yt-dlp` kennt (YouTube, Vimeo, Mediatheken …), nicht nur für
 YouTube. `--force-audio` ist auch dann nützlich, wenn die automatischen YouTube-Untertitel zu
 schlecht sind: Whisper liefert meist deutlich saubereren Text (richtige Satzzeichen,
 korrekte Eigennamen).
 Hinweis: Beim Audio-Weg wird die Audiodatei als Quell-Asset hinterlegt, die ursprüngliche
 Video-URL steht dann nicht mehr am Eintrag (beim Untertitel-Weg schon).
 ### YouTube-Video wird als leere Quelle angelegt
 Behoben. Symptom war: Quelle hinzufügen scheint zu klappen (der **Titel** erscheint), aber es gibt
 keinen Inhalt — kein Text, nichts zum Chatten, nichts zum Zusammenfassen.
 Ursache: Open Notebook holt YouTube-Untertitel über die Bibliothek `content-core`, und die suchte
 nur nach Transkripten in **Englisch, Spanisch und Portugiesisch**. Ein Video mit ausschließlich
 deutschem Transkript fiel damit durch; der Fehler stand nur im Container-Log, in der Oberfläche
 sah die Quelle einfach leer aus. Behoben durch `config/content_core.yaml` (per Bind-Mount
 eingehängt), die Deutsch an erste Stelle setzt.
 Falls doch mal ein Video leer bleibt, ist meist **gar kein Untertitel** vorhanden (weder manuell
 noch automatisch) — das lässt sich so prüfen:
 ```bash
 docker compose exec open_notebook /app/.venv/bin/python -c "
 from youtube_transcript_api import YouTubeTranscriptApi
 for t in YouTubeTranscriptApi().list('VIDEO_ID'): print(t.language_code, t.is_generated)"
 ```
 Kommt hier nichts zurück, hat das Video keine Untertitel — dann hilft nur, das Video separat
 herunterzuladen und über die STT-Funktion (Audio-Upload) zu transkribieren.
 ### Podcast schlägt beim Vertonen fehl („Failed to generate speech")
 Text und Transkript sind fertig, erst die Sprachausgabe scheitert. Die genaue Fehlermeldung
 (UI-Episodenliste oder `docker compose logs open_notebook`) unterscheidet **zwei verschiedene
 Ursachen** — die Verwechslung kostet sonst viel Zeit:
 | Meldung | Bedeutung | Abhilfe |
 |---|---|---|
 | `All connection attempts failed` (`ConnectError`) | Der TTS-Server **läuft nicht** — niemand nimmt den Request an. | `systemctl --user status open-notebook-tts`, ggf. `restart`. |
 | `OpenAI-compatible TTS endpoint error: HTTP 500` | Der TTS-Server **lebt**, bricht aber bei der Arbeit ab — meist `CUDA out of memory`. | `journalctl --user -u open-notebook-tts -n 50` prüfen. |
 Der GPU-Speicherfall entstand so: GPU 2 teilen sich drei Prozesse (TTS, STT und der separate
 `chatterbox-tts`-MCP-Dienst auf Port 9999), es bleiben nur ~12 GB übrig. Der Podcast-Generator
 schickte aber 5 Clips gleichzeitig, die im TTS-Server echt parallel auf derselben GPU liefen —
 der Speicherbedarf vervielfachte sich (bis 16,7 GB) und sprengte die Karte. Behoben durch zwei
 Maßnahmen: `tts_server.py` serialisiert seine Generierung intern (ein Clip zur Zeit — auf einer
 einzelnen GPU kostet das keinen Durchsatz), und `TTS_BATCH_SIZE=1` in `docker-compose.yml` sorgt
 dafür, dass gar nicht erst mehrere Anfragen gleichzeitig eintreffen. Ein fehlgeschlagener Podcast
 lässt sich anschließend ohne Neuanlage wiederholen:
 ```bash
 curl -X POST http://127.0.0.1:5055/api/podcasts/episodes/{episode_id}/retry
 ```
 (Der Retry legt einen neuen Episoden-Eintrag an; der alte, fehlgeschlagene verschwindet.)
 ### Weitere technische Details
 Siehe [`CLAUDE.md`](CLAUDE.md) — dort stehen alle bisher aufgetretenen Probleme mit exakter
--- a/CLAUDE.md
+++ b/CLAUDE.md
@ -4,25 +4,80 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 ## What this repo is
-This is a local Docker Compose deployment of [Open Notebook](https://github.com/lfnovo/open-notebook) (upstream image `lfnovo/open_notebook`), not a clone of its source. There is no application source code here — only deployment config (`docker-compose.yml`, `.env`, `.gitignore`). Application bugs/features belong upstream; this repo only concerns itself with how the stack is run locally.
+This is a local Docker Compose deployment of [Open Notebook](https://github.com/lfnovo/open-notebook) (upstream image `lfnovo/open_notebook`), not a clone of its source. The application's own code is **not** here and its bugs/features belong upstream — this repo only concerns itself with how the stack is run locally.
 What *is* here, beyond `docker-compose.yml`/`.env`:
 - `services/` — our own source: the TTS/STT wrapper servers (Open Notebook has no local TTS/STT provider) plus their systemd units.
 - `scripts/` — setup, maintenance and update tooling (see Commands below).
 - `prompts/podcast/`, `config/content_core.yaml` — **overlays** bind-mounted over files inside the image, to fix upstream behaviour we can't configure otherwise (podcast language/reliability, YouTube caption languages). These are the fragile part: they can silently rot when the image is updated, which is what `scripts/smoke_test.sh` exists to catch.
 ## Commands
 ```bash
-docker compose up -d              # start (pulls images per pull_policy: always)
+docker compose up -d              # start (images are digest-pinned; no implicit upgrade)
 docker compose down                # stop and remove containers (data volumes persist)
 docker compose logs -f open_notebook   # follow app logs
 docker compose ps                  # container status
 ./scripts/check_updates.sh         # is there a newer release? (reports only, changes nothing)
 ./scripts/update_stack.sh          # backup -> update -> smoke test -> rollback on failure
 ./scripts/smoke_test.sh            # verify the stack still does what we rely on
 ```
-No build/lint/test suite exists in this repo — nothing here to run.
+`smoke_test.sh` is the closest thing to a test suite here — run it after touching
 `docker-compose.yml`, the bind-mounted overlays, or the host TTS/STT services.
 ## Image versioning
 Images are **pinned by digest**, and updating is an explicit act — never a side effect of
 starting the stack.
 The previous setup (`v1-latest` + `pull_policy: always`) was unreliable in *both* directions:
 `docker compose up -d` silently pulled a new application — including an irreversible SurrealDB
 schema migration — while a reboot (`restart: always`) or `docker compose start` kept running the
 **old** image without any pull. So the running version was essentially unpredictable.
 Note `v1-latest` tracks *releases*, not `main`: the tag lags the repo by weeks. Chasing `main`
 would mean building from source and running unreleased code — explicitly not wanted here.
 Updating (`scripts/update_stack.sh`): resolve the new digest → stop → back up `surreal_data` +
 `notebook_data` + `docker-compose.yml` into `backups/<ts>/` → rewrite the digest → start → run
 `smoke_test.sh` → **roll back data and compose file if the smoke test fails**. The data backup is
 the point: the app migrates the database on startup, and an older application generally cannot
 read a migrated database, so a bad update would otherwise be a one-way door. Backup/restore runs
 `tar` **inside a container** because the data directories are root-owned (the container writes as
 root) and the host user cannot restore over them.
 Both scripts read the digest from the `lfnovo/open_notebook` line specifically — a naive "first
 sha256 in the file" grep picks up **surrealdb** (it is listed first), which would make the update
 script rewrite the database image instead of the application.
 ### Why a smoke test, and not just "does it start"
 Every local adaptation in this repo leans on upstream internals, and an update can break them
 *silently* — the stack comes up fine and misbehaves later:
 - the `prompts/podcast/` **directory** mount masks the image's directory: a third template added
  upstream would simply be invisible;
 - `config/content_core.yaml` is a full dump of content-core's defaults (because `CCORE_CONFIG_PATH`
  replaces rather than merges), so new upstream defaults would be frozen out;
 - the env-var workarounds (`TTS_BATCH_SIZE`, `OPENAI_COMPATIBLE_BASE_URL_STT`,
  `ESPERANTO_TTS_TIMEOUT`) depend on how content-core / esperanto / podcast_creator behave today.
 `smoke_test.sh` therefore checks those assumptions explicitly (template file list vs. the image,
 `{{ language }}` and `/no_think` still present, `language` still plumbed by podcast_creator,
 `TTS_BATCH_SIZE` still read, `preferred_languages` still used, esperanto still honouring
 `OPENAI_COMPATIBLE_BASE_URL_STT`, our config still covering all default keys) on top of the obvious
 end-to-end ones (API, TTS audio, STT, YouTube transcript). `SKIP_NET=1` skips the internet-dependent
 check.
 ## Architecture
 Two services, defined in `docker-compose.yml`:
 - **surrealdb** (`surrealdb/surrealdb:v2`) — the database, RocksDB-backed, bound to `127.0.0.1:8000` only (local debugging access, e.g. `surreal sql`). Credentials come from `SURREAL_USER`/`SURREAL_PASSWORD` (default `root`/`root` if unset in `.env`) and are shared with the `open_notebook` service so they stay in sync.
- **open_notebook** (`lfnovo/open_notebook:v1-latest`) — bundles the web UI (port 8502) and REST API (port 5055) in one image, both bound to `127.0.0.1` only. Connects to `surrealdb` over the internal compose network (`ws://surrealdb:8000/rpc`).
+- **open_notebook** (`lfnovo/open_notebook:v1-latest`, digest-pinned — see *Image versioning*) — bundles the web UI (port 8502) and REST API (port 5055) in one image, both bound to `127.0.0.1` only. Connects to `surrealdb` over the internal compose network (`ws://surrealdb:8000/rpc`).
 Data persists in `./surreal_data` and `./notebook_data` (bind mounts, gitignored).
@ -41,6 +96,117 @@ A separate local `llama.cpp` server exists on this machine (managed via `~/llama
 **Chat model context window:** the Ollama credential used for `qwen3.5:27b` (`credential:di8b31l16zikyyb6isyi`) has `num_ctx=98304` set explicitly. Without this, esperanto's `OllamaLanguageModel` silently defaults `num_ctx` to **8192** regardless of the model's real capability or `OLLAMA_CONTEXT_LENGTH` — any chat context (e.g. "full content" mode on a real document) beyond that gets truncated by Ollama, and because `qwen3.5:27b` is a thinking model, the truncated prompt frequently produces an empty final answer (the response is silently swallowed by `clean_thinking_content`) instead of a visible error — looks exactly like "chat gives no answer" from the UI. `98304` is the largest value that still keeps the whole model on GPU 1 (`ollama ps` shows `100% GPU`); `131072` spills ~91% onto CPU and makes a single response take 5-6 minutes instead of ~1-2. Even at 98304, a full-book "Volltext" chat (~70k+ tokens of context) takes ~2-3 minutes — that's inherent to processing that much context on one consumer GPU, not a bug. For large documents, the default short/RAG context mode (only relevant chunks, not the whole document) stays fast; reserve "Volltext" for shorter sources. If `qwen3.5:27b` ever gets swapped for a different Ollama chat model, re-check this credential's `num_ctx` still makes sense for its VRAM footprint.
 **Podcast language + reliability — fixed via patched prompt templates (`prompts/podcast/`,
 bind-mounted).** Two separate problems, one fix location:
 1. *Language.* The episode profile's `language` field (`"de"` → resolved to `"German"`) **is**
   passed into both `podcast_creator` prompt templates as a `{{ language }}` variable
   (`nodes.py`, both `generate_outline_node` and `generate_transcript_node`), but the stock
   templates shipped in the image never reference it — so setting `language` had no effect and
   podcasts came out in English (the stock briefings and speaker backstories are English, which
   is the only language signal the model then sees). English text read aloud by the German-locked
   Chatterbox TTS is what produced the "English with a bad German accent" symptom.
 2. *Reliability.* `qwen3.5:27b` is a thinking model. Podcast content is the **entire notebook**
   (`Notebook.get_context()` → every source's full text + all insights; for a whole book that's
   ~70k+ tokens), fed into *each* transcript-segment call. On long segments the `<think>` block
   eats the 5000-token response budget and the JSON gets truncated → `Invalid json output` →
   whole job fails. (Confirmed: `/no_think` cut a segment from 98s to 34s and left valid JSON.)
 Fix — both handled in the two templates under `prompts/podcast/`, bind-mounted read-only over
 the image's copies via `docker-compose.yml`:
 - Prepended `/no_think` as the first line of both templates (Qwen honors it anywhere; harmless
  text for non-Qwen models) — disables thinking, freeing the token budget and ~3x speedup.
 - Added a `{% if language %}CRITICAL LANGUAGE REQUIREMENT … in {{ language }}{% endif %}` block
  near the end of both templates, so the already-plumbed `language` field now actually forces the
  output language.
 The bundled briefings and speaker backstory/personality were **also** translated to German (belt
 and suspenders; reduces English drift from the source content), but the template change is what
 makes `language` authoritative. **Editing these templates requires a container restart** to take
 effect — they're bind-mounted, and the mount is the directory `prompts/podcast/` (not individual
 files, which go stale on inode-replacing edits). If you switch off `qwen3.5:27b`, the `/no_think`
 line becomes inert but harmless.
 If you create your own episode profile, just set `language` — the template handles the rest.
 **Deleting an episode leaks its working directory (upstream gap).** `DELETE /api/podcasts/episodes/{id}`
 (`api/routers/podcasts.py`) resolves `episode.audio_file` and calls `audio_path.unlink()` on it — the
 final MP3 only. The enclosing `data/podcasts/episodes/<uuid>/` directory, holding `clips/` (one MP3 per
 dialogue segment), `outline.json` and `transcript.json`, is never touched, and nothing else reaps it.
 Failed and `/retry`-replaced runs leak a directory too (the retry writes a *new* episode row with a new
 uuid dir; the old one loses its DB entry and just stays). Upstream this would be roughly
 `shutil.rmtree(audio_path.parent.parent)` in that handler. We deliberately do **not** patch it here by
 bind-mounting a modified router — that would fork app logic into a deployment repo and rot against
 image updates (the same trap the existing overlays already carry). Instead:
 `scripts/prune_podcast_data.py` (dry-run by default, `--yes` to
 apply) reconciles the episode list from the API against the directories on disk and removes orphans plus
 the `clips/` of completed episodes. Two guards matter — it aborts if any job is `running`/`pending`
 (a running job has no `audio_file` yet, so its directory is indistinguishable from an orphan) and it
 skips directories touched in the last 60 minutes. **Deletion runs inside the container** (`docker compose
 exec … rm -rf`), because the container writes as root and the host user can't remove those directories.
 ### YouTube sources — German transcripts need `CCORE_CONFIG_PATH`
 Open Notebook extracts YouTube via `content-core`, which pulls captions with `youtube-transcript-api`
 (no `yt-dlp`/`pytube` in the image). `content_core/processors/youtube.py` only ever looks for the
 languages in `preferred_languages`, and the built-in default is `["en", "es", "pt"]` — **no German**.
 All four fallback attempts in `_fetch_best_transcript` use that same list, so a video with only a
 German transcript raises `NoTranscriptFound`, `get_best_transcript` swallows it and returns `None`,
 and the source is created with an **empty body**. The title is still fetched (separate HTTP scrape),
 so the failure is quiet: a source that looks fine but has no content, with the real reason only in
 the container log (`Failed to get transcript for video <id> after retries`).
 The `youtube_transcripts.preferred_languages` key in the package's `cc_config.yaml` looks like the
 fix, but it is **dead code** on the default path: `load_config()` copies only the `extraction` block
 out of `cc_config.yaml`, so the key never reaches `CONFIG` and the hardcoded `["en","es","pt"]`
 fallback always wins. The only way to set it is `CCORE_CONFIG_PATH`.
 Fix: `config/content_core.yaml` (bind-mounted read-only, `CCORE_CONFIG_PATH=/app/config/content_core.yaml`
 in `docker-compose.yml`) sets `preferred_languages: ["de", "en", "es", "pt"]`.
 **`CCORE_CONFIG_PATH` replaces the config wholesale — it does not merge** (`config.py`:
 `return yaml.safe_load(file)`). So that file is a full dump of the effective default config plus the
 `youtube_transcripts` key, not a small override; a minimal file would silently drop `extraction`
 (engine selection) and the model/timeout defaults. Its header comment carries the one-liner that
 regenerates it from inside the container — do that after an image update if content-core's defaults
 change. The `openai`/`gpt-4o-mini` entries in it are content-core's internal defaults and are unused
 here (Open Notebook picks its own models); they're only present because the file must be complete.
 Note this only fixes *caption* languages. A video with **no** captions is handled by the audio
 fallback below.
 ### Video without captions — audio fallback (`scripts/add_video_source.py`)
 `yt-dlp` exists on the **host** only (`~/miniforge3/bin/yt-dlp`), not in the image, so the chain
 can't live inside the app. `scripts/add_video_source.py` bridges it:
 - captions available → hands the URL to Open Notebook as a `link` source (fast, no download);
 - no captions → `yt-dlp -x` pulls just the audio track (mono 16 kHz mp3, ~6 MB per 17 min) and
  uploads it as an `upload` source. Open Notebook then transcribes it **itself** via its configured
  speech-to-text model — i.e. the local faster-whisper server — and embeds it. We deliberately do
  not transcribe in the script: the app's own pipeline already does STT, chunking and embedding.
 Caption availability is probed with `youtube-transcript-api` **inside the container**, not with
 `yt-dlp`. `yt-dlp`'s `automatic_captions` lists YouTube's ~100 auto-*translation* targets (German
 is always among them), which `youtube-transcript-api` does not accept as transcripts — trusting it
 would send a Japanese-only video down the caption path and produce an empty source again.
 **`OPENAI_COMPATIBLE_BASE_URL_STT` is what makes the upload path work at all.** Open Notebook passes
 its STT model (`openai_compatible`/`faster-whisper-large-v3`) into content-core as
 `audio_provider`/`audio_model` (`open_notebook/graphs/source.py`), but content-core builds it with
 `AIFactory.create_speech_to_text(provider, model, {'timeout': …})` — **no `base_url`**
 (`content_core/processors/audio.py`). Esperanto then can't locate the local server, content-core
 silently falls back to its default (`openai`/`whisper-1`) and the job dies with *"OpenAI API key not
 found"* — which reads like a missing key but is really a missing URL. The env var supplies it. Do
 **not** "fix" this by putting an `OPENAI_API_KEY` into the container: that would send audio to OpenAI
 (paid, off-machine) while a working local Whisper sits idle on GPU 2, and it contradicts the
 two-provider rule above.
 The script uploads with `delete_source=true` and names the file after the video id, so
 `notebook_data/uploads/` doesn't accumulate a few MB per video and a second video can't collide with
 the previous `audio.mp3`. Known gap: the `upload` path stores the audio file as the source's asset,
 so the original video URL is not recorded on the source (the `link` path does record it).
 ### Text-to-speech / speech-to-text
 Open Notebook's `openai_compatible` provider type talks to any endpoint implementing OpenAI's audio API shape (`POST /audio/speech`, `POST /audio/transcriptions` — see `esperanto.providers.tts.openai_compatible` / `.stt.openai_compatible` inside the app container for the exact contract). Since neither Ollama nor OpenRouter support TTS/STT, two thin wrapper servers in `services/` expose the locally installed tools this way:
@ -48,15 +214,61 @@ Open Notebook's `openai_compatible` provider type talks to any endpoint implemen
 - `services/tts_server.py` — wraps `chatterbox-tts` (via `~/chatterbox-tts-cli/chatterbox_cli_v4.py`), run with the `chatterbox` conda env (`~/miniforge3/envs/chatterbox/bin/python`).
 - `services/stt_server.py` — wraps `faster-whisper` (`large-v3` model), run with the base `python3`.
-Both are plain background processes (`nohup ... &`), **not** systemd units — they don't survive a reboot; restart manually if needed:
+Both run as **systemd user units**. The unit files live in the repo (`services/systemd/*.service`,
 using `%h` rather than a hardcoded home) and `scripts/start_services.sh` installs them into
 `~/.config/systemd/user/`, enables `linger` and starts them — so they come up on boot without a
 login session. Re-run that script after editing a unit; it's idempotent. They log to the journal,
 not to `services/logs/` (those files are leftovers from the earlier `nohup` setup):
 ```bash
-cd ~/open-notebook/services
+systemctl --user status open-notebook-tts open-notebook-stt
-CUDA_VISIBLE_DEVICES=2 TTS_PORT=8901 nohup ~/miniforge3/envs/chatterbox/bin/python tts_server.py > logs/tts_server.log 2>&1 &
+systemctl --user restart open-notebook-tts        # after editing tts_server.py or adding voices
-CUDA_VISIBLE_DEVICES=2 STT_PORT=8902 nohup python3 stt_server.py > logs/stt_server.log 2>&1 &
+journalctl --user -u open-notebook-tts -f
 ```
-They're pinned to `CUDA_VISIBLE_DEVICES=2` deliberately — GPU 1 tends to fill up with Ollama's resident models (`OLLAMA_KEEP_ALIVE=-1` keeps them loaded), so TTS/STT get GPU 2 to themselves rather than risking an OOM race with Ollama.
+They were previously plain `nohup ... &` background processes, which silently did not survive a
 reboot — a podcast then failed at the audio stage with `Failed to generate speech: All connection
 attempts failed` (`httpx.ConnectError` against `host.docker.internal:8901`) even though outline
 and transcript had generated fine. That's the signature of a dead TTS server, not a model problem.
 Both units pin GPU 2 **by UUID** (`CUDA_VISIBLE_DEVICES=GPU-83ba6d1f-…`), not by index: GPU 1 is
 occupied by Ollama's resident models (`OLLAMA_KEEP_ALIVE=-1` keeps them loaded), so TTS/STT go to
 GPU 2 rather than racing Ollama for VRAM. The index form (`CUDA_VISIBLE_DEVICES=2`) is **not**
 reliable here — CUDA's default device order is "fastest first", not nvidia-smi's PCI order, so
 index 2 can resolve to the T600. Re-check the UUID with
 `nvidia-smi --query-gpu=index,name,uuid --format=csv` if the GPUs are ever reseated.
 #### GPU 2 is shared — TTS concurrency is capped on purpose
 GPU 2 does **not** belong to the TTS server alone. Three processes sit on it: `open-notebook-tts`
 (~4 GB idle-loaded), `open-notebook-stt` (faster-whisper `large-v3`, ~4 GB), and the unrelated
 `chatterbox-tts.service` MCP server on port 9999 (~3.4 GB). That leaves roughly 12 GB of working
 headroom, and TTS inference must fit inside it.
 `podcast_creator` requests `TTS_BATCH_SIZE` clips **concurrently** (`asyncio.gather` per batch,
 `nodes.py`; upstream default **5**). `tts_server.py`'s `/audio/speech` is a *sync* FastAPI handler,
 so FastAPI runs each request in a threadpool thread — five clips therefore generate genuinely in
 parallel on one shared model, and the activation memory multiplies. That drove the TTS process to
 **16.7 GB** and produced `torch.OutOfMemoryError` inside chatterbox's `s3gen.embed_ref`, surfacing
 in the app as `Failed to generate speech: OpenAI-compatible TTS endpoint error: HTTP 500`. Note
 this is a *different* failure from the dead-server `ConnectError` above — a 500 means the server is
 alive and rejecting the work, so check the TTS journal for the CUDA OOM before assuming a restart
 will help.
 Two changes keep it bounded, and both matter:
 - **`tts_server.py` serializes generation behind `_GPU_LOCK`** (a `threading.Lock` around model load
  + the chunk loop, plus `torch.cuda.empty_cache()` afterwards). Generation is GPU-bound, so running
  clips in parallel buys no throughput on a single card — it only multiplies peak VRAM. This is the
  hard guarantee: it holds no matter which client calls, and no matter what batch size they use.
 - **`TTS_BATCH_SIZE=1` in `docker-compose.yml`** so the client sends clips one at a time. Without
  this, the lock still prevents the OOM, but 4 of the 5 batched requests sit queued in it — and with
  ~40-60 s per clip the last one approaches esperanto's **300 s** TTS timeout. `ESPERANTO_TTS_TIMEOUT=600`
  is set alongside it as extra headroom for long segments.
 Measured after the fix: 5 concurrent `/audio/speech` requests all return 200, GPU 2 peaks at
 ~11.5 GB total (vs. 16.7 GB for the TTS process alone before), i.e. ~12 GB of headroom left.
 If you add another GPU-resident service to GPU 2, re-check that budget.
 Reachability from the `open_notebook` container requires **two things**, both already done: the `extra_hosts: host.docker.internal:host-gateway` entry in `docker-compose.yml`, and UFW rules allowing inbound `8901/tcp` and `8902/tcp` (same pattern as the existing `11434/tcp` rule for Ollama — UFW defaults to deny-incoming, so any new host-side port a container needs to reach must be explicitly opened).
@ -74,7 +286,7 @@ Three reference clips exist in `services/voices/` (gitignored, not committed):
  piper --model de_DE-thorsten-medium.onnx --output_file male_thorsten.wav <<< "some German sentence"
  ```
-Podcast speaker profiles are assigned across these three: `business_panel` (Marcus→male_thorsten, Elena→female_kerstin, Johny→default), `tech_experts` (Alex→male_thorsten, Jamie→female_kerstin), `solo_expert`/`test_speaker_local` (→female_kerstin/Anna). To add another distinct voice, drop a new `<name>.wav` (~10-30s, clean single-speaker audio, licensing-checked) into `services/voices/`, restart `tts_server.py` (voices are scanned once at startup), and set the speaker's `voice_id` to `<name>` via `PUT /api/speaker-profiles/{id}`.
+Podcast speaker profiles are assigned across these three: `business_panel` (Marcus→male_thorsten, Elena→female_kerstin, Johny→default), `tech_experts` (Alex→male_thorsten, Jamie→female_kerstin), `solo_expert`/`test_speaker_local` (→female_kerstin/Anna). To add another distinct voice, drop a new `<name>.wav` (~10-30s, clean single-speaker audio, licensing-checked) into `services/voices/`, restart the TTS server (`systemctl --user restart open-notebook-tts` — voices are scanned once at startup), and set the speaker's `voice_id` to `<name>` via `PUT /api/speaker-profiles/{id}`.
 ## Secrets
--- a/README.md
+++ b/README.md
@ -24,8 +24,8 @@ Weiterentwicklung: siehe [`CLAUDE.md`](CLAUDE.md).
 | `surrealdb` | Docker | Datenbank (nur `127.0.0.1:8000`) |
 | `open_notebook` | Docker | Web-UI (`:8502`) + REST-API (`:5055`) |
 | Ollama | Host (systemd) | Chat-/Embedding-Modelle, GPU 1+2 |
-| `services/tts_server.py` | Host (Hintergrundprozess) | Chatterbox TTS, GPU 2, Port `8901` |
+| `services/tts_server.py` | Host (systemd-User-Dienst) | Chatterbox TTS, GPU 2, Port `8901` |
-| `services/stt_server.py` | Host (Hintergrundprozess) | faster-whisper STT, GPU 2, Port `8902` |
+| `services/stt_server.py` | Host (systemd-User-Dienst) | faster-whisper STT, GPU 2, Port `8902` |
 Der `open_notebook`-Container erreicht Ollama und die TTS/STT-Wrapper über
 `host.docker.internal` (Linux-Route via `extra_hosts: host-gateway`).
@ -49,6 +49,8 @@ Der `open_notebook`-Container erreicht Ollama und die TTS/STT-Wrapper über
  in Schritt 2)
 - **`ufw`** (falls als Firewall aktiv) — Standardrichtlinie „deny incoming“ wird vorausgesetzt
 - **`openssl`**, **`ffmpeg`**, **`curl`**, **`python3`**
 - **`yt-dlp`** — nur für `scripts/add_video_source.py` (Videos ohne Untertitel); der übrige Stack
  läuft auch ohne
 ---
@ -80,11 +82,14 @@ chmod 600 .env
 ### 2.3 Container starten
 ```bash
 docker compose pull
 docker compose up -d
 docker compose ps        # beide Container sollten "Up" sein
 ```
 Die Images sind in `docker-compose.yml` auf einen **Digest gepinnt** (`pull_policy: missing`) —
 `up -d` lädt sie beim ersten Mal und danach nie wieder unbemerkt eine andere Version. Aktualisiert
 wird bewusst, siehe [Abschnitt 4](#4-updates).
 Web-UI: `http://localhost:8502` · REST-API: `http://localhost:5055` (beide nur `127.0.0.1`).
 ### 2.4 Firewall öffnen für TTS/STT
@ -101,19 +106,29 @@ sudo ufw allow 8902/tcp comment 'open-notebook STT (faster-whisper)'
 Ohne diesen Schritt bleiben TTS/STT vom Container aus unerreichbar (Timeout), obwohl sie lokal
 laufen und funktionieren.
-### 2.5 TTS/STT-Server starten
+### 2.5 TTS/STT-Server einrichten
 ```bash
 ./scripts/start_services.sh
 ```
-Startet `tts_server.py` und `stt_server.py` als Hintergrundprozesse auf GPU 2 (nicht GPU 0/1,
+Installiert `services/systemd/*.service` nach `~/.config/systemd/user/`, aktiviert `linger`
-um Ollama und die Desktop-GPU nicht zu belasten). Nicht systemd-verwaltet — nach einem Reboot
+(damit die Dienste auch ohne aktive Login-Session laufen) und startet beide. Sie laufen auf
-erneut ausführen. Logs: `services/logs/`.
+GPU 2 — nicht GPU 0/1, um die Desktop-GPU und Ollamas Chat-Modell nicht zu verdrängen.
 Das Skript ist idempotent und **überlebt Reboots**: die Dienste starten automatisch mit. Status
 und Logs:
 ```bash
 systemctl --user status open-notebook-tts open-notebook-stt
 journalctl --user -u open-notebook-tts -f
 ```
 > Der erste Start dauert bis zu einer Minute, weil whisper `large-v3` geladen wird.
 Für Stimmklonung mindestens eine Referenz-WAV in `services/voices/default.wav` ablegen (z. B.
 Symlink auf eine eigene Sprachaufnahme, 10–30s, WAV) — siehe
-[`BEDIENUNGSANLEITUNG.md`](BEDIENUNGSANLEITUNG.md#stimmklonung).
+[`BEDIENUNGSANLEITUNG.md`](BEDIENUNGSANLEITUNG.md#5-stimmklonung).
 ### 2.6 KI-Provider und Modelle registrieren
@ -151,6 +166,12 @@ curl -s http://127.0.0.1:8901/health   # Chatterbox TTS
 curl -s http://127.0.0.1:8902/health   # faster-whisper STT
 ```
 Gründlicher — prüft API, TTS, STT, YouTube-Extraktion und alle lokalen Anpassungen auf einmal:
 ```bash
 ./scripts/smoke_test.sh
 ```
 UI öffnen (`http://localhost:8502`), ein Notebook anlegen, eine Quelle hinzufügen und chatten —
 siehe [`BEDIENUNGSANLEITUNG.md`](BEDIENUNGSANLEITUNG.md) für die weitere Bedienung.
@ -163,12 +184,57 @@ docker compose up -d              # starten
 docker compose down                # stoppen (Daten bleiben erhalten)
 docker compose logs -f open_notebook   # Logs verfolgen
 docker compose ps                  # Status
-./scripts/start_services.sh        # TTS/STT (re-)starten, z.B. nach Reboot
+
 systemctl --user status open-notebook-tts open-notebook-stt   # TTS/STT-Status
 systemctl --user restart open-notebook-tts                     # z.B. nach neuer Stimme
 journalctl --user -u open-notebook-tts -f                      # TTS-Logs
 ./scripts/start_services.sh        # Units neu installieren + starten (idempotent)
 ./scripts/prune_podcast_data.py         # Podcast-Datenmüll anzeigen (Trockenlauf)
 ./scripts/prune_podcast_data.py --yes   # ... und löschen
 ./scripts/add_video_source.py <url>     # Video als Quelle (auch ohne Untertitel)
 ./scripts/check_updates.sh              # gibt es ein neues Release? (ändert nichts)
 ./scripts/update_stack.sh               # Update mit Backup, Rauchtest und Rollback
 ./scripts/smoke_test.sh                 # tut der Stack noch, worauf wir uns verlassen?
 ```
 TTS/STT starten nach einem Reboot von selbst — `start_services.sh` ist nur für die
 Ersteinrichtung bzw. nach Änderungen an den Unit-Dateien nötig.
 ---
-## 4. Verzeichnisstruktur
+## 4. Updates
 Die Images sind **auf einen Digest gepinnt** — es läuft immer genau der Stand, der im Repo steht.
 Weder ein `docker compose up -d` noch ein Reboot tauscht die Anwendung unbemerkt aus.
 Beim Sitzungsstart (oder wann immer Sie mögen):
 ```bash
 ./scripts/check_updates.sh     # zeigt nur: läuft X, verfügbar ist Y
 ```
 Wenn ein neues Release da ist:
 ```bash
 ./scripts/update_stack.sh      # Backup -> Update -> Rauchtest -> bei Fehler Rollback
 ```
 Warum nicht automatisch das Neueste bei jedem Start? Ein Update **migriert die Datenbank**, und
 eine ältere Anwendung kann eine migrierte Datenbank in der Regel nicht mehr lesen — ein
 misslungenes Update ohne Backup wäre eine Einbahnstraße. Zudem hängen die lokalen Anpassungen
 dieses Projekts (Podcast-Vorlagen, content-core-Config, Env-Variablen) an Interna der Anwendung
 und können still brechen. Genau das prüft der Rauchtest; schlägt er fehl, rollt das Skript
 Daten und Compose-Datei automatisch zurück.
 Hinweis: `v1-latest` folgt den **Releases**, nicht dem `main`-Branch — der Repository-Stand ist
 typischerweise Wochen voraus, aber unveröffentlicht. Wer den will, müsste selbst bauen.
 ---
 ## 5. Verzeichnisstruktur
 ```
 open-notebook/
@ -177,14 +243,23 @@ open-notebook/
 ├── .env.example             # Template
 ├── surreal_data/            # DB-Daten (gitignored)
 ├── notebook_data/           # Notebook-Dateien/Uploads (gitignored)
 ├── config/
 │   └── content_core.yaml    # content-core (u.a. deutsche YouTube-Transkripte)
 ├── services/
 │   ├── tts_server.py        # Chatterbox-Wrapper (OpenAI-API-kompatibel)
 │   ├── stt_server.py        # faster-whisper-Wrapper
 │   ├── systemd/              # Unit-Dateien für die beiden Wrapper
 │   ├── voices/               # Referenz-WAVs fürs Voice-Cloning (gitignored)
-│   └── logs/                 # (gitignored)
+│   └── logs/                 # (gitignored, Altlast der früheren nohup-Variante)
 ├── backups/                  # Sicherungen vor Image-Updates (gitignored)
 ├── scripts/
 │   ├── setup_models.sh       # Provider/Modelle einrichten (idempotent)
-│   └── start_services.sh     # TTS/STT starten
+│   ├── start_services.sh     # TTS/STT als systemd-User-Dienste installieren+starten
 │   ├── prune_podcast_data.py # verwaiste Podcast-Ordner + Zwischenclips aufräumen
 │   ├── add_video_source.py   # Video als Quelle (Untertitel oder Tonspur+Whisper)
 │   ├── check_updates.sh      # neues Release verfügbar? (ändert nichts)
 │   ├── update_stack.sh       # Update mit Backup/Rauchtest/Rollback
 │   └── smoke_test.sh         # prüft die Annahmen, auf denen die Anpassungen beruhen
 ├── README.md                 # diese Datei
 ├── BEDIENUNGSANLEITUNG.md     # ausführliche Bedienungsanleitung (Deutsch)
 └── CLAUDE.md                  # technische Architektur-Doku für Weiterentwicklung
@ -192,7 +267,7 @@ open-notebook/
 ---
-## 5. Bekannte Stolpersteine
+## 6. Bekannte Stolpersteine
 Kurzreferenz — Details jeweils in [`CLAUDE.md`](CLAUDE.md):
@ -204,3 +279,33 @@ Kurzreferenz — Details jeweils in [`CLAUDE.md`](CLAUDE.md):
  leere Chat-Antworten bei größerem Kontext, ohne sichtbaren Fehler.
 - **Neue Host-Ports brauchen eine `ufw`-Regel**, sonst kann der Container sie nicht erreichen
  (Timeout, keine Fehlermeldung in Open Notebook selbst).
 - **YouTube-Transkripte brauchen `config/content_core.yaml`.** `content-core` sucht Untertitel
  standardmäßig nur in `en`/`es`/`pt` — deutsche Videos landen sonst als **leere Quelle** im
  Notebook (Titel da, Inhalt leer, Fehler nur im Log). Die eingehängte Config setzt Deutsch an
  erste Stelle. Achtung: `CCORE_CONFIG_PATH` ersetzt die Konfiguration komplett (kein Merge) —
  die Datei muss vollständig bleiben, siehe Kopfkommentar darin.
 - **Videos ohne Untertitel** gehen über `./scripts/add_video_source.py <url>` (yt-dlp lädt die
  Tonspur, der lokale Whisper-Server transkribiert sie). Dafür ist
  `OPENAI_COMPATIBLE_BASE_URL_STT` in `docker-compose.yml` zwingend: content-core reicht dem
  STT-Modell sonst keine `base_url` durch, fällt auf OpenAI zurück und scheitert mit
  „OpenAI API key not found" — das ist **kein** fehlender Key, sondern eine fehlende URL. Keinen
  `OPENAI_API_KEY` in den Container legen: das schickte Audio in die Cloud, obwohl Whisper lokal
  auf GPU 2 bereitsteht.
 - **Episoden löschen räumt nicht auf.** Der Mülleimer-Button (bzw.
  `DELETE /api/podcasts/episodes/{id}`) entfernt nur die finale MP3 und den DB-Eintrag — der
  Ordner `notebook_data/podcasts/episodes/<uuid>/` mit den Einzelclips, `outline.json` und
  `transcript.json` bleibt liegen. Das ist eine Lücke im Upstream-Code, kein lokales
  Konfigurationsproblem. Abhilfe: `./scripts/prune_podcast_data.py`.
 - **GPU 2 teilen sich drei Prozesse** (TTS, STT und der separate `chatterbox-tts`-MCP-Dienst auf
  Port 9999) — es bleiben nur ~12 GB Arbeitsspeicher für die Sprachsynthese. Deshalb serialisiert
  `tts_server.py` seine Generierung intern und `TTS_BATCH_SIZE=1` steht in `docker-compose.yml`:
  ohne beides schickt der Podcast-Generator 5 Clips gleichzeitig, die parallel auf der GPU laufen
  und sie mit `CUDA out of memory` sprengen (sichtbar als `HTTP 500` beim Vertonen).
 - **Podcast-Sprache** wird über das `language`-Feld des Episode-Profils gesteuert, aber nur
  dank der gepatchten Prompt-Vorlagen unter `prompts/podcast/` (per Bind-Mount in
  `docker-compose.yml` eingehängt) — die Original-Vorlagen im Image ignorieren `language`, was
  englische Podcasts (von der deutsch-fixierten TTS mit Akzent vorgelesen) verursachte. Die
  Vorlagen enthalten zusätzlich `/no_think`, damit `qwen3.5:27b` bei langem Quellinhalt nicht
  durch überlange Reasoning-Blöcke ungültiges JSON liefert. Änderst du die Vorlagen, danach
  `docker compose up -d --force-recreate open_notebook`. Details:
  [`BEDIENUNGSANLEITUNG.md`](BEDIENUNGSANLEITUNG.md#11-troubleshooting) und [`CLAUDE.md`](CLAUDE.md).
--- a/config/content_core.yaml
+++ b/config/content_core.yaml
@ -0,0 +1,78 @@
 # content-core Konfiguration fuer dieses Deployment.
 #
 # Zweck: YouTube-Quellen mit deutschem Transkript nutzbar machen.
 #
 # content-core sucht Transkripte nur in "preferred_languages". Der eingebaute
 # Standard ist ["en", "es", "pt"] — ohne Deutsch. Ein Video mit ausschliesslich
 # deutschem Transkript liefert deshalb NoTranscriptFound, und Open Notebook legt
 # die Quelle mit LEEREM Inhalt an (der Titel wird trotzdem geholt, der Fehler
 # taucht nur im Log auf).
 #
 # Der Schluessel youtube_transcripts steht zwar in der cc_config.yaml des Pakets,
 # wird von load_config() im Standardfall aber gar nicht eingelesen (dort wird nur
 # der Block "extraction" uebernommen) — er ist also wirkungslos. Einziger Weg,
 # ihn zu setzen: CCORE_CONFIG_PATH auf eine eigene Datei zeigen lassen.
 #
 # ACHTUNG: CCORE_CONFIG_PATH ERSETZT die Konfiguration vollstaendig, es wird
 # nicht zusammengefuehrt (content_core/config.py: return yaml.safe_load(file)).
 # Diese Datei ist deshalb ein vollstaendiger Abzug der effektiven Standardconfig
 # plus youtube_transcripts. Bei einem Image-Update mit geaenderten Defaults neu
 # erzeugen:
 #
 #   docker compose exec -T open_notebook /app/.venv/bin/python -c "
 #   from content_core.config import CONFIG; import yaml, copy
 #   cfg = copy.deepcopy(CONFIG)
 #   cfg['youtube_transcripts'] = {'preferred_languages': ['de','en','es','pt']}
 #   print(yaml.safe_dump(cfg, sort_keys=False, allow_unicode=True))" > config/content_core.yaml
 #
 # Die openai/gpt-4o-mini-Eintraege unten sind content-core-interne Defaults; sie
 # werden hier nicht benutzt (Open Notebook waehlt seine Modelle selbst) und
 # stehen nur drin, weil die Datei vollstaendig sein muss.
 speech_to_text:
  provider: openai
  model_name: whisper-1
  timeout: 3600
 default_model:
  provider: openai
  model_name: gpt-4o-mini
  config:
    temperature: 0.5
    top_p: 1
    max_tokens: 2000
    timeout: 300
 cleanup_model:
  provider: openai
  model_name: gpt-4o-mini
  config:
    temperature: 0
    max_tokens: 8000
    output_format: json
    timeout: 600
 summary_model:
  provider: openai
  model_name: gpt-4o-mini
  config:
    temperature: 0
    top_p: 1
    max_tokens: 2000
    timeout: 300
 extraction:
  document_engine: auto
  url_engine: auto
  audio:
    concurrency: 3
  firecrawl:
    api_url: null
  docling:
    output_format: markdown
  pymupdf:
    enable_formula_ocr: false
    formula_threshold: 3
    ocr_fallback: true
 youtube_transcripts:
  preferred_languages:
  - de
  - en
  - es
  - pt
--- a/docker-compose.yml
+++ b/docker-compose.yml
@ -1,8 +1,21 @@
 # Images sind bewusst auf einen Digest gepinnt, nicht auf einen beweglichen Tag.
 #
 # Warum: "v1-latest" ist ein wandernder Zeiger. Mit pull_policy: always zog jedes
 # `docker compose up -d` unbemerkt eine neue Anwendung — inklusive Schema-Migration der
 # SurrealDB, die sich nicht ohne Weiteres zurueckdrehen laesst. Gleichzeitig griff das
 # nur bei `up -d`: nach einem Reboot startete `restart: always` weiter das ALTE Image.
 # Also beides unzuverlaessig: mal heimlich neu, mal heimlich alt.
 #
 # Mit Digest-Pin laeuft immer exakt der Stand, der hier im Repo steht. Aktualisiert wird
 # bewusst und mit Rueckfallpunkt:
 #   ./scripts/check_updates.sh   -> gibt es ein neues Release? (aendert nichts)
 #   ./scripts/update_stack.sh    -> Backup, Update, Rauchtest, bei Fehler Rollback
 services:
  surrealdb:
-    image: surrealdb/surrealdb:v2
+    # surrealdb:v2 (Digest-Pin: auch ein Minor-Sprung kann das Storage-Format anfassen)
    image: surrealdb/surrealdb:v2@sha256:d653f6c8a89e81f865ee31cd2f587c50f50ace922175e04150b1e385d2f86011
    restart: always
-    pull_policy: always
+    pull_policy: missing
    command: start --log info --user ${SURREAL_USER:-root} --pass ${SURREAL_PASSWORD:-root} rocksdb:/mydata/mydatabase.db
    user: root  # Required for bind mounts on Linux
    environment:
@ -13,9 +26,11 @@ services:
      - ./surreal_data:/mydata
  open_notebook:
-    image: lfnovo/open_notebook:v1-latest
+    # Open Notebook v1.10.0 (Release vom 2026-06-18, Image vom selben Tag).
    # Digest von scripts/update_stack.sh gepflegt — nicht von Hand aendern.
    image: lfnovo/open_notebook:v1-latest@sha256:c8112fbd4b8fee7f2a20d3bdbea24e7d72267acfc990b803fd0ff4de30899b57
    restart: always
-    pull_policy: always
+    pull_policy: missing
    depends_on:
      - surrealdb
    ports:
@ -30,7 +45,35 @@ services:
      - SURREAL_DATABASE=open_notebook
      - OLLAMA_BASE_URL=http://host.docker.internal:11434
      - OPENROUTER_API_KEY=${OPENROUTER_API_KEY}
      # podcast_creator schickt sonst 5 Audio-Clips gleichzeitig an den TTS-Server.
      # Die laufen dort echt parallel auf einer GPU -> Aktivierungsspeicher
      # vervielfacht sich -> CUDA OOM (GPU 2 teilt sich TTS, STT und den
      # chatterbox-MCP-Dienst). Der TTS-Server serialisiert intern zwar ohnehin,
      # aber bei Batch 5 warten 4 Requests im Lock und laufen ins 300s-Timeout.
      - TTS_BATCH_SIZE=1
      # Kopfraum fuer lange Segmente; Default waere 300s pro Clip.
      - ESPERANTO_TTS_TIMEOUT=600
      # content-core sucht YouTube-Transkripte sonst nur in en/es/pt — deutsche
      # Videos landen dann als LEERE Quelle im Notebook. Siehe config/content_core.yaml.
      - CCORE_CONFIG_PATH=/app/config/content_core.yaml
      # Audio-Quellen (Upload, Video-Tonspur): Open Notebook reicht sein STT-Modell
      # (openai_compatible/faster-whisper-large-v3) an content-core weiter, aber
      # content-core baut das Modell OHNE base_url (es gibt nur timeout mit). Esperanto
      # weiss dann nicht, wo der lokale Whisper-Server steht, faellt auf OpenAI zurueck
      # und scheitert mit "OpenAI API key not found". Diese Variable liefert die URL nach.
      - OPENAI_COMPATIBLE_BASE_URL_STT=http://host.docker.internal:8902
    extra_hosts:
      - "host.docker.internal:host-gateway"
    volumes:
      - ./notebook_data:/app/data
      # Podcast-Prompt-Vorlagen mit expliziter Sprachanweisung ({{ language }})
      # und abgeschaltetem Thinking-Modus (/no_think), damit Podcasts in der
      # eingestellten Sprache erzeugt werden (statt Englisch) und das JSON bei
      # langen Segmenten nicht durch riesige Reasoning-Blöcke abgeschnitten wird.
      # Verzeichnis-Mount (nicht Einzeldateien), damit Edits ohne Inode-Probleme
      # nach einem Container-Neustart greifen. Der Ordner im Image enthält nur
      # genau diese zwei Vorlagen.
      - ./prompts/podcast:/app/prompts/podcast:ro
      # content-core-Konfiguration (u.a. deutsche YouTube-Transkripte), per
      # CCORE_CONFIG_PATH oben referenziert.
      - ./config:/app/config:ro
--- a/prompts/podcast/outline.jinja
+++ b/prompts/podcast/outline.jinja
@ -0,0 +1,91 @@
 /no_think
 You are an AI assistant specialized in creating podcast outlines. Your task is to create a detailed outline for a podcast episode based on a provided briefing. The outline you create will be used to generate the podcast transcript.
 Here is the briefing for the podcast episode:
 <briefing>
 {{ briefing }}
 </briefing>
 The user has provided content to be used as the context for this podcast episode:
 <context>
 {% if context is string %}
 {{ context }}
 {% else %}
 {% for item in context %}
 <content_piece>
 {{ item }}
 </content_piece>
 {% endfor %}
 {% endif %}
 </context>
 The podcast will feature the following speakers:
 <speakers>
 {% for speaker in speakers %}
 - **{{ speaker.name }}**: {{ speaker.backstory }}
  Personality: {{ speaker.personality }}
 {% endfor %}
 </speakers>
 Please create an outline based on this briefing. Your outline should consist of {{ num_segments }} main segments for the podcast episode, along with a description of each segment. Follow these guidelines:
 1. Read the briefing carefully and identify the main topics and themes.
 2. Create {{ num_segments }} distinct segments that cover the entire scope of the briefing.
 3. For each segment, provide a clear and concise name that reflects its content.
 4. Write a detailed description for each segment, explaining what will be discussed and provide suggestions of topics according to the context given. The writer will use your suggestion to design the dialogs.
 5. Consider the speaker personalities and backstories when planning segments - match content to speaker expertise.
 6. Ensure that the segments flow logically from one to the next.
 7. This is a whole podcast so no need to reintroduce speakers or topics on each segment. Segments are just markers for us to know to change the topics, nothing else.
 8. Include an introduction segment at the beginning and a conclusion or wrap-up segment at the end.
 Format your outline using the following structure:
 ```json
 {
    "segments": [
        {
            "name": "[Segment Name]",
            "description": "[Description of the segment content]",
            "size": "short"
        },
        {
            "name": "[Segment Name]",
            "description": "[Description of the segment content]",
            "size": "medium"
        },
        {
            "name": "[Segment Name]",
            "description": "[Description of the segment content]",
            "size": "long"
        },
    ...
    ]
 }
 ```
 Formatting instructions:
 {{ format_instructions}}
 Additional tips:
 - Make sure the segment names are catchy and informative.
 - In the descriptions, include key points or questions that will be addressed in each segment.
 - Consider the target audience mentioned in the briefing when crafting your outline.
 - If the briefing mentions a guest, include segments for introducing the guest and featuring their expertise.
 - The size of the segment should be short, medium or long. Think about the content of the segment and how important it is to the episode.
 {% if language %}
 CRITICAL LANGUAGE REQUIREMENT:
 - Write the ENTIRE outline — every segment name and every segment description — exclusively in {{ language }}.
 - Do NOT use English or any other language, not even for technical terms or proper concepts where a {{ language }} wording exists.
 - These instructions are written in English, but your output MUST be in {{ language }}.
 {% endif %}
 IMPORTANT OUTPUT FORMAT:
 - If you use extended thinking with <think> tags, put ALL your reasoning inside <think></think> tags
 - Put the final JSON output OUTSIDE and AFTER any <think> tags
 - Do NOT wrap the JSON in ```json code blocks - return the raw JSON object only
 - Example correct format:
  <think>Let me analyze the briefing...</think>
  {"segments": [...]}
 Please provide your outline now, following the format and guidelines provided above.
--- a/prompts/podcast/transcript.jinja
+++ b/prompts/podcast/transcript.jinja
@ -0,0 +1,142 @@
 /no_think
 You are an AI assistant specialized in creating podcast transcripts.
 Your task is to generate a transcript for a specific segment of a podcast episode based on a provided briefing and outline.
 The transcript will be used to generate podcast audio. Follow these instructions carefully:
 First, review the briefing for the podcast episode:
 <briefing>
 {{ briefing }}
 </briefing>
 The user has provided content to be used as the context for this podcast episode:
 <context>
 {% if context is string %}
 {{ context }}
 {% else %}
 {% for item in context %}
 <content_piece>
 {{ item }}
 </content_piece>
 {% endfor %}
 {% endif %}
 </context>
 The podcast features the following speakers:
 <speakers>
 {% for speaker in speakers %}
 - **{{ speaker.name }}**: {{ speaker.backstory }}
  Personality: {{ speaker.personality }}
 {% endfor %}
 </speakers>
 Next, examine the outline produced by our director:
 <outline>
 {{ outline }}
 </outline>
 {% if transcript %}
 Here is the current transcript so far:
 <transcript>
 {{ transcript }}
 </transcript>
 {% endif %}
 {% if is_final %}
 {% if speakers|length == 1 %}
 This is the final segment of the podcast. Make sure to wrap up the presentation and provide a conclusion.
 {% else %}
 This is the final segment of the podcast. Make sure to wrap up the conversation and provide a conclusion.
 {% endif %}
 {% endif %}
 You will focus on creating the dialogue for the following segment ONLY:
 <segment>
 {{ segment }}
 </segment>
 {% if speakers|length == 1 %}
 IMPORTANT: This is a SOLO podcast with only ONE speaker ({{ speaker_names[0] }}). Do NOT invent or add any other speakers.
 All dialogue entries must use "{{ speaker_names[0] }}" as the speaker name.
 Follow these format requirements strictly:
   - Use ONLY the speaker name "{{ speaker_names[0] }}" for all dialogue entries.
   - Do NOT create or invent any additional speakers.
   - Stick to the segment, do not go further than what's requested. Other agents will do the rest of the podcast.
   - The transcript must have at least {{ turns }} dialogue segments from the speaker.
   - The speaker should present the content in an engaging, educational manner.
 {% else %}
 Follow these format requirements strictly:
   - Use the actual speaker names ({{ speaker_names|join(', ') }}) to denote speakers.
   - Choose which speaker should speak based on their personality, backstory, and the content being discussed.
   - Stick to the segment, do not go further than what's requested. Other agents will do the rest of the podcast.
   - The transcript must have at least {{ turns }} turns of messages between the speakers.
   - Each speaker should contribute meaningfully based on their expertise and personality.
 {% endif %}
 ```json
 {
    "transcript": [
        {
            "speaker": "[Actual Speaker Name]",
            "dialogue": "[Speaker's dialogue based on their personality and expertise]"
        },
    ...
    ]
 }
 ```
 Formatting instructions:
 {{ format_instructions}}
 {% if speakers|length == 1 %}
 Guidelines for creating the transcript:
   - Ensure the presentation flows naturally and covers all points in the outline.
   - Ensure you return the root "transcript" key in your response.
   - Make the content sound engaging and educational.
   - Include relevant details from the briefing.
   - Break up the content into digestible segments with natural transitions.
   - Use appropriate transitions between topics.
   - Match the speaker's dialogue to their personality and expertise.
   - This is a whole podcast so no need to reintroduce the speaker or topics on each segment. Segments are just markers for us to know to change the topics, nothing else.
   - CRITICAL: There is only ONE speaker. Use ONLY: {{ speaker_names[0] }}. Do NOT invent additional speakers.
 {% else %}
 Guidelines for creating the transcript:
   - Ensure the conversation flows naturally and covers all points in the outline.
   - Ensure you return the root "transcript" key in your response.
   - Make the dialogue sound conversational and engaging.
   - Include relevant details from the briefing.
   - Avoid long monologues; keep exchanges between speakers balanced.
   - Use appropriate transitions between topics.
   - Match each speaker's dialogue to their personality and expertise.
   - Choose speakers strategically based on who would naturally contribute to each topic.
   - This is a whole podcast so no need to reintroduce speakers or topics on each segment. Segments are just markers for us to know to change the topics, nothing else.
   - IMPORTANT: Only use the provided speaker names: {{ speaker_names|join(', ') }}
 {% endif %}
 {% if language %}
 CRITICAL LANGUAGE REQUIREMENT:
 - Write ALL dialogue lines exclusively in {{ language }}.
 - Do NOT use English or any other language, not even for technical terms or proper concepts where a {{ language }} wording exists.
 - These instructions are written in English, but every spoken line you produce MUST be in {{ language }}.
 {% endif %}
 IMPORTANT OUTPUT FORMAT:
 - If you use extended thinking with <think> tags, put ALL your reasoning inside <think></think> tags
 - Put the final JSON output OUTSIDE and AFTER any <think> tags
 - Do NOT wrap the JSON in ```json code blocks - return the raw JSON object only
 - Example correct format:
  <think>Let me plan the dialogue...</think>
  {"transcript": [...]}
 When you're ready, provide the transcript.
 {% if speakers|length == 1 %}
 Remember, you are creating a realistic solo podcast presentation based on the given information.
 Make it informative, engaging, and natural-sounding while adhering to the format requirements.
 There is only ONE speaker - do not add any other speakers.
 {% else %}
 Remember, you are creating a realistic podcast conversation based on the given information.
 Make it informative, engaging, and natural-sounding while adhering to the format requirements.
 {% endif %}
--- a/scripts/add_video_source.py
+++ b/scripts/add_video_source.py
@ -0,0 +1,260 @@
 #!/usr/bin/env python3
 """Fuegt ein Video (YouTube & alles, was yt-dlp kennt) als Quelle in ein Notebook ein —
 auch ohne Untertitel.
 Zwei Wege, automatisch gewaehlt:
  Untertitel vorhanden  -> die URL wird direkt als Link-Quelle uebergeben. Open Notebook
                           holt das Transkript selbst (schnell, kein Download).
  keine Untertitel      -> yt-dlp laedt nur die Tonspur, die wird als Audio-Quelle
                           hochgeladen. Open Notebook transkribiert sie mit dem
                           eingestellten Speech-to-Text-Modell (hier: der lokale
                           faster-whisper-Server auf :8902) und bettet sie ein.
 Der zweite Weg ist noetig, weil Open Notebook fuer Videos nur die vorhandenen Untertitel
 liest — gibt es keine, bleibt die Quelle leer. yt-dlp laeuft nur auf dem Host (nicht im
 Container), deshalb diese Bruecke.
 Beispiele:
    ./scripts/add_video_source.py https://www.youtube.com/watch?v=...
    ./scripts/add_video_source.py <url> --notebook notebook:abc --language de
    ./scripts/add_video_source.py <url> --force-audio     # Untertitel ignorieren
 """
 from __future__ import annotations
 import argparse
 import json
 import re
 import subprocess
 import sys
 import tempfile
 import time
 import urllib.error
 import urllib.parse
 import urllib.request
 from pathlib import Path
 API = "http://127.0.0.1:5055"
 REPO = Path(__file__).resolve().parent.parent
 # Sprachen, deren Untertitel wir akzeptieren. Deckungsgleich mit
 # config/content_core.yaml (youtube_transcripts.preferred_languages) — sonst wuerde
 # dieses Skript den Link-Weg waehlen, obwohl Open Notebook die Sprache gar nicht sucht.
 CAPTION_LANGS = ["de", "en", "es", "pt"]
 # Whisper braucht nicht mehr: mono, 16 kHz. Haelt den Upload klein (~15 MB/Stunde).
 AUDIO_POSTPROC = "-ac 1 -ar 16000 -b:a 32k"
 def die(msg: str) -> int:
    print(f"FEHLER: {msg}", file=sys.stderr)
    return 1
 def run(cmd: list[str], **kw) -> subprocess.CompletedProcess:
    return subprocess.run(cmd, check=True, capture_output=True, text=True, **kw)
 def api_get(path: str) -> dict | list:
    with urllib.request.urlopen(f"{API}{path}", timeout=30) as r:
        return json.load(r)
 def probe(url: str) -> dict:
    """Metadaten holen, ohne etwas herunterzuladen."""
    try:
        out = run(["yt-dlp", "--dump-single-json", "--no-warnings", "--skip-download", url])
    except subprocess.CalledProcessError as e:
        raise SystemExit(die(f"yt-dlp kann die URL nicht lesen:\n{e.stderr.strip()[:500]}"))
    return json.loads(out.stdout)
 YOUTUBE_ID = re.compile(
    r"(?:youtu\.be/|youtube\.com/(?:embed/|v/|watch\?v=|watch\?.+&v=))([\w-]{11})"
 )
 def caption_langs(url: str) -> list[str]:
    """Untertitelsprachen, die Open Notebook tatsaechlich verwenden kann.
    Bewusst NICHT ueber yt-dlp: dessen "automatic_captions" listet auch YouTubes
    Auto-Uebersetzungen (>100 Sprachen, u.a. immer "de"), die von
    youtube-transcript-api gar nicht als Transkript gefunden werden. Wer sich darauf
    verlaesst, waehlt bei einem z.B. rein japanischen Video den Link-Weg — und Open
    Notebook legt wieder eine leere Quelle an. Also fragen wir genau die Bibliothek,
    die im Container auch die Entscheidung trifft.
    Nicht-YouTube-Quellen: leere Liste — fuer die kennt content-core keinen
    Untertitel-Weg, da fuehrt ohnehin nur die Tonspur zum Ziel.
    """
    m = YOUTUBE_ID.search(url)
    if not m:
        return []
    probe_code = (
        "from youtube_transcript_api import YouTubeTranscriptApi as A;"
        f"print(' '.join(t.language_code for t in A().list('{m.group(1)}')))"
    )
    try:
        out = run(["docker", "compose", "exec", "-T", "open_notebook",
                   "/app/.venv/bin/python", "-c", probe_code], cwd=REPO)
    except subprocess.CalledProcessError:
        return []  # kein Transkript vorhanden (oder Abruf blockiert) -> Tonspur
    have = out.stdout.split()
    return [lang for lang in CAPTION_LANGS
            if any(h == lang or h.startswith(f"{lang}-") for h in have)]
 def download_audio(url: str, dest_dir: Path, stem: str) -> Path:
    """Tonspur laden. `stem` macht den Dateinamen eindeutig — Open Notebook legt den
    Upload unter seinem Originalnamen in data/uploads/ ab, ein fixer Name wie
    "audio.mp3" wuerde beim naechsten Video kollidieren."""
    print("Lade Tonspur (yt-dlp)...")
    run([
        "yt-dlp", "-x", "--audio-format", "mp3", "--audio-quality", "5",
        "--postprocessor-args", f"ExtractAudio:{AUDIO_POSTPROC}",
        "--no-warnings", "--no-playlist",
        "-o", str(dest_dir / f"{stem}.%(ext)s"), url,
    ])
    files = list(dest_dir.glob(f"{stem}.mp3"))
    if not files:
        raise SystemExit(die("yt-dlp hat keine Audiodatei erzeugt."))
    f = files[0]
    print(f"  {f.name}: {f.stat().st_size / 1e6:.1f} MB")
    return f
 def post_source(fields: dict[str, str], file: Path | None = None) -> dict:
    """POST /api/sources als multipart/form-data (das erwartet die API)."""
    cmd = ["curl", "-sS", "-X", "POST", f"{API}/api/sources"]
    for k, v in fields.items():
        cmd += ["-F", f"{k}={v}"]
    if file is not None:
        cmd += ["-F", f"file=@{file}"]
    out = run(cmd).stdout
    try:
        return json.loads(out)
    except json.JSONDecodeError:
        raise SystemExit(die(f"Unerwartete Antwort der API: {out[:300]}"))
 def cleanup_upload(filename: str) -> None:
    """Die hochgeladene MP3 nach dem Transkribieren wegraeumen.
    Eigentlich ist dafuer das API-Flag delete_source=true gedacht — das ist in dieser
    content-core-Version aber wirkungslos: extract_content verliert das Feld beim
    Aufbau des Graph-State (der zurueckgegebene State zeigt delete_source=None), der
    delete_file-Knoten sieht es nie und ueberspringt das Loeschen. Ohne diesen Schritt
    blieben pro Video ein paar MB in notebook_data/uploads/ liegen, die niemand mehr
    braucht — der Text steckt ja laengst in der Quelle.
    Loeschen im Container, weil die Datei dort als root angelegt wird.
    """
    path = f"/app/data/uploads/{filename}"
    try:
        run(["docker", "compose", "exec", "-T", "open_notebook", "rm", "-f", "--", path],
            cwd=REPO)
        print(f"Aufgeraeumt: uploads/{filename}")
    except subprocess.CalledProcessError as e:
        print(f"Hinweis: uploads/{filename} konnte nicht geloescht werden "
              f"({e.stderr.strip()[:100]}) — stoert nicht, belegt nur Platz.",
              file=sys.stderr)
 def wait_for_content(source_id: str, timeout_s: int) -> dict | None:
    """Auf die asynchrone Verarbeitung warten (Transkription kann Minuten dauern)."""
    quoted = urllib.parse.quote(source_id, safe="")
    deadline = time.time() + timeout_s
    last = None
    while time.time() < deadline:
        try:
            last = api_get(f"/api/sources/{quoted}")
        except urllib.error.HTTPError:
            pass
        if last:
            text = last.get("full_text") or ""
            if text.strip():
                return last
        print("  ... wird verarbeitet", end="\r", flush=True)
        time.sleep(10)
    return None
 def main() -> int:
    ap = argparse.ArgumentParser(description=__doc__,
                                 formatter_class=argparse.RawDescriptionHelpFormatter)
    ap.add_argument("url")
    ap.add_argument("--notebook", help="Notebook-ID (Default: das einzige vorhandene)")
    ap.add_argument("--language", help="Sprache fuers Transkribieren erzwingen, z.B. de")
    ap.add_argument("--force-audio", action="store_true",
                    help="Untertitel ignorieren und immer die Tonspur transkribieren")
    ap.add_argument("--timeout", type=int, default=1800, metavar="S",
                    help="Wartezeit auf die Verarbeitung (Default: 1800s)")
    args = ap.parse_args()
    try:
        notebooks = api_get("/api/notebooks")
    except (urllib.error.URLError, TimeoutError) as e:
        return die(f"Open-Notebook-API unter {API} nicht erreichbar ({e}).")
    notebook_id = args.notebook
    if not notebook_id:
        if len(notebooks) != 1:
            return die("Mehrere (oder keine) Notebooks vorhanden — bitte --notebook angeben:\n"
                       + "\n".join(f"  {n['id']}  {n.get('name')}" for n in notebooks))
        notebook_id = notebooks[0]["id"]
        print(f"Notebook: {notebooks[0].get('name')} ({notebook_id})")
    meta = probe(args.url)
    title = meta.get("title") or args.url
    dur = meta.get("duration")
    langs = [] if args.force_audio else caption_langs(args.url)
    print(f"Video   : {title}" + (f"  ({dur // 60}:{dur % 60:02d} min)" if dur else ""))
    use_audio = args.force_audio or not langs
    if args.force_audio:
        print("Untertitel werden ignoriert (--force-audio) — Tonspur wird transkribiert.")
    elif langs:
        print(f"Untertitel vorhanden ({', '.join(langs)}) — nutze den direkten Weg.")
    else:
        print("Keine brauchbaren Untertitel — Tonspur wird transkribiert.")
    common = {
        "notebook_id": notebook_id,
        "embed": "true",
        "async_processing": "true",
        "title": title,
    }
    uploaded_name: str | None = None
    if not use_audio:
        src = post_source({**common, "type": "link", "url": args.url})
    else:
        stem = meta.get("id") or "video"
        with tempfile.TemporaryDirectory() as td:
            audio = download_audio(args.url, Path(td), stem)
            print("Lade zu Open Notebook hoch; die Transkription laeuft dort auf dem "
                  "lokalen Whisper-Server...")
            src = post_source({**common, "type": "upload"}, file=audio)
            uploaded_name = audio.name
    source_id = src.get("id")
    if not source_id:
        return die(f"Quelle wurde nicht angelegt: {json.dumps(src)[:300]}")
    print(f"Quelle angelegt: {source_id}")
    done = wait_for_content(source_id, args.timeout)
    if not done:
        return die(f"Nach {args.timeout}s kein Inhalt. Status pruefen mit:\n"
                   f"  docker compose logs --tail 50 open_notebook")
    text = done.get("full_text") or ""
    print(f"\nFertig: {len(text)} Zeichen Inhalt.")
    print(f"Anfang: {text[:160].strip()}...")
    if uploaded_name:
        cleanup_upload(uploaded_name)
    return 0
 if __name__ == "__main__":
    sys.exit(main())
--- a/scripts/check_updates.sh
+++ b/scripts/check_updates.sh
@ -0,0 +1,57 @@
 #!/usr/bin/env bash
 # Zeigt, ob es ein neues Open-Notebook-Release gibt. Aendert NICHTS.
 #
 # Gedacht fuer den Sitzungsstart: Du siehst, worauf du laeufst und ob etwas Neues da
 # ist — und entscheidest selbst, wann du migrierst (./scripts/update_stack.sh).
 # Bewusst kein Auto-Update: ein Image-Update migriert die Datenbank und kann unsere
 # Anpassungen brechen (siehe scripts/smoke_test.sh).
 set -uo pipefail
 REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
 cd "$REPO"
 TAG="lfnovo/open_notebook:v1-latest"
 # Gezielt aus der open_notebook-Zeile: ein blindes grep auf den ersten Digest der Datei
 # erwischt surrealdb (steht weiter oben) und vergleicht dann Aepfel mit Birnen.
 pinned="$(grep -oP 'lfnovo/open_notebook\S*@\Ksha256:[0-9a-f]+' docker-compose.yml | head -1)"
 running_ver="$(docker compose exec -T open_notebook grep -m1 '^version' /app/pyproject.toml 2>/dev/null |
               grep -oP '(?<=")[^"]+' || echo '?')"
 echo "Laeuft gerade"
 echo "  Version : ${running_ver}"
 echo "  Digest  : ${pinned:-(nicht gepinnt!)}"
 echo
 echo "Verfuegbar"
 latest_rel="$(curl -sf -m 10 https://api.github.com/repos/lfnovo/open-notebook/releases/latest |
              python3 -c "import json,sys;d=json.load(sys.stdin);print(d['tag_name'],'vom',d['published_at'][:10])" 2>/dev/null)"
 echo "  Release : ${latest_rel:-(GitHub nicht erreichbar)}"
 remote="$(docker buildx imagetools inspect "$TAG" --format '{{.Manifest.Digest}}' 2>/dev/null)"
 if [ -z "$remote" ]; then
  echo "  Image   : (Registry nicht erreichbar)"
  exit 0
 fi
 echo "  Image   : $remote"
 echo
 if [ "$remote" = "$pinned" ]; then
  echo "Du laeufst auf dem aktuellen Release. Nichts zu tun."
  exit 0
 fi
 cat <<EOF
 NEUES IMAGE VERFUEGBAR.
  gepinnt : $pinned
  neu     : $remote
 Update mit Backup, Rauchtest und Rollback:
  ./scripts/update_stack.sh
 Vorher lohnt ein Blick in die Release-Notes — vor allem auf Aenderungen an den
 Podcast-Vorlagen, an content-core und an der Datenbank:
  https://github.com/lfnovo/open-notebook/releases
 EOF
 exit 0
--- a/scripts/prune_podcast_data.py
+++ b/scripts/prune_podcast_data.py
@ -0,0 +1,209 @@
 #!/usr/bin/env python3
 """Raeumt Podcast-Daten auf, die Open Notebook selbst liegen laesst.
 Zwei Arten von Muell entstehen unter notebook_data/podcasts/episodes/<uuid>/:
 1. Verwaiste Episodenordner. Beim Loeschen einer Episode entfernt Open Notebook
   nur die finale MP3 (api/routers/podcasts.py: audio_path.unlink()), nicht den
   Ordner mit clips/, outline.json und transcript.json. Auch fehlgeschlagene und
   per /retry ersetzte Laeufe lassen ihren Ordner zurueck.
 2. clips/ fertiger Episoden. Die Einzelclips sind nur Zwischenprodukt; sobald die
   finale MP3 existiert, werden sie nicht mehr gebraucht.
 Standardmaessig Trockenlauf — es wird nur angezeigt, was wegfiele. Erst mit --yes
 wird geloescht.
    ./scripts/prune_podcast_data.py            # anzeigen
    ./scripts/prune_podcast_data.py --yes      # loeschen
    ./scripts/prune_podcast_data.py --keep-clips --yes   # nur Verwaiste loeschen
 """
 from __future__ import annotations
 import argparse
 import json
 import subprocess
 import sys
 import time
 import urllib.error
 import urllib.request
 from pathlib import Path
 API = "http://127.0.0.1:5055"
 REPO = Path(__file__).resolve().parent.parent
 EPISODES_DIR = REPO / "notebook_data" / "podcasts" / "episodes"
 # Pfade in der DB sind Container-Pfade (/app/data/...), auf dem Host liegen sie
 # unter notebook_data/ (Bind-Mount aus docker-compose.yml).
 CONTAINER_PREFIX = "/app/data/"
 HOST_PREFIX = REPO / "notebook_data"
 # Ein laufender Job hat noch keine audio_file, sein Ordner ist also nicht als
 # "in Benutzung" erkennbar. Frisch angefasste Ordner deshalb grundsaetzlich in
 # Ruhe lassen.
 DEFAULT_MIN_AGE_MIN = 60
 def die(msg: str) -> None:
    print(f"FEHLER: {msg}", file=sys.stderr)
    sys.exit(1)
 def fetch_episodes() -> list[dict]:
    try:
        with urllib.request.urlopen(f"{API}/api/podcasts/episodes", timeout=10) as r:
            return json.load(r)
    except (urllib.error.URLError, TimeoutError) as e:
        die(f"API unter {API} nicht erreichbar ({e}). Laeuft der Container? "
            "Ohne die Episodenliste laesst sich nicht sicher unterscheiden, "
            "welche Ordner noch gebraucht werden.")
 def to_host_path(audio_file: str) -> Path | None:
    """Container-Pfad der DB -> Pfad auf dem Host."""
    if audio_file.startswith("file://"):
        audio_file = audio_file[len("file://"):]
    if audio_file.startswith(CONTAINER_PREFIX):
        return HOST_PREFIX / audio_file[len(CONTAINER_PREFIX):]
    p = Path(audio_file)
    return p if p.is_absolute() and p.exists() else None
 def to_container_path(host_path: Path) -> str:
    """Pfad auf dem Host -> Container-Pfad (Umkehrung von to_host_path)."""
    rel = host_path.resolve().relative_to(HOST_PREFIX.resolve())
    return CONTAINER_PREFIX + str(rel)
 def delete_in_container(paths: list[Path]) -> None:
    """Loescht im Container statt auf dem Host.
    Der open_notebook-Container laeuft als root und legt die Episodenordner
    entsprechend root:root an — der Host-User darf sie nicht entfernen. Statt
    dafuer sudo zu verlangen, loeschen wir dort, wo die Rechte ohnehin stimmen.
    """
    targets = [to_container_path(p) for p in paths]
    guard = CONTAINER_PREFIX + "podcasts/episodes/"
    for t in targets:  # Sicherheitsnetz gegen Pfade ausserhalb der Episoden.
        if not t.startswith(guard):
            die(f"Abbruch: {t} liegt ausserhalb von {guard}")
    subprocess.run(
        ["docker", "compose", "exec", "-T", "open_notebook", "rm", "-rf", "--", *targets],
        cwd=REPO, check=True, capture_output=True,
    )
 def dir_size_mb(path: Path) -> float:
    return sum(f.stat().st_size for f in path.rglob("*") if f.is_file()) / 1e6
 def age_minutes(path: Path) -> float:
    newest = max((f.stat().st_mtime for f in path.rglob("*")), default=path.stat().st_mtime)
    return (time.time() - newest) / 60
 def main() -> int:
    ap = argparse.ArgumentParser(description=__doc__,
                                 formatter_class=argparse.RawDescriptionHelpFormatter)
    ap.add_argument("--yes", action="store_true",
                    help="wirklich loeschen (ohne dies nur Trockenlauf)")
    ap.add_argument("--keep-clips", action="store_true",
                    help="clips/ fertiger Episoden behalten, nur verwaiste Ordner loeschen")
    ap.add_argument("--min-age", type=int, default=DEFAULT_MIN_AGE_MIN, metavar="MIN",
                    help=f"Ordner juenger als MIN Minuten nie anfassen (Default: {DEFAULT_MIN_AGE_MIN})")
    args = ap.parse_args()
    if not EPISODES_DIR.is_dir():
        print(f"Nichts zu tun: {EPISODES_DIR} existiert nicht.")
        return 0
    episodes = fetch_episodes()
    # Laufende Jobs schreiben gerade in ihren Ordner, den wir nicht zuordnen
    # koennen (noch keine audio_file). Dann lieber gar nichts loeschen.
    running = [e for e in episodes if e.get("job_status") in ("running", "pending", "submitted")]
    if running:
        die(f"{len(running)} Podcast-Job(s) laufen gerade "
            f"({', '.join(e.get('name') or e['id'] for e in running)}). "
            "Bitte warten, bis sie fertig sind — sonst wuerde ihr Arbeitsverzeichnis geloescht.")
    # Ordner, die zu einer existierenden Episode gehoeren.
    live: dict[Path, dict] = {}
    for e in episodes:
        af = e.get("audio_file")
        if not af:
            continue
        host = to_host_path(af)
        if host:
            live[host.parent.parent.resolve()] = e  # <uuid>/audio/<uuid>.mp3 -> <uuid>/
    orphans: list[tuple[Path, float]] = []
    clip_dirs: list[tuple[Path, float, str]] = []
    skipped_young: list[str] = []
    for d in sorted(EPISODES_DIR.iterdir()):
        if not d.is_dir():
            continue
        d = d.resolve()
        if d.parent != EPISODES_DIR.resolve():  # Symlink-Ausbruch o.ae.
            continue
        if age_minutes(d) < args.min_age:
            skipped_young.append(d.name)
            continue
        ep = live.get(d)
        if ep is None:
            orphans.append((d, dir_size_mb(d)))
            continue
        clips = d / "clips"
        final_mp3 = to_host_path(ep["audio_file"])
        if not args.keep_clips and clips.is_dir() and final_mp3 and final_mp3.exists():
            n = len(list(clips.glob("*")))
            if n:
                clip_dirs.append((clips, dir_size_mb(clips), ep.get("name") or ep["id"]))
    if orphans:
        print(f"Verwaiste Episodenordner (kein DB-Eintrag mehr) — {len(orphans)}:")
        for d, mb in orphans:
            print(f"  {d.name}  {mb:6.1f} MB")
    if clip_dirs:
        print(f"\nZwischenclips fertiger Episoden (finale MP3 existiert) — {len(clip_dirs)}:")
        for c, mb, name in clip_dirs:
            print(f"  {c.parent.name}/clips  {mb:6.1f} MB   ({name})")
    if skipped_young:
        print(f"\nUebersprungen (juenger als {args.min_age} min): {', '.join(skipped_young)}")
    total = sum(mb for _, mb in orphans) + sum(mb for _, mb, _ in clip_dirs)
    if not orphans and not clip_dirs:
        print("Nichts aufzuraeumen.")
        return 0
    print(f"\nFreizugeben: {total:.1f} MB")
    if not args.yes:
        print("Trockenlauf — nichts geloescht. Mit --yes wirklich loeschen.")
        return 0
    victims = [d for d, _ in orphans] + [c for c, _, _ in clip_dirs]
    try:
        delete_in_container(victims)
    except FileNotFoundError:
        die("docker nicht gefunden — das Loeschen laeuft im Container "
            "(die Ordner gehoeren root).")
    except subprocess.CalledProcessError as e:
        die(f"Loeschen im Container fehlgeschlagen: {e.stderr.decode().strip()}")
    for d, _ in orphans:
        print(f"geloescht: {d.name}")
    for c, _, _ in clip_dirs:
        print(f"geloescht: {c.parent.name}/clips")
    rest = [p for p in victims if p.exists()]
    if rest:
        die(f"{len(rest)} Pfad(e) existieren noch: {', '.join(p.name for p in rest)}")
    print(f"\n{total:.1f} MB freigegeben.")
    return 0
 if __name__ == "__main__":
    sys.exit(main())
--- a/scripts/smoke_test.sh
+++ b/scripts/smoke_test.sh
@ -0,0 +1,161 @@
 #!/usr/bin/env bash
 # Rauchtest fuer den laufenden Stack.
 #
 # Prueft nicht "laeuft der Container", sondern genau die Dinge, die bei diesem
 # Deployment schon einmal kaputt waren — inklusive der Annahmen, auf denen unsere
 # Anpassungen beruhen. Genau die koennen ein Image-Update stillschweigend brechen:
 # der Bind-Mount ueber prompts/podcast/ verdeckt neue Dateien, config/content_core.yaml
 # friert alte Defaults ein (CCORE_CONFIG_PATH ersetzt, statt zu mergen), und die
 # Env-Var-Workarounds haengen an Interna von content-core/esperanto/podcast_creator.
 #
 #   ./scripts/smoke_test.sh          # alles
 #   SKIP_NET=1 ./scripts/smoke_test.sh   # ohne Tests, die ins Internet muessen
 #
 # Exit 0 = alles gruen. Exit 1 = mindestens ein Test rot.
 set -uo pipefail
 REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
 cd "$REPO"
 API="http://127.0.0.1:5055"
 IMAGE="$(grep -oP '(?<=image: )lfnovo/open_notebook\S+' docker-compose.yml | head -1)"
 PY=/app/.venv/bin/python
 fails=0
 ok()   { printf '  \033[32mOK\033[0m   %s\n' "$1"; }
 fail() { printf '  \033[31mFEHL\033[0m %s\n' "$1"; fails=$((fails + 1)); }
 sec()  { printf '\n%s\n' "$1"; }
 inapp() { docker compose exec -T open_notebook "$@" 2>/dev/null; }
 sec "Grundfunktion"
 if curl -sf -m 10 "$API/api/models/providers" -o /tmp/.st_prov 2>/dev/null; then
  if grep -q ollama /tmp/.st_prov && grep -q openrouter /tmp/.st_prov; then
    ok "API erreichbar, Provider ollama + openrouter verfuegbar"
  else
    fail "API erreichbar, aber Provider fehlen: $(cat /tmp/.st_prov)"
  fi
 else
  fail "API $API nicht erreichbar"
 fi
 rm -f /tmp/.st_prov
 sec "Lokale Sprachdienste (vom Container aus)"
 if inapp "$PY" -c "
 import httpx,sys
 r = httpx.post('http://host.docker.internal:8901/audio/speech',
    json={'model':'chatterbox','voice':'default','input':'Test.','response_format':'wav'}, timeout=180)
 sys.exit(0 if r.status_code == 200 and len(r.content) > 1000 else 1)"; then
  ok "TTS antwortet mit Audio (host.docker.internal:8901)"
 else
  fail "TTS liefert kein Audio — Dienst tot (ConnectError) oder VRAM voll (HTTP 500)?
         systemctl --user status open-notebook-tts; journalctl --user -u open-notebook-tts -n 30"
 fi
 if inapp "$PY" -c "
 import httpx,sys
 r = httpx.get('http://host.docker.internal:8902/health', timeout=30)
 sys.exit(0 if r.status_code == 200 else 1)"; then
  ok "STT erreichbar (host.docker.internal:8902)"
 else
  fail "STT nicht erreichbar — systemctl --user status open-notebook-stt"
 fi
 sec "Annahmen unserer Anpassungen (brechen leise bei Image-Updates)"
 # 1) prompts/podcast: Verzeichnis-Mount verdeckt das Image-Verzeichnis komplett.
 #    Kommt upstream eine dritte Vorlage dazu, waere sie unsichtbar.
 img_tpl="$(docker run --rm --entrypoint ls "$IMAGE" /app/prompts/podcast 2>/dev/null | sort | tr '\n' ' ')"
 our_tpl="$(ls prompts/podcast | sort | tr '\n' ' ')"
 if [ "$img_tpl" = "$our_tpl" ]; then
  ok "prompts/podcast: Dateiliste deckt sich mit dem Image ($our_tpl)"
 else
  fail "prompts/podcast weicht ab — unser Mount verdeckt Image-Dateien!
         Image: $img_tpl
         wir  : $our_tpl"
 fi
 # 2) Unsere Patches in den Vorlagen sind noch drin.
 if grep -q '{{ language }}' prompts/podcast/outline.jinja prompts/podcast/transcript.jinja &&
   head -1 prompts/podcast/outline.jinja | grep -q '/no_think'; then
  ok "Vorlagen enthalten Sprachanweisung ({{ language }}) und /no_think"
 else
  fail "Vorlagen-Patches fehlen — Podcasts kaemen wieder auf Englisch / mit kaputtem JSON"
 fi
 # 3) podcast_creator reicht `language` ueberhaupt noch in die Vorlagen durch.
 if inapp grep -q "language" /app/.venv/lib/python3.12/site-packages/podcast_creator/nodes.py; then
  ok "podcast_creator kennt weiterhin ein language-Feld"
 else
  fail "podcast_creator reicht kein language mehr durch — Vorlagen-Patch wirkungslos"
 fi
 # 4) TTS_BATCH_SIZE existiert noch als Stellschraube (sonst wieder 5 parallele Clips -> OOM).
 if inapp grep -q "TTS_BATCH_SIZE" /app/.venv/lib/python3.12/site-packages/podcast_creator/nodes.py; then
  ok "podcast_creator liest weiterhin TTS_BATCH_SIZE"
 else
  fail "TTS_BATCH_SIZE wird nicht mehr gelesen — Gefahr paralleler Clips (CUDA OOM)"
 fi
 # 5) content-core sucht YouTube-Untertitel weiterhin ueber preferred_languages.
 if inapp grep -q "preferred_languages" /app/.venv/lib/python3.12/site-packages/content_core/processors/youtube.py; then
  ok "content-core nutzt weiterhin preferred_languages (deutsche Untertitel)"
 else
  fail "content-core sucht Untertitel anders — config/content_core.yaml pruefen"
 fi
 # 6) content-core baut das STT-Modell weiterhin ohne base_url (deshalb die Env-Var).
 if inapp grep -q "OPENAI_COMPATIBLE_BASE_URL" /app/.venv/lib/python3.12/site-packages/esperanto/providers/stt/openai_compatible.py; then
  ok "esperanto akzeptiert weiterhin OPENAI_COMPATIBLE_BASE_URL_STT (Audio-Transkription)"
 else
  fail "esperanto liest die Env-Var nicht mehr — Audio-Quellen fielen auf OpenAI zurueck"
 fi
 # 7) CCORE_CONFIG_PATH ersetzt die Config komplett. Bringt ein Update neue
 #    Top-Level-Schluessel mit, frieren wir die alten ein — hier faellt das auf.
 img_keys="$(docker run --rm --entrypoint "$PY" -e CCORE_CONFIG_PATH= "$IMAGE" \
  -c "from content_core.config import load_config; print(' '.join(sorted(load_config())))" 2>/dev/null)"
 our_keys="$(inapp "$PY" -c "
 import yaml; print(' '.join(sorted(yaml.safe_load(open('/app/config/content_core.yaml')))))")"
 missing="$(comm -23 <(tr ' ' '\n' <<<"$img_keys" | sort -u) <(tr ' ' '\n' <<<"$our_keys" | sort -u) | tr '\n' ' ')"
 if [ -z "${missing// /}" ] && [ -n "$img_keys" ]; then
  ok "config/content_core.yaml enthaelt alle Default-Schluessel des Images"
 else
  fail "config/content_core.yaml fehlen Schluessel aus dem Image: ${missing:-(Image-Defaults nicht lesbar)}
         Neu erzeugen — siehe Kopfkommentar in config/content_core.yaml"
 fi
 # 8) Episode-Profile haben eine Sprache (sonst wieder englische Podcasts).
 if curl -sf -m 10 "$API/api/episode-profiles" | python3 -c "
 import json,sys
 p=json.load(sys.stdin)
 sys.exit(0 if p and all(x.get('language') for x in p) else 1)" 2>/dev/null; then
  ok "alle Episode-Profile haben ein language-Feld gesetzt"
 else
  fail "mindestens ein Episode-Profil ohne language — Podcast kaeme auf Englisch"
 fi
 if [ "${SKIP_NET:-0}" != "1" ]; then
  sec "Ende-zu-Ende (braucht Internet)"
  if inapp "$PY" -c "
 import asyncio,sys
 from content_core import extract_content
 async def m():
    r = await extract_content({'url':'https://www.youtube.com/watch?v=hzxiegk9QAg'})
    sys.exit(0 if len((r.content or '')) > 1000 else 1)
 asyncio.run(m())"; then
    ok "YouTube-Quelle liefert Transkript (deutsche Untertitel werden gefunden)"
  else
    fail "YouTube-Extraktion liefert keinen Text — CCORE_CONFIG_PATH/Sprachen pruefen"
  fi
 fi
 sec "Ergebnis"
 if [ "$fails" -eq 0 ]; then
  echo "  Alles gruen."
  exit 0
 fi
 echo "  $fails Test(s) fehlgeschlagen."
 exit 1
--- a/scripts/start_services.sh
+++ b/scripts/start_services.sh
@ -1,31 +1,57 @@
 #!/usr/bin/env bash
-# Startet die lokalen TTS/STT-Wrapper-Server (services/tts_server.py,
+# Installiert und startet die lokalen TTS/STT-Wrapper-Server (services/tts_server.py,
-# services/stt_server.py) als Hintergrundprozesse auf GPU 2. Überspringt
+# services/stt_server.py) als systemd-User-Dienste auf GPU 2.
-# einen Service, falls er unter seinem Port bereits antwortet.
+#
 # Frueher liefen die beiden als nohup-Hintergrundprozesse — die ueberleben keinen
 # Reboot, und ein Podcast scheitert dann in der Vertonungsphase mit
 # "Failed to generate speech: All connection attempts failed". Als systemd-Units
 # starten sie automatisch mit (dank enable-linger auch ohne Login).
 #
 # Idempotent: mehrfach ausfuehrbar, aktualisiert die Units und startet sie neu.
 set -euo pipefail
-DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../services" && pwd)"
+REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
-mkdir -p "$DIR/logs"
+UNIT_SRC="$REPO/services/systemd"
-cd "$DIR"
+UNIT_DST="$HOME/.config/systemd/user"
 UNITS=(open-notebook-tts.service open-notebook-stt.service)
-start_if_needed() {
+mkdir -p "$UNIT_DST"
-  local name="$1" port="$2" cmd="$3"
+for unit in "${UNITS[@]}"; do
-  if curl -sf -m 2 "http://127.0.0.1:${port}/health" >/dev/null 2>&1; then
+  install -m 644 "$UNIT_SRC/$unit" "$UNIT_DST/$unit"
-    echo "$name läuft bereits auf Port $port"
+done
-    return
+systemctl --user daemon-reload
-  fi
+
-  echo "Starte $name auf Port $port..."
+# Ohne Linger laufen User-Dienste nur waehrend einer aktiven Login-Session und
-  eval "$cmd"
+# starten nicht beim Boot.
-  disown
+if [ "$(loginctl show-user "$USER" --property=Linger --value 2>/dev/null)" != "yes" ]; then
-  for _ in $(seq 1 15); do
+  echo "Aktiviere linger (Dienste sollen auch ohne Login laufen)..."
-    curl -sf -m 2 "http://127.0.0.1:${port}/health" >/dev/null 2>&1 && { echo "$name ist bereit."; return; }
+  loginctl enable-linger "$USER"
-    sleep 2
+fi
 echo "Starte TTS/STT-Dienste..."
 systemctl --user enable --quiet "${UNITS[@]}"
 systemctl --user restart "${UNITS[@]}"
 # Modelle werden beim Start geladen (whisper large-v3 braucht ~1 Minute).
 wait_healthy() {
  local name="$1" port="$2"
  for _ in $(seq 1 45); do
    if curl -sf -m 2 "http://127.0.0.1:${port}/health" >/dev/null 2>&1; then
      echo "  $name (Port $port) ist bereit."
      return 0
    fi
    sleep 4
  done
-  echo "WARNUNG: $name antwortet nach 30s nicht auf /health — siehe logs/${name}.log" >&2
+  echo "WARNUNG: $name antwortet nach 180s nicht auf /health." >&2
  echo "         Logs: journalctl --user -u open-notebook-${name} -n 50" >&2
  return 1
 }
-start_if_needed "tts" 8901 \
+rc=0
-  'CUDA_VISIBLE_DEVICES=2 TTS_PORT=8901 nohup ~/miniforge3/envs/chatterbox/bin/python tts_server.py > logs/tts_server.log 2>&1 &'
+wait_healthy tts 8901 || rc=1
 wait_healthy stt 8902 || rc=1
-start_if_needed "stt" 8902 \
+if [ "$rc" -eq 0 ]; then
-  'CUDA_VISIBLE_DEVICES=2 STT_PORT=8902 nohup python3 stt_server.py > logs/stt_server.log 2>&1 &'
+  echo "Fertig. Status: systemctl --user status ${UNITS[*]}"
 fi
 exit "$rc"
--- a/scripts/update_stack.sh
+++ b/scripts/update_stack.sh
@ -0,0 +1,104 @@
 #!/usr/bin/env bash
 # Aktualisiert Open Notebook auf das neueste veroeffentlichte Image — mit Rueckfallpunkt.
 #
 # Ablauf:
 #   1. Neuen Digest aus der Registry holen (nichts zu tun, wenn schon aktuell).
 #   2. Stack stoppen und Daten sichern (surreal_data + notebook_data + docker-compose.yml).
 #   3. Digest in docker-compose.yml eintragen, Stack starten.
 #   4. Rauchtest (scripts/smoke_test.sh).
 #   5. Rot -> alles zurueckrollen (Daten UND Compose-Datei). Gruen -> fertig.
 #
 # Der Rollback der Daten ist der Grund fuer das Backup: Open Notebook migriert die
 # SurrealDB beim Start, und ein Downgrade der Anwendung kann eine migrierte Datenbank
 # nicht wieder lesen. Ohne Backup waere ein missgluecktes Update eine Einbahnstrasse.
 #
 #   ./scripts/update_stack.sh              # auf neuestes Image
 #   ./scripts/update_stack.sh --dry-run    # nur zeigen, was passieren wuerde
 set -euo pipefail
 REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
 cd "$REPO"
 TAG="lfnovo/open_notebook:v1-latest"
 BACKUP_DIR="$REPO/backups"
 KEEP_BACKUPS=3
 DRY_RUN=0
 [ "${1:-}" = "--dry-run" ] && DRY_RUN=1
 say() { printf '\n\033[1m%s\033[0m\n' "$1"; }
 # Gezielt aus der open_notebook-Zeile. Ein blindes grep auf den ersten Digest der Datei
 # erwischt surrealdb (steht weiter oben) — das sed weiter unten wuerde dann den falschen
 # Digest ersetzen und die Datenbank austauschen statt der Anwendung.
 pinned="$(grep -oP 'lfnovo/open_notebook\S*@\Ksha256:[0-9a-f]+' docker-compose.yml | head -1)"
 [ -n "$pinned" ] || { echo "FEHLER: kein open_notebook-Digest in docker-compose.yml gepinnt." >&2; exit 1; }
 say "1/5  Registry abfragen"
 remote="$(docker buildx imagetools inspect "$TAG" --format '{{.Manifest.Digest}}')"
 echo "  gepinnt: $pinned"
 echo "  neu    : $remote"
 if [ "$remote" = "$pinned" ]; then
  echo "  Bereits aktuell — nichts zu tun."
  exit 0
 fi
 if [ "$DRY_RUN" = 1 ]; then
  echo "  (--dry-run: hier wuerde das Update starten)"
  exit 0
 fi
 ts="$(date +%Y%m%d-%H%M%S)"
 backup="$BACKUP_DIR/$ts"
 mkdir -p "$backup"
 say "2/5  Stack stoppen und sichern -> backups/$ts"
 docker compose down
 cp docker-compose.yml "$backup/docker-compose.yml"
 # Die Datenverzeichnisse gehoeren root (der Container schreibt als root), deshalb wird
 # im Container gepackt statt auf dem Host — spart sudo.
 docker run --rm -v "$REPO:/work" -w /work --entrypoint tar \
  "lfnovo/open_notebook@$pinned" -czf "backups/$ts/data.tgz" surreal_data notebook_data
 echo "  $(du -sh "$backup/data.tgz" | cut -f1) gesichert"
 restore() {
  say "ROLLBACK"
  docker compose down || true
  cp "$backup/docker-compose.yml" docker-compose.yml
  docker run --rm -v "$REPO:/work" -w /work --entrypoint sh \
    "lfnovo/open_notebook@$pinned" -c \
    "rm -rf surreal_data notebook_data && tar -xzf backups/$ts/data.tgz"
  docker compose up -d
  echo "  Alter Stand ($pinned) ist wiederhergestellt, Daten aus backups/$ts."
  echo "  Das neue Image bleibt lokal liegen — Ursache pruefen, dann erneut versuchen."
 }
 say "3/5  Auf neues Image umstellen"
 sed -i "s|@$pinned|@$remote|" docker-compose.yml
 docker compose up -d
 echo "  gestartet, warte auf die API..."
 for _ in $(seq 1 30); do
  curl -sf -m 3 http://127.0.0.1:5055/api/models/providers >/dev/null 2>&1 && break
  sleep 5
 done
 say "4/5  Rauchtest"
 if ./scripts/smoke_test.sh; then
  say "5/5  Fertig"
  newver="$(docker compose exec -T open_notebook grep -m1 '^version' /app/pyproject.toml |
            grep -oP '(?<=")[^"]+' || echo '?')"
  echo "  Update auf $newver erfolgreich, Rauchtest gruen."
  echo "  Backup: backups/$ts (bleibt liegen)"
  echo
  echo "  Bitte den neuen Digest committen:"
  echo "    git add docker-compose.yml && git commit -m 'Update Open Notebook auf $newver'"
 else
  echo
  echo "  Rauchtest ROT — das Update wird zurueckgenommen."
  restore
  exit 1
 fi
 # Alte Backups aufraeumen (Platte ist knapp).
 ls -1dt "$BACKUP_DIR"/*/ 2>/dev/null | tail -n +$((KEEP_BACKUPS + 1)) | while read -r old; do
  echo "  raeume altes Backup weg: $(basename "$old")"
  rm -rf "$old"
 done
--- a/services/systemd/open-notebook-stt.service
+++ b/services/systemd/open-notebook-stt.service
@ -0,0 +1,21 @@
 [Unit]
 Description=Open Notebook STT (faster-whisper, OpenAI-compatible wrapper on :8902)
 After=network.target
 [Service]
 Type=simple
 WorkingDirectory=%h/open-notebook/services
 ExecStart=%h/miniforge3/bin/python3 stt_server.py
 Restart=on-failure
 RestartSec=5
 StandardOutput=journal
 StandardError=journal
 Environment="STT_HOST=0.0.0.0"
 Environment="STT_PORT=8902"
 # GPU 2 eindeutig per UUID, NICHT per Index — siehe Kommentar in
 # open-notebook-tts.service.
 Environment="CUDA_VISIBLE_DEVICES=GPU-83ba6d1f-ebbe-f8c8-1433-153e0ebc3866"
 Environment="PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True"
 [Install]
 WantedBy=default.target
--- a/services/systemd/open-notebook-tts.service
+++ b/services/systemd/open-notebook-tts.service
@ -0,0 +1,23 @@
 [Unit]
 Description=Open Notebook TTS (chatterbox, OpenAI-compatible wrapper on :8901)
 After=network.target
 [Service]
 Type=simple
 WorkingDirectory=%h/open-notebook/services
 ExecStart=%h/miniforge3/envs/chatterbox/bin/python tts_server.py
 Restart=on-failure
 RestartSec=5
 StandardOutput=journal
 StandardError=journal
 Environment="TTS_HOST=0.0.0.0"
 Environment="TTS_PORT=8901"
 # GPU 2 eindeutig per UUID, NICHT per Index: CUDA sortiert standardmaessig
 # "fastest first" statt nach PCI-Reihenfolge, dadurch kann Index 2 auf der
 # T600 landen. UUID pruefen mit:
 #   nvidia-smi --query-gpu=index,name,uuid --format=csv
 Environment="CUDA_VISIBLE_DEVICES=GPU-83ba6d1f-ebbe-f8c8-1433-153e0ebc3866"
 Environment="PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True"
 [Install]
 WantedBy=default.target
--- a/services/tts_server.py
+++ b/services/tts_server.py
@ -25,6 +25,7 @@ import os
 import subprocess
 import sys
 import tempfile
 import threading
 from pathlib import Path
 sys.path.insert(0, str(Path.home() / "chatterbox-tts-cli"))
@ -40,6 +41,14 @@ app = FastAPI(title="Chatterbox TTS (OpenAI-compatible, voice cloning)", version
 _DEVICE = tts.get_device(None)
 _model_cache: dict[str, tuple] = {}
 # Serializes model load + inference. FastAPI runs sync handlers in a threadpool, so
 # without this every concurrent request generates in parallel on the same GPU and the
 # activation memory multiplies — podcast_creator sends TTS_BATCH_SIZE (default 5) clips
 # at once, which OOM'd this server (peak 16.7 GB) since GPU 2 is shared with the STT
 # server and the chatterbox MCP service. Generation is GPU-bound, so serializing costs
 # no real throughput; it just bounds peak VRAM to a single clip.
 _GPU_LOCK = threading.Lock()
 VOICES_DIR = Path(__file__).parent / "voices"
 CLONE_LANG = "de"  # language used for every cloned reference voice
 VOICE_LANG_OVERRIDES: dict[str, str] = {}  # e.g. {"john": "en"} if you add an English clip
@ -124,12 +133,18 @@ def speech(req: SpeechRequest):
    if not chunks:
        raise HTTPException(status_code=422, detail="Kein synthetisierbarer Text übrig.")
-    model, model_kind, sr = _get_model(lang)
+    with _GPU_LOCK:
-
+        model, model_kind, sr = _get_model(lang)
-    wavs = []
+        try:
-    for chunk in chunks:
+            wavs = []
-        wavs.append(tts.generate_chunk(model, model_kind, chunk, lang, voice_path))
+            for chunk in chunks:
-    final = wavs[0] if len(wavs) == 1 else torch.cat(wavs, dim=-1)
+                wavs.append(tts.generate_chunk(model, model_kind, chunk, lang, voice_path))
            final = wavs[0] if len(wavs) == 1 else torch.cat(wavs, dim=-1)
        finally:
            # GPU 2 is shared; hand cached blocks back so a neighbour process can't be
            # starved by fragmentation we're holding on to.
            if torch.cuda.is_available():
                torch.cuda.empty_cache()
    with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as wav_tmp:
        wav_path = wav_tmp.name
Author	SHA1	Message	Date
dschlueter	09c6d910a7	Docs: close the gaps an audit turned up Audited README, BEDIENUNGSANLEITUNG and CLAUDE.md against what the repo and the running stack actually do. Findings, all fixed: - CLAUDE.md claimed the repo holds "only deployment config (docker-compose.yml, .env, .gitignore)". It has not been that for a while: services/ is our own source (TTS/STT servers + systemd units), scripts/ is tooling, and prompts/podcast/ + config/content_core.yaml are overlays bind-mounted over the image. Those overlays are the fragile part, so the section now says so and points at smoke_test.sh. - CLAUDE.md still described the image as a moving tag and argued against patching app code "because it would rot against pull_policy: always" — both obsolete since the digest pin. - README listed neither yt-dlp (needed by add_video_source.py) nor smoke_test.sh in the verification step, and section 2.3 still told you to `docker compose pull` as if the tag moved. - BEDIENUNGSANLEITUNG had nothing at all about updates — the very thing a user has to do periodically. New section 8 covers check_updates / update_stack / smoke_test and, importantly, why updating is not automatic (DB migration is one-way without a backup; the local adaptations can break silently without crashing). - Renumbered the ad-hoc "3a. Updates" in README into a real section 4, and the BEDIENUNGSANLEITUNG sections after the new 8 accordingly. Verified mechanically: every scripts//services//config//prompts/ path named in the docs exists, and all internal and cross-document anchors resolve (0 dead links — one was already broken before this change: README pointed at BEDIENUNGSANLEITUNG.md#stimmklonung instead of #5-stimmklonung). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 14:17:06 +02:00
dschlueter	4805fe75ed	Pin images by digest; add update, smoke-test and status scripts Goal: at every session start, run exactly the application the repo says — and be able to move to a new upstream release deliberately, with a way back. The old setup (v1-latest + pull_policy: always) was unreliable in both directions. `docker compose up -d` silently pulled a new application, including an irreversible SurrealDB schema migration, while a reboot (restart: always) or `docker compose start` kept running the old image without pulling anything. The running version was effectively unpredictable. Note v1-latest tracks releases, not main: the current image (v1.10.0, 2026-06-18) IS the latest release — the repo being "ahead" is unreleased code, which is explicitly not wanted here. So this is about guaranteeing and detecting, not catching up. - docker-compose.yml: both images pinned by digest, pull_policy: missing. - scripts/check_updates.sh: reports running version/digest vs the latest release and registry digest. Changes nothing; meant for the session-start ritual. - scripts/update_stack.sh: resolve new digest -> stop -> back up surreal_data, notebook_data and docker-compose.yml -> repin -> start -> smoke test -> roll back data AND compose file if the smoke test fails. The data backup is the point: the app migrates the DB on startup and an older app cannot read a migrated DB, so a bad update would otherwise be a one-way door. tar runs inside a container because the data dirs are root-owned and the host user cannot restore over them. - scripts/smoke_test.sh: checks what actually breaks here, not just "does it start". Every local adaptation leans on upstream internals and can break silently: the prompts/podcast directory mount masks the image's directory (a new template upstream would be invisible), config/content_core.yaml freezes content-core's defaults (because CCORE_CONFIG_PATH replaces rather than merges), and the env-var workarounds depend on current content-core/esperanto/podcast_creator behaviour. So it verifies those assumptions explicitly, plus API, TTS audio, STT and a real YouTube transcript. Both scripts read the digest from the lfnovo/open_notebook line specifically. A naive "first sha256 in the file" grep matches surrealdb (listed first) — caught while testing: check_updates.sh falsely reported an update, and update_stack.sh would have rewritten the database image instead of the application. Verified: smoke test green against the current stack, and red (exit 1) when failures are injected (dead TTS port, removed template variable). Backup/restore mechanics exercised separately: 44 MB in 3.3s, restore yields 13 DB files and 91 notebook files. The update path itself cannot be exercised end to end until a newer image exists. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 14:12:31 +02:00
dschlueter	f350e49bd5	Add audio fallback so videos without captions work Open Notebook only reads existing captions from a video. Without them the source is created empty. yt-dlp lives on the host, not in the image, so the chain can't run inside the app: scripts/add_video_source.py bridges it. Captions present -> the URL goes in as a normal "link" source (fast, no download). No captions -> yt-dlp pulls just the audio track (mono 16 kHz mp3, ~6 MB per 17 min) and uploads it as an "upload" source. Open Notebook then transcribes it itself with its configured speech-to-text model — the local faster-whisper server — and embeds it. The script deliberately does not transcribe: the app's own pipeline already does STT, chunking and embedding. OPENAI_COMPATIBLE_BASE_URL_STT is what makes the upload path work at all. Open Notebook passes its STT model (openai_compatible/faster-whisper-large-v3) into content-core, but content-core builds it with AIFactory.create_speech_to_text(provider, model, {'timeout': ...}) — with no base_url. Esperanto can't locate the local server, content-core falls back to its default (openai/whisper-1) and the job dies with "OpenAI API key not found". That message is misleading: nothing is missing but the URL. The env var supplies it. An OPENAI_API_KEY is deliberately NOT put into the container — that would ship audio to a paid cloud service while a working Whisper sits idle on GPU 2. Caption availability is probed with youtube-transcript-api inside the container, not with yt-dlp. yt-dlp's automatic_captions lists YouTube's ~100 auto-translation targets (German is always among them), which youtube-transcript-api does not accept as transcripts — trusting it would route a Japanese-only video down the caption path and produce an empty source again. The uploaded mp3 is removed afterwards by the script. The API's delete_source=true flag is meant for exactly this but is inert in this content-core version: extract_content drops the field when building the graph state (the returned state carries delete_source=None), so the delete_file node never fires. Verified by running the flag through the real API path and watching the file survive. Files are also named after the video id, since every upload previously landed as audio.mp3 and a second video would have collided with the first. Verified end to end: the reported video (hzxiegk9QAg, 17:14) transcribes to 22069 characters via the local Whisper server (two segment requests, both 200), embeds into 14 chunks, and uploads/ is empty afterwards. An English video (aircAruvnKk) works with automatic language detection. Whisper's text is noticeably cleaner than YouTube's auto-captions ("GitHub" vs "Gitub", real punctuation), so --force-audio is useful even when captions exist. Known gap: the audio path stores the audio file as the source asset, so the original video URL is not recorded on the source (the caption path records it). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 13:49:42 +02:00
dschlueter	2a057d04e1	Fix YouTube sources with German transcripts landing empty Adding a German YouTube video created a source with a title but no content — nothing to chat with, nothing to summarise. The failure was silent in the UI; the reason only showed up in the container log as "Failed to get transcript for video <id> after retries: No suitable transcript found". Open Notebook extracts YouTube via content-core, which reads captions with youtube-transcript-api. content_core/processors/youtube.py only looks for the languages listed in preferred_languages, and the built-in default is ["en", "es", "pt"] — no German. All four fallback attempts in _fetch_best_transcript use that same list, so a video carrying only a German transcript raises NoTranscriptFound, get_best_transcript swallows it and returns None, and the source is stored with an empty body. The title still arrives because it is scraped separately, which is what makes this look like a success. The youtube_transcripts.preferred_languages key in the package's own cc_config.yaml appears to be the knob for this, but it is dead code on the default path: load_config() copies only the "extraction" block out of that file, so the key never reaches CONFIG and the hardcoded fallback always wins. The only way to set it is CCORE_CONFIG_PATH. config/content_core.yaml (bind-mounted read-only) therefore sets preferred_languages: ["de", "en", "es", "pt"]. Note CCORE_CONFIG_PATH REPLACES the config wholesale rather than merging it (config.py: return yaml.safe_load(file)). The file is consequently a full dump of the effective default config plus the new key — a minimal override file would silently drop the extraction engines and the model/timeout defaults. Its header comment carries the command that regenerates it from inside the container, for when an image update changes content-core's defaults. Verified end to end on the reported video (hzxiegk9QAg): extraction now yields 21553 characters of German transcript, the source is created through POST /api/sources with that full text, embedding produces 17 chunks in 3.4s, and vector search returns the video as top hit. This only covers caption languages. A video with no captions at all still yields an empty source — there is no download-and-transcribe fallback in the image. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 13:20:14 +02:00
dschlueter	ba98ac4eb4	Add prune_podcast_data.py: clean up podcast data Open Notebook leaves behind Deleting an episode does not delete its data. DELETE /api/podcasts/episodes/{id} resolves episode.audio_file and unlinks that one MP3 — the enclosing data/podcasts/episodes/<uuid>/ directory, holding clips/ (one MP3 per dialogue segment), outline.json and transcript.json, is never touched. Failed and /retry-replaced runs leak a directory as well. Nothing reaps any of it: this deployment had 15 directories on disk against 4 episodes in the database. That is an upstream gap (roughly a shutil.rmtree of audio_path.parent.parent in that handler), not something configurable here. Deliberately not patched by bind-mounting a modified router — that would fork app logic into a deployment repo and rot silently against pull_policy: always. Local cleanup instead. scripts/prune_podcast_data.py reconciles the episode list from the API against the directories on disk and removes: - orphaned directories (no corresponding episode), and - clips/ of completed episodes, since the clips are intermediate output once the final MP3 exists. Dry-run by default; --yes applies, --keep-clips restricts it to orphans. Two guards, both tested: it aborts when any job is running/pending (a running job has no audio_file yet, so its working directory is indistinguishable from an orphan) and it skips directories touched within the last 60 minutes (--min-age). It also refuses to act if the API is unreachable, rather than guessing. Deletion runs inside the container (docker compose exec … rm -rf): the container writes as root, so the host user cannot remove those directories — a plain host-side rmtree fails with EPERM after the first directory. Verified on this deployment: freed 13 MB (10 orphaned dirs + clips of 3 episodes, 49 MB -> 36 MB); all four surviving episodes still stream byte-identical MP3s from /audio afterwards. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 13:14:25 +02:00
dschlueter	03877588e1	Fix two podcast failures: dead TTS server after reboot + CUDA OOM Podcast generation failed twice at the audio stage, for two unrelated reasons that look similar from the UI but need opposite fixes. 1) TTS/STT ran as nohup background processes and silently did not survive a reboot. Podcasts then failed with "Failed to generate speech: All connection attempts failed" (httpx.ConnectError) even though outline and transcript had generated fine. Both now run as systemd user units. The unit files are versioned in services/systemd/ (using %h, not a hardcoded home) and installed by scripts/start_services.sh, which also enables linger so they start on boot without a login session. They pin GPU 2 by UUID, not by index: CUDA orders devices "fastest first", so index 2 can resolve to the T600. 2) GPU 2 is shared by three processes (TTS, STT and the separate chatterbox-tts MCP service on :9999), leaving ~12 GB of headroom. podcast_creator sends TTS_BATCH_SIZE (default 5) clips concurrently, and since /audio/speech is a sync FastAPI handler, they generated genuinely in parallel on one shared model. Activation memory multiplied, the TTS process hit 16.7 GB and threw torch.OutOfMemoryError, surfacing as "HTTP 500" from the endpoint. tts_server.py now serializes generation behind a lock (GPU-bound work, so parallelism buys no throughput — it only multiplies peak VRAM) and frees the cache afterwards. TTS_BATCH_SIZE=1 keeps the client from queuing requests in that lock and running into esperanto's 300s TTS timeout; ESPERANTO_TTS_TIMEOUT is raised to 600s as headroom. Verified: 5 concurrent /audio/speech requests all return 200 with GPU 2 peaking at ~11.5 GB (was 16.7 GB for the TTS process alone), and the previously failed episode now completes end to end — 38/38 batches, 10:56 min of audio, zero OOM. Docs record both failure signatures side by side, since ConnectError (server dead) and HTTP 500 (server alive, out of VRAM) have very different remedies. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 13:02:32 +02:00
dschlueter	054f867a8a	Fix podcasts generating in English + failing with invalid JSON Two root causes, both fixed in the bind-mounted podcast prompt templates (prompts/podcast/*.jinja): 1. Language: the episode profile's `language` field IS passed to both podcast_creator templates as {{ language }}, but the stock templates never reference it — so podcasts came out English (driven only by the English stock briefings/speakers), then read aloud by the German-locked Chatterbox TTS = "English with a German accent". Added a CRITICAL LANGUAGE REQUIREMENT block keyed on {{ language }} to both templates; now `language: "de"` actually forces German. Verified end-to-end: a full run produced a 44-line all-German transcript + audio. 2. Invalid json output failures: qwen3.5:27b is a thinking model; on long segments the <think> block ate the response-token budget and truncated the JSON. Prepended /no_think to both templates (~3x faster, valid JSON). Templates are bind-mounted read-only via docker-compose (directory mount, so edits survive a container restart without inode-staleness). Bundled briefings and speaker backstories were also translated to German to reduce drift. Known limitation documented: feeding an entire book (~70k tokens) as podcast content makes each of the 6 LLM calls take ~3.5 min and is unreliable; use a shorter source or summary. Confirmed working with concise content. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-11 02:30:05 +02:00
dschlueter	e2fb113213	Add dedicated API usage section (curl + Python) to BEDIENUNGSANLEITUNG.md Consolidates and extends the curl patterns already scattered across other sections into one reference point, notes the API requires no auth (loopback- only), and points to the interactive /docs Swagger UI as the best starting point for exploring endpoints not yet covered here. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-11 01:02:17 +02:00
dschlueter	83fd13fb98	Close two documentation gaps found during an accuracy audit - README's "Bekannte Stolpersteine" quick-reference list was missing the podcast-language pitfall (already documented in CLAUDE.md/BEDIENUNGSANLEITUNG.md). - The search/ask feature was only mentioned in passing; added a full section with the working curl examples and the SSE response shape, since it's the fastest/most accurate way to query long documents. Everything else cross-checked against live state (credential num_ctx, default models, running services, episode profile languages) and matched. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-11 00:59:06 +02:00
dschlueter	81b2e90326	Fix podcasts defaulting to English: set language=de on default episode profiles The three bundled episode profiles (business_analysis, solo_expert, tech_discussion) shipped with language=null, which podcast_creator treats as "unspecified" and defaults to English for outline/transcript generation regardless of source language. Set language="de" on all three. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-11 00:37:09 +02:00