Add prune_podcast_data.py: clean up podcast data Open Notebook leaves behind
Deleting an episode does not delete its data. DELETE /api/podcasts/episodes/{id}
resolves episode.audio_file and unlinks that one MP3 — the enclosing
data/podcasts/episodes/<uuid>/ directory, holding clips/ (one MP3 per dialogue
segment), outline.json and transcript.json, is never touched. Failed and
/retry-replaced runs leak a directory as well. Nothing reaps any of it: this
deployment had 15 directories on disk against 4 episodes in the database.
That is an upstream gap (roughly a shutil.rmtree of audio_path.parent.parent in
that handler), not something configurable here. Deliberately not patched by
bind-mounting a modified router — that would fork app logic into a deployment
repo and rot silently against pull_policy: always. Local cleanup instead.
scripts/prune_podcast_data.py reconciles the episode list from the API against
the directories on disk and removes:
- orphaned directories (no corresponding episode), and
- clips/ of completed episodes, since the clips are intermediate output once
the final MP3 exists.
Dry-run by default; --yes applies, --keep-clips restricts it to orphans.
Two guards, both tested: it aborts when any job is running/pending (a running
job has no audio_file yet, so its working directory is indistinguishable from an
orphan) and it skips directories touched within the last 60 minutes (--min-age).
It also refuses to act if the API is unreachable, rather than guessing.
Deletion runs inside the container (docker compose exec … rm -rf): the container
writes as root, so the host user cannot remove those directories — a plain
host-side rmtree fails with EPERM after the first directory.
Verified on this deployment: freed 13 MB (10 orphaned dirs + clips of 3 episodes,
49 MB -> 36 MB); all four surviving episodes still stream byte-identical MP3s
from /audio afterwards.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
03877588e1
commit
ba98ac4eb4
5 changed files with 268 additions and 1 deletions
11
README.md
11
README.md
|
|
@ -178,6 +178,9 @@ systemctl --user status open-notebook-tts open-notebook-stt # TTS/STT-Status
|
|||
systemctl --user restart open-notebook-tts # z.B. nach neuer Stimme
|
||||
journalctl --user -u open-notebook-tts -f # TTS-Logs
|
||||
./scripts/start_services.sh # Units neu installieren + starten (idempotent)
|
||||
|
||||
./scripts/prune_podcast_data.py # Podcast-Datenmüll anzeigen (Trockenlauf)
|
||||
./scripts/prune_podcast_data.py --yes # ... und löschen
|
||||
```
|
||||
|
||||
TTS/STT starten nach einem Reboot von selbst — `start_services.sh` ist nur für die
|
||||
|
|
@ -202,7 +205,8 @@ open-notebook/
|
|||
│ └── logs/ # (gitignored, Altlast der früheren nohup-Variante)
|
||||
├── scripts/
|
||||
│ ├── setup_models.sh # Provider/Modelle einrichten (idempotent)
|
||||
│ └── start_services.sh # TTS/STT als systemd-User-Dienste installieren+starten
|
||||
│ ├── start_services.sh # TTS/STT als systemd-User-Dienste installieren+starten
|
||||
│ └── prune_podcast_data.py # verwaiste Podcast-Ordner + Zwischenclips aufräumen
|
||||
├── README.md # diese Datei
|
||||
├── BEDIENUNGSANLEITUNG.md # ausführliche Bedienungsanleitung (Deutsch)
|
||||
└── CLAUDE.md # technische Architektur-Doku für Weiterentwicklung
|
||||
|
|
@ -222,6 +226,11 @@ Kurzreferenz — Details jeweils in [`CLAUDE.md`](CLAUDE.md):
|
|||
leere Chat-Antworten bei größerem Kontext, ohne sichtbaren Fehler.
|
||||
- **Neue Host-Ports brauchen eine `ufw`-Regel**, sonst kann der Container sie nicht erreichen
|
||||
(Timeout, keine Fehlermeldung in Open Notebook selbst).
|
||||
- **Episoden löschen räumt nicht auf.** Der Mülleimer-Button (bzw.
|
||||
`DELETE /api/podcasts/episodes/{id}`) entfernt nur die finale MP3 und den DB-Eintrag — der
|
||||
Ordner `notebook_data/podcasts/episodes/<uuid>/` mit den Einzelclips, `outline.json` und
|
||||
`transcript.json` bleibt liegen. Das ist eine Lücke im Upstream-Code, kein lokales
|
||||
Konfigurationsproblem. Abhilfe: `./scripts/prune_podcast_data.py`.
|
||||
- **GPU 2 teilen sich drei Prozesse** (TTS, STT und der separate `chatterbox-tts`-MCP-Dienst auf
|
||||
Port 9999) — es bleiben nur ~12 GB Arbeitsspeicher für die Sprachsynthese. Deshalb serialisiert
|
||||
`tts_server.py` seine Generierung intern und `TTS_BATCH_SIZE=1` steht in `docker-compose.yml`:
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue