Open Notebook only reads existing captions from a video. Without them the source
is created empty. yt-dlp lives on the host, not in the image, so the chain can't
run inside the app: scripts/add_video_source.py bridges it.
Captions present -> the URL goes in as a normal "link" source (fast, no download).
No captions -> yt-dlp pulls just the audio track (mono 16 kHz mp3, ~6 MB per
17 min) and uploads it as an "upload" source. Open Notebook
then transcribes it itself with its configured speech-to-text
model — the local faster-whisper server — and embeds it. The
script deliberately does not transcribe: the app's own pipeline
already does STT, chunking and embedding.
OPENAI_COMPATIBLE_BASE_URL_STT is what makes the upload path work at all. Open
Notebook passes its STT model (openai_compatible/faster-whisper-large-v3) into
content-core, but content-core builds it with
AIFactory.create_speech_to_text(provider, model, {'timeout': ...}) — with no
base_url. Esperanto can't locate the local server, content-core falls back to its
default (openai/whisper-1) and the job dies with "OpenAI API key not found". That
message is misleading: nothing is missing but the URL. The env var supplies it.
An OPENAI_API_KEY is deliberately NOT put into the container — that would ship
audio to a paid cloud service while a working Whisper sits idle on GPU 2.
Caption availability is probed with youtube-transcript-api inside the container,
not with yt-dlp. yt-dlp's automatic_captions lists YouTube's ~100 auto-translation
targets (German is always among them), which youtube-transcript-api does not
accept as transcripts — trusting it would route a Japanese-only video down the
caption path and produce an empty source again.
The uploaded mp3 is removed afterwards by the script. The API's delete_source=true
flag is meant for exactly this but is inert in this content-core version:
extract_content drops the field when building the graph state (the returned state
carries delete_source=None), so the delete_file node never fires. Verified by
running the flag through the real API path and watching the file survive. Files
are also named after the video id, since every upload previously landed as
audio.mp3 and a second video would have collided with the first.
Verified end to end: the reported video (hzxiegk9QAg, 17:14) transcribes to 22069
characters via the local Whisper server (two segment requests, both 200), embeds
into 14 chunks, and uploads/ is empty afterwards. An English video (aircAruvnKk)
works with automatic language detection. Whisper's text is noticeably cleaner than
YouTube's auto-captions ("GitHub" vs "Gitub", real punctuation), so --force-audio
is useful even when captions exist.
Known gap: the audio path stores the audio file as the source asset, so the
original video URL is not recorded on the source (the caption path records it).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Deleting an episode does not delete its data. DELETE /api/podcasts/episodes/{id}
resolves episode.audio_file and unlinks that one MP3 — the enclosing
data/podcasts/episodes/<uuid>/ directory, holding clips/ (one MP3 per dialogue
segment), outline.json and transcript.json, is never touched. Failed and
/retry-replaced runs leak a directory as well. Nothing reaps any of it: this
deployment had 15 directories on disk against 4 episodes in the database.
That is an upstream gap (roughly a shutil.rmtree of audio_path.parent.parent in
that handler), not something configurable here. Deliberately not patched by
bind-mounting a modified router — that would fork app logic into a deployment
repo and rot silently against pull_policy: always. Local cleanup instead.
scripts/prune_podcast_data.py reconciles the episode list from the API against
the directories on disk and removes:
- orphaned directories (no corresponding episode), and
- clips/ of completed episodes, since the clips are intermediate output once
the final MP3 exists.
Dry-run by default; --yes applies, --keep-clips restricts it to orphans.
Two guards, both tested: it aborts when any job is running/pending (a running
job has no audio_file yet, so its working directory is indistinguishable from an
orphan) and it skips directories touched within the last 60 minutes (--min-age).
It also refuses to act if the API is unreachable, rather than guessing.
Deletion runs inside the container (docker compose exec … rm -rf): the container
writes as root, so the host user cannot remove those directories — a plain
host-side rmtree fails with EPERM after the first directory.
Verified on this deployment: freed 13 MB (10 orphaned dirs + clips of 3 episodes,
49 MB -> 36 MB); all four surviving episodes still stream byte-identical MP3s
from /audio afterwards.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
tts_server.py now resolves `voice` against services/voices/*.wav (gitignored,
personal recordings) and clones via chatterbox's audio_prompt_path, falling
back to "default" (symlinked to ~/chatterbox-tts-cli/my_voice_deutsch_60s.wav)
for unknown names, or to the old built-in-voice-by-language-code behavior if
`voice` is a recognized language. All podcast speaker profiles now point at
voice_id "default" until distinct per-speaker reference clips are added.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Wraps the locally installed chatterbox-tts and faster-whisper packages in thin
FastAPI servers implementing OpenAI's audio API shape, since neither Ollama nor
OpenRouter support speech. Pinned to GPU 2 to avoid contending with Ollama's
resident models on GPU 1. Requires UFW rules for 8901/8902 (same pattern as
the existing 11434 rule) since UFW defaults to deny-incoming.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
