Adding a German YouTube video created a source with a title but no content —
nothing to chat with, nothing to summarise. The failure was silent in the UI;
the reason only showed up in the container log as "Failed to get transcript for
video <id> after retries: No suitable transcript found".
Open Notebook extracts YouTube via content-core, which reads captions with
youtube-transcript-api. content_core/processors/youtube.py only looks for the
languages listed in preferred_languages, and the built-in default is
["en", "es", "pt"] — no German. All four fallback attempts in
_fetch_best_transcript use that same list, so a video carrying only a German
transcript raises NoTranscriptFound, get_best_transcript swallows it and returns
None, and the source is stored with an empty body. The title still arrives
because it is scraped separately, which is what makes this look like a success.
The youtube_transcripts.preferred_languages key in the package's own
cc_config.yaml appears to be the knob for this, but it is dead code on the
default path: load_config() copies only the "extraction" block out of that file,
so the key never reaches CONFIG and the hardcoded fallback always wins. The only
way to set it is CCORE_CONFIG_PATH.
config/content_core.yaml (bind-mounted read-only) therefore sets
preferred_languages: ["de", "en", "es", "pt"].
Note CCORE_CONFIG_PATH REPLACES the config wholesale rather than merging it
(config.py: return yaml.safe_load(file)). The file is consequently a full dump of
the effective default config plus the new key — a minimal override file would
silently drop the extraction engines and the model/timeout defaults. Its header
comment carries the command that regenerates it from inside the container, for
when an image update changes content-core's defaults.
Verified end to end on the reported video (hzxiegk9QAg): extraction now yields
21553 characters of German transcript, the source is created through
POST /api/sources with that full text, embedding produces 17 chunks in 3.4s, and
vector search returns the video as top hit.
This only covers caption languages. A video with no captions at all still yields
an empty source — there is no download-and-transcribe fallback in the image.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Deleting an episode does not delete its data. DELETE /api/podcasts/episodes/{id}
resolves episode.audio_file and unlinks that one MP3 — the enclosing
data/podcasts/episodes/<uuid>/ directory, holding clips/ (one MP3 per dialogue
segment), outline.json and transcript.json, is never touched. Failed and
/retry-replaced runs leak a directory as well. Nothing reaps any of it: this
deployment had 15 directories on disk against 4 episodes in the database.
That is an upstream gap (roughly a shutil.rmtree of audio_path.parent.parent in
that handler), not something configurable here. Deliberately not patched by
bind-mounting a modified router — that would fork app logic into a deployment
repo and rot silently against pull_policy: always. Local cleanup instead.
scripts/prune_podcast_data.py reconciles the episode list from the API against
the directories on disk and removes:
- orphaned directories (no corresponding episode), and
- clips/ of completed episodes, since the clips are intermediate output once
the final MP3 exists.
Dry-run by default; --yes applies, --keep-clips restricts it to orphans.
Two guards, both tested: it aborts when any job is running/pending (a running
job has no audio_file yet, so its working directory is indistinguishable from an
orphan) and it skips directories touched within the last 60 minutes (--min-age).
It also refuses to act if the API is unreachable, rather than guessing.
Deletion runs inside the container (docker compose exec … rm -rf): the container
writes as root, so the host user cannot remove those directories — a plain
host-side rmtree fails with EPERM after the first directory.
Verified on this deployment: freed 13 MB (10 orphaned dirs + clips of 3 episodes,
49 MB -> 36 MB); all four surviving episodes still stream byte-identical MP3s
from /audio afterwards.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Podcast generation failed twice at the audio stage, for two unrelated reasons
that look similar from the UI but need opposite fixes.
1) TTS/STT ran as nohup background processes and silently did not survive a
reboot. Podcasts then failed with "Failed to generate speech: All connection
attempts failed" (httpx.ConnectError) even though outline and transcript had
generated fine.
Both now run as systemd user units. The unit files are versioned in
services/systemd/ (using %h, not a hardcoded home) and installed by
scripts/start_services.sh, which also enables linger so they start on boot
without a login session. They pin GPU 2 by UUID, not by index: CUDA orders
devices "fastest first", so index 2 can resolve to the T600.
2) GPU 2 is shared by three processes (TTS, STT and the separate chatterbox-tts
MCP service on :9999), leaving ~12 GB of headroom. podcast_creator sends
TTS_BATCH_SIZE (default 5) clips concurrently, and since /audio/speech is a
sync FastAPI handler, they generated genuinely in parallel on one shared
model. Activation memory multiplied, the TTS process hit 16.7 GB and threw
torch.OutOfMemoryError, surfacing as "HTTP 500" from the endpoint.
tts_server.py now serializes generation behind a lock (GPU-bound work, so
parallelism buys no throughput — it only multiplies peak VRAM) and frees the
cache afterwards. TTS_BATCH_SIZE=1 keeps the client from queuing requests in
that lock and running into esperanto's 300s TTS timeout; ESPERANTO_TTS_TIMEOUT
is raised to 600s as headroom.
Verified: 5 concurrent /audio/speech requests all return 200 with GPU 2 peaking
at ~11.5 GB (was 16.7 GB for the TTS process alone), and the previously failed
episode now completes end to end — 38/38 batches, 10:56 min of audio, zero OOM.
Docs record both failure signatures side by side, since ConnectError (server
dead) and HTTP 500 (server alive, out of VRAM) have very different remedies.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two root causes, both fixed in the bind-mounted podcast prompt templates
(prompts/podcast/*.jinja):
1. Language: the episode profile's `language` field IS passed to both
podcast_creator templates as {{ language }}, but the stock templates never
reference it — so podcasts came out English (driven only by the English
stock briefings/speakers), then read aloud by the German-locked Chatterbox
TTS = "English with a German accent". Added a CRITICAL LANGUAGE REQUIREMENT
block keyed on {{ language }} to both templates; now `language: "de"`
actually forces German. Verified end-to-end: a full run produced a 44-line
all-German transcript + audio.
2. Invalid json output failures: qwen3.5:27b is a thinking model; on long
segments the <think> block ate the response-token budget and truncated the
JSON. Prepended /no_think to both templates (~3x faster, valid JSON).
Templates are bind-mounted read-only via docker-compose (directory mount, so
edits survive a container restart without inode-staleness). Bundled briefings
and speaker backstories were also translated to German to reduce drift.
Known limitation documented: feeding an entire book (~70k tokens) as podcast
content makes each of the 6 LLM calls take ~3.5 min and is unreliable; use a
shorter source or summary. Confirmed working with concise content.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Consolidates and extends the curl patterns already scattered across other
sections into one reference point, notes the API requires no auth (loopback-
only), and points to the interactive /docs Swagger UI as the best starting
point for exploring endpoints not yet covered here.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- README's "Bekannte Stolpersteine" quick-reference list was missing the
podcast-language pitfall (already documented in CLAUDE.md/BEDIENUNGSANLEITUNG.md).
- The search/ask feature was only mentioned in passing; added a full section
with the working curl examples and the SSE response shape, since it's the
fastest/most accurate way to query long documents.
Everything else cross-checked against live state (credential num_ctx,
default models, running services, episode profile languages) and matched.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The three bundled episode profiles (business_analysis, solo_expert,
tech_discussion) shipped with language=null, which podcast_creator treats as
"unspecified" and defaults to English for outline/transcript generation
regardless of source language. Set language="de" on all three.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
README.md gives the precise, reproducible install path (tested the two
scripts against the live instance - both idempotent, correctly detect
already-registered credentials/models). BEDIENUNGSANLEITUNG.md covers daily
usage: notebooks/sources, the three chat context modes (and why "nur
Erkenntnisse" needs a transformation run first), podcasts, voice cloning,
and troubleshooting for the issues actually hit during setup (num_ctx
truncation, embedding GPU contention, ufw).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
