Fix podcasts generating in English + failing with invalid JSON

Two root causes, both fixed in the bind-mounted podcast prompt templates
(prompts/podcast/*.jinja):

1. Language: the episode profile's `language` field IS passed to both
   podcast_creator templates as {{ language }}, but the stock templates never
   reference it — so podcasts came out English (driven only by the English
   stock briefings/speakers), then read aloud by the German-locked Chatterbox
   TTS = "English with a German accent". Added a CRITICAL LANGUAGE REQUIREMENT
   block keyed on {{ language }} to both templates; now `language: "de"`
   actually forces German. Verified end-to-end: a full run produced a 44-line
   all-German transcript + audio.

2. Invalid json output failures: qwen3.5:27b is a thinking model; on long
   segments the <think> block ate the response-token budget and truncated the
   JSON. Prepended /no_think to both templates (~3x faster, valid JSON).

Templates are bind-mounted read-only via docker-compose (directory mount, so
edits survive a container restart without inode-staleness). Bundled briefings
and speaker backstories were also translated to German to reduce drift.

Known limitation documented: feeding an entire book (~70k tokens) as podcast
content makes each of the 6 LLM calls take ~3.5 min and is unreliable; use a
shorter source or summary. Confirmed working with concise content.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Dieter Schlüter 2026-07-11 02:30:05 +02:00
commit 054f867a8a
6 changed files with 304 additions and 18 deletions

View file

@ -41,13 +41,38 @@ A separate local `llama.cpp` server exists on this machine (managed via `~/llama
**Chat model context window:** the Ollama credential used for `qwen3.5:27b` (`credential:di8b31l16zikyyb6isyi`) has `num_ctx=98304` set explicitly. Without this, esperanto's `OllamaLanguageModel` silently defaults `num_ctx` to **8192** regardless of the model's real capability or `OLLAMA_CONTEXT_LENGTH` — any chat context (e.g. "full content" mode on a real document) beyond that gets truncated by Ollama, and because `qwen3.5:27b` is a thinking model, the truncated prompt frequently produces an empty final answer (the response is silently swallowed by `clean_thinking_content`) instead of a visible error — looks exactly like "chat gives no answer" from the UI. `98304` is the largest value that still keeps the whole model on GPU 1 (`ollama ps` shows `100% GPU`); `131072` spills ~91% onto CPU and makes a single response take 5-6 minutes instead of ~1-2. Even at 98304, a full-book "Volltext" chat (~70k+ tokens of context) takes ~2-3 minutes — that's inherent to processing that much context on one consumer GPU, not a bug. For large documents, the default short/RAG context mode (only relevant chunks, not the whole document) stays fast; reserve "Volltext" for shorter sources. If `qwen3.5:27b` ever gets swapped for a different Ollama chat model, re-check this credential's `num_ctx` still makes sense for its VRAM footprint.
**Podcast language:** the three bundled episode profiles (`business_analysis`, `solo_expert`,
`tech_discussion`) ship with `language: null`, which the underlying `podcast_creator` library
treats as "no language specified" and defaults to English for both outline and transcript
generation — regardless of what language the source content or the briefing text is in. All
three are set to `language: "de"` here. If you create a new episode profile, set `language`
explicitly (`PUT /api/episode-profiles/{id}`, full body) — it doesn't inherit from the notebook
or the source.
**Podcast language + reliability — fixed via patched prompt templates (`prompts/podcast/`,
bind-mounted).** Two separate problems, one fix location:
1. *Language.* The episode profile's `language` field (`"de"` → resolved to `"German"`) **is**
passed into both `podcast_creator` prompt templates as a `{{ language }}` variable
(`nodes.py`, both `generate_outline_node` and `generate_transcript_node`), but the stock
templates shipped in the image never reference it — so setting `language` had no effect and
podcasts came out in English (the stock briefings and speaker backstories are English, which
is the only language signal the model then sees). English text read aloud by the German-locked
Chatterbox TTS is what produced the "English with a bad German accent" symptom.
2. *Reliability.* `qwen3.5:27b` is a thinking model. Podcast content is the **entire notebook**
(`Notebook.get_context()` → every source's full text + all insights; for a whole book that's
~70k+ tokens), fed into *each* transcript-segment call. On long segments the `<think>` block
eats the 5000-token response budget and the JSON gets truncated → `Invalid json output`
whole job fails. (Confirmed: `/no_think` cut a segment from 98s to 34s and left valid JSON.)
Fix — both handled in the two templates under `prompts/podcast/`, bind-mounted read-only over
the image's copies via `docker-compose.yml`:
- Prepended `/no_think` as the first line of both templates (Qwen honors it anywhere; harmless
text for non-Qwen models) — disables thinking, freeing the token budget and ~3x speedup.
- Added a `{% if language %}CRITICAL LANGUAGE REQUIREMENT … in {{ language }}{% endif %}` block
near the end of both templates, so the already-plumbed `language` field now actually forces the
output language.
The bundled briefings and speaker backstory/personality were **also** translated to German (belt
and suspenders; reduces English drift from the source content), but the template change is what
makes `language` authoritative. **Editing these templates requires a container restart** to take
effect — they're bind-mounted, and the mount is the directory `prompts/podcast/` (not individual
files, which go stale on inode-replacing edits). If you switch off `qwen3.5:27b`, the `/no_think`
line becomes inert but harmless.
If you create your own episode profile, just set `language` — the template handles the rest.
### Text-to-speech / speech-to-text