Fix podcasts generating in English + failing with invalid JSON
Two root causes, both fixed in the bind-mounted podcast prompt templates
(prompts/podcast/*.jinja):
1. Language: the episode profile's `language` field IS passed to both
podcast_creator templates as {{ language }}, but the stock templates never
reference it — so podcasts came out English (driven only by the English
stock briefings/speakers), then read aloud by the German-locked Chatterbox
TTS = "English with a German accent". Added a CRITICAL LANGUAGE REQUIREMENT
block keyed on {{ language }} to both templates; now `language: "de"`
actually forces German. Verified end-to-end: a full run produced a 44-line
all-German transcript + audio.
2. Invalid json output failures: qwen3.5:27b is a thinking model; on long
segments the <think> block ate the response-token budget and truncated the
JSON. Prepended /no_think to both templates (~3x faster, valid JSON).
Templates are bind-mounted read-only via docker-compose (directory mount, so
edits survive a container restart without inode-staleness). Bundled briefings
and speaker backstories were also translated to German to reduce drift.
Known limitation documented: feeding an entire book (~70k tokens) as podcast
content makes each of the 6 LLM calls take ~3.5 min and is unreliable; use a
shorter source or summary. Confirmed working with concise content.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
parent
e2fb113213
commit
054f867a8a
6 changed files with 304 additions and 18 deletions
39
CLAUDE.md
39
CLAUDE.md
|
|
@ -41,13 +41,38 @@ A separate local `llama.cpp` server exists on this machine (managed via `~/llama
|
|||
|
||||
**Chat model context window:** the Ollama credential used for `qwen3.5:27b` (`credential:di8b31l16zikyyb6isyi`) has `num_ctx=98304` set explicitly. Without this, esperanto's `OllamaLanguageModel` silently defaults `num_ctx` to **8192** regardless of the model's real capability or `OLLAMA_CONTEXT_LENGTH` — any chat context (e.g. "full content" mode on a real document) beyond that gets truncated by Ollama, and because `qwen3.5:27b` is a thinking model, the truncated prompt frequently produces an empty final answer (the response is silently swallowed by `clean_thinking_content`) instead of a visible error — looks exactly like "chat gives no answer" from the UI. `98304` is the largest value that still keeps the whole model on GPU 1 (`ollama ps` shows `100% GPU`); `131072` spills ~91% onto CPU and makes a single response take 5-6 minutes instead of ~1-2. Even at 98304, a full-book "Volltext" chat (~70k+ tokens of context) takes ~2-3 minutes — that's inherent to processing that much context on one consumer GPU, not a bug. For large documents, the default short/RAG context mode (only relevant chunks, not the whole document) stays fast; reserve "Volltext" for shorter sources. If `qwen3.5:27b` ever gets swapped for a different Ollama chat model, re-check this credential's `num_ctx` still makes sense for its VRAM footprint.
|
||||
|
||||
**Podcast language:** the three bundled episode profiles (`business_analysis`, `solo_expert`,
|
||||
`tech_discussion`) ship with `language: null`, which the underlying `podcast_creator` library
|
||||
treats as "no language specified" and defaults to English for both outline and transcript
|
||||
generation — regardless of what language the source content or the briefing text is in. All
|
||||
three are set to `language: "de"` here. If you create a new episode profile, set `language`
|
||||
explicitly (`PUT /api/episode-profiles/{id}`, full body) — it doesn't inherit from the notebook
|
||||
or the source.
|
||||
**Podcast language + reliability — fixed via patched prompt templates (`prompts/podcast/`,
|
||||
bind-mounted).** Two separate problems, one fix location:
|
||||
|
||||
1. *Language.* The episode profile's `language` field (`"de"` → resolved to `"German"`) **is**
|
||||
passed into both `podcast_creator` prompt templates as a `{{ language }}` variable
|
||||
(`nodes.py`, both `generate_outline_node` and `generate_transcript_node`), but the stock
|
||||
templates shipped in the image never reference it — so setting `language` had no effect and
|
||||
podcasts came out in English (the stock briefings and speaker backstories are English, which
|
||||
is the only language signal the model then sees). English text read aloud by the German-locked
|
||||
Chatterbox TTS is what produced the "English with a bad German accent" symptom.
|
||||
2. *Reliability.* `qwen3.5:27b` is a thinking model. Podcast content is the **entire notebook**
|
||||
(`Notebook.get_context()` → every source's full text + all insights; for a whole book that's
|
||||
~70k+ tokens), fed into *each* transcript-segment call. On long segments the `<think>` block
|
||||
eats the 5000-token response budget and the JSON gets truncated → `Invalid json output` →
|
||||
whole job fails. (Confirmed: `/no_think` cut a segment from 98s to 34s and left valid JSON.)
|
||||
|
||||
Fix — both handled in the two templates under `prompts/podcast/`, bind-mounted read-only over
|
||||
the image's copies via `docker-compose.yml`:
|
||||
- Prepended `/no_think` as the first line of both templates (Qwen honors it anywhere; harmless
|
||||
text for non-Qwen models) — disables thinking, freeing the token budget and ~3x speedup.
|
||||
- Added a `{% if language %}CRITICAL LANGUAGE REQUIREMENT … in {{ language }}{% endif %}` block
|
||||
near the end of both templates, so the already-plumbed `language` field now actually forces the
|
||||
output language.
|
||||
|
||||
The bundled briefings and speaker backstory/personality were **also** translated to German (belt
|
||||
and suspenders; reduces English drift from the source content), but the template change is what
|
||||
makes `language` authoritative. **Editing these templates requires a container restart** to take
|
||||
effect — they're bind-mounted, and the mount is the directory `prompts/podcast/` (not individual
|
||||
files, which go stale on inode-replacing edits). If you switch off `qwen3.5:27b`, the `/no_think`
|
||||
line becomes inert but harmless.
|
||||
|
||||
If you create your own episode profile, just set `language` — the template handles the rest.
|
||||
|
||||
### Text-to-speech / speech-to-text
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue