German TTS mispronounced English terms and foreign names. The fix lives upstream in ~/chatterbox-tts-cli (pronunciation.py + pronunciation/de.json), and services/tts_server.py picks it up with no code change here — worth recording, since nothing in this repo hints that the mechanism exists. Also records why it is respelling and not phonetics: chatterbox is grapheme-based, and the [ipa] token found in its vocabulary was tested and does not speak IPA. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
346 lines
28 KiB
Markdown
346 lines
28 KiB
Markdown
# CLAUDE.md
|
||
|
||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||
|
||
## What this repo is
|
||
|
||
This is a local Docker Compose deployment of [Open Notebook](https://github.com/lfnovo/open-notebook) (upstream image `lfnovo/open_notebook`), not a clone of its source. The application's own code is **not** here and its bugs/features belong upstream — this repo only concerns itself with how the stack is run locally.
|
||
|
||
What *is* here, beyond `docker-compose.yml`/`.env`:
|
||
|
||
- `services/` — our own source: the TTS/STT wrapper servers (Open Notebook has no local TTS/STT provider) plus their systemd units.
|
||
- `scripts/` — setup, maintenance and update tooling (see Commands below).
|
||
- `prompts/podcast/`, `config/content_core.yaml` — **overlays** bind-mounted over files inside the image, to fix upstream behaviour we can't configure otherwise (podcast language/reliability, YouTube caption languages). These are the fragile part: they can silently rot when the image is updated, which is what `scripts/smoke_test.sh` exists to catch.
|
||
|
||
## Commands
|
||
|
||
```bash
|
||
docker compose up -d # start (images are digest-pinned; no implicit upgrade)
|
||
docker compose down # stop and remove containers (data volumes persist)
|
||
docker compose logs -f open_notebook # follow app logs
|
||
docker compose ps # container status
|
||
|
||
./scripts/check_updates.sh # is there a newer release? (reports only, changes nothing)
|
||
./scripts/update_stack.sh # backup -> update -> smoke test -> rollback on failure
|
||
./scripts/smoke_test.sh # verify the stack still does what we rely on
|
||
```
|
||
|
||
`smoke_test.sh` is the closest thing to a test suite here — run it after touching
|
||
`docker-compose.yml`, the bind-mounted overlays, or the host TTS/STT services.
|
||
|
||
## Image versioning
|
||
|
||
Images are **pinned by digest**, and updating is an explicit act — never a side effect of
|
||
starting the stack.
|
||
|
||
The previous setup (`v1-latest` + `pull_policy: always`) was unreliable in *both* directions:
|
||
`docker compose up -d` silently pulled a new application — including an irreversible SurrealDB
|
||
schema migration — while a reboot (`restart: always`) or `docker compose start` kept running the
|
||
**old** image without any pull. So the running version was essentially unpredictable.
|
||
|
||
Note `v1-latest` tracks *releases*, not `main`: the tag lags the repo by weeks. Chasing `main`
|
||
would mean building from source and running unreleased code — explicitly not wanted here.
|
||
|
||
Updating (`scripts/update_stack.sh`): resolve the new digest → stop → back up `surreal_data` +
|
||
`notebook_data` + `docker-compose.yml` into `backups/<ts>/` → rewrite the digest → start → run
|
||
`smoke_test.sh` → **roll back data and compose file if the smoke test fails**. The data backup is
|
||
the point: the app migrates the database on startup, and an older application generally cannot
|
||
read a migrated database, so a bad update would otherwise be a one-way door. Backup/restore runs
|
||
plain `tar` on the host, which works because the containers run as the host user (see *Container
|
||
user* below).
|
||
|
||
Both scripts read the digest from the `lfnovo/open_notebook` line specifically — a naive "first
|
||
sha256 in the file" grep picks up **surrealdb** (it is listed first), which would make the update
|
||
script rewrite the database image instead of the application.
|
||
|
||
## Container user
|
||
|
||
Both services run as the **host user** (`user: "1000:1000"`), not as root, and `surreal_data/` +
|
||
`notebook_data/` are owned by that uid.
|
||
|
||
Default would be root: the `open_notebook` image declares no `USER`, and the compose file used to
|
||
override surrealdb's own non-root user (`65532`) with `user: root` under the (wrong) comment
|
||
"required for bind mounts on Linux". Everything a root container writes into a bind mount is
|
||
root-owned, so the host user could not delete or back up its own podcast data — every cleanup had
|
||
to detour through `docker compose exec`.
|
||
|
||
Two things this needs, and both are load-bearing:
|
||
|
||
- **`HOME=/tmp` for `open_notebook`.** Without it, `HOME` resolves to `/` for a non-root uid, `uv`
|
||
fails to create `/.cache/uv` (permission denied), and api + worker die at startup with exit 2.
|
||
The failure is immediate and loud, but the cause is not obvious from the message.
|
||
- **The data directories must already be owned by that uid.** On a fresh checkout `mkdir -p
|
||
surreal_data notebook_data` is enough. To adopt existing root-owned data without `sudo`:
|
||
```bash
|
||
docker run --rm -u 0 -v "$PWD:/work" -w /work --entrypoint chown \
|
||
lfnovo/open_notebook:v1-latest -R 1000:1000 surreal_data notebook_data
|
||
```
|
||
|
||
Note this is a deviation from what the image expects (it assumes root), so it is exactly the kind of
|
||
thing an update can break. `smoke_test.sh` therefore asserts the container's uid matches the host
|
||
user and that no foreign-owned files exist under the data directories.
|
||
|
||
### Why a smoke test, and not just "does it start"
|
||
|
||
Every local adaptation in this repo leans on upstream internals, and an update can break them
|
||
*silently* — the stack comes up fine and misbehaves later:
|
||
|
||
- the `prompts/podcast/` **directory** mount masks the image's directory: a third template added
|
||
upstream would simply be invisible;
|
||
- `config/content_core.yaml` is a full dump of content-core's defaults (because `CCORE_CONFIG_PATH`
|
||
replaces rather than merges), so new upstream defaults would be frozen out;
|
||
- the env-var workarounds (`TTS_BATCH_SIZE`, `OPENAI_COMPATIBLE_BASE_URL_STT`,
|
||
`ESPERANTO_TTS_TIMEOUT`) depend on how content-core / esperanto / podcast_creator behave today.
|
||
|
||
`smoke_test.sh` therefore checks those assumptions explicitly (template file list vs. the image,
|
||
`{{ language }}` and `/no_think` still present, `language` still plumbed by podcast_creator,
|
||
`TTS_BATCH_SIZE` still read, `preferred_languages` still used, esperanto still honouring
|
||
`OPENAI_COMPATIBLE_BASE_URL_STT`, our config still covering all default keys) on top of the obvious
|
||
end-to-end ones (API, TTS audio, STT, YouTube transcript). `SKIP_NET=1` skips the internet-dependent
|
||
check.
|
||
|
||
## Architecture
|
||
|
||
Two services, defined in `docker-compose.yml`:
|
||
|
||
- **surrealdb** (`surrealdb/surrealdb:v2`) — the database, RocksDB-backed, bound to `127.0.0.1:8000` only (local debugging access, e.g. `surreal sql`). Credentials come from `SURREAL_USER`/`SURREAL_PASSWORD` (default `root`/`root` if unset in `.env`) and are shared with the `open_notebook` service so they stay in sync.
|
||
- **open_notebook** (`lfnovo/open_notebook:v1-latest`, digest-pinned — see *Image versioning*) — bundles the web UI (port 8502) and REST API (port 5055) in one image, both bound to `127.0.0.1` only. Connects to `surrealdb` over the internal compose network (`ws://surrealdb:8000/rpc`).
|
||
|
||
Data persists in `./surreal_data` and `./notebook_data` (bind mounts, gitignored).
|
||
|
||
### AI providers
|
||
|
||
This deployment intentionally uses only two AI providers — no OpenAI/Anthropic/Google keys are wired in, even though such keys may exist in the host environment:
|
||
|
||
- **Ollama**, running natively on the host (not containerized) at `0.0.0.0:11434`, restricted to GPU 1+2 (RTX 3090s) via `CUDA_VISIBLE_DEVICES=1,2` in its systemd unit — GPU 0 (a T600 used for the desktop) is intentionally excluded. The `open_notebook` container reaches it via `OLLAMA_BASE_URL=http://host.docker.internal:11434`, which requires the `extra_hosts: host.docker.internal:host-gateway` entry in the compose file (Linux has no Docker-Desktop magic DNS for this).
|
||
- **OpenRouter**, the only cloud provider, configured via `OPENROUTER_API_KEY` (read from the host environment into `.env`).
|
||
|
||
Neither `surrealdb` nor `open_notebook` do local model inference themselves, so no GPU device reservation is needed in the compose file — all GPU usage happens inside the host's Ollama process.
|
||
|
||
**Embedding model:** use `nomic-embed-text` (274 MB, 2048 native context), not `qwen3-embedding` (a repurposed 8B-class LLM, ~13GB loaded). With the resident `qwen3.5:27b` chat model already using 17GB of GPU 1's 24GB, `qwen3-embedding` didn't fit and Ollama silently ran it 44-56% on CPU — a single embedding call took 39-54s, so any real document (many chunks, batched) blew past esperanto's 60s embedding timeout (`ESPERANTO_EMBEDDING_TIMEOUT`) and got stuck in an infinite retry loop that looked like "adding the source failed" even though extraction worked fine. Note: the credential's `num_ctx` field (meant to shrink Ollama's context/VRAM footprint) does **not** work for embeddings — esperanto's `OllamaEmbeddingModel` puts it at the top level of the `/api/embed` payload instead of nesting it under `options`, so Ollama silently ignores it. Don't rely on `num_ctx` to fix embedding VRAM contention; use a genuinely small embedding model instead.
|
||
|
||
A separate local `llama.cpp` server exists on this machine (managed via `~/llamacppctl-server_neu/llamacppctl/`, container name `llama_cpp_server`) but is **not** wired into this stack. It can be added later as an additional OpenAI-compatible provider through the Open Notebook UI (Settings → AI Providers) if needed — note it competes for the same GPU 1/2 VRAM as Ollama, so don't run large models on both simultaneously.
|
||
|
||
**Chat model context window:** the Ollama credential used for `qwen3.5:27b` (`credential:di8b31l16zikyyb6isyi`) has `num_ctx=98304` set explicitly. Without this, esperanto's `OllamaLanguageModel` silently defaults `num_ctx` to **8192** regardless of the model's real capability or `OLLAMA_CONTEXT_LENGTH` — any chat context (e.g. "full content" mode on a real document) beyond that gets truncated by Ollama, and because `qwen3.5:27b` is a thinking model, the truncated prompt frequently produces an empty final answer (the response is silently swallowed by `clean_thinking_content`) instead of a visible error — looks exactly like "chat gives no answer" from the UI. `98304` is the largest value that still keeps the whole model on GPU 1 (`ollama ps` shows `100% GPU`); `131072` spills ~91% onto CPU and makes a single response take 5-6 minutes instead of ~1-2. Even at 98304, a full-book "Volltext" chat (~70k+ tokens of context) takes ~2-3 minutes — that's inherent to processing that much context on one consumer GPU, not a bug. For large documents, the default short/RAG context mode (only relevant chunks, not the whole document) stays fast; reserve "Volltext" for shorter sources. If `qwen3.5:27b` ever gets swapped for a different Ollama chat model, re-check this credential's `num_ctx` still makes sense for its VRAM footprint.
|
||
|
||
**Podcast language + reliability — fixed via patched prompt templates (`prompts/podcast/`,
|
||
bind-mounted).** Two separate problems, one fix location:
|
||
|
||
1. *Language.* The episode profile's `language` field (`"de"` → resolved to `"German"`) **is**
|
||
passed into both `podcast_creator` prompt templates as a `{{ language }}` variable
|
||
(`nodes.py`, both `generate_outline_node` and `generate_transcript_node`), but the stock
|
||
templates shipped in the image never reference it — so setting `language` had no effect and
|
||
podcasts came out in English (the stock briefings and speaker backstories are English, which
|
||
is the only language signal the model then sees). English text read aloud by the German-locked
|
||
Chatterbox TTS is what produced the "English with a bad German accent" symptom.
|
||
2. *Reliability.* `qwen3.5:27b` is a thinking model. Podcast content is the **entire notebook**
|
||
(`Notebook.get_context()` → every source's full text + all insights; for a whole book that's
|
||
~70k+ tokens), fed into *each* transcript-segment call. On long segments the `<think>` block
|
||
eats the 5000-token response budget and the JSON gets truncated → `Invalid json output` →
|
||
whole job fails. (Confirmed: `/no_think` cut a segment from 98s to 34s and left valid JSON.)
|
||
|
||
Fix — both handled in the two templates under `prompts/podcast/`, bind-mounted read-only over
|
||
the image's copies via `docker-compose.yml`:
|
||
- Prepended `/no_think` as the first line of both templates (Qwen honors it anywhere; harmless
|
||
text for non-Qwen models) — disables thinking, freeing the token budget and ~3x speedup.
|
||
- Added a `{% if language %}CRITICAL LANGUAGE REQUIREMENT … in {{ language }}{% endif %}` block
|
||
near the end of both templates, so the already-plumbed `language` field now actually forces the
|
||
output language.
|
||
|
||
The bundled briefings and speaker backstory/personality were **also** translated to German (belt
|
||
and suspenders; reduces English drift from the source content), but the template change is what
|
||
makes `language` authoritative. **Editing these templates requires a container restart** to take
|
||
effect — they're bind-mounted, and the mount is the directory `prompts/podcast/` (not individual
|
||
files, which go stale on inode-replacing edits). If you switch off `qwen3.5:27b`, the `/no_think`
|
||
line becomes inert but harmless.
|
||
|
||
If you create your own episode profile, just set `language` — the template handles the rest.
|
||
|
||
**Deleting an episode leaks its working directory (upstream gap).** `DELETE /api/podcasts/episodes/{id}`
|
||
(`api/routers/podcasts.py`) resolves `episode.audio_file` and calls `audio_path.unlink()` on it — the
|
||
final MP3 only. The enclosing `data/podcasts/episodes/<uuid>/` directory, holding `clips/` (one MP3 per
|
||
dialogue segment), `outline.json` and `transcript.json`, is never touched, and nothing else reaps it.
|
||
Failed and `/retry`-replaced runs leak a directory too (the retry writes a *new* episode row with a new
|
||
uuid dir; the old one loses its DB entry and just stays). Upstream this would be roughly
|
||
`shutil.rmtree(audio_path.parent.parent)` in that handler. We deliberately do **not** patch it here by
|
||
bind-mounting a modified router — that would fork app logic into a deployment repo and rot against
|
||
image updates (the same trap the existing overlays already carry). Instead:
|
||
`scripts/prune_podcast_data.py` (dry-run by default, `--yes` to
|
||
apply) reconciles the episode list from the API against the directories on disk and removes orphans plus
|
||
the `clips/` of completed episodes. Two guards matter — it aborts if any job is `running`/`pending`
|
||
(a running job has no `audio_file` yet, so its directory is indistinguishable from an orphan) and it
|
||
skips directories touched in the last 60 minutes. Deletion is a plain `shutil.rmtree` on the host —
|
||
possible because the containers run as the host user (see *Container user*); as root-owned data it
|
||
would need a container detour.
|
||
|
||
### YouTube sources — German transcripts need `CCORE_CONFIG_PATH`
|
||
|
||
Open Notebook extracts YouTube via `content-core`, which pulls captions with `youtube-transcript-api`
|
||
(no `yt-dlp`/`pytube` in the image). `content_core/processors/youtube.py` only ever looks for the
|
||
languages in `preferred_languages`, and the built-in default is `["en", "es", "pt"]` — **no German**.
|
||
All four fallback attempts in `_fetch_best_transcript` use that same list, so a video with only a
|
||
German transcript raises `NoTranscriptFound`, `get_best_transcript` swallows it and returns `None`,
|
||
and the source is created with an **empty body**. The title is still fetched (separate HTTP scrape),
|
||
so the failure is quiet: a source that looks fine but has no content, with the real reason only in
|
||
the container log (`Failed to get transcript for video <id> after retries`).
|
||
|
||
The `youtube_transcripts.preferred_languages` key in the package's `cc_config.yaml` looks like the
|
||
fix, but it is **dead code** on the default path: `load_config()` copies only the `extraction` block
|
||
out of `cc_config.yaml`, so the key never reaches `CONFIG` and the hardcoded `["en","es","pt"]`
|
||
fallback always wins. The only way to set it is `CCORE_CONFIG_PATH`.
|
||
|
||
Fix: `config/content_core.yaml` (bind-mounted read-only, `CCORE_CONFIG_PATH=/app/config/content_core.yaml`
|
||
in `docker-compose.yml`) sets `preferred_languages: ["de", "en", "es", "pt"]`.
|
||
|
||
**`CCORE_CONFIG_PATH` replaces the config wholesale — it does not merge** (`config.py`:
|
||
`return yaml.safe_load(file)`). So that file is a full dump of the effective default config plus the
|
||
`youtube_transcripts` key, not a small override; a minimal file would silently drop `extraction`
|
||
(engine selection) and the model/timeout defaults. Its header comment carries the one-liner that
|
||
regenerates it from inside the container — do that after an image update if content-core's defaults
|
||
change. The `openai`/`gpt-4o-mini` entries in it are content-core's internal defaults and are unused
|
||
here (Open Notebook picks its own models); they're only present because the file must be complete.
|
||
|
||
Note this only fixes *caption* languages. A video with **no** captions is handled by the audio
|
||
fallback below.
|
||
|
||
### Video without captions — audio fallback (`scripts/add_video_source.py`)
|
||
|
||
`yt-dlp` exists on the **host** only (`~/miniforge3/bin/yt-dlp`), not in the image, so the chain
|
||
can't live inside the app. `scripts/add_video_source.py` bridges it:
|
||
|
||
- captions available → hands the URL to Open Notebook as a `link` source (fast, no download);
|
||
- no captions → `yt-dlp -x` pulls just the audio track (mono 16 kHz mp3, ~6 MB per 17 min) and
|
||
uploads it as an `upload` source. Open Notebook then transcribes it **itself** via its configured
|
||
speech-to-text model — i.e. the local faster-whisper server — and embeds it. We deliberately do
|
||
not transcribe in the script: the app's own pipeline already does STT, chunking and embedding.
|
||
|
||
Caption availability is probed with `youtube-transcript-api` **inside the container**, not with
|
||
`yt-dlp`. `yt-dlp`'s `automatic_captions` lists YouTube's ~100 auto-*translation* targets (German
|
||
is always among them), which `youtube-transcript-api` does not accept as transcripts — trusting it
|
||
would send a Japanese-only video down the caption path and produce an empty source again.
|
||
|
||
**`OPENAI_COMPATIBLE_BASE_URL_STT` is what makes the upload path work at all.** Open Notebook passes
|
||
its STT model (`openai_compatible`/`faster-whisper-large-v3`) into content-core as
|
||
`audio_provider`/`audio_model` (`open_notebook/graphs/source.py`), but content-core builds it with
|
||
`AIFactory.create_speech_to_text(provider, model, {'timeout': …})` — **no `base_url`**
|
||
(`content_core/processors/audio.py`). Esperanto then can't locate the local server, content-core
|
||
silently falls back to its default (`openai`/`whisper-1`) and the job dies with *"OpenAI API key not
|
||
found"* — which reads like a missing key but is really a missing URL. The env var supplies it. Do
|
||
**not** "fix" this by putting an `OPENAI_API_KEY` into the container: that would send audio to OpenAI
|
||
(paid, off-machine) while a working local Whisper sits idle on GPU 2, and it contradicts the
|
||
two-provider rule above.
|
||
|
||
The script uploads with `delete_source=true` and names the file after the video id, so
|
||
`notebook_data/uploads/` doesn't accumulate a few MB per video and a second video can't collide with
|
||
the previous `audio.mp3`. Known gap: the `upload` path stores the audio file as the source's asset,
|
||
so the original video URL is not recorded on the source (the `link` path does record it).
|
||
|
||
### Text-to-speech / speech-to-text
|
||
|
||
Open Notebook's `openai_compatible` provider type talks to any endpoint implementing OpenAI's audio API shape (`POST /audio/speech`, `POST /audio/transcriptions` — see `esperanto.providers.tts.openai_compatible` / `.stt.openai_compatible` inside the app container for the exact contract). Since neither Ollama nor OpenRouter support TTS/STT, two thin wrapper servers in `services/` expose the locally installed tools this way:
|
||
|
||
- `services/tts_server.py` — wraps `chatterbox-tts` (via `~/chatterbox-tts-cli/chatterbox_cli_v4.py`), run with the `chatterbox` conda env (`~/miniforge3/envs/chatterbox/bin/python`).
|
||
- `services/stt_server.py` — wraps `faster-whisper` (`large-v3` model), run with the base `python3`.
|
||
|
||
Both run as **systemd user units**. The unit files live in the repo (`services/systemd/*.service`,
|
||
using `%h` rather than a hardcoded home) and `scripts/start_services.sh` installs them into
|
||
`~/.config/systemd/user/`, enables `linger` and starts them — so they come up on boot without a
|
||
login session. Re-run that script after editing a unit; it's idempotent. They log to the journal,
|
||
not to `services/logs/` (those files are leftovers from the earlier `nohup` setup):
|
||
|
||
```bash
|
||
systemctl --user status open-notebook-tts open-notebook-stt
|
||
systemctl --user restart open-notebook-tts # after editing tts_server.py or adding voices
|
||
journalctl --user -u open-notebook-tts -f
|
||
```
|
||
|
||
They were previously plain `nohup ... &` background processes, which silently did not survive a
|
||
reboot — a podcast then failed at the audio stage with `Failed to generate speech: All connection
|
||
attempts failed` (`httpx.ConnectError` against `host.docker.internal:8901`) even though outline
|
||
and transcript had generated fine. That's the signature of a dead TTS server, not a model problem.
|
||
|
||
Both units pin GPU 2 **by UUID** (`CUDA_VISIBLE_DEVICES=GPU-83ba6d1f-…`), not by index: GPU 1 is
|
||
occupied by Ollama's resident models (`OLLAMA_KEEP_ALIVE=-1` keeps them loaded), so TTS/STT go to
|
||
GPU 2 rather than racing Ollama for VRAM. The index form (`CUDA_VISIBLE_DEVICES=2`) is **not**
|
||
reliable here — CUDA's default device order is "fastest first", not nvidia-smi's PCI order, so
|
||
index 2 can resolve to the T600. Re-check the UUID with
|
||
`nvidia-smi --query-gpu=index,name,uuid --format=csv` if the GPUs are ever reseated.
|
||
|
||
#### GPU 2 is shared — TTS concurrency is capped on purpose
|
||
|
||
GPU 2 does **not** belong to the TTS server alone. Three processes sit on it: `open-notebook-tts`
|
||
(~4 GB idle-loaded), `open-notebook-stt` (faster-whisper `large-v3`, ~4 GB), and the unrelated
|
||
`chatterbox-tts.service` MCP server on port 9999 (~3.4 GB). That leaves roughly 12 GB of working
|
||
headroom, and TTS inference must fit inside it.
|
||
|
||
`podcast_creator` requests `TTS_BATCH_SIZE` clips **concurrently** (`asyncio.gather` per batch,
|
||
`nodes.py`; upstream default **5**). `tts_server.py`'s `/audio/speech` is a *sync* FastAPI handler,
|
||
so FastAPI runs each request in a threadpool thread — five clips therefore generate genuinely in
|
||
parallel on one shared model, and the activation memory multiplies. That drove the TTS process to
|
||
**16.7 GB** and produced `torch.OutOfMemoryError` inside chatterbox's `s3gen.embed_ref`, surfacing
|
||
in the app as `Failed to generate speech: OpenAI-compatible TTS endpoint error: HTTP 500`. Note
|
||
this is a *different* failure from the dead-server `ConnectError` above — a 500 means the server is
|
||
alive and rejecting the work, so check the TTS journal for the CUDA OOM before assuming a restart
|
||
will help.
|
||
|
||
Two changes keep it bounded, and both matter:
|
||
|
||
- **`tts_server.py` serializes generation behind `_GPU_LOCK`** (a `threading.Lock` around model load
|
||
+ the chunk loop, plus `torch.cuda.empty_cache()` afterwards). Generation is GPU-bound, so running
|
||
clips in parallel buys no throughput on a single card — it only multiplies peak VRAM. This is the
|
||
hard guarantee: it holds no matter which client calls, and no matter what batch size they use.
|
||
- **`TTS_BATCH_SIZE=1` in `docker-compose.yml`** so the client sends clips one at a time. Without
|
||
this, the lock still prevents the OOM, but 4 of the 5 batched requests sit queued in it — and with
|
||
~40-60 s per clip the last one approaches esperanto's **300 s** TTS timeout. `ESPERANTO_TTS_TIMEOUT=600`
|
||
is set alongside it as extra headroom for long segments.
|
||
|
||
Measured after the fix: 5 concurrent `/audio/speech` requests all return 200, GPU 2 peaks at
|
||
~11.5 GB total (vs. 16.7 GB for the TTS process alone before), i.e. ~12 GB of headroom left.
|
||
If you add another GPU-resident service to GPU 2, re-check that budget.
|
||
|
||
Reachability from the `open_notebook` container requires **two things**, both already done: the `extra_hosts: host.docker.internal:host-gateway` entry in `docker-compose.yml`, and UFW rules allowing inbound `8901/tcp` and `8902/tcp` (same pattern as the existing `11434/tcp` rule for Ollama — UFW defaults to deny-incoming, so any new host-side port a container needs to reach must be explicitly opened).
|
||
|
||
Registered in Open Notebook as `openai_compatible` credentials with `base_url=http://host.docker.internal:8901` (TTS) / `:8902` (STT), and set as `default_text_to_speech_model` / `default_speech_to_text_model`.
|
||
|
||
#### Pronunciation (English terms, proper names)
|
||
|
||
German TTS mispronounces English technical terms and foreign names ("Repository", "Donald Trump").
|
||
The fix lives **upstream in `~/chatterbox-tts-cli/`**, not here: `pronunciation.py` (a torch-free
|
||
module) plus the curated `pronunciation/de.json`, which `chatterbox_cli_v4.preprocess_tts_text()`
|
||
applies automatically for `lang="de"`. `services/tts_server.py` needs **no code** for this — it
|
||
already calls that function, so podcasts get the corrections for free.
|
||
|
||
The mechanism is **respelling in German orthography** ("Repository" → "Ripositori"), not phonetics.
|
||
Chatterbox is grapheme-based (tokenizer: `grapheme_mtl_merged_expanded_v1`); its vocabulary does
|
||
contain IPA symbols and an `[ipa]` language token, but that was tested and does **not** work —
|
||
`ɹɪˈpɑzɪˌtɔɹi` comes out audibly as "Ripacitari". Splitting the sentence and switching to `[en]`
|
||
per term works phonetically but glues segments together without crossfade: 4 seams cost 0.5–0.8 s
|
||
of silence each, growing a 3.9 s sentence to 7.1 s. Respelling is the only usable route.
|
||
|
||
To add project-specific terms without touching the shared file, point
|
||
`CHATTERBOX_PRONUNCIATION_DICT` at a JSON file (it is read by `load_pronunciation_dict()` and
|
||
overrides the shipped list). Find candidates in a real transcript with
|
||
`~/chatterbox-tts-cli/pronunciation_candidates.py` (lists words hunspell's German dictionary
|
||
doesn't know and that aren't in the dictionary yet).
|
||
|
||
#### Voice cloning
|
||
|
||
`tts_server.py` resolves the incoming `voice` field via `services/voices/*.wav` (gitignored — personal voice recordings): `<name>.wav` becomes a selectable voice `"<name>"`, cloned via chatterbox's `audio_prompt_path`. Unknown/omitted voices fall back to `"default"`; a bare language code (e.g. `"en"`) still works and skips cloning (built-in voice for that language). All cloned voices are generated in German (`CLONE_LANG` in the script) since only German reference clips exist here — add entries to `VOICE_LANG_OVERRIDES` in the script if you add clips in other languages.
|
||
|
||
Three reference clips exist in `services/voices/` (gitignored, not committed):
|
||
|
||
- `default.wav` — symlink to `~/chatterbox-tts-cli/my_voice_deutsch_60s.wav`, the owner's own voice.
|
||
- `male_thorsten.wav` / `female_kerstin.wav` — synthesized with [Piper](https://github.com/rhasspy/piper) using the CC0-licensed `de_DE-thorsten-medium` and `de_DE-kerstin-low` voice models (downloaded from `huggingface.co/rhasspy/piper-voices`). Deliberately **not** real people's recordings scraped from the internet — Piper's voices are explicitly released for this kind of downstream synthesis/cloning use, unlike e.g. Common Voice (consented for ASR training, not cloning) or random web audio (no consent at all). Regenerate with:
|
||
```bash
|
||
piper --model de_DE-thorsten-medium.onnx --output_file male_thorsten.wav <<< "some German sentence"
|
||
```
|
||
|
||
Podcast speaker profiles are assigned across these three: `business_panel` (Marcus→male_thorsten, Elena→female_kerstin, Johny→default), `tech_experts` (Alex→male_thorsten, Jamie→female_kerstin), `solo_expert`/`test_speaker_local` (→female_kerstin/Anna). To add another distinct voice, drop a new `<name>.wav` (~10-30s, clean single-speaker audio, licensing-checked) into `services/voices/`, restart the TTS server (`systemctl --user restart open-notebook-tts` — voices are scanned once at startup), and set the speaker's `voice_id` to `<name>` via `PUT /api/speaker-profiles/{id}`.
|
||
|
||
## Secrets
|
||
|
||
`.env` (gitignored) holds `OPEN_NOTEBOOK_ENCRYPTION_KEY` (encrypts provider credentials stored in SurrealDB), `SURREAL_PASSWORD`, and `OPENROUTER_API_KEY`. `.env.example` is the committed template. Provider API keys can also be entered directly in the Open Notebook UI after first startup — the project's own docs recommend this over env vars for better security since they get encrypted at rest.
|
||
|
||
## Git
|
||
|
||
This directory is its own independent git repository (`main` branch) even though the parent home directory is also a git repo — this is intentional, not an accident to fix.
|