dschlueter/open_notebook
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Add voice cloning to the TTS wrapper, default to the owner's own voice sample

Browse source 
tts_server.py now resolves `voice` against services/voices/*.wav (gitignored,
personal recordings) and clones via chatterbox's audio_prompt_path, falling
back to "default" (symlinked to ~/chatterbox-tts-cli/my_voice_deutsch_60s.wav)
for unknown names, or to the old built-in-voice-by-language-code behavior if
`voice` is a recognized language. All podcast speaker profiles now point at
voice_id "default" until distinct per-speaker reference clips are added.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Dieter Schlüter 2026-07-10 22:38:03 +02:00
commit 0d8af23598
3 changed files with 62 additions and 7 deletions
Download patch file Download diff file Expand all files Collapse all files

6
CLAUDE.md
View file

@ -58,6 +58,12 @@ Reachability from the `open_notebook` container requires **two things**, both al
Registered in Open Notebook as `openai_compatible` credentials with `base_url=http://host.docker.internal:8901` (TTS) / `:8902` (STT), and set as `default_text_to_speech_model` / `default_speech_to_text_model`.
#### Voice cloning
`tts_server.py` resolves the incoming `voice` field via `services/voices/*.wav` (gitignored — personal voice recordings): `<name>.wav` becomes a selectable voice `"<name>"`, cloned via chatterbox's `audio_prompt_path`. Unknown/omitted voices fall back to `"default"`; a bare language code (e.g. `"en"`) still works and skips cloning (built-in voice for that language). All cloned voices are generated in German (`CLONE_LANG` in the script) since only German reference clips exist here — add entries to `VOICE_LANG_OVERRIDES` in the script if you add clips in other languages.
`services/voices/default.wav` is a symlink to `~/chatterbox-tts-cli/my_voice_deutsch_60s.wav` (the owner's own voice). All podcast speaker profiles (`business_panel`, `tech_experts`, `solo_expert`, `test_speaker_local`) currently point every speaker at `voice_id: "default"` — there's only one reference clip, so multi-speaker podcasts don't yet sound like distinct people, only the transcript labels differ. To give a speaker their own voice, drop a new `<name>.wav` (~10-30s, clean single-speaker audio) into `services/voices/`, restart `tts_server.py` (voices are scanned once at startup), and set that speaker's `voice_id` to `<name>` via `PUT /api/speaker-profiles/{id}`.
## Secrets
`.env` (gitignored) holds `OPEN_NOTEBOOK_ENCRYPTION_KEY` (encrypts provider credentials stored in SurrealDB), `SURREAL_PASSWORD`, and `OPENROUTER_API_KEY`. `.env.example` is the committed template. Provider API keys can also be entered directly in the Open Notebook UI after first startup — the project's own docs recommend this over env vars for better security since they get encrypted at rest.