open_notebook/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

What this repo is

This is a local Docker Compose deployment of Open Notebook (upstream image lfnovo/open_notebook), not a clone of its source. There is no application source code here — only deployment config (docker-compose.yml, .env, .gitignore). Application bugs/features belong upstream; this repo only concerns itself with how the stack is run locally.

Commands

docker compose up -d              # start (pulls images per pull_policy: always)
docker compose down                # stop and remove containers (data volumes persist)
docker compose logs -f open_notebook   # follow app logs
docker compose ps                  # container status

No build/lint/test suite exists in this repo — nothing here to run.

Architecture

Two services, defined in docker-compose.yml:

• surrealdb (surrealdb/surrealdb:v2) — the database, RocksDB-backed, bound to 127.0.0.1:8000 only (local debugging access, e.g. surreal sql). Credentials come from SURREAL_USER/SURREAL_PASSWORD (default root/root if unset in .env) and are shared with the open_notebook service so they stay in sync.
• open_notebook (lfnovo/open_notebook:v1-latest) — bundles the web UI (port 8502) and REST API (port 5055) in one image, both bound to 127.0.0.1 only. Connects to surrealdb over the internal compose network (ws://surrealdb:8000/rpc).

Data persists in ./surreal_data and ./notebook_data (bind mounts, gitignored).

AI providers

This deployment intentionally uses only two AI providers — no OpenAI/Anthropic/Google keys are wired in, even though such keys may exist in the host environment:

• Ollama, running natively on the host (not containerized) at 0.0.0.0:11434, restricted to GPU 1+2 (RTX 3090s) via CUDA_VISIBLE_DEVICES=1,2 in its systemd unit — GPU 0 (a T600 used for the desktop) is intentionally excluded. The open_notebook container reaches it via OLLAMA_BASE_URL=http://host.docker.internal:11434, which requires the extra_hosts: host.docker.internal:host-gateway entry in the compose file (Linux has no Docker-Desktop magic DNS for this).
• OpenRouter, the only cloud provider, configured via OPENROUTER_API_KEY (read from the host environment into .env).

Neither surrealdb nor open_notebook do local model inference themselves, so no GPU device reservation is needed in the compose file — all GPU usage happens inside the host's Ollama process.

Embedding model: use nomic-embed-text (274 MB, 2048 native context), not qwen3-embedding (a repurposed 8B-class LLM, ~13GB loaded). With the resident qwen3.5:27b chat model already using 17GB of GPU 1's 24GB, qwen3-embedding didn't fit and Ollama silently ran it 44-56% on CPU — a single embedding call took 39-54s, so any real document (many chunks, batched) blew past esperanto's 60s embedding timeout (ESPERANTO_EMBEDDING_TIMEOUT) and got stuck in an infinite retry loop that looked like "adding the source failed" even though extraction worked fine. Note: the credential's num_ctx field (meant to shrink Ollama's context/VRAM footprint) does not work for embeddings — esperanto's OllamaEmbeddingModel puts it at the top level of the /api/embed payload instead of nesting it under options, so Ollama silently ignores it. Don't rely on num_ctx to fix embedding VRAM contention; use a genuinely small embedding model instead.

A separate local llama.cpp server exists on this machine (managed via ~/llamacppctl-server_neu/llamacppctl/, container name llama_cpp_server) but is not wired into this stack. It can be added later as an additional OpenAI-compatible provider through the Open Notebook UI (Settings → AI Providers) if needed — note it competes for the same GPU 1/2 VRAM as Ollama, so don't run large models on both simultaneously.

Chat model context window: the Ollama credential used for qwen3.5:27b (credential:di8b31l16zikyyb6isyi) has num_ctx=98304 set explicitly. Without this, esperanto's OllamaLanguageModel silently defaults num_ctx to 8192 regardless of the model's real capability or OLLAMA_CONTEXT_LENGTH — any chat context (e.g. "full content" mode on a real document) beyond that gets truncated by Ollama, and because qwen3.5:27b is a thinking model, the truncated prompt frequently produces an empty final answer (the response is silently swallowed by clean_thinking_content) instead of a visible error — looks exactly like "chat gives no answer" from the UI. 98304 is the largest value that still keeps the whole model on GPU 1 (ollama ps shows 100% GPU); 131072 spills ~91% onto CPU and makes a single response take 5-6 minutes instead of ~1-2. Even at 98304, a full-book "Volltext" chat (~70k+ tokens of context) takes ~2-3 minutes — that's inherent to processing that much context on one consumer GPU, not a bug. For large documents, the default short/RAG context mode (only relevant chunks, not the whole document) stays fast; reserve "Volltext" for shorter sources. If qwen3.5:27b ever gets swapped for a different Ollama chat model, re-check this credential's num_ctx still makes sense for its VRAM footprint.

Podcast language + reliability — fixed via patched prompt templates (prompts/podcast/, bind-mounted). Two separate problems, one fix location:

1. Language. The episode profile's language field ("de" → resolved to "German") is passed into both podcast_creator prompt templates as a {{ language }} variable (nodes.py, both generate_outline_node and generate_transcript_node), but the stock templates shipped in the image never reference it — so setting language had no effect and podcasts came out in English (the stock briefings and speaker backstories are English, which is the only language signal the model then sees). English text read aloud by the German-locked Chatterbox TTS is what produced the "English with a bad German accent" symptom.
2. Reliability. qwen3.5:27b is a thinking model. Podcast content is the entire notebook (Notebook.get_context() → every source's full text + all insights; for a whole book that's ~70k+ tokens), fed into each transcript-segment call. On long segments the <think> block eats the 5000-token response budget and the JSON gets truncated → Invalid json output → whole job fails. (Confirmed: /no_think cut a segment from 98s to 34s and left valid JSON.)

Fix — both handled in the two templates under prompts/podcast/, bind-mounted read-only over the image's copies via docker-compose.yml:

• Prepended /no_think as the first line of both templates (Qwen honors it anywhere; harmless text for non-Qwen models) — disables thinking, freeing the token budget and ~3x speedup.
• Added a {% if language %}CRITICAL LANGUAGE REQUIREMENT … in {{ language }}{% endif %} block near the end of both templates, so the already-plumbed language field now actually forces the output language.

The bundled briefings and speaker backstory/personality were also translated to German (belt and suspenders; reduces English drift from the source content), but the template change is what makes language authoritative. Editing these templates requires a container restart to take effect — they're bind-mounted, and the mount is the directory prompts/podcast/ (not individual files, which go stale on inode-replacing edits). If you switch off qwen3.5:27b, the /no_think line becomes inert but harmless.

If you create your own episode profile, just set language — the template handles the rest.

Deleting an episode leaks its working directory (upstream gap). DELETE /api/podcasts/episodes/{id} (api/routers/podcasts.py) resolves episode.audio_file and calls audio_path.unlink() on it — the final MP3 only. The enclosing data/podcasts/episodes/<uuid>/ directory, holding clips/ (one MP3 per dialogue segment), outline.json and transcript.json, is never touched, and nothing else reaps it. Failed and /retry-replaced runs leak a directory too (the retry writes a new episode row with a new uuid dir; the old one loses its DB entry and just stays). Upstream this would be roughly shutil.rmtree(audio_path.parent.parent) in that handler. We deliberately do not patch it here by bind-mounting a modified router — that would fork app logic into a deployment repo and silently rot against pull_policy: always. Instead: scripts/prune_podcast_data.py (dry-run by default, --yes to apply) reconciles the episode list from the API against the directories on disk and removes orphans plus the clips/ of completed episodes. Two guards matter — it aborts if any job is running/pending (a running job has no audio_file yet, so its directory is indistinguishable from an orphan) and it skips directories touched in the last 60 minutes. Deletion runs inside the container (docker compose exec … rm -rf), because the container writes as root and the host user can't remove those directories.

Text-to-speech / speech-to-text

Open Notebook's openai_compatible provider type talks to any endpoint implementing OpenAI's audio API shape (POST /audio/speech, POST /audio/transcriptions — see esperanto.providers.tts.openai_compatible / .stt.openai_compatible inside the app container for the exact contract). Since neither Ollama nor OpenRouter support TTS/STT, two thin wrapper servers in services/ expose the locally installed tools this way:

• services/tts_server.py — wraps chatterbox-tts (via ~/chatterbox-tts-cli/chatterbox_cli_v4.py), run with the chatterbox conda env (~/miniforge3/envs/chatterbox/bin/python).
• services/stt_server.py — wraps faster-whisper (large-v3 model), run with the base python3.

Both run as systemd user units. The unit files live in the repo (services/systemd/*.service, using %h rather than a hardcoded home) and scripts/start_services.sh installs them into ~/.config/systemd/user/, enables linger and starts them — so they come up on boot without a login session. Re-run that script after editing a unit; it's idempotent. They log to the journal, not to services/logs/ (those files are leftovers from the earlier nohup setup):

systemctl --user status open-notebook-tts open-notebook-stt
systemctl --user restart open-notebook-tts        # after editing tts_server.py or adding voices
journalctl --user -u open-notebook-tts -f

They were previously plain nohup ... & background processes, which silently did not survive a reboot — a podcast then failed at the audio stage with Failed to generate speech: All connection attempts failed (httpx.ConnectError against host.docker.internal:8901) even though outline and transcript had generated fine. That's the signature of a dead TTS server, not a model problem.

Both units pin GPU 2 by UUID (CUDA_VISIBLE_DEVICES=GPU-83ba6d1f-…), not by index: GPU 1 is occupied by Ollama's resident models (OLLAMA_KEEP_ALIVE=-1 keeps them loaded), so TTS/STT go to GPU 2 rather than racing Ollama for VRAM. The index form (CUDA_VISIBLE_DEVICES=2) is not reliable here — CUDA's default device order is "fastest first", not nvidia-smi's PCI order, so index 2 can resolve to the T600. Re-check the UUID with nvidia-smi --query-gpu=index,name,uuid --format=csv if the GPUs are ever reseated.

GPU 2 is shared — TTS concurrency is capped on purpose

GPU 2 does not belong to the TTS server alone. Three processes sit on it: open-notebook-tts (~4 GB idle-loaded), open-notebook-stt (faster-whisper large-v3, ~4 GB), and the unrelated chatterbox-tts.service MCP server on port 9999 (~3.4 GB). That leaves roughly 12 GB of working headroom, and TTS inference must fit inside it.

podcast_creator requests TTS_BATCH_SIZE clips concurrently (asyncio.gather per batch, nodes.py; upstream default 5). tts_server.py's /audio/speech is a sync FastAPI handler, so FastAPI runs each request in a threadpool thread — five clips therefore generate genuinely in parallel on one shared model, and the activation memory multiplies. That drove the TTS process to 16.7 GB and produced torch.OutOfMemoryError inside chatterbox's s3gen.embed_ref, surfacing in the app as Failed to generate speech: OpenAI-compatible TTS endpoint error: HTTP 500. Note this is a different failure from the dead-server ConnectError above — a 500 means the server is alive and rejecting the work, so check the TTS journal for the CUDA OOM before assuming a restart will help.

Two changes keep it bounded, and both matter:

• tts_server.py serializes generation behind _GPU_LOCK (a threading.Lock around model load
  • the chunk loop, plus torch.cuda.empty_cache() afterwards). Generation is GPU-bound, so running clips in parallel buys no throughput on a single card — it only multiplies peak VRAM. This is the hard guarantee: it holds no matter which client calls, and no matter what batch size they use.
• TTS_BATCH_SIZE=1 in docker-compose.yml so the client sends clips one at a time. Without this, the lock still prevents the OOM, but 4 of the 5 batched requests sit queued in it — and with ~40-60 s per clip the last one approaches esperanto's 300 s TTS timeout. ESPERANTO_TTS_TIMEOUT=600 is set alongside it as extra headroom for long segments.

Measured after the fix: 5 concurrent /audio/speech requests all return 200, GPU 2 peaks at ~11.5 GB total (vs. 16.7 GB for the TTS process alone before), i.e. ~12 GB of headroom left. If you add another GPU-resident service to GPU 2, re-check that budget.

Reachability from the open_notebook container requires two things, both already done: the extra_hosts: host.docker.internal:host-gateway entry in docker-compose.yml, and UFW rules allowing inbound 8901/tcp and 8902/tcp (same pattern as the existing 11434/tcp rule for Ollama — UFW defaults to deny-incoming, so any new host-side port a container needs to reach must be explicitly opened).

Registered in Open Notebook as openai_compatible credentials with base_url=http://host.docker.internal:8901 (TTS) / :8902 (STT), and set as default_text_to_speech_model / default_speech_to_text_model.

Voice cloning

tts_server.py resolves the incoming voice field via services/voices/*.wav (gitignored — personal voice recordings): <name>.wav becomes a selectable voice "<name>", cloned via chatterbox's audio_prompt_path. Unknown/omitted voices fall back to "default"; a bare language code (e.g. "en") still works and skips cloning (built-in voice for that language). All cloned voices are generated in German (CLONE_LANG in the script) since only German reference clips exist here — add entries to VOICE_LANG_OVERRIDES in the script if you add clips in other languages.

Three reference clips exist in services/voices/ (gitignored, not committed):

• default.wav — symlink to ~/chatterbox-tts-cli/my_voice_deutsch_60s.wav, the owner's own voice.
• male_thorsten.wav / female_kerstin.wav — synthesized with Piper using the CC0-licensed de_DE-thorsten-medium and de_DE-kerstin-low voice models (downloaded from huggingface.co/rhasspy/piper-voices). Deliberately not real people's recordings scraped from the internet — Piper's voices are explicitly released for this kind of downstream synthesis/cloning use, unlike e.g. Common Voice (consented for ASR training, not cloning) or random web audio (no consent at all). Regenerate with:

  piper --model de_DE-thorsten-medium.onnx --output_file male_thorsten.wav <<< "some German sentence"
  

Podcast speaker profiles are assigned across these three: business_panel (Marcus→male_thorsten, Elena→female_kerstin, Johny→default), tech_experts (Alex→male_thorsten, Jamie→female_kerstin), solo_expert/test_speaker_local (→female_kerstin/Anna). To add another distinct voice, drop a new <name>.wav (~10-30s, clean single-speaker audio, licensing-checked) into services/voices/, restart the TTS server (systemctl --user restart open-notebook-tts — voices are scanned once at startup), and set the speaker's voice_id to <name> via PUT /api/speaker-profiles/{id}.

Secrets

.env (gitignored) holds OPEN_NOTEBOOK_ENCRYPTION_KEY (encrypts provider credentials stored in SurrealDB), SURREAL_PASSWORD, and OPENROUTER_API_KEY. .env.example is the committed template. Provider API keys can also be entered directly in the Open Notebook UI after first startup — the project's own docs recommend this over env vars for better security since they get encrypted at rest.

Git

This directory is its own independent git repository (main branch) even though the parent home directory is also a git repo — this is intentional, not an accident to fix.