dschlueter
631ed26558
qwen3-embedding (repurposed 8B LLM, ~13GB loaded) didn't fit on GPU 1 alongside the resident 17GB chat model, so Ollama silently ran it 44-56% on CPU. A single embed call took 39-54s, so any real document's chunk batches blew past esperanto's 60s embedding timeout and got stuck retrying forever - looked like "adding the source failed" but extraction was fine. Also note: credential num_ctx doesn't work for embeddings (esperanto sends it at the wrong JSON level, Ollama ignores it) - a small dedicated embedding model is the real fix, not context-window tuning. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
83 lines
8.4 KiB
Markdown
83 lines
8.4 KiB
Markdown
# CLAUDE.md
|
|
|
|
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
|
|
## What this repo is
|
|
|
|
This is a local Docker Compose deployment of [Open Notebook](https://github.com/lfnovo/open-notebook) (upstream image `lfnovo/open_notebook`), not a clone of its source. There is no application source code here — only deployment config (`docker-compose.yml`, `.env`, `.gitignore`). Application bugs/features belong upstream; this repo only concerns itself with how the stack is run locally.
|
|
|
|
## Commands
|
|
|
|
```bash
|
|
docker compose up -d # start (pulls images per pull_policy: always)
|
|
docker compose down # stop and remove containers (data volumes persist)
|
|
docker compose logs -f open_notebook # follow app logs
|
|
docker compose ps # container status
|
|
```
|
|
|
|
No build/lint/test suite exists in this repo — nothing here to run.
|
|
|
|
## Architecture
|
|
|
|
Two services, defined in `docker-compose.yml`:
|
|
|
|
- **surrealdb** (`surrealdb/surrealdb:v2`) — the database, RocksDB-backed, bound to `127.0.0.1:8000` only (local debugging access, e.g. `surreal sql`). Credentials come from `SURREAL_USER`/`SURREAL_PASSWORD` (default `root`/`root` if unset in `.env`) and are shared with the `open_notebook` service so they stay in sync.
|
|
- **open_notebook** (`lfnovo/open_notebook:v1-latest`) — bundles the web UI (port 8502) and REST API (port 5055) in one image, both bound to `127.0.0.1` only. Connects to `surrealdb` over the internal compose network (`ws://surrealdb:8000/rpc`).
|
|
|
|
Data persists in `./surreal_data` and `./notebook_data` (bind mounts, gitignored).
|
|
|
|
### AI providers
|
|
|
|
This deployment intentionally uses only two AI providers — no OpenAI/Anthropic/Google keys are wired in, even though such keys may exist in the host environment:
|
|
|
|
- **Ollama**, running natively on the host (not containerized) at `0.0.0.0:11434`, restricted to GPU 1+2 (RTX 3090s) via `CUDA_VISIBLE_DEVICES=1,2` in its systemd unit — GPU 0 (a T600 used for the desktop) is intentionally excluded. The `open_notebook` container reaches it via `OLLAMA_BASE_URL=http://host.docker.internal:11434`, which requires the `extra_hosts: host.docker.internal:host-gateway` entry in the compose file (Linux has no Docker-Desktop magic DNS for this).
|
|
- **OpenRouter**, the only cloud provider, configured via `OPENROUTER_API_KEY` (read from the host environment into `.env`).
|
|
|
|
Neither `surrealdb` nor `open_notebook` do local model inference themselves, so no GPU device reservation is needed in the compose file — all GPU usage happens inside the host's Ollama process.
|
|
|
|
**Embedding model:** use `nomic-embed-text` (274 MB, 2048 native context), not `qwen3-embedding` (a repurposed 8B-class LLM, ~13GB loaded). With the resident `qwen3.5:27b` chat model already using 17GB of GPU 1's 24GB, `qwen3-embedding` didn't fit and Ollama silently ran it 44-56% on CPU — a single embedding call took 39-54s, so any real document (many chunks, batched) blew past esperanto's 60s embedding timeout (`ESPERANTO_EMBEDDING_TIMEOUT`) and got stuck in an infinite retry loop that looked like "adding the source failed" even though extraction worked fine. Note: the credential's `num_ctx` field (meant to shrink Ollama's context/VRAM footprint) does **not** work for embeddings — esperanto's `OllamaEmbeddingModel` puts it at the top level of the `/api/embed` payload instead of nesting it under `options`, so Ollama silently ignores it. Don't rely on `num_ctx` to fix embedding VRAM contention; use a genuinely small embedding model instead.
|
|
|
|
A separate local `llama.cpp` server exists on this machine (managed via `~/llamacppctl-server_neu/llamacppctl/`, container name `llama_cpp_server`) but is **not** wired into this stack. It can be added later as an additional OpenAI-compatible provider through the Open Notebook UI (Settings → AI Providers) if needed — note it competes for the same GPU 1/2 VRAM as Ollama, so don't run large models on both simultaneously.
|
|
|
|
### Text-to-speech / speech-to-text
|
|
|
|
Open Notebook's `openai_compatible` provider type talks to any endpoint implementing OpenAI's audio API shape (`POST /audio/speech`, `POST /audio/transcriptions` — see `esperanto.providers.tts.openai_compatible` / `.stt.openai_compatible` inside the app container for the exact contract). Since neither Ollama nor OpenRouter support TTS/STT, two thin wrapper servers in `services/` expose the locally installed tools this way:
|
|
|
|
- `services/tts_server.py` — wraps `chatterbox-tts` (via `~/chatterbox-tts-cli/chatterbox_cli_v4.py`), run with the `chatterbox` conda env (`~/miniforge3/envs/chatterbox/bin/python`).
|
|
- `services/stt_server.py` — wraps `faster-whisper` (`large-v3` model), run with the base `python3`.
|
|
|
|
Both are plain background processes (`nohup ... &`), **not** systemd units — they don't survive a reboot; restart manually if needed:
|
|
|
|
```bash
|
|
cd ~/open-notebook/services
|
|
CUDA_VISIBLE_DEVICES=2 TTS_PORT=8901 nohup ~/miniforge3/envs/chatterbox/bin/python tts_server.py > logs/tts_server.log 2>&1 &
|
|
CUDA_VISIBLE_DEVICES=2 STT_PORT=8902 nohup python3 stt_server.py > logs/stt_server.log 2>&1 &
|
|
```
|
|
|
|
They're pinned to `CUDA_VISIBLE_DEVICES=2` deliberately — GPU 1 tends to fill up with Ollama's resident models (`OLLAMA_KEEP_ALIVE=-1` keeps them loaded), so TTS/STT get GPU 2 to themselves rather than risking an OOM race with Ollama.
|
|
|
|
Reachability from the `open_notebook` container requires **two things**, both already done: the `extra_hosts: host.docker.internal:host-gateway` entry in `docker-compose.yml`, and UFW rules allowing inbound `8901/tcp` and `8902/tcp` (same pattern as the existing `11434/tcp` rule for Ollama — UFW defaults to deny-incoming, so any new host-side port a container needs to reach must be explicitly opened).
|
|
|
|
Registered in Open Notebook as `openai_compatible` credentials with `base_url=http://host.docker.internal:8901` (TTS) / `:8902` (STT), and set as `default_text_to_speech_model` / `default_speech_to_text_model`.
|
|
|
|
#### Voice cloning
|
|
|
|
`tts_server.py` resolves the incoming `voice` field via `services/voices/*.wav` (gitignored — personal voice recordings): `<name>.wav` becomes a selectable voice `"<name>"`, cloned via chatterbox's `audio_prompt_path`. Unknown/omitted voices fall back to `"default"`; a bare language code (e.g. `"en"`) still works and skips cloning (built-in voice for that language). All cloned voices are generated in German (`CLONE_LANG` in the script) since only German reference clips exist here — add entries to `VOICE_LANG_OVERRIDES` in the script if you add clips in other languages.
|
|
|
|
Three reference clips exist in `services/voices/` (gitignored, not committed):
|
|
|
|
- `default.wav` — symlink to `~/chatterbox-tts-cli/my_voice_deutsch_60s.wav`, the owner's own voice.
|
|
- `male_thorsten.wav` / `female_kerstin.wav` — synthesized with [Piper](https://github.com/rhasspy/piper) using the CC0-licensed `de_DE-thorsten-medium` and `de_DE-kerstin-low` voice models (downloaded from `huggingface.co/rhasspy/piper-voices`). Deliberately **not** real people's recordings scraped from the internet — Piper's voices are explicitly released for this kind of downstream synthesis/cloning use, unlike e.g. Common Voice (consented for ASR training, not cloning) or random web audio (no consent at all). Regenerate with:
|
|
```bash
|
|
piper --model de_DE-thorsten-medium.onnx --output_file male_thorsten.wav <<< "some German sentence"
|
|
```
|
|
|
|
Podcast speaker profiles are assigned across these three: `business_panel` (Marcus→male_thorsten, Elena→female_kerstin, Johny→default), `tech_experts` (Alex→male_thorsten, Jamie→female_kerstin), `solo_expert`/`test_speaker_local` (→female_kerstin/Anna). To add another distinct voice, drop a new `<name>.wav` (~10-30s, clean single-speaker audio, licensing-checked) into `services/voices/`, restart `tts_server.py` (voices are scanned once at startup), and set the speaker's `voice_id` to `<name>` via `PUT /api/speaker-profiles/{id}`.
|
|
|
|
## Secrets
|
|
|
|
`.env` (gitignored) holds `OPEN_NOTEBOOK_ENCRYPTION_KEY` (encrypts provider credentials stored in SurrealDB), `SURREAL_PASSWORD`, and `OPENROUTER_API_KEY`. `.env.example` is the committed template. Provider API keys can also be entered directly in the Open Notebook UI after first startup — the project's own docs recommend this over env vars for better security since they get encrypted at rest.
|
|
|
|
## Git
|
|
|
|
This directory is its own independent git repository (`main` branch) even though the parent home directory is also a git repo — this is intentional, not an accident to fix.
|