dschlueter
fb1fbcfabe
Wraps the locally installed chatterbox-tts and faster-whisper packages in thin FastAPI servers implementing OpenAI's audio API shape, since neither Ollama nor OpenRouter support speech. Pinned to GPU 2 to avoid contending with Ollama's resident models on GPU 1. Requires UFW rules for 8901/8902 (same pattern as the existing 11434 rule) since UFW defaults to deny-incoming. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
67 lines
5.5 KiB
Markdown
67 lines
5.5 KiB
Markdown
# CLAUDE.md
|
|
|
|
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
|
|
## What this repo is
|
|
|
|
This is a local Docker Compose deployment of [Open Notebook](https://github.com/lfnovo/open-notebook) (upstream image `lfnovo/open_notebook`), not a clone of its source. There is no application source code here — only deployment config (`docker-compose.yml`, `.env`, `.gitignore`). Application bugs/features belong upstream; this repo only concerns itself with how the stack is run locally.
|
|
|
|
## Commands
|
|
|
|
```bash
|
|
docker compose up -d # start (pulls images per pull_policy: always)
|
|
docker compose down # stop and remove containers (data volumes persist)
|
|
docker compose logs -f open_notebook # follow app logs
|
|
docker compose ps # container status
|
|
```
|
|
|
|
No build/lint/test suite exists in this repo — nothing here to run.
|
|
|
|
## Architecture
|
|
|
|
Two services, defined in `docker-compose.yml`:
|
|
|
|
- **surrealdb** (`surrealdb/surrealdb:v2`) — the database, RocksDB-backed, bound to `127.0.0.1:8000` only (local debugging access, e.g. `surreal sql`). Credentials come from `SURREAL_USER`/`SURREAL_PASSWORD` (default `root`/`root` if unset in `.env`) and are shared with the `open_notebook` service so they stay in sync.
|
|
- **open_notebook** (`lfnovo/open_notebook:v1-latest`) — bundles the web UI (port 8502) and REST API (port 5055) in one image, both bound to `127.0.0.1` only. Connects to `surrealdb` over the internal compose network (`ws://surrealdb:8000/rpc`).
|
|
|
|
Data persists in `./surreal_data` and `./notebook_data` (bind mounts, gitignored).
|
|
|
|
### AI providers
|
|
|
|
This deployment intentionally uses only two AI providers — no OpenAI/Anthropic/Google keys are wired in, even though such keys may exist in the host environment:
|
|
|
|
- **Ollama**, running natively on the host (not containerized) at `0.0.0.0:11434`, restricted to GPU 1+2 (RTX 3090s) via `CUDA_VISIBLE_DEVICES=1,2` in its systemd unit — GPU 0 (a T600 used for the desktop) is intentionally excluded. The `open_notebook` container reaches it via `OLLAMA_BASE_URL=http://host.docker.internal:11434`, which requires the `extra_hosts: host.docker.internal:host-gateway` entry in the compose file (Linux has no Docker-Desktop magic DNS for this).
|
|
- **OpenRouter**, the only cloud provider, configured via `OPENROUTER_API_KEY` (read from the host environment into `.env`).
|
|
|
|
Neither `surrealdb` nor `open_notebook` do local model inference themselves, so no GPU device reservation is needed in the compose file — all GPU usage happens inside the host's Ollama process.
|
|
|
|
A separate local `llama.cpp` server exists on this machine (managed via `~/llamacppctl-server_neu/llamacppctl/`, container name `llama_cpp_server`) but is **not** wired into this stack. It can be added later as an additional OpenAI-compatible provider through the Open Notebook UI (Settings → AI Providers) if needed — note it competes for the same GPU 1/2 VRAM as Ollama, so don't run large models on both simultaneously.
|
|
|
|
### Text-to-speech / speech-to-text
|
|
|
|
Open Notebook's `openai_compatible` provider type talks to any endpoint implementing OpenAI's audio API shape (`POST /audio/speech`, `POST /audio/transcriptions` — see `esperanto.providers.tts.openai_compatible` / `.stt.openai_compatible` inside the app container for the exact contract). Since neither Ollama nor OpenRouter support TTS/STT, two thin wrapper servers in `services/` expose the locally installed tools this way:
|
|
|
|
- `services/tts_server.py` — wraps `chatterbox-tts` (via `~/chatterbox-tts-cli/chatterbox_cli_v4.py`), run with the `chatterbox` conda env (`~/miniforge3/envs/chatterbox/bin/python`).
|
|
- `services/stt_server.py` — wraps `faster-whisper` (`large-v3` model), run with the base `python3`.
|
|
|
|
Both are plain background processes (`nohup ... &`), **not** systemd units — they don't survive a reboot; restart manually if needed:
|
|
|
|
```bash
|
|
cd ~/open-notebook/services
|
|
CUDA_VISIBLE_DEVICES=2 TTS_PORT=8901 nohup ~/miniforge3/envs/chatterbox/bin/python tts_server.py > logs/tts_server.log 2>&1 &
|
|
CUDA_VISIBLE_DEVICES=2 STT_PORT=8902 nohup python3 stt_server.py > logs/stt_server.log 2>&1 &
|
|
```
|
|
|
|
They're pinned to `CUDA_VISIBLE_DEVICES=2` deliberately — GPU 1 tends to fill up with Ollama's resident models (`OLLAMA_KEEP_ALIVE=-1` keeps them loaded), so TTS/STT get GPU 2 to themselves rather than risking an OOM race with Ollama.
|
|
|
|
Reachability from the `open_notebook` container requires **two things**, both already done: the `extra_hosts: host.docker.internal:host-gateway` entry in `docker-compose.yml`, and UFW rules allowing inbound `8901/tcp` and `8902/tcp` (same pattern as the existing `11434/tcp` rule for Ollama — UFW defaults to deny-incoming, so any new host-side port a container needs to reach must be explicitly opened).
|
|
|
|
Registered in Open Notebook as `openai_compatible` credentials with `base_url=http://host.docker.internal:8901` (TTS) / `:8902` (STT), and set as `default_text_to_speech_model` / `default_speech_to_text_model`.
|
|
|
|
## Secrets
|
|
|
|
`.env` (gitignored) holds `OPEN_NOTEBOOK_ENCRYPTION_KEY` (encrypts provider credentials stored in SurrealDB), `SURREAL_PASSWORD`, and `OPENROUTER_API_KEY`. `.env.example` is the committed template. Provider API keys can also be entered directly in the Open Notebook UI after first startup — the project's own docs recommend this over env vars for better security since they get encrypted at rest.
|
|
|
|
## Git
|
|
|
|
This directory is its own independent git repository (`main` branch) even though the parent home directory is also a git repo — this is intentional, not an accident to fix.
|