diff --git a/CLAUDE.md b/CLAUDE.md index 757fd5c..4174e30 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -35,6 +35,8 @@ This deployment intentionally uses only two AI providers — no OpenAI/Anthropic Neither `surrealdb` nor `open_notebook` do local model inference themselves, so no GPU device reservation is needed in the compose file — all GPU usage happens inside the host's Ollama process. +**Embedding model:** use `nomic-embed-text` (274 MB, 2048 native context), not `qwen3-embedding` (a repurposed 8B-class LLM, ~13GB loaded). With the resident `qwen3.5:27b` chat model already using 17GB of GPU 1's 24GB, `qwen3-embedding` didn't fit and Ollama silently ran it 44-56% on CPU — a single embedding call took 39-54s, so any real document (many chunks, batched) blew past esperanto's 60s embedding timeout (`ESPERANTO_EMBEDDING_TIMEOUT`) and got stuck in an infinite retry loop that looked like "adding the source failed" even though extraction worked fine. Note: the credential's `num_ctx` field (meant to shrink Ollama's context/VRAM footprint) does **not** work for embeddings — esperanto's `OllamaEmbeddingModel` puts it at the top level of the `/api/embed` payload instead of nesting it under `options`, so Ollama silently ignores it. Don't rely on `num_ctx` to fix embedding VRAM contention; use a genuinely small embedding model instead. + A separate local `llama.cpp` server exists on this machine (managed via `~/llamacppctl-server_neu/llamacppctl/`, container name `llama_cpp_server`) but is **not** wired into this stack. It can be added later as an additional OpenAI-compatible provider through the Open Notebook UI (Settings → AI Providers) if needed — note it competes for the same GPU 1/2 VRAM as Ollama, so don't run large models on both simultaneously. ### Text-to-speech / speech-to-text