dschlueter
2a057d04e1
Adding a German YouTube video created a source with a title but no content — nothing to chat with, nothing to summarise. The failure was silent in the UI; the reason only showed up in the container log as "Failed to get transcript for video <id> after retries: No suitable transcript found". Open Notebook extracts YouTube via content-core, which reads captions with youtube-transcript-api. content_core/processors/youtube.py only looks for the languages listed in preferred_languages, and the built-in default is ["en", "es", "pt"] — no German. All four fallback attempts in _fetch_best_transcript use that same list, so a video carrying only a German transcript raises NoTranscriptFound, get_best_transcript swallows it and returns None, and the source is stored with an empty body. The title still arrives because it is scraped separately, which is what makes this look like a success. The youtube_transcripts.preferred_languages key in the package's own cc_config.yaml appears to be the knob for this, but it is dead code on the default path: load_config() copies only the "extraction" block out of that file, so the key never reaches CONFIG and the hardcoded fallback always wins. The only way to set it is CCORE_CONFIG_PATH. config/content_core.yaml (bind-mounted read-only) therefore sets preferred_languages: ["de", "en", "es", "pt"]. Note CCORE_CONFIG_PATH REPLACES the config wholesale rather than merging it (config.py: return yaml.safe_load(file)). The file is consequently a full dump of the effective default config plus the new key — a minimal override file would silently drop the extraction engines and the model/timeout defaults. Its header comment carries the command that regenerates it from inside the container, for when an image update changes content-core's defaults. Verified end to end on the reported video (hzxiegk9QAg): extraction now yields 21553 characters of German transcript, the source is created through POST /api/sources with that full text, embedding produces 17 chunks in 3.4s, and vector search returns the video as top hit. This only covers caption languages. A video with no captions at all still yields an empty source — there is no download-and-transcribe fallback in the image. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
58 lines
2.5 KiB
YAML
58 lines
2.5 KiB
YAML
services:
|
|
surrealdb:
|
|
image: surrealdb/surrealdb:v2
|
|
restart: always
|
|
pull_policy: always
|
|
command: start --log info --user ${SURREAL_USER:-root} --pass ${SURREAL_PASSWORD:-root} rocksdb:/mydata/mydatabase.db
|
|
user: root # Required for bind mounts on Linux
|
|
environment:
|
|
- SURREAL_EXPERIMENTAL_GRAPHQL=true
|
|
ports:
|
|
- "127.0.0.1:8000:8000"
|
|
volumes:
|
|
- ./surreal_data:/mydata
|
|
|
|
open_notebook:
|
|
image: lfnovo/open_notebook:v1-latest
|
|
restart: always
|
|
pull_policy: always
|
|
depends_on:
|
|
- surrealdb
|
|
ports:
|
|
- "127.0.0.1:8502:8502"
|
|
- "127.0.0.1:5055:5055"
|
|
environment:
|
|
- OPEN_NOTEBOOK_ENCRYPTION_KEY=${OPEN_NOTEBOOK_ENCRYPTION_KEY}
|
|
- SURREAL_URL=ws://surrealdb:8000/rpc
|
|
- SURREAL_USER=${SURREAL_USER:-root}
|
|
- SURREAL_PASSWORD=${SURREAL_PASSWORD:-root}
|
|
- SURREAL_NAMESPACE=open_notebook
|
|
- SURREAL_DATABASE=open_notebook
|
|
- OLLAMA_BASE_URL=http://host.docker.internal:11434
|
|
- OPENROUTER_API_KEY=${OPENROUTER_API_KEY}
|
|
# podcast_creator schickt sonst 5 Audio-Clips gleichzeitig an den TTS-Server.
|
|
# Die laufen dort echt parallel auf einer GPU -> Aktivierungsspeicher
|
|
# vervielfacht sich -> CUDA OOM (GPU 2 teilt sich TTS, STT und den
|
|
# chatterbox-MCP-Dienst). Der TTS-Server serialisiert intern zwar ohnehin,
|
|
# aber bei Batch 5 warten 4 Requests im Lock und laufen ins 300s-Timeout.
|
|
- TTS_BATCH_SIZE=1
|
|
# Kopfraum fuer lange Segmente; Default waere 300s pro Clip.
|
|
- ESPERANTO_TTS_TIMEOUT=600
|
|
# content-core sucht YouTube-Transkripte sonst nur in en/es/pt — deutsche
|
|
# Videos landen dann als LEERE Quelle im Notebook. Siehe config/content_core.yaml.
|
|
- CCORE_CONFIG_PATH=/app/config/content_core.yaml
|
|
extra_hosts:
|
|
- "host.docker.internal:host-gateway"
|
|
volumes:
|
|
- ./notebook_data:/app/data
|
|
# Podcast-Prompt-Vorlagen mit expliziter Sprachanweisung ({{ language }})
|
|
# und abgeschaltetem Thinking-Modus (/no_think), damit Podcasts in der
|
|
# eingestellten Sprache erzeugt werden (statt Englisch) und das JSON bei
|
|
# langen Segmenten nicht durch riesige Reasoning-Blöcke abgeschnitten wird.
|
|
# Verzeichnis-Mount (nicht Einzeldateien), damit Edits ohne Inode-Probleme
|
|
# nach einem Container-Neustart greifen. Der Ordner im Image enthält nur
|
|
# genau diese zwei Vorlagen.
|
|
- ./prompts/podcast:/app/prompts/podcast:ro
|
|
# content-core-Konfiguration (u.a. deutsche YouTube-Transkripte), per
|
|
# CCORE_CONFIG_PATH oben referenziert.
|
|
- ./config:/app/config:ro
|