Pin images by digest; add update, smoke-test and status scripts
Goal: at every session start, run exactly the application the repo says — and be able to move to a new upstream release deliberately, with a way back. The old setup (v1-latest + pull_policy: always) was unreliable in both directions. `docker compose up -d` silently pulled a new application, including an irreversible SurrealDB schema migration, while a reboot (restart: always) or `docker compose start` kept running the old image without pulling anything. The running version was effectively unpredictable. Note v1-latest tracks releases, not main: the current image (v1.10.0, 2026-06-18) IS the latest release — the repo being "ahead" is unreleased code, which is explicitly not wanted here. So this is about guaranteeing and detecting, not catching up. - docker-compose.yml: both images pinned by digest, pull_policy: missing. - scripts/check_updates.sh: reports running version/digest vs the latest release and registry digest. Changes nothing; meant for the session-start ritual. - scripts/update_stack.sh: resolve new digest -> stop -> back up surreal_data, notebook_data and docker-compose.yml -> repin -> start -> smoke test -> roll back data AND compose file if the smoke test fails. The data backup is the point: the app migrates the DB on startup and an older app cannot read a migrated DB, so a bad update would otherwise be a one-way door. tar runs inside a container because the data dirs are root-owned and the host user cannot restore over them. - scripts/smoke_test.sh: checks what actually breaks here, not just "does it start". Every local adaptation leans on upstream internals and can break silently: the prompts/podcast directory mount masks the image's directory (a new template upstream would be invisible), config/content_core.yaml freezes content-core's defaults (because CCORE_CONFIG_PATH replaces rather than merges), and the env-var workarounds depend on current content-core/esperanto/podcast_creator behaviour. So it verifies those assumptions explicitly, plus API, TTS audio, STT and a real YouTube transcript. Both scripts read the digest from the lfnovo/open_notebook line specifically. A naive "first sha256 in the file" grep matches surrealdb (listed first) — caught while testing: check_updates.sh falsely reported an update, and update_stack.sh would have rewritten the database image instead of the application. Verified: smoke test green against the current stack, and red (exit 1) when failures are injected (dead TTS port, removed template variable). Backup/restore mechanics exercised separately: 44 MB in 3.3s, restore yields 13 DB files and 91 notebook files. The update path itself cannot be exercised end to end until a newer image exists. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
f350e49bd5
commit
4805fe75ed
7 changed files with 434 additions and 7 deletions
3
.gitignore
vendored
3
.gitignore
vendored
|
|
@ -10,3 +10,6 @@ __pycache__/
|
|||
|
||||
# Artefakt versehentlicher Shell-Redirects
|
||||
/3
|
||||
|
||||
# Backups vor Image-Updates (scripts/update_stack.sh)
|
||||
backups/
|
||||
|
|
|
|||
53
CLAUDE.md
53
CLAUDE.md
|
|
@ -9,13 +9,62 @@ This is a local Docker Compose deployment of [Open Notebook](https://github.com/
|
|||
## Commands
|
||||
|
||||
```bash
|
||||
docker compose up -d # start (pulls images per pull_policy: always)
|
||||
docker compose up -d # start (images are digest-pinned; no implicit upgrade)
|
||||
docker compose down # stop and remove containers (data volumes persist)
|
||||
docker compose logs -f open_notebook # follow app logs
|
||||
docker compose ps # container status
|
||||
|
||||
./scripts/check_updates.sh # is there a newer release? (reports only, changes nothing)
|
||||
./scripts/update_stack.sh # backup -> update -> smoke test -> rollback on failure
|
||||
./scripts/smoke_test.sh # verify the stack still does what we rely on
|
||||
```
|
||||
|
||||
No build/lint/test suite exists in this repo — nothing here to run.
|
||||
`smoke_test.sh` is the closest thing to a test suite here — run it after touching
|
||||
`docker-compose.yml`, the bind-mounted overlays, or the host TTS/STT services.
|
||||
|
||||
## Image versioning
|
||||
|
||||
Images are **pinned by digest**, and updating is an explicit act — never a side effect of
|
||||
starting the stack.
|
||||
|
||||
The previous setup (`v1-latest` + `pull_policy: always`) was unreliable in *both* directions:
|
||||
`docker compose up -d` silently pulled a new application — including an irreversible SurrealDB
|
||||
schema migration — while a reboot (`restart: always`) or `docker compose start` kept running the
|
||||
**old** image without any pull. So the running version was essentially unpredictable.
|
||||
|
||||
Note `v1-latest` tracks *releases*, not `main`: the tag lags the repo by weeks. Chasing `main`
|
||||
would mean building from source and running unreleased code — explicitly not wanted here.
|
||||
|
||||
Updating (`scripts/update_stack.sh`): resolve the new digest → stop → back up `surreal_data` +
|
||||
`notebook_data` + `docker-compose.yml` into `backups/<ts>/` → rewrite the digest → start → run
|
||||
`smoke_test.sh` → **roll back data and compose file if the smoke test fails**. The data backup is
|
||||
the point: the app migrates the database on startup, and an older application generally cannot
|
||||
read a migrated database, so a bad update would otherwise be a one-way door. Backup/restore runs
|
||||
`tar` **inside a container** because the data directories are root-owned (the container writes as
|
||||
root) and the host user cannot restore over them.
|
||||
|
||||
Both scripts read the digest from the `lfnovo/open_notebook` line specifically — a naive "first
|
||||
sha256 in the file" grep picks up **surrealdb** (it is listed first), which would make the update
|
||||
script rewrite the database image instead of the application.
|
||||
|
||||
### Why a smoke test, and not just "does it start"
|
||||
|
||||
Every local adaptation in this repo leans on upstream internals, and an update can break them
|
||||
*silently* — the stack comes up fine and misbehaves later:
|
||||
|
||||
- the `prompts/podcast/` **directory** mount masks the image's directory: a third template added
|
||||
upstream would simply be invisible;
|
||||
- `config/content_core.yaml` is a full dump of content-core's defaults (because `CCORE_CONFIG_PATH`
|
||||
replaces rather than merges), so new upstream defaults would be frozen out;
|
||||
- the env-var workarounds (`TTS_BATCH_SIZE`, `OPENAI_COMPATIBLE_BASE_URL_STT`,
|
||||
`ESPERANTO_TTS_TIMEOUT`) depend on how content-core / esperanto / podcast_creator behave today.
|
||||
|
||||
`smoke_test.sh` therefore checks those assumptions explicitly (template file list vs. the image,
|
||||
`{{ language }}` and `/no_think` still present, `language` still plumbed by podcast_creator,
|
||||
`TTS_BATCH_SIZE` still read, `preferred_languages` still used, esperanto still honouring
|
||||
`OPENAI_COMPATIBLE_BASE_URL_STT`, our config still covering all default keys) on top of the obvious
|
||||
end-to-end ones (API, TTS audio, STT, YouTube transcript). `SKIP_NET=1` skips the internet-dependent
|
||||
check.
|
||||
|
||||
## Architecture
|
||||
|
||||
|
|
|
|||
40
README.md
40
README.md
|
|
@ -183,6 +183,10 @@ journalctl --user -u open-notebook-tts -f # TTS-Logs
|
|||
./scripts/prune_podcast_data.py --yes # ... und löschen
|
||||
|
||||
./scripts/add_video_source.py <url> # Video als Quelle (auch ohne Untertitel)
|
||||
|
||||
./scripts/check_updates.sh # gibt es ein neues Release? (ändert nichts)
|
||||
./scripts/update_stack.sh # Update mit Backup, Rauchtest und Rollback
|
||||
./scripts/smoke_test.sh # tut der Stack noch, worauf wir uns verlassen?
|
||||
```
|
||||
|
||||
TTS/STT starten nach einem Reboot von selbst — `start_services.sh` ist nur für die
|
||||
|
|
@ -190,6 +194,35 @@ Ersteinrichtung bzw. nach Änderungen an den Unit-Dateien nötig.
|
|||
|
||||
---
|
||||
|
||||
## 3a. Updates
|
||||
|
||||
Die Images sind **auf einen Digest gepinnt** — es läuft immer genau der Stand, der im Repo steht.
|
||||
Weder ein `docker compose up -d` noch ein Reboot tauscht die Anwendung unbemerkt aus.
|
||||
|
||||
Beim Sitzungsstart (oder wann immer Sie mögen):
|
||||
|
||||
```bash
|
||||
./scripts/check_updates.sh # zeigt nur: läuft X, verfügbar ist Y
|
||||
```
|
||||
|
||||
Wenn ein neues Release da ist:
|
||||
|
||||
```bash
|
||||
./scripts/update_stack.sh # Backup -> Update -> Rauchtest -> bei Fehler Rollback
|
||||
```
|
||||
|
||||
Warum nicht automatisch das Neueste bei jedem Start? Ein Update **migriert die Datenbank**, und
|
||||
eine ältere Anwendung kann eine migrierte Datenbank in der Regel nicht mehr lesen — ein
|
||||
misslungenes Update ohne Backup wäre eine Einbahnstraße. Zudem hängen die lokalen Anpassungen
|
||||
dieses Projekts (Podcast-Vorlagen, content-core-Config, Env-Variablen) an Interna der Anwendung
|
||||
und können still brechen. Genau das prüft der Rauchtest; schlägt er fehl, rollt das Skript
|
||||
Daten und Compose-Datei automatisch zurück.
|
||||
|
||||
Hinweis: `v1-latest` folgt den **Releases**, nicht dem `main`-Branch — der Repository-Stand ist
|
||||
typischerweise Wochen voraus, aber unveröffentlicht. Wer den will, müsste selbst bauen.
|
||||
|
||||
---
|
||||
|
||||
## 4. Verzeichnisstruktur
|
||||
|
||||
```
|
||||
|
|
@ -207,10 +240,15 @@ open-notebook/
|
|||
│ ├── systemd/ # Unit-Dateien für die beiden Wrapper
|
||||
│ ├── voices/ # Referenz-WAVs fürs Voice-Cloning (gitignored)
|
||||
│ └── logs/ # (gitignored, Altlast der früheren nohup-Variante)
|
||||
├── backups/ # Sicherungen vor Image-Updates (gitignored)
|
||||
├── scripts/
|
||||
│ ├── setup_models.sh # Provider/Modelle einrichten (idempotent)
|
||||
│ ├── start_services.sh # TTS/STT als systemd-User-Dienste installieren+starten
|
||||
│ └── prune_podcast_data.py # verwaiste Podcast-Ordner + Zwischenclips aufräumen
|
||||
│ ├── prune_podcast_data.py # verwaiste Podcast-Ordner + Zwischenclips aufräumen
|
||||
│ ├── add_video_source.py # Video als Quelle (Untertitel oder Tonspur+Whisper)
|
||||
│ ├── check_updates.sh # neues Release verfügbar? (ändert nichts)
|
||||
│ ├── update_stack.sh # Update mit Backup/Rauchtest/Rollback
|
||||
│ └── smoke_test.sh # prüft die Annahmen, auf denen die Anpassungen beruhen
|
||||
├── README.md # diese Datei
|
||||
├── BEDIENUNGSANLEITUNG.md # ausführliche Bedienungsanleitung (Deutsch)
|
||||
└── CLAUDE.md # technische Architektur-Doku für Weiterentwicklung
|
||||
|
|
|
|||
|
|
@ -1,8 +1,21 @@
|
|||
# Images sind bewusst auf einen Digest gepinnt, nicht auf einen beweglichen Tag.
|
||||
#
|
||||
# Warum: "v1-latest" ist ein wandernder Zeiger. Mit pull_policy: always zog jedes
|
||||
# `docker compose up -d` unbemerkt eine neue Anwendung — inklusive Schema-Migration der
|
||||
# SurrealDB, die sich nicht ohne Weiteres zurueckdrehen laesst. Gleichzeitig griff das
|
||||
# nur bei `up -d`: nach einem Reboot startete `restart: always` weiter das ALTE Image.
|
||||
# Also beides unzuverlaessig: mal heimlich neu, mal heimlich alt.
|
||||
#
|
||||
# Mit Digest-Pin laeuft immer exakt der Stand, der hier im Repo steht. Aktualisiert wird
|
||||
# bewusst und mit Rueckfallpunkt:
|
||||
# ./scripts/check_updates.sh -> gibt es ein neues Release? (aendert nichts)
|
||||
# ./scripts/update_stack.sh -> Backup, Update, Rauchtest, bei Fehler Rollback
|
||||
services:
|
||||
surrealdb:
|
||||
image: surrealdb/surrealdb:v2
|
||||
# surrealdb:v2 (Digest-Pin: auch ein Minor-Sprung kann das Storage-Format anfassen)
|
||||
image: surrealdb/surrealdb:v2@sha256:d653f6c8a89e81f865ee31cd2f587c50f50ace922175e04150b1e385d2f86011
|
||||
restart: always
|
||||
pull_policy: always
|
||||
pull_policy: missing
|
||||
command: start --log info --user ${SURREAL_USER:-root} --pass ${SURREAL_PASSWORD:-root} rocksdb:/mydata/mydatabase.db
|
||||
user: root # Required for bind mounts on Linux
|
||||
environment:
|
||||
|
|
@ -13,9 +26,11 @@ services:
|
|||
- ./surreal_data:/mydata
|
||||
|
||||
open_notebook:
|
||||
image: lfnovo/open_notebook:v1-latest
|
||||
# Open Notebook v1.10.0 (Release vom 2026-06-18, Image vom selben Tag).
|
||||
# Digest von scripts/update_stack.sh gepflegt — nicht von Hand aendern.
|
||||
image: lfnovo/open_notebook:v1-latest@sha256:c8112fbd4b8fee7f2a20d3bdbea24e7d72267acfc990b803fd0ff4de30899b57
|
||||
restart: always
|
||||
pull_policy: always
|
||||
pull_policy: missing
|
||||
depends_on:
|
||||
- surrealdb
|
||||
ports:
|
||||
|
|
|
|||
57
scripts/check_updates.sh
Executable file
57
scripts/check_updates.sh
Executable file
|
|
@ -0,0 +1,57 @@
|
|||
#!/usr/bin/env bash
|
||||
# Zeigt, ob es ein neues Open-Notebook-Release gibt. Aendert NICHTS.
|
||||
#
|
||||
# Gedacht fuer den Sitzungsstart: Du siehst, worauf du laeufst und ob etwas Neues da
|
||||
# ist — und entscheidest selbst, wann du migrierst (./scripts/update_stack.sh).
|
||||
# Bewusst kein Auto-Update: ein Image-Update migriert die Datenbank und kann unsere
|
||||
# Anpassungen brechen (siehe scripts/smoke_test.sh).
|
||||
set -uo pipefail
|
||||
|
||||
REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
cd "$REPO"
|
||||
|
||||
TAG="lfnovo/open_notebook:v1-latest"
|
||||
|
||||
# Gezielt aus der open_notebook-Zeile: ein blindes grep auf den ersten Digest der Datei
|
||||
# erwischt surrealdb (steht weiter oben) und vergleicht dann Aepfel mit Birnen.
|
||||
pinned="$(grep -oP 'lfnovo/open_notebook\S*@\Ksha256:[0-9a-f]+' docker-compose.yml | head -1)"
|
||||
running_ver="$(docker compose exec -T open_notebook grep -m1 '^version' /app/pyproject.toml 2>/dev/null |
|
||||
grep -oP '(?<=")[^"]+' || echo '?')"
|
||||
|
||||
echo "Laeuft gerade"
|
||||
echo " Version : ${running_ver}"
|
||||
echo " Digest : ${pinned:-(nicht gepinnt!)}"
|
||||
|
||||
echo
|
||||
echo "Verfuegbar"
|
||||
latest_rel="$(curl -sf -m 10 https://api.github.com/repos/lfnovo/open-notebook/releases/latest |
|
||||
python3 -c "import json,sys;d=json.load(sys.stdin);print(d['tag_name'],'vom',d['published_at'][:10])" 2>/dev/null)"
|
||||
echo " Release : ${latest_rel:-(GitHub nicht erreichbar)}"
|
||||
|
||||
remote="$(docker buildx imagetools inspect "$TAG" --format '{{.Manifest.Digest}}' 2>/dev/null)"
|
||||
if [ -z "$remote" ]; then
|
||||
echo " Image : (Registry nicht erreichbar)"
|
||||
exit 0
|
||||
fi
|
||||
echo " Image : $remote"
|
||||
|
||||
echo
|
||||
if [ "$remote" = "$pinned" ]; then
|
||||
echo "Du laeufst auf dem aktuellen Release. Nichts zu tun."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
cat <<EOF
|
||||
NEUES IMAGE VERFUEGBAR.
|
||||
|
||||
gepinnt : $pinned
|
||||
neu : $remote
|
||||
|
||||
Update mit Backup, Rauchtest und Rollback:
|
||||
./scripts/update_stack.sh
|
||||
|
||||
Vorher lohnt ein Blick in die Release-Notes — vor allem auf Aenderungen an den
|
||||
Podcast-Vorlagen, an content-core und an der Datenbank:
|
||||
https://github.com/lfnovo/open-notebook/releases
|
||||
EOF
|
||||
exit 0
|
||||
161
scripts/smoke_test.sh
Executable file
161
scripts/smoke_test.sh
Executable file
|
|
@ -0,0 +1,161 @@
|
|||
#!/usr/bin/env bash
|
||||
# Rauchtest fuer den laufenden Stack.
|
||||
#
|
||||
# Prueft nicht "laeuft der Container", sondern genau die Dinge, die bei diesem
|
||||
# Deployment schon einmal kaputt waren — inklusive der Annahmen, auf denen unsere
|
||||
# Anpassungen beruhen. Genau die koennen ein Image-Update stillschweigend brechen:
|
||||
# der Bind-Mount ueber prompts/podcast/ verdeckt neue Dateien, config/content_core.yaml
|
||||
# friert alte Defaults ein (CCORE_CONFIG_PATH ersetzt, statt zu mergen), und die
|
||||
# Env-Var-Workarounds haengen an Interna von content-core/esperanto/podcast_creator.
|
||||
#
|
||||
# ./scripts/smoke_test.sh # alles
|
||||
# SKIP_NET=1 ./scripts/smoke_test.sh # ohne Tests, die ins Internet muessen
|
||||
#
|
||||
# Exit 0 = alles gruen. Exit 1 = mindestens ein Test rot.
|
||||
set -uo pipefail
|
||||
|
||||
REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
cd "$REPO"
|
||||
|
||||
API="http://127.0.0.1:5055"
|
||||
IMAGE="$(grep -oP '(?<=image: )lfnovo/open_notebook\S+' docker-compose.yml | head -1)"
|
||||
PY=/app/.venv/bin/python
|
||||
fails=0
|
||||
|
||||
ok() { printf ' \033[32mOK\033[0m %s\n' "$1"; }
|
||||
fail() { printf ' \033[31mFEHL\033[0m %s\n' "$1"; fails=$((fails + 1)); }
|
||||
sec() { printf '\n%s\n' "$1"; }
|
||||
|
||||
inapp() { docker compose exec -T open_notebook "$@" 2>/dev/null; }
|
||||
|
||||
sec "Grundfunktion"
|
||||
|
||||
if curl -sf -m 10 "$API/api/models/providers" -o /tmp/.st_prov 2>/dev/null; then
|
||||
if grep -q ollama /tmp/.st_prov && grep -q openrouter /tmp/.st_prov; then
|
||||
ok "API erreichbar, Provider ollama + openrouter verfuegbar"
|
||||
else
|
||||
fail "API erreichbar, aber Provider fehlen: $(cat /tmp/.st_prov)"
|
||||
fi
|
||||
else
|
||||
fail "API $API nicht erreichbar"
|
||||
fi
|
||||
rm -f /tmp/.st_prov
|
||||
|
||||
sec "Lokale Sprachdienste (vom Container aus)"
|
||||
|
||||
if inapp "$PY" -c "
|
||||
import httpx,sys
|
||||
r = httpx.post('http://host.docker.internal:8901/audio/speech',
|
||||
json={'model':'chatterbox','voice':'default','input':'Test.','response_format':'wav'}, timeout=180)
|
||||
sys.exit(0 if r.status_code == 200 and len(r.content) > 1000 else 1)"; then
|
||||
ok "TTS antwortet mit Audio (host.docker.internal:8901)"
|
||||
else
|
||||
fail "TTS liefert kein Audio — Dienst tot (ConnectError) oder VRAM voll (HTTP 500)?
|
||||
systemctl --user status open-notebook-tts; journalctl --user -u open-notebook-tts -n 30"
|
||||
fi
|
||||
|
||||
if inapp "$PY" -c "
|
||||
import httpx,sys
|
||||
r = httpx.get('http://host.docker.internal:8902/health', timeout=30)
|
||||
sys.exit(0 if r.status_code == 200 else 1)"; then
|
||||
ok "STT erreichbar (host.docker.internal:8902)"
|
||||
else
|
||||
fail "STT nicht erreichbar — systemctl --user status open-notebook-stt"
|
||||
fi
|
||||
|
||||
sec "Annahmen unserer Anpassungen (brechen leise bei Image-Updates)"
|
||||
|
||||
# 1) prompts/podcast: Verzeichnis-Mount verdeckt das Image-Verzeichnis komplett.
|
||||
# Kommt upstream eine dritte Vorlage dazu, waere sie unsichtbar.
|
||||
img_tpl="$(docker run --rm --entrypoint ls "$IMAGE" /app/prompts/podcast 2>/dev/null | sort | tr '\n' ' ')"
|
||||
our_tpl="$(ls prompts/podcast | sort | tr '\n' ' ')"
|
||||
if [ "$img_tpl" = "$our_tpl" ]; then
|
||||
ok "prompts/podcast: Dateiliste deckt sich mit dem Image ($our_tpl)"
|
||||
else
|
||||
fail "prompts/podcast weicht ab — unser Mount verdeckt Image-Dateien!
|
||||
Image: $img_tpl
|
||||
wir : $our_tpl"
|
||||
fi
|
||||
|
||||
# 2) Unsere Patches in den Vorlagen sind noch drin.
|
||||
if grep -q '{{ language }}' prompts/podcast/outline.jinja prompts/podcast/transcript.jinja &&
|
||||
head -1 prompts/podcast/outline.jinja | grep -q '/no_think'; then
|
||||
ok "Vorlagen enthalten Sprachanweisung ({{ language }}) und /no_think"
|
||||
else
|
||||
fail "Vorlagen-Patches fehlen — Podcasts kaemen wieder auf Englisch / mit kaputtem JSON"
|
||||
fi
|
||||
|
||||
# 3) podcast_creator reicht `language` ueberhaupt noch in die Vorlagen durch.
|
||||
if inapp grep -q "language" /app/.venv/lib/python3.12/site-packages/podcast_creator/nodes.py; then
|
||||
ok "podcast_creator kennt weiterhin ein language-Feld"
|
||||
else
|
||||
fail "podcast_creator reicht kein language mehr durch — Vorlagen-Patch wirkungslos"
|
||||
fi
|
||||
|
||||
# 4) TTS_BATCH_SIZE existiert noch als Stellschraube (sonst wieder 5 parallele Clips -> OOM).
|
||||
if inapp grep -q "TTS_BATCH_SIZE" /app/.venv/lib/python3.12/site-packages/podcast_creator/nodes.py; then
|
||||
ok "podcast_creator liest weiterhin TTS_BATCH_SIZE"
|
||||
else
|
||||
fail "TTS_BATCH_SIZE wird nicht mehr gelesen — Gefahr paralleler Clips (CUDA OOM)"
|
||||
fi
|
||||
|
||||
# 5) content-core sucht YouTube-Untertitel weiterhin ueber preferred_languages.
|
||||
if inapp grep -q "preferred_languages" /app/.venv/lib/python3.12/site-packages/content_core/processors/youtube.py; then
|
||||
ok "content-core nutzt weiterhin preferred_languages (deutsche Untertitel)"
|
||||
else
|
||||
fail "content-core sucht Untertitel anders — config/content_core.yaml pruefen"
|
||||
fi
|
||||
|
||||
# 6) content-core baut das STT-Modell weiterhin ohne base_url (deshalb die Env-Var).
|
||||
if inapp grep -q "OPENAI_COMPATIBLE_BASE_URL" /app/.venv/lib/python3.12/site-packages/esperanto/providers/stt/openai_compatible.py; then
|
||||
ok "esperanto akzeptiert weiterhin OPENAI_COMPATIBLE_BASE_URL_STT (Audio-Transkription)"
|
||||
else
|
||||
fail "esperanto liest die Env-Var nicht mehr — Audio-Quellen fielen auf OpenAI zurueck"
|
||||
fi
|
||||
|
||||
# 7) CCORE_CONFIG_PATH ersetzt die Config komplett. Bringt ein Update neue
|
||||
# Top-Level-Schluessel mit, frieren wir die alten ein — hier faellt das auf.
|
||||
img_keys="$(docker run --rm --entrypoint "$PY" -e CCORE_CONFIG_PATH= "$IMAGE" \
|
||||
-c "from content_core.config import load_config; print(' '.join(sorted(load_config())))" 2>/dev/null)"
|
||||
our_keys="$(inapp "$PY" -c "
|
||||
import yaml; print(' '.join(sorted(yaml.safe_load(open('/app/config/content_core.yaml')))))")"
|
||||
missing="$(comm -23 <(tr ' ' '\n' <<<"$img_keys" | sort -u) <(tr ' ' '\n' <<<"$our_keys" | sort -u) | tr '\n' ' ')"
|
||||
if [ -z "${missing// /}" ] && [ -n "$img_keys" ]; then
|
||||
ok "config/content_core.yaml enthaelt alle Default-Schluessel des Images"
|
||||
else
|
||||
fail "config/content_core.yaml fehlen Schluessel aus dem Image: ${missing:-(Image-Defaults nicht lesbar)}
|
||||
Neu erzeugen — siehe Kopfkommentar in config/content_core.yaml"
|
||||
fi
|
||||
|
||||
# 8) Episode-Profile haben eine Sprache (sonst wieder englische Podcasts).
|
||||
if curl -sf -m 10 "$API/api/episode-profiles" | python3 -c "
|
||||
import json,sys
|
||||
p=json.load(sys.stdin)
|
||||
sys.exit(0 if p and all(x.get('language') for x in p) else 1)" 2>/dev/null; then
|
||||
ok "alle Episode-Profile haben ein language-Feld gesetzt"
|
||||
else
|
||||
fail "mindestens ein Episode-Profil ohne language — Podcast kaeme auf Englisch"
|
||||
fi
|
||||
|
||||
if [ "${SKIP_NET:-0}" != "1" ]; then
|
||||
sec "Ende-zu-Ende (braucht Internet)"
|
||||
if inapp "$PY" -c "
|
||||
import asyncio,sys
|
||||
from content_core import extract_content
|
||||
async def m():
|
||||
r = await extract_content({'url':'https://www.youtube.com/watch?v=hzxiegk9QAg'})
|
||||
sys.exit(0 if len((r.content or '')) > 1000 else 1)
|
||||
asyncio.run(m())"; then
|
||||
ok "YouTube-Quelle liefert Transkript (deutsche Untertitel werden gefunden)"
|
||||
else
|
||||
fail "YouTube-Extraktion liefert keinen Text — CCORE_CONFIG_PATH/Sprachen pruefen"
|
||||
fi
|
||||
fi
|
||||
|
||||
sec "Ergebnis"
|
||||
if [ "$fails" -eq 0 ]; then
|
||||
echo " Alles gruen."
|
||||
exit 0
|
||||
fi
|
||||
echo " $fails Test(s) fehlgeschlagen."
|
||||
exit 1
|
||||
104
scripts/update_stack.sh
Executable file
104
scripts/update_stack.sh
Executable file
|
|
@ -0,0 +1,104 @@
|
|||
#!/usr/bin/env bash
|
||||
# Aktualisiert Open Notebook auf das neueste veroeffentlichte Image — mit Rueckfallpunkt.
|
||||
#
|
||||
# Ablauf:
|
||||
# 1. Neuen Digest aus der Registry holen (nichts zu tun, wenn schon aktuell).
|
||||
# 2. Stack stoppen und Daten sichern (surreal_data + notebook_data + docker-compose.yml).
|
||||
# 3. Digest in docker-compose.yml eintragen, Stack starten.
|
||||
# 4. Rauchtest (scripts/smoke_test.sh).
|
||||
# 5. Rot -> alles zurueckrollen (Daten UND Compose-Datei). Gruen -> fertig.
|
||||
#
|
||||
# Der Rollback der Daten ist der Grund fuer das Backup: Open Notebook migriert die
|
||||
# SurrealDB beim Start, und ein Downgrade der Anwendung kann eine migrierte Datenbank
|
||||
# nicht wieder lesen. Ohne Backup waere ein missgluecktes Update eine Einbahnstrasse.
|
||||
#
|
||||
# ./scripts/update_stack.sh # auf neuestes Image
|
||||
# ./scripts/update_stack.sh --dry-run # nur zeigen, was passieren wuerde
|
||||
set -euo pipefail
|
||||
|
||||
REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
cd "$REPO"
|
||||
|
||||
TAG="lfnovo/open_notebook:v1-latest"
|
||||
BACKUP_DIR="$REPO/backups"
|
||||
KEEP_BACKUPS=3
|
||||
DRY_RUN=0
|
||||
[ "${1:-}" = "--dry-run" ] && DRY_RUN=1
|
||||
|
||||
say() { printf '\n\033[1m%s\033[0m\n' "$1"; }
|
||||
|
||||
# Gezielt aus der open_notebook-Zeile. Ein blindes grep auf den ersten Digest der Datei
|
||||
# erwischt surrealdb (steht weiter oben) — das sed weiter unten wuerde dann den falschen
|
||||
# Digest ersetzen und die Datenbank austauschen statt der Anwendung.
|
||||
pinned="$(grep -oP 'lfnovo/open_notebook\S*@\Ksha256:[0-9a-f]+' docker-compose.yml | head -1)"
|
||||
[ -n "$pinned" ] || { echo "FEHLER: kein open_notebook-Digest in docker-compose.yml gepinnt." >&2; exit 1; }
|
||||
|
||||
say "1/5 Registry abfragen"
|
||||
remote="$(docker buildx imagetools inspect "$TAG" --format '{{.Manifest.Digest}}')"
|
||||
echo " gepinnt: $pinned"
|
||||
echo " neu : $remote"
|
||||
if [ "$remote" = "$pinned" ]; then
|
||||
echo " Bereits aktuell — nichts zu tun."
|
||||
exit 0
|
||||
fi
|
||||
if [ "$DRY_RUN" = 1 ]; then
|
||||
echo " (--dry-run: hier wuerde das Update starten)"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
ts="$(date +%Y%m%d-%H%M%S)"
|
||||
backup="$BACKUP_DIR/$ts"
|
||||
mkdir -p "$backup"
|
||||
|
||||
say "2/5 Stack stoppen und sichern -> backups/$ts"
|
||||
docker compose down
|
||||
cp docker-compose.yml "$backup/docker-compose.yml"
|
||||
# Die Datenverzeichnisse gehoeren root (der Container schreibt als root), deshalb wird
|
||||
# im Container gepackt statt auf dem Host — spart sudo.
|
||||
docker run --rm -v "$REPO:/work" -w /work --entrypoint tar \
|
||||
"lfnovo/open_notebook@$pinned" -czf "backups/$ts/data.tgz" surreal_data notebook_data
|
||||
echo " $(du -sh "$backup/data.tgz" | cut -f1) gesichert"
|
||||
|
||||
restore() {
|
||||
say "ROLLBACK"
|
||||
docker compose down || true
|
||||
cp "$backup/docker-compose.yml" docker-compose.yml
|
||||
docker run --rm -v "$REPO:/work" -w /work --entrypoint sh \
|
||||
"lfnovo/open_notebook@$pinned" -c \
|
||||
"rm -rf surreal_data notebook_data && tar -xzf backups/$ts/data.tgz"
|
||||
docker compose up -d
|
||||
echo " Alter Stand ($pinned) ist wiederhergestellt, Daten aus backups/$ts."
|
||||
echo " Das neue Image bleibt lokal liegen — Ursache pruefen, dann erneut versuchen."
|
||||
}
|
||||
|
||||
say "3/5 Auf neues Image umstellen"
|
||||
sed -i "s|@$pinned|@$remote|" docker-compose.yml
|
||||
docker compose up -d
|
||||
echo " gestartet, warte auf die API..."
|
||||
for _ in $(seq 1 30); do
|
||||
curl -sf -m 3 http://127.0.0.1:5055/api/models/providers >/dev/null 2>&1 && break
|
||||
sleep 5
|
||||
done
|
||||
|
||||
say "4/5 Rauchtest"
|
||||
if ./scripts/smoke_test.sh; then
|
||||
say "5/5 Fertig"
|
||||
newver="$(docker compose exec -T open_notebook grep -m1 '^version' /app/pyproject.toml |
|
||||
grep -oP '(?<=")[^"]+' || echo '?')"
|
||||
echo " Update auf $newver erfolgreich, Rauchtest gruen."
|
||||
echo " Backup: backups/$ts (bleibt liegen)"
|
||||
echo
|
||||
echo " Bitte den neuen Digest committen:"
|
||||
echo " git add docker-compose.yml && git commit -m 'Update Open Notebook auf $newver'"
|
||||
else
|
||||
echo
|
||||
echo " Rauchtest ROT — das Update wird zurueckgenommen."
|
||||
restore
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Alte Backups aufraeumen (Platte ist knapp).
|
||||
ls -1dt "$BACKUP_DIR"/*/ 2>/dev/null | tail -n +$((KEEP_BACKUPS + 1)) | while read -r old; do
|
||||
echo " raeume altes Backup weg: $(basename "$old")"
|
||||
rm -rf "$old"
|
||||
done
|
||||
Loading…
Add table
Add a link
Reference in a new issue