Pin images by digest; add update, smoke-test and status scripts
Goal: at every session start, run exactly the application the repo says — and be able to move to a new upstream release deliberately, with a way back. The old setup (v1-latest + pull_policy: always) was unreliable in both directions. `docker compose up -d` silently pulled a new application, including an irreversible SurrealDB schema migration, while a reboot (restart: always) or `docker compose start` kept running the old image without pulling anything. The running version was effectively unpredictable. Note v1-latest tracks releases, not main: the current image (v1.10.0, 2026-06-18) IS the latest release — the repo being "ahead" is unreleased code, which is explicitly not wanted here. So this is about guaranteeing and detecting, not catching up. - docker-compose.yml: both images pinned by digest, pull_policy: missing. - scripts/check_updates.sh: reports running version/digest vs the latest release and registry digest. Changes nothing; meant for the session-start ritual. - scripts/update_stack.sh: resolve new digest -> stop -> back up surreal_data, notebook_data and docker-compose.yml -> repin -> start -> smoke test -> roll back data AND compose file if the smoke test fails. The data backup is the point: the app migrates the DB on startup and an older app cannot read a migrated DB, so a bad update would otherwise be a one-way door. tar runs inside a container because the data dirs are root-owned and the host user cannot restore over them. - scripts/smoke_test.sh: checks what actually breaks here, not just "does it start". Every local adaptation leans on upstream internals and can break silently: the prompts/podcast directory mount masks the image's directory (a new template upstream would be invisible), config/content_core.yaml freezes content-core's defaults (because CCORE_CONFIG_PATH replaces rather than merges), and the env-var workarounds depend on current content-core/esperanto/podcast_creator behaviour. So it verifies those assumptions explicitly, plus API, TTS audio, STT and a real YouTube transcript. Both scripts read the digest from the lfnovo/open_notebook line specifically. A naive "first sha256 in the file" grep matches surrealdb (listed first) — caught while testing: check_updates.sh falsely reported an update, and update_stack.sh would have rewritten the database image instead of the application. Verified: smoke test green against the current stack, and red (exit 1) when failures are injected (dead TTS port, removed template variable). Backup/restore mechanics exercised separately: 44 MB in 3.3s, restore yields 13 DB files and 91 notebook files. The update path itself cannot be exercised end to end until a newer image exists. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
f350e49bd5
commit
4805fe75ed
7 changed files with 434 additions and 7 deletions
3
.gitignore
vendored
3
.gitignore
vendored
|
|
@ -10,3 +10,6 @@ __pycache__/
|
||||||
|
|
||||||
# Artefakt versehentlicher Shell-Redirects
|
# Artefakt versehentlicher Shell-Redirects
|
||||||
/3
|
/3
|
||||||
|
|
||||||
|
# Backups vor Image-Updates (scripts/update_stack.sh)
|
||||||
|
backups/
|
||||||
|
|
|
||||||
53
CLAUDE.md
53
CLAUDE.md
|
|
@ -9,13 +9,62 @@ This is a local Docker Compose deployment of [Open Notebook](https://github.com/
|
||||||
## Commands
|
## Commands
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
docker compose up -d # start (pulls images per pull_policy: always)
|
docker compose up -d # start (images are digest-pinned; no implicit upgrade)
|
||||||
docker compose down # stop and remove containers (data volumes persist)
|
docker compose down # stop and remove containers (data volumes persist)
|
||||||
docker compose logs -f open_notebook # follow app logs
|
docker compose logs -f open_notebook # follow app logs
|
||||||
docker compose ps # container status
|
docker compose ps # container status
|
||||||
|
|
||||||
|
./scripts/check_updates.sh # is there a newer release? (reports only, changes nothing)
|
||||||
|
./scripts/update_stack.sh # backup -> update -> smoke test -> rollback on failure
|
||||||
|
./scripts/smoke_test.sh # verify the stack still does what we rely on
|
||||||
```
|
```
|
||||||
|
|
||||||
No build/lint/test suite exists in this repo — nothing here to run.
|
`smoke_test.sh` is the closest thing to a test suite here — run it after touching
|
||||||
|
`docker-compose.yml`, the bind-mounted overlays, or the host TTS/STT services.
|
||||||
|
|
||||||
|
## Image versioning
|
||||||
|
|
||||||
|
Images are **pinned by digest**, and updating is an explicit act — never a side effect of
|
||||||
|
starting the stack.
|
||||||
|
|
||||||
|
The previous setup (`v1-latest` + `pull_policy: always`) was unreliable in *both* directions:
|
||||||
|
`docker compose up -d` silently pulled a new application — including an irreversible SurrealDB
|
||||||
|
schema migration — while a reboot (`restart: always`) or `docker compose start` kept running the
|
||||||
|
**old** image without any pull. So the running version was essentially unpredictable.
|
||||||
|
|
||||||
|
Note `v1-latest` tracks *releases*, not `main`: the tag lags the repo by weeks. Chasing `main`
|
||||||
|
would mean building from source and running unreleased code — explicitly not wanted here.
|
||||||
|
|
||||||
|
Updating (`scripts/update_stack.sh`): resolve the new digest → stop → back up `surreal_data` +
|
||||||
|
`notebook_data` + `docker-compose.yml` into `backups/<ts>/` → rewrite the digest → start → run
|
||||||
|
`smoke_test.sh` → **roll back data and compose file if the smoke test fails**. The data backup is
|
||||||
|
the point: the app migrates the database on startup, and an older application generally cannot
|
||||||
|
read a migrated database, so a bad update would otherwise be a one-way door. Backup/restore runs
|
||||||
|
`tar` **inside a container** because the data directories are root-owned (the container writes as
|
||||||
|
root) and the host user cannot restore over them.
|
||||||
|
|
||||||
|
Both scripts read the digest from the `lfnovo/open_notebook` line specifically — a naive "first
|
||||||
|
sha256 in the file" grep picks up **surrealdb** (it is listed first), which would make the update
|
||||||
|
script rewrite the database image instead of the application.
|
||||||
|
|
||||||
|
### Why a smoke test, and not just "does it start"
|
||||||
|
|
||||||
|
Every local adaptation in this repo leans on upstream internals, and an update can break them
|
||||||
|
*silently* — the stack comes up fine and misbehaves later:
|
||||||
|
|
||||||
|
- the `prompts/podcast/` **directory** mount masks the image's directory: a third template added
|
||||||
|
upstream would simply be invisible;
|
||||||
|
- `config/content_core.yaml` is a full dump of content-core's defaults (because `CCORE_CONFIG_PATH`
|
||||||
|
replaces rather than merges), so new upstream defaults would be frozen out;
|
||||||
|
- the env-var workarounds (`TTS_BATCH_SIZE`, `OPENAI_COMPATIBLE_BASE_URL_STT`,
|
||||||
|
`ESPERANTO_TTS_TIMEOUT`) depend on how content-core / esperanto / podcast_creator behave today.
|
||||||
|
|
||||||
|
`smoke_test.sh` therefore checks those assumptions explicitly (template file list vs. the image,
|
||||||
|
`{{ language }}` and `/no_think` still present, `language` still plumbed by podcast_creator,
|
||||||
|
`TTS_BATCH_SIZE` still read, `preferred_languages` still used, esperanto still honouring
|
||||||
|
`OPENAI_COMPATIBLE_BASE_URL_STT`, our config still covering all default keys) on top of the obvious
|
||||||
|
end-to-end ones (API, TTS audio, STT, YouTube transcript). `SKIP_NET=1` skips the internet-dependent
|
||||||
|
check.
|
||||||
|
|
||||||
## Architecture
|
## Architecture
|
||||||
|
|
||||||
|
|
|
||||||
40
README.md
40
README.md
|
|
@ -183,6 +183,10 @@ journalctl --user -u open-notebook-tts -f # TTS-Logs
|
||||||
./scripts/prune_podcast_data.py --yes # ... und löschen
|
./scripts/prune_podcast_data.py --yes # ... und löschen
|
||||||
|
|
||||||
./scripts/add_video_source.py <url> # Video als Quelle (auch ohne Untertitel)
|
./scripts/add_video_source.py <url> # Video als Quelle (auch ohne Untertitel)
|
||||||
|
|
||||||
|
./scripts/check_updates.sh # gibt es ein neues Release? (ändert nichts)
|
||||||
|
./scripts/update_stack.sh # Update mit Backup, Rauchtest und Rollback
|
||||||
|
./scripts/smoke_test.sh # tut der Stack noch, worauf wir uns verlassen?
|
||||||
```
|
```
|
||||||
|
|
||||||
TTS/STT starten nach einem Reboot von selbst — `start_services.sh` ist nur für die
|
TTS/STT starten nach einem Reboot von selbst — `start_services.sh` ist nur für die
|
||||||
|
|
@ -190,6 +194,35 @@ Ersteinrichtung bzw. nach Änderungen an den Unit-Dateien nötig.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## 3a. Updates
|
||||||
|
|
||||||
|
Die Images sind **auf einen Digest gepinnt** — es läuft immer genau der Stand, der im Repo steht.
|
||||||
|
Weder ein `docker compose up -d` noch ein Reboot tauscht die Anwendung unbemerkt aus.
|
||||||
|
|
||||||
|
Beim Sitzungsstart (oder wann immer Sie mögen):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/check_updates.sh # zeigt nur: läuft X, verfügbar ist Y
|
||||||
|
```
|
||||||
|
|
||||||
|
Wenn ein neues Release da ist:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/update_stack.sh # Backup -> Update -> Rauchtest -> bei Fehler Rollback
|
||||||
|
```
|
||||||
|
|
||||||
|
Warum nicht automatisch das Neueste bei jedem Start? Ein Update **migriert die Datenbank**, und
|
||||||
|
eine ältere Anwendung kann eine migrierte Datenbank in der Regel nicht mehr lesen — ein
|
||||||
|
misslungenes Update ohne Backup wäre eine Einbahnstraße. Zudem hängen die lokalen Anpassungen
|
||||||
|
dieses Projekts (Podcast-Vorlagen, content-core-Config, Env-Variablen) an Interna der Anwendung
|
||||||
|
und können still brechen. Genau das prüft der Rauchtest; schlägt er fehl, rollt das Skript
|
||||||
|
Daten und Compose-Datei automatisch zurück.
|
||||||
|
|
||||||
|
Hinweis: `v1-latest` folgt den **Releases**, nicht dem `main`-Branch — der Repository-Stand ist
|
||||||
|
typischerweise Wochen voraus, aber unveröffentlicht. Wer den will, müsste selbst bauen.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## 4. Verzeichnisstruktur
|
## 4. Verzeichnisstruktur
|
||||||
|
|
||||||
```
|
```
|
||||||
|
|
@ -207,10 +240,15 @@ open-notebook/
|
||||||
│ ├── systemd/ # Unit-Dateien für die beiden Wrapper
|
│ ├── systemd/ # Unit-Dateien für die beiden Wrapper
|
||||||
│ ├── voices/ # Referenz-WAVs fürs Voice-Cloning (gitignored)
|
│ ├── voices/ # Referenz-WAVs fürs Voice-Cloning (gitignored)
|
||||||
│ └── logs/ # (gitignored, Altlast der früheren nohup-Variante)
|
│ └── logs/ # (gitignored, Altlast der früheren nohup-Variante)
|
||||||
|
├── backups/ # Sicherungen vor Image-Updates (gitignored)
|
||||||
├── scripts/
|
├── scripts/
|
||||||
│ ├── setup_models.sh # Provider/Modelle einrichten (idempotent)
|
│ ├── setup_models.sh # Provider/Modelle einrichten (idempotent)
|
||||||
│ ├── start_services.sh # TTS/STT als systemd-User-Dienste installieren+starten
|
│ ├── start_services.sh # TTS/STT als systemd-User-Dienste installieren+starten
|
||||||
│ └── prune_podcast_data.py # verwaiste Podcast-Ordner + Zwischenclips aufräumen
|
│ ├── prune_podcast_data.py # verwaiste Podcast-Ordner + Zwischenclips aufräumen
|
||||||
|
│ ├── add_video_source.py # Video als Quelle (Untertitel oder Tonspur+Whisper)
|
||||||
|
│ ├── check_updates.sh # neues Release verfügbar? (ändert nichts)
|
||||||
|
│ ├── update_stack.sh # Update mit Backup/Rauchtest/Rollback
|
||||||
|
│ └── smoke_test.sh # prüft die Annahmen, auf denen die Anpassungen beruhen
|
||||||
├── README.md # diese Datei
|
├── README.md # diese Datei
|
||||||
├── BEDIENUNGSANLEITUNG.md # ausführliche Bedienungsanleitung (Deutsch)
|
├── BEDIENUNGSANLEITUNG.md # ausführliche Bedienungsanleitung (Deutsch)
|
||||||
└── CLAUDE.md # technische Architektur-Doku für Weiterentwicklung
|
└── CLAUDE.md # technische Architektur-Doku für Weiterentwicklung
|
||||||
|
|
|
||||||
|
|
@ -1,8 +1,21 @@
|
||||||
|
# Images sind bewusst auf einen Digest gepinnt, nicht auf einen beweglichen Tag.
|
||||||
|
#
|
||||||
|
# Warum: "v1-latest" ist ein wandernder Zeiger. Mit pull_policy: always zog jedes
|
||||||
|
# `docker compose up -d` unbemerkt eine neue Anwendung — inklusive Schema-Migration der
|
||||||
|
# SurrealDB, die sich nicht ohne Weiteres zurueckdrehen laesst. Gleichzeitig griff das
|
||||||
|
# nur bei `up -d`: nach einem Reboot startete `restart: always` weiter das ALTE Image.
|
||||||
|
# Also beides unzuverlaessig: mal heimlich neu, mal heimlich alt.
|
||||||
|
#
|
||||||
|
# Mit Digest-Pin laeuft immer exakt der Stand, der hier im Repo steht. Aktualisiert wird
|
||||||
|
# bewusst und mit Rueckfallpunkt:
|
||||||
|
# ./scripts/check_updates.sh -> gibt es ein neues Release? (aendert nichts)
|
||||||
|
# ./scripts/update_stack.sh -> Backup, Update, Rauchtest, bei Fehler Rollback
|
||||||
services:
|
services:
|
||||||
surrealdb:
|
surrealdb:
|
||||||
image: surrealdb/surrealdb:v2
|
# surrealdb:v2 (Digest-Pin: auch ein Minor-Sprung kann das Storage-Format anfassen)
|
||||||
|
image: surrealdb/surrealdb:v2@sha256:d653f6c8a89e81f865ee31cd2f587c50f50ace922175e04150b1e385d2f86011
|
||||||
restart: always
|
restart: always
|
||||||
pull_policy: always
|
pull_policy: missing
|
||||||
command: start --log info --user ${SURREAL_USER:-root} --pass ${SURREAL_PASSWORD:-root} rocksdb:/mydata/mydatabase.db
|
command: start --log info --user ${SURREAL_USER:-root} --pass ${SURREAL_PASSWORD:-root} rocksdb:/mydata/mydatabase.db
|
||||||
user: root # Required for bind mounts on Linux
|
user: root # Required for bind mounts on Linux
|
||||||
environment:
|
environment:
|
||||||
|
|
@ -13,9 +26,11 @@ services:
|
||||||
- ./surreal_data:/mydata
|
- ./surreal_data:/mydata
|
||||||
|
|
||||||
open_notebook:
|
open_notebook:
|
||||||
image: lfnovo/open_notebook:v1-latest
|
# Open Notebook v1.10.0 (Release vom 2026-06-18, Image vom selben Tag).
|
||||||
|
# Digest von scripts/update_stack.sh gepflegt — nicht von Hand aendern.
|
||||||
|
image: lfnovo/open_notebook:v1-latest@sha256:c8112fbd4b8fee7f2a20d3bdbea24e7d72267acfc990b803fd0ff4de30899b57
|
||||||
restart: always
|
restart: always
|
||||||
pull_policy: always
|
pull_policy: missing
|
||||||
depends_on:
|
depends_on:
|
||||||
- surrealdb
|
- surrealdb
|
||||||
ports:
|
ports:
|
||||||
|
|
|
||||||
57
scripts/check_updates.sh
Executable file
57
scripts/check_updates.sh
Executable file
|
|
@ -0,0 +1,57 @@
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
# Zeigt, ob es ein neues Open-Notebook-Release gibt. Aendert NICHTS.
|
||||||
|
#
|
||||||
|
# Gedacht fuer den Sitzungsstart: Du siehst, worauf du laeufst und ob etwas Neues da
|
||||||
|
# ist — und entscheidest selbst, wann du migrierst (./scripts/update_stack.sh).
|
||||||
|
# Bewusst kein Auto-Update: ein Image-Update migriert die Datenbank und kann unsere
|
||||||
|
# Anpassungen brechen (siehe scripts/smoke_test.sh).
|
||||||
|
set -uo pipefail
|
||||||
|
|
||||||
|
REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||||
|
cd "$REPO"
|
||||||
|
|
||||||
|
TAG="lfnovo/open_notebook:v1-latest"
|
||||||
|
|
||||||
|
# Gezielt aus der open_notebook-Zeile: ein blindes grep auf den ersten Digest der Datei
|
||||||
|
# erwischt surrealdb (steht weiter oben) und vergleicht dann Aepfel mit Birnen.
|
||||||
|
pinned="$(grep -oP 'lfnovo/open_notebook\S*@\Ksha256:[0-9a-f]+' docker-compose.yml | head -1)"
|
||||||
|
running_ver="$(docker compose exec -T open_notebook grep -m1 '^version' /app/pyproject.toml 2>/dev/null |
|
||||||
|
grep -oP '(?<=")[^"]+' || echo '?')"
|
||||||
|
|
||||||
|
echo "Laeuft gerade"
|
||||||
|
echo " Version : ${running_ver}"
|
||||||
|
echo " Digest : ${pinned:-(nicht gepinnt!)}"
|
||||||
|
|
||||||
|
echo
|
||||||
|
echo "Verfuegbar"
|
||||||
|
latest_rel="$(curl -sf -m 10 https://api.github.com/repos/lfnovo/open-notebook/releases/latest |
|
||||||
|
python3 -c "import json,sys;d=json.load(sys.stdin);print(d['tag_name'],'vom',d['published_at'][:10])" 2>/dev/null)"
|
||||||
|
echo " Release : ${latest_rel:-(GitHub nicht erreichbar)}"
|
||||||
|
|
||||||
|
remote="$(docker buildx imagetools inspect "$TAG" --format '{{.Manifest.Digest}}' 2>/dev/null)"
|
||||||
|
if [ -z "$remote" ]; then
|
||||||
|
echo " Image : (Registry nicht erreichbar)"
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
echo " Image : $remote"
|
||||||
|
|
||||||
|
echo
|
||||||
|
if [ "$remote" = "$pinned" ]; then
|
||||||
|
echo "Du laeufst auf dem aktuellen Release. Nichts zu tun."
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
cat <<EOF
|
||||||
|
NEUES IMAGE VERFUEGBAR.
|
||||||
|
|
||||||
|
gepinnt : $pinned
|
||||||
|
neu : $remote
|
||||||
|
|
||||||
|
Update mit Backup, Rauchtest und Rollback:
|
||||||
|
./scripts/update_stack.sh
|
||||||
|
|
||||||
|
Vorher lohnt ein Blick in die Release-Notes — vor allem auf Aenderungen an den
|
||||||
|
Podcast-Vorlagen, an content-core und an der Datenbank:
|
||||||
|
https://github.com/lfnovo/open-notebook/releases
|
||||||
|
EOF
|
||||||
|
exit 0
|
||||||
161
scripts/smoke_test.sh
Executable file
161
scripts/smoke_test.sh
Executable file
|
|
@ -0,0 +1,161 @@
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
# Rauchtest fuer den laufenden Stack.
|
||||||
|
#
|
||||||
|
# Prueft nicht "laeuft der Container", sondern genau die Dinge, die bei diesem
|
||||||
|
# Deployment schon einmal kaputt waren — inklusive der Annahmen, auf denen unsere
|
||||||
|
# Anpassungen beruhen. Genau die koennen ein Image-Update stillschweigend brechen:
|
||||||
|
# der Bind-Mount ueber prompts/podcast/ verdeckt neue Dateien, config/content_core.yaml
|
||||||
|
# friert alte Defaults ein (CCORE_CONFIG_PATH ersetzt, statt zu mergen), und die
|
||||||
|
# Env-Var-Workarounds haengen an Interna von content-core/esperanto/podcast_creator.
|
||||||
|
#
|
||||||
|
# ./scripts/smoke_test.sh # alles
|
||||||
|
# SKIP_NET=1 ./scripts/smoke_test.sh # ohne Tests, die ins Internet muessen
|
||||||
|
#
|
||||||
|
# Exit 0 = alles gruen. Exit 1 = mindestens ein Test rot.
|
||||||
|
set -uo pipefail
|
||||||
|
|
||||||
|
REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||||
|
cd "$REPO"
|
||||||
|
|
||||||
|
API="http://127.0.0.1:5055"
|
||||||
|
IMAGE="$(grep -oP '(?<=image: )lfnovo/open_notebook\S+' docker-compose.yml | head -1)"
|
||||||
|
PY=/app/.venv/bin/python
|
||||||
|
fails=0
|
||||||
|
|
||||||
|
ok() { printf ' \033[32mOK\033[0m %s\n' "$1"; }
|
||||||
|
fail() { printf ' \033[31mFEHL\033[0m %s\n' "$1"; fails=$((fails + 1)); }
|
||||||
|
sec() { printf '\n%s\n' "$1"; }
|
||||||
|
|
||||||
|
inapp() { docker compose exec -T open_notebook "$@" 2>/dev/null; }
|
||||||
|
|
||||||
|
sec "Grundfunktion"
|
||||||
|
|
||||||
|
if curl -sf -m 10 "$API/api/models/providers" -o /tmp/.st_prov 2>/dev/null; then
|
||||||
|
if grep -q ollama /tmp/.st_prov && grep -q openrouter /tmp/.st_prov; then
|
||||||
|
ok "API erreichbar, Provider ollama + openrouter verfuegbar"
|
||||||
|
else
|
||||||
|
fail "API erreichbar, aber Provider fehlen: $(cat /tmp/.st_prov)"
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
fail "API $API nicht erreichbar"
|
||||||
|
fi
|
||||||
|
rm -f /tmp/.st_prov
|
||||||
|
|
||||||
|
sec "Lokale Sprachdienste (vom Container aus)"
|
||||||
|
|
||||||
|
if inapp "$PY" -c "
|
||||||
|
import httpx,sys
|
||||||
|
r = httpx.post('http://host.docker.internal:8901/audio/speech',
|
||||||
|
json={'model':'chatterbox','voice':'default','input':'Test.','response_format':'wav'}, timeout=180)
|
||||||
|
sys.exit(0 if r.status_code == 200 and len(r.content) > 1000 else 1)"; then
|
||||||
|
ok "TTS antwortet mit Audio (host.docker.internal:8901)"
|
||||||
|
else
|
||||||
|
fail "TTS liefert kein Audio — Dienst tot (ConnectError) oder VRAM voll (HTTP 500)?
|
||||||
|
systemctl --user status open-notebook-tts; journalctl --user -u open-notebook-tts -n 30"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if inapp "$PY" -c "
|
||||||
|
import httpx,sys
|
||||||
|
r = httpx.get('http://host.docker.internal:8902/health', timeout=30)
|
||||||
|
sys.exit(0 if r.status_code == 200 else 1)"; then
|
||||||
|
ok "STT erreichbar (host.docker.internal:8902)"
|
||||||
|
else
|
||||||
|
fail "STT nicht erreichbar — systemctl --user status open-notebook-stt"
|
||||||
|
fi
|
||||||
|
|
||||||
|
sec "Annahmen unserer Anpassungen (brechen leise bei Image-Updates)"
|
||||||
|
|
||||||
|
# 1) prompts/podcast: Verzeichnis-Mount verdeckt das Image-Verzeichnis komplett.
|
||||||
|
# Kommt upstream eine dritte Vorlage dazu, waere sie unsichtbar.
|
||||||
|
img_tpl="$(docker run --rm --entrypoint ls "$IMAGE" /app/prompts/podcast 2>/dev/null | sort | tr '\n' ' ')"
|
||||||
|
our_tpl="$(ls prompts/podcast | sort | tr '\n' ' ')"
|
||||||
|
if [ "$img_tpl" = "$our_tpl" ]; then
|
||||||
|
ok "prompts/podcast: Dateiliste deckt sich mit dem Image ($our_tpl)"
|
||||||
|
else
|
||||||
|
fail "prompts/podcast weicht ab — unser Mount verdeckt Image-Dateien!
|
||||||
|
Image: $img_tpl
|
||||||
|
wir : $our_tpl"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# 2) Unsere Patches in den Vorlagen sind noch drin.
|
||||||
|
if grep -q '{{ language }}' prompts/podcast/outline.jinja prompts/podcast/transcript.jinja &&
|
||||||
|
head -1 prompts/podcast/outline.jinja | grep -q '/no_think'; then
|
||||||
|
ok "Vorlagen enthalten Sprachanweisung ({{ language }}) und /no_think"
|
||||||
|
else
|
||||||
|
fail "Vorlagen-Patches fehlen — Podcasts kaemen wieder auf Englisch / mit kaputtem JSON"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# 3) podcast_creator reicht `language` ueberhaupt noch in die Vorlagen durch.
|
||||||
|
if inapp grep -q "language" /app/.venv/lib/python3.12/site-packages/podcast_creator/nodes.py; then
|
||||||
|
ok "podcast_creator kennt weiterhin ein language-Feld"
|
||||||
|
else
|
||||||
|
fail "podcast_creator reicht kein language mehr durch — Vorlagen-Patch wirkungslos"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# 4) TTS_BATCH_SIZE existiert noch als Stellschraube (sonst wieder 5 parallele Clips -> OOM).
|
||||||
|
if inapp grep -q "TTS_BATCH_SIZE" /app/.venv/lib/python3.12/site-packages/podcast_creator/nodes.py; then
|
||||||
|
ok "podcast_creator liest weiterhin TTS_BATCH_SIZE"
|
||||||
|
else
|
||||||
|
fail "TTS_BATCH_SIZE wird nicht mehr gelesen — Gefahr paralleler Clips (CUDA OOM)"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# 5) content-core sucht YouTube-Untertitel weiterhin ueber preferred_languages.
|
||||||
|
if inapp grep -q "preferred_languages" /app/.venv/lib/python3.12/site-packages/content_core/processors/youtube.py; then
|
||||||
|
ok "content-core nutzt weiterhin preferred_languages (deutsche Untertitel)"
|
||||||
|
else
|
||||||
|
fail "content-core sucht Untertitel anders — config/content_core.yaml pruefen"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# 6) content-core baut das STT-Modell weiterhin ohne base_url (deshalb die Env-Var).
|
||||||
|
if inapp grep -q "OPENAI_COMPATIBLE_BASE_URL" /app/.venv/lib/python3.12/site-packages/esperanto/providers/stt/openai_compatible.py; then
|
||||||
|
ok "esperanto akzeptiert weiterhin OPENAI_COMPATIBLE_BASE_URL_STT (Audio-Transkription)"
|
||||||
|
else
|
||||||
|
fail "esperanto liest die Env-Var nicht mehr — Audio-Quellen fielen auf OpenAI zurueck"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# 7) CCORE_CONFIG_PATH ersetzt die Config komplett. Bringt ein Update neue
|
||||||
|
# Top-Level-Schluessel mit, frieren wir die alten ein — hier faellt das auf.
|
||||||
|
img_keys="$(docker run --rm --entrypoint "$PY" -e CCORE_CONFIG_PATH= "$IMAGE" \
|
||||||
|
-c "from content_core.config import load_config; print(' '.join(sorted(load_config())))" 2>/dev/null)"
|
||||||
|
our_keys="$(inapp "$PY" -c "
|
||||||
|
import yaml; print(' '.join(sorted(yaml.safe_load(open('/app/config/content_core.yaml')))))")"
|
||||||
|
missing="$(comm -23 <(tr ' ' '\n' <<<"$img_keys" | sort -u) <(tr ' ' '\n' <<<"$our_keys" | sort -u) | tr '\n' ' ')"
|
||||||
|
if [ -z "${missing// /}" ] && [ -n "$img_keys" ]; then
|
||||||
|
ok "config/content_core.yaml enthaelt alle Default-Schluessel des Images"
|
||||||
|
else
|
||||||
|
fail "config/content_core.yaml fehlen Schluessel aus dem Image: ${missing:-(Image-Defaults nicht lesbar)}
|
||||||
|
Neu erzeugen — siehe Kopfkommentar in config/content_core.yaml"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# 8) Episode-Profile haben eine Sprache (sonst wieder englische Podcasts).
|
||||||
|
if curl -sf -m 10 "$API/api/episode-profiles" | python3 -c "
|
||||||
|
import json,sys
|
||||||
|
p=json.load(sys.stdin)
|
||||||
|
sys.exit(0 if p and all(x.get('language') for x in p) else 1)" 2>/dev/null; then
|
||||||
|
ok "alle Episode-Profile haben ein language-Feld gesetzt"
|
||||||
|
else
|
||||||
|
fail "mindestens ein Episode-Profil ohne language — Podcast kaeme auf Englisch"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ "${SKIP_NET:-0}" != "1" ]; then
|
||||||
|
sec "Ende-zu-Ende (braucht Internet)"
|
||||||
|
if inapp "$PY" -c "
|
||||||
|
import asyncio,sys
|
||||||
|
from content_core import extract_content
|
||||||
|
async def m():
|
||||||
|
r = await extract_content({'url':'https://www.youtube.com/watch?v=hzxiegk9QAg'})
|
||||||
|
sys.exit(0 if len((r.content or '')) > 1000 else 1)
|
||||||
|
asyncio.run(m())"; then
|
||||||
|
ok "YouTube-Quelle liefert Transkript (deutsche Untertitel werden gefunden)"
|
||||||
|
else
|
||||||
|
fail "YouTube-Extraktion liefert keinen Text — CCORE_CONFIG_PATH/Sprachen pruefen"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
sec "Ergebnis"
|
||||||
|
if [ "$fails" -eq 0 ]; then
|
||||||
|
echo " Alles gruen."
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
echo " $fails Test(s) fehlgeschlagen."
|
||||||
|
exit 1
|
||||||
104
scripts/update_stack.sh
Executable file
104
scripts/update_stack.sh
Executable file
|
|
@ -0,0 +1,104 @@
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
# Aktualisiert Open Notebook auf das neueste veroeffentlichte Image — mit Rueckfallpunkt.
|
||||||
|
#
|
||||||
|
# Ablauf:
|
||||||
|
# 1. Neuen Digest aus der Registry holen (nichts zu tun, wenn schon aktuell).
|
||||||
|
# 2. Stack stoppen und Daten sichern (surreal_data + notebook_data + docker-compose.yml).
|
||||||
|
# 3. Digest in docker-compose.yml eintragen, Stack starten.
|
||||||
|
# 4. Rauchtest (scripts/smoke_test.sh).
|
||||||
|
# 5. Rot -> alles zurueckrollen (Daten UND Compose-Datei). Gruen -> fertig.
|
||||||
|
#
|
||||||
|
# Der Rollback der Daten ist der Grund fuer das Backup: Open Notebook migriert die
|
||||||
|
# SurrealDB beim Start, und ein Downgrade der Anwendung kann eine migrierte Datenbank
|
||||||
|
# nicht wieder lesen. Ohne Backup waere ein missgluecktes Update eine Einbahnstrasse.
|
||||||
|
#
|
||||||
|
# ./scripts/update_stack.sh # auf neuestes Image
|
||||||
|
# ./scripts/update_stack.sh --dry-run # nur zeigen, was passieren wuerde
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||||
|
cd "$REPO"
|
||||||
|
|
||||||
|
TAG="lfnovo/open_notebook:v1-latest"
|
||||||
|
BACKUP_DIR="$REPO/backups"
|
||||||
|
KEEP_BACKUPS=3
|
||||||
|
DRY_RUN=0
|
||||||
|
[ "${1:-}" = "--dry-run" ] && DRY_RUN=1
|
||||||
|
|
||||||
|
say() { printf '\n\033[1m%s\033[0m\n' "$1"; }
|
||||||
|
|
||||||
|
# Gezielt aus der open_notebook-Zeile. Ein blindes grep auf den ersten Digest der Datei
|
||||||
|
# erwischt surrealdb (steht weiter oben) — das sed weiter unten wuerde dann den falschen
|
||||||
|
# Digest ersetzen und die Datenbank austauschen statt der Anwendung.
|
||||||
|
pinned="$(grep -oP 'lfnovo/open_notebook\S*@\Ksha256:[0-9a-f]+' docker-compose.yml | head -1)"
|
||||||
|
[ -n "$pinned" ] || { echo "FEHLER: kein open_notebook-Digest in docker-compose.yml gepinnt." >&2; exit 1; }
|
||||||
|
|
||||||
|
say "1/5 Registry abfragen"
|
||||||
|
remote="$(docker buildx imagetools inspect "$TAG" --format '{{.Manifest.Digest}}')"
|
||||||
|
echo " gepinnt: $pinned"
|
||||||
|
echo " neu : $remote"
|
||||||
|
if [ "$remote" = "$pinned" ]; then
|
||||||
|
echo " Bereits aktuell — nichts zu tun."
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
if [ "$DRY_RUN" = 1 ]; then
|
||||||
|
echo " (--dry-run: hier wuerde das Update starten)"
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
ts="$(date +%Y%m%d-%H%M%S)"
|
||||||
|
backup="$BACKUP_DIR/$ts"
|
||||||
|
mkdir -p "$backup"
|
||||||
|
|
||||||
|
say "2/5 Stack stoppen und sichern -> backups/$ts"
|
||||||
|
docker compose down
|
||||||
|
cp docker-compose.yml "$backup/docker-compose.yml"
|
||||||
|
# Die Datenverzeichnisse gehoeren root (der Container schreibt als root), deshalb wird
|
||||||
|
# im Container gepackt statt auf dem Host — spart sudo.
|
||||||
|
docker run --rm -v "$REPO:/work" -w /work --entrypoint tar \
|
||||||
|
"lfnovo/open_notebook@$pinned" -czf "backups/$ts/data.tgz" surreal_data notebook_data
|
||||||
|
echo " $(du -sh "$backup/data.tgz" | cut -f1) gesichert"
|
||||||
|
|
||||||
|
restore() {
|
||||||
|
say "ROLLBACK"
|
||||||
|
docker compose down || true
|
||||||
|
cp "$backup/docker-compose.yml" docker-compose.yml
|
||||||
|
docker run --rm -v "$REPO:/work" -w /work --entrypoint sh \
|
||||||
|
"lfnovo/open_notebook@$pinned" -c \
|
||||||
|
"rm -rf surreal_data notebook_data && tar -xzf backups/$ts/data.tgz"
|
||||||
|
docker compose up -d
|
||||||
|
echo " Alter Stand ($pinned) ist wiederhergestellt, Daten aus backups/$ts."
|
||||||
|
echo " Das neue Image bleibt lokal liegen — Ursache pruefen, dann erneut versuchen."
|
||||||
|
}
|
||||||
|
|
||||||
|
say "3/5 Auf neues Image umstellen"
|
||||||
|
sed -i "s|@$pinned|@$remote|" docker-compose.yml
|
||||||
|
docker compose up -d
|
||||||
|
echo " gestartet, warte auf die API..."
|
||||||
|
for _ in $(seq 1 30); do
|
||||||
|
curl -sf -m 3 http://127.0.0.1:5055/api/models/providers >/dev/null 2>&1 && break
|
||||||
|
sleep 5
|
||||||
|
done
|
||||||
|
|
||||||
|
say "4/5 Rauchtest"
|
||||||
|
if ./scripts/smoke_test.sh; then
|
||||||
|
say "5/5 Fertig"
|
||||||
|
newver="$(docker compose exec -T open_notebook grep -m1 '^version' /app/pyproject.toml |
|
||||||
|
grep -oP '(?<=")[^"]+' || echo '?')"
|
||||||
|
echo " Update auf $newver erfolgreich, Rauchtest gruen."
|
||||||
|
echo " Backup: backups/$ts (bleibt liegen)"
|
||||||
|
echo
|
||||||
|
echo " Bitte den neuen Digest committen:"
|
||||||
|
echo " git add docker-compose.yml && git commit -m 'Update Open Notebook auf $newver'"
|
||||||
|
else
|
||||||
|
echo
|
||||||
|
echo " Rauchtest ROT — das Update wird zurueckgenommen."
|
||||||
|
restore
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Alte Backups aufraeumen (Platte ist knapp).
|
||||||
|
ls -1dt "$BACKUP_DIR"/*/ 2>/dev/null | tail -n +$((KEEP_BACKUPS + 1)) | while read -r old; do
|
||||||
|
echo " raeume altes Backup weg: $(basename "$old")"
|
||||||
|
rm -rf "$old"
|
||||||
|
done
|
||||||
Loading…
Add table
Add a link
Reference in a new issue