open_notebook/scripts/update_stack.sh

120 lines
4.6 KiB
Bash
Raw Permalink Normal View History

Pin images by digest; add update, smoke-test and status scripts Goal: at every session start, run exactly the application the repo says — and be able to move to a new upstream release deliberately, with a way back. The old setup (v1-latest + pull_policy: always) was unreliable in both directions. `docker compose up -d` silently pulled a new application, including an irreversible SurrealDB schema migration, while a reboot (restart: always) or `docker compose start` kept running the old image without pulling anything. The running version was effectively unpredictable. Note v1-latest tracks releases, not main: the current image (v1.10.0, 2026-06-18) IS the latest release — the repo being "ahead" is unreleased code, which is explicitly not wanted here. So this is about guaranteeing and detecting, not catching up. - docker-compose.yml: both images pinned by digest, pull_policy: missing. - scripts/check_updates.sh: reports running version/digest vs the latest release and registry digest. Changes nothing; meant for the session-start ritual. - scripts/update_stack.sh: resolve new digest -> stop -> back up surreal_data, notebook_data and docker-compose.yml -> repin -> start -> smoke test -> roll back data AND compose file if the smoke test fails. The data backup is the point: the app migrates the DB on startup and an older app cannot read a migrated DB, so a bad update would otherwise be a one-way door. tar runs inside a container because the data dirs are root-owned and the host user cannot restore over them. - scripts/smoke_test.sh: checks what actually breaks here, not just "does it start". Every local adaptation leans on upstream internals and can break silently: the prompts/podcast directory mount masks the image's directory (a new template upstream would be invisible), config/content_core.yaml freezes content-core's defaults (because CCORE_CONFIG_PATH replaces rather than merges), and the env-var workarounds depend on current content-core/esperanto/podcast_creator behaviour. So it verifies those assumptions explicitly, plus API, TTS audio, STT and a real YouTube transcript. Both scripts read the digest from the lfnovo/open_notebook line specifically. A naive "first sha256 in the file" grep matches surrealdb (listed first) — caught while testing: check_updates.sh falsely reported an update, and update_stack.sh would have rewritten the database image instead of the application. Verified: smoke test green against the current stack, and red (exit 1) when failures are injected (dead TTS port, removed template variable). Backup/restore mechanics exercised separately: 44 MB in 3.3s, restore yields 13 DB files and 91 notebook files. The update path itself cannot be exercised end to end until a newer image exists. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 14:12:31 +02:00
#!/usr/bin/env bash
# Aktualisiert Open Notebook auf das neueste veroeffentlichte Image — mit Rueckfallpunkt.
#
# Ablauf:
# 1. Neuen Digest aus der Registry holen (nichts zu tun, wenn schon aktuell).
# 2. Stack stoppen und Daten sichern (surreal_data + notebook_data + docker-compose.yml).
# 3. Digest in docker-compose.yml eintragen, Stack starten.
# 4. Rauchtest (scripts/smoke_test.sh).
# 5. Rot -> alles zurueckrollen (Daten UND Compose-Datei). Gruen -> fertig.
#
# Der Rollback der Daten ist der Grund fuer das Backup: Open Notebook migriert die
# SurrealDB beim Start, und ein Downgrade der Anwendung kann eine migrierte Datenbank
# nicht wieder lesen. Ohne Backup waere ein missgluecktes Update eine Einbahnstrasse.
#
# ./scripts/update_stack.sh # auf neuestes Image
# ./scripts/update_stack.sh --dry-run # nur zeigen, was passieren wuerde
set -euo pipefail
REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$REPO"
TAG="lfnovo/open_notebook:v1-latest"
BACKUP_DIR="$REPO/backups"
KEEP_BACKUPS=3
DRY_RUN=0
[ "${1:-}" = "--dry-run" ] && DRY_RUN=1
say() { printf '\n\033[1m%s\033[0m\n' "$1"; }
# Gezielt aus der open_notebook-Zeile. Ein blindes grep auf den ersten Digest der Datei
# erwischt surrealdb (steht weiter oben) — das sed weiter unten wuerde dann den falschen
# Digest ersetzen und die Datenbank austauschen statt der Anwendung.
pinned="$(grep -oP 'lfnovo/open_notebook\S*@\Ksha256:[0-9a-f]+' docker-compose.yml | head -1)"
[ -n "$pinned" ] || { echo "FEHLER: kein open_notebook-Digest in docker-compose.yml gepinnt." >&2; exit 1; }
say "1/5 Registry abfragen"
remote="$(docker buildx imagetools inspect "$TAG" --format '{{.Manifest.Digest}}')"
echo " gepinnt: $pinned"
echo " neu : $remote"
if [ "$remote" = "$pinned" ]; then
echo " Bereits aktuell — nichts zu tun."
exit 0
fi
if [ "$DRY_RUN" = 1 ]; then
echo " (--dry-run: hier wuerde das Update starten)"
exit 0
fi
# Ein laufender Podcast ueberlebt das Stoppen nicht: der Job ist weg, sein Status bleibt
# aber auf "running" stehen. Er wird nie fertig, blockiert prune_podcast_data.py und laesst
# sich nicht per /retry wiederholen (das will "failed"). Also vorher fragen, nicht nachher
# aufraeumen.
running_jobs="$(curl -sf -m 10 http://127.0.0.1:5055/api/podcasts/episodes 2>/dev/null |
python3 -c "
import json,sys
try: eps = json.load(sys.stdin)
except Exception: sys.exit(0)
print('; '.join(e['name'] for e in eps if e.get('job_status') in ('running','pending','submitted')))" 2>/dev/null)"
if [ -n "${running_jobs// /}" ]; then
echo
echo "ABBRUCH: Es laeuft gerade ein Podcast-Job: $running_jobs"
echo " Das Stoppen wuerde ihn abschiessen und als Karteileiche zuruecklassen."
echo " Bitte warten, bis er fertig ist, und danach erneut starten."
exit 1
fi
Pin images by digest; add update, smoke-test and status scripts Goal: at every session start, run exactly the application the repo says — and be able to move to a new upstream release deliberately, with a way back. The old setup (v1-latest + pull_policy: always) was unreliable in both directions. `docker compose up -d` silently pulled a new application, including an irreversible SurrealDB schema migration, while a reboot (restart: always) or `docker compose start` kept running the old image without pulling anything. The running version was effectively unpredictable. Note v1-latest tracks releases, not main: the current image (v1.10.0, 2026-06-18) IS the latest release — the repo being "ahead" is unreleased code, which is explicitly not wanted here. So this is about guaranteeing and detecting, not catching up. - docker-compose.yml: both images pinned by digest, pull_policy: missing. - scripts/check_updates.sh: reports running version/digest vs the latest release and registry digest. Changes nothing; meant for the session-start ritual. - scripts/update_stack.sh: resolve new digest -> stop -> back up surreal_data, notebook_data and docker-compose.yml -> repin -> start -> smoke test -> roll back data AND compose file if the smoke test fails. The data backup is the point: the app migrates the DB on startup and an older app cannot read a migrated DB, so a bad update would otherwise be a one-way door. tar runs inside a container because the data dirs are root-owned and the host user cannot restore over them. - scripts/smoke_test.sh: checks what actually breaks here, not just "does it start". Every local adaptation leans on upstream internals and can break silently: the prompts/podcast directory mount masks the image's directory (a new template upstream would be invisible), config/content_core.yaml freezes content-core's defaults (because CCORE_CONFIG_PATH replaces rather than merges), and the env-var workarounds depend on current content-core/esperanto/podcast_creator behaviour. So it verifies those assumptions explicitly, plus API, TTS audio, STT and a real YouTube transcript. Both scripts read the digest from the lfnovo/open_notebook line specifically. A naive "first sha256 in the file" grep matches surrealdb (listed first) — caught while testing: check_updates.sh falsely reported an update, and update_stack.sh would have rewritten the database image instead of the application. Verified: smoke test green against the current stack, and red (exit 1) when failures are injected (dead TTS port, removed template variable). Backup/restore mechanics exercised separately: 44 MB in 3.3s, restore yields 13 DB files and 91 notebook files. The update path itself cannot be exercised end to end until a newer image exists. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 14:12:31 +02:00
ts="$(date +%Y%m%d-%H%M%S)"
backup="$BACKUP_DIR/$ts"
mkdir -p "$backup"
say "2/5 Stack stoppen und sichern -> backups/$ts"
docker compose down
cp docker-compose.yml "$backup/docker-compose.yml"
Run containers as the host user instead of root Everything the containers wrote into the bind mounts (surreal_data, notebook_data) was owned by root, so the host user could not delete or back up his own podcast data — prune_podcast_data.py and update_stack.sh had to detour through `docker compose exec` for every rm and tar. That was not a requirement, just the default: the open_notebook image declares no USER, and the compose file even overrode surrealdb's own non-root user (65532) with `user: root`, under the comment "Required for bind mounts on Linux" — which is not true. Both services now run as user: "1000:1000". Two things this needs: - HOME=/tmp for open_notebook. Without it HOME resolves to "/" for a non-root uid, uv cannot create /.cache/uv, and api + worker exit 2 at startup. Verified by running the image as uid 1000 both ways. - The data directories must be owned by that uid. Existing data was adopted with a throwaway root container (chown -R), no sudo needed. Both scripts drop the container detour and operate on the host directly, which is simpler and now honest. smoke_test.sh gained two checks so a silent regression to root cannot go unnoticed: the container's uid must match the host user, and no foreign-owned files may exist under the data directories. Verified: containers run as uid 1000, new files land as dschlueter and are deletable without sudo, SurrealDB writes as 1000, a source can be created, embedded and deleted through the API, and the full smoke test is green. Note this deviates from what the image expects (it assumes root), so it is exactly the kind of assumption an update can break — hence the smoke-test checks. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 15:14:47 +02:00
# Direkt auf dem Host, weil die Container als Host-User laufen (user: "1000:1000")
# und die Daten uns gehoeren. Liefen sie als root, scheiterte das an den Rechten.
tar -czf "$backup/data.tgz" surreal_data notebook_data
Pin images by digest; add update, smoke-test and status scripts Goal: at every session start, run exactly the application the repo says — and be able to move to a new upstream release deliberately, with a way back. The old setup (v1-latest + pull_policy: always) was unreliable in both directions. `docker compose up -d` silently pulled a new application, including an irreversible SurrealDB schema migration, while a reboot (restart: always) or `docker compose start` kept running the old image without pulling anything. The running version was effectively unpredictable. Note v1-latest tracks releases, not main: the current image (v1.10.0, 2026-06-18) IS the latest release — the repo being "ahead" is unreleased code, which is explicitly not wanted here. So this is about guaranteeing and detecting, not catching up. - docker-compose.yml: both images pinned by digest, pull_policy: missing. - scripts/check_updates.sh: reports running version/digest vs the latest release and registry digest. Changes nothing; meant for the session-start ritual. - scripts/update_stack.sh: resolve new digest -> stop -> back up surreal_data, notebook_data and docker-compose.yml -> repin -> start -> smoke test -> roll back data AND compose file if the smoke test fails. The data backup is the point: the app migrates the DB on startup and an older app cannot read a migrated DB, so a bad update would otherwise be a one-way door. tar runs inside a container because the data dirs are root-owned and the host user cannot restore over them. - scripts/smoke_test.sh: checks what actually breaks here, not just "does it start". Every local adaptation leans on upstream internals and can break silently: the prompts/podcast directory mount masks the image's directory (a new template upstream would be invisible), config/content_core.yaml freezes content-core's defaults (because CCORE_CONFIG_PATH replaces rather than merges), and the env-var workarounds depend on current content-core/esperanto/podcast_creator behaviour. So it verifies those assumptions explicitly, plus API, TTS audio, STT and a real YouTube transcript. Both scripts read the digest from the lfnovo/open_notebook line specifically. A naive "first sha256 in the file" grep matches surrealdb (listed first) — caught while testing: check_updates.sh falsely reported an update, and update_stack.sh would have rewritten the database image instead of the application. Verified: smoke test green against the current stack, and red (exit 1) when failures are injected (dead TTS port, removed template variable). Backup/restore mechanics exercised separately: 44 MB in 3.3s, restore yields 13 DB files and 91 notebook files. The update path itself cannot be exercised end to end until a newer image exists. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 14:12:31 +02:00
echo " $(du -sh "$backup/data.tgz" | cut -f1) gesichert"
restore() {
say "ROLLBACK"
docker compose down || true
cp "$backup/docker-compose.yml" docker-compose.yml
Run containers as the host user instead of root Everything the containers wrote into the bind mounts (surreal_data, notebook_data) was owned by root, so the host user could not delete or back up his own podcast data — prune_podcast_data.py and update_stack.sh had to detour through `docker compose exec` for every rm and tar. That was not a requirement, just the default: the open_notebook image declares no USER, and the compose file even overrode surrealdb's own non-root user (65532) with `user: root`, under the comment "Required for bind mounts on Linux" — which is not true. Both services now run as user: "1000:1000". Two things this needs: - HOME=/tmp for open_notebook. Without it HOME resolves to "/" for a non-root uid, uv cannot create /.cache/uv, and api + worker exit 2 at startup. Verified by running the image as uid 1000 both ways. - The data directories must be owned by that uid. Existing data was adopted with a throwaway root container (chown -R), no sudo needed. Both scripts drop the container detour and operate on the host directly, which is simpler and now honest. smoke_test.sh gained two checks so a silent regression to root cannot go unnoticed: the container's uid must match the host user, and no foreign-owned files may exist under the data directories. Verified: containers run as uid 1000, new files land as dschlueter and are deletable without sudo, SurrealDB writes as 1000, a source can be created, embedded and deleted through the API, and the full smoke test is green. Note this deviates from what the image expects (it assumes root), so it is exactly the kind of assumption an update can break — hence the smoke-test checks. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 15:14:47 +02:00
rm -rf surreal_data notebook_data
tar -xzf "$backup/data.tgz"
Pin images by digest; add update, smoke-test and status scripts Goal: at every session start, run exactly the application the repo says — and be able to move to a new upstream release deliberately, with a way back. The old setup (v1-latest + pull_policy: always) was unreliable in both directions. `docker compose up -d` silently pulled a new application, including an irreversible SurrealDB schema migration, while a reboot (restart: always) or `docker compose start` kept running the old image without pulling anything. The running version was effectively unpredictable. Note v1-latest tracks releases, not main: the current image (v1.10.0, 2026-06-18) IS the latest release — the repo being "ahead" is unreleased code, which is explicitly not wanted here. So this is about guaranteeing and detecting, not catching up. - docker-compose.yml: both images pinned by digest, pull_policy: missing. - scripts/check_updates.sh: reports running version/digest vs the latest release and registry digest. Changes nothing; meant for the session-start ritual. - scripts/update_stack.sh: resolve new digest -> stop -> back up surreal_data, notebook_data and docker-compose.yml -> repin -> start -> smoke test -> roll back data AND compose file if the smoke test fails. The data backup is the point: the app migrates the DB on startup and an older app cannot read a migrated DB, so a bad update would otherwise be a one-way door. tar runs inside a container because the data dirs are root-owned and the host user cannot restore over them. - scripts/smoke_test.sh: checks what actually breaks here, not just "does it start". Every local adaptation leans on upstream internals and can break silently: the prompts/podcast directory mount masks the image's directory (a new template upstream would be invisible), config/content_core.yaml freezes content-core's defaults (because CCORE_CONFIG_PATH replaces rather than merges), and the env-var workarounds depend on current content-core/esperanto/podcast_creator behaviour. So it verifies those assumptions explicitly, plus API, TTS audio, STT and a real YouTube transcript. Both scripts read the digest from the lfnovo/open_notebook line specifically. A naive "first sha256 in the file" grep matches surrealdb (listed first) — caught while testing: check_updates.sh falsely reported an update, and update_stack.sh would have rewritten the database image instead of the application. Verified: smoke test green against the current stack, and red (exit 1) when failures are injected (dead TTS port, removed template variable). Backup/restore mechanics exercised separately: 44 MB in 3.3s, restore yields 13 DB files and 91 notebook files. The update path itself cannot be exercised end to end until a newer image exists. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 14:12:31 +02:00
docker compose up -d
echo " Alter Stand ($pinned) ist wiederhergestellt, Daten aus backups/$ts."
echo " Das neue Image bleibt lokal liegen — Ursache pruefen, dann erneut versuchen."
}
say "3/5 Auf neues Image umstellen"
sed -i "s|@$pinned|@$remote|" docker-compose.yml
docker compose up -d
echo " gestartet, warte auf die API..."
for _ in $(seq 1 30); do
curl -sf -m 3 http://127.0.0.1:5055/api/models/providers >/dev/null 2>&1 && break
sleep 5
done
say "4/5 Rauchtest"
if ./scripts/smoke_test.sh; then
say "5/5 Fertig"
newver="$(docker compose exec -T open_notebook grep -m1 '^version' /app/pyproject.toml |
grep -oP '(?<=")[^"]+' || echo '?')"
echo " Update auf $newver erfolgreich, Rauchtest gruen."
echo " Backup: backups/$ts (bleibt liegen)"
echo
echo " Bitte den neuen Digest committen:"
echo " git add docker-compose.yml && git commit -m 'Update Open Notebook auf $newver'"
else
echo
echo " Rauchtest ROT — das Update wird zurueckgenommen."
restore
exit 1
fi
# Alte Backups aufraeumen (Platte ist knapp).
ls -1dt "$BACKUP_DIR"/*/ 2>/dev/null | tail -n +$((KEEP_BACKUPS + 1)) | while read -r old; do
echo " raeume altes Backup weg: $(basename "$old")"
rm -rf "$old"
done