Run containers as the host user instead of root

Everything the containers wrote into the bind mounts (surreal_data, notebook_data)
was owned by root, so the host user could not delete or back up his own podcast
data — prune_podcast_data.py and update_stack.sh had to detour through
`docker compose exec` for every rm and tar.

That was not a requirement, just the default: the open_notebook image declares no
USER, and the compose file even overrode surrealdb's own non-root user (65532)
with `user: root`, under the comment "Required for bind mounts on Linux" — which
is not true.

Both services now run as user: "1000:1000". Two things this needs:

- HOME=/tmp for open_notebook. Without it HOME resolves to "/" for a non-root uid,
  uv cannot create /.cache/uv, and api + worker exit 2 at startup. Verified by
  running the image as uid 1000 both ways.
- The data directories must be owned by that uid. Existing data was adopted with a
  throwaway root container (chown -R), no sudo needed.

Both scripts drop the container detour and operate on the host directly, which is
simpler and now honest. smoke_test.sh gained two checks so a silent regression to
root cannot go unnoticed: the container's uid must match the host user, and no
foreign-owned files may exist under the data directories.

Verified: containers run as uid 1000, new files land as dschlueter and are
deletable without sudo, SurrealDB writes as 1000, a source can be created,
embedded and deleted through the API, and the full smoke test is green.

Note this deviates from what the image expects (it assumes root), so it is exactly
the kind of assumption an update can break — hence the smoke-test checks.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

CLAUDE.md

@@ -46,13 +46,40 @@ Updating (`scripts/update_stack.sh`): resolve the new digest → stop → back u
 `smoke_test.sh` → **roll back data and compose file if the smoke test fails**. The data backup is
 the point: the app migrates the database on startup, and an older application generally cannot
 read a migrated database, so a bad update would otherwise be a one-way door. Backup/restore runs
-`tar` **inside a container** because the data directories are root-owned (the container writes as
-root) and the host user cannot restore over them.
+plain `tar` on the host, which works because the containers run as the host user (see *Container
+user* below).
 
 Both scripts read the digest from the `lfnovo/open_notebook` line specifically — a naive "first
 sha256 in the file" grep picks up **surrealdb** (it is listed first), which would make the update
 script rewrite the database image instead of the application.
 
+## Container user
+
+Both services run as the **host user** (`user: "1000:1000"`), not as root, and `surreal_data/` +
+`notebook_data/` are owned by that uid.
+
+Default would be root: the `open_notebook` image declares no `USER`, and the compose file used to
+override surrealdb's own non-root user (`65532`) with `user: root` under the (wrong) comment
+"required for bind mounts on Linux". Everything a root container writes into a bind mount is
+root-owned, so the host user could not delete or back up its own podcast data — every cleanup had
+to detour through `docker compose exec`.
+
+Two things this needs, and both are load-bearing:
+
+- **`HOME=/tmp` for `open_notebook`.** Without it, `HOME` resolves to `/` for a non-root uid, `uv`
+  fails to create `/.cache/uv` (permission denied), and api + worker die at startup with exit 2.
+  The failure is immediate and loud, but the cause is not obvious from the message.
+- **The data directories must already be owned by that uid.** On a fresh checkout `mkdir -p
+  surreal_data notebook_data` is enough. To adopt existing root-owned data without `sudo`:
+  ```bash
+  docker run --rm -u 0 -v "$PWD:/work" -w /work --entrypoint chown \
+    lfnovo/open_notebook:v1-latest -R 1000:1000 surreal_data notebook_data
+  ```
+
+Note this is a deviation from what the image expects (it assumes root), so it is exactly the kind of
+thing an update can break. `smoke_test.sh` therefore asserts the container's uid matches the host
+user and that no foreign-owned files exist under the data directories.
+
 ### Why a smoke test, and not just "does it start"
 
 Every local adaptation in this repo leans on upstream internals, and an update can break them
@@ -142,8 +169,9 @@ image updates (the same trap the existing overlays already carry). Instead:
 apply) reconciles the episode list from the API against the directories on disk and removes orphans plus
 the `clips/` of completed episodes. Two guards matter — it aborts if any job is `running`/`pending`
 (a running job has no `audio_file` yet, so its directory is indistinguishable from an orphan) and it
-skips directories touched in the last 60 minutes. **Deletion runs inside the container** (`docker compose
-exec … rm -rf`), because the container writes as root and the host user can't remove those directories.
+skips directories touched in the last 60 minutes. Deletion is a plain `shutil.rmtree` on the host —
+possible because the containers run as the host user (see *Container user*); as root-owned data it
+would need a container detour.
 
 ### YouTube sources — German transcripts need `CCORE_CONFIG_PATH`
 

README.md

@@ -82,10 +82,16 @@ chmod 600 .env
 ### 2.3 Container starten
 
 ```bash
+mkdir -p surreal_data notebook_data   # müssen dem eigenen User gehören
 docker compose up -d
 docker compose ps        # beide Container sollten "Up" sein
 ```
 
+Die Container laufen als **Host-User** (`user: "1000:1000"`), nicht als root — sonst gehörten alle
+Daten, die sie schreiben (Notebooks, Podcasts, Datenbank), root, und Sie könnten sie ohne `sudo`
+weder löschen noch sichern. Bei abweichender UID/GID (`id -u`, `id -g`) den Eintrag in
+`docker-compose.yml` anpassen.
+
 Die Images sind in `docker-compose.yml` auf einen **Digest gepinnt** (`pull_policy: missing`) —
 `up -d` lädt sie beim ersten Mal und danach nie wieder unbemerkt eine andere Version. Aktualisiert
 wird bewusst, siehe [Abschnitt 4](#4-updates).

docker-compose.yml

@@ -17,7 +17,10 @@ services:
     restart: always
     pull_policy: missing
     command: start --log info --user ${SURREAL_USER:-root} --pass ${SURREAL_PASSWORD:-root} rocksdb:/mydata/mydatabase.db
-    user: root  # Required for bind mounts on Linux
+    # Als Host-User laufen, nicht als root: sonst gehoeren alle Dateien in
+    # ./surreal_data root und der Host-User kann sie weder loeschen noch zurueckspielen.
+    # (Die Verzeichnisse gehoeren 1000:1000 — siehe Kommentar bei open_notebook.)
+    user: "1000:1000"
     environment:
       - SURREAL_EXPERIMENTAL_GRAPHQL=true
     ports:
@@ -31,12 +34,22 @@ services:
     image: lfnovo/open_notebook:v1-latest@sha256:c8112fbd4b8fee7f2a20d3bdbea24e7d72267acfc990b803fd0ff4de30899b57
     restart: always
     pull_policy: missing
+    # Als Host-User (dschlueter, 1000:1000) laufen statt als root. Das Image definiert
+    # keinen USER, liefe also als root — und alles, was es nach ./notebook_data schreibt,
+    # gehoerte dann root: der Host-User koennte Podcast-Daten weder loeschen noch sichern.
+    # Voraussetzung: surreal_data/ und notebook_data/ gehoeren 1000:1000. Bei einem
+    # Neuaufbau anlegen mit:  mkdir -p surreal_data notebook_data
+    user: "1000:1000"
     depends_on:
       - surrealdb
     ports:
       - "127.0.0.1:8502:8502"
       - "127.0.0.1:5055:5055"
     environment:
+      # Ohne HOME zeigt es bei Nicht-Root auf "/", und uv scheitert beim Anlegen
+      # seines Caches (/.cache/uv, Permission denied) -> api/worker starten nicht.
+      # /tmp ist im Container schreibbar; der Cache ist ohnehin fluechtig.
+      - HOME=/tmp
       - OPEN_NOTEBOOK_ENCRYPTION_KEY=${OPEN_NOTEBOOK_ENCRYPTION_KEY}
       - SURREAL_URL=ws://surrealdb:8000/rpc
       - SURREAL_USER=${SURREAL_USER:-root}

scripts/prune_podcast_data.py

@@ -22,7 +22,7 @@ from __future__ import annotations
 
 import argparse
 import json
-import subprocess
+import shutil
 import sys
 import time
 import urllib.error
@@ -69,28 +69,25 @@ def to_host_path(audio_file: str) -> Path | None:
     return p if p.is_absolute() and p.exists() else None
 
 
-def to_container_path(host_path: Path) -> str:
-    """Pfad auf dem Host -> Container-Pfad (Umkehrung von to_host_path)."""
-    rel = host_path.resolve().relative_to(HOST_PREFIX.resolve())
-    return CONTAINER_PREFIX + str(rel)
+def delete_dirs(paths: list[Path]) -> None:
+    """Loescht die Verzeichnisse direkt auf dem Host.
 
-
-def delete_in_container(paths: list[Path]) -> None:
-    """Loescht im Container statt auf dem Host.
-
-    Der open_notebook-Container laeuft als root und legt die Episodenordner
-    entsprechend root:root an — der Host-User darf sie nicht entfernen. Statt
-    dafuer sudo zu verlangen, loeschen wir dort, wo die Rechte ohnehin stimmen.
+    Das geht, weil die Container als Host-User laufen (user: "1000:1000" in
+    docker-compose.yml) und die Episodenordner deshalb uns gehoeren. Liefen sie als
+    root — der Default, wenn man den user-Eintrag entfernt — schlaegt das hier mit
+    PermissionError fehl; dann gehoert der Eintrag zurueck in die Compose-Datei.
     """
-    targets = [to_container_path(p) for p in paths]
-    guard = CONTAINER_PREFIX + "podcasts/episodes/"
-    for t in targets:  # Sicherheitsnetz gegen Pfade ausserhalb der Episoden.
-        if not t.startswith(guard):
-            die(f"Abbruch: {t} liegt ausserhalb von {guard}")
-    subprocess.run(
-        ["docker", "compose", "exec", "-T", "open_notebook", "rm", "-rf", "--", *targets],
-        cwd=REPO, check=True, capture_output=True,
-    )
+    guard = (EPISODES_DIR).resolve()
+    for p in paths:  # Sicherheitsnetz gegen Pfade ausserhalb der Episoden.
+        if guard not in p.resolve().parents:
+            die(f"Abbruch: {p} liegt ausserhalb von {guard}")
+    for p in paths:
+        try:
+            shutil.rmtree(p)
+        except PermissionError as e:
+            die(f"Keine Berechtigung fuer {p} ({e}).\n"
+                "Gehoeren die Daten root? Dann laeuft der Container als root — pruefe\n"
+                "den Eintrag user: \"1000:1000\" in docker-compose.yml.")
 
 
 def dir_size_mb(path: Path) -> float:
@@ -185,13 +182,7 @@ def main() -> int:
         return 0
 
     victims = [d for d, _ in orphans] + [c for c, _, _ in clip_dirs]
-    try:
-        delete_in_container(victims)
-    except FileNotFoundError:
-        die("docker nicht gefunden — das Loeschen laeuft im Container "
-            "(die Ordner gehoeren root).")
-    except subprocess.CalledProcessError as e:
-        die(f"Loeschen im Container fehlgeschlagen: {e.stderr.decode().strip()}")
+    delete_dirs(victims)
 
     for d, _ in orphans:
         print(f"geloescht: {d.name}")

scripts/smoke_test.sh

@@ -41,6 +41,26 @@ else
 fi
 rm -f /tmp/.st_prov
 
+# Laufen die Container als Host-User? Sonst gehoeren alle neu geschriebenen Daten
+# root, und Aufraeumen/Sichern auf dem Host scheitert an den Rechten (prune_podcast_data.py,
+# update_stack.sh verlassen sich darauf).
+uid="$(id -u)"
+if [ "$(inapp id -u | tr -d '\r')" = "$uid" ]; then
+  ok "open_notebook laeuft als Host-User (uid $uid), nicht als root"
+else
+  fail "open_notebook laeuft NICHT als uid $uid — neue Dateien gehoerten root.
+         user: \"$uid:$(id -g)\" in docker-compose.yml pruefen."
+fi
+
+foreign="$(find surreal_data notebook_data ! -user "$USER" 2>/dev/null | wc -l)"
+if [ "$foreign" -eq 0 ]; then
+  ok "keine fremd-eigenen Dateien in surreal_data/ und notebook_data/"
+else
+  fail "$foreign Datei(en) in surreal_data/notebook_data gehoeren nicht $USER —
+         uebernehmen mit: docker run --rm -u 0 -v \"\$PWD:/work\" -w /work \\
+           --entrypoint chown \$IMAGE -R $uid:$(id -g) surreal_data notebook_data"
+fi
+
 sec "Lokale Sprachdienste (vom Container aus)"
 
 if inapp "$PY" -c "

scripts/update_stack.sh

@@ -53,19 +53,17 @@ mkdir -p "$backup"
 say "2/5  Stack stoppen und sichern -> backups/$ts"
 docker compose down
 cp docker-compose.yml "$backup/docker-compose.yml"
-# Die Datenverzeichnisse gehoeren root (der Container schreibt als root), deshalb wird
-# im Container gepackt statt auf dem Host — spart sudo.
-docker run --rm -v "$REPO:/work" -w /work --entrypoint tar \
-  "lfnovo/open_notebook@$pinned" -czf "backups/$ts/data.tgz" surreal_data notebook_data
+# Direkt auf dem Host, weil die Container als Host-User laufen (user: "1000:1000")
+# und die Daten uns gehoeren. Liefen sie als root, scheiterte das an den Rechten.
+tar -czf "$backup/data.tgz" surreal_data notebook_data
 echo "  $(du -sh "$backup/data.tgz" | cut -f1) gesichert"
 
 restore() {
   say "ROLLBACK"
   docker compose down || true
   cp "$backup/docker-compose.yml" docker-compose.yml
-  docker run --rm -v "$REPO:/work" -w /work --entrypoint sh \
-    "lfnovo/open_notebook@$pinned" -c \
-    "rm -rf surreal_data notebook_data && tar -xzf backups/$ts/data.tgz"
+  rm -rf surreal_data notebook_data
+  tar -xzf "$backup/data.tgz"
   docker compose up -d
   echo "  Alter Stand ($pinned) ist wiederhergestellt, Daten aus backups/$ts."
   echo "  Das neue Image bleibt lokal liegen — Ursache pruefen, dann erneut versuchen."