Compare commits
22 changed files with 21 additions and 1446 deletions
2
.gitignore
vendored
2
.gitignore
vendored
|
|
@ -12,7 +12,7 @@ htmlcov/
|
||||||
build/
|
build/
|
||||||
dist/
|
dist/
|
||||||
|
|
||||||
# QA-Läufe (Prompt_Kontrolle.md): erzeugte Daten von tools/qa/
|
# QA-Läufe (Prompt_Kontrolle.md)
|
||||||
.qa/
|
.qa/
|
||||||
.qa-output/
|
.qa-output/
|
||||||
.qa-fixtures/
|
.qa-fixtures/
|
||||||
|
|
|
||||||
|
|
@ -163,9 +163,8 @@ python3 epub2md.py python-buch.epub output/python-buch
|
||||||
```
|
```
|
||||||
|
|
||||||
Programmcode bleibt zeichengenau erhalten, inklusive Einrückung. Die Programmiersprache wird nur
|
Programmcode bleibt zeichengenau erhalten, inklusive Einrückung. Die Programmiersprache wird nur
|
||||||
eingetragen, wenn sie eindeutig erkennbar ist. Ist ein Buch erkennbar in **einer** Sprache geschrieben,
|
eingetragen, wenn sie eindeutig erkennbar ist. Ist das ganze Buch in **einer** Sprache geschrieben,
|
||||||
erhalten auch kurze Codeschnipsel dieses Label (etwa `char ch = 65;` in einem C-Buch). Programmausgaben
|
kannst du sie für alle übrigen Codeblöcke vorgeben:
|
||||||
bleiben ohne Label. Willst du wirklich alle übrigen Blöcke kennzeichnen, gib die Sprache vor:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
python3 epub2md.py python-buch.epub output/python-buch --code-language python
|
python3 epub2md.py python-buch.epub output/python-buch --code-language python
|
||||||
|
|
@ -187,9 +186,7 @@ python3 epub2md.py mathe.epub output/mathe
|
||||||
```
|
```
|
||||||
|
|
||||||
Formeln, die im E-Book als echte Formeln (MathML oder TeX) vorliegen, werden zu TeX: `$…$` im Text,
|
Formeln, die im E-Book als echte Formeln (MathML oder TeX) vorliegen, werden zu TeX: `$…$` im Text,
|
||||||
`$$…$$` als eigene Zeile. Viele Verlage legen das TeX einer Formel in den Alternativtext eines Formelbildes;
|
`$$…$$` als eigene Zeile. Formeln, die nur als Bild vorliegen, bleiben Bilder und erscheinen in
|
||||||
epub2md übernimmt es und passt Schreibweisen an, die Pandoc nicht versteht (z. B. `\"u` → `ü`), ohne den
|
|
||||||
Inhalt zu ändern. Formeln, die nur als Bild vorliegen, bleiben Bilder und erscheinen in
|
|
||||||
`diagnostics/warnings.md`.
|
`diagnostics/warnings.md`.
|
||||||
|
|
||||||
| Gewünschte Formeldarstellung | Option |
|
| Gewünschte Formeldarstellung | Option |
|
||||||
|
|
@ -327,7 +324,6 @@ P=~/Python_Programs/epub2md/pandoc # Pfad zu deinem epub2md-Ordner anpassen
|
||||||
| Webseite (HTML) | `pandoc meinbuch.md -s --lua-filter $P/book.lua --lua-filter $P/callouts.lua -o meinbuch.html` |
|
| Webseite (HTML) | `pandoc meinbuch.md -s --lua-filter $P/book.lua --lua-filter $P/callouts.lua -o meinbuch.html` |
|
||||||
| Webseite mit Formeln | wie HTML, zusätzlich `--mathml` |
|
| Webseite mit Formeln | wie HTML, zusätzlich `--mathml` |
|
||||||
| E-Book (EPUB) | `pandoc meinbuch.md --toc --lua-filter $P/callouts.lua -o meinbuch.epub` |
|
| E-Book (EPUB) | `pandoc meinbuch.md --toc --lua-filter $P/callouts.lua -o meinbuch.epub` |
|
||||||
| E-Book mit Formeln | wie EPUB, zusätzlich `--mathml` |
|
|
||||||
| Word | `pandoc meinbuch.md --toc --lua-filter $P/callouts.lua -o meinbuch.docx` |
|
| Word | `pandoc meinbuch.md --toc --lua-filter $P/callouts.lua -o meinbuch.docx` |
|
||||||
| LibreOffice | `pandoc meinbuch.md --toc --lua-filter $P/callouts.lua -o meinbuch.odt` |
|
| LibreOffice | `pandoc meinbuch.md --toc --lua-filter $P/callouts.lua -o meinbuch.odt` |
|
||||||
| PDF mit LaTeX (Bücher mit Formeln) | `pandoc meinbuch.md -s --mathml --lua-filter $P/book.lua --lua-filter $P/callouts.lua -o meinbuch.html && pandoc meinbuch.html --lua-filter $P/book.lua --lua-filter $P/callouts.lua --pdf-engine=xelatex -o meinbuch.pdf` |
|
| PDF mit LaTeX (Bücher mit Formeln) | `pandoc meinbuch.md -s --mathml --lua-filter $P/book.lua --lua-filter $P/callouts.lua -o meinbuch.html && pandoc meinbuch.html --lua-filter $P/book.lua --lua-filter $P/callouts.lua --pdf-engine=xelatex -o meinbuch.pdf` |
|
||||||
|
|
@ -347,9 +343,8 @@ Die drei Hilfsdateien:
|
||||||
Schreibweise nicht und druckt „[!TIP]“ sonst einfach mit. In HTML, EPUB und im PDF über weasyprint werden
|
Schreibweise nicht und druckt „[!TIP]“ sonst einfach mit. In HTML, EPUB und im PDF über weasyprint werden
|
||||||
daraus gerahmte, farbige Kästen mit Titel („Tipp: …“), im PDF über LaTeX und in Word/LibreOffice ein
|
daraus gerahmte, farbige Kästen mit Titel („Tipp: …“), im PDF über LaTeX und in Word/LibreOffice ein
|
||||||
eingerückter Block mit fettem Titel.
|
eingerückter Block mit fettem Titel.
|
||||||
- **`print.css`** liefert das Druck-Layout für weasyprint: A4, Seitenzahlen (nicht auf dem Cover), jedes
|
- **`print.css`** liefert das Druck-Layout für weasyprint: A4, Seitenzahlen, jedes Kapitel auf einer neuen
|
||||||
Kapitel auf einer neuen Seite, Inhaltsverzeichnis mit Seitenzahlen, Silbentrennung und umbrechende lange
|
Seite, Inhaltsverzeichnis mit Seitenzahlen, Silbentrennung und umbrechende lange Codezeilen. Ohne sie
|
||||||
Codezeilen. Ohne sie
|
|
||||||
übernimmt weasyprint das Bildschirmlayout, und lange Codezeilen werden am Seitenrand abgeschnitten.
|
übernimmt weasyprint das Bildschirmlayout, und lange Codezeilen werden am Seitenrand abgeschnitten.
|
||||||
|
|
||||||
Du musst die Befehle nicht abtippen: Die `README.md` im Ausgabeordner enthält sie fertig, mit allen Pfaden.
|
Du musst die Befehle nicht abtippen: Die `README.md` im Ausgabeordner enthält sie fertig, mit allen Pfaden.
|
||||||
|
|
@ -364,8 +359,7 @@ Du musst die Befehle nicht abtippen: Die `README.md` im Ausgabeordner enthält s
|
||||||
Kastenrahmen gehen dabei verloren; Kästen erscheinen als eingerückter Block mit fettem Titel. `book.lua`
|
Kastenrahmen gehen dabei verloren; Kästen erscheinen als eingerückter Block mit fettem Titel. `book.lua`
|
||||||
stellt dabei A4 ein und wählt die Schrift DejaVu, die auch Zeichen wie ∈, λ oder ≠ im Fließtext enthält.
|
stellt dabei A4 ein und wählt die Schrift DejaVu, die auch Zeichen wie ∈, λ oder ≠ im Fließtext enthält.
|
||||||
|
|
||||||
Bei Büchern mit Formeln für HTML und EPUB zusätzlich `--mathml` angeben, sonst bleiben Matrizen und
|
Bei Büchern mit Formeln für HTML zusätzlich `--mathml` angeben, sonst bleiben Matrizen als TeX-Text stehen.
|
||||||
ähnliche Formeln als TeX-Text stehen.
|
|
||||||
|
|
||||||
#### Direkt aus den Kapiteldateien
|
#### Direkt aus den Kapiteldateien
|
||||||
|
|
||||||
|
|
|
||||||
15
CHANGELOG.md
15
CHANGELOG.md
|
|
@ -5,21 +5,6 @@ Versionsnummern nach [Semantic Versioning](https://semver.org/lang/de/).
|
||||||
|
|
||||||
## [Unveröffentlicht]
|
## [Unveröffentlicht]
|
||||||
|
|
||||||
### Neu
|
|
||||||
|
|
||||||
- `tools/qa/`: QA-Skripte versioniert (CLI-Matrix mit 89 Fällen, Generator für Test-EPUBs, unabhängiger
|
|
||||||
Markdown-Validator, Wortvergleich gegen Datenverlust, Prüfung des Pandoc-Wegs für HTML, EPUB und PDF).
|
|
||||||
Beschreibung in `tools/qa/README.md`; erzeugte Daten landen in `.qa/`.
|
|
||||||
|
|
||||||
### Verbessert
|
|
||||||
|
|
||||||
- Sprachlabels für Codeblöcke (R1): Gedruckte Zeilennummern (`00 #include …`) verdecken die Sprache
|
|
||||||
nicht mehr. Python wird zusätzlich per `ast.parse` erkannt (Kontrollstrukturen, `def`/`class`, Import
|
|
||||||
mit Code) sowie an `>>>`-Sitzungen. Ein Vordurchlauf bestimmt die Sprache des Buches aus den sicher
|
|
||||||
erkannten Blöcken. Passt sie, reichen für kurze Schnipsel schwächere Signale (`detected_by:
|
|
||||||
"book-context"`); Programmausgaben bleiben ohne Label. Ohne Label waren vorher im C-Buch 518 von 574
|
|
||||||
Blöcken, jetzt 132 (fast nur Ausgaben). Im Python-Buch waren es 492 von 747, jetzt 326 (Ausgaben).
|
|
||||||
|
|
||||||
## [0.3.0] – 2026-10-06
|
## [0.3.0] – 2026-10-06
|
||||||
|
|
||||||
### Neu
|
### Neu
|
||||||
|
|
|
||||||
40
README.md
40
README.md
|
|
@ -158,9 +158,9 @@ output/mein-buch/
|
||||||
## Wie Kapitel gebildet werden
|
## Wie Kapitel gebildet werden
|
||||||
|
|
||||||
1. Die **OPF-Spine** bestimmt die Lesereihenfolge (nicht Dateinamen, nicht das Inhaltsverzeichnis).
|
1. Die **OPF-Spine** bestimmt die Lesereihenfolge (nicht Dateinamen, nicht das Inhaltsverzeichnis).
|
||||||
2. Überschriften `h1`…`hN` (N = `--chapter-level`, bei `auto` 1 bzw. 2 in sehr großen Kapiteln) und
|
2. Überschriften `h1`…`hN` (N = `--chapter-level`, bei `auto` 1 bzw. 2 in sehr großen Kapiteln) und semantische Abschnitte (`epub:type`/`role`:
|
||||||
semantische Abschnitte (`epub:type`/`role`: chapter, part, appendix, glossary, bibliography, index,
|
chapter, part, appendix, glossary, bibliography, index, preface, …) beginnen neue Kapiteldateien,
|
||||||
preface, …) beginnen neue Kapiteldateien, auch mitten in einer XHTML-Datei.
|
auch mitten in einer XHTML-Datei.
|
||||||
3. Eine XHTML-Datei, die nur mit einer tieferen Überschrift beginnt (z. B. eine Datei pro Abschnitt
|
3. Eine XHTML-Datei, die nur mit einer tieferen Überschrift beginnt (z. B. eine Datei pro Abschnitt
|
||||||
`1.2`), wird an das laufende Kapitel angehängt.
|
`1.2`), wird an das laufende Kapitel angehängt.
|
||||||
4. Dateien ohne Überschrift (Cover, Titelei, Widmung) werden eigene Kapitel. Ihr Titel kommt aus
|
4. Dateien ohne Überschrift (Cover, Titelei, Widmung) werden eigene Kapitel. Ihr Titel kommt aus
|
||||||
|
|
@ -246,14 +246,9 @@ weil CommonMark keine `[^…]`-Syntax kennt.
|
||||||
- **Syntax-Highlighting:** Farb-Spans, `<strong>`/`<em>` in Listings und Zeilennummern werden entfernt.
|
- **Syntax-Highlighting:** Farb-Spans, `<strong>`/`<em>` in Listings und Zeilennummern werden entfernt.
|
||||||
Einrückung, Tabs, `<`, `>`, `&`, `$`, `|`, Backslashes und Backticks bleiben exakt erhalten. Die Sprache
|
Einrückung, Tabs, `<`, `>`, `&`, `$`, `|`, Backslashes und Backticks bleiben exakt erhalten. Die Sprache
|
||||||
kommt aus Klassen/Attributen (`language-python`, `data-lang`, …) oder starken Inhaltssignalen
|
kommt aus Klassen/Attributen (`language-python`, `data-lang`, …) oder starken Inhaltssignalen
|
||||||
(`#include`, Shebang, `>>>`, Python-Code, der sich parsen lässt und `def`/`class`/Kontrollstrukturen
|
(`#include`, `def …:`, Shebang, …). Ist sie unsicher, bleibt das Label leer. Bei einsprachigen
|
||||||
enthält, …). Gedruckte Zeilennummern als Text (`00 #include …`) stören die Erkennung nicht und bleiben
|
Büchern kann `--code-language python` (o. Ä.) alle übrigen Blöcke kennzeichnen. Achtung: Das betrifft
|
||||||
im Block stehen. Sind sich die sicher erkannten Blöcke einig (mindestens 5, davon 90 % eine Sprache),
|
auch Programmausgaben und Shell-Befehle im Buch.
|
||||||
genügen für die übrigen schwächere Signale: in einem Python-Buch Zuweisungen und Aufrufe, in einem
|
|
||||||
Buch mit C-artiger Sprache Anweisungen und Deklarationen mit `;` oder `{` (im Bericht
|
|
||||||
`detected_by: "book-context"`). Programmausgaben (Literale, Zahlen, Text) bleiben ohne Label.
|
|
||||||
`--code-language python` (o. Ä.) kennzeichnet alle übrigen Blöcke, auch Programmausgaben und
|
|
||||||
Shell-Befehle im Buch.
|
|
||||||
- **CSS-Formatierung:** Viele Verlage setzen Hervorhebungen nur über Stylesheets. epub2md wertet die
|
- **CSS-Formatierung:** Viele Verlage setzen Hervorhebungen nur über Stylesheets. epub2md wertet die
|
||||||
CSS-Dateien des Buches aus: Klassen-, Element- und ID-Selektoren, Nachfahren- und Kind-Kombinatoren,
|
CSS-Dateien des Buches aus: Klassen-, Element- und ID-Selektoren, Nachfahren- und Kind-Kombinatoren,
|
||||||
Spezifität und `style`-Attribute. `@media`-Blöcke und Selektoren mit Pseudoklassen werden ignoriert.
|
Spezifität und `style`-Attribute. `@media`-Blöcke und Selektoren mit Pseudoklassen werden ignoriert.
|
||||||
|
|
@ -361,7 +356,6 @@ cd output/mathematikbuch # Formeln: PDF über HTML und LaTeX (weasypri
|
||||||
pandoc mathematikbuch.md -s --mathml --lua-filter $P/book.lua --lua-filter $P/callouts.lua -o mathematikbuch.html
|
pandoc mathematikbuch.md -s --mathml --lua-filter $P/book.lua --lua-filter $P/callouts.lua -o mathematikbuch.html
|
||||||
pandoc mathematikbuch.html --lua-filter $P/book.lua --lua-filter $P/callouts.lua --pdf-engine=xelatex \
|
pandoc mathematikbuch.html --lua-filter $P/book.lua --lua-filter $P/callouts.lua --pdf-engine=xelatex \
|
||||||
-o mathematikbuch.pdf
|
-o mathematikbuch.pdf
|
||||||
pandoc mathematikbuch.md --toc --mathml --lua-filter $P/callouts.lua -o mathematikbuch.epub # EPUB: MathML
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Hilfsdateien in `pandoc/`:
|
Hilfsdateien in `pandoc/`:
|
||||||
|
|
@ -382,11 +376,11 @@ Hilfsdateien in `pandoc/`:
|
||||||
- Der Vorspann reicht bis vor das erste Kapitel des Hauptteils (Teil, Vorwort, Einleitung, Prolog,
|
- Der Vorspann reicht bis vor das erste Kapitel des Hauptteils (Teil, Vorwort, Einleitung, Prolog,
|
||||||
nummerierter Titel oder mehr als 1000 Wörter), sofern er Cover, Titelseite oder Impressum enthält. Ein
|
nummerierter Titel oder mehr als 1000 Wörter), sofern er Cover, Titelseite oder Impressum enthält. Ein
|
||||||
erstes Kapitel nur aus einem Bild gilt als Cover, ein Kapitel „Titel“ als Titelseite.
|
erstes Kapitel nur aus einem Bild gilt als Cover, ein Kapitel „Titel“ als Titelseite.
|
||||||
- **`print.css`** (weasyprint): A4, Seitenzahlen (nicht auf dem Cover), jedes Kapitel (auch Cover,
|
- **`print.css`** (weasyprint): A4, Seitenzahlen, jedes Kapitel (auch Cover, Titelseite, Impressum) und das
|
||||||
Titelseite, Impressum) und das Inhaltsverzeichnis auf neuer Seite, Inhaltsverzeichnis mit Punktlinie und
|
Inhaltsverzeichnis auf neuer Seite, Inhaltsverzeichnis mit Punktlinie und Seitenzahl, Silbentrennung
|
||||||
Seitenzahl, Silbentrennung (nicht in Code), umbrechende Codezeilen. Mit `--css` lässt Pandoc sein eigenes
|
(nicht in Code), umbrechende Codezeilen. Mit `--css` lässt Pandoc sein eigenes Stylesheet weg, deshalb
|
||||||
Stylesheet weg, deshalb bringt `print.css` auch Grundschrift, Zeilenabstand und Titelkopf mit. Getestet
|
bringt `print.css` auch Grundschrift, Zeilenabstand und Titelkopf mit. Getestet mit dem Python-Buch
|
||||||
mit dem Python-Buch (362 Seiten).
|
(362 Seiten).
|
||||||
|
|
||||||
**Infokästen:** Pandoc kennt die Schreibweise `> [!TIP]` erst ab neueren Versionen (Erweiterung `alerts`);
|
**Infokästen:** Pandoc kennt die Schreibweise `> [!TIP]` erst ab neueren Versionen (Erweiterung `alerts`);
|
||||||
3.1.3 (Ubuntu 24.04) zeigt „[!TIP]“ als Text. Der Filter `pandoc/callouts.lua` behebt das: In HTML und
|
3.1.3 (Ubuntu 24.04) zeigt „[!TIP]“ als Text. Der Filter `pandoc/callouts.lua` behebt das: In HTML und
|
||||||
|
|
@ -461,11 +455,6 @@ Verlustfreiheit behauptet (`lossless: false`).
|
||||||
realen Buch geprüft (434 Bilder, 246 Figuren, ausführliche Alternativtexte).
|
realen Buch geprüft (434 Bilder, 246 Figuren, ausführliche Alternativtexte).
|
||||||
- Rein per CSS erzeugte Inhalte (`::before`/`content:`, z. B. automatische Schrittnummern) werden nicht
|
- Rein per CSS erzeugte Inhalte (`::before`/`content:`, z. B. automatische Schrittnummern) werden nicht
|
||||||
übernommen, weil Pseudo-Elemente nicht ausgewertet werden.
|
übernommen, weil Pseudo-Elemente nicht ausgewertet werden.
|
||||||
- Die eingebundenen Stylesheets gelten für das ganze Buch, nicht je Kapiteldokument. Bindet nur ein
|
|
||||||
Kapitel ein bestimmtes Stylesheet ein, wirkt es auch auf die anderen. In Fußnoten greifen CSS-Regeln
|
|
||||||
mit Vorfahren-Selektoren nicht, weil die Noten vor dem Rendern aus dem Text gelöst werden.
|
|
||||||
- Von Hand gebaute TeX-Konstrukte (z. B. `\mathinner{…\raise…}`) kann Pandoc nicht in MathML umsetzen;
|
|
||||||
sie bleiben als TeX-Text stehen (Mathebuch: 1 von 4986 Formeln).
|
|
||||||
|
|
||||||
## Architektur
|
## Architektur
|
||||||
|
|
||||||
|
|
@ -496,7 +485,6 @@ pandoc/
|
||||||
├── callouts.lua Pandoc-Filter: '> [!TIP]'-Kästen → gerahmte Kästen (HTML/EPUB) bzw. Zitatblock mit Titel
|
├── callouts.lua Pandoc-Filter: '> [!TIP]'-Kästen → gerahmte Kästen (HTML/EPUB) bzw. Zitatblock mit Titel
|
||||||
├── callouts.css Gestaltung der Kästen (vom Filter eingebunden; muss neben callouts.lua liegen)
|
├── callouts.css Gestaltung der Kästen (vom Filter eingebunden; muss neben callouts.lua liegen)
|
||||||
└── print.css Druck-Layout für PDFs mit weasyprint (A4, Seitenzahlen, Kapitel auf neuer Seite, Codeumbruch)
|
└── print.css Druck-Layout für PDFs mit weasyprint (A4, Seitenzahlen, Kapitel auf neuer Seite, Codeumbruch)
|
||||||
tools/qa/ QA-Skripte: CLI-Matrix, Validator, Wortvergleich, Pandoc-Prüfung (siehe tools/qa/README.md)
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Statt BeautifulSoup wird direkt lxml verwendet: Es ist für die XHTML-Bäume ausreichend, schneller und
|
Statt BeautifulSoup wird direkt lxml verwendet: Es ist für die XHTML-Bäume ausreichend, schneller und
|
||||||
|
|
@ -520,8 +508,4 @@ unsichtbare Zeichen, Szenenwechsel, Gedichte, Infokästen, `--styles`, `--page-m
|
||||||
in der Buch-README. Liegen EPUBs unter `Beispiele/`, werden sie zusätzlich auf
|
in der Buch-README. Liegen EPUBs unter `Beispiele/`, werden sie zusätzlich auf
|
||||||
Struktur-Invarianten geprüft: keine kaputten Links, balancierte Code-Fences, gültiges YAML, alle Medien
|
Struktur-Invarianten geprüft: keine kaputten Links, balancierte Code-Fences, gültiges YAML, alle Medien
|
||||||
vorhanden. Dabei werden keine Buchtexte verglichen. `Beispiele/` ist per `.gitignore` vom Repository
|
vorhanden. Dabei werden keine Buchtexte verglichen. `Beispiele/` ist per `.gitignore` vom Repository
|
||||||
ausgeschlossen.`tests/test_pandoc.py` prüft den Pandoc-Weg (nur mit installiertem Pandoc).
|
ausgeschlossen.
|
||||||
|
|
||||||
Für gründlichere Prüfungen nach `Prompt_Kontrolle.md` gibt es `tools/qa/`: eine CLI-Matrix über alle Optionen
|
|
||||||
und Fehlercodes, einen unabhängigen Markdown-Validator, einen Wortvergleich gegen Datenverlust und die
|
|
||||||
Prüfung des Pandoc-Wegs (HTML, EPUB, PDF) für alle Beispielbücher. Details in `tools/qa/README.md`.
|
|
||||||
|
|
|
||||||
|
|
@ -2,10 +2,8 @@
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import ast
|
|
||||||
import json
|
import json
|
||||||
import re
|
import re
|
||||||
import warnings
|
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from .utils import classes, is_element, local
|
from .utils import classes, is_element, local
|
||||||
|
|
@ -129,114 +127,9 @@ def detect_language(el: Any) -> tuple[str, str]:
|
||||||
return "", ""
|
return "", ""
|
||||||
|
|
||||||
|
|
||||||
LINE_NUMBER_PREFIX = re.compile(r"^(\d{1,4})( +|$)")
|
|
||||||
|
|
||||||
|
|
||||||
def strip_line_numbers(code: str) -> str:
|
|
||||||
"""Remove printed line numbers ("00 #include …") – for language detection only, the block keeps them.
|
|
||||||
|
|
||||||
Applies when at least 80 % of the non-blank lines (at least two) start with consecutive numbers.
|
|
||||||
"""
|
|
||||||
lines = code.split("\n")
|
|
||||||
filled = [ln for ln in lines if ln.strip()]
|
|
||||||
matches = [LINE_NUMBER_PREFIX.match(ln) for ln in filled]
|
|
||||||
numbers = [int(m.group(1)) for m in matches if m]
|
|
||||||
if len(numbers) < 2 or len(numbers) < 0.8 * len(filled):
|
|
||||||
return code
|
|
||||||
if any(b != a + 1 for a, b in zip(numbers, numbers[1:])):
|
|
||||||
return code
|
|
||||||
width = min((len(m.group(2)) for m in matches if m and m.group(2)), default=0)
|
|
||||||
out = []
|
|
||||||
for ln in lines:
|
|
||||||
m = LINE_NUMBER_PREFIX.match(ln) if ln.strip() else None
|
|
||||||
out.append(ln[len(m.group(1)) + min(width, len(m.group(2))):] if m else ln)
|
|
||||||
return "\n".join(out)
|
|
||||||
|
|
||||||
|
|
||||||
def _python_module(text: str) -> ast.Module | None:
|
|
||||||
"""The parsed module, or None when the text is not valid Python (never executed, warnings suppressed)."""
|
|
||||||
try:
|
|
||||||
with warnings.catch_warnings():
|
|
||||||
warnings.simplefilter("ignore")
|
|
||||||
return ast.parse(text)
|
|
||||||
except (SyntaxError, ValueError, RecursionError, MemoryError):
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
PY_COMPOUND = (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef, ast.If, ast.For, ast.AsyncFor, ast.While,
|
|
||||||
ast.With, ast.AsyncWith, ast.Try)
|
|
||||||
PY_CODE_EXPR = (ast.Call, ast.ListComp, ast.SetComp, ast.DictComp, ast.GeneratorExp, ast.Lambda, ast.Await)
|
|
||||||
|
|
||||||
|
|
||||||
def _python_strength(text: str) -> int:
|
|
||||||
"""2 = surely Python (block statement, or import plus code), 1 = Python code (assignment, call,
|
|
||||||
comprehension …), 0 = not Python or only literals/names – which is what program output looks like."""
|
|
||||||
if any(ln.rstrip().endswith((";", "{")) for ln in text.split("\n")):
|
|
||||||
return 0
|
|
||||||
module = _python_module(text)
|
|
||||||
if module is None or not module.body:
|
|
||||||
return 0
|
|
||||||
imports = sum(isinstance(n, (ast.Import, ast.ImportFrom)) for n in module.body)
|
|
||||||
code = 0
|
|
||||||
for node in module.body:
|
|
||||||
if isinstance(node, PY_COMPOUND):
|
|
||||||
return 2
|
|
||||||
if isinstance(node, ast.Expr):
|
|
||||||
value = node.value
|
|
||||||
if isinstance(value, ast.Constant) and isinstance(value.value, str):
|
|
||||||
continue # docstring
|
|
||||||
if not isinstance(value, PY_CODE_EXPR):
|
|
||||||
return 0 # a bare name, number or literal line is program output ("Jane", "42", "[1, 4]")
|
|
||||||
code += 1
|
|
||||||
elif isinstance(node, ast.AnnAssign) and node.value is None:
|
|
||||||
pass # "name: value" is YAML or a Makefile rule as often as Python
|
|
||||||
elif not isinstance(node, (ast.Import, ast.ImportFrom, ast.Pass)):
|
|
||||||
code += 1
|
|
||||||
if imports and code:
|
|
||||||
return 2
|
|
||||||
return 1 if code else 0
|
|
||||||
|
|
||||||
|
|
||||||
CURLY_STATEMENT = re.compile(r"[=(]|^\s*[A-Za-z_]\w*[\s*]+[A-Za-z_*]") # assignment/call or "type name"
|
|
||||||
|
|
||||||
|
|
||||||
def _curly_signal(text: str) -> bool:
|
|
||||||
"""C-family syntax: a statement or declaration ending with ';' or '{', or a function body on one line."""
|
|
||||||
lines = [re.sub(r"(^|\s)//.*$", "", ln).rstrip() for ln in text.split("\n")]
|
|
||||||
if any(ln.endswith((";", "{")) and CURLY_STATEMENT.search(ln) for ln in lines):
|
|
||||||
return True
|
|
||||||
return bool(re.search(r"\w\s*\([^)]*\)\s*\{.*\}", text))
|
|
||||||
|
|
||||||
|
|
||||||
CURLY_LANGS = {"c", "cpp", "java", "csharp", "javascript", "typescript", "go", "rust", "php", "kotlin", "swift", "scala"}
|
|
||||||
|
|
||||||
|
|
||||||
def detect_language_from_book(code: str, book_language: str) -> str:
|
|
||||||
"""Weak content signals, accepted only when they fit the book's dominant language."""
|
|
||||||
if not book_language:
|
|
||||||
return ""
|
|
||||||
text = strip_line_numbers(code).strip()
|
|
||||||
if not text:
|
|
||||||
return ""
|
|
||||||
if book_language == "python" and _python_strength(text) >= 1:
|
|
||||||
return "python"
|
|
||||||
if book_language in CURLY_LANGS and _curly_signal(text):
|
|
||||||
return book_language
|
|
||||||
return ""
|
|
||||||
|
|
||||||
|
|
||||||
def dominant_language(languages: list[str]) -> str:
|
|
||||||
"""The book's language when the reliably detected blocks agree (at least 5 blocks, 90 %)."""
|
|
||||||
detected = [lang for lang in languages if lang and lang != "text"]
|
|
||||||
if len(detected) < 5:
|
|
||||||
return ""
|
|
||||||
lang = max(set(detected), key=detected.count)
|
|
||||||
return lang if detected.count(lang) >= 0.9 * len(detected) else ""
|
|
||||||
|
|
||||||
|
|
||||||
def detect_language_from_content(code: str) -> str:
|
def detect_language_from_content(code: str) -> str:
|
||||||
"""Very conservative content heuristics – only strong signals."""
|
"""Very conservative content heuristics – only strong signals."""
|
||||||
text = strip_line_numbers(code).strip()
|
text = code.strip()
|
||||||
if not text:
|
if not text:
|
||||||
return ""
|
return ""
|
||||||
first = text.split("\n", 1)[0]
|
first = text.split("\n", 1)[0]
|
||||||
|
|
@ -274,8 +167,6 @@ def detect_language_from_content(code: str) -> str:
|
||||||
c_like = sum(1 for ln in lines if ln.rstrip().endswith((";", "{")))
|
c_like = sum(1 for ln in lines if ln.rstrip().endswith((";", "{")))
|
||||||
if py_signals >= 2 and c_like == 0:
|
if py_signals >= 2 and c_like == 0:
|
||||||
return "python"
|
return "python"
|
||||||
if re.search(r"^>>> \S", text, re.M) or _python_strength(text) == 2:
|
|
||||||
return "python"
|
|
||||||
if re.match(r"\s*(SELECT|INSERT\s+INTO|CREATE\s+(TABLE|INDEX|VIEW)|UPDATE\s+\w+\s+SET|DELETE\s+FROM)\b", text):
|
if re.match(r"\s*(SELECT|INSERT\s+INTO|CREATE\s+(TABLE|INDEX|VIEW)|UPDATE\s+\w+\s+SET|DELETE\s+FROM)\b", text):
|
||||||
return "sql"
|
return "sql"
|
||||||
if re.match(r"(?m)^FROM\s+\S+", text) and re.search(r"(?m)^(RUN|CMD|COPY|ENTRYPOINT)\s", text):
|
if re.match(r"(?m)^FROM\s+\S+", text) and re.search(r"(?m)^(RUN|CMD|COPY|ENTRYPOINT)\s", text):
|
||||||
|
|
|
||||||
|
|
@ -22,7 +22,6 @@ from .css import StyleSheet, emphasis, has_highlight, inline_style, style_props
|
||||||
from .codeblocks import (
|
from .codeblocks import (
|
||||||
code_text,
|
code_text,
|
||||||
detect_language,
|
detect_language,
|
||||||
detect_language_from_book,
|
|
||||||
detect_language_from_content,
|
detect_language_from_content,
|
||||||
fence_for,
|
fence_for,
|
||||||
inline_code,
|
inline_code,
|
||||||
|
|
@ -324,7 +323,6 @@ class Renderer:
|
||||||
self.consumed_formula_images: set[str] = set()
|
self.consumed_formula_images: set[str] = set()
|
||||||
self.formulas: list[dict[str, str]] = []
|
self.formulas: list[dict[str, str]] = []
|
||||||
self.code_blocks: list[dict[str, Any]] = []
|
self.code_blocks: list[dict[str, Any]] = []
|
||||||
self.book_code_language = "" # dominant language of the reliably detected blocks (pre-scan)
|
|
||||||
self.endnotes_file = ""
|
self.endnotes_file = ""
|
||||||
# per chapter state
|
# per chapter state
|
||||||
self.ch: Chapter | None = None
|
self.ch: Chapter | None = None
|
||||||
|
|
@ -801,9 +799,6 @@ class Renderer:
|
||||||
if not lang:
|
if not lang:
|
||||||
lang = detect_language_from_content(code)
|
lang = detect_language_from_content(code)
|
||||||
how = "content" if lang else ""
|
how = "content" if lang else ""
|
||||||
if not lang:
|
|
||||||
lang = detect_language_from_book(code, self.book_code_language)
|
|
||||||
how = "book-context" if lang else ""
|
|
||||||
if not lang and self.opt.code_language:
|
if not lang and self.opt.code_language:
|
||||||
lang, how = self.opt.code_language, "option"
|
lang, how = self.opt.code_language, "option"
|
||||||
if self.ch:
|
if self.ch:
|
||||||
|
|
|
||||||
|
|
@ -17,8 +17,6 @@ from urllib.parse import unquote
|
||||||
|
|
||||||
from . import TOOL_NAME, __version__
|
from . import TOOL_NAME, __version__
|
||||||
from .chapters import SPECIAL_KINDS, ChapterBuilder, heading_level
|
from .chapters import SPECIAL_KINDS, ChapterBuilder, heading_level
|
||||||
from .codeblocks import (code_text, detect_language, detect_language_from_content, dominant_language,
|
|
||||||
is_block_code_container, join_wrap_markers)
|
|
||||||
from .diagnostics import Diagnostics
|
from .diagnostics import Diagnostics
|
||||||
from .css import StyleSheet, base_style, emphasis
|
from .css import StyleSheet, base_style, emphasis
|
||||||
from .epub_reader import EpubReader, decode_markup, parse_xml
|
from .epub_reader import EpubReader, decode_markup, parse_xml
|
||||||
|
|
@ -294,7 +292,6 @@ def _render(reader: EpubReader, package: Any, chapters: list[Chapter], resolver:
|
||||||
notes_title = "Anmerkungen" if language.startswith("de") else "Notes"
|
notes_title = "Anmerkungen" if language.startswith("de") else "Notes"
|
||||||
endnotes_number = 999 if len(chapters) < 999 else len(chapters) + 1
|
endnotes_number = 999 if len(chapters) < 999 else len(chapters) + 1
|
||||||
renderer.endnotes_file = f"chapters/{endnotes_number:0{width}d}-{slugify(notes_title, opts.slug_language)}.md"
|
renderer.endnotes_file = f"chapters/{endnotes_number:0{width}d}-{slugify(notes_title, opts.slug_language)}.md"
|
||||||
renderer.book_code_language = _book_code_language(chapters)
|
|
||||||
for ch in chapters:
|
for ch in chapters:
|
||||||
diag.step(f"Rendere {ch.filename}")
|
diag.step(f"Rendere {ch.filename}")
|
||||||
ch.markdown = renderer.render_chapter(ch)
|
ch.markdown = renderer.render_chapter(ch)
|
||||||
|
|
@ -315,20 +312,6 @@ def _render(reader: EpubReader, package: Any, chapters: list[Chapter], resolver:
|
||||||
return _Rendered(renderer, media, cover_out, endnotes, endnotes_md, skipped)
|
return _Rendered(renderer, media, cover_out, endnotes, endnotes_md, skipped)
|
||||||
|
|
||||||
|
|
||||||
def _book_code_language(chapters: list[Chapter]) -> str:
|
|
||||||
"""Pre-scan: the language of the book's code blocks, from the reliably detected blocks only."""
|
|
||||||
langs: list[str] = []
|
|
||||||
for ch in chapters:
|
|
||||||
for _, item in ch.items:
|
|
||||||
if not is_element(item):
|
|
||||||
continue
|
|
||||||
for el in iter_elements(item):
|
|
||||||
if local(el.tag) in ("pre", "listing", "xmp", "plaintext") or is_block_code_container(el):
|
|
||||||
lang, _ = detect_language(el)
|
|
||||||
langs.append(lang or detect_language_from_content(join_wrap_markers(code_text(el))[0]))
|
|
||||||
return dominant_language(langs)
|
|
||||||
|
|
||||||
|
|
||||||
def _suggest_code_language(renderer: Renderer, diag: Diagnostics) -> None:
|
def _suggest_code_language(renderer: Renderer, diag: Diagnostics) -> None:
|
||||||
"""Book-level hint: many unlabeled blocks while every recognized block has the same language."""
|
"""Book-level hint: many unlabeled blocks while every recognized block has the same language."""
|
||||||
blocks = renderer.code_blocks
|
blocks = renderer.code_blocks
|
||||||
|
|
|
||||||
|
|
@ -118,57 +118,3 @@ def test_print_wrap_markers_can_be_kept(convert):
|
||||||
run = convert(Book(docs=[Doc("c.xhtml", body)]), "--keep-wrap-markers")
|
run = convert(Book(docs=[Doc("c.xhtml", body)]), "--keep-wrap-markers")
|
||||||
(_, _, code), = fenced_blocks(run.chapter("001"))
|
(_, _, code), = fenced_blocks(run.chapter("001"))
|
||||||
assert code == "a = (1,\n↪ 2)"
|
assert code == "a = (1,\n↪ 2)"
|
||||||
|
|
||||||
|
|
||||||
def test_printed_line_numbers_do_not_hide_the_language(convert):
|
|
||||||
listing = ("00 // Kapitel1/hallo.c\n01 #include <stdio.h>\n \n02 int main(void) {\n"
|
|
||||||
"03 printf(\"Hallo\\n\");\n04 return 0;\n05 }")
|
|
||||||
run = convert(Book(docs=[Doc("c.xhtml", f"<h1>N</h1><pre>{listing}</pre>")]))
|
|
||||||
(_, lang, code), = fenced_blocks(run.chapter("001"))
|
|
||||||
assert lang == "c"
|
|
||||||
assert code.startswith("00 // Kapitel1/hallo.c\n01 #include <stdio.h>") # numbers stay in the block
|
|
||||||
|
|
||||||
|
|
||||||
def test_python_is_recognized_by_parsing(convert):
|
|
||||||
blocks = ["from collections import ChainMap\n\nchain = ChainMap({}, {})\nprint(chain)",
|
|
||||||
"zahl = -1\nif zahl < 0:\n print(\"negativ\")",
|
|
||||||
">>> 2 + 3\n5",
|
|
||||||
"name: epub2md\nversion: 1", # YAML is valid Python syntax, but no Python code
|
|
||||||
"[1, 4, 9]", # program output
|
|
||||||
"x = compute()"] # Python, but too weak without book context
|
|
||||||
body = "<h1>P</h1>" + "".join(f"<pre>{b}</pre>" for b in blocks)
|
|
||||||
run = convert(Book(docs=[Doc("c.xhtml", body)]))
|
|
||||||
langs = [lang for _, lang, _ in fenced_blocks(run.chapter("001"))]
|
|
||||||
assert langs == ["python", "python", "python", "", "", ""]
|
|
||||||
|
|
||||||
|
|
||||||
def test_weak_signals_follow_the_book_language(convert):
|
|
||||||
c_listing = "#include <stdio.h>\nint main(void) { return 0; }"
|
|
||||||
snippets = ["char ch = 65; // 'A'", "if (a || b) {\n x();\n}", "int remove(const char *pathname);",
|
|
||||||
"struct node {\n int value;\n};", "Ergebnis: 12345", "|00003.14|", "gcc -Wall -o prog prog.c",
|
|
||||||
"Dezimal: 65; Zeichen: A"]
|
|
||||||
body = "<h1>C</h1>" + "".join(f"<pre>{c_listing}</pre>" for _ in range(5))
|
|
||||||
body += "".join(f"<pre>{s}</pre>" for s in snippets)
|
|
||||||
run = convert(Book(docs=[Doc("c.xhtml", body)]))
|
|
||||||
langs = [lang for _, lang, _ in fenced_blocks(run.chapter("001"))][5:]
|
|
||||||
assert langs == ["c", "c", "c", "c", "", "", "", ""]
|
|
||||||
by = [item["detected_by"] for item in run.report["code_blocks"]["items"]][5:9]
|
|
||||||
assert by == ["book-context"] * 4
|
|
||||||
|
|
||||||
|
|
||||||
def test_python_book_labels_assignments_but_not_output(convert):
|
|
||||||
py = "def f(x):\n return x"
|
|
||||||
snippets = ["foo = decorate(foo)", "random.sample(population, k)", "[x * x for x in range(5)]",
|
|
||||||
"True", "('A', 'B', 'C')", "Hallo Welt", "4\n3\nnohtyP", "Jane\n42\nPerson(name='Jane')"]
|
|
||||||
body = "<h1>P</h1>" + "".join(f"<pre>{py}</pre>" for _ in range(5))
|
|
||||||
body += "".join(f"<pre>{s}</pre>" for s in snippets)
|
|
||||||
run = convert(Book(docs=[Doc("c.xhtml", body)]))
|
|
||||||
langs = [lang for _, lang, _ in fenced_blocks(run.chapter("001"))][5:]
|
|
||||||
assert langs == ["python", "python", "python", "", "", "", "", ""]
|
|
||||||
|
|
||||||
|
|
||||||
def test_mixed_book_gets_no_context_labels(convert):
|
|
||||||
body = "<h1>M</h1>" + "<pre>def f(x):\n return x</pre>" * 3
|
|
||||||
body += "<pre>#include <stdio.h>\nint main(void) { return 0; }</pre>" * 3 + "<pre>x = compute();</pre>"
|
|
||||||
run = convert(Book(docs=[Doc("c.xhtml", body)]))
|
|
||||||
assert fenced_blocks(run.chapter("001"))[-1][1] == ""
|
|
||||||
|
|
|
||||||
|
|
@ -39,7 +39,7 @@ def test_qa2_10_book_readme_recommends_pdf_via_html(convert):
|
||||||
|
|
||||||
def test_qa2_11_code_language_hint(convert):
|
def test_qa2_11_code_language_hint(convert):
|
||||||
py = "def f(x):\n return x\n\nclass A:\n pass"
|
py = "def f(x):\n return x\n\nclass A:\n pass"
|
||||||
body = "<h1>C</h1>" + "".join(f"<pre>{py}\n# {i}</pre><pre>Ergebnis: {i}</pre>" for i in range(10))
|
body = "<h1>C</h1>" + "".join(f"<pre>{py}\n# {i}</pre><pre>print({i})</pre>" for i in range(10))
|
||||||
run = convert(Book(docs=[Doc("c.xhtml", body)]))
|
run = convert(Book(docs=[Doc("c.xhtml", body)]))
|
||||||
hints = [i for i in run.report["infos"] if i["code"] == "code-language-hint"]
|
hints = [i for i in run.report["infos"] if i["code"] == "code-language-hint"]
|
||||||
assert hints and "--code-language python" in hints[0]["message"]
|
assert hints and "--code-language python" in hints[0]["message"]
|
||||||
|
|
|
||||||
|
|
@ -1,34 +0,0 @@
|
||||||
# QA-Werkzeuge
|
|
||||||
|
|
||||||
Skripte für die Qualitätsprüfung nach `Prompt_Kontrolle.md`. Sie ergänzen `pytest` um CLI-Integrationstests,
|
|
||||||
einen unabhängigen Validator und die Prüfung des Pandoc-Wegs. Mit ihnen wurden die Befunde QA3-01 bis
|
|
||||||
QA3-06 gefunden (siehe `CHANGELOG.md`). Alle Aufrufe im Repo-Wurzelverzeichnis; erzeugte Daten landen in
|
|
||||||
`.qa/` (von Git ignoriert), Kommandos mit Rückgabecode in `.qa/commands.log`.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python3 -m venv .qa/venv && . .qa/venv/bin/activate
|
|
||||||
pip install -r tools/qa/requirements.txt
|
|
||||||
```
|
|
||||||
|
|
||||||
| Skript | Zweck |
|
|
||||||
| --- | --- |
|
|
||||||
| `make_fixtures.py` | erzeugt synthetische Test-EPUBs 01–14 (u. a. Path Traversal, kaputter Container, MathML, Fußnoten, Typografie) in `.qa/generated-fixtures/` |
|
|
||||||
| `run_matrix.py` | CLI-Matrix (89 Fälle) über alle Optionen und Fehlercodes → `.qa/output/matrix-results.json`; braucht die Fixtures |
|
|
||||||
| `validate_output.py AUSGABE…` | unabhängiger Validator (markdown-it-py statt `epub2md/linkcheck.py`): Links, Anker, Bilder, Fences, YAML, JSON, Python-Einrückung, Fußnotenpaare → JSON |
|
|
||||||
| `worddiff.py ALT NEU` | Wortvergleich zweier Ausgaben – zeigt Datenverlust nach einer Codeänderung |
|
|
||||||
| `pyblocks.py DATEIEN…` | Python-Codeblöcke mit Einrückungsfehlern samt Inhalt (Vergleich mit der Quelle) |
|
|
||||||
| `pandoc_full.sh` | alle EPUBs aus `Beispiele/` konvertieren, zusammenfügen und mit den Pandoc-Befehlen der Buch-README nach HTML, EPUB und PDF (weasyprint, xelatex) bringen → `.qa/output/pandoc/` |
|
|
||||||
| `pandoc_run.sh`, `pandoc_pdfs.sh` | Teilschritte von `pandoc_full.sh` für einzelne Buchordner |
|
|
||||||
| `check_pandoc.py BUCHORDNER…` | HTML/EPUB von Pandoc: interne Links und Anker, Bilder, `[!TIP]`-Reste, Formeln, Lage des Inhaltsverzeichnisses |
|
|
||||||
| `check_pdf.sh PDF…` | Seiten, Seite des Inhaltsverzeichnisses, Text-Reste, nicht eingebettete Schriften (poppler-utils) |
|
|
||||||
| `log.sh` | `. tools/qa/log.sh; qa BEFEHL` protokolliert Befehl und rc |
|
|
||||||
|
|
||||||
Typischer Ablauf nach einer Änderung:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python -m pytest -q
|
|
||||||
python tools/qa/make_fixtures.py && python tools/qa/run_matrix.py
|
|
||||||
python epub2md.py Beispiele/BUCH.epub .qa/output/neu/BUCH
|
|
||||||
python tools/qa/worddiff.py .qa/output/alt/BUCH .qa/output/neu/BUCH # alt: z. B. mit git stash erzeugt
|
|
||||||
python tools/qa/validate_output.py .qa/output/neu/BUCH
|
|
||||||
```
|
|
||||||
|
|
@ -1,118 +0,0 @@
|
||||||
"""Prüft Pandoc-Ausgaben (HTML und EPUB) eines konvertierten Buches.
|
|
||||||
|
|
||||||
Aufruf: python tools/qa/check_pandoc.py BUCHORDNER [...] -> JSON auf stdout
|
|
||||||
Geprüft: interne Links/Anker, Bilder, Reste von '[!TIP]', Lage des Inhaltsverzeichnisses, TeX-Reste.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import json
|
|
||||||
import posixpath
|
|
||||||
import re
|
|
||||||
import sys
|
|
||||||
import zipfile
|
|
||||||
from pathlib import Path
|
|
||||||
from urllib.parse import unquote
|
|
||||||
|
|
||||||
from lxml import html as lhtml
|
|
||||||
|
|
||||||
ID_RE = re.compile(r"""\b(?:id|name)\s*=\s*["']([^"']+)["']""")
|
|
||||||
|
|
||||||
|
|
||||||
def _ids(text: str) -> set[str]:
|
|
||||||
return set(ID_RE.findall(text))
|
|
||||||
|
|
||||||
|
|
||||||
def _links(doc) -> list[str]:
|
|
||||||
return [a.get("href") for a in doc.iter("a") if a.get("href")]
|
|
||||||
|
|
||||||
|
|
||||||
def _images(doc) -> list[str]:
|
|
||||||
srcs = [i.get("src") for i in doc.iter("img") if i.get("src")]
|
|
||||||
srcs += [i.get("{http://www.w3.org/1999/xlink}href") or i.get("href") for i in doc.iter("image")]
|
|
||||||
return [s for s in srcs if s and not s.startswith("data:")]
|
|
||||||
|
|
||||||
|
|
||||||
def _parse(text: str):
|
|
||||||
return lhtml.fromstring(text.encode("utf-8"))
|
|
||||||
|
|
||||||
|
|
||||||
def _toc_position(doc) -> dict:
|
|
||||||
"""Where does the TOC (nav#TOC) stand relative to the first chapter heading?"""
|
|
||||||
nav = doc.xpath("//*[@id='TOC']")
|
|
||||||
if not nav:
|
|
||||||
return {"toc": False}
|
|
||||||
body_text = lhtml.tostring(doc, encoding="unicode")
|
|
||||||
pos = body_text.find('id="TOC"')
|
|
||||||
before = re.sub(r"<[^>]+>", " ", body_text[:pos])
|
|
||||||
h1_before = len(re.findall(r"<h1\b", body_text[:pos]))
|
|
||||||
return {"toc": True, "h1_before_toc": h1_before, "words_before_toc": len(before.split()),
|
|
||||||
"toc_entries": len(nav[0].xpath(".//a"))}
|
|
||||||
|
|
||||||
|
|
||||||
def check_html(path: Path) -> dict:
|
|
||||||
text = path.read_text(encoding="utf-8")
|
|
||||||
doc = _parse(text)
|
|
||||||
ids = _ids(text)
|
|
||||||
broken = [h for h in _links(doc) if h.startswith("#") and unquote(h[1:]) not in ids]
|
|
||||||
missing_img = [s for s in _images(doc) if not (path.parent / unquote(s)).exists()]
|
|
||||||
plain = doc.text_content()
|
|
||||||
return {
|
|
||||||
"file": path.name, "links_internal": sum(1 for h in _links(doc) if h.startswith("#")),
|
|
||||||
"broken_internal": len(broken), "broken_examples": broken[:5],
|
|
||||||
"images": len(_images(doc)), "missing_images": missing_img[:5],
|
|
||||||
"callout_residue": len(re.findall(r"\[!(?:NOTE|TIP|IMPORTANT|WARNING|CAUTION)\]", plain)),
|
|
||||||
"tex_residue": len(re.findall(r"\\(?:frac|begin\{|mathbb|alpha|sum_)", plain)),
|
|
||||||
"math_elements": len(doc.xpath("//*[local-name()='math']")),
|
|
||||||
**_toc_position(doc),
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def check_epub(path: Path) -> dict:
|
|
||||||
with zipfile.ZipFile(path) as z:
|
|
||||||
names = set(z.namelist())
|
|
||||||
docs = {n: z.read(n).decode("utf-8") for n in names if n.endswith((".xhtml", ".html"))}
|
|
||||||
ids = {n: _ids(t) for n, t in docs.items()}
|
|
||||||
broken: list[str] = []
|
|
||||||
internal = 0
|
|
||||||
missing_img: list[str] = []
|
|
||||||
residue = tex = math = 0
|
|
||||||
for name, text in docs.items():
|
|
||||||
doc = _parse(text)
|
|
||||||
base = posixpath.dirname(name)
|
|
||||||
for h in _links(doc):
|
|
||||||
if re.match(r"^[a-z]+:", h):
|
|
||||||
continue
|
|
||||||
internal += 1
|
|
||||||
file, _, frag = h.partition("#")
|
|
||||||
target = posixpath.normpath(posixpath.join(base, unquote(file))) if file else name
|
|
||||||
if target not in names:
|
|
||||||
broken.append(f"{name}: {h}")
|
|
||||||
elif frag and unquote(frag) not in ids.get(target, set()):
|
|
||||||
broken.append(f"{name}: {h}")
|
|
||||||
for s in _images(doc):
|
|
||||||
if posixpath.normpath(posixpath.join(base, unquote(s))) not in names:
|
|
||||||
missing_img.append(f"{name}: {s}")
|
|
||||||
plain = doc.text_content()
|
|
||||||
residue += len(re.findall(r"\[!(?:NOTE|TIP|IMPORTANT|WARNING|CAUTION)\]", plain))
|
|
||||||
tex += len(re.findall(r"\\(?:frac|begin\{|mathbb|alpha|sum_)", plain))
|
|
||||||
math += len(doc.xpath("//*[local-name()='math']"))
|
|
||||||
return {"file": path.name, "documents": len(docs), "links_internal": internal, "broken_internal": len(broken),
|
|
||||||
"broken_examples": broken[:5], "missing_images": missing_img[:5], "callout_residue": residue,
|
|
||||||
"tex_residue": tex, "math_elements": math, "nav": any(n.endswith("nav.xhtml") for n in names)}
|
|
||||||
|
|
||||||
|
|
||||||
def main() -> None:
|
|
||||||
results = []
|
|
||||||
for d in map(Path, sys.argv[1:]):
|
|
||||||
entry = {"book": d.name}
|
|
||||||
for f in sorted(d.glob("*.html")):
|
|
||||||
entry["html"] = check_html(f)
|
|
||||||
for f in sorted(d.glob("*.epub")):
|
|
||||||
entry["epub"] = check_epub(f)
|
|
||||||
results.append(entry)
|
|
||||||
json.dump(results, sys.stdout, ensure_ascii=False, indent=1)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
main()
|
|
||||||
|
|
@ -1,11 +0,0 @@
|
||||||
#!/bin/bash
|
|
||||||
# Kurzprüfung von PDFs: Seiten, Inhaltsverzeichnis-Seite, Text-Reste, nicht eingebettete Schriften
|
|
||||||
for pdf in "$@"; do
|
|
||||||
pages=$(pdfinfo "$pdf" 2>/dev/null | awk '/^Pages:/{print $2}')
|
|
||||||
txt=$(pdftotext -enc UTF-8 "$pdf" - 2>/dev/null)
|
|
||||||
tocpage=$(for p in $(seq 1 15); do pdftotext -f $p -l $p "$pdf" - 2>/dev/null | grep -qiE '^(Inhalt|Inhaltsverzeichnis|Contents)$' && { echo $p; break; }; done)
|
|
||||||
first=$(for p in 1 2 3 4; do printf '%s | ' "$(pdftotext -f $p -l $p "$pdf" - 2>/dev/null | grep -v '^\s*$' | head -1 | cut -c1-28)"; done)
|
|
||||||
printf '%s\tSeiten=%s\tInhalt-S.=%s\t[!]=%s\t<a id=%s\tTeX=%s\tfehlende-Einbettung=%s\n\t\tS.1-4: %s\n' "$(basename "$pdf" | cut -c1-14)" "$pages" "${tocpage:--}" \
|
|
||||||
"$(grep -c '\[!' <<<"$txt")" "$(grep -c '<a id' <<<"$txt")" "$(grep -cE '\\\\(frac|begin|mathbb)|\$\$' <<<"$txt")" \
|
|
||||||
"$(pdffonts "$pdf" 2>/dev/null | awk 'NR>2 && $(NF-4)=="no"' | wc -l)" "$first"
|
|
||||||
done
|
|
||||||
|
|
@ -1,9 +0,0 @@
|
||||||
# usage: . tools/qa/log.sh; qa <command...> – runs a command, appends command + rc to .qa/commands.log
|
|
||||||
qa() {
|
|
||||||
local start rc
|
|
||||||
start=$(date -Iseconds)
|
|
||||||
"$@"; rc=$?
|
|
||||||
mkdir -p "$(git rev-parse --show-toplevel)/.qa"
|
|
||||||
printf '%s\trc=%s\t%s\n' "$start" "$rc" "$*" >> "$(git rev-parse --show-toplevel)/.qa/commands.log"
|
|
||||||
return $rc
|
|
||||||
}
|
|
||||||
|
|
@ -1,266 +0,0 @@
|
||||||
"""Erzeugt die QA-Fixtures 01–14 (synthetisch, DRM-frei) unter .qa/generated-fixtures/."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import sys
|
|
||||||
import zipfile
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
ROOT = Path(__file__).resolve().parents[2]
|
|
||||||
sys.path.insert(0, str(ROOT / "tests"))
|
|
||||||
|
|
||||||
from conftest import PNG, SVG, Book, Doc, build_epub # noqa: E402
|
|
||||||
|
|
||||||
OUT = ROOT / ".qa" / "generated-fixtures"
|
|
||||||
M = 'xmlns="http://www.w3.org/1998/Math/MathML"'
|
|
||||||
|
|
||||||
|
|
||||||
def rewrite(path: Path, changes: dict[str, bytes | None], extra: dict[str, bytes] | None = None) -> None:
|
|
||||||
"""Replace/delete entries of an existing ZIP and add raw entries (names unchanged, even unsafe ones)."""
|
|
||||||
with zipfile.ZipFile(path) as z:
|
|
||||||
entries = [(i, z.read(i)) for i in z.infolist()]
|
|
||||||
with zipfile.ZipFile(path, "w") as z:
|
|
||||||
for info, data in entries:
|
|
||||||
if info.filename in changes:
|
|
||||||
if changes[info.filename] is None:
|
|
||||||
continue
|
|
||||||
data = changes[info.filename]
|
|
||||||
z.writestr(info, data)
|
|
||||||
for name, data in (extra or {}).items():
|
|
||||||
z.writestr(zipfile.ZipInfo(name), data)
|
|
||||||
|
|
||||||
|
|
||||||
def f01() -> Book:
|
|
||||||
return Book(version="2.0", ncx=True, title="Minimal EPUB 2", authors=["Ada Lovelace"], language="en",
|
|
||||||
identifier="urn:uuid:11111111-2222-3333-4444-555555555555",
|
|
||||||
docs=[Doc("chapter1.xhtml", "<h1>Chapter One</h1><p>Hello <em>EPUB 2</em> world.</p>", "Chapter One")])
|
|
||||||
|
|
||||||
|
|
||||||
def f02() -> Book:
|
|
||||||
return Book(title="Minimal EPUB 3", authors=["Grace Hopper"],
|
|
||||||
docs=[Doc("c1.xhtml", "<h1>Erstes Kapitel</h1><p>Eins.</p>", "Erstes Kapitel"),
|
|
||||||
Doc("c2.xhtml", "<h1>Zweites Kapitel</h1><p>Zwei.</p>", "Zweites Kapitel")])
|
|
||||||
|
|
||||||
|
|
||||||
def f03() -> Book:
|
|
||||||
a = ('<h1 id="a-start">Kapitel A</h1><p>Zu <a href="b.xhtml">Kapitel B</a>, zum <a href="#a-unten">Anker unten</a>, '
|
|
||||||
'zum <a href="b.xhtml#b-ziel">Ziel in B</a> und ins <a href="nirgendwo.xhtml#weg">Nichts</a>, '
|
|
||||||
'sowie <a href="b.xhtml#gibt-es-nicht">fehlender Anker</a>.</p>'
|
|
||||||
'<h2 id="a-sec">A.1 Abschnitt</h2><p>Text A.1</p>'
|
|
||||||
'<h1 id="a2">Kapitel A2 in derselben Datei</h1><p id="a-unten">Unten in A2.</p>'
|
|
||||||
'<h2>A2.1 Unterabschnitt</h2><p>Mehr.</p>')
|
|
||||||
b = ('<h1>Kapitel B</h1><p>B-Text.</p><h2 id="b-sec">B.1</h2><p id="b-ziel">Ziel in B.</p>'
|
|
||||||
'<p><a href="a.xhtml#a-sec">zurück zu A.1</a></p>')
|
|
||||||
c = '<h1>Kapitel C</h1><p>Ende mit <a href="a.xhtml">Link auf A</a>.</p>'
|
|
||||||
return Book(title="Kapitel und Links", docs=[Doc("a.xhtml", a, "A"), Doc("b.xhtml", b, "B"), Doc("c.xhtml", c, "C")])
|
|
||||||
|
|
||||||
|
|
||||||
PY = """class Stack:
|
|
||||||
def __init__(self):
|
|
||||||
self.items = []
|
|
||||||
|
|
||||||
def push(self, x):
|
|
||||||
\tself.items.append(x) # Tab-Einrückung
|
|
||||||
if len(self.items) > 10 and x < 0:
|
|
||||||
raise ValueError("& zu voll")
|
|
||||||
"""
|
|
||||||
|
|
||||||
|
|
||||||
def f04() -> Book:
|
|
||||||
body = (
|
|
||||||
"<h1>Code und Tabellen</h1>"
|
|
||||||
"<p>Inline: <code>len(x) < 3 && y</code>, mit Backtick <code>a`b</code> und <code>*nicht kursiv*</code>.</p>"
|
|
||||||
f'<pre><code class="language-python">{PY}</code></pre>'
|
|
||||||
"<pre class=\"shell\">$ find . -name '*.py' | xargs grep -n \"TODO\" \\\n | sort -u > todo.txt\n$ echo $HOME</pre>"
|
|
||||||
"<pre><code>Markdown mit ```fences``` und `ticks`\n eingerückt</code></pre>"
|
|
||||||
'<pre data-language="json">{\n "a": [1, 2],\n "b": "<tag>"\n}</pre>'
|
|
||||||
"<pre>#include <stdio.h>\nint main(void) { return 0; }</pre>"
|
|
||||||
"<ul><li>Punkt 1<ul><li>Punkt 1.1<ol><li>Schritt a</li><li>Schritt b</li></ol></li></ul></li>"
|
|
||||||
"<li>Punkt 2</li></ul>"
|
|
||||||
"<ol start=\"3\"><li>drei</li><li>vier<ul><li>vier.a</li></ul></li></ol>"
|
|
||||||
'<p>Unbekanntes Element: <x-widget data-k="v" onclick="evil()">Widget-Inhalt</x-widget>.</p>'
|
|
||||||
"<table><thead><tr><th>Befehl</th><th>Bedeutung</th></tr></thead>"
|
|
||||||
"<tbody><tr><td><code>ls | wc -l</code></td><td>Dateien <strong>zählen</strong></td></tr>"
|
|
||||||
"<tr><td><code>a && b</code></td><td>und</td></tr></tbody></table>"
|
|
||||||
)
|
|
||||||
return Book(title="Code und Tabellen", docs=[Doc("code.xhtml", body, "Code")])
|
|
||||||
|
|
||||||
|
|
||||||
def f05() -> Book:
|
|
||||||
body = (
|
|
||||||
"<h1>Bilder</h1>"
|
|
||||||
'<figure id="fig1"><img src="images/foto%201%20%C3%A4.png" alt="Ein Foto" title="Fototitel"/>'
|
|
||||||
"<figcaption>Abbildung 1.1: Ein Foto mit Unterschrift.</figcaption></figure>"
|
|
||||||
'<p>Nochmals dasselbe Foto: <img src="images/foto%201%20%C3%A4.png" alt="Wiederholung"/></p>'
|
|
||||||
'<figure><img src="images/diagramm.svg" alt=""/><figcaption>Abbildung 1.2: Ein SVG-Diagramm.</figcaption></figure>'
|
|
||||||
'<p>Inline-SVG: <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20"><title>Kreis</title>'
|
|
||||||
'<circle cx="10" cy="10" r="8"/></svg></p>'
|
|
||||||
'<p>Siehe <a href="#fig1">Abbildung 1.1</a> und <a href="images/diagramm.svg">SVG direkt</a>.</p>'
|
|
||||||
'<p>Fehlendes Bild: <img src="images/fehlt.png" alt="weg"/></p>'
|
|
||||||
)
|
|
||||||
return Book(title="Bilder und SVG", docs=[Doc("bilder.xhtml", body, "Bilder")],
|
|
||||||
media={"images/foto 1 ä.png": PNG, "images/diagramm.svg": SVG, "images/unbenutzt.png": PNG})
|
|
||||||
|
|
||||||
|
|
||||||
def f06() -> Book:
|
|
||||||
body = (
|
|
||||||
"<h1>Formeln</h1>"
|
|
||||||
f"<p>Bruch inline: <math {M}><mfrac><mi>a</mi><mi>b</mi></mfrac></math>.</p>"
|
|
||||||
f"<p>Potenz: <math {M}><msup><mi>x</mi><mn>2</mn></msup></math>, Index: <math {M}><msub><mi>a</mi><mi>i</mi></msub></math>, "
|
|
||||||
f"beides: <math {M}><msubsup><mi>x</mi><mi>i</mi><mn>2</mn></msubsup></math>.</p>"
|
|
||||||
f'<div class="equation" id="eq1"><math {M} display="block"><mrow><msup><mi>e</mi><mrow><mi>i</mi><mi>π</mi></mrow></msup>'
|
|
||||||
'<mo>+</mo><mn>1</mn><mo>=</mo><mn>0</mn></mrow></math><span class="eqno">(2.1)</span></div>'
|
|
||||||
'<p>Nach Gleichung <a href="#eq1">(2.1)</a> folgt …</p>'
|
|
||||||
f'<p>Nicht unterstützt: <math {M}><menclose notation="box"><mi>z</mi></menclose></math>.</p>'
|
|
||||||
f'<p>Mit TeX-Annotation: <math {M}><semantics><mi>y</mi><annotation encoding="application/x-tex">\\vec{{y}}</annotation>'
|
|
||||||
"</semantics></math>.</p>"
|
|
||||||
f'<p>alttext: <math {M} alttext="\\sqrt{{2}}"><msqrt><mn>2</mn></msqrt></math>.</p>'
|
|
||||||
'<p>SVG-Formel: <span class="math"><img src="f/formel.svg" alt="Formel E=mc²"/></span></p>'
|
|
||||||
'<p>Bildformel: <span class="equation"><img src="f/formel.png" alt="Summe über i"/></span></p>'
|
|
||||||
'<div class="Equation"><img src="f/tex.png" alt="$$\\int_0^1 x\\,dx = \\frac{1}{2}$$"/> <span>(2.2)</span></div>'
|
|
||||||
)
|
|
||||||
return Book(title="MathML und Formeln", docs=[Doc("math.xhtml", body, "Formeln")],
|
|
||||||
media={"f/formel.svg": SVG, "f/formel.png": PNG, "f/tex.png": PNG})
|
|
||||||
|
|
||||||
|
|
||||||
def f07() -> Book:
|
|
||||||
ch1 = (
|
|
||||||
"<h1>Kapitel mit Fußnoten</h1>"
|
|
||||||
'<p>Aussage A<sup><a epub:type="noteref" href="#fn1" id="ref1">1</a></sup> und B'
|
|
||||||
'<sup><a role="doc-noteref" href="#fn2" id="ref2">2</a></sup>, nochmal A<sup><a epub:type="noteref" href="#fn1" id="ref1b">1</a></sup>.</p>'
|
|
||||||
'<p>Endnote<sup><a epub:type="noteref" href="notes.xhtml#en1" id="eref1">3</a></sup> und kaputt<a epub:type="noteref" href="#fehlt">9</a>.</p>'
|
|
||||||
'<aside epub:type="footnote" id="fn1"><p><a href="#ref1">1</a> Erster Absatz mit <em>Hervorhebung</em> und '
|
|
||||||
'<a href="https://example.org/quelle">Link</a>.</p><p>Zweiter Absatz.</p></aside>'
|
|
||||||
f'<aside epub:type="footnote" id="fn2"><p><a href="#ref2">2</a> Mit Formel <math {M}><msup><mi>x</mi><mn>3</mn></msup></math> '
|
|
||||||
"und Code <code>f(x)</code>.</p></aside>"
|
|
||||||
)
|
|
||||||
notes = (
|
|
||||||
'<section epub:type="endnotes"><h1>Anmerkungen</h1>'
|
|
||||||
'<aside epub:type="endnote" id="en1"><p><a href="ch1.xhtml#eref1">3</a> Eine Endnote mit Rücklink.</p></aside></section>'
|
|
||||||
)
|
|
||||||
return Book(title="Fußnoten und Endnoten", docs=[Doc("ch1.xhtml", ch1, "Kapitel"), Doc("notes.xhtml", notes, "Anmerkungen")])
|
|
||||||
|
|
||||||
|
|
||||||
def f08() -> Book:
|
|
||||||
ch = ('<h1>Hauptteil</h1><p>Der Begriff <a href="glossar.xhtml#g-api">API</a> und '
|
|
||||||
'<span id="idx-rekursion">Rekursion</span> nach <span id="p-knuth">Knuth</span> [<a href="lit.xhtml#knuth">Knu97</a>].</p>')
|
|
||||||
glossary = ('<section epub:type="glossary"><h1>Glossar</h1><dl>'
|
|
||||||
'<dt id="g-api">API</dt><dd>Application Programming Interface, eine Schnittstelle.</dd>'
|
|
||||||
"<dt>Rekursion</dt><dd><p>Selbstaufruf einer Funktion.</p><p>Siehe auch Iteration.</p></dd></dl></section>")
|
|
||||||
lit = ('<section epub:type="bibliography"><h1>Literatur</h1><ol>'
|
|
||||||
'<li id="knuth">Knuth, D. E.: <em>The Art of Computer Programming</em>, Bd. 1, 3. Aufl., Addison-Wesley, 1997, '
|
|
||||||
"ISBN 978-0-201-89683-1, S. 1–650.</li>"
|
|
||||||
'<li>Doe, J.: Ein Artikel. <em>Journal</em> 12 (2020), DOI <a href="https://doi.org/10.1000/xyz123">10.1000/xyz123</a>.</li>'
|
|
||||||
'<li>Webquelle: <a href="https://example.org/seite?a=1&b=2">https://example.org/seite?a=1&b=2</a> '
|
|
||||||
"(Zugriff: 01.02.2024).</li></ol></section>")
|
|
||||||
sach = ('<section epub:type="index"><h1>Sachregister</h1>'
|
|
||||||
'<div class="group"><h2>A</h2><p class="index1">API <a href="glossar.xhtml#g-api">Glossar</a></p></div>'
|
|
||||||
'<div class="group"><h2>R</h2><p class="index1">Rekursion <a href="main.xhtml#idx-rekursion">17</a>, 42</p>'
|
|
||||||
'<p class="index2">endständige, 45</p></div></section>')
|
|
||||||
pers = ('<section epub:type="index"><h1>Personenregister</h1><h2>K</h2>'
|
|
||||||
'<p>Knuth, Donald E. <a href="main.xhtml#p-knuth">17</a></p></section>')
|
|
||||||
return Book(title="Glossar, Literatur, Register", docs=[
|
|
||||||
Doc("main.xhtml", ch, "Hauptteil"), Doc("glossar.xhtml", glossary, "Glossar"), Doc("lit.xhtml", lit, "Literatur"),
|
|
||||||
Doc("sach.xhtml", sach, "Sachregister"), Doc("pers.xhtml", pers, "Personenregister")])
|
|
||||||
|
|
||||||
|
|
||||||
def f09() -> Book:
|
|
||||||
body = (
|
|
||||||
"<h1>Komplexe Tabelle</h1>"
|
|
||||||
"<table><caption>Tabelle 9.1: Messwerte</caption>"
|
|
||||||
'<thead><tr><th rowspan="2">Probe</th><th colspan="2">Messung</th></tr><tr><th>vorher</th><th>nachher</th></tr></thead>'
|
|
||||||
"<tbody><tr><td>A</td><td><p>1,5</p><p><em>korrigiert</em></p></td><td><strong>2,0</strong></td></tr>"
|
|
||||||
'<tr><td>B</td><td colspan="2">ausgefallen<br/>siehe Text</td></tr></tbody></table>'
|
|
||||||
)
|
|
||||||
return Book(title="Komplexe Tabelle", docs=[Doc("t.xhtml", body, "Tabelle")])
|
|
||||||
|
|
||||||
|
|
||||||
def f10() -> Book:
|
|
||||||
body = ('<h1>Böse Pfade</h1><p><img src="../../outside.png" alt="raus"/> '
|
|
||||||
'<a href="../escape.xhtml">escape</a> <img src="/absolute/path.png" alt="abs"/></p>')
|
|
||||||
return Book(title="Malicious", docs=[Doc("ok.xhtml", body, "OK")],
|
|
||||||
extra_entries={"../../outside.txt": b"evil", "../escape.xhtml": b"<html/>", "/absolute/path.txt": b"evil",
|
|
||||||
"OEBPS/../../../tmp/qa-epub2md-escape.txt": b"evil"})
|
|
||||||
|
|
||||||
|
|
||||||
def f12() -> Book:
|
|
||||||
text = ("Ärger über Öl und Straße – „deutsche Anführungszeichen“, ‚einfach‘ und «guillemets» — "
|
|
||||||
"café, crème brûlée, naïve, Noël; Ελληνικά: αβγ ΔΣΩ; Mathe: ∀x∈ℝ: x² ≥ 0, ∑ᵢ aᵢ ≈ π, ∞ ≠ ∅ … ")
|
|
||||||
body = f'<h1 id="größe">Größe & Übermaß</h1><p>{text}</p><p><a href="#größe">Rücksprung</a></p>'
|
|
||||||
return Book(title="Kodierung – Ümläute", authors=["Zoë Ørsted"],
|
|
||||||
docs=[Doc("kapitel-ü.xhtml", body, "Größe"), Doc("café.xhtml", "<h1>Café crème</h1><p>Ça va.</p>", "Café")])
|
|
||||||
|
|
||||||
|
|
||||||
TYPO_CSS = b"""
|
|
||||||
p { font-family: Georgia, serif; }
|
|
||||||
.kursiv { font-style: italic; } .fett { font-weight: bold; } span.u { text-decoration: underline; }
|
|
||||||
.weg { display: none; } .markiert { background-color: #ffff00; } .rot { color: #cc0000; }
|
|
||||||
.gross { font-size: 1.6em; } .kap { font-variant: small-caps; } p.mitte { text-align: center; }
|
|
||||||
@media print { .kursiv { font-weight: bold; } }
|
|
||||||
"""
|
|
||||||
|
|
||||||
|
|
||||||
def f14() -> Book:
|
|
||||||
"""Typografie: CSS-Hervorhebungen, unsichtbare Zeichen, Szenenwechsel, Gedicht, Kästen, Stile, Seitenmarken."""
|
|
||||||
c1 = ('<h1>Gestaltung</h1>'
|
|
||||||
'<p>Ein <span class="kursiv">kursives</span> und <span class="fett">fettes</span> Wort, '
|
|
||||||
'<span class="u">unterstrichen</span><span class="weg">VERSTECKT</span>.</p>'
|
|
||||||
'<p>Null\u200bbreite und wei\u00adches Trenn\u00adzeichen.</p>'
|
|
||||||
'<p>Szene eins endet hier.</p><p class="leer"> </p><p>Szene zwei beginnt hier.</p>'
|
|
||||||
'<div class="poem"><p>Zeile eins</p><p>Zeile zwei</p></div>'
|
|
||||||
'<p>Ein <span class="markiert">markierter</span>, <span class="rot">roter</span> und '
|
|
||||||
'<span class="gross">großer</span> Text in <span class="kap">Kapitälchen</span>, '
|
|
||||||
'<span lang="en">an English phrase</span>.</p>'
|
|
||||||
'<p class="mitte">Zentrierte Widmung</p>'
|
|
||||||
'<p>Seite elf endet<span epub:type="pagebreak" id="page12" title="12"/> und Seite zwölf '
|
|
||||||
'geht weiter<a id="page_13"/> bis dreizehn.</p>'
|
|
||||||
'<pre><code>x = 1\u200b # bleibt</code></pre>')
|
|
||||||
c2 = ('<h1>Kästen</h1>'
|
|
||||||
'<div class="box hinweis"><h3 class="boxheading"><span class="box_icon">[»]</span>Wichtige Regel</h3>'
|
|
||||||
'<p>Immer testen.</p></div>'
|
|
||||||
'<aside epub:type="tip"><p>Zwischendurch speichern.</p></aside><p>Nachher.</p>')
|
|
||||||
return Book(title="Typografie", authors=["Tina Typo"], media={"style.css": TYPO_CSS},
|
|
||||||
docs=[Doc("c1.xhtml", c1, "Gestaltung"), Doc("c2.xhtml", c2, "Kästen")])
|
|
||||||
|
|
||||||
|
|
||||||
def main() -> None:
|
|
||||||
OUT.mkdir(parents=True, exist_ok=True)
|
|
||||||
for name, book in [
|
|
||||||
("01-minimal-epub2.epub", f01()), ("02-minimal-epub3-nav.epub", f02()), ("03-chapters-and-links.epub", f03()),
|
|
||||||
("04-code-and-tables.epub", f04()), ("05-images-and-svg.epub", f05()), ("06-mathml-and-equations.epub", f06()),
|
|
||||||
("07-footnotes-and-endnotes.epub", f07()), ("08-glossary-bibliography-index.epub", f08()),
|
|
||||||
("09-complex-table.epub", f09()), ("10-malicious-paths.epub", f10()), ("12-encoding-edge-cases.epub", f12()),
|
|
||||||
("14-typography.epub", f14()),
|
|
||||||
]:
|
|
||||||
build_epub(OUT / name, book)
|
|
||||||
# 10: additionally a manifest item that points outside the EPUB root
|
|
||||||
p10 = OUT / "10-malicious-paths.epub"
|
|
||||||
with zipfile.ZipFile(p10) as z:
|
|
||||||
opf = z.read("OEBPS/content.opf").decode()
|
|
||||||
opf = opf.replace("</manifest>", '<item id="evil" href="../../escape.xhtml" media-type="application/xhtml+xml"/></manifest>')
|
|
||||||
rewrite(p10, {"OEBPS/content.opf": opf.encode()})
|
|
||||||
|
|
||||||
# 11: broken container variants
|
|
||||||
base = Book(title="Defekt", docs=[Doc("a.xhtml", "<h1>A</h1><p>x</p>", "A")])
|
|
||||||
p11 = build_epub(OUT / "11-broken-container.epub", base)
|
|
||||||
rewrite(p11, {"META-INF/container.xml": (
|
|
||||||
b'<?xml version="1.0"?><container version="1.0" xmlns="urn:oasis:names:tc:opendocument:xmlns:container">'
|
|
||||||
b'<rootfiles><rootfile full-path="OEBPS/gibt-es-nicht.opf" media-type="application/oebps-package+xml"/>'
|
|
||||||
b"</rootfiles></container>")})
|
|
||||||
p11b = build_epub(OUT / "11b-broken-spine.epub", Book(title="Spine", docs=[Doc("a.xhtml", "<h1>A</h1><p>x</p>", "A")],
|
|
||||||
spine_extra='<itemref idref="fehlt"/>'))
|
|
||||||
assert p11b.exists()
|
|
||||||
p11c = build_epub(OUT / "11c-broken-opf-xml.epub", base)
|
|
||||||
rewrite(p11c, {"OEBPS/content.opf": b"<package><metadata><dc:title>kaputt"})
|
|
||||||
p11d = build_epub(OUT / "11d-missing-spine-file.epub", base)
|
|
||||||
rewrite(p11d, {"OEBPS/a.xhtml": None})
|
|
||||||
(OUT / "11e-not-a-zip.epub").write_bytes(b"PK\x03\x04 kein echtes zip")
|
|
||||||
p11f = build_epub(OUT / "11f-missing-container.epub", base)
|
|
||||||
rewrite(p11f, {"META-INF/container.xml": None})
|
|
||||||
p11g = build_epub(OUT / "11g-broken-xhtml.epub", base)
|
|
||||||
rewrite(p11g, {"OEBPS/a.xhtml": b"<html><body><h1>Kaputt<p>nicht geschlossen &unbekannt; <b>fett</body>"})
|
|
||||||
print("\n".join(sorted(p.name for p in OUT.glob("*.epub"))))
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
main()
|
|
||||||
|
|
@ -1,18 +0,0 @@
|
||||||
#!/bin/bash
|
|
||||||
# Kompletter Pandoc-Durchlauf: jedes EPUB aus Beispiele/ konvertieren, zusammenfügen, alle Pandoc-Befehle der
|
|
||||||
# Buch-README ausführen, beide PDF-Varianten erzeugen und prüfen. Aufruf im Repo-Wurzelverzeichnis:
|
|
||||||
# tools/qa/pandoc_full.sh Ergebnis in .qa/output/pandoc/ (check.json, pdf-check.txt)
|
|
||||||
# Python über $PYTHON (Standard: python3); braucht pandoc, weasyprint, xelatex, poppler-utils.
|
|
||||||
PY=${PYTHON:-python3}
|
|
||||||
Q=$(dirname "$0")
|
|
||||||
P=.qa/output/pandoc; rm -rf $P; mkdir -p $P
|
|
||||||
for f in Beispiele/*.epub; do
|
|
||||||
n=$(basename "$f" .epub | cut -d_ -f1 | cut -d- -f1); n=${n/Krooß/Krooss}
|
|
||||||
$PY epub2md.py "$f" "$P/$n" > /dev/null 2> "$P/$n.convert.err"; echo "$n convert rc=$?"
|
|
||||||
$PY epub2md_merge.py "$P/$n" > "$P/$n.merge.log" 2>&1; echo "$n merge rc=$? $(head -1 "$P/$n.merge.log" | cut -c1-120)"
|
|
||||||
SKIP_PDF=1 "$Q/pandoc_run.sh" "$P/$n"
|
|
||||||
"$Q/pandoc_pdfs.sh" "$P/$n"
|
|
||||||
done
|
|
||||||
$PY "$Q/check_pandoc.py" $P/*/ > $P/check.json
|
|
||||||
"$Q/check_pdf.sh" $P/*/*.pdf > $P/pdf-check.txt
|
|
||||||
echo FERTIG
|
|
||||||
|
|
@ -1,15 +0,0 @@
|
||||||
#!/bin/bash
|
|
||||||
# Erzeugt beide PDF-Varianten aus der Buch-README unter getrennten Namen: *.weasy.pdf und *.latex.pdf
|
|
||||||
# (in der README schreiben beide Befehle dieselbe Datei). Aufruf im Repo-Wurzelverzeichnis.
|
|
||||||
mkdir -p .qa
|
|
||||||
for dir in "$@"; do
|
|
||||||
mkdir -p "$dir/pandoc-logs"
|
|
||||||
grep -E '^pandoc .*\.pdf' "$dir/README.md" | while IFS= read -r cmd; do
|
|
||||||
case "$cmd" in *weasyprint*) kind=weasy;; *) kind=latex;; esac
|
|
||||||
cmd=$(printf '%s' "$cmd" | sed -E "s/\.pdf('?)\$/.$kind.pdf\1/")
|
|
||||||
s=$(date +%s)
|
|
||||||
(cd "$dir" && timeout 1800 bash -c "$cmd") > "$dir/pandoc-logs/$kind.out" 2> "$dir/pandoc-logs/$kind.err"; rc=$?
|
|
||||||
printf '%s\t%s\trc=%s\t%ss\n' "$(basename "$dir")" $kind $rc $(( $(date +%s)-s ))
|
|
||||||
printf '%s\trc=%s\tpandoc %s-PDF (%s)\n' "$(date -Iseconds)" $rc $kind "$(basename "$dir")" >> .qa/commands.log
|
|
||||||
done
|
|
||||||
done
|
|
||||||
|
|
@ -1,18 +0,0 @@
|
||||||
#!/bin/bash
|
|
||||||
# Führt jede "pandoc …"-Zeile aus der README.md eines konvertierten Buches im Buchordner aus.
|
|
||||||
# Aufruf im Repo-Wurzelverzeichnis: tools/qa/pandoc_run.sh BUCHORDNER... (SKIP_PDF=1: ohne PDF-Befehle)
|
|
||||||
# Logs: BUCHORDNER/pandoc-logs/NN.{cmd,out,err} und summary.tsv; Kommandos mit rc in .qa/commands.log
|
|
||||||
mkdir -p .qa
|
|
||||||
for dir in "$@"; do
|
|
||||||
mkdir -p "$dir/pandoc-logs"
|
|
||||||
n=0
|
|
||||||
grep -E '^pandoc ' "$dir/README.md" | { [ -n "$SKIP_PDF" ] && grep -v '\.pdf' || cat; } | while IFS= read -r cmd; do
|
|
||||||
n=$((n+1)); f=$(printf '%s/pandoc-logs/%02d' "$dir" $n)
|
|
||||||
printf '%s\n' "$cmd" > "$f.cmd"
|
|
||||||
s=$(date +%s)
|
|
||||||
(cd "$dir" && timeout 1800 bash -c "$cmd") > "$f.out" 2> "$f.err"; rc=$?
|
|
||||||
printf '%s\trc=%s\t%ss\t%s warn\t%s\n' "$(basename "$dir")" $rc $(( $(date +%s)-s )) "$(grep -c . "$f.err")" \
|
|
||||||
"$(echo "$cmd" | grep -o -- '-o [^ ]*\.[a-z]*' | tail -1)" | tee -a "$dir/pandoc-logs/summary.tsv"
|
|
||||||
printf '%s\trc=%s\tpandoc (%s)\n' "$(date -Iseconds)" $rc "$(basename "$dir") #$n" >> .qa/commands.log
|
|
||||||
done
|
|
||||||
done
|
|
||||||
|
|
@ -1,36 +0,0 @@
|
||||||
"""Listet Python-Codeblöcke mit Einrückungsfehlern in Markdown-Dateien (mit Inhalt, zum Vergleich mit der Quelle).
|
|
||||||
|
|
||||||
Aufruf: python tools/qa/pyblocks.py AUSGABE/chapters/*.md
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import ast
|
|
||||||
import re
|
|
||||||
import sys
|
|
||||||
|
|
||||||
|
|
||||||
def main() -> None:
|
|
||||||
for path in sys.argv[1:]:
|
|
||||||
with open(path, encoding="utf-8") as fh:
|
|
||||||
lines = fh.read().split("\n")
|
|
||||||
i = 0
|
|
||||||
while i < len(lines):
|
|
||||||
m = re.match(r"^(`{3,})python\s*$", lines[i])
|
|
||||||
if m:
|
|
||||||
j = i + 1
|
|
||||||
while j < len(lines) and not lines[j].startswith(m.group(1)):
|
|
||||||
j += 1
|
|
||||||
try:
|
|
||||||
ast.parse("\n".join(lines[i + 1:j]))
|
|
||||||
except IndentationError as exc:
|
|
||||||
print(f"== {path}:{i + 1} {exc.msg} (Blockzeile {exc.lineno})")
|
|
||||||
print("\n".join(f"{k + 1:3}|{line}" for k, line in enumerate(lines[i + 1:j])))
|
|
||||||
except SyntaxError:
|
|
||||||
pass # fragments and program output are no indentation defects
|
|
||||||
i = j
|
|
||||||
i += 1
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
main()
|
|
||||||
|
|
@ -1,5 +0,0 @@
|
||||||
# Zusätzlich zu requirements-dev.txt für die QA-Skripte (unabhängiger Markdown-Validator, Coverage)
|
|
||||||
-r ../../requirements-dev.txt
|
|
||||||
markdown-it-py>=3.0
|
|
||||||
mdit-py-plugins>=0.4
|
|
||||||
coverage>=7.0
|
|
||||||
|
|
@ -1,418 +0,0 @@
|
||||||
"""CLI-Testmatrix gegen die QA-Fixtures. Jeder Lauf schreibt in ein neues Verzeichnis unter .qa/output/.
|
|
||||||
|
|
||||||
Ergebnis: .qa/output/matrix-results.json, Kommandos mit Rückgabecode in .qa/commands.log
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
import re
|
|
||||||
import shutil
|
|
||||||
import stat
|
|
||||||
import subprocess
|
|
||||||
import sys
|
|
||||||
from datetime import datetime
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
ROOT = Path(__file__).resolve().parents[2]
|
|
||||||
QA = ROOT / ".qa"
|
|
||||||
FX = QA / "generated-fixtures"
|
|
||||||
OUT = QA / "output" / "matrix"
|
|
||||||
PY = sys.executable
|
|
||||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
|
||||||
from validate_output import validate # noqa: E402
|
|
||||||
|
|
||||||
results: list[dict] = []
|
|
||||||
|
|
||||||
|
|
||||||
def run(args: list[str]) -> subprocess.CompletedProcess:
|
|
||||||
cmd = [PY, "epub2md.py", *args]
|
|
||||||
cp = subprocess.run(cmd, cwd=ROOT, capture_output=True, text=True, timeout=600)
|
|
||||||
with open(QA / "commands.log", "a", encoding="utf-8") as fh:
|
|
||||||
rel = " ".join(a.replace(str(ROOT) + "/", "") for a in cmd[1:])
|
|
||||||
fh.write(f"{datetime.now().isoformat(timespec='seconds')}\trc={cp.returncode}\tpython {rel}\n")
|
|
||||||
return cp
|
|
||||||
|
|
||||||
|
|
||||||
def merge_case(name: str, book_dir: Path, expect_rc: int, checks=None) -> dict:
|
|
||||||
"""epub2md_merge.py on an existing output directory."""
|
|
||||||
cmd = [PY, "epub2md_merge.py", str(book_dir)]
|
|
||||||
cp = subprocess.run(cmd, cwd=ROOT, capture_output=True, text=True, timeout=600)
|
|
||||||
with open(QA / "commands.log", "a", encoding="utf-8") as fh:
|
|
||||||
fh.write(f"{datetime.now().isoformat(timespec='seconds')}\trc={cp.returncode}\tpython epub2md_merge.py "
|
|
||||||
f"{str(book_dir).replace(str(ROOT) + '/', '')}\n")
|
|
||||||
entry: dict = {"case": name, "args": [str(book_dir)], "rc": cp.returncode, "expected_rc": expect_rc,
|
|
||||||
"ok": cp.returncode == expect_rc and "Traceback" not in cp.stderr, "notes": []}
|
|
||||||
for label, fn in checks or []:
|
|
||||||
good = bool(fn(book_dir, cp))
|
|
||||||
entry["ok"] = entry["ok"] and good
|
|
||||||
entry["notes"].append(("OK: " if good else "FEHLER: ") + label)
|
|
||||||
results.append(entry)
|
|
||||||
print(("PASS " if entry["ok"] else "FAIL ") + name, "rc=", cp.returncode)
|
|
||||||
return entry
|
|
||||||
|
|
||||||
|
|
||||||
def read(out: Path, pattern: str) -> str:
|
|
||||||
return "\n".join(p.read_text(encoding="utf-8") for p in sorted(out.glob(pattern)))
|
|
||||||
|
|
||||||
|
|
||||||
def case(name: str, fixture: str, args: list[str], expect_rc: int, checks=None, validate_out: bool = True,
|
|
||||||
out: Path | None = None, keep: bool = False) -> dict:
|
|
||||||
out = out or OUT / name
|
|
||||||
if out.exists() and not keep:
|
|
||||||
shutil.rmtree(out)
|
|
||||||
src = str(FX / fixture) if fixture and not fixture.startswith("/") else fixture
|
|
||||||
argv = ([src] if src else []) + ([str(out)] if src else []) + args
|
|
||||||
cp = run(argv)
|
|
||||||
entry: dict = {"case": name, "fixture": fixture, "args": args, "rc": cp.returncode, "expected_rc": expect_rc,
|
|
||||||
"ok": cp.returncode == expect_rc, "notes": [], "stderr_tail": cp.stderr.strip().splitlines()[-3:]}
|
|
||||||
if not entry["ok"]:
|
|
||||||
entry["notes"].append(f"Rückgabecode {cp.returncode}, erwartet {expect_rc}")
|
|
||||||
if "Traceback (most recent call last)" in cp.stderr:
|
|
||||||
entry["ok"] = False
|
|
||||||
entry["notes"].append("Traceback im Normalmodus")
|
|
||||||
if checks:
|
|
||||||
for label, fn in checks:
|
|
||||||
try:
|
|
||||||
good = bool(fn(out, cp))
|
|
||||||
except Exception as exc: # noqa: BLE001
|
|
||||||
good = False
|
|
||||||
label = f"{label} (Exception {type(exc).__name__}: {exc})"
|
|
||||||
if not good:
|
|
||||||
entry["ok"] = False
|
|
||||||
entry["notes"].append(("OK: " if good else "FEHLER: ") + label)
|
|
||||||
if validate_out and cp.returncode == 0 and out.exists() and "--dry-run" not in args:
|
|
||||||
v = validate(out)
|
|
||||||
entry["validator"] = v["severity_counts"]
|
|
||||||
entry["validator_issues"] = [i for i in v["issues"] if i["severity"] != "info"][:15]
|
|
||||||
if any(i["severity"] in ("critical", "high") for i in v["issues"]):
|
|
||||||
entry["ok"] = False
|
|
||||||
entry["notes"].append("Validator: kritische/hohe Befunde")
|
|
||||||
results.append(entry)
|
|
||||||
print(("PASS " if entry["ok"] else "FAIL ") + name, "rc=", cp.returncode, "; ".join(n for n in entry["notes"] if n.startswith("FEHLER") or "erwartet" in n or "Validator" in n or "Traceback" in n))
|
|
||||||
return entry
|
|
||||||
|
|
||||||
|
|
||||||
def ch(out: Path, prefix: str = "") -> str:
|
|
||||||
return read(out / "chapters", f"{prefix}*.md")
|
|
||||||
|
|
||||||
|
|
||||||
def report(out: Path) -> dict:
|
|
||||||
return json.loads((out / "conversion-report.json").read_text(encoding="utf-8"))
|
|
||||||
|
|
||||||
|
|
||||||
def main() -> None:
|
|
||||||
if OUT.exists():
|
|
||||||
shutil.rmtree(OUT)
|
|
||||||
OUT.mkdir(parents=True)
|
|
||||||
# --- help
|
|
||||||
cp = run(["--help"])
|
|
||||||
opts = ["--format", "--extract-media", "--no-extract-media", "--copy-original", "--chapter-level", "--toc", "--no-toc",
|
|
||||||
"--frontmatter", "--no-frontmatter", "--footnotes", "--math", "--images", "--slug-language", "--keep-html",
|
|
||||||
"--strict", "--force", "--verbose", "--dry-run", "--report", "--code-language", "--keep-wrap-markers",
|
|
||||||
"--keep-invisible-chars", "--css-emphasis", "--no-css-emphasis", "--boxes", "--styles", "--page-markers"]
|
|
||||||
missing = [o for o in opts if o not in cp.stdout]
|
|
||||||
results.append({"case": "help", "rc": cp.returncode, "expected_rc": 0, "ok": cp.returncode == 0 and not missing,
|
|
||||||
"notes": [f"fehlende Optionen in --help: {missing}"] if missing else ["alle Optionen dokumentiert"]})
|
|
||||||
print("PASS help" if not missing else f"FAIL help {missing}")
|
|
||||||
|
|
||||||
# --- basics on all good fixtures
|
|
||||||
for fx in ["01-minimal-epub2", "02-minimal-epub3-nav", "03-chapters-and-links", "04-code-and-tables", "05-images-and-svg",
|
|
||||||
"06-mathml-and-equations", "07-footnotes-and-endnotes", "08-glossary-bibliography-index", "09-complex-table",
|
|
||||||
"12-encoding-edge-cases"]:
|
|
||||||
case(f"basic-{fx[:2]}", f"{fx}.epub", [], 0)
|
|
||||||
|
|
||||||
f03 = "03-chapters-and-links.epub"
|
|
||||||
case("dry-run", f03, ["--dry-run", "--report", str(OUT / "dry-report.json")], 0, [
|
|
||||||
("kein Ausgabeordner", lambda o, c: not o.exists()),
|
|
||||||
("kein Report", lambda o, c: not (OUT / "dry-report.json").exists()),
|
|
||||||
("Planung im stdout", lambda o, c: "chapters/001-" in c.stdout)])
|
|
||||||
case("verbose", f03, ["--verbose"], 0, [("Schritte auf stderr", lambda o, c: "Rendere" in c.stderr and "Kapitel gebildet" in c.stderr)])
|
|
||||||
case("chapter-level-1", f03, ["--chapter-level", "1"], 0, [
|
|
||||||
("3+1 Kapitel (A, A2, B, C)", lambda o, c: len(list((o / "chapters").glob("*.md"))) == 4)])
|
|
||||||
case("chapter-level-2", f03, ["--chapter-level", "2"], 0, [
|
|
||||||
("h2 erzeugt eigene Dateien", lambda o, c: len(list((o / "chapters").glob("*.md"))) == 7)])
|
|
||||||
case("links-03", f03, [], 0, [
|
|
||||||
("Link Kapitel→Kapitel", lambda o, c: "[Kapitel B](003-kapitel-b.md)" in ch(o, "001")),
|
|
||||||
("Anker im selben EPUB-Dokument, anderes Kapitel", lambda o, c: "[Anker unten](002-kapitel-a2-in-derselben-datei.md#a-unten)" in ch(o, "001")),
|
|
||||||
("Anker in anderem Kapitel", lambda o, c: "[Ziel in B](003-kapitel-b.md#b-ziel)" in ch(o, "001")),
|
|
||||||
("nicht auflösbar → Text + Report", lambda o, c: "Nichts" in ch(o, "001") and "nirgendwo" not in ch(o, "001")
|
|
||||||
and any(u["href"] == "nirgendwo.xhtml#weg" for u in report(o)["unresolved_links"])),
|
|
||||||
("fehlender Anker → Warnung", lambda o, c: any(w["code"] == "anchor-missing" for w in report(o)["warnings"]))])
|
|
||||||
|
|
||||||
f04 = "04-code-and-tables.epub"
|
|
||||||
case("format-gfm", f04, ["--format", "gfm"], 0, [
|
|
||||||
("Tabelle mit | in Code als HTML (QA2-01)", lambda o, c: "<code>ls | wc -l</code>" in ch(o) and "| Befehl |" not in ch(o))])
|
|
||||||
case("format-commonmark", f04, ["--format", "commonmark"], 0, [("HTML-Tabelle statt Pipe", lambda o, c: "<table>" in ch(o) and "| Befehl |" not in ch(o))])
|
|
||||||
case("code-04", f04, [], 0, [
|
|
||||||
("Python-Einrückung + Tab exakt", lambda o, c: " def push(self, x):\n\tself.items.append(x) # Tab-Einrückung\n if len(self.items) > 10 and x < 0:" in ch(o)),
|
|
||||||
("Sprachlabel python", lambda o, c: "```python\nclass Stack:" in ch(o)),
|
|
||||||
("Shell exakt, Label aus class=shell", lambda o, c: "```bash\n$ find . -name '*.py' | xargs grep -n \"TODO\" \\\n | sort -u > todo.txt\n$ echo $HOME\n```" in ch(o)),
|
|
||||||
("Fence länger als Backticks im Code", lambda o, c: "````\nMarkdown mit ```fences```" in ch(o)),
|
|
||||||
("json via data-language", lambda o, c: "```json\n{" in ch(o)),
|
|
||||||
("c via #include", lambda o, c: "```c\n#include <stdio.h>" in ch(o)),
|
|
||||||
("Inline-Code mit < & ", lambda o, c: "`len(x) < 3 && y`" in ch(o)),
|
|
||||||
("Inline-Code mit Backtick", lambda o, c: "``a`b``" in ch(o)),
|
|
||||||
("verschachtelte Listen", lambda o, c: "- Punkt 1\n\n - Punkt 1.1\n\n 1. Schritt a\n 2. Schritt b" in ch(o) or "- Punkt 1\n - Punkt 1.1" in ch(o)),
|
|
||||||
("ol start=3", lambda o, c: "3. drei" in ch(o)),
|
|
||||||
("unbekanntes Element gemeldet", lambda o, c: "x-widget" in report(o)["unsupported_elements"] and "Widget-Inhalt" in ch(o))])
|
|
||||||
case("keep-html", f04, ["--keep-html"], 0, [
|
|
||||||
("unbekanntes Element als HTML, ohne onclick", lambda o, c: '<x-widget data-k="v">Widget-Inhalt</x-widget>' in ch(o) and "evil" not in ch(o))])
|
|
||||||
|
|
||||||
f05 = "05-images-and-svg.epub"
|
|
||||||
case("images-relative", f05, ["--images", "relative"], 0, [
|
|
||||||
("Foto einmal extrahiert", lambda o, c: len([p for p in (o / "media/images").iterdir() if "foto" in p.name]) == 1),
|
|
||||||
("SVG bleibt SVG", lambda o, c: any(p.suffix == ".svg" for p in (o / "media/images").iterdir())),
|
|
||||||
("alt + title", lambda o, c: ' and '"Fototitel")' in ch(o)),
|
|
||||||
("Bildunterschrift", lambda o, c: "*Abbildung 1.1: Ein Foto mit Unterschrift.*" in ch(o)),
|
|
||||||
("Caption als Alt bei leerem alt", lambda o, c: "![Abbildung 1.2: Ein SVG-Diagramm.](" in ch(o)),
|
|
||||||
("Inline-SVG extrahiert", lambda o, c: any(m["role"] == "inline-svg" for m in report(o)["media"]["files"])),
|
|
||||||
("fehlendes Bild gewarnt", lambda o, c: any(w["code"] == "image-missing" for w in report(o)["warnings"])),
|
|
||||||
("alles im Ausgabeordner", lambda o, c: all(o.resolve() in p.resolve().parents for p in o.rglob("*")))])
|
|
||||||
case("images-flat", f05, ["--images", "flat"], 0, [("media/<name>", lambda o, c: (o / "media" / "foto 1 ä.png").exists())])
|
|
||||||
case("images-original-path", f05, ["--images", "original-path"], 0, [("media/images/<orig>", lambda o, c: (o / "media/images/foto 1 ä.png").exists())])
|
|
||||||
case("no-extract-media", f05, ["--no-extract-media"], 0, [("kein media/", lambda o, c: not (o / "media").exists()),
|
|
||||||
("Platzhalter statt Link", lambda o, c: "Bild: Ein Foto" in ch(o))])
|
|
||||||
case("extract-media", f05, ["--extract-media"], 0, [("media/ vorhanden", lambda o, c: (o / "media").exists())])
|
|
||||||
|
|
||||||
f06 = "06-mathml-and-equations.epub"
|
|
||||||
math_common = [
|
|
||||||
("Gleichungsnummer sichtbar", lambda o, c: "(2.1)" in ch(o) and "(2.2)" in ch(o)),
|
|
||||||
("SVG-Formel als SVG-Bild", lambda o, c: "![Formel E=mc²](" in ch(o)),
|
|
||||||
("Bildformel als Bild", lambda o, c: "![Summe über i](" in ch(o)),
|
|
||||||
("unbekanntes MathML nicht verloren", lambda o, c: "menclose" in ch(o))]
|
|
||||||
case("math-auto", f06, ["--math", "auto"], 0, math_common + [
|
|
||||||
("Bruch", lambda o, c: r"$\frac{a}{b}$" in ch(o)),
|
|
||||||
("Hoch/Index", lambda o, c: "$x^{2}$" in ch(o) and "$a_{i}$" in ch(o) and "$x_{i}^{2}$" in ch(o)),
|
|
||||||
("Block-Formel", lambda o, c: "$$\ne^{i \\pi } + 1 = 0\n$$" in ch(o) or "$$\ne^{i \\pi} + 1 = 0\n$$" in ch(o)),
|
|
||||||
("TeX-Annotation bevorzugt", lambda o, c: r"$\vec{y}$" in ch(o)),
|
|
||||||
("TeX aus alt-Bild", lambda o, c: r"\int_0^1 x\,dx = \frac{1}{2}" in ch(o)),
|
|
||||||
("Warnungen im Report", lambda o, c: {"mathml-kept", "formula-image"} <= {w["code"] for w in report(o)["warnings"]}),
|
|
||||||
("SVG-Formel als svg im Report", lambda o, c: "svg" in {f["origin"] for f in report(o)["formulas"]["items"]})])
|
|
||||||
case("math-tex", f06, ["--math", "tex"], 0, math_common + [
|
|
||||||
("\\( \\) Begrenzer", lambda o, c: r"\(\frac{a}{b}\)" in ch(o)),
|
|
||||||
("keine $-Begrenzer", lambda o, c: not re.search(r"(?<!\\)\$", ch(o).split("---\n", 2)[2]))])
|
|
||||||
case("math-mathml", f06, ["--math", "mathml"], 0, math_common + [("MathML erhalten", lambda o, c: "<mfrac><mi>a</mi><mi>b</mi></mfrac>" in ch(o))])
|
|
||||||
case("math-image", f06, ["--math", "image"], 0, math_common + [("TeX-Bild als Bild", lambda o, c: "tex" in " ".join(m["source"] for m in report(o)["media"]["files"]))])
|
|
||||||
|
|
||||||
f07 = "07-footnotes-and-endnotes.epub"
|
|
||||||
fn_common = [("Fußnotentext vorhanden", lambda o, c: "Erster Absatz mit" in read(o, "chapters/*.md") and "Zweiter Absatz." in read(o, "chapters/*.md")),
|
|
||||||
("Endnote vorhanden", lambda o, c: "Eine Endnote mit Rücklink." in read(o, "chapters/*.md")),
|
|
||||||
("Formel in Fußnote", lambda o, c: "x^{3}" in read(o, "chapters/*.md")),
|
|
||||||
("kaputter Verweis gewarnt", lambda o, c: any(w["code"] == "noteref-unresolved" for w in report(o)["warnings"]))]
|
|
||||||
case("footnotes-end-of-chapter", f07, ["--footnotes", "end-of-chapter"], 0, fn_common + [
|
|
||||||
("Mehrfachverweis auf eine Definition", lambda o, c: ch(o, "001").count("[^fn-001]") == 3),
|
|
||||||
("keine Doppelung im Fließtext", lambda o, c: ch(o, "001").count("Erster Absatz") == 1)])
|
|
||||||
case("footnotes-end-of-book", f07, ["--footnotes", "end-of-book"], 0, fn_common + [
|
|
||||||
("Anmerkungsdatei", lambda o, c: (o / "chapters" / "999-anmerkungen.md").exists()),
|
|
||||||
("Rücklinks", lambda o, c: "[↩](001-" in ch(o, "999"))])
|
|
||||||
case("footnotes-inline", f07, ["--footnotes", "inline"], 0, fn_common + [("inline", lambda o, c: "(Anmerkung: Erster Absatz" in ch(o, "001"))])
|
|
||||||
case("footnotes-keep-links", f07, ["--footnotes", "keep-links"], 0, fn_common + [("Links", lambda o, c: "](#fn1)" in ch(o, "001"))])
|
|
||||||
|
|
||||||
f08 = "08-glossary-bibliography-index.epub"
|
|
||||||
case("special-sections", f08, [], 0, [
|
|
||||||
("Typen erkannt", lambda o, c: {s["type"] for s in report(o)["special_sections"]} >= {"glossary", "bibliography", "index", "index-persons"}),
|
|
||||||
("Glossarbegriff", lambda o, c: "**API:** Application Programming Interface" in ch(o)),
|
|
||||||
("mehrteilige Definition", lambda o, c: "Selbstaufruf einer Funktion." in ch(o) and "Siehe auch Iteration." in ch(o)),
|
|
||||||
("Literatur vollständig", lambda o, c: all(s in ch(o) for s in ("ISBN 978-0-201-89683-1", "S. 1–650", "(Zugriff: 01.02.2024)"))),
|
|
||||||
("DOI/URL als Link", lambda o, c: "](https://doi.org/10.1000/xyz123)" in ch(o) and "](https://example.org/seite?a=1&b=2)" in ch(o)),
|
|
||||||
("Registerlink ins Kapitel", lambda o, c: "[17](001-hauptteil.md#idx-rekursion)" in ch(o)),
|
|
||||||
("Seitenzahl als Text", lambda o, c: ", 42" in ch(o)),
|
|
||||||
("Untereintrag eingerückt", lambda o, c: "– endständige, 45" in ch(o)),
|
|
||||||
("Buchstabengruppen", lambda o, c: "## R" in ch(o)),
|
|
||||||
("Querverweis ins Glossar", lambda o, c: "[API](002-glossar.md#g-api)" in ch(o, "001"))])
|
|
||||||
|
|
||||||
case("complex-table", "09-complex-table.epub", [], 0, [
|
|
||||||
("HTML-Tabelle mit colspan/rowspan", lambda o, c: 'rowspan="2"' in ch(o) and 'colspan="2"' in ch(o)),
|
|
||||||
("Zellinhalte vollständig", lambda o, c: all(s in ch(o) for s in ("1,5", "korrigiert", "2,0", "ausgefallen", "siehe Text", "Messwerte"))),
|
|
||||||
("Warnung", lambda o, c: any(w["code"] == "table-html" for w in report(o)["warnings"]))])
|
|
||||||
|
|
||||||
f12 = "12-encoding-edge-cases.epub"
|
|
||||||
enc_text = "Ärger über Öl und Straße – „deutsche Anführungszeichen“, ‚einfach‘ und «guillemets» — café, crème brûlée"
|
|
||||||
case("slug-de", f12, ["--slug-language", "de"], 0, [("ä→ae, ß→ss", lambda o, c: (o / "chapters/001-groesse-uebermass.md").exists()),
|
|
||||||
("Text unverändert UTF-8", lambda o, c: enc_text in ch(o) and "αβγ ΔΣΩ" in ch(o) and "∀x∈ℝ: x² ≥ 0" in ch(o))])
|
|
||||||
case("slug-en", f12, ["--slug-language", "en"], 0, [("ä→a", lambda o, c: (o / "chapters/001-grosse-ubermass.md").exists())])
|
|
||||||
case("slug-none", f12, ["--slug-language", "none"], 0, [("Unicode", lambda o, c: (o / "chapters/001-größe-übermaß.md").exists())])
|
|
||||||
|
|
||||||
f02 = "02-minimal-epub3-nav.epub"
|
|
||||||
case("copy-original", f02, ["--copy-original"], 0, [("source/", lambda o, c: (o / "source" / f02).exists())])
|
|
||||||
case("no-copy-original", f02, ["--no-copy-original"], 0, [("kein source/", lambda o, c: not (o / "source").exists())])
|
|
||||||
case("toc", f02, ["--toc"], 0, [("README mit TOC", lambda o, c: "## Inhaltsverzeichnis" in (o / "README.md").read_text())])
|
|
||||||
case("no-toc", f02, ["--no-toc"], 0, [("kein README", lambda o, c: not (o / "README.md").exists())])
|
|
||||||
case("frontmatter", f02, ["--frontmatter"], 0, [("YAML", lambda o, c: ch(o).startswith("---\n"))])
|
|
||||||
case("no-frontmatter", f02, ["--no-frontmatter"], 0, [("kein YAML", lambda o, c: not ch(o).startswith("---"))])
|
|
||||||
rep = OUT / "reports" / "r.json"
|
|
||||||
case("report", f02, ["--report", str(rep)], 0, [("Report-Datei", lambda o, c: json.loads(rep.read_text())["tool"] == "epub2md")])
|
|
||||||
|
|
||||||
# --- force / existing directory
|
|
||||||
fo = OUT / "force-target"
|
|
||||||
case("force-1-initial", f02, [], 0, out=fo)
|
|
||||||
case("force-2-exists-no-force", f02, [], 1, [("Meldung", lambda o, c: "existiert bereits" in c.stderr)], out=fo, keep=True, validate_out=False)
|
|
||||||
(fo / "zusatz.txt").write_text("x")
|
|
||||||
case("force-3-with-force", f02, ["--force"], 0, [("ersetzt", lambda o, c: not (o / "zusatz.txt").exists())], out=fo, keep=True)
|
|
||||||
foreign = OUT / "fremd"
|
|
||||||
foreign.mkdir()
|
|
||||||
(foreign / "wichtig.txt").write_text("nicht löschen")
|
|
||||||
case("force-4-foreign-dir", f02, ["--force"], 1, [("fremder Ordner bleibt", lambda o, c: (o / "wichtig.txt").exists())],
|
|
||||||
out=foreign, keep=True, validate_out=False)
|
|
||||||
|
|
||||||
# --- invalid options
|
|
||||||
case("invalid-math", f02, ["--math", "foo"], 1, [("Meldung", lambda o, c: "invalid choice" in c.stderr)], validate_out=False)
|
|
||||||
case("invalid-chapter-level", f02, ["--chapter-level", "0"], 1, validate_out=False)
|
|
||||||
case("invalid-chapter-level-text", f02, ["--chapter-level", "x"], 1, validate_out=False)
|
|
||||||
case("missing-input", "/nonexistent/datei.epub", [], 1, validate_out=False)
|
|
||||||
case("unknown-option", f02, ["--gibt-es-nicht"], 1, validate_out=False)
|
|
||||||
|
|
||||||
# --- broken input
|
|
||||||
case("broken-container", "11-broken-container.epub", [], 2, [("verständlich", lambda o, c: "OPF" in c.stderr), ("keine Ausgabe", lambda o, c: not o.exists())], validate_out=False)
|
|
||||||
case("broken-spine", "11b-broken-spine.epub", [], 0, [("Fehler im Report", lambda o, c: report(o)["status"] == "success-with-errors")])
|
|
||||||
case("broken-spine-strict", "11b-broken-spine.epub", ["--strict"], 3, [("keine Ausgabe", lambda o, c: not o.exists())], validate_out=False)
|
|
||||||
case("broken-opf-xml", "11c-broken-opf-xml.epub", [], 3, [("keine Ausgabe", lambda o, c: not o.exists())], validate_out=False)
|
|
||||||
case("missing-spine-file", "11d-missing-spine-file.epub", [], 3, [("verständlich", lambda o, c: "Spine" in c.stderr)], validate_out=False)
|
|
||||||
case("not-a-zip", "11e-not-a-zip.epub", [], 2, validate_out=False)
|
|
||||||
case("missing-container", "11f-missing-container.epub", [], 2, validate_out=False)
|
|
||||||
case("broken-xhtml", "11g-broken-xhtml.epub", [], 0, [("Text gerettet", lambda o, c: "nicht geschlossen" in ch(o)),
|
|
||||||
("Hinweis html-fallback/Entity", lambda o, c: True)])
|
|
||||||
case("strict-ok", "01-minimal-epub2.epub", ["--strict"], 0)
|
|
||||||
|
|
||||||
# --- security
|
|
||||||
escape = Path("/tmp/qa-epub2md-escape.txt")
|
|
||||||
escape.unlink(missing_ok=True)
|
|
||||||
case("malicious", "10-malicious-paths.epub", [], 0, [
|
|
||||||
("nichts außerhalb geschrieben", lambda o, c: not escape.exists() and not (ROOT / "outside.txt").exists()
|
|
||||||
and not (QA / "outside.txt").exists() and not (OUT / "outside.txt").exists()),
|
|
||||||
("alle Dateien im Zielordner", lambda o, c: all(o.resolve() in p.resolve().parents for p in o.rglob("*"))),
|
|
||||||
("unsichere Einträge gemeldet", lambda o, c: any(e["code"] == "unsafe-zip-path" for e in report(o)["errors"])),
|
|
||||||
("unsicherer Manifest-Pfad gemeldet", lambda o, c: any(e["code"] == "unsafe-href" for e in report(o)["errors"]))])
|
|
||||||
case("malicious-strict", "10-malicious-paths.epub", ["--strict"], 2, [("keine Ausgabe", lambda o, c: not o.exists())], validate_out=False)
|
|
||||||
|
|
||||||
# --- write error: parent directory not writable
|
|
||||||
ro = OUT / "readonly"
|
|
||||||
ro.mkdir()
|
|
||||||
os.chmod(ro, stat.S_IRUSR | stat.S_IXUSR)
|
|
||||||
try:
|
|
||||||
case("write-error", f02, [], 1, [("verständliche Meldung", lambda o, c: "Traceback" not in c.stderr and c.stderr.strip() != "")],
|
|
||||||
out=ro / "ziel", validate_out=False)
|
|
||||||
finally:
|
|
||||||
os.chmod(ro, stat.S_IRWXU)
|
|
||||||
|
|
||||||
# --- typography (P1–P7): CSS emphasis, invisible chars, scene breaks, poems, boxes, styles, page markers
|
|
||||||
f14 = "14-typography.epub"
|
|
||||||
body = lambda o: ch(o, "001") # noqa: E731
|
|
||||||
boxes = lambda o: ch(o, "002") # noqa: E731
|
|
||||||
case("typo-default", f14, [], 0, [
|
|
||||||
("CSS-kursiv/fett/unterstrichen", lambda o, c: "Ein *kursives* und **fettes** Wort, <u>unterstrichen</u>." in body(o)),
|
|
||||||
("display:none entfällt ohne Leerzeichen-Rest", lambda o, c: "VERSTECKT" not in body(o) and "</u>." in body(o)),
|
|
||||||
("@media print ignoriert", lambda o, c: "***kursives***" not in body(o)),
|
|
||||||
("unsichtbare Zeichen im Text entfernt", lambda o, c: "Nullbreite und weiches Trennzeichen." in body(o)),
|
|
||||||
("unsichtbare Zeichen im Code erhalten", lambda o, c: "x = 1\u200b # bleibt" in body(o)),
|
|
||||||
("Szenenwechsel", lambda o, c: "Szene eins endet hier.\n\n* * *\n\nSzene zwei beginnt hier." in body(o)),
|
|
||||||
("Gedicht als Zitatblock", lambda o, c: "> Zeile eins\\\n> Zeile zwei" in body(o)),
|
|
||||||
("semantic: <mark>, keine Stile", lambda o, c: "<mark>markierter</mark>" in body(o) and "style=" not in body(o)),
|
|
||||||
("Kasten mit Klassentyp und Titel ohne Symbol", lambda o, c: "> [!NOTE]\n> **Wichtige Regel**\n>\n> Immer testen." in boxes(o)),
|
|
||||||
("aside tip", lambda o, c: "> [!TIP]\n> Zwischendurch speichern." in boxes(o)),
|
|
||||||
("keine Seitenmarken", lambda o, c: "S. 12" not in body(o)),
|
|
||||||
("Bericht", lambda o, c: report(o)["layout"] == {"scene_breaks": 1, "poems": 1, "boxes": 2, "page_markers": 0}
|
|
||||||
and report(o)["text_cleanup"]["invisible_chars_removed"] == 3 and report(o)["css"]["hidden_elements"] == 1),
|
|
||||||
("Buch-README mit Pandoc-Filter", lambda o, c: "--lua-filter" in (o / "README.md").read_text(encoding="utf-8"))])
|
|
||||||
case("typo-no-css-emphasis", f14, ["--no-css-emphasis"], 0, [
|
|
||||||
("keine CSS-Hervorhebung, Verstecktes bleibt", lambda o, c: "Ein kursives und fettes Wort" in body(o) and "VERSTECKT" in body(o))])
|
|
||||||
case("typo-keep-invisible-chars", f14, ["--keep-invisible-chars"], 0, [
|
|
||||||
("Zeichen erhalten", lambda o, c: "Null\u200bbreite" in body(o) and "wei\u00adches" in body(o)),
|
|
||||||
("Bericht 0", lambda o, c: report(o)["text_cleanup"]["invisible_chars_removed"] == 0)])
|
|
||||||
case("typo-styles-html", f14, ["--styles", "html"], 0, [
|
|
||||||
("Farbe/Größe/Kapitälchen/Hintergrund", lambda o, c: all(x in body(o) for x in (
|
|
||||||
'<span style="color: #cc0000">roter</span>', '<span style="font-size: 1.6em">großer</span>',
|
|
||||||
'<span style="font-variant: small-caps">Kapitälchen</span>',
|
|
||||||
'<span style="background-color: #ffff00">markierter</span>'))),
|
|
||||||
("Sprache", lambda o, c: '<span lang="en">an English phrase</span>' in body(o)),
|
|
||||||
("Ausrichtung", lambda o, c: '<div style="text-align: center">\n\nZentrierte Widmung\n\n</div>' in body(o)),
|
|
||||||
("Grundschrift nicht wiederholt", lambda o, c: "Georgia" not in body(o))])
|
|
||||||
case("typo-page-markers-visible", f14, ["--page-markers", "visible"], 0, [
|
|
||||||
("Marker", lambda o, c: "Seite elf endet\\[S. 12\\] und Seite zwölf geht weiter\\[S. 13\\] bis dreizehn." in body(o)),
|
|
||||||
("Bericht", lambda o, c: report(o)["layout"]["page_markers"] == 2)])
|
|
||||||
case("typo-page-markers-comment", f14, ["--page-markers", "comment"], 0, [
|
|
||||||
("Kommentar", lambda o, c: "endet<!-- S. 12 --> und" in body(o) and "weiter<!-- S. 13 --> bis" in body(o))])
|
|
||||||
case("typo-boxes-quote", f14, ["--boxes", "quote"], 0, [
|
|
||||||
("Zitatblock ohne [!", lambda o, c: "> **Wichtige Regel**\n>\n> Immer testen." in boxes(o) and "[!" not in boxes(o))])
|
|
||||||
case("typo-boxes-html", f14, ["--boxes", "html"], 0, [
|
|
||||||
("div mit Typklasse", lambda o, c: '<div class="box box-note">' in boxes(o) and '<div class="box box-tip">' in boxes(o))])
|
|
||||||
case("typo-boxes-plain", f14, ["--boxes", "plain"], 0, [
|
|
||||||
("normaler Text", lambda o, c: "Immer testen." in boxes(o) and "> " not in boxes(o) and "<div" not in boxes(o)),
|
|
||||||
("Bericht 0 Kästen, kein Filter", lambda o, c: report(o)["layout"]["boxes"] == 0
|
|
||||||
and "callouts.lua" not in (o / "README.md").read_text(encoding="utf-8"))])
|
|
||||||
case("invalid-boxes", f14, ["--boxes", "bunt"], 1, [("Meldung", lambda o, c: "invalid choice" in c.stderr)], validate_out=False)
|
|
||||||
case("invalid-styles", f14, ["--styles", "css"], 1, validate_out=False)
|
|
||||||
case("invalid-page-markers", f14, ["--page-markers", "ja"], 1, validate_out=False)
|
|
||||||
|
|
||||||
# --- epub2md_merge.py on a converted book (+ Pandoc filter if Pandoc is installed)
|
|
||||||
merge_case("merge-typo", OUT / "typo-default", 0, [
|
|
||||||
("Gesamtdatei mit Original-Namen", lambda o, c: (o / "14-typography.md").exists()),
|
|
||||||
("Inhalt beider Kapitel", lambda o, c: all(x in (o / "14-typography.md").read_text(encoding="utf-8")
|
|
||||||
for x in ("*kursives*", "> [!NOTE]", "Zwischendurch speichern.")))])
|
|
||||||
merge_case("merge-again-no-force", OUT / "typo-default", 1, [("Meldung", lambda o, c: c.stderr.strip() != "")])
|
|
||||||
merge_case("merge-missing-dir", OUT / "gibt-es-nicht", 1)
|
|
||||||
if shutil.which("pandoc"):
|
|
||||||
src = OUT / "typo-default" / "14-typography.md"
|
|
||||||
cp = subprocess.run(["pandoc", str(src), "-s", "--lua-filter", str(ROOT / "pandoc" / "callouts.lua"), "-o", "-",
|
|
||||||
"-t", "html"], capture_output=True, text=True, timeout=120)
|
|
||||||
good = cp.returncode == 0 and 'class="callout callout-note"' in cp.stdout and "[!NOTE]" not in cp.stdout \
|
|
||||||
and "<strong>Tipp</strong>" in cp.stdout
|
|
||||||
results.append({"case": "pandoc-callouts-filter", "ok": good, "rc": cp.returncode,
|
|
||||||
"notes": ["Kästen als div.callout, keine [!…]-Marken" if good else cp.stderr[-300:]]})
|
|
||||||
print(("PASS " if good else "FAIL ") + "pandoc-callouts-filter")
|
|
||||||
cp = subprocess.run(["pandoc", str(src), "-s", "--lua-filter", str(ROOT / "pandoc" / "book.lua"), "-t", "html"],
|
|
||||||
capture_output=True, text=True, timeout=120)
|
|
||||||
h = cp.stdout
|
|
||||||
pos = [h.find(x) for x in ('id="title-block-header"', 'id="TOC"', 'class="chapter-start"')]
|
|
||||||
good = cp.returncode == 0 and -1 not in pos and pos == sorted(pos) and h.count('id="title-block-header"') == 1 \
|
|
||||||
and "epub2md-toc" not in h and 'href="#' in h[pos[1]:pos[2]]
|
|
||||||
results.append({"case": "pandoc-book-filter", "ok": good, "rc": cp.returncode,
|
|
||||||
"notes": ["Titelkopf → Inhaltsverzeichnis → Kapitel, Platzhalter ersetzt" if good else f"Reihenfolge {pos}"]})
|
|
||||||
print(("PASS " if good else "FAIL ") + "pandoc-book-filter")
|
|
||||||
step1 = OUT / "typo-step1.html"
|
|
||||||
subprocess.run(["pandoc", str(src), "-s", "--lua-filter", str(ROOT / "pandoc" / "book.lua"), "-o", str(step1)],
|
|
||||||
capture_output=True, text=True, timeout=120)
|
|
||||||
cp = subprocess.run(["pandoc", str(step1), "-s", "--lua-filter", str(ROOT / "pandoc" / "book.lua"), "-t", "latex"],
|
|
||||||
capture_output=True, text=True, timeout=120, cwd=OUT)
|
|
||||||
t = cp.stdout
|
|
||||||
good = cp.returncode == 0 and t.count("\\tableofcontents") == 1 and "\\maketitle" not in t \
|
|
||||||
and t.index("Typografie}") < t.index("\\tableofcontents") and "a4paper" in t
|
|
||||||
results.append({"case": "pandoc-book-filter-latex", "ok": good, "rc": cp.returncode,
|
|
||||||
"notes": ["LaTeX: Titel → \\tableofcontents, kein \\maketitle, A4" if good else "LaTeX-Aufbau falsch"]})
|
|
||||||
print(("PASS " if good else "FAIL ") + "pandoc-book-filter-latex")
|
|
||||||
hdr = OUT / "header.html"
|
|
||||||
hdr.write_text("<style>body { color: black; }</style>\n", encoding="utf-8")
|
|
||||||
cp = subprocess.run(["pandoc", str(src), "-s", "-H", str(hdr), "--lua-filter", str(ROOT / "pandoc" / "callouts.lua"),
|
|
||||||
"-t", "html"], capture_output=True, text=True, timeout=120)
|
|
||||||
good = cp.returncode == 0 and ".callout-tip {" in cp.stdout
|
|
||||||
results.append({"case": "pandoc-callouts-filter-with-H", "ok": good, "rc": cp.returncode,
|
|
||||||
"notes": ["Kasten-CSS trotz -H vorhanden" if good else "Kasten-CSS fehlt bei -H"]})
|
|
||||||
print(("PASS " if good else "FAIL ") + "pandoc-callouts-filter-with-H")
|
|
||||||
epub = OUT / "typo.epub"
|
|
||||||
cp = subprocess.run(["pandoc", str(src), "--lua-filter", str(ROOT / "pandoc" / "callouts.lua"), "-o", str(epub)],
|
|
||||||
capture_output=True, text=True, timeout=120)
|
|
||||||
import zipfile
|
|
||||||
with zipfile.ZipFile(epub) as z:
|
|
||||||
css = "".join(z.read(n).decode() for n in z.namelist() if n.endswith(".css"))
|
|
||||||
chapters = [z.read(n).decode() for n in z.namelist() if "/ch0" in n]
|
|
||||||
good = cp.returncode == 0 and ".callout-tip {" in css and "font-family" in css \
|
|
||||||
and all("stylesheet" in c for c in chapters) and any('class="callout' in c for c in chapters)
|
|
||||||
results.append({"case": "pandoc-callouts-filter-epub", "ok": good, "rc": cp.returncode,
|
|
||||||
"notes": ["EPUB: Kasten-CSS + Standard-CSS, in allen Kapiteln verlinkt" if good else "EPUB-CSS fehlt"]})
|
|
||||||
print(("PASS " if good else "FAIL ") + "pandoc-callouts-filter-epub")
|
|
||||||
|
|
||||||
# --- determinism
|
|
||||||
a = case("determinism-a", "06-mathml-and-equations.epub", [], 0, out=OUT / "det-a")
|
|
||||||
b = case("determinism-b", "06-mathml-and-equations.epub", [], 0, out=OUT / "det-b")
|
|
||||||
diff = [p.name for p in (OUT / "det-a/chapters").glob("*.md") if p.read_bytes() != (OUT / "det-b/chapters" / p.name).read_bytes()]
|
|
||||||
results.append({"case": "determinism", "ok": not diff and a["ok"] and b["ok"], "notes": [f"abweichende Dateien: {diff}"] if diff else ["Kapiteldateien identisch"]})
|
|
||||||
|
|
||||||
(QA / "output" / "matrix-results.json").write_text(json.dumps(results, ensure_ascii=False, indent=2), encoding="utf-8")
|
|
||||||
failed = [r["case"] for r in results if not r["ok"]]
|
|
||||||
print(f"\n{len(results)} Fälle, {len(results) - len(failed)} ok, {len(failed)} fehlgeschlagen: {failed}")
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
main()
|
|
||||||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Reference in a new issue