fix: restliche Befunde der unabhängigen QA (QA2-09 bis QA2-19)
- QA2-09: Formeln in HTML-Tabellen als MathML/Formelbild statt $…$; Inline-Noten dort als HTML-Verweis (Markdown wird in HTML-Blöcken nicht gerendert) - QA2-10: PDF-Weg Markdown → HTML → PDF in README, Bedienungsanleitung und Buch-README. Beim direkten Weg Markdown → LaTeX verwirft Pandoc HTML-Tabellen und <a id>-Ziele. Geprüft: C-Buch 405 Seiten, 34 Tabellen, 728/755 Links. - QA2-11: Hinweis code-language-hint bei einsprachigen Büchern - QA2-12: --strict bricht bei nicht dekodierbarem XHTML ab (Exit 3); Hilfetext präzisiert - QA2-13: verdächtig große Medien werden übersprungen (Fehler media-skipped, Platzhalter) statt die ganze Konvertierung abzubrechen - QA2-14: Formelklasse am <img> selbst wird erkannt - QA2-15: wiederholte Diagnosen werden gezählt (count, ×n); MathML in HTML-Tabellen zählt im Formelbericht mit - QA2-16: Unicode-Buchstaben in Ankernamen bleiben erhalten - QA2-17: keine leeren Links (#) auf den eigenen Kapitelanfang; keine namespaced Attribute (epub:type → type=) in HTML-Tabellen - QA2-18: nicht definierte Entitäten werden gemeldet (undefined-entity) - QA2-19: Bild-Platzhalter ohne Kursivschrift (benachbarte ergaben **) Regressionstests: tests/test_qa2_open.py (11 Tests, vorher alle rot). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
daa99415b5
commit
109612db28
12 changed files with 247 additions and 49 deletions
|
|
@ -161,7 +161,9 @@ kannst du sie für alle übrigen Codeblöcke vorgeben:
|
||||||
python3 epub2md.py python-buch.epub output/python-buch --code-language python
|
python3 epub2md.py python-buch.epub output/python-buch --code-language python
|
||||||
```
|
```
|
||||||
|
|
||||||
Das hilft Editoren beim Einfärben des Codes. Nachteil: Auch Programmausgaben und Shell-Befehle im Buch
|
Hat das Programm viele Codeblöcke ohne Sprache gefunden, aber alle erkannten in derselben Sprache,
|
||||||
|
schlägt es diese Option im Bericht von sich aus vor (Hinweis `code-language-hint`). Das hilft Editoren
|
||||||
|
beim Einfärben des Codes. Nachteil: Auch Programmausgaben und Shell-Befehle im Buch
|
||||||
werden dann als Python markiert.
|
werden dann als Python markiert.
|
||||||
|
|
||||||
Zu lange Codezeilen sind im gedruckten Buch oft umbrochen und die Fortsetzung mit `↪` markiert. Das
|
Zu lange Codezeilen sind im gedruckten Buch oft umbrochen und die Fortsetzung mit `↪` markiert. Das
|
||||||
|
|
@ -231,11 +233,16 @@ python3 epub2md_merge.py output/meinbuch
|
||||||
cd output/meinbuch
|
cd output/meinbuch
|
||||||
pandoc meinbuch.md --toc -s -o meinbuch.html # Webseite
|
pandoc meinbuch.md --toc -s -o meinbuch.html # Webseite
|
||||||
pandoc meinbuch.md --toc -o meinbuch.docx # Word
|
pandoc meinbuch.md --toc -o meinbuch.docx # Word
|
||||||
pandoc meinbuch.md --pdf-engine=xelatex -o meinbuch.pdf # PDF (LaTeX nötig)
|
pandoc meinbuch.md -s -o meinbuch.html && \
|
||||||
|
pandoc meinbuch.html --toc --pdf-engine=xelatex -o meinbuch.pdf # PDF (LaTeX nötig)
|
||||||
```
|
```
|
||||||
|
|
||||||
Bei Büchern mit Formeln für HTML zusätzlich `--mathml` angeben, sonst bleiben Matrizen als TeX-Text stehen.
|
Bei Büchern mit Formeln für HTML zusätzlich `--mathml` angeben, sonst bleiben Matrizen als TeX-Text stehen.
|
||||||
|
|
||||||
|
Für ein PDF immer den Umweg über HTML nehmen, wie oben gezeigt. Beim direkten Weg von Markdown nach PDF
|
||||||
|
lässt Pandoc komplexe Tabellen weg, und Querverweise im PDF funktionieren nicht. Die `README.md` im
|
||||||
|
Ausgabeordner enthält die passenden Befehle für dein Buch.
|
||||||
|
|
||||||
### Für ein lokales Sprachmodell (RAG)
|
### Für ein lokales Sprachmodell (RAG)
|
||||||
|
|
||||||
- Die einzelnen Kapiteldateien eignen sich als Abschnitte. Die Metadaten im Datenkopf (Buch, Kapitel,
|
- Die einzelnen Kapiteldateien eignen sich als Abschnitte. Die Metadaten im Datenkopf (Buch, Kapitel,
|
||||||
|
|
@ -285,9 +292,13 @@ stehen gesammelt in `diagnostics/warnings.md`.
|
||||||
| `noteref-unresolved` | Fußnotenzeichen ohne zugehörige Fußnote | Fehler der Vorlage |
|
| `noteref-unresolved` | Fußnotenzeichen ohne zugehörige Fußnote | Fehler der Vorlage |
|
||||||
| `note-unreferenced` | Fußnote, auf die nichts verweist | Bleibt an ihrer Stelle im Text |
|
| `note-unreferenced` | Fußnote, auf die nichts verweist | Bleibt an ihrer Stelle im Text |
|
||||||
| `decode-error` | Zeichen im E-Book waren beschädigt | Betroffene Zeichen erscheinen als `<60>` |
|
| `decode-error` | Zeichen im E-Book waren beschädigt | Betroffene Zeichen erscheinen als `<60>` |
|
||||||
|
| `undefined-entity` | Das E-Book verwendet Sonderzeichen-Namen (`&name;`), die nirgends definiert sind | Diese Zeichen fehlen in der Ausgabe; Fehler der Vorlage |
|
||||||
|
| `media-skipped` (Fehler) | Ein Bild/Medium war verdächtig groß oder stark komprimiert und wurde aus Sicherheitsgründen nicht übernommen | An seiner Stelle steht `[Bild nicht übernommen: …]`; mit `--strict` bricht das Programm ab |
|
||||||
| `no-toc` | E-Book ohne Inhaltsverzeichnis | Kapiteltitel werden aus dem Inhalt abgeleitet |
|
| `no-toc` | E-Book ohne Inhaltsverzeichnis | Kapiteltitel werden aus dem Inhalt abgeleitet |
|
||||||
| `broken-output-link` | Interne Prüfung fand einen kaputten Link in der Ausgabe | Bitte melden, das sollte nicht vorkommen |
|
| `broken-output-link` | Interne Prüfung fand einen kaputten Link in der Ausgabe | Bitte melden, das sollte nicht vorkommen |
|
||||||
|
|
||||||
|
Steht hinter einer Meldung `(×5)`, ist dieselbe Meldung an derselben Stelle mehrfach aufgetreten.
|
||||||
|
|
||||||
Fehler, nach denen trotzdem weitergearbeitet wurde, etwa eine fehlende Kapiteldatei im E-Book, stehen
|
Fehler, nach denen trotzdem weitergearbeitet wurde, etwa eine fehlende Kapiteldatei im E-Book, stehen
|
||||||
ebenfalls dort. Mit `--strict` bricht das Programm bei solchen kritischen Fehlern stattdessen ab, ohne
|
ebenfalls dort. Mit `--strict` bricht das Programm bei solchen kritischen Fehlern stattdessen ab, ohne
|
||||||
eine Ausgabe zu schreiben.
|
eine Ausgabe zu schreiben.
|
||||||
|
|
|
||||||
14
README.md
14
README.md
|
|
@ -222,7 +222,9 @@ weil CommonMark keine `[^…]`-Syntax kennt.
|
||||||
- **Layouttabellen / komplexe Tabellen** (`colspan`, `rowspan`, Blockinhalte in Zellen, verschachtelte
|
- **Layouttabellen / komplexe Tabellen** (`colspan`, `rowspan`, Blockinhalte in Zellen, verschachtelte
|
||||||
Tabellen, senkrechte Striche `|` in Code, Formeln oder Link-Zielen – GFM würde die Zeile dort in
|
Tabellen, senkrechte Striche `|` in Code, Formeln oder Link-Zielen – GFM würde die Zeile dort in
|
||||||
zusätzliche Zellen zerlegen) bleiben als bereinigtes HTML im Markdown erhalten, mit Warnung. Einfache Tabellen werden
|
zusätzliche Zellen zerlegen) bleiben als bereinigtes HTML im Markdown erhalten, mit Warnung. Einfache Tabellen werden
|
||||||
GFM-Pipe-Tabellen. Einzellige Layouttabellen werden als normaler Text ausgegeben.
|
GFM-Pipe-Tabellen. Einzellige Layouttabellen werden als normaler Text ausgegeben. Innerhalb solcher
|
||||||
|
HTML-Tabellen rendern Markdown-Programme kein Markdown. Formeln stehen dort deshalb als MathML oder als
|
||||||
|
Formelbild, Hervorhebungen als HTML-Tags, Fußnotenverweise als HTML-Links (auch bei `--footnotes inline`).
|
||||||
- **Syntax-Highlighting:** Farb-Spans, `<strong>`/`<em>` in Listings und Zeilennummern werden entfernt.
|
- **Syntax-Highlighting:** Farb-Spans, `<strong>`/`<em>` in Listings und Zeilennummern werden entfernt.
|
||||||
Einrückung, Tabs, `<`, `>`, `&`, `$`, `|`, Backslashes und Backticks bleiben exakt erhalten. Die Sprache
|
Einrückung, Tabs, `<`, `>`, `&`, `$`, `|`, Backslashes und Backticks bleiben exakt erhalten. Die Sprache
|
||||||
kommt aus Klassen/Attributen (`language-python`, `data-lang`, …) oder starken Inhaltssignalen
|
kommt aus Klassen/Attributen (`language-python`, `data-lang`, …) oder starken Inhaltssignalen
|
||||||
|
|
@ -293,9 +295,17 @@ pandoc programmierbuch.md --toc -o programmierbuch.epub
|
||||||
|
|
||||||
cd output/mathematikbuch
|
cd output/mathematikbuch
|
||||||
pandoc mathematikbuch.md --mathml -s -o mathematikbuch.html
|
pandoc mathematikbuch.md --mathml -s -o mathematikbuch.html
|
||||||
pandoc mathematikbuch.md --pdf-engine=xelatex -o mathematikbuch.pdf
|
# PDF über den Zwischenschritt HTML (siehe unten)
|
||||||
|
pandoc mathematikbuch.html --toc --pdf-engine=xelatex -o mathematikbuch.pdf
|
||||||
```
|
```
|
||||||
|
|
||||||
|
**PDF immer über HTML erzeugen.** Beim direkten Weg Markdown → LaTeX verwirft Pandoc allen rohen HTML-Code.
|
||||||
|
Dabei gehen HTML-Tabellen samt Inhalt verloren, und `<a id>`-Sprungziele fehlen, sodass interne Links im PDF
|
||||||
|
ins Leere zeigen. Liest Pandoc dagegen die HTML-Fassung, macht es daraus echte Tabellen und Sprungziele.
|
||||||
|
Getestet mit dem C-Buch (405 Seiten, 34 Tabellen, 728 von 755 internen Links funktionieren; die übrigen
|
||||||
|
stehen in Codeblöcken innerhalb von Tabellen). Die `README.md` jedes konvertierten Buches enthält den
|
||||||
|
passenden Befehl.
|
||||||
|
|
||||||
Alternativ direkt aus den Kapiteldateien:
|
Alternativ direkt aus den Kapiteldateien:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
|
|
||||||
|
|
@ -86,7 +86,8 @@ def build_parser() -> argparse.ArgumentParser:
|
||||||
help="Druck-Umbruchzeichen (↪/➥) in Codeblöcken nicht entfernen; Standard: fortgesetzte "
|
help="Druck-Umbruchzeichen (↪/➥) in Codeblöcken nicht entfernen; Standard: fortgesetzte "
|
||||||
"Zeilen werden wieder zur ursprünglichen Codezeile zusammengefügt")
|
"Zeilen werden wieder zur ursprünglichen Codezeile zusammengefügt")
|
||||||
p.add_argument("--strict", action="store_true",
|
p.add_argument("--strict", action="store_true",
|
||||||
help="Bei kritischen Fehlern (fehlende Spine-Dateien, unsichere Pfade, kaputtes XHTML) abbrechen")
|
help="Bei kritischen Fehlern abbrechen: fehlende/unlesbare Spine-Dokumente, nicht dekodierbares "
|
||||||
|
"XHTML, unsichere ZIP-Pfade, übersprungene (verdächtige) Medien")
|
||||||
p.add_argument("--force", action="store_true",
|
p.add_argument("--force", action="store_true",
|
||||||
help="Bestehendes Ausgabeverzeichnis ersetzen – nur wenn es eine frühere epub2md-Ausgabe ist")
|
help="Bestehendes Ausgabeverzeichnis ersetzen – nur wenn es eine frühere epub2md-Ausgabe ist")
|
||||||
p.add_argument("--verbose", "-v", action="store_true", help="Verarbeitungsschritte und Warnungen ausgeben")
|
p.add_argument("--verbose", "-v", action="store_true", help="Verarbeitungsschritte und Warnungen ausgeben")
|
||||||
|
|
|
||||||
|
|
@ -17,14 +17,16 @@ class Diagnostics:
|
||||||
self.unsupported: Counter[str] = Counter()
|
self.unsupported: Counter[str] = Counter()
|
||||||
self.unsupported_examples: dict[str, str] = {}
|
self.unsupported_examples: dict[str, str] = {}
|
||||||
self.unresolved_links: list[dict[str, str]] = []
|
self.unresolved_links: list[dict[str, str]] = []
|
||||||
self._seen: set[tuple[str, str, str]] = set()
|
self._seen: dict[tuple[str, str, str], Diagnostic] = {}
|
||||||
|
|
||||||
def add(self, level: str, code: str, message: str, source: str = "", chapter: str = "") -> None:
|
def add(self, level: str, code: str, message: str, source: str = "", chapter: str = "") -> None:
|
||||||
key = (code, message, source)
|
key = (code, message, source)
|
||||||
if key in self._seen:
|
if key in self._seen:
|
||||||
|
self._seen[key].count += 1
|
||||||
return
|
return
|
||||||
self._seen.add(key)
|
item = Diagnostic(level, code, message, source, chapter)
|
||||||
self.items.append(Diagnostic(level, code, message, source, chapter))
|
self._seen[key] = item
|
||||||
|
self.items.append(item)
|
||||||
if self.verbose:
|
if self.verbose:
|
||||||
print(f"[{level}] {message}" + (f" ({source})" if source else ""), file=sys.stderr)
|
print(f"[{level}] {message}" + (f" ({source})" if source else ""), file=sys.stderr)
|
||||||
|
|
||||||
|
|
@ -54,4 +56,5 @@ class Diagnostics:
|
||||||
print(f"… {message}", file=sys.stderr)
|
print(f"… {message}", file=sys.stderr)
|
||||||
|
|
||||||
def count(self, level: str) -> int:
|
def count(self, level: str) -> int:
|
||||||
return sum(1 for d in self.items if d.level == level)
|
"""Number of occurrences (repeated identical diagnostics included)."""
|
||||||
|
return sum(d.count for d in self.items if d.level == level)
|
||||||
|
|
|
||||||
|
|
@ -43,13 +43,15 @@ def _secure_xml_parser(recover: bool = True) -> etree.XMLParser:
|
||||||
_KEEP_ENTITIES = {"amp", "lt", "gt", "quot", "apos"}
|
_KEEP_ENTITIES = {"amp", "lt", "gt", "quot", "apos"}
|
||||||
|
|
||||||
|
|
||||||
def _replace_named_entities(text: str) -> str:
|
def _replace_named_entities(text: str, unknown: list[str] | None = None) -> str:
|
||||||
def repl(m: re.Match[str]) -> str:
|
def repl(m: re.Match[str]) -> str:
|
||||||
name = m.group(1)
|
name = m.group(1)
|
||||||
if name in _KEEP_ENTITIES:
|
if name in _KEEP_ENTITIES:
|
||||||
return m.group(0)
|
return m.group(0)
|
||||||
cp = html.entities.name2codepoint.get(name)
|
cp = html.entities.name2codepoint.get(name)
|
||||||
if cp is None:
|
if cp is None:
|
||||||
|
if unknown is not None and name not in unknown:
|
||||||
|
unknown.append(name)
|
||||||
return m.group(0)
|
return m.group(0)
|
||||||
return f"&#{cp};"
|
return f"&#{cp};"
|
||||||
|
|
||||||
|
|
@ -75,11 +77,10 @@ def decode_markup(data: bytes, source: str, diag: Diagnostics) -> str:
|
||||||
except (UnicodeDecodeError, LookupError):
|
except (UnicodeDecodeError, LookupError):
|
||||||
text = data.decode("utf-8", errors="replace")
|
text = data.decode("utf-8", errors="replace")
|
||||||
n = text.count("<EFBFBD>")
|
n = text.count("<EFBFBD>")
|
||||||
diag.warn(
|
message = f"Ungültige Zeichenkodierung ({encoding}); {n} Zeichen durch U+FFFD ersetzt."
|
||||||
"decode-error",
|
if diag.strict:
|
||||||
f"Ungültige Zeichenkodierung ({encoding}); {n} Zeichen durch U+FFFD ersetzt.",
|
diag.error("decode-error", message, source, critical=True)
|
||||||
source,
|
diag.warn("decode-error", message, source)
|
||||||
)
|
|
||||||
return text
|
return text
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -87,7 +88,12 @@ def parse_xml(data: bytes, source: str, diag: Diagnostics, html_fallback: bool =
|
||||||
"""Parse XML securely; optionally fall back to the HTML parser for tag soup."""
|
"""Parse XML securely; optionally fall back to the HTML parser for tag soup."""
|
||||||
text = decode_markup(data, source, diag)
|
text = decode_markup(data, source, diag)
|
||||||
text = re.sub(r"^\s*<\?xml[^>]*\?>", "", text)
|
text = re.sub(r"^\s*<\?xml[^>]*\?>", "", text)
|
||||||
text = _replace_named_entities(text)
|
unknown: list[str] = []
|
||||||
|
text = _replace_named_entities(text, unknown)
|
||||||
|
if unknown:
|
||||||
|
# never resolved (XXE protection) and therefore dropped by the parser – say so instead of losing them silently
|
||||||
|
diag.warn("undefined-entity", "Nicht definierte Entitäten entfernt: " + ", ".join(f"&{n};" for n in unknown[:10])
|
||||||
|
+ (" …" if len(unknown) > 10 else ""), source)
|
||||||
root = None
|
root = None
|
||||||
try:
|
try:
|
||||||
root = etree.fromstring(text, _secure_xml_parser())
|
root = etree.fromstring(text, _secure_xml_parser())
|
||||||
|
|
@ -165,14 +171,24 @@ class EpubReader:
|
||||||
def exists(self, path: str) -> bool:
|
def exists(self, path: str) -> bool:
|
||||||
return path in self.names
|
return path in self.names
|
||||||
|
|
||||||
|
def entry_problem(self, path: str) -> str | None:
|
||||||
|
"""Reason why an entry must not be read (too large / suspicious compression ratio), else None."""
|
||||||
|
info = self.names.get(path)
|
||||||
|
if info is None:
|
||||||
|
return None
|
||||||
|
if info.file_size > MAX_MEMBER_SIZE or (
|
||||||
|
info.compress_size and info.file_size > 50 * 1024 * 1024 and info.file_size / info.compress_size > MAX_RATIO
|
||||||
|
):
|
||||||
|
return "ZIP-Eintrag zu groß oder verdächtig stark komprimiert"
|
||||||
|
return None
|
||||||
|
|
||||||
def read(self, path: str) -> bytes:
|
def read(self, path: str) -> bytes:
|
||||||
info = self.names.get(path)
|
info = self.names.get(path)
|
||||||
if info is None:
|
if info is None:
|
||||||
raise KeyError(path)
|
raise KeyError(path)
|
||||||
if info.file_size > MAX_MEMBER_SIZE or (
|
problem = self.entry_problem(path)
|
||||||
info.compress_size and info.file_size > 50 * 1024 * 1024 and info.file_size / info.compress_size > MAX_RATIO
|
if problem:
|
||||||
):
|
raise ContainerError(f"{problem}: {path}")
|
||||||
raise ContainerError(f"ZIP-Eintrag zu groß oder verdächtig stark komprimiert: {path}")
|
|
||||||
try:
|
try:
|
||||||
with self.zip.open(info) as fh:
|
with self.zip.open(info) as fh:
|
||||||
data = fh.read(MAX_MEMBER_SIZE + 1)
|
data = fh.read(MAX_MEMBER_SIZE + 1)
|
||||||
|
|
|
||||||
|
|
@ -9,7 +9,7 @@ from __future__ import annotations
|
||||||
|
|
||||||
import html
|
import html
|
||||||
import re
|
import re
|
||||||
from collections.abc import Sequence
|
from collections.abc import Callable, Sequence
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from typing import Any
|
from typing import Any
|
||||||
from urllib.parse import unquote
|
from urllib.parse import unquote
|
||||||
|
|
@ -28,6 +28,7 @@ from .codeblocks import (
|
||||||
is_inline_code,
|
is_inline_code,
|
||||||
)
|
)
|
||||||
from .diagnostics import Diagnostics
|
from .diagnostics import Diagnostics
|
||||||
|
from .errors import ContainerError
|
||||||
from .footnotes import NoteIndex
|
from .footnotes import NoteIndex
|
||||||
from .mathml import Unsupported, is_display_tex, looks_like_tex, mathml_to_tex, strip_tex_delimiters
|
from .mathml import Unsupported, is_display_tex, looks_like_tex, mathml_to_tex, strip_tex_delimiters
|
||||||
from .media import MediaRegistry
|
from .media import MediaRegistry
|
||||||
|
|
@ -227,7 +228,9 @@ def serialize_mathml(math: Any) -> str:
|
||||||
|
|
||||||
class Renderer:
|
class Renderer:
|
||||||
def __init__(self, options: RenderOptions, resolver: LinkResolver, media: MediaRegistry,
|
def __init__(self, options: RenderOptions, resolver: LinkResolver, media: MediaRegistry,
|
||||||
notes: NoteIndex, diag: Diagnostics, zip_names: set[str]) -> None:
|
notes: NoteIndex, diag: Diagnostics, zip_names: set[str],
|
||||||
|
entry_problem: Callable[[str], str | None] | None = None) -> None:
|
||||||
|
self.entry_problem = entry_problem or (lambda path: None)
|
||||||
self.opt = options
|
self.opt = options
|
||||||
self.zip_names = zip_names
|
self.zip_names = zip_names
|
||||||
self.in_table_html = False
|
self.in_table_html = False
|
||||||
|
|
@ -902,7 +905,7 @@ class Renderer:
|
||||||
num = int(note.label.split("-")[1])
|
num = int(note.label.split("-")[1])
|
||||||
if self.ch:
|
if self.ch:
|
||||||
self.ch.stats.footnotes += 1
|
self.ch.stats.footnotes += 1
|
||||||
if mode == "inline" and not self.in_note:
|
if mode == "inline" and not self.in_note and not self.in_table_html:
|
||||||
text = self._note_inline_text(note)
|
text = self._note_inline_text(note)
|
||||||
if text is not None:
|
if text is not None:
|
||||||
label = "Anmerkung" if self.opt.language.startswith("de") else "Note"
|
label = "Anmerkung" if self.opt.language.startswith("de") else "Note"
|
||||||
|
|
@ -1015,6 +1018,11 @@ class Renderer:
|
||||||
if not self._exists(path):
|
if not self._exists(path):
|
||||||
self.diag.warn("image-missing", f"Bild {src} nicht im EPUB gefunden.", self._source(el), self.file)
|
self.diag.warn("image-missing", f"Bild {src} nicht im EPUB gefunden.", self._source(el), self.file)
|
||||||
return [("t", md_escape(f"[Bild fehlt: {alt or src}]"))]
|
return [("t", md_escape(f"[Bild fehlt: {alt or src}]"))]
|
||||||
|
problem = self.entry_problem(path)
|
||||||
|
if problem:
|
||||||
|
self.diag.error("media-skipped", f"Medium {path} nicht übernommen: {problem}", self._source(el), self.file,
|
||||||
|
critical=True, exc=ContainerError)
|
||||||
|
return [("t", md_escape(f"[Bild nicht übernommen: {alt or path.rsplit('/', 1)[-1]}]"))]
|
||||||
if self._is_formula_image(el, alt, path):
|
if self._is_formula_image(el, alt, path):
|
||||||
return self.formula_image(el, alt, path)
|
return self.formula_image(el, alt, path)
|
||||||
if not alt and self.figure_caption:
|
if not alt and self.figure_caption:
|
||||||
|
|
@ -1031,7 +1039,7 @@ class Renderer:
|
||||||
else:
|
else:
|
||||||
self.ch.stats.images += 1
|
self.ch.stats.images += 1
|
||||||
if not self.opt.extract_media:
|
if not self.opt.extract_media:
|
||||||
return [("r", f"*\\[Bild: {self._alt(alt)} – {md_escape(path)}\\]*")]
|
return [("r", f"\\[Bild: {self._alt(alt)} – {md_escape(path)}\\] ")]
|
||||||
mf = self.media.register(path, role=role, hint=alt[:40] if self.media.strategy == "relative" and role != "formula" else "")
|
mf = self.media.register(path, role=role, hint=alt[:40] if self.media.strategy == "relative" and role != "formula" else "")
|
||||||
if self.in_table_html:
|
if self.in_table_html:
|
||||||
return [("r", f'<img src="{html.escape(url_path(rel_link(self.file, mf.output)))}" alt="{html.escape(alt)}"/>')]
|
return [("r", f'<img src="{html.escape(url_path(rel_link(self.file, mf.output)))}" alt="{html.escape(alt)}"/>')]
|
||||||
|
|
@ -1044,13 +1052,13 @@ class Renderer:
|
||||||
def _is_formula_image(self, el: Any, alt: str, path: str) -> bool:
|
def _is_formula_image(self, el: Any, alt: str, path: str) -> bool:
|
||||||
if looks_like_tex(alt):
|
if looks_like_tex(alt):
|
||||||
return True
|
return True
|
||||||
for anc in _ancestors(el, 4):
|
for anc in [el] + _ancestors(el, 4):
|
||||||
if any(FORMULA_CLASS.search(c) for c in classes(anc)) or {"equation"} & set(epub_types(anc)):
|
if any(FORMULA_CLASS.search(c) for c in classes(anc)) or {"equation"} & set(epub_types(anc)):
|
||||||
return True
|
return True
|
||||||
return False
|
return False
|
||||||
|
|
||||||
def _formula_display(self, el: Any, alt: str) -> bool:
|
def _formula_display(self, el: Any, alt: str) -> bool:
|
||||||
for anc in _ancestors(el, 5):
|
for anc in [el] + _ancestors(el, 5):
|
||||||
cls = classes(anc)
|
cls = classes(anc)
|
||||||
if any(INLINE_FORMULA_CLASS.search(c) for c in cls):
|
if any(INLINE_FORMULA_CLASS.search(c) for c in cls):
|
||||||
return False
|
return False
|
||||||
|
|
@ -1062,7 +1070,7 @@ class Renderer:
|
||||||
display = self._formula_display(el, alt)
|
display = self._formula_display(el, alt)
|
||||||
tex = strip_tex_delimiters(alt) if looks_like_tex(alt) else ""
|
tex = strip_tex_delimiters(alt) if looks_like_tex(alt) else ""
|
||||||
source = self._source(el)
|
source = self._source(el)
|
||||||
if tex and self.opt.math in ("auto", "tex"):
|
if tex and self.opt.math in ("auto", "tex") and not self.in_table_html:
|
||||||
self.consumed_formula_images.add(path)
|
self.consumed_formula_images.add(path)
|
||||||
self.formulas.append({"source": source, "chapter": self.file, "origin": "image-alt-tex",
|
self.formulas.append({"source": source, "chapter": self.file, "origin": "image-alt-tex",
|
||||||
"representation": "tex", "display": str(display).lower()})
|
"representation": "tex", "display": str(display).lower()})
|
||||||
|
|
@ -1183,24 +1191,19 @@ class Renderer:
|
||||||
def _html(self, el: Any, in_pre: bool, keep: bool = False) -> str:
|
def _html(self, el: Any, in_pre: bool, keep: bool = False) -> str:
|
||||||
name = local(el.tag)
|
name = local(el.tag)
|
||||||
if name == "math":
|
if name == "math":
|
||||||
if self.opt.math in ("auto", "tex"):
|
# Markdown math ($…$, \(…\)) is not rendered inside an HTML block; MathML is.
|
||||||
try:
|
display = el.get("display") == "block"
|
||||||
tex = mathml_to_tex(el)
|
if self.ch:
|
||||||
return html.escape(f"\\({tex}\\)" if self.opt.math == "tex" else f"${tex}$", quote=False)
|
self.ch.stats.math_as_mathml += 1
|
||||||
except Unsupported:
|
if display:
|
||||||
pass
|
self.ch.stats.math_block += 1
|
||||||
|
else:
|
||||||
|
self.ch.stats.math_inline += 1
|
||||||
|
self.formulas.append({"source": self._source(el), "chapter": self.file, "origin": "mathml",
|
||||||
|
"representation": "mathml-in-html-table", "display": str(display).lower()})
|
||||||
return serialize_mathml(el)
|
return serialize_mathml(el)
|
||||||
if name in ("img", "svg", "image"):
|
if name in ("img", "svg", "image"):
|
||||||
if name == "img" and self._is_formula_image(el, el.get("alt") or "", ""):
|
return join_inline(self.inline(el))
|
||||||
alt = el.get("alt") or ""
|
|
||||||
if looks_like_tex(alt) and self.opt.math in ("auto", "tex"):
|
|
||||||
path = resolve_href(self.doc, el.get("src") or "")
|
|
||||||
if path:
|
|
||||||
self.consumed_formula_images.add(path)
|
|
||||||
tex = strip_tex_delimiters(alt)
|
|
||||||
return html.escape(f"\\({tex}\\)" if self.opt.math == "tex" else f"${tex}$", quote=False)
|
|
||||||
pieces = self.inline(el)
|
|
||||||
return join_inline(pieces)
|
|
||||||
if name == "a" and self.notes.extracted and self.notes.note_for_ref(el) is not None:
|
if name == "a" and self.notes.extracted and self.notes.note_for_ref(el) is not None:
|
||||||
return join_inline(self.link(el))
|
return join_inline(self.link(el))
|
||||||
text_fn = (lambda t: html.escape(t, quote=False).replace("\n", " ")) if in_pre else (lambda t: html.escape(WS.sub(" ", t), quote=False))
|
text_fn = (lambda t: html.escape(t, quote=False).replace("\n", " ")) if in_pre else (lambda t: html.escape(WS.sub(" ", t), quote=False))
|
||||||
|
|
@ -1224,6 +1227,8 @@ class Renderer:
|
||||||
return anchor + content
|
return anchor + content
|
||||||
attrs = []
|
attrs = []
|
||||||
for k, v in el.attrib.items():
|
for k, v in el.attrib.items():
|
||||||
|
if k.startswith("{"):
|
||||||
|
continue # namespaced (epub:type, xml:lang …) – not valid HTML
|
||||||
key = k.rsplit("}", 1)[-1]
|
key = k.rsplit("}", 1)[-1]
|
||||||
if key in HTML_KEEP_ATTRS:
|
if key in HTML_KEEP_ATTRS:
|
||||||
attrs.append(f'{key}="{html.escape(v)}"')
|
attrs.append(f'{key}="{html.escape(v)}"')
|
||||||
|
|
|
||||||
|
|
@ -15,6 +15,7 @@ INLINE_CODE = re.compile(r"(?<![\\`])(`+)(?!`)((?:[^`\n]|\n(?![ \t]*\n)|`(?!\1))
|
||||||
MD_LINK = re.compile(r"(?<!\\)!?\[((?:[^\[\]\\\n]|\\.|\[(?:[^\[\]\\\n]|\\.)*\]|\n(?![ \t]*\n))*)\]\((<[^>]*>|[^)\s]+)(?:\s+\"[^\"]*\")?\)")
|
MD_LINK = re.compile(r"(?<!\\)!?\[((?:[^\[\]\\\n]|\\.|\[(?:[^\[\]\\\n]|\\.)*\]|\n(?![ \t]*\n))*)\]\((<[^>]*>|[^)\s]+)(?:\s+\"[^\"]*\")?\)")
|
||||||
HTML_ATTR = re.compile(r"""\b(?:href|src)="([^"]*)\"""")
|
HTML_ATTR = re.compile(r"""\b(?:href|src)="([^"]*)\"""")
|
||||||
ANCHOR = re.compile(r"""\bid="([^"]+)\"""")
|
ANCHOR = re.compile(r"""\bid="([^"]+)\"""")
|
||||||
|
PANDOC_ANCHOR = re.compile(r"\{#([^}\s]+)[^}]*\}") # []{#id} spans and {#id} heading attributes
|
||||||
HEADING = re.compile(r"^#{1,6}\s+(.*?)\s*#*\s*$", re.M)
|
HEADING = re.compile(r"^#{1,6}\s+(.*?)\s*#*\s*$", re.M)
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -33,7 +34,7 @@ def gfm_slug(heading: str) -> str:
|
||||||
|
|
||||||
def anchors_in(text: str) -> set[str]:
|
def anchors_in(text: str) -> set[str]:
|
||||||
body = strip_code(text)
|
body = strip_code(text)
|
||||||
names = set(ANCHOR.findall(body))
|
names = set(ANCHOR.findall(body)) | set(PANDOC_ANCHOR.findall(body))
|
||||||
names.update(gfm_slug(h) for h in HEADING.findall(body))
|
names.update(gfm_slug(h) for h in HEADING.findall(body))
|
||||||
return names
|
return names
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -82,6 +82,7 @@ class Diagnostic:
|
||||||
message: str
|
message: str
|
||||||
source: str = ""
|
source: str = ""
|
||||||
chapter: str = ""
|
chapter: str = ""
|
||||||
|
count: int = 1 # identical diagnostics (code, message, source) are counted, not repeated
|
||||||
|
|
||||||
def as_dict(self) -> dict[str, Any]:
|
def as_dict(self) -> dict[str, Any]:
|
||||||
return {
|
return {
|
||||||
|
|
@ -90,6 +91,7 @@ class Diagnostic:
|
||||||
"message": self.message,
|
"message": self.message,
|
||||||
"source": self.source,
|
"source": self.source,
|
||||||
"chapter": self.chapter,
|
"chapter": self.chapter,
|
||||||
|
"count": self.count,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -236,7 +236,7 @@ def _render(reader: EpubReader, package: Any, chapters: list[Chapter], resolver:
|
||||||
RenderOptions(format=opts.format, footnotes=opts.footnotes, math=opts.math,
|
RenderOptions(format=opts.format, footnotes=opts.footnotes, math=opts.math,
|
||||||
extract_media=opts.extract_media, keep_html=opts.keep_html, language=language,
|
extract_media=opts.extract_media, keep_html=opts.keep_html, language=language,
|
||||||
code_language=opts.code_language, keep_wrap_markers=opts.keep_wrap_markers),
|
code_language=opts.code_language, keep_wrap_markers=opts.keep_wrap_markers),
|
||||||
resolver, media, notes, diag, set(reader.names),
|
resolver, media, notes, diag, set(reader.names), reader.entry_problem,
|
||||||
)
|
)
|
||||||
width = max(3, len(str(len(chapters) + 1)))
|
width = max(3, len(str(len(chapters) + 1)))
|
||||||
notes_title = "Anmerkungen" if language.startswith("de") else "Notes"
|
notes_title = "Anmerkungen" if language.startswith("de") else "Notes"
|
||||||
|
|
@ -248,6 +248,8 @@ def _render(reader: EpubReader, package: Any, chapters: list[Chapter], resolver:
|
||||||
ch.stats.word_count = _word_count(ch.markdown)
|
ch.stats.word_count = _word_count(ch.markdown)
|
||||||
_check_chapter_size(ch, opts.chapter_level, diag)
|
_check_chapter_size(ch, opts.chapter_level, diag)
|
||||||
_report_wrap_markers(renderer, diag)
|
_report_wrap_markers(renderer, diag)
|
||||||
|
if not opts.code_language:
|
||||||
|
_suggest_code_language(renderer, diag)
|
||||||
endnotes: Chapter | None = None
|
endnotes: Chapter | None = None
|
||||||
endnotes_md = ""
|
endnotes_md = ""
|
||||||
if opts.footnotes == "end-of-book" and renderer.book_notes:
|
if opts.footnotes == "end-of-book" and renderer.book_notes:
|
||||||
|
|
@ -256,10 +258,25 @@ def _render(reader: EpubReader, package: Any, chapters: list[Chapter], resolver:
|
||||||
endnotes.filename = renderer.endnotes_file
|
endnotes.filename = renderer.endnotes_file
|
||||||
endnotes.stats.footnotes = len(renderer.book_notes)
|
endnotes.stats.footnotes = len(renderer.book_notes)
|
||||||
endnotes.stats.word_count = _word_count(endnotes_md)
|
endnotes.stats.word_count = _word_count(endnotes_md)
|
||||||
skipped = _collect_remaining_media(reader, package, media, renderer, opts)
|
skipped = _collect_remaining_media(reader, package, media, renderer, opts, diag)
|
||||||
return _Rendered(renderer, media, cover_out, endnotes, endnotes_md, skipped)
|
return _Rendered(renderer, media, cover_out, endnotes, endnotes_md, skipped)
|
||||||
|
|
||||||
|
|
||||||
|
def _suggest_code_language(renderer: Renderer, diag: Diagnostics) -> None:
|
||||||
|
"""Book-level hint: many unlabeled blocks while every recognized block has the same language."""
|
||||||
|
blocks = renderer.code_blocks
|
||||||
|
detected = [b["language"] for b in blocks if b["language"]]
|
||||||
|
unlabeled = len(blocks) - len(detected)
|
||||||
|
if len(blocks) < 10 or len(detected) < 5 or unlabeled < 0.3 * len(blocks):
|
||||||
|
return
|
||||||
|
lang = max(set(detected), key=detected.count)
|
||||||
|
share = detected.count(lang) / len(detected)
|
||||||
|
if share >= 0.9:
|
||||||
|
diag.info("code-language-hint",
|
||||||
|
f"{unlabeled} von {len(blocks)} Codeblöcken ohne erkannte Sprache; {round(share * 100)} % der erkannten "
|
||||||
|
f"Blöcke sind {lang}. Bei einem einsprachigen Buch kennzeichnet --code-language {lang} die übrigen Blöcke.")
|
||||||
|
|
||||||
|
|
||||||
def _report_wrap_markers(renderer: Renderer, diag: Diagnostics) -> None:
|
def _report_wrap_markers(renderer: Renderer, diag: Diagnostics) -> None:
|
||||||
wraps: dict[str, int] = {}
|
wraps: dict[str, int] = {}
|
||||||
for block in renderer.code_blocks:
|
for block in renderer.code_blocks:
|
||||||
|
|
@ -271,7 +288,7 @@ def _report_wrap_markers(renderer: Renderer, diag: Diagnostics) -> None:
|
||||||
|
|
||||||
|
|
||||||
def _collect_remaining_media(reader: EpubReader, package: Any, media: MediaRegistry, renderer: Renderer,
|
def _collect_remaining_media(reader: EpubReader, package: Any, media: MediaRegistry, renderer: Renderer,
|
||||||
opts: Options) -> int:
|
opts: Options, diag: Diagnostics) -> int:
|
||||||
"""Register fonts, CSS, audio and unreferenced images; returns the number of formula images replaced by TeX."""
|
"""Register fonts, CSS, audio and unreferenced images; returns the number of formula images replaced by TeX."""
|
||||||
skipped_formula_images = 0
|
skipped_formula_images = 0
|
||||||
if opts.extract_media:
|
if opts.extract_media:
|
||||||
|
|
@ -281,6 +298,11 @@ def _collect_remaining_media(reader: EpubReader, package: Any, media: MediaRegis
|
||||||
if item.media_type in ("application/xhtml+xml", "text/html", "application/x-dtbncx+xml",
|
if item.media_type in ("application/xhtml+xml", "text/html", "application/x-dtbncx+xml",
|
||||||
"application/oebps-package+xml") or item.path == package.opf_path:
|
"application/oebps-package+xml") or item.path == package.opf_path:
|
||||||
continue
|
continue
|
||||||
|
problem = reader.entry_problem(item.path)
|
||||||
|
if problem:
|
||||||
|
diag.error("media-skipped", f"Medium {item.path} nicht übernommen: {problem}", item.path,
|
||||||
|
critical=True, exc=ContainerError)
|
||||||
|
continue
|
||||||
if item.path in renderer.consumed_formula_images:
|
if item.path in renderer.consumed_formula_images:
|
||||||
skipped_formula_images += 1
|
skipped_formula_images += 1
|
||||||
continue
|
continue
|
||||||
|
|
|
||||||
|
|
@ -16,7 +16,7 @@ EXTERNAL_SCHEMES = {"http", "https", "ftp", "mailto", "tel", "urn", "doi", "news
|
||||||
|
|
||||||
|
|
||||||
def anchor_name(raw: str) -> str:
|
def anchor_name(raw: str) -> str:
|
||||||
name = re.sub(r"[^A-Za-z0-9_.:-]", "-", raw).strip("-")
|
name = re.sub(r"[^\w.:-]", "-", raw).strip("-") # \w keeps Unicode letters (über, été)
|
||||||
if not name or not name[0].isalpha():
|
if not name or not name[0].isalpha():
|
||||||
name = "a-" + name
|
name = "a-" + name
|
||||||
return name
|
return name
|
||||||
|
|
@ -136,5 +136,7 @@ class LinkResolver:
|
||||||
file = self.chapters[self.doc_start[path]].filename
|
file = self.chapters[self.doc_start[path]].filename
|
||||||
rel = rel_link(from_file, file)
|
rel = rel_link(from_file, file)
|
||||||
status = "ok" if not frag else "anchor-missing"
|
status = "ok" if not frag else "anchor-missing"
|
||||||
return (url_path(rel) or "#"), status
|
if not rel:
|
||||||
|
return None, "self" # start of the same file: no useful link target
|
||||||
|
return url_path(rel), status
|
||||||
return None, "missing"
|
return None, "missing"
|
||||||
|
|
|
||||||
|
|
@ -136,6 +136,9 @@ def readme(md: BookMetadata, chapters: list[Chapter], endnotes: Chapter | None,
|
||||||
"# danach in diesem Ordner:",
|
"# danach in diesem Ordner:",
|
||||||
f"pandoc {shlex.quote(stem + '.md')} --toc -s{math} -o {shlex.quote(stem + '.html')}",
|
f"pandoc {shlex.quote(stem + '.md')} --toc -s{math} -o {shlex.quote(stem + '.html')}",
|
||||||
f"pandoc {shlex.quote(stem + '.md')} --toc -o {shlex.quote(stem + '.epub')}",
|
f"pandoc {shlex.quote(stem + '.md')} --toc -o {shlex.quote(stem + '.epub')}",
|
||||||
|
"# PDF über HTML: Pandoc übernimmt dann auch HTML-Tabellen und interne Sprungziele",
|
||||||
|
f"pandoc {shlex.quote(stem + '.md')} -s{math} -o {shlex.quote(stem + '.html')} && "
|
||||||
|
f"pandoc {shlex.quote(stem + '.html')} --toc --pdf-engine=xelatex -o {shlex.quote(stem + '.pdf')}",
|
||||||
]
|
]
|
||||||
out.append("## Weiterverarbeitung")
|
out.append("## Weiterverarbeitung")
|
||||||
out.append("```bash\n" + "\n".join(commands) + "\n```")
|
out.append("```bash\n" + "\n".join(commands) + "\n```")
|
||||||
|
|
@ -163,7 +166,8 @@ def warnings_md(diag: Diagnostics) -> str:
|
||||||
out += ["", f"## {key} ({len(items)})", ""]
|
out += ["", f"## {key} ({len(items)})", ""]
|
||||||
for d in items:
|
for d in items:
|
||||||
where = " · ".join(x for x in (d.source, d.chapter) if x)
|
where = " · ".join(x for x in (d.source, d.chapter) if x)
|
||||||
out.append(f"- {md_escape(d.message)}" + (f" (`{where}`)" if where else ""))
|
times = f" (×{d.count})" if d.count > 1 else ""
|
||||||
|
out.append(f"- {md_escape(d.message)}{times}" + (f" (`{where}`)" if where else ""))
|
||||||
return "\n".join(out) + "\n"
|
return "\n".join(out) + "\n"
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
121
tests/test_qa2_open.py
Normal file
121
tests/test_qa2_open.py
Normal file
|
|
@ -0,0 +1,121 @@
|
||||||
|
"""Regression tests for the remaining findings of the independent QA (QA2-09 … QA2-19)."""
|
||||||
|
|
||||||
|
import re
|
||||||
|
|
||||||
|
from conftest import PNG, SVG, Book, Doc
|
||||||
|
|
||||||
|
M = 'xmlns="http://www.w3.org/1998/Math/MathML"'
|
||||||
|
|
||||||
|
|
||||||
|
def html_table(text: str) -> str:
|
||||||
|
return text[text.index("<table"): text.index("</table>")]
|
||||||
|
|
||||||
|
|
||||||
|
def test_qa2_09_math_and_notes_in_html_tables_are_html(convert):
|
||||||
|
body = ('<h1>T</h1><table><tr><td colspan="2">Kopf</td></tr>'
|
||||||
|
f'<tr><td><math {M}><mfrac><mi>a</mi><mi>b</mi></mfrac></math></td>'
|
||||||
|
'<td><span class="InlineEquation"><img src="f.png" alt="$$x^2$$"/></span>'
|
||||||
|
'<a epub:type="noteref" href="#n1">1</a></td></tr></table>'
|
||||||
|
'<aside epub:type="footnote" id="n1"><p>Notiz <em>wichtig</em>.</p></aside>')
|
||||||
|
for mode in ("auto", "tex"):
|
||||||
|
run = convert(Book(docs=[Doc("c.xhtml", body)], media={"f.png": PNG}), "--math", mode, "--footnotes", "inline")
|
||||||
|
table = html_table(run.chapter("001"))
|
||||||
|
visible = re.sub(r'"[^"]*"', '""', table) # ignore attribute values (alt texts)
|
||||||
|
assert "<math" in table and "$" not in visible and "\\(" not in visible # renderable MathML
|
||||||
|
assert '<img src="../media/images/' in table and 'alt="$$x^2$$"' in table # formula image kept
|
||||||
|
assert "(Anmerkung:" not in table and '<sup><a href="#fn-001">1</a></sup>' in table
|
||||||
|
assert '<a id="fn-001"></a>' in run.chapter("001")
|
||||||
|
assert run.report["output_link_check"]["broken_count"] == 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_qa2_10_book_readme_recommends_pdf_via_html(convert):
|
||||||
|
"""Pandoc drops raw HTML (tables, <a id>) when going Markdown -> LaTeX; the HTML route keeps both."""
|
||||||
|
run = convert(Book(docs=[Doc("a.xhtml", "<h1>A</h1><p>x</p>")]))
|
||||||
|
readme = (run.out / "README.md").read_text(encoding="utf-8")
|
||||||
|
assert re.search(r"pandoc \S+\.md -s -o \S+\.html && pandoc \S+\.html --toc --pdf-engine=xelatex -o \S+\.pdf", readme)
|
||||||
|
|
||||||
|
|
||||||
|
def test_qa2_11_code_language_hint(convert):
|
||||||
|
py = "def f(x):\n return x\n\nclass A:\n pass"
|
||||||
|
body = "<h1>C</h1>" + "".join(f"<pre>{py}\n# {i}</pre><pre>print({i})</pre>" for i in range(10))
|
||||||
|
run = convert(Book(docs=[Doc("c.xhtml", body)]))
|
||||||
|
hints = [i for i in run.report["infos"] if i["code"] == "code-language-hint"]
|
||||||
|
assert hints and "--code-language python" in hints[0]["message"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_qa2_12_strict_aborts_on_undecodable_xhtml(convert, tmp_path):
|
||||||
|
import zipfile
|
||||||
|
from conftest import build_epub
|
||||||
|
path = build_epub(tmp_path / "g.epub", Book(docs=[Doc("a.xhtml", "<h1>A</h1><p>x</p>")]))
|
||||||
|
with zipfile.ZipFile(path) as z:
|
||||||
|
entries = {n: z.read(n) for n in z.namelist()}
|
||||||
|
entries["OEBPS/a.xhtml"] = b'<?xml version="1.0" encoding="utf-8"?><html><body><p>\xff\xfe\x00\x81 kaputt</p></body></html>'
|
||||||
|
with zipfile.ZipFile(path, "w") as z:
|
||||||
|
for n, d in entries.items():
|
||||||
|
z.writestr(n, d)
|
||||||
|
assert convert(path).rc == 0
|
||||||
|
assert convert(path, "--strict").rc == 3
|
||||||
|
|
||||||
|
|
||||||
|
def test_qa2_13_suspicious_media_is_skipped_not_fatal(convert, tmp_path):
|
||||||
|
import zipfile
|
||||||
|
from conftest import build_epub
|
||||||
|
path = build_epub(tmp_path / "b.epub", Book(docs=[Doc("a.xhtml", '<h1>A</h1><p><img src="big.png" alt="Riesig"/> Text</p>')],
|
||||||
|
media={"big.png": PNG}))
|
||||||
|
with zipfile.ZipFile(path) as z:
|
||||||
|
entries = [(i, z.read(i)) for i in z.infolist()]
|
||||||
|
with zipfile.ZipFile(path, "w", compression=zipfile.ZIP_DEFLATED) as z:
|
||||||
|
for info, data in entries:
|
||||||
|
z.writestr(info.filename, b"\0" * 60_000_000 if info.filename == "OEBPS/big.png" else data)
|
||||||
|
run = convert(path)
|
||||||
|
assert run.rc == 0
|
||||||
|
assert "Riesig" in run.chapter("001") and "big.png" not in run.chapter("001")
|
||||||
|
assert any(e["code"] == "media-skipped" for e in run.report["errors"])
|
||||||
|
assert run.report["output_link_check"]["broken_count"] == 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_qa2_14_formula_class_on_img_itself(convert):
|
||||||
|
body = '<h1>F</h1><p>Wert <img class="inlineequation" src="f.png" alt="Summe a plus b"/> ist.</p>'
|
||||||
|
run = convert(Book(docs=[Doc("c.xhtml", body)], media={"f.png": PNG}))
|
||||||
|
assert [f["origin"] for f in run.report["formulas"]["items"]] == ["image"]
|
||||||
|
assert any(w["code"] == "formula-image" for w in run.report["warnings"])
|
||||||
|
|
||||||
|
|
||||||
|
def test_qa2_15_diagnostics_count_repeats_and_html_table_formulas(convert):
|
||||||
|
img = '<span class="equation"><img src="f.png" alt="Summe"/></span>'
|
||||||
|
body = (f"<h1>F</h1><p>{img} und {img}</p>"
|
||||||
|
f'<table><tr><td colspan="2"><math {M}><mi>x</mi></math></td></tr><tr><td>a</td><td>b</td></tr></table>')
|
||||||
|
run = convert(Book(docs=[Doc("c.xhtml", body)], media={"f.png": PNG}))
|
||||||
|
w = [w for w in run.report["warnings"] if w["code"] == "formula-image"]
|
||||||
|
assert len(w) == 1 and w[0]["count"] == 2
|
||||||
|
assert run.report["formulas"]["count"] == 3
|
||||||
|
|
||||||
|
|
||||||
|
def test_qa2_16_non_ascii_ids_are_kept(convert):
|
||||||
|
body = '<h1 id="über-uns">Über</h1><p id="été">Sommer</p><p><a href="#été">S</a> <a href="#über-uns">Ü</a></p>'
|
||||||
|
text = convert(Book(docs=[Doc("c.xhtml", body)])).chapter("001")
|
||||||
|
assert '<a id="été"></a>' in text and '<a id="über-uns"></a>' in text
|
||||||
|
assert "[S](#été)" in text
|
||||||
|
|
||||||
|
|
||||||
|
def test_qa2_17_no_empty_fragment_links_and_no_epub_type_attr(convert):
|
||||||
|
body = ('<h1>K</h1><p>Text<a epub:type="noteref" href="#n">1</a></p>'
|
||||||
|
'<table><tr><td colspan="2"><a epub:type="noteref" href="#n">1</a></td></tr><tr><td>a</td><td>b</td></tr></table>'
|
||||||
|
'<aside epub:type="footnote" id="n"><p><a href="c.xhtml">↩</a> Note.</p></aside>')
|
||||||
|
text = convert(Book(docs=[Doc("c.xhtml", body)]), "--footnotes", "keep-links").chapter("001")
|
||||||
|
assert "](#)" not in text
|
||||||
|
assert ' type="noteref"' not in text
|
||||||
|
|
||||||
|
|
||||||
|
def test_qa2_18_undefined_entities_are_reported(convert):
|
||||||
|
body = "<h1>E</h1><p>A &unbekannt; B &auch;</p>"
|
||||||
|
run = convert(Book(docs=[Doc("c.xhtml", body)]))
|
||||||
|
w = [w for w in run.report["warnings"] if w["code"] == "undefined-entity"]
|
||||||
|
assert w and "unbekannt" in w[0]["message"] and "auch" in w[0]["message"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_qa2_19_adjacent_placeholders_without_media(convert):
|
||||||
|
body = '<h1>B</h1><p><img src="a.png" alt="Eins"/><img src="b.svg" alt="Zwei"/></p>'
|
||||||
|
text = convert(Book(docs=[Doc("c.xhtml", body)], media={"a.png": PNG, "b.svg": SVG}), "--no-extract-media").chapter("001")
|
||||||
|
assert "**" not in text
|
||||||
|
assert re.search(r"Bild: Eins.*Bild: Zwei", text)
|
||||||
Loading…
Reference in a new issue