feat: Hervorhebungen aus dem CSS des Buches übernehmen (P1)
Neues Modul epub2md/css.py: Es wertet die Stylesheets des Buches aus
(Typ-, Klassen- und ID-Selektoren, Nachfahre/Kind, Spezifität,
style-Attribute; @media und Pseudoklassen werden ignoriert). Daraus werden:
- fett/kursiv → **/*, unterstrichen → <u>, durchgestrichen → ~~,
vertical-align super/sub → <sup>/<sub>
- Monospace-Einzelwörter → Inline-Code
- display:none → Element entfällt
Das gilt im Fließtext, für ganze Absätze und in HTML-Tabellen (dort als
HTML-Tags). Direkt aneinanderstoßende gleiche Hervorhebungen werden
zusammengefasst ('*x**y*' brach Markdown; Mathebuch: 305 → 249 Stellen).
Beispiele: C-Buch 799× kursiv und 648× fett, die vorher verloren gingen.
Abschaltbar mit --no-css-emphasis.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
a54fdda2fa
commit
5f7c352df1
7 changed files with 395 additions and 9 deletions
|
|
@ -435,6 +435,7 @@ python3 epub2md_merge.py --help
|
||||||
| `--code-language SPRACHE` | Sprache für Codeblöcke ohne erkannte Sprache |
|
| `--code-language SPRACHE` | Sprache für Codeblöcke ohne erkannte Sprache |
|
||||||
| `--keep-wrap-markers` | Druck-Umbruchzeichen `↪` im Code stehen lassen |
|
| `--keep-wrap-markers` | Druck-Umbruchzeichen `↪` im Code stehen lassen |
|
||||||
| `--keep-invisible-chars` | unsichtbare Zeichen (Nullbreiten-Leerzeichen, weiche Trennstriche) behalten |
|
| `--keep-invisible-chars` | unsichtbare Zeichen (Nullbreiten-Leerzeichen, weiche Trennstriche) behalten |
|
||||||
|
| `--no-css-emphasis` | Hervorhebungen nur aus HTML-Tags übernehmen, nicht aus dem Stylesheet des Buches |
|
||||||
| `--math auto/tex/mathml/image` | Formeldarstellung |
|
| `--math auto/tex/mathml/image` | Formeldarstellung |
|
||||||
| `--footnotes end-of-chapter/end-of-book/inline/keep-links` | Fußnoten-Platzierung |
|
| `--footnotes end-of-chapter/end-of-book/inline/keep-links` | Fußnoten-Platzierung |
|
||||||
| `--format gfm/commonmark` | Markdown-Variante; `commonmark` für sehr strikte Programme |
|
| `--format gfm/commonmark` | Markdown-Variante; `commonmark` für sehr strikte Programme |
|
||||||
|
|
|
||||||
|
|
@ -97,6 +97,7 @@ python3 epub2md.py buch.epub output/buch --force --report berichte/buch.json
|
||||||
| `--slug-language {de,en,none}` | `de` | Dateinamen: `de` ä→ae, ß→ss; `en` ä→a, ß→ss; `none` Unicode bleibt |
|
| `--slug-language {de,en,none}` | `de` | Dateinamen: `de` ä→ae, ß→ss; `en` ä→a, ß→ss; `none` Unicode bleibt |
|
||||||
| `--code-language SPRACHE` | – | Sprachlabel für Codeblöcke ohne erkannte Sprache (z. B. `python`, `c`; Aliase wie `py` erlaubt). Erkannte Labels haben Vorrang; im Bericht `detected_by: "option"` |
|
| `--code-language SPRACHE` | – | Sprachlabel für Codeblöcke ohne erkannte Sprache (z. B. `python`, `c`; Aliase wie `py` erlaubt). Erkannte Labels haben Vorrang; im Bericht `detected_by: "option"` |
|
||||||
| `--keep-wrap-markers` | aus | Druck-Umbruchzeichen `↪`/`➥` in Codeblöcken stehen lassen. Standard: Fortsetzungszeilen werden wieder zur ursprünglichen Codezeile zusammengefügt (Anzahl im Bericht: `wrap_lines_joined`) |
|
| `--keep-wrap-markers` | aus | Druck-Umbruchzeichen `↪`/`➥` in Codeblöcken stehen lassen. Standard: Fortsetzungszeilen werden wieder zur ursprünglichen Codezeile zusammengefügt (Anzahl im Bericht: `wrap_lines_joined`) |
|
||||||
|
| `--css-emphasis` / `--no-css-emphasis` | an | Fett, kursiv, unterstrichen, durchgestrichen, hoch-/tiefgestellt auch übernehmen, wenn das Buch sie nur per CSS-Klasse setzt (z. B. `span.italic`). Einzelwörter in Monospace-Schrift werden zu Inline-Code; per CSS ausgeblendete Elemente (`display:none`) entfallen. Statistik im Bericht unter `css` |
|
||||||
| `--keep-invisible-chars` | aus | Nullbreiten-Zeichen und weiche Trennstriche im Fließtext behalten. Standard: entfernen, weil sie Suche und RAG stören (Anzahl im Bericht: `text_cleanup`). Code bleibt immer unverändert |
|
| `--keep-invisible-chars` | aus | Nullbreiten-Zeichen und weiche Trennstriche im Fließtext behalten. Standard: entfernen, weil sie Suche und RAG stören (Anzahl im Bericht: `text_cleanup`). Code bleibt immer unverändert |
|
||||||
| `--keep-html` | aus | Unbekannte/nicht sicher konvertierbare Elemente als bereinigtes HTML erhalten (sonst: Text übernommen + Eintrag in `diagnostics/unsupported-elements.md`) |
|
| `--keep-html` | aus | Unbekannte/nicht sicher konvertierbare Elemente als bereinigtes HTML erhalten (sonst: Text übernommen + Eintrag in `diagnostics/unsupported-elements.md`) |
|
||||||
| `--strict` | aus | Abbruch bei kritischen Fehlern (fehlende/unlesbare Spine-Dokumente, unsichere ZIP-Pfade, ungültige Spine-Verweise) |
|
| `--strict` | aus | Abbruch bei kritischen Fehlern (fehlende/unlesbare Spine-Dokumente, unsichere ZIP-Pfade, ungültige Spine-Verweise) |
|
||||||
|
|
@ -238,6 +239,10 @@ weil CommonMark keine `[^…]`-Syntax kennt.
|
||||||
(`#include`, `def …:`, Shebang, …). Ist sie unsicher, bleibt das Label leer. Bei einsprachigen
|
(`#include`, `def …:`, Shebang, …). Ist sie unsicher, bleibt das Label leer. Bei einsprachigen
|
||||||
Büchern kann `--code-language python` (o. Ä.) alle übrigen Blöcke kennzeichnen. Achtung: Das betrifft
|
Büchern kann `--code-language python` (o. Ä.) alle übrigen Blöcke kennzeichnen. Achtung: Das betrifft
|
||||||
auch Programmausgaben und Shell-Befehle im Buch.
|
auch Programmausgaben und Shell-Befehle im Buch.
|
||||||
|
- **CSS-Formatierung:** Viele Verlage setzen Hervorhebungen nur über Stylesheets. epub2md wertet die
|
||||||
|
CSS-Dateien des Buches aus: Klassen-, Element- und ID-Selektoren, Nachfahren- und Kind-Kombinatoren,
|
||||||
|
Spezifität und `style`-Attribute. `@media`-Blöcke und Selektoren mit Pseudoklassen werden ignoriert.
|
||||||
|
Direkt aneinanderstoßende gleiche Hervorhebungen werden zusammengefasst (`*x**y*` → `*xy*`).
|
||||||
- **Gescannte Texte:** nur Bilder, kein Text (keine OCR).
|
- **Gescannte Texte:** nur Bilder, kein Text (keine OCR).
|
||||||
- **DRM:** wird erkannt und abgelehnt.
|
- **DRM:** wird erkannt und abgelehnt.
|
||||||
- **Register:** Buchstabengruppen werden zu `##`-Überschriften, Untereinträge erhalten `– `. Interne
|
- **Register:** Buchstabengruppen werden zu `##`-Überschriften, Untereinträge erhalten `– `. Interne
|
||||||
|
|
|
||||||
|
|
@ -85,6 +85,9 @@ def build_parser() -> argparse.ArgumentParser:
|
||||||
p.add_argument("--keep-wrap-markers", action="store_true",
|
p.add_argument("--keep-wrap-markers", action="store_true",
|
||||||
help="Druck-Umbruchzeichen (↪/➥) in Codeblöcken nicht entfernen; Standard: fortgesetzte "
|
help="Druck-Umbruchzeichen (↪/➥) in Codeblöcken nicht entfernen; Standard: fortgesetzte "
|
||||||
"Zeilen werden wieder zur ursprünglichen Codezeile zusammengefügt")
|
"Zeilen werden wieder zur ursprünglichen Codezeile zusammengefügt")
|
||||||
|
p.add_argument("--css-emphasis", action=argparse.BooleanOptionalAction, default=True,
|
||||||
|
help="Fett/kursiv/unterstrichen/hoch-/tiefgestellt auch übernehmen, wenn das Buch sie nur per CSS "
|
||||||
|
"setzt; per CSS ausgeblendete Elemente weglassen (Standard: an)")
|
||||||
p.add_argument("--keep-invisible-chars", action="store_true",
|
p.add_argument("--keep-invisible-chars", action="store_true",
|
||||||
help="Nullbreiten-Zeichen und weiche Trennstriche (U+00AD) im Fließtext behalten; Standard: entfernen "
|
help="Nullbreiten-Zeichen und weiche Trennstriche (U+00AD) im Fließtext behalten; Standard: entfernen "
|
||||||
"(stören Suche und Weiterverarbeitung). Code bleibt immer unverändert")
|
"(stören Suche und Weiterverarbeitung). Code bleibt immer unverändert")
|
||||||
|
|
@ -123,6 +126,7 @@ def main(argv: list[str] | None = None) -> int:
|
||||||
code_language=args.code_language or "",
|
code_language=args.code_language or "",
|
||||||
keep_wrap_markers=args.keep_wrap_markers,
|
keep_wrap_markers=args.keep_wrap_markers,
|
||||||
keep_invisible_chars=args.keep_invisible_chars,
|
keep_invisible_chars=args.keep_invisible_chars,
|
||||||
|
css_emphasis=args.css_emphasis,
|
||||||
strict=args.strict,
|
strict=args.strict,
|
||||||
verbose=args.verbose,
|
verbose=args.verbose,
|
||||||
dry_run=args.dry_run,
|
dry_run=args.dry_run,
|
||||||
|
|
|
||||||
194
epub2md/css.py
Normal file
194
epub2md/css.py
Normal file
|
|
@ -0,0 +1,194 @@
|
||||||
|
"""Minimal CSS evaluation for formatting that publishers express only through stylesheets.
|
||||||
|
|
||||||
|
Supports type/class/id selectors, descendant (` `) and child (`>`) combinators, specificity and source
|
||||||
|
order, and `style` attributes. Selectors with pseudo-classes, attribute selectors or sibling combinators
|
||||||
|
are ignored (conservative: an unknown rule never adds formatting). @-blocks (@media, @font-face, @page …)
|
||||||
|
are skipped.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from .utils import classes, local
|
||||||
|
|
||||||
|
COMPOUND = re.compile(r"^(\*|[a-zA-Z][\w-]*)?((?:[.#][\w-]+)*)$")
|
||||||
|
MONOSPACE = re.compile(r"monospace|courier|consolas|menlo|monaco|lucida console|source code|dejavu sans mono|"
|
||||||
|
r"liberation mono|fira mono|fira code|inconsolata|andale mono|\bmono\b", re.I)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class _Compound:
|
||||||
|
tag: str | None
|
||||||
|
classes: frozenset[str]
|
||||||
|
ids: frozenset[str]
|
||||||
|
|
||||||
|
def matches(self, el: Any) -> bool:
|
||||||
|
if self.tag and self.tag != "*" and local(el.tag) != self.tag:
|
||||||
|
return False
|
||||||
|
if self.ids and (el.get("id") or "").lower() not in self.ids:
|
||||||
|
return False
|
||||||
|
return not self.classes or self.classes <= set(classes(el))
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class _Rule:
|
||||||
|
parts: list[tuple[str, _Compound]] # (combinator to the previous part, compound); first combinator is ""
|
||||||
|
specificity: tuple[int, int, int]
|
||||||
|
order: int
|
||||||
|
decls: dict[str, str]
|
||||||
|
|
||||||
|
def matches(self, el: Any) -> bool:
|
||||||
|
if not self.parts[-1][1].matches(el):
|
||||||
|
return False
|
||||||
|
node = el
|
||||||
|
for i in range(len(self.parts) - 1, 0, -1):
|
||||||
|
comb = self.parts[i][0]
|
||||||
|
compound = self.parts[i - 1][1]
|
||||||
|
node = node.getparent()
|
||||||
|
if comb == ">":
|
||||||
|
if node is None or not compound.matches(node):
|
||||||
|
return False
|
||||||
|
else: # descendant
|
||||||
|
while node is not None and not compound.matches(node):
|
||||||
|
node = node.getparent()
|
||||||
|
if node is None:
|
||||||
|
return False
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_declarations(text: str) -> dict[str, str]:
|
||||||
|
decls: dict[str, str] = {}
|
||||||
|
for part in text.split(";"):
|
||||||
|
if ":" not in part:
|
||||||
|
continue
|
||||||
|
prop, value = part.split(":", 1)
|
||||||
|
prop = prop.strip().lower()
|
||||||
|
value = re.sub(r"\s*!important\s*$", "", value.strip(), flags=re.I).strip()
|
||||||
|
if prop and value:
|
||||||
|
decls[prop] = value
|
||||||
|
return decls
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_selector(sel: str) -> list[tuple[str, _Compound]] | None:
|
||||||
|
sel = sel.strip().lower()
|
||||||
|
if not sel or re.search(r"[:\[\]+~]", sel):
|
||||||
|
return None
|
||||||
|
tokens = re.split(r"\s*(>)\s*|\s+", sel)
|
||||||
|
parts: list[tuple[str, _Compound]] = []
|
||||||
|
comb = ""
|
||||||
|
for tok in tokens:
|
||||||
|
if tok is None or tok == "":
|
||||||
|
continue
|
||||||
|
if tok == ">":
|
||||||
|
comb = ">"
|
||||||
|
continue
|
||||||
|
m = COMPOUND.match(tok)
|
||||||
|
if not m:
|
||||||
|
return None
|
||||||
|
rest = m.group(2) or ""
|
||||||
|
parts.append((comb if parts else "", _Compound(
|
||||||
|
tag=m.group(1),
|
||||||
|
classes=frozenset(re.findall(r"\.([\w-]+)", rest)),
|
||||||
|
ids=frozenset(re.findall(r"#([\w-]+)", rest)),
|
||||||
|
)))
|
||||||
|
comb = " "
|
||||||
|
return parts or None
|
||||||
|
|
||||||
|
|
||||||
|
def _strip_at_blocks(css: str) -> str:
|
||||||
|
"""Remove @-rules including nested blocks (@media, @font-face, @supports, @page …)."""
|
||||||
|
out: list[str] = []
|
||||||
|
i = 0
|
||||||
|
while i < len(css):
|
||||||
|
if css[i] == "@":
|
||||||
|
brace = css.find("{", i)
|
||||||
|
semi = css.find(";", i)
|
||||||
|
if semi != -1 and (brace == -1 or semi < brace): # @import/@charset …;
|
||||||
|
i = semi + 1
|
||||||
|
continue
|
||||||
|
if brace == -1:
|
||||||
|
break
|
||||||
|
depth, j = 0, brace
|
||||||
|
while j < len(css):
|
||||||
|
if css[j] == "{":
|
||||||
|
depth += 1
|
||||||
|
elif css[j] == "}":
|
||||||
|
depth -= 1
|
||||||
|
if depth == 0:
|
||||||
|
break
|
||||||
|
j += 1
|
||||||
|
i = j + 1
|
||||||
|
continue
|
||||||
|
out.append(css[i])
|
||||||
|
i += 1
|
||||||
|
return "".join(out)
|
||||||
|
|
||||||
|
|
||||||
|
class StyleSheet:
|
||||||
|
def __init__(self, css_texts: list[str]) -> None:
|
||||||
|
self.rules: list[_Rule] = []
|
||||||
|
self._index: dict[str, list[_Rule]] = {}
|
||||||
|
self._cache: dict[Any, dict[str, str]] = {}
|
||||||
|
order = 0
|
||||||
|
for css in css_texts:
|
||||||
|
css = re.sub(r"/\*.*?\*/", "", css, flags=re.S)
|
||||||
|
css = _strip_at_blocks(css)
|
||||||
|
for selectors, body in re.findall(r"([^{}]+)\{([^{}]*)\}", css):
|
||||||
|
decls = _parse_declarations(body)
|
||||||
|
if not decls:
|
||||||
|
continue
|
||||||
|
for sel in selectors.split(","):
|
||||||
|
parts = _parse_selector(sel)
|
||||||
|
if not parts:
|
||||||
|
continue
|
||||||
|
spec = (sum(len(c.ids) for _, c in parts), sum(len(c.classes) for _, c in parts),
|
||||||
|
sum(1 for _, c in parts if c.tag and c.tag != "*"))
|
||||||
|
rule = _Rule(parts, spec, order, decls)
|
||||||
|
order += 1
|
||||||
|
self.rules.append(rule)
|
||||||
|
last = parts[-1][1]
|
||||||
|
keys = [f".{c}" for c in last.classes] or ([last.tag] if last.tag and last.tag != "*" else ["*"])
|
||||||
|
for key in keys[:1]: # one index entry suffices: every key must match anyway
|
||||||
|
self._index.setdefault(key, []).append(rule)
|
||||||
|
|
||||||
|
def computed(self, el: Any) -> dict[str, str]:
|
||||||
|
"""Declared properties for ``el`` (no inheritance), including its ``style`` attribute."""
|
||||||
|
cached = self._cache.get(el)
|
||||||
|
if cached is not None:
|
||||||
|
return cached
|
||||||
|
candidates: list[_Rule] = []
|
||||||
|
for key in [f".{c}" for c in classes(el)] + [local(el.tag), "*"]:
|
||||||
|
candidates.extend(self._index.get(key, ()))
|
||||||
|
result: dict[str, str] = {}
|
||||||
|
for rule in sorted({id(r): r for r in candidates}.values(), key=lambda r: (r.specificity, r.order)):
|
||||||
|
if rule.matches(el):
|
||||||
|
result.update(rule.decls)
|
||||||
|
if el.get("style"):
|
||||||
|
result.update(_parse_declarations(el.get("style")))
|
||||||
|
self._cache[el] = result
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def inline_style(el: Any) -> dict[str, str]:
|
||||||
|
"""Only the ``style`` attribute (used when no stylesheet is available)."""
|
||||||
|
return _parse_declarations(el.get("style") or "")
|
||||||
|
|
||||||
|
|
||||||
|
def emphasis(style: dict[str, str]) -> dict[str, bool]:
|
||||||
|
"""Semantic formatting expressed by a computed style."""
|
||||||
|
weight = style.get("font-weight", "").lower()
|
||||||
|
deco = (style.get("text-decoration-line", "") + " " + style.get("text-decoration", "")).lower()
|
||||||
|
valign = style.get("vertical-align", "").lower()
|
||||||
|
return {
|
||||||
|
"bold": weight in ("bold", "bolder") or (weight.isdigit() and int(weight) >= 600),
|
||||||
|
"italic": style.get("font-style", "").lower() in ("italic", "oblique"),
|
||||||
|
"underline": "underline" in deco,
|
||||||
|
"strike": "line-through" in deco,
|
||||||
|
"sup": valign == "super",
|
||||||
|
"sub": valign == "sub",
|
||||||
|
"mono": bool(MONOSPACE.search(style.get("font-family", ""))),
|
||||||
|
"hidden": style.get("display", "").lower() == "none",
|
||||||
|
}
|
||||||
|
|
@ -9,6 +9,7 @@ from __future__ import annotations
|
||||||
|
|
||||||
import html
|
import html
|
||||||
import re
|
import re
|
||||||
|
from collections import Counter
|
||||||
from collections.abc import Callable, Sequence
|
from collections.abc import Callable, Sequence
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
@ -17,6 +18,7 @@ from urllib.parse import unquote
|
||||||
from lxml import etree
|
from lxml import etree
|
||||||
|
|
||||||
from .chapters import IdCarrier, Marker, heading_level
|
from .chapters import IdCarrier, Marker, heading_level
|
||||||
|
from .css import StyleSheet, emphasis, inline_style
|
||||||
from .codeblocks import (
|
from .codeblocks import (
|
||||||
code_text,
|
code_text,
|
||||||
detect_language,
|
detect_language,
|
||||||
|
|
@ -95,6 +97,7 @@ class RenderOptions:
|
||||||
code_language: str = "" # --code-language: label for blocks without a detected language
|
code_language: str = "" # --code-language: label for blocks without a detected language
|
||||||
keep_wrap_markers: bool = False # --keep-wrap-markers: do not join ↪-continued code lines
|
keep_wrap_markers: bool = False # --keep-wrap-markers: do not join ↪-continued code lines
|
||||||
keep_invisible_chars: bool = False # --keep-invisible-chars: keep zero-width chars and soft hyphens
|
keep_invisible_chars: bool = False # --keep-invisible-chars: keep zero-width chars and soft hyphens
|
||||||
|
css_emphasis: bool = True # --no-css-emphasis: ignore bold/italic/… that only the stylesheet expresses
|
||||||
|
|
||||||
|
|
||||||
def md_escape(text: str, table: bool = False) -> str:
|
def md_escape(text: str, table: bool = False) -> str:
|
||||||
|
|
@ -126,9 +129,23 @@ def escape_line_starts(text: str) -> str:
|
||||||
return "\n".join(out)
|
return "\n".join(out)
|
||||||
|
|
||||||
|
|
||||||
|
EMPH_SEP = "\x1f" # separates open marker, content and close marker inside an ("e", …) piece
|
||||||
|
|
||||||
|
|
||||||
def join_inline(pieces: list[Piece], hard_break: str = "\\\n") -> str:
|
def join_inline(pieces: list[Piece], hard_break: str = "\\\n") -> str:
|
||||||
out = ""
|
out = ""
|
||||||
|
last_e: tuple[int, str, str, str, int] | None = None # (start, open, core, close, end) of the last emphasis
|
||||||
for kind, s in pieces:
|
for kind, s in pieces:
|
||||||
|
if kind == "e":
|
||||||
|
open_, core, close = s.split(EMPH_SEP)
|
||||||
|
if last_e and last_e[4] == len(out) and last_e[1] == open_ and last_e[3] == close:
|
||||||
|
# directly adjacent identical emphasis (<em>x</em><em>y</em>): merge, "*x**y*" would break
|
||||||
|
out = out[: last_e[0]]
|
||||||
|
core = last_e[2] + core
|
||||||
|
start = len(out)
|
||||||
|
out += open_ + core + close
|
||||||
|
last_e = (start, open_, core, close, len(out))
|
||||||
|
continue
|
||||||
if kind == "t":
|
if kind == "t":
|
||||||
s = WS.sub(" ", s)
|
s = WS.sub(" ", s)
|
||||||
if not out or out.endswith((" ", "\n")):
|
if not out or out.endswith((" ", "\n")):
|
||||||
|
|
@ -178,6 +195,17 @@ def _collapse_blank_lines(text: str) -> str:
|
||||||
return "\n".join(out)
|
return "\n".join(out)
|
||||||
|
|
||||||
|
|
||||||
|
ITALIC_TAGS = {"em", "i", "cite", "dfn", "var"}
|
||||||
|
BOLD_TAGS = {"strong", "b"}
|
||||||
|
CSS_SKIP_TAGS = {"br", "wbr", "img", "svg", "image", "math", "code", "samp", "tt", "kbd", "pre"}
|
||||||
|
|
||||||
|
|
||||||
|
def _wrap_text(text: str, marker: str) -> str:
|
||||||
|
"""Wrap a finished paragraph in emphasis markers (whitespace stays outside)."""
|
||||||
|
core = text.strip()
|
||||||
|
return f"{marker}{core}{marker}" if core else text
|
||||||
|
|
||||||
|
|
||||||
_BLOCK_SYNTAX = ("```", "~~~", "|", "<table", "- ", "> ", "#", "$$", "\\[", "* * *")
|
_BLOCK_SYNTAX = ("```", "~~~", "|", "<table", "- ", "> ", "#", "$$", "\\[", "* * *")
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -231,8 +259,12 @@ def serialize_mathml(math: Any) -> str:
|
||||||
class Renderer:
|
class Renderer:
|
||||||
def __init__(self, options: RenderOptions, resolver: LinkResolver, media: MediaRegistry,
|
def __init__(self, options: RenderOptions, resolver: LinkResolver, media: MediaRegistry,
|
||||||
notes: NoteIndex, diag: Diagnostics, zip_names: set[str],
|
notes: NoteIndex, diag: Diagnostics, zip_names: set[str],
|
||||||
entry_problem: Callable[[str], str | None] | None = None) -> None:
|
entry_problem: Callable[[str], str | None] | None = None,
|
||||||
|
stylesheet: StyleSheet | None = None) -> None:
|
||||||
self.entry_problem = entry_problem or (lambda path: None)
|
self.entry_problem = entry_problem or (lambda path: None)
|
||||||
|
self.css = stylesheet
|
||||||
|
self.css_applied: Counter[str] = Counter()
|
||||||
|
self.hidden_elements = 0
|
||||||
self.invisible_removed = 0
|
self.invisible_removed = 0
|
||||||
self.opt = options
|
self.opt = options
|
||||||
self.zip_names = zip_names
|
self.zip_names = zip_names
|
||||||
|
|
@ -364,6 +396,8 @@ class Renderer:
|
||||||
|
|
||||||
def block(self, el: Any) -> list[str]:
|
def block(self, el: Any) -> list[str]:
|
||||||
name = local(el.tag)
|
name = local(el.tag)
|
||||||
|
if self._hidden(el):
|
||||||
|
return []
|
||||||
ch = self.ch
|
ch = self.ch
|
||||||
if ch is not None and el is ch.title_el and not heading_level(el):
|
if ch is not None and el is ch.title_el and not heading_level(el):
|
||||||
return self.heading(el, forced_level=1)
|
return self.heading(el, forced_level=1)
|
||||||
|
|
@ -404,7 +438,24 @@ class Renderer:
|
||||||
special = self._index_block(el, anchor)
|
special = self._index_block(el, anchor)
|
||||||
if special is not None:
|
if special is not None:
|
||||||
return special
|
return special
|
||||||
blocks = self.flow(el)
|
italic = bold = False
|
||||||
|
if self.opt.css_emphasis and local(el.tag) in ("p", "div") and not self.in_heading \
|
||||||
|
and not any(local(d.tag) in BLOCK_TAGS for d in iter_elements(el) if d is not el) \
|
||||||
|
and text_content(el).strip():
|
||||||
|
e = self._emphasis(el)
|
||||||
|
italic, bold = e["italic"] and not self.em, e["bold"] and not self.strong
|
||||||
|
self.em += italic
|
||||||
|
self.strong += bold
|
||||||
|
try:
|
||||||
|
blocks = self.flow(el)
|
||||||
|
finally:
|
||||||
|
self.em -= italic
|
||||||
|
self.strong -= bold
|
||||||
|
if (italic or bold) and len(blocks) == 1 and not _is_block_syntax(blocks[0]) and not blocks[0].startswith("<"):
|
||||||
|
for flag, marker, key in ((italic, "*", "italic"), (bold, "**", "bold")):
|
||||||
|
if flag:
|
||||||
|
blocks[0] = _wrap_text(blocks[0], marker)
|
||||||
|
self.css_applied[key] += 1
|
||||||
if anchor:
|
if anchor:
|
||||||
if blocks and not blocks[0].startswith(("#", "`", "|", "<table", "> ", "- ", "$$", "\\[", "* * *")) \
|
if blocks and not blocks[0].startswith(("#", "`", "|", "<table", "> ", "- ", "$$", "\\[", "* * *")) \
|
||||||
and not re.match(r"\d+\. ", blocks[0]):
|
and not re.match(r"\d+\. ", blocks[0]):
|
||||||
|
|
@ -766,7 +817,58 @@ class Renderer:
|
||||||
pieces.append(("t", self._text(child.tail)))
|
pieces.append(("t", self._text(child.tail)))
|
||||||
return pieces
|
return pieces
|
||||||
|
|
||||||
|
def _emphasis(self, el: Any) -> dict[str, bool]:
|
||||||
|
style = self.css.computed(el) if self.css is not None else inline_style(el)
|
||||||
|
return emphasis(style)
|
||||||
|
|
||||||
|
def _hidden(self, el: Any) -> bool:
|
||||||
|
"""display:none in the book's CSS: readers do not show the element, so neither do we."""
|
||||||
|
if not self.opt.css_emphasis or not is_element(el) or not self._emphasis(el)["hidden"]:
|
||||||
|
return False
|
||||||
|
self.hidden_elements += 1
|
||||||
|
return True
|
||||||
|
|
||||||
def inline(self, el: Any) -> list[Piece]:
|
def inline(self, el: Any) -> list[Piece]:
|
||||||
|
"""Inline rendering plus formatting that only the stylesheet expresses (P1)."""
|
||||||
|
name = local(el.tag)
|
||||||
|
if not self.opt.css_emphasis or name in CSS_SKIP_TAGS:
|
||||||
|
return self._inline_core(el)
|
||||||
|
if self._hidden(el):
|
||||||
|
return []
|
||||||
|
if self.in_heading or is_inline_code(el) or not text_content(el).strip():
|
||||||
|
return self._inline_core(el)
|
||||||
|
e = self._emphasis(el)
|
||||||
|
if e["mono"] and name != "a" and re.fullmatch(r"\S+", norm_ws(text_content(el))) \
|
||||||
|
and not any(local(d.tag) in ("img", "svg", "math", "a") for d in iter_elements(el) if d is not el):
|
||||||
|
self.css_applied["code"] += 1
|
||||||
|
anchor = self.anchors_of(el)
|
||||||
|
return ([("r", anchor)] if anchor else []) + [("r", inline_code(norm_ws(text_content(el))))]
|
||||||
|
has_link = name == "a" or any(local(d.tag) == "a" for d in iter_elements(el))
|
||||||
|
italic = e["italic"] and name not in ITALIC_TAGS and not self.em
|
||||||
|
bold = e["bold"] and name not in BOLD_TAGS and not self.strong
|
||||||
|
strike_marker = ("~~", "~~") if self.opt.format == "gfm" else ("<del>", "</del>")
|
||||||
|
wrappers = [
|
||||||
|
(e["sup"] and name != "sup", "<sup>", "</sup>", "sup"),
|
||||||
|
(e["sub"] and name != "sub", "<sub>", "</sub>", "sub"),
|
||||||
|
(e["underline"] and name not in ("u", "ins") and not has_link, "<u>", "</u>", "underline"),
|
||||||
|
(e["strike"] and name not in ("del", "s", "strike"), strike_marker[0], strike_marker[1], "strike"),
|
||||||
|
(italic, "*", "*", "italic"),
|
||||||
|
(bold, "**", "**", "bold"),
|
||||||
|
]
|
||||||
|
self.em += italic
|
||||||
|
self.strong += bold
|
||||||
|
try:
|
||||||
|
pieces = self._inline_core(el)
|
||||||
|
finally:
|
||||||
|
self.em -= italic
|
||||||
|
self.strong -= bold
|
||||||
|
for flag, open_, close, key in wrappers:
|
||||||
|
if flag:
|
||||||
|
pieces = self._wrap(pieces, open_, close)
|
||||||
|
self.css_applied[key] += 1
|
||||||
|
return pieces
|
||||||
|
|
||||||
|
def _inline_core(self, el: Any) -> list[Piece]:
|
||||||
name = local(el.tag)
|
name = local(el.tag)
|
||||||
anchor = self.anchors_of(el) if name not in ("a",) else ""
|
anchor = self.anchors_of(el) if name not in ("a",) else ""
|
||||||
pre: list[Piece] = [("r", anchor)] if anchor and not self.in_heading else []
|
pre: list[Piece] = [("r", anchor)] if anchor and not self.in_heading else []
|
||||||
|
|
@ -844,7 +946,7 @@ class Renderer:
|
||||||
core = core[: -len(hard)].rstrip()
|
core = core[: -len(hard)].rstrip()
|
||||||
if not core:
|
if not core:
|
||||||
return [("t", lead or trail)] if (lead or trail) else []
|
return [("t", lead or trail)] if (lead or trail) else []
|
||||||
return [("t", lead), ("r", f"{open_}{core}{close}"), ("t", trail)]
|
return [("t", lead), ("e", f"{open_}{EMPH_SEP}{core}{EMPH_SEP}{close}"), ("t", trail)]
|
||||||
|
|
||||||
# ------------------------------------------------------------------ links
|
# ------------------------------------------------------------------ links
|
||||||
|
|
||||||
|
|
@ -1236,6 +1338,17 @@ class Renderer:
|
||||||
return f"<{name}{(' ' + ' '.join(safe)) if safe else ''}>{content}</{name}>"
|
return f"<{name}{(' ' + ' '.join(safe)) if safe else ''}>{content}</{name}>"
|
||||||
if name not in HTML_KEEP_TAGS or name == "span":
|
if name not in HTML_KEEP_TAGS or name == "span":
|
||||||
anchor = self.anchors_of(el)
|
anchor = self.anchors_of(el)
|
||||||
|
if self.opt.css_emphasis and content.strip():
|
||||||
|
e = self._emphasis(el)
|
||||||
|
if e["hidden"]:
|
||||||
|
self.hidden_elements += 1
|
||||||
|
return ""
|
||||||
|
for flag, tag, key in ((e["italic"], "em", "italic"), (e["bold"], "strong", "bold"),
|
||||||
|
(e["underline"] and "<a " not in content, "u", "underline"),
|
||||||
|
(e["strike"], "del", "strike"), (e["sup"], "sup", "sup"), (e["sub"], "sub", "sub")):
|
||||||
|
if flag:
|
||||||
|
content = f"<{tag}>{content}</{tag}>"
|
||||||
|
self.css_applied[key] += 1
|
||||||
return anchor + content
|
return anchor + content
|
||||||
attrs = []
|
attrs = []
|
||||||
for k, v in el.attrib.items():
|
for k, v in el.attrib.items():
|
||||||
|
|
|
||||||
|
|
@ -16,7 +16,8 @@ from typing import Any
|
||||||
from . import TOOL_NAME, __version__
|
from . import TOOL_NAME, __version__
|
||||||
from .chapters import ChapterBuilder
|
from .chapters import ChapterBuilder
|
||||||
from .diagnostics import Diagnostics
|
from .diagnostics import Diagnostics
|
||||||
from .epub_reader import EpubReader, parse_xml
|
from .css import StyleSheet
|
||||||
|
from .epub_reader import EpubReader, decode_markup, parse_xml
|
||||||
from .errors import ContainerError, InputError, StructureError
|
from .errors import ContainerError, InputError, StructureError
|
||||||
from .footnotes import collect_notes
|
from .footnotes import collect_notes
|
||||||
from .html_converter import RenderOptions, Renderer
|
from .html_converter import RenderOptions, Renderer
|
||||||
|
|
@ -25,7 +26,7 @@ from .media import MediaRegistry
|
||||||
from .models import Chapter
|
from .models import Chapter
|
||||||
from .normalize import Doc, book_uses_title_tables, is_html_toc_doc, make_doc, normalize_doc, remove_keep_tail
|
from .normalize import Doc, book_uses_title_tables, is_html_toc_doc, make_doc, normalize_doc, remove_keep_tail
|
||||||
from .references import LinkResolver
|
from .references import LinkResolver
|
||||||
from .utils import is_inside, iter_elements, sha256_file, slugify
|
from .utils import is_inside, iter_elements, local, sha256_file, slugify, text_content
|
||||||
from . import writer
|
from . import writer
|
||||||
|
|
||||||
MARKER_FILES = ("conversion-report.json", "manifest.json")
|
MARKER_FILES = ("conversion-report.json", "manifest.json")
|
||||||
|
|
@ -48,6 +49,7 @@ class Options:
|
||||||
code_language: str = ""
|
code_language: str = ""
|
||||||
keep_wrap_markers: bool = False
|
keep_wrap_markers: bool = False
|
||||||
keep_invisible_chars: bool = False
|
keep_invisible_chars: bool = False
|
||||||
|
css_emphasis: bool = True
|
||||||
strict: bool = False
|
strict: bool = False
|
||||||
verbose: bool = False
|
verbose: bool = False
|
||||||
dry_run: bool = False
|
dry_run: bool = False
|
||||||
|
|
@ -134,9 +136,10 @@ def _convert(reader: EpubReader, input_path: Path, output_dir: Path, opts: Optio
|
||||||
package = reader.load_package()
|
package = reader.load_package()
|
||||||
diag.step(f"EPUB {package.version}: {package.metadata.title!r}, {len(package.spine)} Spine-Dokumente")
|
diag.step(f"EPUB {package.version}: {package.metadata.title!r}, {len(package.spine)} Spine-Dokumente")
|
||||||
docs = _load_docs(reader, package, diag)
|
docs = _load_docs(reader, package, diag)
|
||||||
|
stylesheet = _load_stylesheet(reader, package, docs, diag) # before normalization drops <style>
|
||||||
_prepare_docs(docs, package, diag)
|
_prepare_docs(docs, package, diag)
|
||||||
notes, chapters, resolver = _build_structure(docs, package, opts, diag)
|
notes, chapters, resolver = _build_structure(docs, package, opts, diag)
|
||||||
rendered = _render(reader, package, chapters, resolver, notes, opts, diag)
|
rendered = _render(reader, package, chapters, resolver, notes, opts, diag, stylesheet)
|
||||||
files, all_chapters, report = _assemble(input_path, output_dir, opts, diag, generated_at, sha, package, docs,
|
files, all_chapters, report = _assemble(input_path, output_dir, opts, diag, generated_at, sha, package, docs,
|
||||||
notes, chapters, resolver, rendered)
|
notes, chapters, resolver, rendered)
|
||||||
if opts.dry_run:
|
if opts.dry_run:
|
||||||
|
|
@ -224,8 +227,22 @@ def _build_structure(docs: list[Doc], package: Any, opts: Options,
|
||||||
return notes, chapters, resolver
|
return notes, chapters, resolver
|
||||||
|
|
||||||
|
|
||||||
|
def _load_stylesheet(reader: EpubReader, package: Any, docs: list[Doc], diag: Diagnostics) -> StyleSheet:
|
||||||
|
"""All CSS of the book: manifest stylesheets (manifest order) plus <style> elements of the documents."""
|
||||||
|
texts: list[str] = []
|
||||||
|
for item in package.manifest.values():
|
||||||
|
if item.media_type == "text/css" or item.path.lower().endswith(".css"):
|
||||||
|
try:
|
||||||
|
texts.append(decode_markup(reader.read(item.path), item.path, diag))
|
||||||
|
except (KeyError, ContainerError):
|
||||||
|
diag.warn("css-unreadable", f"Stylesheet {item.path} nicht lesbar; seine Formatierung entfällt.", item.path)
|
||||||
|
for doc in docs:
|
||||||
|
texts.extend(text_content(e) for e in iter_elements(doc.root) if local(e.tag) == "style")
|
||||||
|
return StyleSheet(texts)
|
||||||
|
|
||||||
|
|
||||||
def _render(reader: EpubReader, package: Any, chapters: list[Chapter], resolver: LinkResolver, notes: Any,
|
def _render(reader: EpubReader, package: Any, chapters: list[Chapter], resolver: LinkResolver, notes: Any,
|
||||||
opts: Options, diag: Diagnostics) -> _Rendered:
|
opts: Options, diag: Diagnostics, stylesheet: StyleSheet | None = None) -> _Rendered:
|
||||||
"""Render all chapters (and the end-of-book notes file) to Markdown and register media."""
|
"""Render all chapters (and the end-of-book notes file) to Markdown and register media."""
|
||||||
md = package.metadata
|
md = package.metadata
|
||||||
media = MediaRegistry(package, opts.images, opts.slug_language)
|
media = MediaRegistry(package, opts.images, opts.slug_language)
|
||||||
|
|
@ -237,8 +254,8 @@ def _render(reader: EpubReader, package: Any, chapters: list[Chapter], resolver:
|
||||||
RenderOptions(format=opts.format, footnotes=opts.footnotes, math=opts.math,
|
RenderOptions(format=opts.format, footnotes=opts.footnotes, math=opts.math,
|
||||||
extract_media=opts.extract_media, keep_html=opts.keep_html, language=language,
|
extract_media=opts.extract_media, keep_html=opts.keep_html, language=language,
|
||||||
code_language=opts.code_language, keep_wrap_markers=opts.keep_wrap_markers,
|
code_language=opts.code_language, keep_wrap_markers=opts.keep_wrap_markers,
|
||||||
keep_invisible_chars=opts.keep_invisible_chars),
|
keep_invisible_chars=opts.keep_invisible_chars, css_emphasis=opts.css_emphasis),
|
||||||
resolver, media, notes, diag, set(reader.names), reader.entry_problem,
|
resolver, media, notes, diag, set(reader.names), reader.entry_problem, stylesheet,
|
||||||
)
|
)
|
||||||
width = max(3, len(str(len(chapters) + 1)))
|
width = max(3, len(str(len(chapters) + 1)))
|
||||||
notes_title = "Anmerkungen" if language.startswith("de") else "Notes"
|
notes_title = "Anmerkungen" if language.startswith("de") else "Notes"
|
||||||
|
|
@ -498,6 +515,8 @@ def _report(input_path: Path, sha: str, generated_at: str, package: Any, opts: O
|
||||||
"rendered": len(renderer.book_notes)},
|
"rendered": len(renderer.book_notes)},
|
||||||
"special_sections": special,
|
"special_sections": special,
|
||||||
"text_cleanup": {"invisible_chars_removed": renderer.invisible_removed},
|
"text_cleanup": {"invisible_chars_removed": renderer.invisible_removed},
|
||||||
|
"css": {"rules": len(renderer.css.rules) if renderer.css else 0, "applied": dict(renderer.css_applied),
|
||||||
|
"hidden_elements": renderer.hidden_elements},
|
||||||
"warnings": warnings,
|
"warnings": warnings,
|
||||||
"errors": errors,
|
"errors": errors,
|
||||||
"infos": [d.as_dict() for d in diag.items if d.level == "info"],
|
"infos": [d.as_dict() for d in diag.items if d.level == "info"],
|
||||||
|
|
|
||||||
|
|
@ -16,3 +16,53 @@ def test_p6_invisible_characters_removed_outside_code(convert):
|
||||||
assert run.report["text_cleanup"]["invisible_chars_removed"] == 4
|
assert run.report["text_cleanup"]["invisible_chars_removed"] == 4
|
||||||
kept = convert(Book(docs=[Doc("c.xhtml", body)]), "--keep-invisible-chars").chapter("001")
|
kept = convert(Book(docs=[Doc("c.xhtml", body)]), "--keep-invisible-chars").chapter("001")
|
||||||
assert f"Lese{SHY}rinnen" in kept
|
assert f"Lese{SHY}rinnen" in kept
|
||||||
|
|
||||||
|
|
||||||
|
CSS = b""".i { font-style: italic } .b { font-weight: bold } span.u { text-decoration: underline }
|
||||||
|
.s { text-decoration: line-through } .hoch { vertical-align: super } .weg { display: none }
|
||||||
|
li p.first > span.b { font-weight: normal; font-family: monospace } .schrift-c { font-family: "Courier New", monospace }
|
||||||
|
p.zitat { font-style: italic } h2.t { font-weight: bold; font-style: italic } .url { text-decoration: underline }
|
||||||
|
@media amzn-kf8 { .i { font-style: normal } }"""
|
||||||
|
|
||||||
|
|
||||||
|
def styled(body: str, *args: str, convert=None):
|
||||||
|
return convert(Book(docs=[Doc("c.xhtml", body)], media={"style.css": CSS}), *args)
|
||||||
|
|
||||||
|
|
||||||
|
def test_p1_css_classes_become_markdown_emphasis(convert):
|
||||||
|
body = ('<h1>K</h1><p>Ein <span class="i">kursives</span> und <span class="b">fettes</span> Wort, '
|
||||||
|
'<span class="u">unterstrichen</span>, <span class="s">gestrichen</span>, x<span class="hoch">2</span>, '
|
||||||
|
'<span class="weg">[»] </span>sichtbar, <span class="schrift-c">printf</span> und '
|
||||||
|
'<span class="schrift-c">zwei Wörter</span>, <span style="font-weight:700">inline</span>, '
|
||||||
|
'<span class="url"><a href="https://example.org">example.org</a></span>.</p>'
|
||||||
|
'<ul><li><p class="first"><span class="b">nicht fett im Listenkopf</span></p></li></ul>'
|
||||||
|
'<p class="zitat">Ein ganzer kursiver Absatz mit <em>Betonung</em>.</p>'
|
||||||
|
'<h2 class="t">Überschrift bleibt</h2><p><em class="i">nur einmal</em></p><pre><span class="b">code</span></pre>')
|
||||||
|
run = styled(body, convert=convert)
|
||||||
|
text = run.chapter("001")
|
||||||
|
assert "Ein *kursives* und **fettes** Wort, <u>unterstrichen</u>, ~~gestrichen~~, x<sup>2</sup>, sichtbar," in text
|
||||||
|
assert "`printf` und zwei Wörter, **inline**, [example.org](https://example.org)." in text
|
||||||
|
assert "[»]" not in text and "\\[»\\]" not in text
|
||||||
|
assert "- nicht fett im Listenkopf" in text
|
||||||
|
assert "*Ein ganzer kursiver Absatz mit Betonung.*" in text
|
||||||
|
assert "## Überschrift bleibt" in text and "*nur einmal*" in text and "**nur einmal**" not in text
|
||||||
|
assert "```\ncode\n```" in text
|
||||||
|
assert run.report["css"]["hidden_elements"] == 1
|
||||||
|
assert run.report["css"]["applied"]["italic"] >= 2
|
||||||
|
plain = styled(body, "--no-css-emphasis", convert=convert).chapter("001")
|
||||||
|
assert "Ein kursives und fettes Wort" in plain
|
||||||
|
|
||||||
|
|
||||||
|
def test_p1_css_emphasis_inside_html_tables(convert):
|
||||||
|
body = ('<h1>T</h1><table><tr><td colspan="2"><span class="i">kursiv</span> und <span class="b">fett</span></td></tr>'
|
||||||
|
'<tr><td>a</td><td>b</td></tr></table>')
|
||||||
|
text = styled(body, convert=convert).chapter("001")
|
||||||
|
assert "<em>kursiv</em> und <strong>fett</strong>" in text
|
||||||
|
|
||||||
|
|
||||||
|
def test_adjacent_emphasis_is_merged(convert):
|
||||||
|
body = ('<h1>K</h1><p>Fläche (A=r<sup>2</sup><span class="i">×</span><span class="i">π</span>) und '
|
||||||
|
'<em>x</em><em>y</em> sowie <strong>a</strong><strong>b</strong> und <em>c</em> <em>d</em>.</p>')
|
||||||
|
text = styled(body, convert=convert).chapter("001")
|
||||||
|
assert "<sup>2</sup>*×π*" in text
|
||||||
|
assert "*xy*" in text and "**ab**" in text and "*c* *d*" in text
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue