Compare commits

..
Author SHA1 Message Date
5f7c352df1 feat: Hervorhebungen aus dem CSS des Buches übernehmen (P1)
Neues Modul epub2md/css.py: Es wertet die Stylesheets des Buches aus
(Typ-, Klassen- und ID-Selektoren, Nachfahre/Kind, Spezifität,
style-Attribute; @media und Pseudoklassen werden ignoriert). Daraus werden:
- fett/kursiv → **/*, unterstrichen → <u>, durchgestrichen → ~~,
  vertical-align super/sub → <sup>/<sub>
- Monospace-Einzelwörter → Inline-Code
- display:none → Element entfällt
Das gilt im Fließtext, für ganze Absätze und in HTML-Tabellen (dort als
HTML-Tags). Direkt aneinanderstoßende gleiche Hervorhebungen werden
zusammengefasst ('*x**y*' brach Markdown; Mathebuch: 305 → 249 Stellen).
Beispiele: C-Buch 799× kursiv und 648× fett, die vorher verloren gingen.
Abschaltbar mit --no-css-emphasis.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 04:52:36 +02:00
a54fdda2fa feat: unsichtbare Zeichen im Fließtext entfernen (P6)
Nullbreiten-Zeichen (U+200B–U+200D, U+2060, U+FEFF) und weiche
Trennstriche (U+00AD) werden aus Fließtext, Alternativtexten und
Kapiteltiteln entfernt; Code bleibt unverändert. Im Mathebuch sind das
198 Zeichen (z. B. '1.​1'). Abschaltbar mit --keep-invisible-chars; die
Anzahl steht im Bericht unter text_cleanup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 02:15:35 +02:00
9 changed files with 458 additions and 13 deletions

View file

@ -434,6 +434,8 @@ python3 epub2md_merge.py --help
| `--chapter-level N` | ab welcher Überschriftenebene eine neue Datei beginnt (Standard 1) |
| `--code-language SPRACHE` | Sprache für Codeblöcke ohne erkannte Sprache |
| `--keep-wrap-markers` | Druck-Umbruchzeichen `↪` im Code stehen lassen |
| `--keep-invisible-chars` | unsichtbare Zeichen (Nullbreiten-Leerzeichen, weiche Trennstriche) behalten |
| `--no-css-emphasis` | Hervorhebungen nur aus HTML-Tags übernehmen, nicht aus dem Stylesheet des Buches |
| `--math auto/tex/mathml/image` | Formeldarstellung |
| `--footnotes end-of-chapter/end-of-book/inline/keep-links` | Fußnoten-Platzierung |
| `--format gfm/commonmark` | Markdown-Variante; `commonmark` für sehr strikte Programme |

View file

@ -97,6 +97,8 @@ python3 epub2md.py buch.epub output/buch --force --report berichte/buch.json
| `--slug-language {de,en,none}` | `de` | Dateinamen: `de` ä→ae, ß→ss; `en` ä→a, ß→ss; `none` Unicode bleibt |
| `--code-language SPRACHE` | – | Sprachlabel für Codeblöcke ohne erkannte Sprache (z. B. `python`, `c`; Aliase wie `py` erlaubt). Erkannte Labels haben Vorrang; im Bericht `detected_by: "option"` |
| `--keep-wrap-markers` | aus | Druck-Umbruchzeichen `↪`/`➥` in Codeblöcken stehen lassen. Standard: Fortsetzungszeilen werden wieder zur ursprünglichen Codezeile zusammengefügt (Anzahl im Bericht: `wrap_lines_joined`) |
| `--css-emphasis` / `--no-css-emphasis` | an | Fett, kursiv, unterstrichen, durchgestrichen, hoch-/tiefgestellt auch übernehmen, wenn das Buch sie nur per CSS-Klasse setzt (z. B. `span.italic`). Einzelwörter in Monospace-Schrift werden zu Inline-Code; per CSS ausgeblendete Elemente (`display:none`) entfallen. Statistik im Bericht unter `css` |
| `--keep-invisible-chars` | aus | Nullbreiten-Zeichen und weiche Trennstriche im Fließtext behalten. Standard: entfernen, weil sie Suche und RAG stören (Anzahl im Bericht: `text_cleanup`). Code bleibt immer unverändert |
| `--keep-html` | aus | Unbekannte/nicht sicher konvertierbare Elemente als bereinigtes HTML erhalten (sonst: Text übernommen + Eintrag in `diagnostics/unsupported-elements.md`) |
| `--strict` | aus | Abbruch bei kritischen Fehlern (fehlende/unlesbare Spine-Dokumente, unsichere ZIP-Pfade, ungültige Spine-Verweise) |
| `--force` | aus | Bestehendes Ausgabeverzeichnis ersetzen – siehe [Sicherheit](#sicherheit-und-fehlerbehandlung) |
@ -237,6 +239,10 @@ weil CommonMark keine `[^…]`-Syntax kennt.
(`#include`, `def …:`, Shebang, …). Ist sie unsicher, bleibt das Label leer. Bei einsprachigen
Büchern kann `--code-language python` (o. Ä.) alle übrigen Blöcke kennzeichnen. Achtung: Das betrifft
auch Programmausgaben und Shell-Befehle im Buch.
- **CSS-Formatierung:** Viele Verlage setzen Hervorhebungen nur über Stylesheets. epub2md wertet die
CSS-Dateien des Buches aus: Klassen-, Element- und ID-Selektoren, Nachfahren- und Kind-Kombinatoren,
Spezifität und `style`-Attribute. `@media`-Blöcke und Selektoren mit Pseudoklassen werden ignoriert.
Direkt aneinanderstoßende gleiche Hervorhebungen werden zusammengefasst (`*x**y*` → `*xy*`).
- **Gescannte Texte:** nur Bilder, kein Text (keine OCR).
- **DRM:** wird erkannt und abgelehnt.
- **Register:** Buchstabengruppen werden zu `##`-Überschriften, Untereinträge erhalten `– `. Interne

View file

@ -9,7 +9,17 @@ from typing import Any
from .diagnostics import Diagnostics
from .models import Chapter, NavEntry
from .normalize import Doc, has_content
from .utils import classes, epub_types, is_element, iter_elements, local, norm_ws, roles, text_content
from .utils import (
classes,
epub_types,
is_element,
iter_elements,
local,
norm_ws,
remove_invisible,
roles,
text_content,
)
HEADINGS = {"h1": 1, "h2": 2, "h3": 3, "h4": 4, "h5": 5, "h6": 6}
CONTAINERS = {"body", "div", "section", "article", "aside", "main", "header", "footer", "nav", "span", "hgroup"}
@ -270,6 +280,7 @@ class ChapterBuilder:
if not doc.linear and all(not d or d == doc.path for d in ch.docs[:1]):
types.append("auxiliary")
ch.types = list(dict.fromkeys(types))
ch.title = remove_invisible(ch.title)[0] # titles feed file names, frontmatter and README
return ch
def _title_from_content(self, items: list[tuple[str, Any]], number: int) -> tuple[str, str]:

View file

@ -85,6 +85,12 @@ def build_parser() -> argparse.ArgumentParser:
p.add_argument("--keep-wrap-markers", action="store_true",
help="Druck-Umbruchzeichen (↪/➥) in Codeblöcken nicht entfernen; Standard: fortgesetzte "
"Zeilen werden wieder zur ursprünglichen Codezeile zusammengefügt")
p.add_argument("--css-emphasis", action=argparse.BooleanOptionalAction, default=True,
help="Fett/kursiv/unterstrichen/hoch-/tiefgestellt auch übernehmen, wenn das Buch sie nur per CSS "
"setzt; per CSS ausgeblendete Elemente weglassen (Standard: an)")
p.add_argument("--keep-invisible-chars", action="store_true",
help="Nullbreiten-Zeichen und weiche Trennstriche (U+00AD) im Fließtext behalten; Standard: entfernen "
"(stören Suche und Weiterverarbeitung). Code bleibt immer unverändert")
p.add_argument("--strict", action="store_true",
help="Bei kritischen Fehlern abbrechen: fehlende/unlesbare Spine-Dokumente, nicht dekodierbares "
"XHTML, unsichere ZIP-Pfade, übersprungene (verdächtige) Medien")
@ -119,6 +125,8 @@ def main(argv: list[str] | None = None) -> int:
keep_html=args.keep_html,
code_language=args.code_language or "",
keep_wrap_markers=args.keep_wrap_markers,
keep_invisible_chars=args.keep_invisible_chars,
css_emphasis=args.css_emphasis,
strict=args.strict,
verbose=args.verbose,
dry_run=args.dry_run,

194
epub2md/css.py Normal file
View file

@ -0,0 +1,194 @@
"""Minimal CSS evaluation for formatting that publishers express only through stylesheets.
Supports type/class/id selectors, descendant (` `) and child (`>`) combinators, specificity and source
order, and `style` attributes. Selectors with pseudo-classes, attribute selectors or sibling combinators
are ignored (conservative: an unknown rule never adds formatting). @-blocks (@media, @font-face, @page …)
are skipped.
"""
from __future__ import annotations
import re
from dataclasses import dataclass
from typing import Any
from .utils import classes, local
COMPOUND = re.compile(r"^(\*|[a-zA-Z][\w-]*)?((?:[.#][\w-]+)*)$")
MONOSPACE = re.compile(r"monospace|courier|consolas|menlo|monaco|lucida console|source code|dejavu sans mono|"
r"liberation mono|fira mono|fira code|inconsolata|andale mono|\bmono\b", re.I)
@dataclass
class _Compound:
tag: str | None
classes: frozenset[str]
ids: frozenset[str]
def matches(self, el: Any) -> bool:
if self.tag and self.tag != "*" and local(el.tag) != self.tag:
return False
if self.ids and (el.get("id") or "").lower() not in self.ids:
return False
return not self.classes or self.classes <= set(classes(el))
@dataclass
class _Rule:
parts: list[tuple[str, _Compound]] # (combinator to the previous part, compound); first combinator is ""
specificity: tuple[int, int, int]
order: int
decls: dict[str, str]
def matches(self, el: Any) -> bool:
if not self.parts[-1][1].matches(el):
return False
node = el
for i in range(len(self.parts) - 1, 0, -1):
comb = self.parts[i][0]
compound = self.parts[i - 1][1]
node = node.getparent()
if comb == ">":
if node is None or not compound.matches(node):
return False
else: # descendant
while node is not None and not compound.matches(node):
node = node.getparent()
if node is None:
return False
return True
def _parse_declarations(text: str) -> dict[str, str]:
decls: dict[str, str] = {}
for part in text.split(";"):
if ":" not in part:
continue
prop, value = part.split(":", 1)
prop = prop.strip().lower()
value = re.sub(r"\s*!important\s*$", "", value.strip(), flags=re.I).strip()
if prop and value:
decls[prop] = value
return decls
def _parse_selector(sel: str) -> list[tuple[str, _Compound]] | None:
sel = sel.strip().lower()
if not sel or re.search(r"[:\[\]+~]", sel):
return None
tokens = re.split(r"\s*(>)\s*|\s+", sel)
parts: list[tuple[str, _Compound]] = []
comb = ""
for tok in tokens:
if tok is None or tok == "":
continue
if tok == ">":
comb = ">"
continue
m = COMPOUND.match(tok)
if not m:
return None
rest = m.group(2) or ""
parts.append((comb if parts else "", _Compound(
tag=m.group(1),
classes=frozenset(re.findall(r"\.([\w-]+)", rest)),
ids=frozenset(re.findall(r"#([\w-]+)", rest)),
)))
comb = " "
return parts or None
def _strip_at_blocks(css: str) -> str:
"""Remove @-rules including nested blocks (@media, @font-face, @supports, @page …)."""
out: list[str] = []
i = 0
while i < len(css):
if css[i] == "@":
brace = css.find("{", i)
semi = css.find(";", i)
if semi != -1 and (brace == -1 or semi < brace): # @import/@charset …;
i = semi + 1
continue
if brace == -1:
break
depth, j = 0, brace
while j < len(css):
if css[j] == "{":
depth += 1
elif css[j] == "}":
depth -= 1
if depth == 0:
break
j += 1
i = j + 1
continue
out.append(css[i])
i += 1
return "".join(out)
class StyleSheet:
def __init__(self, css_texts: list[str]) -> None:
self.rules: list[_Rule] = []
self._index: dict[str, list[_Rule]] = {}
self._cache: dict[Any, dict[str, str]] = {}
order = 0
for css in css_texts:
css = re.sub(r"/\*.*?\*/", "", css, flags=re.S)
css = _strip_at_blocks(css)
for selectors, body in re.findall(r"([^{}]+)\{([^{}]*)\}", css):
decls = _parse_declarations(body)
if not decls:
continue
for sel in selectors.split(","):
parts = _parse_selector(sel)
if not parts:
continue
spec = (sum(len(c.ids) for _, c in parts), sum(len(c.classes) for _, c in parts),
sum(1 for _, c in parts if c.tag and c.tag != "*"))
rule = _Rule(parts, spec, order, decls)
order += 1
self.rules.append(rule)
last = parts[-1][1]
keys = [f".{c}" for c in last.classes] or ([last.tag] if last.tag and last.tag != "*" else ["*"])
for key in keys[:1]: # one index entry suffices: every key must match anyway
self._index.setdefault(key, []).append(rule)
def computed(self, el: Any) -> dict[str, str]:
"""Declared properties for ``el`` (no inheritance), including its ``style`` attribute."""
cached = self._cache.get(el)
if cached is not None:
return cached
candidates: list[_Rule] = []
for key in [f".{c}" for c in classes(el)] + [local(el.tag), "*"]:
candidates.extend(self._index.get(key, ()))
result: dict[str, str] = {}
for rule in sorted({id(r): r for r in candidates}.values(), key=lambda r: (r.specificity, r.order)):
if rule.matches(el):
result.update(rule.decls)
if el.get("style"):
result.update(_parse_declarations(el.get("style")))
self._cache[el] = result
return result
def inline_style(el: Any) -> dict[str, str]:
"""Only the ``style`` attribute (used when no stylesheet is available)."""
return _parse_declarations(el.get("style") or "")
def emphasis(style: dict[str, str]) -> dict[str, bool]:
"""Semantic formatting expressed by a computed style."""
weight = style.get("font-weight", "").lower()
deco = (style.get("text-decoration-line", "") + " " + style.get("text-decoration", "")).lower()
valign = style.get("vertical-align", "").lower()
return {
"bold": weight in ("bold", "bolder") or (weight.isdigit() and int(weight) >= 600),
"italic": style.get("font-style", "").lower() in ("italic", "oblique"),
"underline": "underline" in deco,
"strike": "line-through" in deco,
"sup": valign == "super",
"sub": valign == "sub",
"mono": bool(MONOSPACE.search(style.get("font-family", ""))),
"hidden": style.get("display", "").lower() == "none",
}

View file

@ -9,6 +9,7 @@ from __future__ import annotations
import html
import re
from collections import Counter
from collections.abc import Callable, Sequence
from dataclasses import dataclass
from typing import Any
@ -17,6 +18,7 @@ from urllib.parse import unquote
from lxml import etree
from .chapters import IdCarrier, Marker, heading_level
from .css import StyleSheet, emphasis, inline_style
from .codeblocks import (
code_text,
detect_language,
@ -45,6 +47,7 @@ from .utils import (
local,
norm_ws,
rel_link,
remove_invisible,
resolve_href,
text_content,
url_path,
@ -93,6 +96,8 @@ class RenderOptions:
language: str = "de"
code_language: str = "" # --code-language: label for blocks without a detected language
keep_wrap_markers: bool = False # --keep-wrap-markers: do not join ↪-continued code lines
keep_invisible_chars: bool = False # --keep-invisible-chars: keep zero-width chars and soft hyphens
css_emphasis: bool = True # --no-css-emphasis: ignore bold/italic/… that only the stylesheet expresses
def md_escape(text: str, table: bool = False) -> str:
@ -124,9 +129,23 @@ def escape_line_starts(text: str) -> str:
return "\n".join(out)
EMPH_SEP = "\x1f" # separates open marker, content and close marker inside an ("e", …) piece
def join_inline(pieces: list[Piece], hard_break: str = "\\\n") -> str:
out = ""
last_e: tuple[int, str, str, str, int] | None = None # (start, open, core, close, end) of the last emphasis
for kind, s in pieces:
if kind == "e":
open_, core, close = s.split(EMPH_SEP)
if last_e and last_e[4] == len(out) and last_e[1] == open_ and last_e[3] == close:
# directly adjacent identical emphasis (<em>x</em><em>y</em>): merge, "*x**y*" would break
out = out[: last_e[0]]
core = last_e[2] + core
start = len(out)
out += open_ + core + close
last_e = (start, open_, core, close, len(out))
continue
if kind == "t":
s = WS.sub(" ", s)
if not out or out.endswith((" ", "\n")):
@ -176,6 +195,17 @@ def _collapse_blank_lines(text: str) -> str:
return "\n".join(out)
ITALIC_TAGS = {"em", "i", "cite", "dfn", "var"}
BOLD_TAGS = {"strong", "b"}
CSS_SKIP_TAGS = {"br", "wbr", "img", "svg", "image", "math", "code", "samp", "tt", "kbd", "pre"}
def _wrap_text(text: str, marker: str) -> str:
"""Wrap a finished paragraph in emphasis markers (whitespace stays outside)."""
core = text.strip()
return f"{marker}{core}{marker}" if core else text
_BLOCK_SYNTAX = ("```", "~~~", "|", "<table", "- ", "> ", "#", "$$", "\\[", "* * *")
@ -229,8 +259,13 @@ def serialize_mathml(math: Any) -> str:
class Renderer:
def __init__(self, options: RenderOptions, resolver: LinkResolver, media: MediaRegistry,
notes: NoteIndex, diag: Diagnostics, zip_names: set[str],
entry_problem: Callable[[str], str | None] | None = None) -> None:
entry_problem: Callable[[str], str | None] | None = None,
stylesheet: StyleSheet | None = None) -> None:
self.entry_problem = entry_problem or (lambda path: None)
self.css = stylesheet
self.css_applied: Counter[str] = Counter()
self.hidden_elements = 0
self.invisible_removed = 0
self.opt = options
self.zip_names = zip_names
self.in_table_html = False
@ -293,7 +328,7 @@ class Renderer:
if doc:
self.doc = doc
if isinstance(item, str):
buf.append(("t", md_escape(item, self.in_table)))
buf.append(("t", self._text(item)))
elif isinstance(item, IdCarrier):
anchor = self.resolver.anchor_for(self.doc, item.element)
if anchor:
@ -361,6 +396,8 @@ class Renderer:
def block(self, el: Any) -> list[str]:
name = local(el.tag)
if self._hidden(el):
return []
ch = self.ch
if ch is not None and el is ch.title_el and not heading_level(el):
return self.heading(el, forced_level=1)
@ -401,7 +438,24 @@ class Renderer:
special = self._index_block(el, anchor)
if special is not None:
return special
blocks = self.flow(el)
italic = bold = False
if self.opt.css_emphasis and local(el.tag) in ("p", "div") and not self.in_heading \
and not any(local(d.tag) in BLOCK_TAGS for d in iter_elements(el) if d is not el) \
and text_content(el).strip():
e = self._emphasis(el)
italic, bold = e["italic"] and not self.em, e["bold"] and not self.strong
self.em += italic
self.strong += bold
try:
blocks = self.flow(el)
finally:
self.em -= italic
self.strong -= bold
if (italic or bold) and len(blocks) == 1 and not _is_block_syntax(blocks[0]) and not blocks[0].startswith("<"):
for flag, marker, key in ((italic, "*", "italic"), (bold, "**", "bold")):
if flag:
blocks[0] = _wrap_text(blocks[0], marker)
self.css_applied[key] += 1
if anchor:
if blocks and not blocks[0].startswith(("#", "`", "|", "<table", "> ", "- ", "$$", "\\[", "* * *")) \
and not re.match(r"\d+\. ", blocks[0]):
@ -755,15 +809,66 @@ class Renderer:
def inline_children(self, el: Any) -> list[Piece]:
pieces: list[Piece] = []
if el.text:
pieces.append(("t", md_escape(el.text, self.in_table)))
pieces.append(("t", self._text(el.text)))
for child in el:
if is_element(child):
pieces.extend(self.inline(child))
if child.tail:
pieces.append(("t", md_escape(child.tail, self.in_table)))
pieces.append(("t", self._text(child.tail)))
return pieces
def _emphasis(self, el: Any) -> dict[str, bool]:
style = self.css.computed(el) if self.css is not None else inline_style(el)
return emphasis(style)
def _hidden(self, el: Any) -> bool:
"""display:none in the book's CSS: readers do not show the element, so neither do we."""
if not self.opt.css_emphasis or not is_element(el) or not self._emphasis(el)["hidden"]:
return False
self.hidden_elements += 1
return True
def inline(self, el: Any) -> list[Piece]:
"""Inline rendering plus formatting that only the stylesheet expresses (P1)."""
name = local(el.tag)
if not self.opt.css_emphasis or name in CSS_SKIP_TAGS:
return self._inline_core(el)
if self._hidden(el):
return []
if self.in_heading or is_inline_code(el) or not text_content(el).strip():
return self._inline_core(el)
e = self._emphasis(el)
if e["mono"] and name != "a" and re.fullmatch(r"\S+", norm_ws(text_content(el))) \
and not any(local(d.tag) in ("img", "svg", "math", "a") for d in iter_elements(el) if d is not el):
self.css_applied["code"] += 1
anchor = self.anchors_of(el)
return ([("r", anchor)] if anchor else []) + [("r", inline_code(norm_ws(text_content(el))))]
has_link = name == "a" or any(local(d.tag) == "a" for d in iter_elements(el))
italic = e["italic"] and name not in ITALIC_TAGS and not self.em
bold = e["bold"] and name not in BOLD_TAGS and not self.strong
strike_marker = ("~~", "~~") if self.opt.format == "gfm" else ("<del>", "</del>")
wrappers = [
(e["sup"] and name != "sup", "<sup>", "</sup>", "sup"),
(e["sub"] and name != "sub", "<sub>", "</sub>", "sub"),
(e["underline"] and name not in ("u", "ins") and not has_link, "<u>", "</u>", "underline"),
(e["strike"] and name not in ("del", "s", "strike"), strike_marker[0], strike_marker[1], "strike"),
(italic, "*", "*", "italic"),
(bold, "**", "**", "bold"),
]
self.em += italic
self.strong += bold
try:
pieces = self._inline_core(el)
finally:
self.em -= italic
self.strong -= bold
for flag, open_, close, key in wrappers:
if flag:
pieces = self._wrap(pieces, open_, close)
self.css_applied[key] += 1
return pieces
def _inline_core(self, el: Any) -> list[Piece]:
name = local(el.tag)
anchor = self.anchors_of(el) if name not in ("a",) else ""
pre: list[Piece] = [("r", anchor)] if anchor and not self.in_heading else []
@ -841,7 +946,7 @@ class Renderer:
core = core[: -len(hard)].rstrip()
if not core:
return [("t", lead or trail)] if (lead or trail) else []
return [("t", lead), ("r", f"{open_}{core}{close}"), ("t", trail)]
return [("t", lead), ("e", f"{open_}{EMPH_SEP}{core}{EMPH_SEP}{close}"), ("t", trail)]
# ------------------------------------------------------------------ links
@ -1046,7 +1151,16 @@ class Renderer:
title_part = f' "{title.replace(chr(34), chr(39))}"' if title else ""
return [("r", f"![{self._alt(alt)}]({url_path(rel_link(self.file, mf.output))}{title_part})")]
def _text(self, text: str) -> str:
"""Escape prose text; zero-width characters and soft hyphens are dropped (P6)."""
if not self.opt.keep_invisible_chars:
text, n = remove_invisible(text)
self.invisible_removed += n
return md_escape(text, self.in_table)
def _alt(self, alt: str) -> str:
if not self.opt.keep_invisible_chars:
alt = remove_invisible(alt)[0]
return WS.sub(" ", alt).replace("\\", "\\\\").replace("[", "\\[").replace("]", "\\]").strip()
def _is_formula_image(self, el: Any, alt: str, path: str) -> bool:
@ -1224,6 +1338,17 @@ class Renderer:
return f"<{name}{(' ' + ' '.join(safe)) if safe else ''}>{content}</{name}>"
if name not in HTML_KEEP_TAGS or name == "span":
anchor = self.anchors_of(el)
if self.opt.css_emphasis and content.strip():
e = self._emphasis(el)
if e["hidden"]:
self.hidden_elements += 1
return ""
for flag, tag, key in ((e["italic"], "em", "italic"), (e["bold"], "strong", "bold"),
(e["underline"] and "<a " not in content, "u", "underline"),
(e["strike"], "del", "strike"), (e["sup"], "sup", "sup"), (e["sub"], "sub", "sub")):
if flag:
content = f"<{tag}>{content}</{tag}>"
self.css_applied[key] += 1
return anchor + content
attrs = []
for k, v in el.attrib.items():

View file

@ -16,7 +16,8 @@ from typing import Any
from . import TOOL_NAME, __version__
from .chapters import ChapterBuilder
from .diagnostics import Diagnostics
from .epub_reader import EpubReader, parse_xml
from .css import StyleSheet
from .epub_reader import EpubReader, decode_markup, parse_xml
from .errors import ContainerError, InputError, StructureError
from .footnotes import collect_notes
from .html_converter import RenderOptions, Renderer
@ -25,7 +26,7 @@ from .media import MediaRegistry
from .models import Chapter
from .normalize import Doc, book_uses_title_tables, is_html_toc_doc, make_doc, normalize_doc, remove_keep_tail
from .references import LinkResolver
from .utils import is_inside, iter_elements, sha256_file, slugify
from .utils import is_inside, iter_elements, local, sha256_file, slugify, text_content
from . import writer
MARKER_FILES = ("conversion-report.json", "manifest.json")
@ -47,6 +48,8 @@ class Options:
keep_html: bool = False
code_language: str = ""
keep_wrap_markers: bool = False
keep_invisible_chars: bool = False
css_emphasis: bool = True
strict: bool = False
verbose: bool = False
dry_run: bool = False
@ -133,9 +136,10 @@ def _convert(reader: EpubReader, input_path: Path, output_dir: Path, opts: Optio
package = reader.load_package()
diag.step(f"EPUB {package.version}: {package.metadata.title!r}, {len(package.spine)} Spine-Dokumente")
docs = _load_docs(reader, package, diag)
stylesheet = _load_stylesheet(reader, package, docs, diag) # before normalization drops <style>
_prepare_docs(docs, package, diag)
notes, chapters, resolver = _build_structure(docs, package, opts, diag)
rendered = _render(reader, package, chapters, resolver, notes, opts, diag)
rendered = _render(reader, package, chapters, resolver, notes, opts, diag, stylesheet)
files, all_chapters, report = _assemble(input_path, output_dir, opts, diag, generated_at, sha, package, docs,
notes, chapters, resolver, rendered)
if opts.dry_run:
@ -223,8 +227,22 @@ def _build_structure(docs: list[Doc], package: Any, opts: Options,
return notes, chapters, resolver
def _load_stylesheet(reader: EpubReader, package: Any, docs: list[Doc], diag: Diagnostics) -> StyleSheet:
"""All CSS of the book: manifest stylesheets (manifest order) plus <style> elements of the documents."""
texts: list[str] = []
for item in package.manifest.values():
if item.media_type == "text/css" or item.path.lower().endswith(".css"):
try:
texts.append(decode_markup(reader.read(item.path), item.path, diag))
except (KeyError, ContainerError):
diag.warn("css-unreadable", f"Stylesheet {item.path} nicht lesbar; seine Formatierung entfällt.", item.path)
for doc in docs:
texts.extend(text_content(e) for e in iter_elements(doc.root) if local(e.tag) == "style")
return StyleSheet(texts)
def _render(reader: EpubReader, package: Any, chapters: list[Chapter], resolver: LinkResolver, notes: Any,
opts: Options, diag: Diagnostics) -> _Rendered:
opts: Options, diag: Diagnostics, stylesheet: StyleSheet | None = None) -> _Rendered:
"""Render all chapters (and the end-of-book notes file) to Markdown and register media."""
md = package.metadata
media = MediaRegistry(package, opts.images, opts.slug_language)
@ -235,8 +253,9 @@ def _render(reader: EpubReader, package: Any, chapters: list[Chapter], resolver:
renderer = Renderer(
RenderOptions(format=opts.format, footnotes=opts.footnotes, math=opts.math,
extract_media=opts.extract_media, keep_html=opts.keep_html, language=language,
code_language=opts.code_language, keep_wrap_markers=opts.keep_wrap_markers),
resolver, media, notes, diag, set(reader.names), reader.entry_problem,
code_language=opts.code_language, keep_wrap_markers=opts.keep_wrap_markers,
keep_invisible_chars=opts.keep_invisible_chars, css_emphasis=opts.css_emphasis),
resolver, media, notes, diag, set(reader.names), reader.entry_problem, stylesheet,
)
width = max(3, len(str(len(chapters) + 1)))
notes_title = "Anmerkungen" if language.startswith("de") else "Notes"
@ -495,6 +514,9 @@ def _report(input_path: Path, sha: str, generated_at: str, package: Any, opts: O
"referenced": sum(1 for n in notes.notes.values() if n.ref_count),
"rendered": len(renderer.book_notes)},
"special_sections": special,
"text_cleanup": {"invisible_chars_removed": renderer.invisible_removed},
"css": {"rules": len(renderer.css.rules) if renderer.css else 0, "applied": dict(renderer.css_applied),
"hidden_elements": renderer.hidden_elements},
"warnings": warnings,
"errors": errors,
"infos": [d.as_dict() for d in diag.items if d.level == "info"],

View file

@ -185,6 +185,15 @@ def text_content(el: Any) -> str:
return "".join(parts)
INVISIBLE = re.compile("[\u200b\u200c\u200d\u2060\ufeff\u00ad]") # zero-width chars, BOM, soft hyphen
def remove_invisible(text: str) -> tuple[str, int]:
"""Remove zero-width characters and soft hyphens; returns (text, number removed)."""
cleaned, n = INVISIBLE.subn("", text)
return cleaned, n
def norm_ws(text: str) -> str:
return re.sub(r"[ \t\r\n\f]+", " ", text or "").strip()

68
tests/test_styles.py Normal file
View file

@ -0,0 +1,68 @@
"""Typography and layout: invisible characters, CSS emphasis, scene breaks, boxes, styles, page markers."""
from conftest import Book, Doc
ZW, SHY = "​", "­"
def test_p6_invisible_characters_removed_outside_code(convert):
body = (f"<h1>Kapitel 1.{ZW}1</h1><p>Lese{SHY}rinnen und 1.{ZW}1 sowie{ZW}Wort.</p>"
f"<pre>a{ZW}b</pre><p><code>x{SHY}y</code></p>")
run = convert(Book(docs=[Doc("c.xhtml", body)]))
text = run.chapter("001")
assert "Leserinnen und 1.1 sowieWort." in text
assert "# Kapitel 1.1" in text and 'chapter_title: "Kapitel 1.1"' in text
assert f"a{ZW}b" in text and f"x{SHY}y" in text # code stays verbatim
assert run.report["text_cleanup"]["invisible_chars_removed"] == 4
kept = convert(Book(docs=[Doc("c.xhtml", body)]), "--keep-invisible-chars").chapter("001")
assert f"Lese{SHY}rinnen" in kept
CSS = b""".i { font-style: italic } .b { font-weight: bold } span.u { text-decoration: underline }
.s { text-decoration: line-through } .hoch { vertical-align: super } .weg { display: none }
li p.first > span.b { font-weight: normal; font-family: monospace } .schrift-c { font-family: "Courier New", monospace }
p.zitat { font-style: italic } h2.t { font-weight: bold; font-style: italic } .url { text-decoration: underline }
@media amzn-kf8 { .i { font-style: normal } }"""
def styled(body: str, *args: str, convert=None):
return convert(Book(docs=[Doc("c.xhtml", body)], media={"style.css": CSS}), *args)
def test_p1_css_classes_become_markdown_emphasis(convert):
body = ('<h1>K</h1><p>Ein <span class="i">kursives</span> und <span class="b">fettes</span> Wort, '
'<span class="u">unterstrichen</span>, <span class="s">gestrichen</span>, x<span class="hoch">2</span>, '
'<span class="weg">[»] </span>sichtbar, <span class="schrift-c">printf</span> und '
'<span class="schrift-c">zwei Wörter</span>, <span style="font-weight:700">inline</span>, '
'<span class="url"><a href="https://example.org">example.org</a></span>.</p>'
'<ul><li><p class="first"><span class="b">nicht fett im Listenkopf</span></p></li></ul>'
'<p class="zitat">Ein ganzer kursiver Absatz mit <em>Betonung</em>.</p>'
'<h2 class="t">Überschrift bleibt</h2><p><em class="i">nur einmal</em></p><pre><span class="b">code</span></pre>')
run = styled(body, convert=convert)
text = run.chapter("001")
assert "Ein *kursives* und **fettes** Wort, <u>unterstrichen</u>, ~~gestrichen~~, x<sup>2</sup>, sichtbar," in text
assert "`printf` und zwei Wörter, **inline**, [example.org](https://example.org)." in text
assert "[»]" not in text and "\\[»\\]" not in text
assert "- nicht fett im Listenkopf" in text
assert "*Ein ganzer kursiver Absatz mit Betonung.*" in text
assert "## Überschrift bleibt" in text and "*nur einmal*" in text and "**nur einmal**" not in text
assert "```\ncode\n```" in text
assert run.report["css"]["hidden_elements"] == 1
assert run.report["css"]["applied"]["italic"] >= 2
plain = styled(body, "--no-css-emphasis", convert=convert).chapter("001")
assert "Ein kursives und fettes Wort" in plain
def test_p1_css_emphasis_inside_html_tables(convert):
body = ('<h1>T</h1><table><tr><td colspan="2"><span class="i">kursiv</span> und <span class="b">fett</span></td></tr>'
'<tr><td>a</td><td>b</td></tr></table>')
text = styled(body, convert=convert).chapter("001")
assert "<em>kursiv</em> und <strong>fett</strong>" in text
def test_adjacent_emphasis_is_merged(convert):
body = ('<h1>K</h1><p>Fläche (A=r<sup>2</sup><span class="i">×</span><span class="i">π</span>) und '
'<em>x</em><em>y</em> sowie <strong>a</strong><strong>b</strong> und <em>c</em> <em>d</em>.</p>')
text = styled(body, convert=convert).chapter("001")
assert "<sup>2</sup>*×π*" in text
assert "*xy*" in text and "**ab**" in text and "*c* *d*" in text