dschlueter/pdf2md
Watch
1
0
Fork
You've already forked pdf2md
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Weiche Trennstriche auflösen, Courier-OCR-Ebenen nicht als Code setzen

Browse source 
- Zeilenende mit U+00AD oder "¬" wird wie Silbentrennung behandelt
  ("wach­ senden" -> "wachsenden")
- Unsichtbarer Text (OCR-Ebene) verliert das Monospace-Flag, sonst setzt
  pymupdf4llm jedes Wort in Backticks
- Tests für beide Fälle und für sichtbaren Monospace-Text

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Dieter Schlüter 2026-10-02 13:51:50 +02:00
commit 217d1ee09b
1 changed files with 49 additions and 7 deletions
Download patch file Download diff file Expand all files Collapse all files

31
tests/test_normalize.py
View file

@ -74,6 +74,37 @@ def test_pymupdf4llm_patch_keeps_compound_hyphen():
assert output.strip() == "Der Militär-Geheimdienst wusste es. Arbeits- und Kriegergeschrei."
def test_pymupdf4llm_patch_resolves_soft_hyphens():
# Weicher Trennstrich (U+00AD) bzw. "¬" am Zeilenende, auch vor Großbuchstaben
spans = [
{"text": "einer wach­", "flags": 0, "char_flags": 16, "font": "x", "block": 0, "line": 0},
{"text": "senden Demo¬", "flags": 0, "char_flags": 16, "font": "x", "block": 0, "line": 1},
{"text": "kratie und Lebens­ ", "flags": 0, "char_flags": 16, "font": "x", "block": 0, "line": 2},
{"text": "Läufe.", "flags": 0, "char_flags": 16, "font": "x", "block": 0, "line": 3},
]
output, _ = p.document_layout.get_styled_text(spans)
assert output.strip() == "einer wachsenden Demokratie und LebensLäufe."
def test_pymupdf4llm_patch_ignores_monospace_of_invisible_text():
# OCR-Ebene in Courier: unsichtbar, aber mit Monospace-Flag
spans = [
{"text": "Nature reioicej", "flags": 8, "char_flags": 0, "alpha": 0, "font": "Courier",
"block": 0, "line": 0},
{"text": "of many frendes", "flags": 8, "char_flags": 0, "alpha": 0, "font": "Courier",
"block": 0, "line": 1},
]
output, _ = p.document_layout.get_styled_text(spans)
assert output.strip() == "Nature reioicej of many frendes"
def test_visible_monospace_stays_code():
span = {"text": "print()", "flags": 8, "char_flags": int(p.VISIBLE_TEXT), "font": "Courier",
"block": 0, "line": 0}
output, _ = p.document_layout.get_styled_text([span])
assert output.strip() == "`print()`"
# --- Überschriften -----------------------------------------------------------------
def test_heading_without_space_and_bold_is_cleaned():