feat: markdown dialect option, faster search (0.1.1)

- split_markdown/load_markdown: dialect="commonmark"|"python-markdown" so list
  blocks match the HTML structure of the renderer (list after paragraph needs a
  blank line; nesting needs 4 spaces in Python-Markdown)
- SearchEngine caches the searched text per block on load
- regex timeout only for /regex/ terms (words and phrases are linear);
  typical queries over 26k blocks: ~30-140 ms instead of 250-700 ms

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
dschlueter 2026-10-12 19:37:40 +02:00
commit d9f81ac918

View file

@ -12,8 +12,8 @@ Boolean full-text search over Markdown (and any other text) – as a **Python li
## Installation
```bash
pip install "mdgrep @ git+https://kitux.de/forgejo/dschlueter/mdgrep.git@v0.1.0" # library + CLI
pip install "mdgrep[web] @ git+https://kitux.de/forgejo/dschlueter/mdgrep.git@v0.1.0" # + Flask API
pip install "mdgrep @ git+https://kitux.de/forgejo/dschlueter/mdgrep.git@v0.1.1" # library + CLI
pip install "mdgrep[web] @ git+https://kitux.de/forgejo/dschlueter/mdgrep.git@v0.1.1" # + Flask API
```
Requires Python ≥ 3.10. Dependencies: [`regex`](https://pypi.org/project/regex/) (and `flask` for `[web]`).
@ -97,6 +97,10 @@ engine.search("login OR sso", filters={"status": "open"})
`load_markdown()` adds `meta["section"]` (nearest heading), `meta["kind"]`
(`file`/`section`/`item`/`paragraph`) and `meta["markdown"]` (raw source).
`dialect` (for `split="list"`) makes blocks line up with the HTML of your Markdown renderer:
`"commonmark"` (default) or `"python-markdown"` (a list needs a blank line before it; nesting
needs 4 spaces of indentation). Important if you link results to element ids in rendered HTML.
### `SearchEngine(source, *, reload_check=None, reload_interval=60)`
`source` is a list of blocks or a callable returning them. With a callable plus `reload_check`