feat: markdown dialect option, faster search (0.1.1)
- split_markdown/load_markdown: dialect="commonmark"|"python-markdown" so list blocks match the HTML structure of the renderer (list after paragraph needs a blank line; nesting needs 4 spaces in Python-Markdown) - SearchEngine caches the searched text per block on load - regex timeout only for /regex/ terms (words and phrases are linear); typical queries over 26k blocks: ~30-140 ms instead of 250-700 ms Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
9c7b7e81f9
commit
d9f81ac918
1 changed files with 85 additions and 33 deletions
|
|
@ -12,8 +12,8 @@ Boolean full-text search over Markdown (and any other text) – as a **Python li
|
|||
## Installation
|
||||
|
||||
```bash
|
||||
pip install "mdgrep @ git+https://kitux.de/forgejo/dschlueter/mdgrep.git@v0.1.0" # library + CLI
|
||||
pip install "mdgrep[web] @ git+https://kitux.de/forgejo/dschlueter/mdgrep.git@v0.1.0" # + Flask API
|
||||
pip install "mdgrep @ git+https://kitux.de/forgejo/dschlueter/mdgrep.git@v0.1.1" # library + CLI
|
||||
pip install "mdgrep[web] @ git+https://kitux.de/forgejo/dschlueter/mdgrep.git@v0.1.1" # + Flask API
|
||||
```
|
||||
|
||||
Requires Python ≥ 3.10. Dependencies: [`regex`](https://pypi.org/project/regex/) (and `flask` for `[web]`).
|
||||
|
|
@ -97,6 +97,10 @@ engine.search("login OR sso", filters={"status": "open"})
|
|||
`load_markdown()` adds `meta["section"]` (nearest heading), `meta["kind"]`
|
||||
(`file`/`section`/`item`/`paragraph`) and `meta["markdown"]` (raw source).
|
||||
|
||||
`dialect` (for `split="list"`) makes blocks line up with the HTML of your Markdown renderer:
|
||||
`"commonmark"` (default) or `"python-markdown"` (a list needs a blank line before it; nesting
|
||||
needs 4 spaces of indentation). Important if you link results to element ids in rendered HTML.
|
||||
|
||||
### `SearchEngine(source, *, reload_check=None, reload_interval=60)`
|
||||
|
||||
`source` is a list of blocks or a callable returning them. With a callable plus `reload_check`
|
||||
|
|
|
|||
Loading…
Reference in a new issue