3a44cfb515
Adds crawl.exclude_paths config option (list of URL path prefixes).
Matching URLs are excluded at two levels:
- Crawler: not fetched at all (saves HTTP requests)
- Differ: filtered from both baseline and current before comparison,
so legacy baseline entries under excluded paths don't appear as
"missing URLs" after the option is added retroactively
Example config:
crawl:
exclude_paths: ["/rss/", "/feed/", "/news/archiv/"]
8 new tests (176 total).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
||
|---|---|---|
| .. | ||
| __init__.py | ||
| __main__.py | ||
| alerter.py | ||
| baseline.py | ||
| checker.py | ||
| config.py | ||
| crawler.py | ||
| differ.py | ||
| extractor.py | ||
| plain.py | ||