dschlueter/integrity_scanner_fuer_statische_Webseiten
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

feat: report broken links (4xx/5xx) in every scan

Browse source 
Neue 4xx/5xx-Seiten (nicht in Baseline) werden bewertet (+20 Pkt) und
lösen ab Schwelle einen Alert aus. Bekannte kaputte Links (in Baseline
vorhanden) erscheinen im Report als [bekannt] ohne Score-Beitrag.

Motivation: erster Crawl fand bredelar.info/www.stadtmarketing-marsberg.de
(404) — CMS-Fehler durch Schema-losen href. Soll dauerhaft sichtbar sein.

Änderungen: checker.find_broken_pages(), differ.compare_snapshots() mit
new_broken_urls/known_broken_urls, score_diff(), Report-Abschnitt,
config-Default new_broken_link:20, 4 neue Tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Your Name 2026-06-12 01:55:19 +02:00
commit b3ef3a90d4
6 changed files with 101 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

26
scanner/differ.py
View file

@ -202,6 +202,25 @@ def compare_snapshots(baseline_pages: dict, current_pages: dict, cfg: dict) -> d
old_ext = _all_external_domains(baseline_pages)
new_ext = _all_external_domains(current_pages)
# Broken link diff (4xx/5xx pages)
baseline_broken = {
url for url, p in baseline_pages.items() if p.get("status", 0) >= 400
}
current_broken_map = {
url: p.get("status", 0)
for url, p in current_pages.items()
if p.get("status", 0) >= 400
}
current_broken = set(current_broken_map)
new_broken = [
{"url": url, "status": current_broken_map[url]}
for url in sorted(current_broken - baseline_broken)
]
known_broken = [
{"url": url, "status": current_broken_map[url]}
for url in sorted(current_broken & baseline_broken)
]
# Analysis of brand-new pages (no baseline to compare against)
new_pages_analysis = [
_quick_check_new_page(current_pages[url])
@ -216,6 +235,8 @@ def compare_snapshots(baseline_pages: dict, current_pages: dict, cfg: dict) -> d
"removed_external_domains": sorted(old_ext - new_ext),
"page_diffs": page_diffs,
"new_pages_analysis": new_pages_analysis,
"new_broken_urls": new_broken,
"known_broken_urls": known_broken,
"total_pages_checked": len(common_urls),
"changed_pages": len(page_diffs),
}
@ -300,6 +321,11 @@ def score_diff(diff: dict, cfg: dict) -> dict:
add(sc.get("large_text_addition", 20),
f"{u}: +{pd['added_chars']} Zeichen neuer Text")
for entry in diff.get("new_broken_urls", []):
add(sc.get("new_broken_link", 20),
f"Neuer kaputte Link: {entry['url']} (HTTP {entry['status']})")
# known_broken_urls are reported but not scored — they were already in the baseline
# New pages: immediate suspicious content counts too
for npa in diff.get("new_pages_analysis", []):
u = npa["url"]