fix: Bugfixes, toten Code entfernt, Cache-Normalisierung & Session

Echte Bugs:
- extractor: JSON-LD-Crash bei Nicht-Dict-Items (isinstance-Guard)
- baseline/__main__: Crawl-Fehler ins Snapshot-Manifest -> erscheinen im Report
- __main__: Whitelist-/Header-/Webshell-Checks nur noch auf status==200
- crawler: Noise-Param-Regex auf Voll-Key (view=/value= nicht mehr verworfen)
- differ/__main__: unerwartetes JSON-LD via eigenem Kanal, auch auf
  unveraenderten Seiten erkannt, kein Re-Alert auf bereits genehmigte Typen

Aufgeraeumt:
- checker: check_internal_paths, find_broken_pages, canonical_is_hijacked,
  check_jsonld_types entfernt (nicht verdrahtet)
- allowed_paths.yaml-Logik und tote Scoring-Keys entfernt
- tote Imports entfernt

Neuer aktiver Schutz:
- Webshell-/Backdoor-Dateinamen-Check verdrahtet (suspicious_filename: 60).
  Regex wortgrenzen-verankert gegen Fehlalarm auf PSEMailerAntispam.js

Effizienz/Struktur:
- crawler nutzt requests.Session (keep-alive)
- Cache-Buster-Normalisierung an einer Stelle (Extraktion) -> stabile
  Snapshots, differ wieder reiner Set-Diff

Tests: 111 gruen (neu: test_checker.py + Regressionstests)
CLAUDE.md aktualisiert

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

scanner/crawler.py

@@ -1,6 +1,5 @@
 """Pure-Python crawler — no wget dependency."""
 import logging
-import re
 import time
 from urllib.parse import parse_qs, urlencode, urljoin, urlparse, urlunparse
 
@@ -9,11 +8,20 @@ from bs4 import BeautifulSoup
 
 logger = logging.getLogger(__name__)
 
-# Query parameters that indicate calendar/session/tracking noise
-_NOISE_PARAM_RE = re.compile(
-    r"^(year|month|day|date|week|session|token|sid|nonce|_|v|cache|utm_|fbclid)",
-    re.I,
-)
+# Query-param keys that indicate calendar / session / tracking / cache noise.
+# Matched as whole keys (not prefixes) so legitimate params like ?view= or
+# ?value= are never silently dropped. utm_* is a known tracking prefix family.
+_NOISE_PARAM_KEYS = {
+    "year", "month", "day", "date", "week",
+    "session", "token", "sid", "nonce", "fbclid",
+    "v", "ver", "version", "cache", "cb", "_", "_t",
+}
+
+
+def _is_noise_param(key: str) -> bool:
+    """True if a query-param key is calendar/session/tracking/cache noise."""
+    k = key.lower()
+    return k in _NOISE_PARAM_KEYS or k.startswith("utm_")
 
 
 def normalize_url(url: str) -> str | None:
@@ -36,7 +44,7 @@ def normalize_url(url: str) -> str | None:
                 (k, v)
                 for k, vs in parse_qs(p.query, keep_blank_values=True).items()
                 for v in vs
-                if not _NOISE_PARAM_RE.match(k)
+                if not _is_noise_param(k)
             )
             p = p._replace(query=urlencode(clean_params))
         return urlunparse(p)
@@ -53,8 +61,8 @@ def should_crawl(url: str, base_netloc: str, skip_extensions: list[str]) -> bool
     for ext in skip_extensions:
         if path_lower.endswith(ext):
             return False
-    # Skip URLs whose path looks like calendar/session noise
-    if _NOISE_PARAM_RE.search(p.query):
+    # Skip URLs whose query carries calendar/session/tracking noise
+    if p.query and any(_is_noise_param(k) for k in parse_qs(p.query)):
         return False
     return True
 
@@ -68,6 +76,8 @@ class Crawler:
         self.skip_ext: list[str] = cfg["crawl"]["skip_extensions"]
         self.timeout: int = cfg["request_timeout"]
         self.headers = {"User-Agent": cfg["user_agent"]}
+        self.session = requests.Session()
+        self.session.headers.update(self.headers)
 
     def crawl(self) -> dict:
         """
@@ -82,6 +92,30 @@ class Crawler:
         pages: list[dict] = []
         errors: list[dict] = []
 
+        try:
+            self._crawl_loop(queue, seen, pages, errors)
+        finally:
+            self.session.close()
+
+        logger.info(
+            "Crawl finished: %d pages, %d errors, %d queued but skipped",
+            len(pages),
+            len(errors),
+            len(queue),
+        )
+        return {
+            "target": self.target,
+            "pages": pages,
+            "errors": errors,
+        }
+
+    def _crawl_loop(
+        self,
+        queue: list[str],
+        seen: set[str],
+        pages: list[dict],
+        errors: list[dict],
+    ) -> None:
         while queue and len(pages) < self.max_pages:
             url = queue.pop(0)
             if url in seen:
@@ -93,9 +127,8 @@ class Crawler:
 
             logger.debug("Crawling %s", url)
             try:
-                resp = requests.get(
+                resp = self.session.get(
                     url,
-                    headers=self.headers,
                     timeout=self.timeout,
                     allow_redirects=True,
                 )
@@ -123,18 +156,6 @@ class Crawler:
                 errors.append({"url": url, "error": str(exc)})
                 logger.warning("Error crawling %s: %s", url, exc)
 
-        logger.info(
-            "Crawl finished: %d pages, %d errors, %d queued but skipped",
-            len(pages),
-            len(errors),
-            len(queue),
-        )
-        return {
-            "target": self.target,
-            "pages": pages,
-            "errors": errors,
-        }
-
     def _extract_follow_links(self, html: str, base_url: str) -> list[str]:
         """Extract internal <a href> links worth following."""
         result = []