Merge pull request #604 from CyberMind-FR/feature/603-waf-port-live-mitmproxy-fixes-to-source

fix(waf): port live mitmproxy fixes to source — mitmproxy-11 routing + routes bind-mount

.claude/HISTORY.md

@@ -25,12 +25,32 @@ Made the apt repo at `https://admin.gk2.secubox.in/repo/` (served from
   secubox-core and others from every build). 1 pkg failed (sentinelle-gsm,
   buildinfo artifact race — deb still produced).
 
-**Blocker for public HTTPS (separate, pre-existing):** `apt.secubox.in` via
-HAProxy returns 503 because the **WAF mitmproxy LXC is crash-looping**
-(restart #45552, `PermissionError: /home/mitmproxy/.mitmproxy/config.yaml`),
-which downs the `mitmproxy_inspector` backend → ALL WAF-inspected vhosts 503
-(analyse.gk2 etc., not just apt). Repo is reachable internally (nginx :9080)
-and via the `/repo/` WebUI; public apt URL needs the WAF restored.
+**Public HTTPS now works — WAF mitmproxy restored (3 stacked bugs).** The WAF
+LXC (`mitmproxy`, served via HAProxy `mitmproxy_inspector` → 10.100.0.60:8080)
+was down board-wide (every inspected vhost 503/400), blocking public
+`apt.secubox.in`. Three compounding faults, all fixed live on gk2:
+
+1. **Crash-loop** (restart #45552): the `cookie-audit.conf` systemd drop-in
+   (added #156) overrode `ExecStart` but dropped `--set confdir=/data/mitmproxy`
+   → mitmdump fell back to `~/.mitmproxy`, which `ProtectHome=true` blocks →
+   `PermissionError: config.yaml`. Restored the flag in the drop-in (+ copied
+   the existing CA into `/data/mitmproxy` to preserve identity).
+2. **mitmproxy-11 routing**: the LXC addon (`secubox_waf.py`, pre-#499) only
+   redirected upstream in the `request` hook, but mitmproxy 11 opens the
+   upstream connection *before* `request` → traffic went to the public IP
+   (82.67.100.75). Added a `requestheaders` hook that sets
+   `flow.server_conn.address` (+ request host/port) before the connect.
+3. **Route-file drift** (the real killer, `routes_count: 0`): the addon reads
+   `/data/mitmproxy/haproxy-routes.json`, but the system maintains
+   `/srv/mitmproxy/haproxy-routes.json` (255 routes). The addon's file was
+   missing. Fixed by **bind-mounting** the host file into the container at the
+   addon's path (`/var/lib/lxc/mitmproxy/config`) so they stay in sync.
+
+Verified: `apt-get update` against `https://apt.secubox.in` fetches a
+**GPG-signed** InRelease + Packages (no signature errors), apt sees 130
+secubox packages, `.deb` downloads (200). Other inspected vhosts recovered.
+Live fixes are durable (container rootfs + LXC config survive restarts);
+porting them into the provisioning package is a follow-up.
 
 ## 2026-06-15 — threat-analyst: global security overview (1.4.3, live on gk2)
 

packages/secubox-mitmproxy/addons/secubox_waf.py

@@ -919,6 +919,28 @@ class SecuBoxWAF:
         except Exception:
             ctx.log.warn(f"BAN FAILED for {ip} ({reason}) : LAPI off + cscli unavailable")
     
+    def requestheaders(self, flow: http.HTTPFlow):
+        # #603 — mitmproxy 11 opens the upstream connection BETWEEN the
+        # requestheaders and request hooks, so upstream redirection must
+        # happen here. Doing it in request() (below) is too late: the
+        # socket is already connected to the original Host, so routed
+        # vhosts went to their public DNS IP instead of the internal
+        # backend. Setting flow.server_conn.address here fixes routing.
+        try:
+            host = flow.request.pretty_host
+            if host in self.routes:
+                bip, bport = self.routes[host]
+                orig = flow.request.headers.get('Host', host)
+                flow.request.host = bip
+                flow.request.port = bport
+                try:
+                    flow.server_conn.address = (bip, bport)
+                except Exception:
+                    pass
+                flow.request.headers['Host'] = orig
+        except Exception as e:
+            ctx.log.warn(f'[requestheaders-route] {e}')
+
     def request(self, flow: http.HTTPFlow):
         # Connection close (Phase 6.J leak fix, ref #496) — prevents mitmproxy
         # from accumulating idle keep-alive sockets to upstream backends.

packages/secubox-mitmproxy/debian/changelog

@@ -1,3 +1,15 @@
+secubox-mitmproxy (1.0.5-1~bookworm1) bookworm; urgency=medium
+
+  * fix(waf): mitmproxy-11 upstream routing (#603). The addon only redirected
+    the upstream in the `request` hook, but mitmproxy 11 opens the upstream
+    connection between `requestheaders` and `request` — so routed vhosts
+    connected to their public DNS IP instead of the internal backend (the WAF
+    was effectively pass-through). Added a `requestheaders` hook that sets
+    `flow.server_conn.address` before the connect. Ported from the live gk2
+    fix that restored apt.secubox.in + all inspected vhosts.
+
+ -- Gerald KERMA <devel@cybermind.fr>  Mon, 15 Jun 2026 16:00:00 +0200
+
 secubox-mitmproxy (1.0.4-1~bookworm1) bookworm; urgency=medium
 
   * Pre-route SELF_HOSTS guard. When a client targets the box by

packages/secubox-waf/debian/changelog

@@ -1,3 +1,18 @@
+secubox-waf (1.2.3-1~bookworm1) bookworm; urgency=medium
+
+  * fix(waf): mitmproxy-11 upstream routing — `requestheaders` hook in the
+    addon copy (#603, kept in sync with secubox-mitmproxy).
+  * fix(wafctl): bind-mount the host-maintained
+    /srv/mitmproxy/haproxy-routes.json into the LXC at the addon's read path
+    (/data/mitmproxy/haproxy-routes.json). The two had drifted (in-LXC copy
+    went stale → routes_count: 0 → no routing). Also ensures the host file
+    exists at provision time.
+  * doc(service): warn that any ExecStart drop-in MUST keep
+    `--set confdir=/data/mitmproxy` (a cookie_audit drop-in once dropped it
+    and crash-looped the WAF on ProtectHome).
+
+ -- Gerald KERMA <devel@cybermind.fr>  Mon, 15 Jun 2026 16:00:00 +0200
+
 secubox-waf (1.2.2-1~bookworm1) bookworm; urgency=medium
 
   * Phase 11+ (#509) — double-buffered cache for WAF stats consumed by

packages/secubox-waf/mitmproxy/secubox_waf.py

@@ -775,6 +775,28 @@ class SecuBoxWAF:
         except Exception:
             ctx.log.warn(f"BAN FAILED for {ip} ({reason})")
     
+    def requestheaders(self, flow: http.HTTPFlow):
+        # #603 — mitmproxy 11 opens the upstream connection BETWEEN the
+        # requestheaders and request hooks, so upstream redirection must
+        # happen here. Doing it in request() (below) is too late: the
+        # socket is already connected to the original Host, so routed
+        # vhosts went to their public DNS IP instead of the internal
+        # backend. Setting flow.server_conn.address here fixes routing.
+        try:
+            host = flow.request.pretty_host
+            if host in self.routes:
+                bip, bport = self.routes[host]
+                orig = flow.request.headers.get('Host', host)
+                flow.request.host = bip
+                flow.request.port = bport
+                try:
+                    flow.server_conn.address = (bip, bport)
+                except Exception:
+                    pass
+                flow.request.headers['Host'] = orig
+        except Exception as e:
+            ctx.log.warn(f'[requestheaders-route] {e}')
+
     def request(self, flow: http.HTTPFlow):
         # Connection close (Phase 6.J leak fix, ref #496) — prevents mitmproxy
         # from accumulating idle keep-alive sockets to upstream backends.

packages/secubox-waf/scripts/wafctl

@@ -82,6 +82,13 @@ cmd_install() {
     # Create symlink for lxc-* commands
     ln -sf "$LXC_PATH/$LXC_NAME" "/var/lib/lxc/$LXC_NAME"
 
+    # Ensure the host-maintained routes file exists so the bind-mount below has
+    # a source (#603). haproxyctl writes this file on the host; the WAF addon
+    # inside the LXC reads /data/mitmproxy/haproxy-routes.json — the bind-mount
+    # keeps them the same file instead of two copies that drift apart.
+    mkdir -p /srv/mitmproxy
+    [ -f /srv/mitmproxy/haproxy-routes.json ] || echo '{}' > /srv/mitmproxy/haproxy-routes.json
+
     # Configure container
     cat >> "$LXC_PATH/$LXC_NAME/config" << CONF
 
@@ -92,6 +99,12 @@ lxc.net.0.flags = up
 lxc.net.0.ipv4.address = $LXC_IP/24
 lxc.net.0.ipv4.gateway = 10.100.0.1
 
+# Routes: bind-mount the host-maintained haproxy routes into the path the WAF
+# addon reads. Without this the addon loaded an empty/stale in-LXC copy
+# (routes_count: 0) so no upstream routing happened and every inspected vhost
+# went to its public DNS IP. (#603)
+lxc.mount.entry = /srv/mitmproxy/haproxy-routes.json data/mitmproxy/haproxy-routes.json none bind,create=file 0 0
+
 # Autostart
 lxc.start.auto = 1
 lxc.start.delay = 5

packages/secubox-waf/systemd/mitmproxy.service

@@ -12,6 +12,10 @@ Type=simple
 User=mitmproxy
 Group=mitmproxy
 WorkingDirectory=/data/mitmproxy
+# WARNING (#603): any ExecStart override (drop-in) MUST keep
+# `--set confdir=/data/mitmproxy`. Without it mitmdump falls back to
+# ~/.mitmproxy, which ProtectHome=true makes inaccessible → PermissionError
+# crash-loop. A cookie_audit drop-in once dropped this flag and downed the WAF.
 ExecStart=/opt/mitmproxy/bin/mitmdump \
     --mode regular \
     --listen-host 0.0.0.0 \