packages/secubox-toolbox/debian/changelog

@@ -1,55 +1,3 @@
-secubox-toolbox (2.5.0-1~bookworm1) bookworm; urgency=medium
-
-  * Phase 9 (#501) — multi-worker fanout for mitm-wg.
-    Single-process mitm-wg saturated one ARM core at ~90 % even with
-    just 2-3 active wg peers, limited by the Python GIL.  Phase 9
-    spreads new TCP flows across N=4 worker instances :
-      - systemd : new template secubox-toolbox-mitm-wg-worker@.service ;
-        per-instance Environment=MITM_WG_LISTEN_PORT=808%i (8081..8084).
-        Per-worker RuntimeMaxSec=3 h, MemoryMax=128M, TasksMax=128.
-      - launcher : reads MITM_WG_LISTEN_PORT (default 8081 for legacy
-        single-worker service).
-      - nft : new drop-in nftables.d/secubox-toolbox-wg-fanout.nft
-        replaces the prerouting chain with a numgen inc round-robin
-        across 4 ports.  Conntrack pins each TCP flow to its initially
-        assigned worker for the lifetime of the connection
-        (sticky-per-flow ; rebalancing only at new connection).
-      - opt-in : single-worker secubox-toolbox-mitm-wg.service stays
-        shipped + functional.  Activation recipe in the worker unit's
-        [Unit] description.
-    Live numbers on gk2 with 2 active Linux peers + 1 iPhone :
-      single  90-95 % CPU on 1 core (saturated)
-      fanout  ~55 % avg per worker × 4, 0-70 % range (headroom)
-    SQLite WAL on toolbox.db handles 4 concurrent writers ; the
-    cert-pin auto-learning dynamic bypass file is the remaining race
-    surface (4 writers can dupe a line, the launcher's sort -u
-    de-dupes at next reload).  A real filelock lands in Phase 9.1.
-
- -- Gérald Kerma <devel@cybermind.fr>  mar., 09 juin 2026 04:27:27 +0000
-
-secubox-toolbox (2.4.3-1~bookworm1) bookworm; urgency=medium
-
-  * Phase 8.2 perf (#500) — defensive performance work :
-    - Captive mitm service flags now match the mitm-wg quick win of
-      2.4.1 (--set http2=true / connection_strategy=eager /
-      keep_host_header=true). No perceptible change today (the
-      captive AP is down so the service idles at ~0 % CPU) but the
-      moment the AP is reactivated the captive picks up the same
-      ×4 CPU win the WG path got.
-    - Addon SQLite writes (local_store + utiq) are now fire-and-
-      forget through a singleton ThreadPoolExecutor. Each addon owns
-      its own bg writer thread (sbx_store_write / sbx_utiq_write).
-      The mitmproxy asyncio event loop never blocks on _conn() open
-      / INSERT / fsync. Live diagnostic showed the actual mitm-wg
-      bottleneck is mitmproxy itself (TLS termination + per-flow
-      H/2 parsing) under multi-peer fan-in, not the addon writes ;
-      the change is still warranted as defensive hygiene before
-      shipping the Phase 9 multi-worker fanout that will benefit
-      from non-blocking writes when 4 workers contend on the same
-      SQLite file.
-
- -- Gérald Kerma <devel@cybermind.fr>  mar., 09 juin 2026 04:19:18 +0000
-
 secubox-toolbox (2.4.2-1~bookworm1) bookworm; urgency=medium
 
   * Landing page kbin.gk2.secubox.in : la section 'Démo install R3'

packages/secubox-toolbox/debian/rules

@@ -35,11 +35,6 @@ override_dh_installsystemd:
 	install -d debian/secubox-toolbox/lib/systemd/system/secubox-toolbox-mitm-wg.service.d
 	install -m 0644 systemd/secubox-toolbox-mitm-wg.service.d/10-runtime-max.conf \
 	  debian/secubox-toolbox/lib/systemd/system/secubox-toolbox-mitm-wg.service.d/
-	# Phase 9 (#501) : multi-worker fanout template — opt-in via
-	# systemctl enable @1..4.  See unit's [Unit] doc string for the
-	# activation + rollback recipe.
-	install -m 0644 systemd/secubox-toolbox-mitm-wg-worker@.service \
-	  debian/secubox-toolbox/lib/systemd/system/
 	# Primary unit goes via dh_installsystemd which also handles the enable helpers.
 	cp systemd/secubox-toolbox.service debian/secubox-toolbox.service
 	dh_installsystemd --no-start --no-enable
@@ -62,10 +57,6 @@ override_dh_strip:
 	install -d debian/secubox-toolbox/usr/share/secubox/toolbox/nftables.d
 	install -m 0644 nftables.d/secubox-toolbox-wg.nft \
 	  debian/secubox-toolbox/usr/share/secubox/toolbox/nftables.d/
-	# Phase 9 (#501) : fanout DNAT drop-in (opt-in).  Operator activates
-	# by symlinking /etc/nftables.d/secubox-toolbox-wg.nft → this file.
-	install -m 0644 nftables.d/secubox-toolbox-wg-fanout.nft \
-	  debian/secubox-toolbox/usr/share/secubox/toolbox/nftables.d/
 	install -m 0755 sbin/secubox-toolbox-wg-restore \
 	  debian/secubox-toolbox/usr/sbin/
 	install -m 0644 systemd/secubox-toolbox-wg-restore.service \

packages/secubox-toolbox/debian/secubox-toolbox-mitm.service

@@ -19,9 +19,6 @@ ExecStart=/usr/bin/mitmdump \
     --set confdir=/etc/secubox/toolbox/mitm \
     --set ssl_insecure=false \
     --set web_open_browser=false \
-    --set http2=true \
-    --set connection_strategy=eager \
-    --set keep_host_header=true \
     -s /usr/lib/secubox/toolbox/mitmproxy_addons/cookies.py \
     -s /usr/lib/secubox/toolbox/mitmproxy_addons/dpi.py \
     -s /usr/lib/secubox/toolbox/mitmproxy_addons/avatar.py \

packages/secubox-toolbox/mitmproxy_addons/local_store.py

@@ -145,15 +145,7 @@ def _peer_ip(flow) -> str | None:
     return None
 
 
-# Phase 8.B perf (#500) — fire-and-forget SQLite writes via a single
+def _insert(mac_hash: str | None, source: str, payload: dict) -> None:
-# background thread so the mitmproxy asyncio event loop never blocks
-# on `fsync()`. Single worker keeps inserts ordered AND avoids SQLite
-# write contention (the engine itself serialises writers in WAL mode).
-import concurrent.futures as _futures
-_executor = _futures.ThreadPoolExecutor(max_workers=1, thread_name_prefix="sbx_store_write")
-
-
-def _insert_sync(mac_hash: str | None, source: str, payload: dict) -> None:
     if not mac_hash:
         return
     try:
@@ -171,22 +163,6 @@ def _insert_sync(mac_hash: str | None, source: str, payload: dict) -> None:
         log.debug("sqlite insert failed: %s", e)
 
 
-def _insert(mac_hash: str | None, source: str, payload: dict) -> None:
-    """Phase 8.B — submit the insert to the bg thread. Hook returns
-    instantly ; the mitmproxy event loop keeps churning flows while
-    the SQLite IO happens off-thread.
-
-    Submit may raise RuntimeError if the executor was shut down during
-    interpreter teardown ; we swallow that to keep the hook silent on
-    shutdown."""
-    if not mac_hash:
-        return
-    try:
-        _executor.submit(_insert_sync, mac_hash, source, payload)
-    except RuntimeError:
-        pass
-
-
 # ──────────────── mitmproxy hooks ────────────────
 
 class LocalStore:

packages/secubox-toolbox/nftables.d/secubox-toolbox-wg-fanout.nft

@@ -1,53 +0,0 @@
-# SPDX-License-Identifier: LicenseRef-CMSD-1.0
-# Phase 9 (#501) — multi-worker fanout drop-in for the R3 wg tunnel mitm.
-#
-# REPLACES the prerouting rules from secubox-toolbox-wg.nft :
-#   iif wg-toolbox tcp dport 443 dnat ip to 10.99.1.1:8081  (single port)
-# with a round-robin numgen mapping to ports 8081..8084.
-#
-# Why numgen inc and not jhash : nftables 1.0.6 (Debian bookworm) doesn't
-# support `jhash` in numgen yet (lands in 1.0.7+).  `inc` is round-robin
-# per-rule-evaluation, but conntrack pins the chosen DNAT translation for
-# the lifetime of the TCP flow — so each individual TCP connection sees
-# exactly one worker from SYN to FIN.  Re-balancing happens only between
-# connections, which is exactly what we want.
-#
-# To apply at boot (the postinst installs this file next to the single-
-# worker drop-in ; the operator picks which is loaded by nftables.service
-# via a symlink at /etc/nftables.d/secubox-toolbox-wg.nft).
-
-flush chain inet wg-toolbox prerouting
-
-table inet wg-toolbox {
-    chain prerouting {
-        type nat hook prerouting priority dstnat; policy accept;
-
-        # Phase 9 (#501) — 4-worker round-robin DNAT.  numgen returns
-        # 0..3 ; the map sends each to one of the 4 worker ports on
-        # 10.99.1.1.  Conntrack pins the choice for the whole flow.
-        iif "wg-toolbox" tcp dport 443 dnat ip to 10.99.1.1 \
-            : numgen inc mod 4 map {
-                0 : 8081,
-                1 : 8082,
-                2 : 8083,
-                3 : 8084
-            }
-        iif "wg-toolbox" tcp dport 80 dnat ip to 10.99.1.1 \
-            : numgen inc mod 4 map {
-                0 : 8081,
-                1 : 8082,
-                2 : 8083,
-                3 : 8084
-            }
-
-        # Phase 7 (#498) — DNS DNAT for legacy peer configs that hand out
-        # DNS = 10.99.0.1.  Single target — these queries are tiny and
-        # don't need worker fanout.
-        iif "wg-toolbox" ip daddr 10.99.0.1 udp dport 53 dnat ip to 10.99.1.1:53
-        iif "wg-toolbox" ip daddr 10.99.0.1 tcp dport 53 dnat ip to 10.99.1.1:53
-
-        # Phase 7 (#498) — captive-portal HTTP probe from the R3
-        # verification page.
-        iif "wg-toolbox" ip daddr 10.99.0.1 tcp dport 8088 dnat ip to 10.99.1.1:8088
-    }
-}

packages/secubox-toolbox/sbin/secubox-toolbox-mitm-wg-launch

@@ -45,15 +45,11 @@ fi
 # Phase 7 (#498) — listen-host is overridable via env. Host (default) binds
 # 10.99.1.1 (the wg-toolbox interface IP) ; LXC variant sets 0.0.0.0 so it
 # accepts the DNAT'd traffic on the 10.100.0.62 br-lxc interface.
-# Phase 9 (#501) — listen-port is overridable too. Each fanout worker
-# instance (secubox-toolbox-mitm-wg-worker@N) sets MITM_WG_LISTEN_PORT
-# to 808N. The legacy single-process service keeps the 8081 default.
 MITM_WG_LISTEN_HOST="${MITM_WG_LISTEN_HOST:-10.99.1.1}"
-MITM_WG_LISTEN_PORT="${MITM_WG_LISTEN_PORT:-8081}"
 ARGS=(
     --mode transparent
     --listen-host "$MITM_WG_LISTEN_HOST"
-    --listen-port "$MITM_WG_LISTEN_PORT"
+    --listen-port 8081
     --set confdir=/etc/secubox/toolbox/ca-wg
     --set ssl_insecure=false
     --set web_open_browser=false

packages/secubox-toolbox/secubox_toolbox/utiq.py

@@ -79,15 +79,17 @@ def _publisher_from_host(host: str) -> str:
     return h or "unknown"
 
 
-# Phase 8.B perf (#500) — fire-and-forget SQLite writes via single
+def record_event(
-# background thread (matches local_store.py pattern). Mitmproxy's
+    *,
-# asyncio event loop never blocks on _conn() open + INSERT + fsync.
+    client_ip: Optional[str],
-import concurrent.futures as _futures
+    host: str,
-_executor = _futures.ThreadPoolExecutor(max_workers=1, thread_name_prefix="sbx_utiq_write")
+    path: Optional[str],
-
+    action: str,
-
+    level: str,
-def _record_sync(client_ip, host, path, action, level,
+    detected_mtid: Optional[str] = None,
-                 detected_mtid, injected_mtid) -> None:
+    injected_mtid: Optional[str] = None,
+) -> None:
+    """Insert one event.  Best-effort — never raises into the addon."""
     try:
         with _conn() as c:
             c.execute(
@@ -110,26 +112,6 @@ def _record_sync(client_ip, host, path, action, level,
         log.warning("record_event failed: %s", e)
 
 
-def record_event(
-    *,
-    client_ip: Optional[str],
-    host: str,
-    path: Optional[str],
-    action: str,
-    level: str,
-    detected_mtid: Optional[str] = None,
-    injected_mtid: Optional[str] = None,
-) -> None:
-    """Insert one event off-thread. Best-effort — never raises into
-    the addon, never blocks the mitmproxy asyncio loop."""
-    try:
-        _executor.submit(_record_sync, client_ip, host, path, action,
-                         level, detected_mtid, injected_mtid)
-    except RuntimeError:
-        # Executor shut down (interpreter teardown) — silent drop.
-        pass
-
-
 def recent(hours: int = 24, limit: int = 200) -> List[Dict]:
     """Return the last events within the window, newest first."""
     since = int(time.time()) - hours * 3600

packages/secubox-toolbox/systemd/secubox-toolbox-mitm-wg-worker@.service

@@ -1,72 +0,0 @@
-# SPDX-License-Identifier: LicenseRef-CMSD-1.0
-# Phase 9 (#501) — multi-worker fanout for the R3 wg tunnel mitm.
-#
-# Why : on gk2 the single-process mitm-wg saturates one ARM core at
-# ~90 % under just 2-3 concurrently-active wg peers.  The Python GIL
-# caps real parallelism inside a single mitmproxy process.  Phase 9
-# runs N=4 worker instances (8081..8084) and lets nft DNAT spread
-# new TCP connections evenly across them via `numgen inc mod 4`,
-# which is sticky-per-connection (the conntrack entry locks the
-# translation for the lifetime of the flow).
-#
-# Each %i ∈ {1..4} → listen on 808%i .  Activate with :
-#
-#   systemctl enable --now secubox-toolbox-mitm-wg-worker@{1,2,3,4}.service
-#   nft -f /etc/nftables.d/secubox-toolbox-wg-fanout.nft
-#   systemctl disable --now secubox-toolbox-mitm-wg.service    # retire single
-#
-# Rollback (single-process) :
-#
-#   systemctl disable --now secubox-toolbox-mitm-wg-worker@{1,2,3,4}.service
-#   nft -f /usr/share/secubox/toolbox/nftables.d/secubox-toolbox-wg.nft
-#   systemctl enable --now secubox-toolbox-mitm-wg.service
-#
-# State coherence : all 4 workers share /var/lib/secubox/toolbox/toolbox.db
-# (WAL mode, multi-writer-safe).  Cert-pin auto-learning's dynamic
-# bypass file is the one source of contention left (4 writers race on
-# /var/lib/secubox/toolbox/mitm-bypass-dynamic.conf) ; the .path
-# watcher already de-bounces 10 s before reload-restart so the worst
-# case is a duplicate line added then deduped by the launcher's
-# sort -u pipeline.  Acceptable for Phase 9 ship ; a real filelock
-# lands in 9.1.
-
-[Unit]
-Description=SecuBox ToolBoX MITM WireGuard worker %i (R3 fanout port 808%i)
-After=network.target wg-quick@wg-toolbox.service
-Wants=wg-quick@wg-toolbox.service
-Documentation=https://github.com/CyberMind-FR/secubox-deb/issues/501
-
-[Service]
-Type=simple
-User=secubox-toolbox
-Group=secubox-toolbox
-WorkingDirectory=/usr/lib/secubox/toolbox
-
-# Phase 9 — per-instance port.  systemd's %i is the instance number.
-Environment="MITM_WG_LISTEN_HOST=10.99.1.1"
-Environment="MITM_WG_LISTEN_PORT=808%i"
-
-ExecStart=/usr/sbin/secubox-toolbox-mitm-wg-launch
-
-Restart=on-failure
-RestartSec=5
-
-# Same hygiene cycle as the single-process unit.  3 h recycle per
-# worker, staggered by 45 min via RuntimeMaxSec randomization
-# (RandomizedDelaySec on the timer would be cleaner ; here we just
-# accept that 4 workers will all recycle at boot+3h with brief 5 s
-# downtime each, mitigated by the others still serving traffic).
-RuntimeMaxSec=3h
-
-# Memory envelope per worker — 4x the single-process budget split
-# evenly is 100 MB each, but real-world per-worker RSS sits at
-# ~60-80 MB so MemoryMax=128M gives a sane upper bound.
-MemoryHigh=100M
-MemoryMax=128M
-
-# Resource isolation between workers.  Without it, one runaway
-# worker can drag the others.
-TasksMax=128
-
-[Install]
-WantedBy=multi-user.target