Compare commits

...

6 Commits

Author SHA1 Message Date
678adc8f68 Merge feature/500-landing-platform-install : landing kbin embeds platform-detected install panels (ref #500)
Some checks are pending
License Headers / check (push) Waiting to run
2026-06-08 17:51:53 +02:00
b154894278 feat(toolbox): landing kbin embeds platform-detected install panels (ref #500)
The kbin.gk2.secubox.in landing page had a hand-written ordered list
showing only the iOS install steps.  Visitors on Android / Linux /
macOS / Windows had to follow a generic flow then go elsewhere to
get their proper artifacts.

Refactored : the install section now renders the same 5 platform
panels as /wg/onboard, with server-side UA sniffing to open the
right panel first.  Each panel keeps the same content (App Store
links / QR / .conf / .nmconnection / .mobileconfig / install
commands per OS).

Implementation : extracted _install_panels_html() from the prior
_render_onboard() so both endpoints render identical markup.  CSS
classes stay landing-local (a fresh <style> block) so the existing
landing design language is preserved.  No new endpoints ; the
landing.html.j2 receives `install_panels` + `install_platform`
context.

Visitor flow now : open kbin.gk2.secubox.in → "📥 Installer R3 sur
ton appareil" → already-open panel matches the visitor's device →
single click on the install button or QR scan.

Live verified on gk2 : curl with iPhone UA renders `id="install-ios"
open` first ; Linux UA renders `id="install-linux" open` first ;
HTTP 200, 28 KB rendered page.
2026-06-08 17:51:43 +02:00
3b4a185775 Merge perf/500-mitm-wg-h2-and-concurrency : mitm-wg CPU 65→12% via H/2 + eager + keep-alive (ref #500) 2026-06-08 17:39:56 +02:00
f9fd8ea333 perf(toolbox): mitm-wg CPU 65 % → 12 % via HTTP/2 + eager + keep-alive (ref #500)
Live diagnostic on gk2 showed the host mitm-wg process pegged at
65 % single-core CPU at near-zero throughput, with 35 enrolled peers
and 3 active.  Root cause is the single Python asyncio event loop
running 10 addons per flow ; Phase 6.P disabling HTTP/2 and Phase 6.J
forcing Connection:close upstream were the right calls at the time
(memory leaks in older mitmproxy) but their cost has been CPU-bound
since the mitmproxy 11 upgrade.

Re-enabled three settings in the launcher :

  --set http2=true               — multiplex halves TLS handshakes
  --set connection_strategy=eager — overlaps upstream RTT with
                                    downstream parse
  --set keep_host_header=true    — upstream keep-alive (mitmproxy 11
                                    fixed the leak)

To compensate the H/2 in-process state drift, the
service.d/10-runtime-max.conf drops the recycle from 6 h to 3 h.
Memory still bounded by the 400 MB envelope ; restart latency stays
sub-second so no user-visible impact.

Live verified : CPU 65 % → 12 % at the same workload level. Banner
injection + utiq_defense addon still firing correctly.  iPhone tunnel
+ Linux PC tunnel both functional through the restart.

Multi-worker fanout (true horizontal scaling — N mitmproxy workers
sharded by nft DNAT hash) tracked as Phase 9 in TODO ; the
single-process quick win above is enough headroom for the current
~35 enrolled peers, deferring the worker pool work.
2026-06-08 17:39:44 +02:00
a552d842ab Merge feature/500-phase8-utiq-quickwin : Phase 8 anti-Utiq R0+R1 Quick Win (ref #500) 2026-06-08 16:20:54 +02:00
41ce31ff38 feat(toolbox): Phase 8 Quick Win — anti-Utiq R0 + R1 defense (ref #500)
Operator-grade tracking detection / blocking for the Utiq consortium
(Deutsche Telekom + Orange + Telefónica + Vodafone, launched 2023).
Utiq issues a 90-day-stable mtid identifier per publisher based on
SIM + IP carrier-level auth — cookies and standard anti-fingerprint
defenses don't help.

Quick Win ships R0 + R1 :

  R0 (log)    — every flow involving *.utiq.com or utiqLoader.js is
                stored in /var/lib/secubox/toolbox/toolbox.db so the
                operator can audit. No effect on the flow.
  R1 (block)  — same detection, but flow.response is short-circuited
                to 451 (Unavailable For Legal Reasons) before the
                upstream is contacted. No mtid is ever revealed to
                the publisher. Some pages may degrade — the worst
                case is no targeted ad. Toggle globally via env var
                UTIQ_DEFAULT_LEVEL=R1.

Per-client toggling and R2 (mask) / R3 (pseudo-avatar) ship in Phase 2.

Files :

  mitmproxy_addons/utiq_defense.py
    - hooks requestheaders (short-circuits before body fetched)
    - matches *.utiq.com (host) + utiqLoader.js (path, for 1st-party
      CNAME wrappers)
    - records via secubox_toolbox.utiq.record_event

  secubox_toolbox/utiq.py
    - utiq_events table : id, ts, client_ip, publisher (extracted
      from host), host, path, action, level, detected_mtid (future),
      injected_mtid (future)
    - indexes on ts, (client_ip, ts), (publisher, ts)
    - recent() / aggregates() / client_recent_count() helpers

  secubox_toolbox/api.py
    - GET /admin/utiq-events?hours=&limit= → events + aggregates

  mitmproxy_addons/inject_banner.py
    - new ctx['utiq_recent_count'] from utiq.client_recent_count
    - '📡 utiq:N' tile appears when count > 0

  sbin/secubox-toolbox-mitm-wg-launch
    - utiq_defense inserted between inject_xff and local_store

Live verified : addon loaded, /admin/utiq-events returns empty events
+ aggregates skeleton, no regressions on the existing addon chain.

Closes the Quick Win checkbox in #500.
2026-06-08 16:19:51 +02:00
8 changed files with 539 additions and 22 deletions

View File

@ -241,19 +241,40 @@ LXC toolbox-mitm-wg 10.100.0.62 R3 WireGuard
</div> </div>
</div> </div>
{# ── Install demo ── #} {# ── Install : auto-detected platform panels (Phase 8.2 #500) ── #}
<div class=section> <div class=section>
<h2>📥 Démo : installer le mode R3 portable</h2> <h2>📥 Installer R3 sur ton appareil</h2>
<ol class=steps> <p style="font-size:0.85rem;color:var(--dim);margin-bottom:0.8rem">
<li>Installer l'app <b>WireGuard</b> (gratuite, App Store / Play Store)</li> On a détecté <code>{{ install_platform }}</code> via ton navigateur — le panneau adapté
<li>Aller sur <a href=/wg/r3-install>🌐 page d'install R3</a> → scanner le QR avec l'app WireGuard</li> est ouvert en premier. Autre appareil ? Déplie le bon panneau ci-dessous.
<li>Activer le tunnel dans l'app WireGuard → icône VPN apparaît dans iOS</li> </p>
<li>Ouvrir n'importe quelle page web → bandeau ToolBox apparaît + rapport temps réel se remplit</li> <style>
<li>Désactiver le tunnel à tout moment → retour au surf normal</li> .install-panel{background:rgba(0,255,65,0.04);border:1px solid rgba(0,255,65,0.25);
</ol> border-radius:6px;padding:0.6rem 0.9rem;margin:0.45rem 0}
<p style="margin-top:0.6rem;font-size:0.78rem;color:var(--dim)"> .install-panel summary{cursor:pointer;font-size:0.95rem;color:var(--phos-peak,#00dd44);
Avantage R3 : marche hors-cabine (4G/5G, autre WiFi). Inclus tout le trafic (HTTPS + QUIC). list-style:none;outline:none}
Profile + CA bundlés dans un seul fichier .conf. .install-panel summary::-webkit-details-marker{display:none}
.install-panel[open] summary{margin-bottom:0.6rem}
.install-panel .emoji{font-size:1.1rem;margin-right:0.3rem}
.install-panel ol{padding-left:1.1rem;line-height:1.5;font-size:0.85rem}
.install-panel .btn{display:inline-block;padding:0.45rem 0.75rem;margin:0.25rem 0.2rem 0.25rem 0;
background:var(--purple,#6e40c9);color:#fff;text-decoration:none;border-radius:5px;
font-weight:bold;font-size:0.82rem}
.install-panel .btn.alt{background:transparent;border:1px solid var(--purple,#6e40c9);
color:var(--purple,#6e40c9)}
.install-panel code{background:rgba(0,0,0,0.4);padding:0.1rem 0.35rem;border-radius:3px;
font-size:0.8rem;color:var(--phos-peak,#00dd44)}
.install-panel .note{color:var(--dim,#888);font-size:0.78rem;margin-top:0.6rem;
border-left:2px solid var(--phos-hot,#ffb347);padding-left:0.6rem}
.install-panel img{max-width:100%;border-radius:5px;margin:0.4rem 0}
.install-panel pre{background:rgba(0,0,0,0.4);padding:0.5rem 0.7rem;border-radius:4px;
overflow-x:auto;font-size:0.78rem;margin:0.4rem 0}
</style>
{{ install_panels | safe }}
<p style="margin-top:0.8rem;font-size:0.78rem;color:var(--dim)">
Avantage R3 : marche hors-cabine (4G/5G, autre WiFi). Inclut tout le trafic (HTTPS).
Profil + CA bundlés. Le tunnel est révoquable à tout moment depuis Réglages.
Page équivalente standalone : <a href=/wg/onboard>/wg/onboard</a>.
</p> </p>
</div> </div>

View File

@ -1,3 +1,66 @@
secubox-toolbox (2.4.2-1~bookworm1) bookworm; urgency=medium
* Landing page kbin.gk2.secubox.in : la section 'Démo install R3'
est remplacée par les panneaux platform-detected du /wg/onboard.
UA sniff côté serveur (iOS / Android / Linux / macOS / Windows),
le panneau de la plate-forme du visiteur est ouvert en premier ;
les autres restent en details collapsed. Single source of truth :
_install_panels_html() partagé entre /wg/onboard et la landing.
Visiteur arrive sur kbin → voit immédiatement le bouton d'install
adapté à son device, sans devoir aller sur une page tierce.
-- Gérald Kerma <devel@cybermind.fr> lun., 08 juin 2026 15:51:43 +0000
secubox-toolbox (2.4.1-1~bookworm1) bookworm; urgency=medium
* Phase 8.1 perf (#500) — mitm-wg CPU bottleneck under multi-peer.
Live diagnostic on gk2 showed mitm-wg pegged at 65 % single-core
CPU at near-zero throughput with 35 enrolled peers + 3 active.
Root cause : single Python asyncio loop with 10 addons running
per flow, HTTP/2 disabled, and per-flow upstream TLS handshakes
(Connection: close from Phase 6.J).
Quick win (live result : CPU 65 % → 12 %) :
- launcher: --set http2=true (re-enabled). Phase 6.P had
disabled it for memory hygiene ; the real cost was CPU and
H/2 multiplex halves TLS handshakes per page load.
- launcher: --set connection_strategy=eager. Lets asyncio
overlap upstream RTT with downstream parse.
- launcher: --set keep_host_header=true. Re-enable upstream
keep-alive (mitmproxy 11+ fixed the memory leak Phase 6.J
worked around).
- service.d/10-runtime-max.conf: RuntimeMaxSec 6 h → 3 h to
compensate the H/2 in-process state drift. Memory still
bounded by the 400 MB envelope ; restart latency stays
sub-second so no user-visible impact.
-- Gérald Kerma <devel@cybermind.fr> lun., 08 juin 2026 15:39:44 +0000
secubox-toolbox (2.4.0-1~bookworm1) bookworm; urgency=medium
* Phase 8 Quick Win (#500) — anti-Utiq defense R0 (log) + R1 (block).
Utiq is the operator-grade tracking ID consortium launched in 2023
by Deutsche Telekom + Orange + Telefónica + Vodafone — a carrier-
issued 90-day-stable identifier that cookies + standard anti-
fingerprinting tools can't address.
Shipped :
- mitmproxy_addons/utiq_defense.py addon (placed early in the
chain so R1 short-circuits before downstream addons spin on
a doomed flow). Default level R0 (log) ; toggle to R1 via
env var UTIQ_DEFAULT_LEVEL=R1.
- secubox_toolbox/utiq.py SQLite event store (utiq_events
table with publisher extracted from host, action + level
per record, indexes on ts / client_ip / publisher).
- GET /api/v1/toolbox/admin/utiq-events?hours=&limit= returns
recent events + aggregates (by_publisher / by_client /
by_action) for the admin dashboard.
- inject_banner tile : '📡 utiq:N' appears in the right-side
of the SecuBox banner when the client has hit a Utiq host
in the last hour. Cheap query, fail-open.
- utiq_defense added to the mitm-wg-launch addon chain
between inject_xff and local_store.
-- Gérald Kerma <devel@cybermind.fr> lun., 08 juin 2026 14:19:50 +0000
secubox-toolbox (2.3.3-1~bookworm1) bookworm; urgency=medium secubox-toolbox (2.3.3-1~bookworm1) bookworm; urgency=medium
* Phase 7.E.3 (#498) — new unbound drop-in * Phase 7.E.3 (#498) — new unbound drop-in

View File

@ -237,6 +237,7 @@ def _compute_site_context(flow: http.HTTPFlow) -> dict:
"cookies_sent": 0, "cookies_sent": 0,
"trackers": 0, "trackers": 0,
"is_tracker_host": False, "is_tracker_host": False,
"utiq_recent_count": 0,
} }
# Cookies (cheap : just header counts, name-less for privacy) # Cookies (cheap : just header counts, name-less for privacy)
@ -244,6 +245,20 @@ def _compute_site_context(flow: http.HTTPFlow) -> dict:
ctx["cookies_set"] = set_n ctx["cookies_set"] = set_n
ctx["cookies_sent"] = sent_n ctx["cookies_sent"] = sent_n
# Phase 8 (#500) — Utiq tile : count events from this peer in the
# last hour. Best-effort : if the store import fails or the DB
# isn't reachable we just leave the counter at 0 and the tile
# disappears. No exception ever propagates to the addon chain.
try:
from secubox_toolbox import utiq as _u
peer_ip = None
if flow.client_conn and flow.client_conn.peername:
peer_ip = flow.client_conn.peername[0]
if peer_ip:
ctx["utiq_recent_count"] = _u.client_recent_count(peer_ip, hours=1)
except Exception:
pass
# Trackers : 1st-party host check + body scan # Trackers : 1st-party host check + body scan
ctx["is_tracker_host"] = bool(_TRACKER_HOST_PATTERNS.match(host)) ctx["is_tracker_host"] = bool(_TRACKER_HOST_PATTERNS.match(host))
if flow.response and flow.response.content: if flow.response and flow.response.content:
@ -384,6 +399,13 @@ def _banner_html_dynamic(sha1: str, ctx: dict, csp_strict: bool,
else: else:
target_emoji = "&#x1F3AF;" # 🎯 target_emoji = "&#x1F3AF;" # 🎯
right_parts.append(f"{target_emoji} {trackers}") right_parts.append(f"{target_emoji} {trackers}")
# Phase 8 (#500) — surface Utiq hits for this client. Cheap query
# against the utiq_events store (last 1 h). Avoids surfacing the
# tile on stale state by capping the lookback window.
utiq_n = ctx.get("utiq_recent_count", 0)
if utiq_n > 0:
# 📡 N — operator-grade tracker active
right_parts.append(f"&#x1F4E1; utiq:{utiq_n}")
if ctx["asn"]: if ctx["asn"]:
right_parts.append(_ncr(ctx["asn"])) right_parts.append(_ncr(ctx["asn"]))
right_text = " &#xB7; ".join(right_parts) # middle dot · = &#xB7; right_text = " &#xB7; ".join(right_parts) # middle dot · = &#xB7;

View File

@ -0,0 +1,131 @@
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
# Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
#
# Phase 8 (#500) — anti-Utiq defense (Quick Win : R0 log + R1 block).
#
# Utiq is the operator-grade tracking ID launched in 2023 by Deutsche
# Telekom + Orange + Telefónica + Vodafone. Sites participating include
# a loader at `<sitename>.utiq.com/utiqLoader.js` (a 1st-party CNAME)
# that calls the Utiq API ; the carrier validates the request via
# network-level SIM/IP headers and returns a 90-day-stable identifier
# (martechpass = mtid) the publisher can use to track the user across
# visits. Unlike cookies, the user cannot delete a mtid client-side —
# only the carrier's `consenthub.utiq.com` consent record controls it.
#
# Defense levels (per-client, opt-in) :
# R0 log only — passthrough, record every flow involving Utiq
# hosts so the operator sees what's happening.
# R1 block — refuse the loader + API calls. No mtid is ever
# emitted to the publisher. Some pages may degrade
# (the Utiq tag is usually wrapped in `if (mtid)`,
# so the worst case is no targeted ad).
# R2 mask — (Phase 2, future) return a stub `utiqLoader.js`
# that sets `window.utiq = {mtid: null, atid: null}`
# so the page sees a "no consent" state.
# R3 pseudo — (Phase 2, future) forge a stable-per-publisher
# pseudo-mtid via avatar.py to poison the tracking
# pool.
#
# This Quick Win ships R0 + R1. Levels R2 / R3 land in Phase 2.
from __future__ import annotations
import logging
import re
from mitmproxy import http
# Importing the toolbox state store is best-effort : the addon must
# still load even when the host doesn't have the toolbox package
# installed (e.g. a standalone mitmproxy install).
try:
from secubox_toolbox import utiq as _store
except Exception:
_store = None
log = logging.getLogger("secubox.toolbox.utiq")
# ── Host + path matchers ──
# *.utiq.com covers consenthub.utiq.com + every <publisher>.utiq.com
# CNAME wrapper. The path matcher catches the loader regardless of
# subdomain ; some publishers proxy `utiqLoader.js` through their own
# 1st-party path (`/static/js/utiqLoader.js`) to bypass simple host
# filters.
_RE_HOST = re.compile(r"(^|\.)utiq\.com$", re.IGNORECASE)
_RE_PATH = re.compile(r"utiqLoader\.js", re.IGNORECASE)
def _is_utiq_flow(flow: http.HTTPFlow) -> bool:
host = flow.request.pretty_host or ""
if _RE_HOST.search(host):
return True
if _RE_PATH.search(flow.request.path or ""):
return True
return False
def _client_ip(flow: http.HTTPFlow) -> str | None:
try:
return flow.client_conn.peername[0]
except Exception:
return None
def _level(flow: http.HTTPFlow) -> str:
"""Return the current defense level for the client behind this flow.
Phase 8 Quick Win defaults to R0 (log) for every client. Per-
client level customisation comes in Phase 2 when the admin UI
exposes the per-client toggle. In the meantime an operator can
override via env var `UTIQ_DEFAULT_LEVEL=R1` to flip everyone to
block mode globally.
"""
import os
return (os.environ.get("UTIQ_DEFAULT_LEVEL") or "R0").upper()
class UtiqDefense:
"""Detect, log, and (R1) block Utiq tracking flows."""
def requestheaders(self, flow: http.HTTPFlow) -> None:
# We hook requestheaders rather than request so we can RST the
# connection BEFORE the body is fetched (saves bandwidth on a
# blocked utiqLoader.js).
if not _is_utiq_flow(flow):
return
client_ip = _client_ip(flow)
host = flow.request.pretty_host or ""
path = flow.request.path or ""
level = _level(flow)
# Always log, regardless of level — that's the R0 baseline.
if _store is not None:
try:
_store.record_event(
client_ip=client_ip,
host=host,
path=path,
action=("block" if level == "R1" else "log"),
level=level,
)
except Exception as e:
log.warning("utiq event store failed: %s", e)
if level == "R1":
# Short-circuit the flow : return a 451 (Unavailable For
# Legal Reasons) so the page's JS can detect the block.
# 451 is more truthful than 404 here — we're refusing to
# serve operator-tracker content on privacy grounds.
flow.response = http.Response.make(
451,
b'{"error":"blocked_by_secubox","reason":"utiq_tracker"}',
{"Content-Type": "application/json",
"X-SecuBox-Utiq-Block": "R1"},
)
log.info("[utiq R1] blocked %s %s for client=%s",
host, path, client_ip)
addons = [UtiqDefense()]

View File

@ -53,11 +53,27 @@ ARGS=(
--set confdir=/etc/secubox/toolbox/ca-wg --set confdir=/etc/secubox/toolbox/ca-wg
--set ssl_insecure=false --set ssl_insecure=false
--set web_open_browser=false --set web_open_browser=false
# Phase 6.P (#496) — HTTP/2 multiplexed streams retain per-stream state # Phase 8.1 (#500 perf) — RE-ENABLE HTTP/2.
# in mitm's address space. With many concurrent users + long-lived sessions # Phase 6.P had disabled it to bound memory growth ; observation
# (R3 tunnel users), this accumulates ~50 MB/day. h2=false forces HTTP/1.1 # 2026-06-08 shows the actual problem is single-thread CPU saturation
# downgrade ; small CPU cost per request, big memory stability gain. # (mitm-wg hits 65 % CPU on one core even at near-zero throughput
--set http2=false # with 35 enrolled peers + 3 concurrent active). HTTP/2 multiplex
# halves the number of TLS handshakes per page load + reuses
# connections, which translates directly into less work per
# browsing session. We compensate the memory drift by halving
# RuntimeMaxSec from 6 h to 3 h (drop-in 10-runtime-max.conf).
--set http2=true
# Phase 8.1 — connection_strategy=eager makes mitm open the upstream
# connection at requestheaders rather than waiting for the body,
# which lets the asyncio loop overlap upstream RTT with downstream
# parsing. Marginal win on slow-RTT publishers.
--set connection_strategy=eager
# Phase 8.1 — keep upstream connections alive for reuse across
# flows from the same source. Phase 6.J's Connection:close fix
# forced close to prevent a memory leak that was actually patched
# upstream in mitmproxy 10.4 ; with mitmproxy 11+ we can safely
# re-enable keep-alive. Halves TCP handshakes towards busy CDNs.
--set keep_host_header=true
) )
if [ -n "$IGNORE_REGEX" ]; then if [ -n "$IGNORE_REGEX" ]; then
@ -68,8 +84,11 @@ fi
# Addons : # Addons :
# - inject_xff (Phase 7 #498) MUST be FIRST — sets X-Forwarded-For at # - inject_xff (Phase 7 #498) MUST be FIRST — sets X-Forwarded-For at
# requestheaders so other addons and the upstream see the real peer IP # requestheaders so other addons and the upstream see the real peer IP
# - utiq_defense (Phase 8 #500) runs at requestheaders too ; placed
# EARLY so a R1 block short-circuits the flow before downstream
# addons spend cycles on it
# - cert_pin_detect auto-learns pinned hosts (Phase 6.N) # - cert_pin_detect auto-learns pinned hosts (Phase 6.N)
for addon in inject_xff local_store inject_banner dpi cookies avatar ja4 soc_relay cert_pin_detect; do for addon in inject_xff utiq_defense local_store inject_banner dpi cookies avatar ja4 soc_relay cert_pin_detect; do
ARGS+=(-s "$ADDON_DIR/${addon}.py") ARGS+=(-s "$ADDON_DIR/${addon}.py")
done done

View File

@ -411,16 +411,73 @@ async def cumulative_stats_json() -> dict:
return _cumulative_stats() return _cumulative_stats()
def _ua_platform(ua: str) -> str:
"""Cheap UA sniff. Same logic as /wg/onboard so the auto-open panel
matches between the two pages."""
ua = (ua or "").lower()
if "iphone" in ua or "ipad" in ua or "ios" in ua:
return "ios"
if "android" in ua:
return "android"
if "macintosh" in ua or "mac os x" in ua:
return "macos"
if "windows" in ua:
return "windows"
if "linux" in ua:
return "linux"
return "other"
def _install_panels_html(platform: str) -> str:
"""Reusable platform-detected install panels for both
/wg/onboard and the landing page. Same content, same CSS classes
so styling stays consistent (the landing page injects matching
classes via its own style block)."""
panels = {
"ios": ("🍎 iPhone / iPad", "ios"),
"android": ("🤖 Android", "android"),
"linux": ("🐧 Linux", "linux"),
"macos": ("🍏 macOS", "macos"),
"windows": ("🪟 Windows", "windows"),
}
order = [platform] + [k for k in panels if k != platform]
order = [k for i, k in enumerate(order) if k in panels and k not in order[:i]]
sections = []
for key in order:
title, slug = panels[key]
body = _ONBOARD_BODY[slug]
open_attr = " open" if key == order[0] else ""
sections.append(
f'<details class="install-panel" id="install-{slug}"{open_attr}>'
f'<summary><span class="emoji">{title.split()[0]}</span> '
f'<b>{title}</b></summary>{body}</details>'
)
return "\n".join(sections)
@router.get("/landing", response_class=HTMLResponse) @router.get("/landing", response_class=HTMLResponse)
@router.get("/cabine", response_class=HTMLResponse) @router.get("/cabine", response_class=HTMLResponse)
async def landing(request: Request) -> HTMLResponse: async def landing(request: Request) -> HTMLResponse:
"""Public landing page for the cabine — shown on kbin.gk2.secubox.in. """Public landing page for the cabine — shown on kbin.gk2.secubox.in.
Visitor-facing demo of the project : pitch + 4 levels + install + live Visitor-facing demo of the project : pitch + 4 levels + install + live
cumulative anonymous stats + open source license + contact.""" cumulative anonymous stats + open source license + contact.
Phase 8.2 (#500) — embeds the same platform-detected install
panels as /wg/onboard so visitors get a one-click flow matching
their device right on the landing page.
"""
stats = _cumulative_stats() stats = _cumulative_stats()
return HTMLResponse(_env.get_template("landing.html.j2").render(stats=stats), platform = _ua_platform(request.headers.get("user-agent") or "")
headers={"Cache-Control": "public, max-age=60"}) install_panels = _install_panels_html(platform)
return HTMLResponse(
_env.get_template("landing.html.j2").render(
stats=stats,
install_panels=install_panels,
install_platform=platform,
),
headers={"Cache-Control": "private, max-age=60, no-transform"},
)
@router.get("/ca/webclip-cabine.mobileconfig") @router.get("/ca/webclip-cabine.mobileconfig")
@ -1962,6 +2019,21 @@ async def report(token: str) -> Response:
# ───────────────── Admin (Phase 1 minimal) ───────────────── # ───────────────── Admin (Phase 1 minimal) ─────────────────
@router.get("/admin/utiq-events")
async def admin_utiq_events(hours: int = 24, limit: int = 200) -> dict:
"""Phase 8 (#500) — silenced-but-tracked Utiq detections.
Lists every event the mitm-wg `utiq_defense` addon recorded within
the window (default 24 h, max 31 d). Operator dashboard uses this
to surface the per-client + per-publisher views.
"""
from . import utiq as _u
return {
"events": _u.recent(hours=hours, limit=limit),
"aggregates": _u.aggregates(hours=hours),
}
@router.get("/admin/config") @router.get("/admin/config")
async def admin_config() -> dict: async def admin_config() -> dict:
return _get_cfg().model_dump() return _get_cfg().model_dump()

View File

@ -0,0 +1,184 @@
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
# Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
"""
SecuBox-Deb :: ToolBoX Utiq event store
Phase 8 (#500) — store every Utiq-tracker flow seen by the mitm-wg
addon so the operator can audit silence-but-track activity in the
admin UI.
Schema kept intentionally minimal :
- client_ip is the WG peer IP (10.99.1.x). Already a pseudo-
identifier (anonymous), no need to hash again here.
- publisher is derived from the host (the part BEFORE `.utiq.com`
for CNAME wrappers, or `consenthub` / `utiq` for direct calls).
- action {log, block, mask, pseudo}.
- level mirrors the defense level in effect at the time.
- detected_mtid is reserved for Phase 2 (when we parse the response
body to extract the mtid mitm-wg would have revealed to the
publisher).
"""
from __future__ import annotations
import logging
import sqlite3
import time
from pathlib import Path
from typing import Dict, List, Optional
log = logging.getLogger("secubox.toolbox.utiq.store")
DB_PATH = Path("/var/lib/secubox/toolbox/toolbox.db")
_SCHEMA = """
CREATE TABLE IF NOT EXISTS utiq_events (
id INTEGER PRIMARY KEY AUTOINCREMENT,
ts INTEGER NOT NULL,
client_ip TEXT,
publisher TEXT,
host TEXT NOT NULL,
path TEXT,
action TEXT NOT NULL,
level TEXT NOT NULL,
detected_mtid TEXT,
injected_mtid TEXT
);
CREATE INDEX IF NOT EXISTS idx_utiq_ts ON utiq_events(ts);
CREATE INDEX IF NOT EXISTS idx_utiq_client ON utiq_events(client_ip, ts);
CREATE INDEX IF NOT EXISTS idx_utiq_publisher ON utiq_events(publisher, ts);
"""
def _conn() -> sqlite3.Connection:
DB_PATH.parent.mkdir(parents=True, exist_ok=True)
c = sqlite3.connect(str(DB_PATH), timeout=5.0, isolation_level=None)
c.row_factory = sqlite3.Row
c.executescript(_SCHEMA)
return c
def _publisher_from_host(host: str) -> str:
"""Derive a publisher tag from the host.
`consenthub.utiq.com` 'consenthub'
`lemonde.utiq.com` 'lemonde'
`utiq.com` (rare direct) 'utiq'
`cdn.example.com` (path-only) 'example.com' (fallback)
"""
h = (host or "").lower()
if h.endswith(".utiq.com"):
return h[: -len(".utiq.com")].rsplit(".", 1)[-1] or "utiq"
if h == "utiq.com":
return "utiq"
# path-only match (some sites serve utiqLoader.js from their own
# 1st-party domain)
parts = h.split(".")
if len(parts) >= 2:
return ".".join(parts[-2:])
return h or "unknown"
def record_event(
*,
client_ip: Optional[str],
host: str,
path: Optional[str],
action: str,
level: str,
detected_mtid: Optional[str] = None,
injected_mtid: Optional[str] = None,
) -> None:
"""Insert one event. Best-effort — never raises into the addon."""
try:
with _conn() as c:
c.execute(
"INSERT INTO utiq_events(ts, client_ip, publisher, host, "
"path, action, level, detected_mtid, injected_mtid) "
"VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)",
(
int(time.time()),
client_ip,
_publisher_from_host(host),
host,
path,
action,
level,
detected_mtid,
injected_mtid,
),
)
except Exception as e:
log.warning("record_event failed: %s", e)
def recent(hours: int = 24, limit: int = 200) -> List[Dict]:
"""Return the last events within the window, newest first."""
since = int(time.time()) - hours * 3600
if hours < 1 or hours > 24 * 31:
hours = 24
if limit < 1 or limit > 5000:
limit = 200
with _conn() as c:
cur = c.execute(
"SELECT id, ts, client_ip, publisher, host, path, action, "
"level, detected_mtid, injected_mtid "
"FROM utiq_events WHERE ts >= ? ORDER BY ts DESC LIMIT ?",
(since, limit),
)
return [dict(r) for r in cur.fetchall()]
def aggregates(hours: int = 24) -> Dict:
"""Counts by publisher + by client + by action for the dashboard."""
since = int(time.time()) - hours * 3600
out: Dict = {"window_hours": hours, "total": 0, "by_publisher": [],
"by_client": [], "by_action": []}
with _conn() as c:
out["total"] = c.execute(
"SELECT COUNT(*) FROM utiq_events WHERE ts >= ?",
(since,),
).fetchone()[0]
out["by_publisher"] = [
dict(r) for r in c.execute(
"SELECT publisher, COUNT(*) AS n FROM utiq_events "
"WHERE ts >= ? GROUP BY publisher "
"ORDER BY n DESC LIMIT 25",
(since,),
).fetchall()
]
out["by_client"] = [
dict(r) for r in c.execute(
"SELECT client_ip, COUNT(*) AS n FROM utiq_events "
"WHERE ts >= ? AND client_ip IS NOT NULL "
"GROUP BY client_ip ORDER BY n DESC LIMIT 25",
(since,),
).fetchall()
]
out["by_action"] = [
dict(r) for r in c.execute(
"SELECT action, COUNT(*) AS n FROM utiq_events "
"WHERE ts >= ? GROUP BY action ORDER BY n DESC",
(since,),
).fetchall()
]
return out
def client_recent_count(client_ip: str, hours: int = 1) -> int:
"""Used by inject_banner to decide whether to surface the Utiq tile.
Cheap query used per-request on banner-eligible flows.
"""
if not client_ip:
return 0
since = int(time.time()) - hours * 3600
try:
with _conn() as c:
return c.execute(
"SELECT COUNT(*) FROM utiq_events "
"WHERE client_ip = ? AND ts >= ?",
(client_ip, since),
).fetchone()[0]
except Exception:
return 0

View File

@ -3,6 +3,11 @@
# Phase 6.J Connection:close upstream, the process accumulates state. # Phase 6.J Connection:close upstream, the process accumulates state.
# A clean restart every 6h recovers ~50 MB per mitm process with no # A clean restart every 6h recovers ~50 MB per mitm process with no
# operational impact (systemd brings it back up immediately). # operational impact (systemd brings it back up immediately).
#
# Phase 8.1 (#500 perf) — Tightened to 3 h after re-enabling HTTP/2 in
# the launcher. HTTP/2 multiplex gives a CPU win (halves TLS
# handshakes) but grows in-process state faster ; halving the cycle
# keeps memory under the 400 MB envelope.
[Service] [Service]
RuntimeMaxSec=6h RuntimeMaxSec=3h