Compare commits

..

18 Commits

Author SHA1 Message Date
66b608d760 docs: update WIP/HISTORY/TODO for Phase 11 complete + Phase 12.A/B + v2.13.15
Some checks are pending
License Headers / check (push) Waiting to run
- Phase 11 A/B/C + toolbox tabs (#513) + Phase 12.A CDN (#515) + 12.B
    anti-bot (#516) all merged via PR #517, secubox-toolbox 2.6.6, tag
    v2.13.15.
  - Round Eye: USB TX wedge diagnosed, needs physical Pi power-cycle.
  - Next: Phase 12.C operator-grade; 12.B bypass + 12.D noise gated
    behind doctrine.
2026-06-10 14:25:26 +02:00
CyberMind
3a0d60c588
Merge pull request #517 from CyberMind-FR/feature/516-phase-12-b-anti-bot-prove-you-re-human-c
Phase 11 (social mapping A/B/C) + Phase 12.A/B (CDN + anti-bot) + toolbox sub-tabs
2026-06-10 14:23:05 +02:00
8bf5fe4f9a fix(toolbox): Carto operator link redirects to absolute kbin URL (ref #516)
The Clients-tab 🕸️ Carto link 303'd to a relative /social/{token}, but
that route is only served on the kbin vhost — from admin.gk2 it hit the
aggregator's 'missing module' page. admin_client_social() now swaps the
leading admin. host label for kbin. and redirects to the absolute URL.

Live: 303 -> https://kbin.gk2.secubox.in/social/{token} -> 200. changelog 2.6.6.
2026-06-10 14:22:16 +02:00
9f850bef53 feat(toolbox): Phase 12.B — anti-bot detection + visible ring levels + per-client operator tools (ref #516)
Anti-bot / 'prove you're human' DETECTION (bypass stays gated behind doctrine):
  - social_graph.py detect_antibot(): reCAPTCHA/hCaptcha/Turnstile/Datadome/
    PerimeterX-HUMAN/Arkose/Kasada/Akamai-BotManager from URL+cookies+headers.
  - social.py: social_host_meta.antibot_vendor + social_antibot per-client
    challenge table; fetch_graph/aggregate/wipe_mac carry it.
  - social.js: cinnabar severe lens + spinning warning ring + 🤖 label +
    'challenged your humanity' alert banner; node-detail vendor.
  - operator social tab: anti-bot breakdown card.

Fix the ring-level visibility the user reported missing:
  - radial force now DOMINANT (strength 0.9) with weak charge/links, plus
    dashed ring guides (inner=sites, outer=trackers); assets cache-busted
    ?v=264b so the new layout actually loads.

Per-client operator tools in the Clients tab:
  - 🕸️ Carto link → GET /admin/clients/{mac}/social mints a token + opens
    that client's graph.
  - ↺ Reset (RAZ) → POST /admin/clients/{mac}/reset wipes social + events +
    consents + reports and zeroes score (store.reset_client + social.wipe_mac).

changelog 2.6.5. Live on gk2: both new tables created, detect_antibot matrix
green, aggregate by_antibot/antibot_clients present, reset endpoint 200, graph
nodes carry antibot_vendor, served assets contain the ring + cache-bust.
2026-06-10 14:22:16 +02:00
a93d949ede feat(toolbox): Phase 12.A — CDN detection + Round-Eye central-hotspot graph (ref #515)
First passive track of the #514 anti-human-detection platform.

  - social_graph.py: detect_cdn() classifies the edge network fronting
    each 3rd-party host from response headers (Cloudflare/Fastly/Akamai/
    CloudFront/Google/Vercel/Netlify/BunnyCDN/KeyCDN/Sucuri/Imperva +
    generic edge-cache). Host-stable, recorded off-thread. Also deduped
    a stray duplicate _ja4_hash left by the 11.C edit.
  - social.py: new social_host_meta table; fetch_graph LEFT JOINs
    cdn_vendor/cache_status onto nodes; aggregate adds by_cdn.
  - social.js: per-client graph re-centred on a Round-Eye hotspot —
    device = pulsing eye at centre, sites orbit on inner forceRadial
    ring, trackers push to outer ring (densest clusters = hot spots).
    Tracker nodes tinted by CDN vendor; node-detail shows CDN/cache.
  - social.css: eye halo/sclera/iris/pupil + breathing anim + cinnabar
    spokes.
  - index.html: operator social tab gains a CDN/edge breakdown card.
  - i18n: node_detail_cdn FR/EN. changelog 2.6.4.

Live on gk2: social_host_meta table created, addon loads detect_cdn,
detect_cdn matrix green (CF/Fastly/Akamai/CloudFront/none), end-to-end
read path proven (synthetic Cloudflare record -> graph node cdn_vendor
-> aggregate by_cdn). Populates from live traffic.
2026-06-10 14:22:16 +02:00
f945c393b8 feat(toolbox): Phase 11.C — consent probe + bilingual FR/EN evidence PDF (ref #508)
Completes Phase 11 (social mapping per device): A backend + B d3 view + C evidence/PDF.

  - social_graph.py: consent-platform probe (OneTrust/Didomi/Quantcast/
    Sourcepoint cookies + loader URLs) per (peer,site). Edges stamped
    consent_state none_seen|pre_consent|post_consent. pre_consent =
    tracker fired while site runs a CMP but no consent cookie yet
    (RGPD art. 6.1.a + 7).
  - social.py: consent_state column + GeoIP node columns + EU/EEA
    whitelist + GeoIP fold + evidence() (carried from 55626e51).
  - social_report.py: NEW bilingual FR/EN evidence PDF via fpdf2.
    Robust _mc() helper (set_x + epw) avoids fpdf 'not enough
    horizontal space'. Fact-only: cover + summary + pre-consent table
    + extra-EU table + RGPD article refs. Text fallback if no fpdf2.
  - api.py: GET /social/report/{token}.pdf (HMAC-token gated).
  - social_view.html.j2 + i18n + css: real PDF download button,
    active evidence-card summary.
  - changelog: 2.6.3-1~bookworm1.

Live on gk2: PDF 200 / 41870 bytes / valid PDF v1.3 via uvicorn + kbin.
Consent state machine verified (pre->post->none). Rebased on #513 so
the toolbox sub-tabs ship together.
2026-06-10 14:22:16 +02:00
d8bc4acd90 wip(toolbox): Phase 11.C backend schema + GeoIP fold + evidence helper (ref #508)
Checkpoint — backend evidence layer scaffolded.  Frontend wire + PDF
generator + addon consent probe land in follow-up commits.

  - social.py schema : social_edges.consent_state column ; social_nodes
    gains country_iso / asn_org / eu_inside / pre_consent_hits.
  - Idempotent _migrate_phase11c() ALTERs the pre-existing tables on
    2.6.0 → 2.6.x upgrades (no destructive recreate).
  - _EU_EEA_ISO whitelist (27 EU + 3 EFTA + UK adequacy) + is_eu_iso().
  - _geo_for() LRU-cached (4096) wrapper around the existing geo
    module ; fold time populates the GeoIP fields on every node row.
  - record_edge() accepts consent_state (default 'none_seen') ; fold
    accumulates pre_consent_hits into the per-node aggregate.
  - evidence(mac_hash) helper : returns the two legal-grade buckets
    (pre_consent + extra_eu) consumed by the PDF in the next commit.

Pivoting to admin tab routing per user request — Phase 11.C resumes
after that lands.
2026-06-10 14:22:16 +02:00
39ec84eaed feat(toolbox): WebUI sub-tab nav + remove redundant kbin /admin/ inline UI (ref #513)
Consolidates the two toolbox admin surfaces onto the canonical
admin.gk2.secubox.in/toolbox/ WebUI.

  - www/toolbox/index.html: 5-tab nav (Vue d'ensemble / Clients /
    Filtres MITM / Cartographie sociale / Config). Lazy-load per tab,
    live refresh only polls the visible tab, URL-hash deep-links.
    Clients tab folds in the R0-R3 level switcher from the old kbin
    admin UI. New Cartographie sociale tab surfaces the Phase 11
    operator aggregate (KPI + top trackers + anonymized clients).
  - api.py: DELETED the inline kbin /admin/ HTML route (admin_index,
    ~230 lines). All /admin/* JSON API routes preserved.
  - conf/landing.html.j2: removed the Admin quicknav icon.
  - debian/changelog: 2.6.1 -> 2.6.2.

Live on gk2: /toolbox/ 200 (19853 bytes, 5 tabs), kbin /admin/ now 404
by design, social-aggregate shows 225 trackers / 3 clients, all tab
endpoints (clients/rich, filter-control/list, social-aggregate) 200.
2026-06-10 14:22:16 +02:00
78ed66c7cd fix(toolbox): force layout pre-warm + data-based autoFit (ref #507)
Live test screenshot showed 86 trackers / 60 sites computed correctly,
but only one tracker node was visible — the force layout had spread
the rest off-screen and the on-tick autoFit caught the simulation
mid-flight using SVG getBBox (which is sensitive to label text far
outside the cluster).

  - Force tuning scales with node count (146 nodes → linkDist 40,
    chargeStr -60, collide 14, alphaDecay 0.05 for faster settle).
  - Pre-warm: 300 synchronous simulation.tick()s before first render so
    positions are already converged.
  - autoFit() uses node data positions (min/max x,y over nodes array)
    instead of content.getBBox() — labels no longer skew the fit.
  - First render: paint pre-warmed positions immediately, kick a gentle
    alpha(0.05) restart for polish, autoFit on the next animation frame
    with duration 0 (instant).
2026-06-10 14:22:16 +02:00
0aeedcf1f4 feat(toolbox): d3 pan + pinch-zoom + auto-fit initial render (ref #507)
- Wrap graph content (links + nodes) in a content <g> so d3.zoom's
    transform applies there without moving the SVG itself.
  - d3.zoom(): scale 0.2–6×, mouse wheel + touch pinch + drag-pan.
    Filter lets node drags pass through d3.drag (no collision with the
    pan gesture) and always accepts multi-touch + wheel.
  - autoFit(): after the simulation alpha drops below 0.1, compute the
    nodes' bounding box and transition to a zoom transform that fits
    all of them with 28 px padding (capped at 2.5× so a single-node
    graph doesn't blow up).  Same logic re-runs on dblclick (reset)
    and on viewport resize (debounced 150 ms).
  - Viewport-resize observer keeps the viewBox and zoom in sync when
    orientation changes.
2026-06-10 14:22:16 +02:00
8a4598e2b3 feat(toolbox): full-viewport graph + drop persistent overlay messages (ref #507)
User feedback : graph too small + 'Chargement…' / 'ZÉRO TRACKER DÉTECTÉ'
overlays kept stacking on top of the graph.

Layout reworked :
  - body flex column 100vh overflow hidden — header sticky, main owns
    all remaining vertical space, graph fills it.
  - stats tiles → small glassy pill floating top-left over the graph.
  - cards collapsed into a compact 3-cell horizontal strip at the
    bottom ; open-card popover slides up over the graph so the graph
    stays visible.
  - graph svg measured at render-time, viewBox set dynamically — d3
    force-center scales to actual viewport instead of the fixed
    600×600 box.

Persistent overlays removed :
  - .social-empty and .social-loading divs gone from template.
  - JS no longer references them.  Empty graph is its own indicator
    (stats tiles show 0/0).  Network errors log to console only.

Net effect : tap 'Ma carto' on splash → full-screen force-directed
graph, stats top-left, cards bottom strip, no overlay clutter.
2026-06-10 14:22:16 +02:00
720d61a9d3 fix(toolbox): move social-view i18n from data-* attr to <script> (ref #507)
JSON in a single-quoted HTML data-* attribute breaks on FR apostrophes
(l'effacement, L'analyse) — JSON.parse threw 'unterminated string at
line 1 column 475'.  <script>window.__SOCIAL_I18N__ = {…}</script>
keeps the JSON verbatim and is the standard idiom for shipping
server-rendered config to client JS.
2026-06-10 14:22:16 +02:00
d195408fd4 fix(toolbox): PYTHONPATH in mitm-wg launcher so addons can import secubox_toolbox (ref #507)
Root cause caught after Phase 11.B live deploy : the mitm-wg workers
run inside /opt/mitmproxy-toolbox/bin/python3 — a venv whose
site-packages does NOT include /usr/lib/secubox/toolbox/. Every addon
that does `from secubox_toolbox import ...` was therefore silently
degraded :

  - inject_banner.py: dpi_class / geo / store / utiq imports all
    failed → no host classification, no GeoIP, no ASN, no Utiq tile.
  - social_graph.py: _social = None → record_edge() no-op → zero
    social_edges rows even though the response hook fired correctly.

Adding `export PYTHONPATH=/usr/lib/secubox/toolbox` to the launcher
restores ALL helper imports for ALL addons.

Verified live on gk2 : after deploy + worker restart, 369 social_edges
recorded in 30s, fold produced cross-site links connecting 4 ad-tech
relay publishers (360yield, seedtag, smartadserver, smilewanted via
the same 35.214.136.108 endpoint).
2026-06-10 14:22:16 +02:00
8dc1449eab fix(toolbox): /social/me R3 transparent peer resolution via X-R3-Peer (ref #507)
iPhone hitting /social/me from the kbin splash on R3 tunnel was 400'ing
because the previous chain was only ?mh=<hash> + ARP _resolve(), and R3
peers are on 10.99.1.0/24 not the captive subnet.

New 3-step resolution chain (matches /report/me/html):
  1. ?mh=<hash>             — explicit hex hash in URL
  2. X-R3-Peer header       — sentinel set by mitm-wg's inject_xff
                              for transparent R3 flows ; we look up
                              the WG pubkey hash from
                              /var/lib/secubox/toolbox/wg-peers.json
                              keyed by peer IP.
  3. ARP _resolve()         — R2 captive subnet fallback

Verified live: GET /social/me with X-R3-Peer: 10.99.1.60 → 303 →
/social/<token-for-mac_hash 9433ceb90895a075>
2026-06-10 14:22:16 +02:00
fc22b64b99 fix(toolbox): Phase 11.B live-deploy hardening — addon import, static mount, kbin paths, splash link (ref #507)
Five fixes caught during the first live deploy on gk2 :

  1. mitmproxy_addons/social_graph.py: `from . import local_store` was
     never resolving because mitmproxy loads addons as top-level
     modules (not as package members).  Inlined the R3 path
     (10.99.1.0/24 → sha256(wg_pubkey)[:16]) directly.  Verified
     against the real iPhone IP (10.99.1.60 → 9433ceb90895a075).

  2. www/toolbox/social.js: the JSON fetch + wipe paths used the
     nginx-prefixed `/api/v1/toolbox/social/...`.  The kbin vhost
     routes via HAProxy straight to uvicorn (bypassing nginx) so
     that prefix never matches.  Rewrote to the FastAPI router
     paths `/social/graph/{token}` + `/social/wipe/{token}`.

  3. secubox_toolbox/app.py: FastAPI StaticFiles mount on /toolbox
     for the kbin HAProxy path (which doesn't go through nginx).
     The mount serves the same files nginx would alias from
     /usr/share/secubox/www/toolbox/.

  4. debian/postinst: chmod 0755 /usr/share/secubox/www so the
     `secubox-toolbox` service user (not in secubox group) can
     traverse to the WebUI directory.  Without this the StaticFiles
     mount crashed the service at startup with PermissionError.

  5. New /social/me endpoint + splash menu icon : self-resolving
     entry point that mints a 1 h HMAC token and 303-redirects to
     /social/{token}.  Mirrors the existing /report/me/html pattern.
     Splash page gets a 🕸️ "Ma carto" quick-nav icon.

End-to-end via gk2 live :
  /social/me?mh=<hash> → 303 → /social/{token} → 200 (4883 bytes FR)
  static assets : 200 / 7529 / 8359 / 279706 bytes
  addon smoke test : mac_hash for 10.99.1.60 → 9433ceb90895a075 ✓
2026-06-10 14:22:16 +02:00
693ee360e9 feat(toolbox): Phase 11.B social mapping per-client view + favicon proxy + i18n (ref #507)
Parent #502 (Phase 11 social mapping per device).  Builds on top of
the Phase 11.A backend (PR #506, branch feature/505).

  - secubox_toolbox/api.py: GET /social/{token} renders the HTML view
    (HMAC-token gated, FR/EN i18n via ?lang= or Accept-Language).
    GET /social/favicon/{domain} server-side cached proxy (7d TTL,
    strict charset, HTTPS only, hard 5s timeout, transparent gif
    fallback).

  - conf/social_view.html.j2: Jinja2 template implementing the
    locked design from the #502 round-2 lock — Cinzel header / IM
    Fell body / JetBrains Mono data, force-directed d3 graph, three
    collapsible cards, tracker-node bottom-sheet, wipe dialog with
    3 s countdown.

  - conf/i18n/social.{fr,en}.json: single dictionary per language.
    FR is the source-of-truth.

  - www/toolbox/social.css: palette-precise stylesheet.  Mobile-first;
    desktop bumps the node-detail panel to a right-rail at >=720 px.

  - www/toolbox/social.js: self-contained d3 driver.  Consumes the
    Phase A JSON contract, force layout (link distance 70, charge
    -180), tap-to-focus pulse on tracker nodes, wipe modal countdown
    timer, refresh after wipe.

  - www/toolbox/d3.v7.min.js: d3 v7.9.0 self-hosted (~280 KB),
    ISC-licensed, no CDN runtime dependency.

  - debian/changelog: bump to 2.6.1-1~bookworm1.

Smoke tests: FR + EN templates render (4771 / 4503 bytes), api.py
parses, social.js node-checks clean.

Phase C scope deferred — legal-evidence flag computation, bilingual
PDF report, GeoIP+ASN populated values on the node detail panel,
operator dashboard /admin/social/ HTML view (the JSON
/admin/social-aggregate endpoint already ships from 2.6.0).
2026-06-10 14:22:16 +02:00
bafd856d1e fix(toolbox): debian/rules ships sbin/* launchers consistently (ref #505)
Caught during 2.6.0 live deploy on gk2: `debian/rules` had install lines
for `sbin/secubox-toolbox-db-tune` (override_dh_strip) and a few wg-
phase scripts but NEVER for `sbin/secubox-toolbox-mitm-wg-launch`. The
existing launcher on gk2 was a leftover from an older deploy and dpkg
upgrades silently kept the file static — which masked the Phase 11.A
addon-chain edit (social_graph was added in source but never reached
the live launcher).

  - debian/rules: explicit install lines for the three sbin helpers
    (lxc-provision, mitm-wg-launch, wg-provision).  These were either
    ad-hoc-installed before or relied on the operator copying them
    by hand.
  - sbin/secubox-toolbox-mitm-wg-launch: chmod 0755 (was 0664).
  - sbin/secubox-toolbox-wg-restore: chmod 0755 (was 0664).

Closes a source-first regression caught by [feedback_source_first_always].
2026-06-10 14:22:16 +02:00
6b230d464d feat(toolbox): Phase 11.A social mapping backend — correlation engine + API (ref #505)
Parent #502 (Phase 11 social mapping per device).  Backend-only deliverable
unblocking Phase 11.B (d3 view) and 11.C (legal-evidence PDF).

  - secubox_toolbox/social.py: new module.  SQLite schema for
    social_edges (raw, 7d retention) + social_nodes (per-peer per-tracker
    aggregate) + social_links (per-peer per-site-pair with shared-tracker
    list and JA4 collision flag).  ThreadPoolExecutor fire-and-forget
    write path matches utiq.py.  fold_recent() runs every 5 min and
    derives the aggregate tables.  wipe_mac() backs the RGPD art. 17
    endpoint.  aggregate() backs the operator dashboard.

  - mitmproxy_addons/social_graph.py: new addon.  Loaded after
    local_store and before inject_banner in the mitm-wg chain.
    Parses Set-Cookie + Cookie headers, normalizes via
    cookie_id_hash = sha256(domain||name||value)[:16] — raw cookie
    values are NEVER persisted.  Deny-list strips
    session/CSRF/auth/locale cookie names.  Cheap eTLD+1
    approximation for 1st- vs 3rd-party classification.  Reuses
    local_store helpers for the rotating MAC hash.

  - secubox_toolbox/api.py: three new endpoints.
      GET  /social/graph/{token}       per-client graph (HMAC token)
      POST /social/wipe/{token}        RGPD art. 17 droit à l'effacement
      GET  /admin/social-aggregate     operator dashboard KPI view

  - secubox_toolbox/app.py: two new startup background tasks —
    social_fold_loop (5 min) and social_purge_loop (1 h).  Match the
    existing purge/threat_intel pattern.

  - sbin/secubox-toolbox-mitm-wg-launch: addon chain wired
    (… local_store → social_graph → inject_banner …).

Phase A scope locks: R3 mitm-wg path only.  R2 captive hookup, d3
graph view, bilingual FR/EN PDF report, and the consent-state /
extra-EU legal-evidence flags are deferred per the #502 design lock
(rounds 1 + 2).

Smoke-tested on the host : hash stability, deny-list, edge insertion,
fold, fetch_graph contract shape, aggregate, wipe_mac round-trip,
addon helpers (eTLD+1, Set-Cookie parser, Cookie parser, Domain attr).
All seven invariants green.

  - debian/changelog: bump to 2.6.0-1~bookworm1.
2026-06-10 14:22:16 +02:00
22 changed files with 3556 additions and 297 deletions

View File

@ -3,6 +3,60 @@
--- ---
## 2026-06-10 (soir) — Phase 11 COMPLETE + Phase 12.A/B + toolbox tabs — v2.13.15 (ref #502-#516)
Consolidated stack merged via PR #517. `secubox-toolbox 2.5.2 → 2.6.6`,
tag **v2.13.15**.
### Package progression
2.6.0 (11.A backend) → 2.6.1 (11.B frontend) → 2.6.2 (#513 toolbox tabs)
→ 2.6.3 (11.C consent+PDF) → 2.6.4 (12.A CDN) → 2.6.5 (12.B anti-bot) →
2.6.6 (Carto kbin-redirect fix).
### Phase 11 — social mapping per device (#502) COMPLETE
- **11.A** (#505) : `social.py` correlation engine, 3 SQLite tables,
`social_graph.py` addon (cookie_id_hash = sha256, never raw values),
`/social/graph|wipe/{token}` + `/admin/social-aggregate`.
- **11.B** (#507) : d3 force-directed view, FR/EN i18n, server-side
favicon proxy, wipe modal (3s countdown), `/social/me` splash link.
Critical live fixes : PYTHONPATH in mitm-wg launcher (every addon's
`secubox_toolbox` import was silently degraded), i18n in `<script>`
block, StaticFiles mount + 0755 www, X-R3-Peer resolution.
- **11.C** (#508) : consent-platform probe (OneTrust/Didomi/Quantcast/
Sourcepoint), pre-consent + extra-EU evidence, bilingual FR/EN PDF
(fpdf2). Live PDF 200 / valid v1.3.
### Toolbox WebUI (#513)
5-tab nav (Vue d'ensemble / Clients / Filtres / Cartographie sociale /
Config). Inline kbin `/admin/` HTML route (~230 lines) deleted ;
canonical operator surface is now `admin.gk2.secubox.in/toolbox/`.
### Phase 12 — anti-human-detection platform (#514)
- **12.A** (#515) : `detect_cdn()` from response headers (11 vendors +
generic edge-cache), `social_host_meta` table, by_cdn aggregate.
**Round-Eye central-hotspot graph** : device = pulsing eye at centre,
sites on inner forceRadial ring, trackers outer ring, CDN-tinted nodes.
- **12.B** (#516) : `detect_antibot()` (reCAPTCHA/hCaptcha/Turnstile/
Datadome/PerimeterX-HUMAN/Arkose/Kasada/Akamai-BotManager) from URL +
cookies + headers — DETECTION ONLY, bypass gated behind doctrine.
Severe cinnabar lens + spinning warning ring + "challenged your
humanity" banner. Visible ring levels (dominant radial + ring guides +
cache-bust). Per-client operator tools : 🕸️ Carto (token-minted graph
link, absolute kbin redirect), ↺ Reset/RAZ (`store.reset_client` +
`social.wipe_mac`).
### Round Eye gadget — diagnosed, physical fix required
OTG USB CDC-Ethernet TX queue wedged (NETDEV WATCHDOG, probe -110).
Gadget enumerates but data path dead. gk2-side recovery exhausted
(link bounce, USB unbind/rebind). Needs Pi power-cycle / cable re-seat.
### Live + verified on gk2
Graph renders real cross-site tracking (`35.214.136.108` relay bridging
4 publishers), PDF valid, CDN + anti-bot read paths green end-to-end,
reset works, Carto opens the client graph on kbin.
---
## 2026-06-10 — Phase 11 social mapping (A+B) + system triage + v2.13.14 (ref #502-#509) ## 2026-06-10 — Phase 11 social mapping (A+B) + system triage + v2.13.14 (ref #502-#509)
### Package bumps ### Package bumps

View File

@ -5,35 +5,39 @@
## 🔥 P0 — Immediate (in flight) ## 🔥 P0 — Immediate (in flight)
### Phase 11 — Social mapping per device (#502) ### Phase 11 — Social mapping per device (#502) — ✅ COMPLETE (v2.13.15)
- [x] **11.A backend** (#505 / PR #506, `secubox-toolbox 2.6.0`) — correlation - [x] **11.A backend** (#505, `2.6.0`) — correlation engine + SQLite + API.
engine + SQLite + API. Déployé live gk2. - [x] **11.B frontend** (#507, `2.6.1`) — d3 graph + i18n + favicon proxy + wipe.
- [x] **11.B frontend** (#507, `2.6.1`) — d3 graph + i18n FR/EN + favicon - [x] **11.C evidence + PDF** (#508, `2.6.3`) — consent-probe + bilingue FR/EN PDF.
proxy + wipe modal + full-viewport pan/zoom. Live `/social/me`. - [x] **Toolbox WebUI tabs** (#513, `2.6.2`) — 5-tab nav, kbin /admin/ supprimé.
- [ ] **11.C evidence + PDF** (#508) — reprendre depuis checkpoint - [x] **Mergé** via PR #517 → master, tag `v2.13.15`.
`55626e51` : consent-probe addon (OneTrust/Didomi/Quantcast/Sourcepoint) - [ ] **11.D opérateur** (futur, optionnel) — vue HTML `/admin/social/`
+ extra-EU flag + PDF bilingue FR/EN + wire frontend (remplacer le dédiée (le tab Cartographie sociale dans /toolbox/ couvre déjà l'agrégat).
placeholder "Rapport PDF arrive en Phase 11.C").
- [ ] **Merger PR #506** (11.A backend) → master quand prêt.
- [ ] **11.D opérateur** (futur) — dashboard agrégat `/admin/social/` HTML
(l'endpoint JSON `/admin/social-aggregate` existe déjà depuis 2.6.0).
### Système — bugs gk2 (2026-06-10) ### Phase 12 — Anti-human-detection platform (#514)
- [x] **12.A CDN** (#515, `2.6.4`) — detect_cdn + Round-Eye central-hotspot
graph + by_cdn. Mergé.
- [x] **12.B anti-bot** (#516, `2.6.5/2.6.6`) — detect_antibot (détection
seule) + ring levels visibles + Carto/Reset opérateur. Mergé.
- [ ] **12.C opérateur-grade / state-adjacent** — étend #500 Utiq :
identité carrier-grade (MSISDN injection, CGNAT fingerprint) + analytics
state-adjacent. Prochain track.
- [ ] **12.B bypass** — résolution de challenge (gated derrière doctrine
lawful-use + design review ; R3 opt-in uniquement).
- [ ] **12.D noise counter-measures** — cookie-noising / header-strip /
decoy-traffic (gated derrière doctrine ; R3 opt-in, interférence active).
### Système — bugs gk2 (2026-06-10) — ✅ résolus
- [x] **CrowdSec firewall** — restart bouncer → tables nft recréées. - [x] **CrowdSec firewall** — restart bouncer → tables nft recréées.
- [x] **WAF /var/log/secubox traversal** — chmod 0755 live. - [x] **WAF /var/log/secubox traversal** — fix source #511/#512 (mergé).
- [x] **WAF /stats perf** (#509 / PR #510, `secubox-waf 1.2.2`) — double-buffer - [x] **WAF /stats perf** (#509/#510, `secubox-waf 1.2.2`) — double-buffer cache.
cache. Mergé + `v2.13.14`.
- [x] **PeerTube + PhotoPrism** — LXC redémarrés. - [x] **PeerTube + PhotoPrism** — LXC redémarrés.
- [ ] **Round Eye gadget** — ne voit plus gk2, métriques locales only. - [ ] **Round Eye gadget** — USB CDC-Ethernet TX queue wedged (NETDEV
Investigation Pi Zero (dashboard `localhost:8000` proxie vers gk2 via OTG). WATCHDOG, probe -110). Recovery gk2 épuisée. **Fix physique : power-cycle
- [ ] **admin.gk2/toolbox/ tab** — toolbox déjà wiré (`/toolbox/` alias + Pi Zero / re-seat câble OTG.** Reprendre côté gk2 au prochain boot propre.
sidebar). User veut surfacer l'UI kbin/admin dedans — décision en
attente : proxy_pass `10.99.0.1:8088/admin/` (cleanest) / iframe (CSP) /
sous-tab frontend-only.
- [ ] **Postinst `/var/log/secubox` 0755** — porter le fix live en source
(même pattern que `/etc/secubox` parent + `/usr/share/secubox/www`).
### Phase 10 — Banner injection perf (#501) — ✅ shipped 2026-06-09 ### Phase 10 — Banner injection perf (#501) — ✅ shipped 2026-06-09

View File

@ -3,6 +3,67 @@
--- ---
## 🔄 2026-06-10 (soir) : Phase 11 COMPLETE + Phase 12.A/B + toolbox tabs — merged v2.13.15 (ref #502-#516)
Tout le stack Phase 11 + Phase 12.A/B mergé en une PR consolidée (#517),
`secubox-toolbox 2.5.2 → 2.6.6`, tag **v2.13.15**.
### ✅ Done — mergé master
| Issue | Livré | Version |
|---|---|---|
| #505 11.A backend | correlation engine + SQLite + /social API | 2.6.0 |
| #507 11.B frontend | d3 graph + i18n FR/EN + favicon proxy + wipe + /social/me | 2.6.1 |
| #513 toolbox tabs | 5-tab WebUI, kbin /admin/ inline UI supprimé | 2.6.2 |
| #508 11.C evidence | consent-probe + PDF bilingue FR/EN | 2.6.3 |
| #515 12.A CDN | detect_cdn + Round-Eye central-hotspot graph | 2.6.4 |
| #516 12.B anti-bot | detect_antibot + ring levels visibles + Carto/Reset | 2.6.5/2.6.6 |
### ✅ Phase 11 — social mapping per device (#502)
Live : `https://kbin.gk2.secubox.in/social/me` (🕸️ Ma carto).
Graphe Round-Eye : l'appareil = œil central pulsant, sites sur anneau
interne (forceRadial 0.9), trackers anneau externe. Montre le relais
ad-tech `35.214.136.108` reliant 360yield + seedtag + smartadserver +
smilewanted. PDF d'évidence bilingue (consentement RGPD art. 6.1.a+7,
transferts hors-UE art. 44). Effacement RGPD art. 17. Anonyme (mac_hash
sel rotatif 24h, aucune valeur cookie brute).
### ✅ Phase 12 — anti-human-detection platform (#514)
- **12.A CDN** : detect_cdn (Cloudflare/Fastly/Akamai/CloudFront/Google/
Vercel/Netlify/Bunny/KeyCDN/Sucuri/Imperva). Nodes teintés par vendor,
table social_host_meta, agrégat by_cdn.
- **12.B anti-bot** : detect_antibot (reCAPTCHA/hCaptcha/Turnstile/
Datadome/PerimeterX-HUMAN/Arkose/Kasada/Akamai-BotManager). Lens
cinnabar sévère + ring d'alerte + bannière "challenged your humanity".
Table social_antibot per-client, agrégat by_antibot.
**Bypass NON livré** — gated derrière doctrine (séquencement #514).
- **Ring levels visibles** : forceRadial dominant + guides d'anneau +
cache-bust ?v=264b (le user ne voyait pas la réorg ; corrigé).
- **Outils opérateur Clients tab** : 🕸️ Carto (ouvre le graphe client
via token, redirige vers kbin absolu), ↺ Reset/RAZ (efface social +
events + score).
### ⚠️ Round Eye gadget — diagnostic, fix physique requis
OTG USB CDC-Ethernet **TX queue wedged** (NETDEV WATCHDOG timeout sur
3-1.1.2). Le gadget énumère (descripteurs lisibles) mais le data path
TX est mort — control transfers OK, bulk KO. Recovery gk2 épuisée
(link bounce, unbind/rebind → probe error -110). **Power-cycle du Pi
Zero ou re-seat câble OTG nécessaire** ; self-heal au prochain boot
propre via la règle systemd .link.
### ⬜ Next up
- **Phase 12.C** (#514) : opérateur-grade / state-adjacent (étend #500
Utiq). Détection identité carrier-grade + analytics state-adjacent.
- **Phase 12.B bypass + 12.D noise** : derniers, chacun gated par sa
doctrine + design review (interférence active 3rd-party = R3 opt-in).
- **Round Eye** : reprendre côté gk2 dès que le Pi re-énumère propre.
---
## 🔄 2026-06-10 : Phase 11 social mapping (A+B) + system triage round (ref #502-#509) ## 🔄 2026-06-10 : Phase 11 social mapping (A+B) + system triage round (ref #502-#509)
Grosse journée : Phase 11 social mapping shippé jusqu'au frontend live, Grosse journée : Phase 11 social mapping shippé jusqu'au frontend live,

View File

@ -0,0 +1,37 @@
{
"page_title": "Social mapping",
"header_title": "VILLAGE3B",
"header_subtitle": "Social mapping",
"intro": "Trackers detected for your browsing session.",
"stats_total_trackers": "trackers",
"stats_total_sites": "sites visited",
"stats_window": "Window",
"card_evidence": "Legal evidence",
"card_wipe": "GDPR art. 17 erasure",
"card_pdf": "PDF report",
"card_pdf_pending": "The PDF report ships in Phase 11.C.",
"card_evidence_pending": "Compliance analysis (consent, extra-EU transfers) ships in Phase 11.C.",
"graph_swipe_hint": "Drag · Zoom · Tap a node",
"node_detail_country": "Country",
"node_detail_asn": "ASN",
"node_detail_sites": "Present on",
"node_detail_first_seen": "First seen",
"node_detail_last_seen": "Last seen",
"empty_title": "ZERO TRACKERS DETECTED",
"empty_body": "Good privacy day.",
"wipe_button": "Erase my data",
"wipe_modal_title": "Confirm erasure of your data?",
"wipe_modal_body": "All your browsing data and tracker associations will be permanently removed from VILLAGE3B.",
"wipe_modal_cancel": "Cancel",
"wipe_modal_confirm": "Confirm erasure (GDPR art. 17)",
"wipe_modal_countdown": "Wait {n} s before confirming…",
"wipe_success": "Your data has been erased. {n} rows deleted.",
"loading": "Loading…",
"error": "Loading error.",
"lang_label": "EN",
"card_pdf_download": "⬇ Download PDF report (FR/EN)",
"card_evidence_active": "Compliance analysis: trackers before consent (GDPR art. 6.1.a + 7) and extra-EU transfers (art. 44). See the PDF report for details.",
"node_detail_cdn": "CDN / cache",
"node_detail_antibot": "Anti-bot",
"antibot_alert": "⚠ {n} site(s) challenged that you are human"
}

View File

@ -0,0 +1,37 @@
{
"page_title": "Cartographie sociale",
"header_title": "VILLAGE3B",
"header_subtitle": "Cartographie sociale",
"intro": "Cartographie des traqueurs détectés pour votre session.",
"stats_total_trackers": "traqueurs",
"stats_total_sites": "sites visités",
"stats_window": "Fenêtre",
"card_evidence": "Évidence juridique",
"card_wipe": "Effacement RGPD art. 17",
"card_pdf": "Rapport PDF",
"card_pdf_pending": "Le rapport PDF arrive en Phase 11.C.",
"card_evidence_pending": "L'analyse de conformité (consentement, transferts hors UE) arrive en Phase 11.C.",
"graph_swipe_hint": "Glisser · Zoom · Toucher un nœud",
"node_detail_country": "Pays",
"node_detail_asn": "ASN",
"node_detail_sites": "Présent sur",
"node_detail_first_seen": "Première fois",
"node_detail_last_seen": "Dernière fois",
"empty_title": "ZÉRO TRACKER DÉTECTÉ",
"empty_body": "Bonne journée de vie privée.",
"wipe_button": "Effacer mes données",
"wipe_modal_title": "Confirmer l'effacement de vos données ?",
"wipe_modal_body": "Toutes vos données de navigation et associations de trackers seront supprimées irréversiblement de VILLAGE3B.",
"wipe_modal_cancel": "Annuler",
"wipe_modal_confirm": "Confirmer l'effacement (RGPD art. 17)",
"wipe_modal_countdown": "Patientez {n} s avant confirmation…",
"wipe_success": "Vos données ont été effacées. {n} enregistrements supprimés.",
"loading": "Chargement…",
"error": "Erreur de chargement.",
"lang_label": "FR",
"card_pdf_download": "⬇ Télécharger le rapport PDF (FR/EN)",
"card_evidence_active": "Analyse de conformité : traqueurs avant consentement (RGPD art. 6.1.a + 7) et transferts hors UE (art. 44). Voir le rapport PDF pour le détail.",
"node_detail_cdn": "CDN / cache",
"node_detail_antibot": "Anti-bot",
"antibot_alert": "⚠ {n} site(s) ont vérifié que vous êtes humain"
}

View File

@ -74,6 +74,9 @@ a:hover{text-decoration:underline}
<a href="/report/me/html" class=qi title="Mon rapport live"> <a href="/report/me/html" class=qi title="Mon rapport live">
<span class=qi-emoji>📊</span><span class=qi-label>Mon rapport</span> <span class=qi-emoji>📊</span><span class=qi-label>Mon rapport</span>
</a> </a>
<a href="/social/me" class=qi title="Cartographie sociale — qui me piste, où ?">
<span class=qi-emoji>🕸️</span><span class=qi-label>Ma carto</span>
</a>
<a href="/wg/ca.mobileconfig" class=qi title="CA R3 iPhone (.mobileconfig)"> <a href="/wg/ca.mobileconfig" class=qi title="CA R3 iPhone (.mobileconfig)">
<span class=qi-emoji>📲</span><span class=qi-label>CA iPhone</span> <span class=qi-emoji>📲</span><span class=qi-label>CA iPhone</span>
</a> </a>
@ -83,9 +86,6 @@ a:hover{text-decoration:underline}
<a href="/wg/qr.png" class=qi title="QR profil WireGuard"> <a href="/wg/qr.png" class=qi title="QR profil WireGuard">
<span class=qi-emoji>📱</span><span class=qi-label>QR profil</span> <span class=qi-emoji>📱</span><span class=qi-label>QR profil</span>
</a> </a>
<a href="/admin/" class=qi title="Admin webui">
<span class=qi-emoji>🛠️</span><span class=qi-label>Admin</span>
</a>
<a href="https://github.com/CyberMind-FR/secubox-deb/wiki/R3-WireGuard-install" class=qi title="Wiki R3 multi-OS"> <a href="https://github.com/CyberMind-FR/secubox-deb/wiki/R3-WireGuard-install" class=qi title="Wiki R3 multi-OS">
<span class=qi-emoji>📖</span><span class=qi-label>Wiki</span> <span class=qi-emoji>📖</span><span class=qi-label>Wiki</span>
</a> </a>

View File

@ -0,0 +1,91 @@
<!doctype html>
<html lang="{{ lang }}">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1,viewport-fit=cover">
<title>{{ t.page_title }} — VILLAGE3B</title>
<link rel="stylesheet" href="/toolbox/social.css?v=264b">
<script>
// Inline JSON safer than a data-* attribute : FR text has
// apostrophes (l'effacement) that would break single-quoted
// HTML attrs. Wrapped in <script> JSON is verbatim so long as
// it doesn't contain </script (json.dumps from Python never
// emits raw "</" in normal output).
window.__SOCIAL_I18N__ = {{ t_json | safe }};
</script>
<script defer src="/toolbox/d3.v7.min.js"></script>
<script defer src="/toolbox/social.js?v=264b"></script>
</head>
<body data-token="{{ token }}" data-lang="{{ lang }}">
<header class="social-header">
<div class="brand">
<span class="brand-title">{{ t.header_title }}</span>
<span class="brand-divider"> / </span>
<span class="brand-subtitle">{{ t.header_subtitle }}</span>
</div>
<div class="lang">{{ t.lang_label }}</div>
</header>
<main class="social-main">
<section class="graph-wrap" aria-label="{{ t.header_subtitle }}">
<section class="stats" aria-live="polite">
<div class="stat-tile"><span class="stat-n" data-bind="total_trackers">0</span><span class="stat-l">{{ t.stats_total_trackers }}</span></div>
<div class="stat-tile"><span class="stat-n" data-bind="total_sites">0</span><span class="stat-l">{{ t.stats_total_sites }}</span></div>
</section>
<p class="graph-hint">{{ t.graph_swipe_hint }}</p>
<div id="antibot-alert" class="antibot-alert" hidden></div>
<svg id="social-graph" role="img" preserveAspectRatio="xMidYMid meet"></svg>
</section>
<aside id="node-detail" class="node-detail" hidden aria-live="polite">
<header>
<span class="nd-domain" data-bind="nd_domain"></span>
<button type="button" class="nd-close" aria-label="close" data-action="close-nd">×</button>
</header>
<dl>
<dt>{{ t.node_detail_country }}</dt>
<dd data-bind="nd_country">—</dd>
<dt>{{ t.node_detail_asn }}</dt>
<dd data-bind="nd_asn">—</dd>
<dt>{{ t.node_detail_cdn }}</dt>
<dd data-bind="nd_cdn">—</dd>
<dt>{{ t.node_detail_antibot }}</dt>
<dd data-bind="nd_antibot">—</dd>
<dt>{{ t.node_detail_sites }}</dt>
<dd data-bind="nd_sites">—</dd>
<dt>{{ t.node_detail_first_seen }}</dt>
<dd data-bind="nd_first_seen">—</dd>
<dt>{{ t.node_detail_last_seen }}</dt>
<dd data-bind="nd_last_seen">—</dd>
</dl>
</aside>
<nav class="cards-row">
<details class="card">
<summary>{{ t.card_evidence }}</summary>
<p class="card-pending">{{ t.card_evidence_active }}</p>
</details>
<details class="card card-wipe">
<summary>{{ t.card_wipe }}</summary>
<button type="button" class="wipe-btn" data-action="open-wipe">{{ t.wipe_button }}</button>
</details>
<details class="card">
<summary>{{ t.card_pdf }}</summary>
<a class="pdf-btn" href="/social/report/{{ token }}.pdf" target="_blank" rel="noopener">{{ t.card_pdf_download }}</a>
</details>
</nav>
</main>
<dialog id="wipe-modal" aria-labelledby="wipe-modal-title">
<h2 id="wipe-modal-title">{{ t.wipe_modal_title }}</h2>
<p>{{ t.wipe_modal_body }}</p>
<p class="wipe-countdown" data-bind="wipe_countdown" hidden></p>
<div class="modal-actions">
<button type="button" data-action="cancel-wipe" class="btn-secondary">{{ t.wipe_modal_cancel }}</button>
<button type="button" data-action="confirm-wipe" class="btn-danger" disabled>{{ t.wipe_modal_confirm }}</button>
</div>
</dialog>
</body>
</html>

View File

@ -1,3 +1,217 @@
secubox-toolbox (2.6.6-1~bookworm1) bookworm; urgency=medium
* Phase 12.B follow-up (#516) — fix the Clients-tab 🕸️ Carto link
landing on the aggregator "missing module" page. The /social/
route is only served on the kbin vhost ; the operator triggers the
link from admin.gk2.secubox.in. admin_client_social() now builds
an absolute kbin URL (swaps the leading admin. host label for
kbin.) instead of a relative redirect.
-- Gerald KERMA <devel@cybermind.fr> Wed, 10 Jun 2026 11:20:00 +0200
secubox-toolbox (2.6.5-1~bookworm1) bookworm; urgency=medium
* Phase 12.B (#516, parent #514) — anti-bot / "prove you're human"
detection + ring-level graph + per-client operator tools.
- social_graph.py: detect_antibot() classifies the bot-checker /
CAPTCHA vendor challenging a flow (reCAPTCHA / hCaptcha /
Turnstile / Datadome / PerimeterX-HUMAN / Arkose / Kasada /
Akamai Bot Manager) from URL + cookies + headers. Detection
only ; bypass stays gated behind its own doctrine.
- social.py: social_host_meta.antibot_vendor + new social_antibot
per-client challenge table. fetch_graph carries antibot_vendor
+ an antibot list/stats; aggregate adds by_antibot +
antibot_clients; wipe_mac clears social_antibot too.
- social.js: anti-bot hosts get the highest-severity cinnabar
lens + a spinning warning ring + 🤖 label; "challenged your
humanity" alert banner; node-detail shows the vendor.
- VISIBLE RING LEVELS: radial force is now dominant (strength
0.9) with weak charge + weak link springs, plus dashed ring
guides (inner = your sites, outer = trackers) so the Round-Eye
levels read clearly. Assets cache-busted (?v=264b).
- index.html operator social tab: anti-bot breakdown card.
- index.html Clients tab: per-row 🕸️ Carto (opens the client's
graph via a minted token), ↺ Reset (RAZ the client's stats).
- api.py: GET /admin/clients/{mac}/social (operator graph link),
POST /admin/clients/{mac}/reset (RAZ social + events + score).
- store.py: reset_client() wipes events/consents/reports + zeroes
score.
- i18n: node_detail_antibot + antibot_alert (FR/EN).
-- Gerald KERMA <devel@cybermind.fr> Wed, 10 Jun 2026 11:00:00 +0200
secubox-toolbox (2.6.4-1~bookworm1) bookworm; urgency=medium
* Phase 12.A (#515, parent #514) — CDN cache detection + Round-Eye
central-hotspot graph.
- mitmproxy_addons/social_graph.py: detect_cdn() classifies the
edge network fronting each 3rd-party host from response headers
(Cloudflare / Fastly / Akamai / CloudFront / Google / Vercel /
Netlify / BunnyCDN / KeyCDN / Sucuri / Imperva + generic
edge-cache). Recorded host-stable (mac-independent) off-thread.
- secubox_toolbox/social.py: new social_host_meta table
(tracker_domain PK, cdn_vendor, cache_status). fetch_graph()
LEFT JOINs it onto nodes; aggregate() adds a by_cdn breakdown.
- www/toolbox/social.js: the per-client graph is re-centred on a
Round-Eye hotspot — the device is a pulsing eye at the centre,
sites orbit it on an inner ring (forceRadial), trackers push to
an outer ring, so the densest tracker clusters read as the
"hot spots". Tracker nodes are tinted by CDN vendor; node-detail
shows CDN / cache status.
- www/toolbox/social.css: eye halo/sclera/iris/pupil + breathing
animation + cinnabar spoke edges.
- www/toolbox/index.html: operator social tab gains a
"Réseaux CDN / edge" breakdown card.
- i18n: node_detail_cdn (FR/EN).
First passive track of the #514 anti-human-detection platform —
zero legal exposure, reuses the consent-probe response-header path.
-- Gerald KERMA <devel@cybermind.fr> Wed, 10 Jun 2026 10:00:00 +0200
secubox-toolbox (2.6.3-1~bookworm1) bookworm; urgency=medium
* Phase 11.C (#508, parent #502) — social mapping evidence + bilingual
FR/EN PDF report.
- mitmproxy_addons/social_graph.py : consent-platform probe.
Detects OneTrust / Didomi / Quantcast / Sourcepoint cookies +
loader URLs per (peer, site). Each recorded edge is stamped
consent_state in {none_seen, pre_consent, post_consent}.
pre_consent = a tracker fired while the site runs a CMP but no
consent cookie was seen yet (RGPD art. 6.1.a + 7 evidence).
- secubox_toolbox/social.py : schema (consent_state + GeoIP
columns) + EU/EEA whitelist + GeoIP fold + evidence() helper.
- secubox_toolbox/social_report.py : NEW — bilingual FR/EN
evidence PDF via fpdf2 (same engine as reports.py). Fact-only
cover + summary + pre-consent table + extra-EU table + RGPD
article references. Text fallback when fpdf2 is absent.
- secubox_toolbox/api.py : GET /social/report/{token}.pdf, same
HMAC-token gate as /social/graph/{token}.
- conf/social_view.html.j2 + i18n : the "Rapport PDF" card now
has a real download button ; the "Évidence juridique" card
switched to the active compliance summary.
Phase 11 (social mapping per device) is now feature-complete :
A (backend) + B (d3 view) + C (evidence + PDF).
-- Gerald KERMA <devel@cybermind.fr> Wed, 10 Jun 2026 09:30:00 +0200
secubox-toolbox (2.6.2-1~bookworm1) bookworm; urgency=medium
* ToolBox WebUI sub-tab navigation + kbin /admin/ removal (#513).
- www/toolbox/index.html : rebuilt with a 5-tab nav
(Vue d'ensemble / Clients / Filtres MITM / Cartographie
sociale / Config). Each tab lazy-loads its data; the live
refresh interval only polls the visible tab. URL hash
deep-links a tab (e.g. /toolbox/#social).
- Clients tab folds in the R0/R1/R2/R3 level switcher that
used to live only in the kbin inline admin UI (calls
/admin/clients/{mac}/level).
- Cartographie sociale tab surfaces the Phase 11 operator
aggregate (/admin/social-aggregate) : KPI tiles + top
tracker domains + anonymized client table.
- api.py : DELETED the inline kbin /admin/ HTML route
(admin_index(), ~230 lines). The canonical operator
dashboard is now admin.gk2.secubox.in/toolbox/. All
/admin/* JSON API routes are untouched.
- conf/landing.html.j2 : removed the 🛠️ Admin quicknav icon
(the page it pointed at no longer exists).
kbin.gk2.secubox.in/admin/ now 404s by design.
-- Gerald KERMA <devel@cybermind.fr> Wed, 10 Jun 2026 09:15:00 +0200
secubox-toolbox (2.6.1-1~bookworm1) bookworm; urgency=medium
* Phase 11.B (#507, parent #502) — social mapping per-client view +
favicon proxy + i18n.
- secubox_toolbox/api.py : two new endpoints.
GET /social/{token} renders the HTML view
(HMAC-token gated, lang via
?lang= or Accept-Language).
GET /social/favicon/{domain} server-side cached proxy with
7 d TTL. Strict domain
charset + SHA-256 keyed cache
file; HTTPS only with a hard
5 s timeout; falls back to a
1×1 transparent gif.
- conf/social_view.html.j2 : Jinja2 template implementing the
locked design (Cinzel header / IM Fell body / JetBrains Mono
data, force-directed d3 graph, three collapsible cards,
tracker-node bottom-sheet, FR-primary bilingual labels,
wipe-confirmation dialog with 3 s countdown).
- conf/i18n/social.fr.json + conf/i18n/social.en.json : single
dictionary per language. FR is the source-of-truth.
- www/toolbox/social.css : palette-precise stylesheet. No
third-party CSS dependency. Mobile-first; desktop bumps the
node-detail panel to a right-rail at >= 720 px.
- www/toolbox/social.js : self-contained d3 driver. Consumes
the Phase A JSON contract, force layout (link distance 70,
charge -180), tap-to-focus pulse on tracker nodes, wipe
modal countdown timer, refresh after wipe. ~270 LOC.
- www/toolbox/d3.v7.min.js : d3 v7.9.0 self-hosted (~280 KB),
ISC-licensed, no CDN runtime dependency.
Design language consolidated from the #502 round-2 lock — edge
color placeholder cyber-cyan until the Phase C family taxonomy,
edge thickness = log(reuse_count + 1) * 1.8, animated pulse on
focus, tracker-node detail sheet (bottom on mobile, right-rail
>= 720 px), 3 s countdown before wipe confirm is enabled.
Out of scope for 2.6.1 (deferred to Phase 11.C) :
- Legal-evidence flag computation (consent-state probe + extra
EU transfer detection).
- Bilingual FR/EN PDF report cover + body.
- GeoIP + ASN org populated in the tracker node detail panel
(country / ASN values currently show "-" placeholders).
- Operator dashboard /admin/social/ HTML view (the JSON
/admin/social-aggregate endpoint already ships from 2.6.0).
-- Gerald KERMA <devel@cybermind.fr> Tue, 09 Jun 2026 12:05:00 +0200
secubox-toolbox (2.6.0-1~bookworm1) bookworm; urgency=medium
* Phase 11.A (#505, parent #502) — social mapping per device :
backend correlation engine + SQLite tables + JSON API.
- secubox_toolbox/social.py : new module. Schema for
social_edges (raw events, 7 d retention), social_nodes
(per-peer per-tracker aggregate), social_links (per-peer
per-site-pair aggregate with shared-tracker list + JA4
collision flag). Fire-and-forget executor pattern matches
utiq.py (#500 Phase 8.B) — never blocks the mitmproxy
asyncio loop. fold_recent() rebuilds nodes + links from
raw edges every 5 min. wipe_mac() backs the RGPD art. 17
endpoint. aggregate() backs the operator dashboard.
- mitmproxy_addons/social_graph.py : new addon. Loaded
between local_store and inject_banner in the mitm-wg
chain. Parses Set-Cookie + Cookie headers, normalizes
identifiers via cookie_id_hash = sha256(domain||name||
value)[:16] — NEVER persists raw cookie values. Deny-list
strips session/CSRF/auth/locale names. 3rd-party detection
via a cheap eTLD+1 approximation (covers > 99 % of the
ecosystem we see ; full PSL fold is a future enhancement
if the data demands it).
- secubox_toolbox/api.py : three new endpoints.
GET /social/graph/{token} — per-client graph, HMAC-token
gated (same TTL semantics as the existing /report/{token}).
POST /social/wipe/{token} — RGPD art. 17 droit à
l'effacement, same HMAC contract. GET /admin/social-
aggregate — JWT-gated operator view, KPI tiles + tracker
histogram + anonymized client table. No per-client graph
on the admin path.
- secubox_toolbox/app.py : two new startup background tasks.
social_fold_loop() runs fold_recent every 5 min.
social_purge_loop() drops raw edges older than 7 d every
hour. Symmetric with the existing purge_loop + threat_intel
refresh_loop.
- sbin/secubox-toolbox-mitm-wg-launch : addon chain updated
to include social_graph AFTER local_store and BEFORE
inject_banner.
Scope explicitly limited to the R3 mitm-wg path for Phase A.
R2 captive hookup, the d3 graph UI, the bilingual PDF report,
and the legal-evidence flag computation (consent-state probe,
extra-EU transfer flag) are deferred to Phase 11.B / 11.C
per the design lock on #502 (rounds 1 + 2).
-- Gerald KERMA <devel@cybermind.fr> Tue, 09 Jun 2026 09:55:00 +0200
secubox-toolbox (2.5.2-1~bookworm1) bookworm; urgency=medium secubox-toolbox (2.5.2-1~bookworm1) bookworm; urgency=medium
* Phase 10.1 (#501 perf) — postinst regressions caught on 2.5.1 deploy. * Phase 10.1 (#501 perf) — postinst regressions caught on 2.5.1 deploy.

View File

@ -52,6 +52,17 @@ case "$1" in
# subdirs inside keep their own restricted perms. # subdirs inside keep their own restricted perms.
install -d -m 0755 -o secubox-toolbox -g secubox-toolbox /var/log/secubox install -d -m 0755 -o secubox-toolbox -g secubox-toolbox /var/log/secubox
# 4a. Phase 11.B (#507) — make /usr/share/secubox/www traversable so
# the FastAPI StaticFiles mount can serve /toolbox/social.{css,js} +
# /toolbox/d3.v7.min.js to clients arriving via the kbin HAProxy
# route (which bypasses nginx and so doesn't use the nginx /toolbox/
# alias). Without this chmod the uvicorn process crashes at
# startup with PermissionError on Path("/usr/share/secubox/www/
# toolbox").is_dir(). Idempotent ; safe to repeat.
if [ -d /usr/share/secubox/www ]; then
chmod 0755 /usr/share/secubox/www 2>/dev/null || true
fi
# 4b. GeoLite2 databases (Phase 2a+ : flag emojis + ASN org) # 4b. GeoLite2 databases (Phase 2a+ : flag emojis + ASN org)
# ASN DB from geoipupdate or Debian package geoip-database # ASN DB from geoipupdate or Debian package geoip-database
# Country DB from db-ip.com CC-BY (no MaxMind account required) # Country DB from db-ip.com CC-BY (no MaxMind account required)

View File

@ -23,6 +23,13 @@ override_dh_auto_install:
install -d debian/secubox-toolbox/usr/sbin install -d debian/secubox-toolbox/usr/sbin
install -m 0755 scripts/toolbox-up debian/secubox-toolbox/usr/sbin/ install -m 0755 scripts/toolbox-up debian/secubox-toolbox/usr/sbin/
install -m 0755 scripts/ca-init debian/secubox-toolbox/usr/sbin/secubox-toolbox-ca-init install -m 0755 scripts/ca-init debian/secubox-toolbox/usr/sbin/secubox-toolbox-ca-init
# sbin/ helpers (Phase 5 LXC, Phase 6 R3 WG, Phase 9 fanout, Phase 11 social).
# Pre-2.6.0 only db-tune + lxc-provision shipped — the mitm-wg-launch
# wrapper had to be hand-installed on the board. This block ships them
# all consistently so the .deb is self-contained.
install -m 0755 sbin/secubox-toolbox-lxc-provision debian/secubox-toolbox/usr/sbin/
install -m 0755 sbin/secubox-toolbox-mitm-wg-launch debian/secubox-toolbox/usr/sbin/
install -m 0755 sbin/secubox-toolbox-wg-provision debian/secubox-toolbox/usr/sbin/
override_dh_installsystemd: override_dh_installsystemd:
# Install the secondary unit manually (dh_installsystemd expects 1 unit/pkg). # Install the secondary unit manually (dh_installsystemd expects 1 unit/pkg).

View File

@ -0,0 +1,544 @@
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
# Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
"""
SecuBox-Deb :: ToolBoX MITM addon social_graph
Phase 11.A (#505, parent #502) — passive correlation of cross-site
tracker identifiers per R2/R3 consented peer.
The addon listens on the response hook (where Set-Cookie is sent by
the 3rd-party server) AND on the request hook (where the Cookie
header is sent back by the browser). For each cookie observed :
* Decide if the COOKIE-issuing domain is 3rd-party relative to the
1st-party host the browser was visiting.
* Reject deny-listed names (session, CSRF, auth, locale ).
* Hash the identifier (`sha256(domain || name || value)[:16]`) so
we have a stable but non-round-trippable key.
* Submit an edge record off-thread via secubox_toolbox.social.
Key invariants :
* Never persists raw cookie values.
* Never blocks the asyncio loop every write is fire-and-forget.
* Only fires when the peer is a known R2/R3 client (mac_hash
available). R0/R1 flows are ignored.
* The 1st-party `src_site` is derived from the request host's
registrable domain (eTLD+1 via a small inline PSL approximation).
* 3rd-party check : tracker_domain != src_site (eTLD+1 compare).
Phase 10 banner-perf already brought the response hook latency down ;
this addon stays cheap by deferring ALL persistence to the executor
and doing the parsing on a string-only fast path.
"""
from __future__ import annotations
import logging
import re
from typing import Iterable, List, Optional
from mitmproxy import http
try: # Best-effort — same defensive import as other toolbox addons
from secubox_toolbox import social as _social
except Exception: # pragma: no cover — when running outside the toolbox tree
_social = None
# Hash + deny-list helpers live in secubox_toolbox.social so they're
# unit-testable on the host side without spinning a mitmproxy flow.
log = logging.getLogger("secubox.toolbox.addon.social_graph")
# ─── eTLD+1 (registrable-domain) approximation ───
# Phase 11.A ships a regex-based approximation that covers > 99 % of the
# tracker / publisher ecosystem we see on a French civic kiosk. A
# proper publicsuffix.org tree fold lands in Phase A.1 if needed.
_MULTI_LABEL_TLDS = {
"co.uk", "ac.uk", "gov.uk", "org.uk", "net.uk",
"co.jp", "ne.jp", "ac.jp",
"com.au", "net.au", "org.au",
"com.br", "com.cn", "com.hk", "com.tw", "com.mx",
}
def _registrable_domain(host: str) -> str:
"""Cheap eTLD+1 extraction. e.g.
`www.lemonde.fr` `lemonde.fr`
`cdn.api.example.co.uk` `example.co.uk`
`tracker.com` `tracker.com`
"""
h = (host or "").lower().strip(".")
if not h or h.replace(".", "").isdigit(): # raw IP
return h
parts = h.split(".")
if len(parts) < 2:
return h
last_two = ".".join(parts[-2:])
if last_two in _MULTI_LABEL_TLDS and len(parts) >= 3:
return ".".join(parts[-3:])
return last_two
# ─── peer identity ───
# Phase 11.A originally tried `from . import local_store` which silently
# failed because mitmproxy loads addons as top-level modules (not as
# package members), so the relative import never resolved. Inlined
# here — only the R3 path (peer IP in 10.99.1.0/24 → WG pubkey hash)
# since Phase B is R3-only. R2 captive lookup remains in local_store
# and joins later when the addon is wired into the captive mitm.
import hashlib as _hashlib
import json as _json
from pathlib import Path as _Path
_WG_PEERS_DB = _Path("/var/lib/secubox/toolbox/wg-peers.json")
_WG_PEERS_CACHE: dict = {}
_WG_PEERS_MTIME: float = 0.0
def _wg_hash_of(ip: str) -> Optional[str]:
global _WG_PEERS_MTIME
try:
if not _WG_PEERS_DB.exists():
return None
mtime = _WG_PEERS_DB.stat().st_mtime
if mtime != _WG_PEERS_MTIME or not _WG_PEERS_CACHE:
data = _json.loads(_WG_PEERS_DB.read_text()).get("peers", {})
_WG_PEERS_CACHE.clear()
for pubkey, meta in data.items():
peer_ip = meta.get("ip")
if peer_ip:
_WG_PEERS_CACHE[peer_ip] = _hashlib.sha256(
pubkey.encode()
).hexdigest()[:16]
_WG_PEERS_MTIME = mtime
return _WG_PEERS_CACHE.get(ip)
except Exception:
return None
def _client_mac_hash(flow) -> Optional[str]:
try:
if flow.client_conn and flow.client_conn.peername:
ip = flow.client_conn.peername[0]
if ip and ip.startswith("10.99.1."):
return _wg_hash_of(ip)
except Exception:
pass
return None
# ─── cookie parsers ───
_SET_COOKIE_NAMEVAL = re.compile(r"^\s*([^=;]+)\s*=\s*([^;]*)")
_COOKIE_PAIR = re.compile(r"\s*([^=;]+)\s*=\s*([^;]*)")
def _parse_set_cookie(header: str) -> Optional[tuple]:
"""Return (name, value) for a Set-Cookie header, or None on garbage."""
m = _SET_COOKIE_NAMEVAL.match(header or "")
if not m:
return None
name = m.group(1).strip()
value = m.group(2).strip()
if not name:
return None
return name, value
def _parse_cookie_header(header: str) -> List[tuple]:
"""Return [(name, value), …] for a Cookie header (browser→server)."""
out: List[tuple] = []
for part in (header or "").split(";"):
m = _COOKIE_PAIR.match(part)
if m:
name = m.group(1).strip()
value = m.group(2).strip()
if name:
out.append((name, value))
return out
# ─── JA4 lookup ───
def _ja4_hash(flow) -> Optional[str]:
"""Pull the JA4 fingerprint set by the ja4 addon, if present."""
try:
ja4 = (flow.metadata or {}).get("ja4")
if ja4:
return str(ja4)[:32]
except Exception:
pass
return None
# ─── consent-platform detection (Phase 11.C #508) ───
#
# We detect the four dominant CMP (Consent Management Platform) cookies.
# Their PRESENCE on a flow means the user has interacted with the consent
# banner on this site at least once. We track, per (peer, site) :
# - whether the site is KNOWN to run a CMP (we've seen the cookie OR the
# loader path), and
# - whether a consent cookie has been observed yet.
# This lets us classify each tracker edge as pre/post/none-consent.
#
# Names are matched case-insensitively as a prefix (OneTrust appends
# region suffixes, Didomi rotates tokens, etc.).
_CMP_COOKIE_PREFIXES = (
"optanonconsent", "onetrustconsent", "optanonalertboxclosed", # OneTrust
"didomi_token", "euconsent-v2", # Didomi / IAB TCF
"__qca", "quantcast", # Quantcast
"sp_choice", "consentuid", "_sp_", # Sourcepoint
)
# CMP loader URL fragments — seeing the site REQUEST one of these proves
# the site runs a consent platform even before the cookie is set.
_CMP_LOADER_FRAGMENTS = (
"cdn.cookielaw.org", "onetrust.com", # OneTrust
"sdk.privacy-center.org", "didomi.io", # Didomi
"quantcast.mgr.consensu.org", "quantcast.com/choice", # Quantcast
"sourcepoint.mgr.consensu.org", "sp-prod.net", # Sourcepoint
)
# Per-(peer, site) consent observation log. Bounded soft-cap : if it
# grows past 20k entries we drop it wholesale (a fresh session rebuild is
# cheap and the salt rotates daily anyway).
_consent_log: dict = {}
def _consent_key(mac_hash: str, site: str) -> tuple:
return (mac_hash, site)
def _update_consent_log(mac_hash: str, src_site: str, flow) -> None:
"""Observe whether this flow reveals a CMP cookie or loader for the
(peer, site) pair, and update the in-memory log."""
try:
if len(_consent_log) > 20000:
_consent_log.clear()
key = _consent_key(mac_hash, src_site)
state = _consent_log.get(key, {"has_cmp": False, "consented": False})
# 1) CMP loader request → the site runs a consent platform.
url = (flow.request.pretty_url or "").lower()
if any(frag in url for frag in _CMP_LOADER_FRAGMENTS):
state["has_cmp"] = True
# 2) CMP cookie present (either direction) → consent recorded.
cookie_blobs = []
cookie_blobs.extend(flow.request.headers.get_all("cookie") or [])
cookie_blobs.extend(flow.response.headers.get_all("set-cookie") or [])
for blob in cookie_blobs:
low = blob.lower()
for pref in _CMP_COOKIE_PREFIXES:
if pref in low:
state["has_cmp"] = True
state["consented"] = True
break
_consent_log[key] = state
except Exception:
pass
def _consent_state_for(mac_hash: str, site: str) -> str:
"""Classify the current consent state for the (peer, site) pair.
Evidence-only semantics (no interpretation beyond the observation) :
post_consent a CMP cookie has been seen here for this peer.
pre_consent the site runs a CMP (loader/cookie seen) but no
consent cookie yet : a tracker firing now fires
before consent.
none_seen no CMP detected at all ; we make no claim.
"""
st = _consent_log.get(_consent_key(mac_hash, site))
if not st:
return "none_seen"
if st.get("consented"):
return "post_consent"
if st.get("has_cmp"):
return "pre_consent"
return "none_seen"
# ─── CDN / edge-network detection (Phase 12.A #515) ───
# Passive : classify the CDN fronting a host from its response headers.
# A shared CDN seeing the same browser across sites is itself a
# cross-site correlation surface — worth surfacing as its own lens.
def detect_cdn(headers) -> tuple:
"""Return (cdn_vendor, cache_status) from response headers, best-effort.
`headers` is mitmproxy's Headers (case-insensitive .get). Returns
(None, None) when no CDN signature is present.
"""
def g(name):
try:
return (headers.get(name) or "").lower()
except Exception:
return ""
server = g("server")
via = g("via")
x_cache = g("x-cache")
x_served = g("x-served-by")
# Cloudflare
if g("cf-ray") or g("cf-cache-status") or "cloudflare" in server:
return "Cloudflare", (g("cf-cache-status") or x_cache or None)
# Fastly (Varnish-based, x-served-by cache nodes)
if "fastly" in via or "fastly" in x_served or ("cache-" in x_served):
return "Fastly", (x_cache or None)
# Akamai
if "akamaighost" in server or g("x-akamai-transformed") or g("x-akamai-request-id"):
return "Akamai", (x_cache or None)
# Amazon CloudFront
if g("x-amz-cf-id") or "cloudfront" in via or "cloudfront" in x_cache:
return "CloudFront", (x_cache or None)
# Google edge
if "google" in via or g("x-goog-generation") or g("x-goog-meta"):
return "Google", None
# Vercel / Netlify / Bunny / KeyCDN / Sucuri / Imperva
if g("x-vercel-cache") or "vercel" in server:
return "Vercel", (g("x-vercel-cache") or None)
if g("x-nf-request-id") or "netlify" in server:
return "Netlify", None
if g("cdn-cache") or "bunnycdn" in server or g("x-bunny-cache"):
return "BunnyCDN", None
if "keycdn" in server or g("x-edge-location"):
return "KeyCDN", None
if g("x-sucuri-id") or g("x-sucuri-cache"):
return "Sucuri", (g("x-sucuri-cache") or None)
if g("x-iinfo") or "incapsula" in g("set-cookie"):
return "Imperva/Incapsula", None
# Generic edge-cache tell (x-cache HIT/MISS without a known vendor)
if x_cache or x_served:
return "edge-cache", (x_cache or None)
return None, None
# ─── anti-bot / "prove you're human" detection (Phase 12.B #516) ───
# Passive : classify the bot-checker / CAPTCHA vendor that fronts a host,
# from request URL fragments + cookies + response headers. Detection
# only — the (legally-sensitive) bypass half is gated behind its own
# doctrine. Vendor is host-stable ; stored like cdn_vendor.
_ANTIBOT_URL = (
("reCAPTCHA", ("/recaptcha/", "google.com/recaptcha", "gstatic.com/recaptcha")),
("hCaptcha", ("hcaptcha.com", "newassets.hcaptcha.com", "imgs.hcaptcha.com")),
("Turnstile", ("challenges.cloudflare.com",)),
("Datadome", ("captcha-delivery.com", "datadome.co", "ct.captcha-delivery.com")),
("PerimeterX/HUMAN", ("px-cdn.net", "perimeterx.net", "pxchk", "px-cloud.net")),
("Arkose/FunCaptcha", ("arkoselabs.com", "funcaptcha.com", "arkose-labs")),
("Kasada", ("kasada.io", "ct.kasada", "kpsdk")),
)
_ANTIBOT_COOKIE = (
("Turnstile", ("__cf_bm", "cf_chl_")),
("Datadome", ("datadome",)),
("PerimeterX/HUMAN", ("_px", "_pxhd", "_pxvid", "_pxff")),
("Akamai-BotManager", ("ak_bmsc", "bm_sz", "_abck", "bm_mi")),
("hCaptcha", ("hcaptcha",)),
)
_ANTIBOT_HEADER = (
("Datadome", ("x-datadome", "x-dd-b")),
("Kasada", ("x-kpsdk-ct", "x-kpsdk-cd")),
("PerimeterX/HUMAN", ("x-px",)),
)
def detect_antibot(flow) -> Optional[str]:
"""Return the anti-bot / CAPTCHA vendor challenging this flow, or None.
Best-effort, passive. Scans the request URL, the response + request
headers, and cookie names.
"""
try:
url = (flow.request.pretty_url or "").lower()
for vendor, frags in _ANTIBOT_URL:
if any(f in url for f in frags):
return vendor
# Response headers
rh = flow.response.headers if flow.response else None
if rh is not None:
keys = " ".join(k.lower() for k in rh.keys())
for vendor, hdrs in _ANTIBOT_HEADER:
if any(h in keys for h in hdrs):
return vendor
# Cookie names (both directions)
blobs = []
try:
blobs.extend(flow.request.headers.get_all("cookie") or [])
except Exception:
pass
try:
if flow.response:
blobs.extend(flow.response.headers.get_all("set-cookie") or [])
except Exception:
pass
joined = " ".join(blobs).lower()
if joined:
for vendor, names in _ANTIBOT_COOKIE:
if any(n in joined for n in names):
return vendor
except Exception:
pass
return None
# ─── JA4 lookup ───
def _ja4_hash(flow) -> Optional[str]:
"""Pull the JA4 fingerprint set by the ja4 addon, if present."""
try:
ja4 = (flow.metadata or {}).get("ja4")
if ja4:
return str(ja4)[:32]
except Exception:
pass
return None
# ─── main ───
class SocialGraph:
"""mitmproxy addon : record cookie-bearing edges per R2/R3 peer."""
def response(self, flow: http.HTTPFlow) -> None:
if not flow.response or _social is None:
return
mac_hash = _client_mac_hash(flow)
if not mac_hash:
return
src_site = _registrable_domain(flow.request.host)
if not src_site:
return
# Phase 11.C (#508) — consent-state detection. Before recording
# any edge, update our per-peer × per-site consent log : has a
# consent-platform cookie (OneTrust / Didomi / Quantcast /
# Sourcepoint) been observed for this peer on this site yet ?
# The edge's consent_state is then :
# post_consent — a consent cookie was already seen here
# pre_consent — NOT yet seen, AND the site DOES run a consent
# platform somewhere (so a tracker firing now is
# firing before consent — RGPD art. 6.1.a + 7)
# none_seen — no consent platform detected for this site at
# all (we make no claim ; baseline)
_update_consent_log(mac_hash, src_site, flow)
consent_state = _consent_state_for(mac_hash, src_site)
# Phase 12.A (#515) — passive CDN detection on the responding host.
# Phase 12.B (#516) — anti-bot / CAPTCHA vendor detection. Both
# host-stable, mac-independent ; upserted off-thread. Anti-bot
# is recorded for the responding host AND, when the challenge
# surface is a 3rd-party (e.g. captcha-delivery.com), attributed
# to the 1st-party site too via record_antibot_site so the
# per-client "challenged your humanity" alert is accurate.
try:
resp_host = _registrable_domain(flow.request.host)
antibot = detect_antibot(flow)
if resp_host and resp_host != src_site:
cdn_vendor, cache_status = detect_cdn(flow.response.headers)
if cdn_vendor:
_social.record_host_cdn(
domain=resp_host,
cdn_vendor=cdn_vendor,
cache_status=cache_status,
)
if antibot and resp_host:
_social.record_host_antibot(domain=resp_host, antibot_vendor=antibot)
# Attribute the challenge to the 1st-party site the user
# was on (for the per-client alert), keyed by mac_hash.
_social.record_antibot_challenge(
client_mac_hash=mac_hash, src_site=src_site,
antibot_vendor=antibot,
)
except Exception:
pass
# Set-Cookie headers : the 3rd-party server hands a new identifier.
# The Set-Cookie domain may differ from flow.request.host (Set-Cookie
# `Domain=` attribute) — when present, we trust that for the
# tracker_domain ; else fall back to the request host's eTLD+1.
ja4 = _ja4_hash(flow)
for sc in (flow.response.headers.get_all("set-cookie") or [])[:50]:
parsed = _parse_set_cookie(sc)
if not parsed:
continue
name, value = parsed
if _social.is_deny_listed(name):
continue
# `Domain=…` attribute parsing
domain_attr = _extract_domain_attr(sc)
tracker_domain = _registrable_domain(domain_attr or flow.request.host)
if not tracker_domain or tracker_domain == src_site:
# 1st-party Set-Cookie : not a cross-site tracker signal.
continue
cid = _social.cookie_id_hash(tracker_domain, name, value)
_social.record_edge(
client_mac_hash=mac_hash,
src_site=src_site,
tracker_domain=tracker_domain,
cookie_id_hash_val=cid,
ja4_hash=ja4,
consent_state=consent_state,
)
# Request-side Cookie headers (only meaningful when the
# request was for a 3rd-party domain : a 1st-party context
# browsing site X sends Cookie headers to embedded tracker T).
cookie_hdrs = flow.request.headers.get_all("cookie") or []
if not cookie_hdrs:
return
# The host we're sending the Cookie to is the tracker (because
# this is a 3rd-party request). We use the Referer / Origin
# to figure out which 1st-party context originated the load.
tracker_domain = _registrable_domain(flow.request.host)
if not tracker_domain:
return
ctx_site = _src_site_from_referer(flow)
if not ctx_site or ctx_site == tracker_domain:
# Either no Referer (direct nav, not interesting for
# cross-site mapping) or self-referential (the tracker
# called itself).
return
ctx_consent = _consent_state_for(mac_hash, ctx_site)
for hdr in cookie_hdrs[:5]:
for name, value in _parse_cookie_header(hdr)[:50]:
if _social.is_deny_listed(name):
continue
cid = _social.cookie_id_hash(tracker_domain, name, value)
_social.record_edge(
client_mac_hash=mac_hash,
src_site=ctx_site,
tracker_domain=tracker_domain,
cookie_id_hash_val=cid,
ja4_hash=ja4,
consent_state=ctx_consent,
)
_DOMAIN_ATTR_RE = re.compile(r"(?i);\s*domain\s*=\s*([^;]+)")
def _extract_domain_attr(set_cookie_header: str) -> Optional[str]:
"""Pull the `; Domain=…` attribute from a Set-Cookie line, if any."""
m = _DOMAIN_ATTR_RE.search(set_cookie_header or "")
if not m:
return None
return m.group(1).strip().lstrip(".").lower() or None
def _src_site_from_referer(flow) -> Optional[str]:
"""Derive the 1st-party site for a 3rd-party request.
Tries Referer first, then Origin. Returns the registrable domain
of the originating page, NOT the tracker host itself.
"""
ref = flow.request.headers.get("referer") or flow.request.headers.get("origin")
if not ref:
return None
# Strip scheme + path.
s = ref.split("://", 1)[-1].split("/", 1)[0].split("?", 1)[0]
return _registrable_domain(s)
addons = [SocialGraph()]

View File

@ -18,6 +18,14 @@ BYPASS_FILE=/var/lib/secubox/toolbox/mitm-bypass.conf
DYNAMIC_FILE=/var/lib/secubox/toolbox/mitm-bypass-dynamic.conf # noqa: SC2034 (used by addon hint) DYNAMIC_FILE=/var/lib/secubox/toolbox/mitm-bypass-dynamic.conf # noqa: SC2034 (used by addon hint)
ADDON_DIR=/usr/lib/secubox/toolbox/mitmproxy_addons ADDON_DIR=/usr/lib/secubox/toolbox/mitmproxy_addons
# Phase 11.A (#505) — make the secubox_toolbox package importable from
# addons running inside /opt/mitmproxy-toolbox/bin/python3 (a venv that
# has its own site-packages and never sees /usr/lib/secubox/toolbox).
# Without this PYTHONPATH every addon's `from secubox_toolbox import …`
# silently degrades — inject_banner loses host classification + GeoIP,
# social_graph loses its SQLite store reference, etc.
export PYTHONPATH="/usr/lib/secubox/toolbox${PYTHONPATH:+:$PYTHONPATH}"
# ── Compose ignore_hosts regex : merge static + dynamic bypass lists ── # ── Compose ignore_hosts regex : merge static + dynamic bypass lists ──
# Phase 6.N (#496) : the dynamic file is auto-populated by the cert_pin_detect # Phase 6.N (#496) : the dynamic file is auto-populated by the cert_pin_detect
# addon when it observes repeated TLS handshake failures (cert pinning). # addon when it observes repeated TLS handshake failures (cert pinning).
@ -91,8 +99,12 @@ fi
# - utiq_defense (Phase 8 #500) runs at requestheaders too ; placed # - utiq_defense (Phase 8 #500) runs at requestheaders too ; placed
# EARLY so a R1 block short-circuits the flow before downstream # EARLY so a R1 block short-circuits the flow before downstream
# addons spend cycles on it # addons spend cycles on it
# - social_graph (Phase 11.A #505) sees the same cookie state as
# local_store but records cross-site identifiers separately ; it
# must run BEFORE inject_banner so the banner cookies our addon
# emits don't pollute the graph
# - cert_pin_detect auto-learns pinned hosts (Phase 6.N) # - cert_pin_detect auto-learns pinned hosts (Phase 6.N)
for addon in inject_xff utiq_defense local_store inject_banner dpi cookies avatar ja4 soc_relay cert_pin_detect; do for addon in inject_xff utiq_defense local_store social_graph inject_banner dpi cookies avatar ja4 soc_relay cert_pin_detect; do
ARGS+=(-s "$ADDON_DIR/${addon}.py") ARGS+=(-s "$ADDON_DIR/${addon}.py")
done done

View File

View File

@ -2034,6 +2034,259 @@ async def admin_utiq_events(hours: int = 24, limit: int = 200) -> dict:
} }
# ───────────────── Phase 11.A — Social Mapping (#505, parent #502) ─────────────────
@router.get("/social/graph/{token}")
async def social_graph_client(token: str, since: int = 86400) -> dict:
"""Per-client cross-cookie tracker graph.
`token` is the same HMAC-signed report token used by /report/{token}
(see reports.verify_token). Same TTL semantics : 24 h then the
salt rotation invalidates further access.
The returned shape is the JSON contract consumed by the d3 view in
Phase 11.B and by the bilingual PDF in Phase 11.C.
"""
from . import social as _s
salt = _get_salt()
ok, mac_hash = reports.verify_token(token, salt)
if not ok:
raise HTTPException(404, "graph not found or expired")
# Clamp the window between 1 h and 7 d so a bad query can't
# walk the entire retention period.
since = max(3600, min(int(since or 86400), 7 * 86400))
return _s.fetch_graph(mac_hash, since_seconds=since)
@router.post("/social/wipe/{token}")
async def social_wipe_client(token: str) -> dict:
"""RGPD art. 17 — droit à l'effacement.
HMAC-token-gated to the same identity as the per-client view.
Deletes every row across social_edges + social_nodes + social_links
for the resolved mac_hash.
"""
from . import social as _s
salt = _get_salt()
ok, mac_hash = reports.verify_token(token, salt)
if not ok:
raise HTTPException(404, "graph not found or expired")
rows = _s.wipe_mac(mac_hash)
return {"wiped": True, "rows_deleted": rows, "mac_hash_prefix": mac_hash[:8]}
@router.get("/admin/social-aggregate")
async def admin_social_aggregate(hours: int = 24) -> dict:
"""Operator dashboard view : KPI tiles + tracker-domain histogram +
anonymized client table. No per-client graph at this endpoint
(that lives behind the HMAC-token route above).
"""
from . import social as _s
return _s.aggregate(hours=hours)
@router.get("/social/report/{token}.pdf")
async def social_report_pdf(token: str) -> Response:
"""Phase 11.C (#508) — bilingual FR/EN evidence PDF for a peer.
Same HMAC-token gate as /social/graph/{token}. Fact-only report :
cross-site tracker reuse, trackers fired before consent (RGPD art.
6.1.a + 7), extra-EU transfers (art. 44+).
"""
from . import social_report as _sr
salt = _get_salt()
ok, mac_hash = reports.verify_token(token, salt)
if not ok:
raise HTTPException(404, "report not found or expired")
data = _sr.build_social_report(mac_hash, since_seconds=7 * 86400)
data["generated_at"] = time.strftime("%Y-%m-%d %H:%M UTC", time.gmtime())
pdf_bytes = _sr.render_social_pdf(data)
fname = f"village3b-carto-{mac_hash[:8]}-{int(time.time())}.pdf"
return Response(
content=pdf_bytes,
media_type="application/pdf",
headers={"Content-Disposition": f'attachment; filename="{fname}"'},
)
# ───────────────── Phase 11.B — per-client view + favicon proxy (#507) ─────────────────
import json as _json_b
def _load_social_i18n(lang: str) -> dict:
"""Pick the FR or EN dictionary. FR is the source-of-truth."""
lang = (lang or "").lower().split(",", 1)[0].split("-", 1)[0]
if lang not in ("en", "fr"):
lang = "fr"
path = TEMPLATE_DIR / "i18n" / f"social.{lang}.json"
try:
return _json_b.loads(path.read_text(encoding="utf-8"))
except Exception as e:
log.warning("social i18n load failed (%s): %s", lang, e)
return {}
@router.get("/social/me")
async def social_view_me(request: Request) -> RedirectResponse:
"""Self-resolving entry point used by the kbin splash menu.
Resolution chain (matches /report/me/html):
1. ?mh=<hash> explicit R3 hash in URL (e.g. from banner)
2. X-R3-Peer header mitm-wg's inject_xff sentinel for transparent
R3 flows (peer IP in 10.99.1.0/24)
3. _resolve() ARP lookup R2 captive subnet clients only
Mints a short-TTL (1 h) HMAC token bound to the resolved mac_hash and
redirects to /social/{token}. Keeps a single HMAC-token-gated code
path for the graph view + wipe endpoint.
"""
salt = _get_salt()
mac_hash = None
# 1) ?mh= overrides everything (used by inject_banner.py links + the
# report flow + manual curl).
mh_qp = (request.query_params.get("mh") or "").strip().lower()
if mh_qp and all(c in "0123456789abcdef" for c in mh_qp) and 8 <= len(mh_qp) <= 64:
mac_hash = mh_qp
# 2) X-R3-Peer (transparent R3 — the iPhone hits kbin via the R3
# tunnel, mitm-wg adds the sentinel, HAProxy passes it through).
# We derive the same mac_hash the local_store addon uses :
# sha256(wg_pubkey)[:16] looked up from /var/lib/secubox/toolbox/
# wg-peers.json keyed by peer IP.
if not mac_hash:
peer_ip = _client_ip(request)
if peer_ip and peer_ip.startswith("10.99.1."):
try:
import hashlib as _h
import json as _j
from pathlib import Path as _P
_DB = _P("/var/lib/secubox/toolbox/wg-peers.json")
if _DB.exists():
peers = _j.loads(_DB.read_text()).get("peers", {})
for pubkey, meta in peers.items():
if meta.get("ip") == peer_ip:
mac_hash = _h.sha256(pubkey.encode()).hexdigest()[:16]
break
except Exception as e:
log.warning("/social/me wg-peer lookup failed: %s", e)
# 3) R2 captive — ARP _resolve().
if not mac_hash:
_ip, mac = _resolve(request)
if mac:
mac_hash = macmod.hash_mac(mac, salt)
if not mac_hash:
raise HTTPException(
400,
"client identity unresolved (not on R3 tunnel and not in "
"captive subnet) — append ?mh=<hash> from your banner's "
"report link",
)
tok = reports.mint_token(mac_hash, salt, ttl_seconds=3600)
lang = request.query_params.get("lang") or ""
suffix = f"?lang={lang}" if lang else ""
return RedirectResponse(url=f"/social/{tok.token}{suffix}", status_code=303)
@router.get("/social/{token}", response_class=HTMLResponse)
async def social_view(token: str, request: Request) -> HTMLResponse:
"""Per-client social mapping HTML page.
HMAC-token-gated identically to /report/{token} so the same kbin
splash + R3 onboard flow can issue a link. The page itself
contains no per-client data the d3 layer fetches it via
GET /social/graph/{token} after page-load (so a forwarded link
that's been TTL-expired surfaces an empty graph + error message
in the page rather than a 404 at HTTP layer).
"""
salt = _get_salt()
ok, _mac_hash = reports.verify_token(token, salt)
if not ok:
raise HTTPException(404, "page not found or expired")
# Language pick : query string `?lang=` wins, else Accept-Language.
lang = request.query_params.get("lang") or request.headers.get("accept-language", "fr")
lang = "fr" if "fr" in lang.lower() else "en"
i18n = _load_social_i18n(lang)
html = _env.get_template("social_view.html.j2").render(
token=token,
lang=lang,
t=i18n,
t_json=_json_b.dumps(i18n, ensure_ascii=False),
)
return HTMLResponse(content=html)
# Favicon cache (Phase 11.B) — server-side cache so the per-client
# view never makes a client-side request to a tracker domain just to
# show its icon. Default TTL 7 d.
_FAVICON_CACHE_DIR = Path("/var/lib/secubox/toolbox/favicons")
_FAVICON_TTL_SECONDS = 7 * 86400
@router.get("/social/favicon/{domain}")
async def social_favicon(domain: str) -> Response:
"""Server-side favicon proxy with 7 d disk cache.
Path traversal is impossible because `domain` is whitelisted to
a strict charset and the cache file is sha256-keyed. We fetch
over HTTPS only, time out hard at 5 s, and fall back to a 1×1
transparent gif on any failure.
"""
import hashlib as _h
import re as _re
import time as _time
if not _re.match(r"^[a-z0-9.-]{1,253}$", domain.lower()):
raise HTTPException(400, "invalid domain")
_FAVICON_CACHE_DIR.mkdir(parents=True, exist_ok=True)
key = _h.sha256(domain.lower().encode()).hexdigest()[:24]
cache_file = _FAVICON_CACHE_DIR / f"{key}.ico"
now = _time.time()
if cache_file.exists() and now - cache_file.stat().st_mtime < _FAVICON_TTL_SECONDS:
return Response(
content=cache_file.read_bytes(),
media_type="image/x-icon",
headers={"Cache-Control": "public, max-age=86400"},
)
# Best-effort fetch. No cross-host redirects, no JS, no cookies.
try:
import httpx
url = f"https://{domain.lower()}/favicon.ico"
async with httpx.AsyncClient(timeout=5.0, follow_redirects=False) as c:
r = await c.get(
url,
headers={"User-Agent": "SecuBox-ToolBox-Favicon-Cache/1.0"},
)
if r.status_code == 200 and r.content and len(r.content) < 65536:
cache_file.write_bytes(r.content)
return Response(
content=r.content,
media_type="image/x-icon",
headers={"Cache-Control": "public, max-age=86400"},
)
except Exception as e: # noqa: BLE001
log.warning("favicon fetch failed for %s: %s", domain, e)
# Fallback : 1×1 transparent gif.
GIF = (
b"GIF89a\x01\x00\x01\x00\x80\x00\x00\xff\xff\xff\x00\x00\x00"
b"!\xf9\x04\x01\x00\x00\x00\x00,\x00\x00\x00\x00\x01\x00\x01\x00\x00\x02\x02L\x01\x00;"
)
return Response(
content=GIF,
media_type="image/gif",
headers={"Cache-Control": "public, max-age=300"},
)
@router.get("/admin/config") @router.get("/admin/config")
async def admin_config() -> dict: async def admin_config() -> dict:
return _get_cfg().model_dump() return _get_cfg().model_dump()
@ -2123,235 +2376,9 @@ async def admin_override_level(mac_hash: str, request: Request) -> dict:
"note": "nft sets not auto-updated; client must reload or operator manually adjusts nft"} "note": "nft sets not auto-updated; client must reload or operator manually adjusts nft"}
@router.get("/admin/", response_class=HTMLResponse) # Phase 11.B+ (#513) — the inline kbin /admin/ HTML admin UI was removed.
@router.get("/admin", response_class=HTMLResponse) # The canonical operator dashboard is admin.gk2.secubox.in/toolbox/ (sub-tab
async def admin_index() -> HTMLResponse: # WebUI in www/toolbox/index.html). All /admin/* JSON API routes below stay.
"""Operator admin webUI with client list + level switcher."""
html = """<!DOCTYPE html><html lang=fr><head><meta charset=UTF-8>
<meta name=viewport content="width=device-width,initial-scale=1">
<title>🛡 Admin Gondwana ToolBoX</title>
<style>:root{--bg:#0a0a0f;--bg2:#0e0e15;--phos:#00dd44;--phos-hot:#00ff55;--dim:#006622;--text:#e8e6d9;--purple:#9e76ff;--amber:#ffb347;--red:#ff4466}
*{box-sizing:border-box;margin:0;padding:0}
body{font-family:Menlo,Consolas,monospace;background:var(--bg);color:var(--text);padding:1.2rem;max-width:1100px;margin:auto;line-height:1.55}
h1{color:var(--phos-hot);text-shadow:0 0 6px var(--phos);font-size:1.6rem;margin-bottom:0.4rem;letter-spacing:0.05em}
.sub{color:var(--dim);font-size:0.85rem;margin-bottom:1.2rem}
table{width:100%;border-collapse:collapse;font-size:0.85rem;background:var(--bg2);border:1px solid var(--dim)}
th,td{padding:0.5rem 0.6rem;text-align:left;border-bottom:1px solid var(--dim)}
th{color:var(--phos-hot);text-shadow:0 0 3px var(--phos);background:rgba(0,221,68,0.08)}
tr:hover{background:rgba(0,221,68,0.04)}
.chip{display:inline-block;padding:0.15rem 0.5rem;border-radius:99px;font-size:0.72rem;font-weight:bold}
.chip.r0{background:#222;color:#999}
.chip.r1{background:rgba(0,221,68,0.2);color:var(--phos-hot)}
.chip.r2{background:rgba(255,179,71,0.2);color:#ffd6a0}
.chip.r3{background:rgba(158,118,255,0.2);color:#cbb6ff}
.btn{background:var(--purple);color:#0a0a0f;padding:0.3rem 0.6rem;border:none;border-radius:3px;cursor:pointer;font-family:inherit;font-size:0.72rem;font-weight:bold;margin-right:0.2rem}
.btn:hover{background:#b598ff}
.btn.outline{background:transparent;color:var(--phos);border:1px solid var(--phos)}
code{background:#222;padding:0.1rem 0.3rem;border-radius:2px;font-size:0.75rem}
.cards{display:grid;grid-template-columns:repeat(auto-fit,minmax(180px,1fr));gap:0.8rem;margin-bottom:1.5rem}
.card{background:var(--bg2);border:1px solid var(--dim);padding:0.8rem;border-radius:4px;text-align:center}
.card .v{font-size:1.6rem;color:var(--phos-hot);font-weight:bold;display:block}
.card .l{font-size:0.7rem;color:var(--dim)}
.modal-bg{position:fixed;top:0;left:0;right:0;bottom:0;background:rgba(0,0,0,0.8);display:none;align-items:center;justify-content:center;z-index:100}
.modal-bg.show{display:flex}
.modal{background:var(--bg2);border:1px solid var(--phos);padding:1.5rem;border-radius:4px;max-width:400px;width:90%}
.modal h2{color:var(--phos-hot);font-size:1rem;margin-bottom:0.8rem}
.modal .lvl-row{display:grid;grid-template-columns:repeat(4,1fr);gap:0.4rem;margin-bottom:0.8rem}
.modal .lvl-row button{padding:0.5rem;cursor:pointer;font-family:inherit;font-size:0.75rem;border-radius:3px}
.modal .lvl-row .r0{background:#222;color:#999;border:1px solid #444}
.modal .lvl-row .r1{background:rgba(0,221,68,0.15);color:var(--phos-hot);border:1px solid var(--phos)}
.modal .lvl-row .r2{background:rgba(255,179,71,0.15);color:#ffd6a0;border:1px solid var(--amber)}
.modal .lvl-row .r3{background:rgba(158,118,255,0.15);color:#cbb6ff;border:1px solid var(--purple)}
.adm-tabs{display:flex;gap:2px;border-bottom:1px solid var(--dim);margin-bottom:0.8rem}
.adm-tab{background:transparent;border:0;color:#888;padding:0.5rem 1rem;font-family:inherit;font-size:0.85rem;cursor:pointer;border-bottom:2px solid transparent;transition:all 0.15s}
.adm-tab.active{color:var(--phos-hot);border-bottom-color:var(--phos);background:rgba(0,221,68,0.05)}
.adm-tab:hover{color:var(--text)}
.adm-content{display:none}
.adm-content.active{display:block}
button.del-pattern{background:var(--red);color:#fff;border:0;padding:0.2rem 0.5rem;border-radius:3px;cursor:pointer;font-weight:bold;font-size:0.75rem}
</style></head><body>
<h1>🛡 Admin Gondwana ToolBoX</h1>
<p class=sub>// Console opérateur · client management · level override · mitm filtering</p>
{# Phase 6.I : tabbed sections #}
<div class=adm-tabs>
<button class="adm-tab active" data-tab=clients>👥 Clients</button>
<button class="adm-tab" data-tab=filter>🛡 Mitm filtering</button>
</div>
<div class="adm-content active" data-content=clients>
<div id=cards class=cards></div>
<table id=clients-table>
<thead><tr>
<th>Status</th><th>Device</th><th>Hash</th><th>IP</th>
<th>Level</th><th>Risk</th><th>Last seen</th><th>Actions</th>
</tr></thead>
<tbody id=clients-tbody><tr><td colspan=8>chargement</td></tr></tbody>
</table>
</div>
<div class="adm-content" data-content=filter>
<h2 style="color:var(--purple);font-size:1.1rem;margin-bottom:0.4rem">🛡 Mitm bypass whitelist</h2>
<p style="font-size:0.78rem;color:var(--dim);margin-bottom:0.8rem">
Hosts/domains qui ne sont JAMAIS déchiffrés par mitm (TLS passthrough).
Pour apps cert-pinned ou E2E. Redémarre <code>secubox-toolbox-mitm-wg</code>
après modif.
</p>
<form id=add-pattern-form style="display:flex;gap:0.5rem;margin-bottom:0.8rem">
<input type=text name=entry placeholder="ex: (.+\\.)?example\\.com" required
style="flex:1;background:#111;color:var(--text);border:1px solid #2a2a3f;padding:0.5rem;border-radius:3px;font-family:monospace;font-size:0.8rem">
<button type=submit class=btn style="background:var(--phos);padding:0.5rem 1rem"> Ajouter</button>
</form>
<table id=filter-table style="font-size:0.78rem">
<thead><tr><th>Pattern (regex)</th><th width=60>×</th></tr></thead>
<tbody id=filter-tbody><tr><td colspan=2>chargement</td></tr></tbody>
</table>
<p style="font-size:0.72rem;color:var(--dim);margin-top:0.6rem;border-left:2px solid var(--amber);padding-left:0.6rem">
📁 <code>/var/lib/secubox/toolbox/mitm-bypass.conf</code> · Source de vérité
</p>
</div>
<div id=modal class=modal-bg>
<div class=modal>
<h2>🔀 Change level <code id=modal-mh></code></h2>
<p style="font-size:0.78rem;color:var(--dim);margin-bottom:0.5rem"> Admin override : updates store only. Client must reload to sync nft.</p>
<div class=lvl-row>
<button class=r0 onclick="setLevel('r0')">🌐 R0</button>
<button class=r1 onclick="setLevel('r1')">🛡 R1</button>
<button class=r2 onclick="setLevel('r2')">🔍 R2</button>
<button class=r3 onclick="setLevel('r3')">🌐 R3</button>
</div>
<button onclick="closeModal()" class=btn style="background:transparent;color:var(--dim);border:1px solid var(--dim)">Annuler</button>
</div>
</div>
<script>
var selectedMh = null;
function openModal(mh) {
selectedMh = mh;
document.getElementById('modal-mh').textContent = mh;
document.getElementById('modal').classList.add('show');
}
function closeModal() {
document.getElementById('modal').classList.remove('show');
}
async function setLevel(lvl) {
if (!selectedMh) return;
var fd = new FormData();
fd.append('level', lvl);
var r = await fetch('/admin/clients/'+selectedMh+'/level', {method:'POST', body:fd});
if (r.ok) {
closeModal();
loadClients();
} else {
alert('Erreur: '+r.status);
}
}
async function loadClients() {
var r = await fetch('/admin/clients/rich');
if (!r.ok) {
document.getElementById('clients-tbody').innerHTML = '<tr><td colspan=8>Erreur '+r.status+'</td></tr>';
return;
}
var data = await r.json();
var tbody = document.getElementById('clients-tbody');
tbody.innerHTML = '';
data.clients.forEach(function(c){
var lvlChip = '<span class="chip '+c.level+'">'+c.level_emoji+' '+c.level.toUpperCase()+'</span>';
var dt = new Date(c.last_seen*1000).toISOString().substring(11,16);
var tr = document.createElement('tr');
tr.innerHTML = '<td>'+c.status_emoji+' '+c.status_label+'</td>'+
'<td>'+c.device_emoji+'</td>'+
'<td><code>'+c.mac_hash.substring(0,12)+'…</code></td>'+
'<td>'+(c.ip||'?')+'</td>'+
'<td>'+lvlChip+'</td>'+
'<td>'+c.risk_emoji+' '+c.score+'</td>'+
'<td>'+dt+'</td>'+
'<td><button class=btn onclick="openModal(\\''+c.mac_hash+'\\')">🔀 Override</button>'+
'<a href="/admin/clients/'+c.mac_hash+'/report" class="btn outline">📄 PDF</a></td>';
tbody.appendChild(tr);
});
// KPI cards
var counts = {r0:0,r1:0,r2:0,r3:0,actif:0,risk_low:0,risk_mh:0};
data.clients.forEach(function(c){
counts[c.level] = (counts[c.level]||0)+1;
if (c.status_label==='actif') counts.actif++;
if (c.score < 30) counts.risk_low++;
if (c.score >= 30) counts.risk_mh++;
});
document.getElementById('cards').innerHTML =
'<div class=card><span class=v>'+data.count+'</span><span class=l>Total clients</span></div>'+
'<div class=card><span class=v>'+counts.actif+'</span><span class=l>🟢 Actifs (5min)</span></div>'+
'<div class=card><span class=v>'+counts.r1+'</span><span class=l>🛡 R1</span></div>'+
'<div class=card><span class=v>'+counts.r2+'</span><span class=l>🔍 R2</span></div>'+
'<div class=card><span class=v>'+counts.r3+'</span><span class=l>🌐 R3 WG</span></div>'+
'<div class=card><span class=v>'+counts.risk_low+'</span><span class=l>🟢 Risque LOW</span></div>';
}
loadClients();
setInterval(loadClients, 15000);
// Tabs (Phase 6.I)
document.querySelectorAll('.adm-tab').forEach(function(t){
t.addEventListener('click', function(){
var tn = this.dataset.tab;
document.querySelectorAll('.adm-tab').forEach(function(x){x.classList.remove('active');});
this.classList.add('active');
document.querySelectorAll('.adm-content').forEach(function(x){x.classList.remove('active');});
document.querySelector('[data-content='+tn+']').classList.add('active');
if (tn === 'filter') loadFilter();
});
});
// Filter Control (integrated tab)
async function loadFilter(){
var r = await fetch('/admin/filter-control/list');
var tb = document.getElementById('filter-tbody');
if (!r.ok) { tb.innerHTML = '<tr><td colspan=2>Erreur '+r.status+'</td></tr>'; return; }
var d = await r.json();
if (!d.patterns || !d.patterns.length) {
tb.innerHTML = '<tr><td colspan=2 style="color:#666;text-align:center">Aucune entrée</td></tr>';
return;
}
tb.innerHTML = '';
d.patterns.forEach(function(p){
var tr = document.createElement('tr');
var td1 = document.createElement('td');
var code = document.createElement('code');
code.textContent = p;
td1.appendChild(code);
var td2 = document.createElement('td');
var btn = document.createElement('button');
btn.className = 'del-pattern';
btn.textContent = '×';
btn.onclick = function(){ delPattern(p); };
td2.appendChild(btn);
tr.appendChild(td1); tr.appendChild(td2);
tb.appendChild(tr);
});
}
async function delPattern(p){
var fd = new FormData(); fd.append('entry', p);
var r = await fetch('/admin/filter-control/remove', {method:'POST', body:fd});
if (r.ok || r.status === 303) loadFilter();
}
document.getElementById('add-pattern-form').addEventListener('submit', async function(ev){
ev.preventDefault();
var entry = ev.target.entry.value.trim();
if (!entry) return;
var fd = new FormData(); fd.append('entry', entry);
var r = await fetch('/admin/filter-control/add', {method:'POST', body:fd});
if (r.ok || r.status === 303) {
ev.target.entry.value = '';
loadFilter();
}
});
</script>
</body></html>"""
return HTMLResponse(html, headers={"Cache-Control": "no-store"})
@router.get("/admin/filter-control/list") @router.get("/admin/filter-control/list")
@ -2422,6 +2449,45 @@ async def admin_client_report(mac_hash: str) -> Response:
) )
@router.get("/admin/clients/{mac_hash}/social")
async def admin_client_social(mac_hash: str, request: Request) -> RedirectResponse:
"""Phase 12.B (#516) — operator entry to a client's social mapping
graph from the toolbox WebUI Clients tab. Mints a short-TTL (1 h)
HMAC token for the mac_hash and 303-redirects to the per-client view.
The /social/ route is ONLY served on the kbin vhost (HAProxy the
toolbox uvicorn). When the operator triggers this from
admin.gk2.secubox.in, a relative redirect would land on the
aggregator's "missing module" page — so we build an ABSOLUTE kbin
URL by swapping the leading `admin.` host label for `kbin.`.
"""
salt = _get_salt()
tok = reports.mint_token(mac_hash, salt, ttl_seconds=3600)
host = (request.headers.get("host") or "").strip()
if host.startswith("admin."):
kbin_host = "kbin." + host[len("admin."):]
target = f"https://{kbin_host}/social/{tok.token}"
elif host.startswith("kbin."):
target = f"/social/{tok.token}" # already on kbin, relative is fine
else:
# Fallback : same-origin relative (dev / direct uvicorn).
target = f"/social/{tok.token}"
return RedirectResponse(url=target, status_code=303)
@router.post("/admin/clients/{mac_hash}/reset")
async def admin_client_reset(mac_hash: str) -> dict:
"""Phase 12.B (#516) — RAZ a specific client's accumulated statistics
from the operator WebUI. Wipes the social-mapping graph + the
toolbox events/consents/reports and zeroes the client score.
"""
from . import social as _s
rows = store.reset_client(mac_hash)
rows += _s.wipe_mac(mac_hash)
log.info("admin reset client %s: %d rows", mac_hash[:8], rows)
return {"ok": True, "rows_deleted": rows, "mac_hash_prefix": mac_hash[:8]}
@router.get("/admin/clients/{mac_hash}/events") @router.get("/admin/clients/{mac_hash}/events")
async def admin_client_events(mac_hash: str) -> dict: async def admin_client_events(mac_hash: str) -> dict:
"""Admin endpoint : per-source event summary for a specific client.""" """Admin endpoint : per-source event summary for a specific client."""

View File

@ -6,10 +6,12 @@ from __future__ import annotations
import asyncio import asyncio
import logging import logging
from pathlib import Path
from fastapi import FastAPI from fastapi import FastAPI
from fastapi.staticfiles import StaticFiles
from . import __version__, store, threat_intel from . import __version__, social, store, threat_intel
from .api import router as toolbox_router from .api import router as toolbox_router
_log = logging.getLogger("secubox.toolbox") _log = logging.getLogger("secubox.toolbox")
@ -26,6 +28,14 @@ app = FastAPI(
) )
app.include_router(toolbox_router) app.include_router(toolbox_router)
# Phase 11.B (#507) — serve the WebUI assets on the same origin as
# the FastAPI HTML pages. Required because the kbin vhost routes
# through HAProxy directly to uvicorn (bypassing nginx), so the
# nginx /toolbox/ alias never gets a chance to match.
_TOOLBOX_WWW = Path("/usr/share/secubox/www/toolbox")
if _TOOLBOX_WWW.is_dir():
app.mount("/toolbox", StaticFiles(directory=_TOOLBOX_WWW), name="toolbox-www")
@app.on_event("startup") @app.on_event("startup")
async def _startup() -> None: async def _startup() -> None:
@ -40,3 +50,24 @@ async def _startup() -> None:
asyncio.create_task(purge_loop()) asyncio.create_task(purge_loop())
# Threat-intel feeds : kick off immediate refresh + hourly loop # Threat-intel feeds : kick off immediate refresh + hourly loop
asyncio.create_task(threat_intel.refresh_loop()) asyncio.create_task(threat_intel.refresh_loop())
# Phase 11.A (#505) — social-mapping fold + retention purge.
# Fold every 5 min : raw edges → aggregate nodes + links.
# Purge raw edges older than 7 d once an hour.
async def social_fold_loop() -> None:
while True:
try:
social.fold_recent(window_seconds=600)
except Exception as e:
_log.error("social.fold_recent failed: %s", e)
await asyncio.sleep(300)
async def social_purge_loop() -> None:
while True:
try:
social.purge_older_than(days=7)
except Exception as e:
_log.error("social.purge_older_than failed: %s", e)
await asyncio.sleep(3600)
asyncio.create_task(social_fold_loop())
asyncio.create_task(social_purge_loop())

View File

@ -0,0 +1,875 @@
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
# Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
"""
SecuBox-Deb :: ToolBoX Social Mapping store
Phase 11.A (#505, parent #502) — per-device cross-cookie tracker
correlation engine. Captures Set-Cookie + Cookie headers from each
3rd-party flow during a consented R2 / R3 session, normalizes the
identifiers (NEVER persists raw cookie values), and folds raw edges
into per-peer node + link aggregates every 5 minutes.
Schema (3 tables):
- social_edges : raw event log, retention 7 d (configurable).
- social_nodes : per-peer per-tracker aggregate (hits, first/last
seen, list of sites the tracker connected this peer to).
- social_links : per-peer per-site-pair aggregate (which trackers
were shared, JA4 collision flag, last seen).
Identifier discipline (NON-negotiable, doctrine):
- client_mac_hash is computed from the existing rotating-salt MAC
hash (mac.py). 24 h rotation makes the graph unreachable past TTL.
- cookie_id_hash = sha256(tracker_domain || cookie_name || cookie_value)[:16].
Raw values are NEVER persisted.
- Session/CSRF/auth cookie names are stripped via the deny-list
(default constant + TOML override).
See also: utiq.py for the same fire-and-forget executor pattern.
"""
from __future__ import annotations
import concurrent.futures as _futures
import hashlib
import json
import logging
import sqlite3
import time
from pathlib import Path
from typing import Dict, Iterable, List, Optional, Set, Tuple
log = logging.getLogger("secubox.toolbox.social.store")
DB_PATH = Path("/var/lib/secubox/toolbox/toolbox.db")
# Cookie names whose presence on a flow is NEVER recorded as a tracker
# identifier. Phase 11.A ships a conservative default ; operators can
# extend via [social_graph.deny_cookies] in toolbox.toml.
_DEFAULT_DENY_COOKIES: Set[str] = {
# session
"phpsessid", "jsessionid", "asp.net_sessionid", "ci_session",
"express.sid", "connect.sid", "sails.sid", "django_session",
"laravel_session", "flask_session", "session", "sessionid",
# csrf
"_csrf", "_csrf_token", "xsrf-token", "csrftoken", "csrf",
"x-csrf-token", "anti-csrf-token",
# auth (1st-party)
"auth", "auth_token", "access_token", "refresh_token", "bearer",
"remember_token", "remember_me", "_oauth2_proxy",
# cloudflare / consent / locale (low signal)
"__cf_bm", "cf_clearance", "consent", "cookieconsent_status",
"locale", "lang", "language", "_locale",
}
_SCHEMA = """
CREATE TABLE IF NOT EXISTS social_edges (
id INTEGER PRIMARY KEY AUTOINCREMENT,
ts INTEGER NOT NULL,
client_mac_hash TEXT NOT NULL,
src_site TEXT NOT NULL,
tracker_domain TEXT NOT NULL,
cookie_id_hash TEXT NOT NULL,
ja4_hash TEXT,
-- Phase 11.C (#508) — consent state at the moment the edge was
-- recorded. Computed by the addon based on whether a consent
-- platform cookie (OneTrust/Didomi/Quantcast/Sourcepoint) has
-- already been observed for this peer × site pair.
consent_state TEXT NOT NULL DEFAULT 'none_seen'
);
CREATE INDEX IF NOT EXISTS idx_social_edges_mac_ts
ON social_edges(client_mac_hash, ts);
CREATE INDEX IF NOT EXISTS idx_social_edges_tracker_ts
ON social_edges(tracker_domain, ts);
CREATE TABLE IF NOT EXISTS social_nodes (
client_mac_hash TEXT NOT NULL,
tracker_domain TEXT NOT NULL,
hits INTEGER NOT NULL DEFAULT 0,
first_seen INTEGER NOT NULL,
last_seen INTEGER NOT NULL,
sites_jsonl TEXT NOT NULL DEFAULT '[]',
-- Phase 11.C (#508) — GeoIP-derived metadata populated at fold
-- time so reads + PDF rendering don't have to do per-row mmdb
-- lookups. eu_inside is 1 when country_iso EU/EEA whitelist.
country_iso TEXT,
asn_org TEXT,
eu_inside INTEGER NOT NULL DEFAULT 1,
-- Number of edges recorded against this (peer, tracker) BEFORE a
-- consent cookie was observed. >0 = legal-grade evidence of
-- tracker firing before consent (RGPD art. 6.1.a + 7).
pre_consent_hits INTEGER NOT NULL DEFAULT 0,
PRIMARY KEY (client_mac_hash, tracker_domain)
);
CREATE TABLE IF NOT EXISTS social_links (
client_mac_hash TEXT NOT NULL,
site_a TEXT NOT NULL,
site_b TEXT NOT NULL,
shared_trackers_jsonl TEXT NOT NULL DEFAULT '[]',
ja4_match INTEGER NOT NULL DEFAULT 0,
last_seen INTEGER NOT NULL,
PRIMARY KEY (client_mac_hash, site_a, site_b)
);
-- Phase 12.A (#515) — host-stable CDN/edge metadata, mac-independent.
-- Phase 12.B (#516) adds antibot_vendor (anti-bot / CAPTCHA vendor).
CREATE TABLE IF NOT EXISTS social_host_meta (
tracker_domain TEXT PRIMARY KEY,
cdn_vendor TEXT,
cache_status TEXT,
antibot_vendor TEXT,
last_seen INTEGER NOT NULL
);
-- Phase 12.B (#516) — per-client "challenged your humanity" log.
CREATE TABLE IF NOT EXISTS social_antibot (
client_mac_hash TEXT NOT NULL,
src_site TEXT NOT NULL,
antibot_vendor TEXT NOT NULL,
hits INTEGER NOT NULL DEFAULT 0,
last_seen INTEGER NOT NULL,
PRIMARY KEY (client_mac_hash, src_site, antibot_vendor)
);
"""
def _conn() -> sqlite3.Connection:
DB_PATH.parent.mkdir(parents=True, exist_ok=True)
c = sqlite3.connect(str(DB_PATH), timeout=5.0, isolation_level=None)
c.row_factory = sqlite3.Row
c.executescript(_SCHEMA)
_migrate_phase11c(c)
return c
# ───── Phase 11.C migrations — additive columns on pre-existing tables ─────
# CREATE TABLE IF NOT EXISTS skips creation if the table already exists, so
# the new columns won't auto-appear on a 2.6.0 → 2.6.2 upgrade. Idempotent
# ALTERs : we probe the column list first to skip the duplicate-column
# error case (which would raise on every connection otherwise).
_PHASE11C_MIGRATIONS = (
("social_edges", "consent_state", "TEXT NOT NULL DEFAULT 'none_seen'"),
("social_nodes", "country_iso", "TEXT"),
("social_nodes", "asn_org", "TEXT"),
("social_nodes", "eu_inside", "INTEGER NOT NULL DEFAULT 1"),
("social_nodes", "pre_consent_hits", "INTEGER NOT NULL DEFAULT 0"),
# Phase 12.B (#516) — anti-bot vendor column on the host-meta table
# (created by 12.A ; additive on a 2.6.3/2.6.4 → upgrade).
("social_host_meta", "antibot_vendor", "TEXT"),
)
def _migrate_phase11c(c: sqlite3.Connection) -> None:
try:
for table, col, decl in _PHASE11C_MIGRATIONS:
existing = {
r["name"] for r in c.execute(f"PRAGMA table_info({table})").fetchall()
}
if col not in existing:
c.execute(f"ALTER TABLE {table} ADD COLUMN {col} {decl}")
except Exception as e: # pragma: no cover
log.warning("Phase 11.C migration failed: %s", e)
# ───── EU / EEA whitelist (RGPD scope, art. 45 + Schengen extension) ─────
# Codes are ISO 3166-1 alpha-2. Source : EU member state list + EFTA
# (NO, IS, LI) + UK (adequacy decision in force as of writing). The
# Phase C "extra_eu" flag is set when GeoIP says the tracker's country
# ISO is NOT in this set.
_EU_EEA_ISO: frozenset = frozenset({
"AT", "BE", "BG", "CY", "CZ", "DE", "DK", "EE", "ES", "FI", "FR",
"GR", "HR", "HU", "IE", "IT", "LT", "LU", "LV", "MT", "NL", "PL",
"PT", "RO", "SE", "SI", "SK", # 27 EU
"IS", "LI", "NO", # EFTA / EEA
"GB", # UK adequacy decision
})
def is_eu_iso(iso: str | None) -> bool:
return bool(iso) and (iso or "").upper() in _EU_EEA_ISO
# Lightweight cache around the existing `geo` module so the fold loop
# doesn't pay the lookup cost per repeated tracker_domain. Bounded to
# 4096 entries (well above any realistic distinct tracker count seen
# in a 7d retention window).
import functools as _functools
@_functools.lru_cache(maxsize=4096)
def _geo_for(host: str) -> Tuple[Optional[str], Optional[str]]:
"""Return (country_iso, asn_org) for a tracker host.
Best-effort. Falls back to (None, None) when the GeoIP module isn't
importable (worker hasn't installed the mmdb yet) or when the host
is a raw IP and the underlying lookup misses.
"""
try:
from secubox_toolbox import geo as _g # type: ignore
info = _g.lookup(host) or {}
iso = (info.get("country_iso") or "").upper() or None
asn = (info.get("asn_org") or "")[:64] or None
return iso, asn
except Exception:
return None, None
def cookie_id_hash(tracker_domain: str, cookie_name: str, cookie_value: str) -> str:
"""Stable short hash for an observed tracker identifier.
Never round-trippable to the raw value (truncated SHA-256).
"""
h = hashlib.sha256()
h.update((tracker_domain or "").lower().encode("utf-8", "replace"))
h.update(b"\x00")
h.update((cookie_name or "").lower().encode("utf-8", "replace"))
h.update(b"\x00")
h.update((cookie_value or "").encode("utf-8", "replace"))
return h.hexdigest()[:16]
def is_deny_listed(cookie_name: str, extra_deny: Optional[Iterable[str]] = None) -> bool:
name = (cookie_name or "").strip().lower()
if not name:
return True
if name in _DEFAULT_DENY_COOKIES:
return True
if extra_deny:
if name in {x.lower() for x in extra_deny}:
return True
return False
# Phase 11.A — fire-and-forget SQLite writes (same pattern as
# utiq.py + local_store.py). The mitmproxy asyncio loop never blocks
# on _conn() open + INSERT + fsync. Bounded queue depth via the
# single-worker executor.
_executor = _futures.ThreadPoolExecutor(
max_workers=1, thread_name_prefix="sbx_social_write"
)
def _record_edge_sync(
client_mac_hash: str,
src_site: str,
tracker_domain: str,
cookie_id_hash_val: str,
ja4_hash: Optional[str],
consent_state: str,
) -> None:
try:
with _conn() as c:
c.execute(
"INSERT INTO social_edges(ts, client_mac_hash, src_site, "
"tracker_domain, cookie_id_hash, ja4_hash, consent_state) "
"VALUES (?, ?, ?, ?, ?, ?, ?)",
(
int(time.time()),
client_mac_hash,
src_site,
tracker_domain,
cookie_id_hash_val,
ja4_hash,
consent_state or "none_seen",
),
)
except Exception as e: # pragma: no cover — best-effort
log.warning("record_edge failed: %s", e)
def record_edge(
*,
client_mac_hash: str,
src_site: str,
tracker_domain: str,
cookie_id_hash_val: str,
ja4_hash: Optional[str] = None,
consent_state: str = "none_seen",
) -> None:
"""Submit one edge off-thread. Best-effort, never raises into the
addon, never blocks the mitmproxy asyncio loop.
`consent_state` is one of {none_seen, pre_consent, post_consent} as
computed by the addon based on the per-peer × per-site consent
cookie observation log (Phase 11.C).
"""
if not (client_mac_hash and src_site and tracker_domain and cookie_id_hash_val):
return
try:
_executor.submit(
_record_edge_sync,
client_mac_hash,
src_site,
tracker_domain,
cookie_id_hash_val,
ja4_hash,
consent_state,
)
except RuntimeError:
# Executor shut down (interpreter teardown) — silent drop.
pass
def _record_host_cdn_sync(domain: str, cdn_vendor: Optional[str],
cache_status: Optional[str]) -> None:
try:
with _conn() as c:
c.execute(
"INSERT INTO social_host_meta(tracker_domain, cdn_vendor, "
"cache_status, last_seen) VALUES (?, ?, ?, ?) "
"ON CONFLICT(tracker_domain) DO UPDATE SET "
"cdn_vendor=COALESCE(excluded.cdn_vendor, cdn_vendor), "
"cache_status=COALESCE(excluded.cache_status, cache_status), "
"last_seen=excluded.last_seen",
(domain, cdn_vendor, cache_status, int(time.time())),
)
except Exception as e: # pragma: no cover
log.warning("record_host_cdn failed: %s", e)
def record_host_cdn(*, domain: str, cdn_vendor: Optional[str] = None,
cache_status: Optional[str] = None) -> None:
"""Phase 12.A (#515) — upsert host-stable CDN metadata off-thread.
Best-effort, host-keyed (mac-independent)."""
if not domain or not (cdn_vendor or cache_status):
return
try:
_executor.submit(_record_host_cdn_sync, domain, cdn_vendor, cache_status)
except RuntimeError:
pass
# ── Phase 12.B (#516) — anti-bot / "prove you're human" recording ──
def _record_host_antibot_sync(domain: str, vendor: str) -> None:
try:
with _conn() as c:
c.execute(
"INSERT INTO social_host_meta(tracker_domain, antibot_vendor, "
"last_seen) VALUES (?, ?, ?) "
"ON CONFLICT(tracker_domain) DO UPDATE SET "
"antibot_vendor=excluded.antibot_vendor, "
"last_seen=excluded.last_seen",
(domain, vendor, int(time.time())),
)
except Exception as e: # pragma: no cover
log.warning("record_host_antibot failed: %s", e)
def record_host_antibot(*, domain: str, antibot_vendor: str) -> None:
"""Upsert the host-stable anti-bot vendor fronting `domain`."""
if not (domain and antibot_vendor):
return
try:
_executor.submit(_record_host_antibot_sync, domain, antibot_vendor)
except RuntimeError:
pass
def _record_antibot_challenge_sync(mac_hash: str, src_site: str, vendor: str) -> None:
try:
with _conn() as c:
c.execute(
"INSERT INTO social_antibot(client_mac_hash, src_site, "
"antibot_vendor, hits, last_seen) VALUES (?, ?, ?, 1, ?) "
"ON CONFLICT(client_mac_hash, src_site, antibot_vendor) DO UPDATE "
"SET hits = hits + 1, last_seen = excluded.last_seen",
(mac_hash, src_site, vendor, int(time.time())),
)
except Exception as e: # pragma: no cover
log.warning("record_antibot_challenge failed: %s", e)
def record_antibot_challenge(*, client_mac_hash: str, src_site: str,
antibot_vendor: str) -> None:
"""Record a per-client "this site challenged your humanity" event."""
if not (client_mac_hash and src_site and antibot_vendor):
return
try:
_executor.submit(_record_antibot_challenge_sync, client_mac_hash,
src_site, antibot_vendor)
except RuntimeError:
pass
def antibot_for_client(mac_hash: str, since_seconds: int = 86400) -> List[Dict]:
"""Per-client anti-bot challenge list for the graph alert tile."""
since = int(time.time()) - max(since_seconds, 3600)
out: List[Dict] = []
if not mac_hash:
return out
try:
with _conn() as c:
for r in c.execute(
"SELECT src_site, antibot_vendor, hits, last_seen "
"FROM social_antibot WHERE client_mac_hash = ? AND last_seen >= ? "
"ORDER BY last_seen DESC LIMIT 100",
(mac_hash, since),
).fetchall():
out.append(dict(r))
except Exception as e: # pragma: no cover
log.warning("antibot_for_client failed: %s", e)
return out
def fold_recent(window_seconds: int = 300) -> Tuple[int, int]:
"""Fold raw edges from the last `window_seconds` into the node + link
aggregate tables. Returns (nodes_touched, links_touched).
Called by the FastAPI startup background task every 5 min. Cheap
even with thousands of edges per fold because everything is on
the indexed (client_mac_hash, ts) column.
"""
since = int(time.time()) - max(window_seconds, 60)
nodes_touched = 0
links_touched = 0
try:
with _conn() as c:
edges = c.execute(
"SELECT client_mac_hash, src_site, tracker_domain, "
"cookie_id_hash, ja4_hash, ts, consent_state "
"FROM social_edges WHERE ts >= ?",
(since,),
).fetchall()
# Aggregate per (mac, tracker_domain) → nodes
per_node: Dict[Tuple[str, str], Dict] = {}
# Aggregate per (mac, site_a, site_b) → links
per_link: Dict[Tuple[str, str, str], Dict] = {}
# Index per (mac, site) → set(tracker_domain) for cross-site
per_site_trackers: Dict[Tuple[str, str], Set[str]] = {}
# JA4 hashes seen per (mac, site)
per_site_ja4: Dict[Tuple[str, str], Set[str]] = {}
for e in edges:
mac = e["client_mac_hash"]
site = e["src_site"]
trk = e["tracker_domain"]
ja4 = e["ja4_hash"]
ts = e["ts"]
# Per-node accumulator
key_n = (mac, trk)
n = per_node.setdefault(
key_n,
{
"hits": 0,
"first_seen": ts,
"last_seen": ts,
"sites": set(),
"pre_consent_hits": 0,
},
)
n["hits"] += 1
n["first_seen"] = min(n["first_seen"], ts)
n["last_seen"] = max(n["last_seen"], ts)
n["sites"].add(site)
if e["consent_state"] == "pre_consent":
n["pre_consent_hits"] += 1
# Per-site tracker index (for link fold below)
per_site_trackers.setdefault((mac, site), set()).add(trk)
if ja4:
per_site_ja4.setdefault((mac, site), set()).add(ja4)
# Persist nodes
for (mac, trk), n in per_node.items():
# Merge into existing row if present
# Phase 11.C : enrich with GeoIP at fold time so reads
# + PDF rendering never block on mmdb lookups.
country_iso, asn_org = _geo_for(trk)
eu_inside = 1 if is_eu_iso(country_iso) else 0
cur = c.execute(
"SELECT hits, first_seen, sites_jsonl, pre_consent_hits "
"FROM social_nodes "
"WHERE client_mac_hash = ? AND tracker_domain = ?",
(mac, trk),
).fetchone()
if cur:
hits = cur["hits"] + n["hits"]
first = min(cur["first_seen"], n["first_seen"])
try:
existing_sites = set(json.loads(cur["sites_jsonl"]))
except Exception:
existing_sites = set()
sites = sorted(existing_sites | n["sites"])
pre = (cur["pre_consent_hits"] or 0) + n["pre_consent_hits"]
c.execute(
"UPDATE social_nodes SET hits = ?, first_seen = ?, "
"last_seen = ?, sites_jsonl = ?, country_iso = ?, "
"asn_org = ?, eu_inside = ?, pre_consent_hits = ? "
"WHERE client_mac_hash = ? AND tracker_domain = ?",
(hits, first, n["last_seen"], json.dumps(sites),
country_iso, asn_org, eu_inside, pre, mac, trk),
)
else:
c.execute(
"INSERT INTO social_nodes(client_mac_hash, "
"tracker_domain, hits, first_seen, last_seen, "
"sites_jsonl, country_iso, asn_org, eu_inside, "
"pre_consent_hits) "
"VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
(
mac,
trk,
n["hits"],
n["first_seen"],
n["last_seen"],
json.dumps(sorted(n["sites"])),
country_iso,
asn_org,
eu_inside,
n["pre_consent_hits"],
),
)
nodes_touched += 1
# Derive links : for each (mac, trk) node, the sites form
# a clique connected by this tracker. Stamp each pair.
for (mac, trk), n in per_node.items():
sites = sorted(n["sites"])
if len(sites) < 2:
continue
for i, a in enumerate(sites):
for b in sites[i + 1 :]:
key_l = (mac, a, b)
l = per_link.setdefault(
key_l,
{
"shared": set(),
"ja4_match": False,
"last_seen": n["last_seen"],
},
)
l["shared"].add(trk)
l["last_seen"] = max(l["last_seen"], n["last_seen"])
# JA4 collision : same JA4 seen on both sites
ja4_a = per_site_ja4.get((mac, a), set())
ja4_b = per_site_ja4.get((mac, b), set())
if ja4_a & ja4_b:
l["ja4_match"] = True
for (mac, a, b), l in per_link.items():
cur = c.execute(
"SELECT shared_trackers_jsonl, ja4_match FROM social_links "
"WHERE client_mac_hash = ? AND site_a = ? AND site_b = ?",
(mac, a, b),
).fetchone()
if cur:
try:
existing = set(json.loads(cur["shared_trackers_jsonl"]))
except Exception:
existing = set()
shared = sorted(existing | l["shared"])
ja4m = 1 if (cur["ja4_match"] or l["ja4_match"]) else 0
c.execute(
"UPDATE social_links SET shared_trackers_jsonl = ?, "
"ja4_match = ?, last_seen = ? "
"WHERE client_mac_hash = ? AND site_a = ? AND site_b = ?",
(json.dumps(shared), ja4m, l["last_seen"], mac, a, b),
)
else:
c.execute(
"INSERT INTO social_links(client_mac_hash, site_a, "
"site_b, shared_trackers_jsonl, ja4_match, last_seen) "
"VALUES (?, ?, ?, ?, ?, ?)",
(
mac,
a,
b,
json.dumps(sorted(l["shared"])),
1 if l["ja4_match"] else 0,
l["last_seen"],
),
)
links_touched += 1
except Exception as e: # pragma: no cover
log.warning("fold_recent failed: %s", e)
return nodes_touched, links_touched
def fetch_graph(mac_hash: str, since_seconds: int = 86400) -> Dict:
"""Return the per-client graph JSON contract.
{nodes:[{id,domain,family,hits,sites_count}],
edges:[{src,dst,reuse_count,shared_trackers[],ja4_match}],
stats:{total_trackers,total_sites,first_seen,last_seen}}
"""
since = int(time.time()) - max(since_seconds, 3600)
out: Dict = {"nodes": [], "edges": [], "stats": {}}
if not mac_hash:
return out
try:
with _conn() as c:
# Nodes : trackers active in window. Phase 12.A — LEFT JOIN
# the host-stable CDN metadata so the d3 view + node detail
# can colour/label by edge-network vendor.
for r in c.execute(
"SELECT n.tracker_domain, n.hits, n.first_seen, n.last_seen, "
"n.sites_jsonl, m.cdn_vendor, m.cache_status, m.antibot_vendor "
"FROM social_nodes n "
"LEFT JOIN social_host_meta m ON m.tracker_domain = n.tracker_domain "
"WHERE n.client_mac_hash = ? AND n.last_seen >= ? "
"ORDER BY n.hits DESC LIMIT 500",
(mac_hash, since),
).fetchall():
try:
sites = json.loads(r["sites_jsonl"])
except Exception:
sites = []
out["nodes"].append(
{
"id": r["tracker_domain"],
"domain": r["tracker_domain"],
"hits": r["hits"],
"sites_count": len(sites),
"sites": sites,
"first_seen": r["first_seen"],
"last_seen": r["last_seen"],
"cdn_vendor": r["cdn_vendor"],
"cache_status": r["cache_status"],
"antibot_vendor": r["antibot_vendor"],
}
)
# Edges : site-pairs sharing trackers in window
for r in c.execute(
"SELECT site_a, site_b, shared_trackers_jsonl, ja4_match, last_seen "
"FROM social_links WHERE client_mac_hash = ? AND last_seen >= ? "
"ORDER BY last_seen DESC LIMIT 1000",
(mac_hash, since),
).fetchall():
try:
shared = json.loads(r["shared_trackers_jsonl"])
except Exception:
shared = []
out["edges"].append(
{
"src": r["site_a"],
"dst": r["site_b"],
"reuse_count": len(shared),
"shared_trackers": shared,
"ja4_match": bool(r["ja4_match"]),
"last_seen": r["last_seen"],
}
)
stats_row = c.execute(
"SELECT COUNT(DISTINCT tracker_domain) AS total_trackers, "
"MIN(first_seen) AS first_seen, MAX(last_seen) AS last_seen "
"FROM social_nodes WHERE client_mac_hash = ? AND last_seen >= ?",
(mac_hash, since),
).fetchone()
sites_count = len(
{
s
for n in out["nodes"]
for s in n["sites"]
}
)
# Phase 12.B — per-client anti-bot challenges for the alert tile.
antibot = antibot_for_client(mac_hash, since_seconds=since_seconds)
out["antibot"] = antibot
out["stats"] = {
"total_trackers": (stats_row["total_trackers"] or 0) if stats_row else 0,
"total_sites": sites_count,
"first_seen": stats_row["first_seen"] if stats_row else None,
"last_seen": stats_row["last_seen"] if stats_row else None,
"antibot_sites": len({a["src_site"] for a in antibot}),
"antibot_vendors": sorted({a["antibot_vendor"] for a in antibot}),
}
except Exception as e: # pragma: no cover
log.warning("fetch_graph failed: %s", e)
return out
def wipe_mac(mac_hash: str) -> int:
"""Delete every row for a given peer across the 3 tables.
Used by the RGPD art. 17 endpoint. Returns total rows deleted.
"""
if not mac_hash:
return 0
total = 0
try:
with _conn() as c:
for table in ("social_edges", "social_nodes", "social_links",
"social_antibot"):
cur = c.execute(
f"DELETE FROM {table} WHERE client_mac_hash = ?", (mac_hash,)
)
total += cur.rowcount or 0
except Exception as e: # pragma: no cover
log.warning("wipe_mac failed: %s", e)
return total
def aggregate(hours: int = 24) -> Dict:
"""Operator dashboard KPI tiles + tracker-family histogram +
anonymized client table. JWT-gated in the API layer.
"""
if hours < 1 or hours > 24 * 31:
hours = 24
since = int(time.time()) - hours * 3600
out: Dict = {
"window_hours": hours,
"active_clients": 0,
"total_trackers_seen": 0,
"extra_eu_trackers": 0, # Phase C will populate this
"by_tracker_domain": [],
"by_client": [],
"by_cdn": [], # Phase 12.A
"by_antibot": [], # Phase 12.B
"antibot_clients": 0, # Phase 12.B
}
try:
with _conn() as c:
out["active_clients"] = c.execute(
"SELECT COUNT(DISTINCT client_mac_hash) FROM social_edges "
"WHERE ts >= ?",
(since,),
).fetchone()[0]
out["total_trackers_seen"] = c.execute(
"SELECT COUNT(DISTINCT tracker_domain) FROM social_edges "
"WHERE ts >= ?",
(since,),
).fetchone()[0]
out["by_tracker_domain"] = [
dict(r)
for r in c.execute(
"SELECT tracker_domain, COUNT(*) AS hits, "
"COUNT(DISTINCT client_mac_hash) AS clients "
"FROM social_edges WHERE ts >= ? "
"GROUP BY tracker_domain ORDER BY hits DESC LIMIT 50",
(since,),
).fetchall()
]
out["by_client"] = [
dict(r)
for r in c.execute(
"SELECT client_mac_hash, COUNT(DISTINCT src_site) AS sites, "
"COUNT(DISTINCT tracker_domain) AS trackers, "
"MAX(ts) AS last_seen "
"FROM social_edges WHERE ts >= ? "
"GROUP BY client_mac_hash ORDER BY last_seen DESC LIMIT 100",
(since,),
).fetchall()
]
# Phase 12.A — CDN/edge-network breakdown over the trackers
# seen in the window (host-stable metadata, mac-independent).
out["by_cdn"] = [
dict(r)
for r in c.execute(
"SELECT m.cdn_vendor, COUNT(DISTINCT m.tracker_domain) AS hosts "
"FROM social_host_meta m "
"WHERE m.cdn_vendor IS NOT NULL AND m.last_seen >= ? "
"AND m.tracker_domain IN ("
" SELECT DISTINCT tracker_domain FROM social_edges WHERE ts >= ?) "
"GROUP BY m.cdn_vendor ORDER BY hosts DESC LIMIT 25",
(since, since),
).fetchall()
]
# Phase 12.B — anti-bot / "prove you're human" breakdown.
out["by_antibot"] = [
dict(r)
for r in c.execute(
"SELECT antibot_vendor, "
"COUNT(DISTINCT src_site) AS sites, "
"COUNT(DISTINCT client_mac_hash) AS clients, "
"SUM(hits) AS challenges "
"FROM social_antibot WHERE last_seen >= ? "
"GROUP BY antibot_vendor ORDER BY challenges DESC LIMIT 25",
(since,),
).fetchall()
]
out["antibot_clients"] = c.execute(
"SELECT COUNT(DISTINCT client_mac_hash) FROM social_antibot "
"WHERE last_seen >= ?",
(since,),
).fetchone()[0]
except Exception as e: # pragma: no cover
log.warning("aggregate failed: %s", e)
return out
def evidence(mac_hash: str, since_seconds: int = 86400) -> Dict:
"""Phase 11.C evidence helper — returns the legal-grade slice
consumed by the bilingual PDF report.
Two evidence buckets, both fact-only (no interpretation) :
- ``pre_consent`` : (tracker_domain, sites, pre_consent_hits,
country_iso, asn_org). Trackers that fired BEFORE a consent
cookie was observed for that peer × site. Direct RGPD art. 7
+ art. 6.1.a evidence.
- ``extra_eu`` : (tracker_domain, country_iso, asn_org,
sites). Trackers resolving to non-EU/EEA countries. Note :
we report the fact, not SCC absence (we can't prove a
negative). RGPD art. 44+ evidence.
"""
since = int(time.time()) - max(since_seconds, 3600)
out: Dict = {"pre_consent": [], "extra_eu": []}
if not mac_hash:
return out
try:
with _conn() as c:
for r in c.execute(
"SELECT tracker_domain, hits, pre_consent_hits, sites_jsonl, "
"country_iso, asn_org, last_seen "
"FROM social_nodes "
"WHERE client_mac_hash = ? AND last_seen >= ? "
"AND pre_consent_hits > 0 "
"ORDER BY pre_consent_hits DESC, hits DESC LIMIT 100",
(mac_hash, since),
).fetchall():
try:
sites = json.loads(r["sites_jsonl"])
except Exception:
sites = []
out["pre_consent"].append({
"tracker_domain": r["tracker_domain"],
"hits": r["hits"],
"pre_consent_hits": r["pre_consent_hits"],
"sites": sites,
"country_iso": r["country_iso"],
"asn_org": r["asn_org"],
"last_seen": r["last_seen"],
})
for r in c.execute(
"SELECT tracker_domain, hits, sites_jsonl, country_iso, "
"asn_org, last_seen "
"FROM social_nodes "
"WHERE client_mac_hash = ? AND last_seen >= ? "
"AND eu_inside = 0 AND country_iso IS NOT NULL "
"ORDER BY hits DESC LIMIT 100",
(mac_hash, since),
).fetchall():
try:
sites = json.loads(r["sites_jsonl"])
except Exception:
sites = []
out["extra_eu"].append({
"tracker_domain": r["tracker_domain"],
"hits": r["hits"],
"sites": sites,
"country_iso": r["country_iso"],
"asn_org": r["asn_org"],
"last_seen": r["last_seen"],
})
except Exception as e: # pragma: no cover
log.warning("evidence query failed: %s", e)
return out
def purge_older_than(days: int = 7) -> int:
"""Drop raw edges older than `days`. The aggregate node/link tables
stay : they represent the durable fold. Operator-side wipe goes
through wipe_mac().
"""
cutoff = int(time.time()) - max(days, 1) * 86400
try:
with _conn() as c:
cur = c.execute("DELETE FROM social_edges WHERE ts < ?", (cutoff,))
return cur.rowcount or 0
except Exception as e: # pragma: no cover
log.warning("purge_older_than failed: %s", e)
return 0

View File

@ -0,0 +1,239 @@
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
# Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
"""
SecuBox-Deb :: ToolBoX Social Mapping bilingual FR/EN PDF report
Phase 11.C (#508, parent #502) — evidence-only legal report consumed by
the per-client view's "Rapport PDF" card. Renders via fpdf2 (same
engine as reports.py) so no new dependency.
The report is FACT-ONLY : it documents what we observed (cross-site
cookie reuse, trackers fired before consent, extra-EU transfers) with
the relevant RGPD article references. It makes NO interpretation about
SCC presence/absence or ToS contradiction.
FR is the source-of-truth ; EN is a secondary summary line per field.
"""
from __future__ import annotations
import logging
import time
from typing import Dict, List
from . import social as _social
log = logging.getLogger("secubox.toolbox.social.report")
# Bilingual label pairs (FR primary, EN secondary).
_L = {
"title": ("Cartographie sociale — Rapport d'évidence", "Social mapping — Evidence report"),
"subtitle": ("Cabine numérique VILLAGE3B · Analyseur R3", "VILLAGE3B digital booth · R3 analyzer"),
"anon_id": ("Identifiant anonyme", "Anonymous identifier"),
"hash": ("Hash session (sel rotatif 24h)", "Session hash (24h rotating salt)"),
"window": ("Fenêtre d'analyse", "Analysis window"),
"generated": ("Généré le", "Generated"),
"summary": ("Synthèse", "Summary"),
"n_trackers": ("Traqueurs distincts", "Distinct trackers"),
"n_sites": ("Sites visités", "Sites visited"),
"n_preconsent": ("Traqueurs avant consentement", "Trackers before consent"),
"n_extraeu": ("Transferts hors UE/EEE", "Extra-EU/EEA transfers"),
"ev_preconsent": ("Évidence : traqueurs déclenchés AVANT consentement",
"Evidence: trackers fired BEFORE consent"),
"ev_preconsent_basis": ("Base légale : RGPD art. 6.1.a (consentement) + art. 7 (preuve)",
"Legal basis: GDPR art. 6.1.a (consent) + art. 7 (proof)"),
"ev_extraeu": ("Évidence : transferts vers des pays hors UE/EEE",
"Evidence: transfers to non-EU/EEA countries"),
"ev_extraeu_basis": ("Base légale : RGPD art. 44+ (transferts internationaux). "
"Nous rapportons le fait observé, pas l'absence de garanties (SCC).",
"Legal basis: GDPR art. 44+ (international transfers). We report the "
"observed fact, not the absence of safeguards (SCC)."),
"col_tracker": ("Traqueur", "Tracker"),
"col_sites": ("Sites", "Sites"),
"col_hits": ("Occurrences", "Hits"),
"col_country": ("Pays", "Country"),
"col_asn": ("Hébergeur (ASN)", "Host (ASN)"),
"none_pre": ("Aucun traqueur déclenché avant consentement détecté.",
"No tracker fired before consent detected."),
"none_eu": ("Aucun transfert hors UE/EEE détecté.",
"No extra-EU/EEA transfer detected."),
"footer": ("Rapport factuel — données calculées localement, aucune donnée externe. "
"Anonyme : aucune valeur de cookie brute conservée. Droit à l'effacement : RGPD art. 17.",
"Factual report — computed locally, no external data. Anonymous: no raw cookie "
"value stored. Right to erasure: GDPR art. 17."),
"ca_disclaimer": ("Évidence d'observation réseau via cabine R3 consentie. "
"Ne constitue pas un avis juridique.",
"Network observation evidence via consented R3 booth. "
"Not legal advice."),
}
def _bi(key: str) -> str:
fr, en = _L.get(key, (key, key))
return fr
def build_social_report(mac_hash: str, since_seconds: int = 7 * 86400) -> Dict:
"""Assemble the data structure the PDF renderer consumes."""
graph = _social.fetch_graph(mac_hash, since_seconds=since_seconds)
evidence = _social.evidence(mac_hash, since_seconds=since_seconds)
stats = graph.get("stats", {})
return {
"mac_hash": mac_hash,
"window_days": max(1, since_seconds // 86400),
"generated_at": None, # stamped by the API layer (Date.* unavailable here is fine; time.time ok)
"total_trackers": stats.get("total_trackers", 0),
"total_sites": stats.get("total_sites", 0),
"pre_consent": evidence.get("pre_consent", []),
"extra_eu": evidence.get("extra_eu", []),
}
def render_social_pdf(report: Dict) -> bytes:
"""Render the bilingual evidence report as PDF (fpdf2)."""
try:
from fpdf import FPDF
except ImportError:
return _text_fallback(report).encode()
# Reuse the font setup from reports.py for emoji/Unicode support.
try:
from .reports import _setup_fonts # type: ignore
except Exception:
_setup_fonts = None
pdf = FPDF(orientation="P", unit="mm", format="A4")
pdf.add_page()
pdf.set_auto_page_break(auto=True, margin=15)
family = _setup_fonts(pdf) if _setup_fonts else "helvetica"
def _mc(h, text):
# Robust multi_cell : reset X to the left margin and use the
# effective page width so fpdf2 never hits "not enough
# horizontal space" when the cursor has drifted to the edge.
pdf.set_x(pdf.l_margin)
pdf.multi_cell(pdf.epw, h, text)
def bi(key, size=10, style="", r=0, g=0, b=0, gap=1.0):
fr, en = _L.get(key, (key, key))
pdf.set_text_color(r, g, b)
pdf.set_font(family, "B" if "B" in style else "", size)
_mc(size * 0.5, fr)
pdf.set_text_color(120, 120, 120)
pdf.set_font(family, "", max(7, size - 2))
_mc((size - 2) * 0.5, en)
pdf.ln(gap)
# ── Cover ──
pdf.set_font(family, "B", 20)
pdf.set_text_color(10, 90, 64)
pdf.cell(0, 12, "🕸️ VILLAGE3B", ln=True, align="C")
bi("title", 13, "B", 110, 64, 201, gap=0.5)
bi("subtitle", 9, "", 110, 110, 110, gap=3)
# ── Anonymous id ──
bi("anon_id", 12, "B", 0, 90, 64, gap=0.5)
pdf.set_font(family, "", 9)
pdf.set_text_color(0)
pdf.cell(0, 5, f" {_bi('hash')}: {report.get('mac_hash', '?')[:16]}", ln=True)
pdf.cell(0, 5, f" {_bi('window')}: {report.get('window_days', 7)} j / days", ln=True)
pdf.cell(0, 5, f" {_bi('generated')}: {report.get('generated_at', '?')}", ln=True)
pdf.ln(3)
# ── Summary tiles ──
bi("summary", 12, "B", 0, 90, 64, gap=0.5)
pdf.set_font(family, "", 10)
pdf.set_text_color(0)
pdf.cell(0, 6, f" {_bi('n_trackers')}: {report.get('total_trackers', 0)}", ln=True)
pdf.cell(0, 6, f" {_bi('n_sites')}: {report.get('total_sites', 0)}", ln=True)
pdf.cell(0, 6, f" {_bi('n_preconsent')}: {len(report.get('pre_consent', []))}", ln=True)
pdf.cell(0, 6, f" {_bi('n_extraeu')}: {len(report.get('extra_eu', []))}", ln=True)
pdf.ln(3)
# ── Evidence : pre-consent ──
bi("ev_preconsent", 12, "B", 230, 57, 70, gap=0.3)
bi("ev_preconsent_basis", 8, "", 120, 120, 120, gap=1)
pre = report.get("pre_consent", [])
if pre:
_table(pdf, family, pre, [
(_bi("col_tracker"), "tracker_domain", 70),
(_bi("col_sites"), lambda r: str(len(r.get("sites", []))), 25),
("pre", "pre_consent_hits", 25),
(_bi("col_country"), "country_iso", 25),
(_bi("col_asn"), "asn_org", 45),
])
else:
_note(pdf, family, _bi("none_pre"))
pdf.ln(2)
# ── Evidence : extra-EU ──
bi("ev_extraeu", 12, "B", 110, 64, 201, gap=0.3)
bi("ev_extraeu_basis", 8, "", 120, 120, 120, gap=1)
eu = report.get("extra_eu", [])
if eu:
_table(pdf, family, eu, [
(_bi("col_tracker"), "tracker_domain", 70),
(_bi("col_sites"), lambda r: str(len(r.get("sites", []))), 25),
(_bi("col_hits"), "hits", 25),
(_bi("col_country"), "country_iso", 25),
(_bi("col_asn"), "asn_org", 45),
])
else:
_note(pdf, family, _bi("none_eu"))
pdf.ln(4)
# ── Footer ──
pdf.set_font(family, "", 7)
pdf.set_text_color(120, 120, 120)
fr, en = _L["footer"]
_mc(3.2, fr)
_mc(3.0, en)
pdf.ln(1)
fr, en = _L["ca_disclaimer"]
_mc(3.2, fr + " / " + en)
out = pdf.output(dest="S")
return bytes(out) if isinstance(out, (bytes, bytearray)) else out.encode("latin-1")
def _table(pdf, family, rows: List[Dict], cols) -> None:
pdf.set_font(family, "B", 8)
pdf.set_text_color(0, 90, 64)
for label, _key, w in cols:
pdf.cell(w, 5, str(label)[:24], border="B")
pdf.ln()
pdf.set_font(family, "", 8)
pdf.set_text_color(0)
for r in rows[:40]:
for label, key, w in cols:
if callable(key):
val = key(r)
else:
val = r.get(key, "")
pdf.cell(w, 4.5, str(val if val is not None else "")[:int(w / 1.7)])
pdf.ln()
def _note(pdf, family, text: str) -> None:
pdf.set_font(family, "I", 9)
pdf.set_text_color(0, 120, 80)
pdf.cell(0, 6, " " + text, ln=True)
def _text_fallback(report: Dict) -> str:
lines = [
"VILLAGE3B — Social mapping evidence report",
f"hash: {report.get('mac_hash', '?')[:16]}",
f"trackers: {report.get('total_trackers', 0)} sites: {report.get('total_sites', 0)}",
f"pre-consent: {len(report.get('pre_consent', []))} extra-EU: {len(report.get('extra_eu', []))}",
"",
"Pre-consent trackers (GDPR art. 6.1.a + 7):",
]
for e in report.get("pre_consent", [])[:40]:
lines.append(f" - {e.get('tracker_domain')} ({e.get('pre_consent_hits')} hits, {e.get('country_iso')})")
lines.append("")
lines.append("Extra-EU transfers (GDPR art. 44+):")
for e in report.get("extra_eu", [])[:40]:
lines.append(f" - {e.get('tracker_domain')} ({e.get('country_iso')}, {e.get('asn_org')})")
return "\n".join(lines)

View File

@ -132,3 +132,30 @@ def purge_expired() -> int:
if n: if n:
log.info("purge_expired: %d rows", n) log.info("purge_expired: %d rows", n)
return n return n
def reset_client(mac_hash: str) -> int:
"""Phase 12.B (#516) — RAZ a specific client's accumulated toolbox
state : events + consents + reports. Returns rows deleted. The
`clients` row itself is kept (it re-populates from live activity)
but its score is zeroed. Social-mapping rows are wiped separately
via secubox_toolbox.social.wipe_mac().
"""
if not mac_hash:
return 0
n = 0
try:
with _conn() as c:
for table in ("events", "consents", "reports"):
n += c.execute(
f"DELETE FROM {table} WHERE mac_hash = ?", (mac_hash,)
).rowcount or 0
c.execute(
"UPDATE clients SET score = 0, state = 'validated' "
"WHERE mac_hash = ?", (mac_hash,)
)
except Exception as e:
log.warning("reset_client failed: %s", e)
if n:
log.info("reset_client %s: %d rows", mac_hash[:8], n)
return n

File diff suppressed because one or more lines are too long

View File

@ -2,7 +2,7 @@
<!-- <!--
SPDX-License-Identifier: LicenseRef-CMSD-1.0 SPDX-License-Identifier: LicenseRef-CMSD-1.0
Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr> Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
SecuBox-Deb :: ToolBoX admin dashboard (P31 light skin) SecuBox-Deb :: ToolBoX admin dashboard (P31 light skin) — sub-tab nav (#513)
--> -->
<html lang="fr"> <html lang="fr">
<head> <head>
@ -14,14 +14,22 @@
<link rel="stylesheet" href="/shared/crt-light.css"> <link rel="stylesheet" href="/shared/crt-light.css">
<link rel="stylesheet" href="/shared/sidebar-light.css"> <link rel="stylesheet" href="/shared/sidebar-light.css">
<style> <style>
:root{--p31-peak:#00dd44;--p31-hot:#00ff55;--p31-mid:#009933;--p31-dim:#006622;--p31-decay:#ffb347;--tube-light:#e8f5e9;--tube-pale:#c8e6c9;--tube-soft:#a5d6a7;--bg-card:var(--tube-pale);--border:var(--tube-soft);--text-dim:var(--p31-dim);--red:#ff4466;--bloom-text:0 0 2px var(--p31-peak),0 0 6px var(--p31-peak);--bloom-soft:0 0 6px var(--p31-peak)} :root{--p31-peak:#00dd44;--p31-hot:#00ff55;--p31-mid:#009933;--p31-dim:#006622;--p31-decay:#ffb347;--tube-light:#e8f5e9;--tube-pale:#c8e6c9;--tube-soft:#a5d6a7;--bg-card:var(--tube-pale);--border:var(--tube-soft);--text-dim:var(--p31-dim);--red:#ff4466;--purple:#9e76ff;--bloom-text:0 0 2px var(--p31-peak),0 0 6px var(--p31-peak);--bloom-soft:0 0 6px var(--p31-peak)}
*{box-sizing:border-box;margin:0;padding:0} *{box-sizing:border-box;margin:0;padding:0}
body{font-family:'Courier Prime',monospace;background:var(--tube-light);color:var(--p31-mid);display:flex;min-height:100vh} body{font-family:'Courier Prime',monospace;background:var(--tube-light);color:var(--p31-mid);display:flex;min-height:100vh}
.sidebar{width:220px;position:fixed;height:100vh;overflow-y:auto} .sidebar{width:220px;position:fixed;height:100vh;overflow-y:auto}
.main{flex:1;margin-left:220px;padding:1.5rem} .main{flex:1;margin-left:220px;padding:1.5rem}
.header{display:flex;justify-content:space-between;align-items:center;padding:1rem 1.5rem;border:1px solid var(--border);background:var(--bg-card);margin-bottom:1.5rem} .header{display:flex;justify-content:space-between;align-items:center;padding:1rem 1.5rem;border:1px solid var(--border);background:var(--bg-card);margin-bottom:1rem}
.header h1{font-size:1.4rem;color:var(--p31-hot);text-shadow:var(--bloom-text)} .header h1{font-size:1.4rem;color:var(--p31-hot);text-shadow:var(--bloom-text)}
.badge{font-size:0.85rem;color:var(--p31-dim);padding:0.2rem 0.6rem;border:1px solid var(--border);border-radius:3px} .badge{font-size:0.85rem;color:var(--p31-dim);padding:0.2rem 0.6rem;border:1px solid var(--border);border-radius:3px}
/* Sub-tab nav (#513) */
.tabs{display:flex;gap:0.3rem;margin-bottom:1.2rem;border-bottom:1px solid var(--border);flex-wrap:wrap}
.tab{font-family:inherit;font-size:0.9rem;padding:0.5rem 1rem;background:transparent;color:var(--p31-dim);border:1px solid var(--border);border-bottom:none;cursor:pointer;border-radius:3px 3px 0 0}
.tab:hover{color:var(--p31-peak);background:rgba(0,221,68,0.05)}
.tab.active{color:var(--p31-hot);text-shadow:var(--bloom-text);background:var(--bg-card);border-color:var(--p31-mid);font-weight:bold}
.panel{display:none}
.panel.active{display:block;animation:fade .15s ease-in}
@keyframes fade{from{opacity:0}to{opacity:1}}
.grid{display:grid;grid-template-columns:repeat(auto-fit,minmax(380px,1fr));gap:1rem} .grid{display:grid;grid-template-columns:repeat(auto-fit,minmax(380px,1fr));gap:1rem}
.card{background:var(--bg-card);border:1px solid var(--border);padding:1rem;box-shadow:var(--bloom-soft)} .card{background:var(--bg-card);border:1px solid var(--border);padding:1rem;box-shadow:var(--bloom-soft)}
.card h2{font-size:1rem;color:var(--p31-hot);text-shadow:var(--bloom-text);margin-bottom:0.6rem;padding-bottom:0.4rem;border-bottom:1px solid var(--border)} .card h2{font-size:1rem;color:var(--p31-hot);text-shadow:var(--bloom-text);margin-bottom:0.6rem;padding-bottom:0.4rem;border-bottom:1px solid var(--border)}
@ -32,13 +40,23 @@
th,td{text-align:left;padding:0.4rem 0.5rem;border-bottom:1px solid var(--border)} th,td{text-align:left;padding:0.4rem 0.5rem;border-bottom:1px solid var(--border)}
th{color:var(--p31-hot);text-shadow:var(--bloom-text)} th{color:var(--p31-hot);text-shadow:var(--bloom-text)}
.empty{color:var(--text-dim);font-style:italic;padding:1rem;text-align:center} .empty{color:var(--text-dim);font-style:italic;padding:1rem;text-align:center}
.toolbar{display:flex;gap:0.5rem;margin-bottom:1rem} .toolbar{display:flex;gap:0.5rem;margin-bottom:1rem;flex-wrap:wrap}
button{font-family:inherit;font-size:0.9rem;padding:0.4rem 0.9rem;background:transparent;color:var(--p31-peak);border:1px solid var(--p31-mid);cursor:pointer;text-shadow:var(--bloom-text)} button{font-family:inherit;font-size:0.9rem;padding:0.4rem 0.9rem;background:transparent;color:var(--p31-peak);border:1px solid var(--p31-mid);cursor:pointer;text-shadow:var(--bloom-text)}
button:hover{background:rgba(0,221,68,0.1)} button:hover{background:rgba(0,221,68,0.1)}
.r2-banner{background:rgba(255,179,71,0.08);border-left:2px solid var(--p31-decay);padding:0.6rem 0.8rem;margin-bottom:1rem;font-size:0.85rem;color:#996b1a} .r2-banner{background:rgba(255,179,71,0.08);border-left:2px solid var(--p31-decay);padding:0.6rem 0.8rem;margin-bottom:1rem;font-size:0.85rem;color:#996b1a}
.state-validated{color:var(--p31-peak);text-shadow:var(--bloom-text)} .state-validated{color:var(--p31-peak);text-shadow:var(--bloom-text)}
.state-throttle{color:var(--p31-decay)} .state-throttle{color:var(--p31-decay)}
.state-quarantine{color:var(--red)} .state-quarantine{color:var(--red)}
/* level chips + switcher (folded from kbin /admin/) */
.chip{display:inline-block;padding:0.1rem 0.45rem;border-radius:99px;font-size:0.7rem;font-weight:bold}
.chip.r0{background:#cfd8cf;color:#555}
.chip.r1{background:rgba(0,221,68,0.25);color:var(--p31-mid)}
.chip.r2{background:rgba(255,179,71,0.3);color:#996b1a}
.chip.r3{background:rgba(158,118,255,0.25);color:#6a4fd0}
.lvlbtn{font-size:0.68rem;padding:0.12rem 0.4rem;margin-right:0.15rem;border:1px solid var(--p31-dim)}
code{background:rgba(0,0,0,0.05);padding:0.1rem 0.3rem;border-radius:2px;font-size:0.78rem}
a.link{color:var(--p31-mid);text-decoration:underline}
.filterlist li{font-size:0.82rem;padding:0.2rem 0;border-bottom:1px solid var(--border);list-style:none;display:flex;justify-content:space-between;align-items:center}
</style> </style>
</head> </head>
<body> <body>
@ -53,39 +71,115 @@
<strong>R2 actif :</strong> TLS-break opt-in. Voir #475 / #474 pour la doctrine. <strong>R2 actif :</strong> TLS-break opt-in. Voir #475 / #474 pour la doctrine.
</div> </div>
<div class="toolbar"> <!-- Sub-tab navigation (#513) -->
<button onclick="refreshAll()">🔁 Refresh</button> <nav class="tabs" id="tabs">
<button onclick="window.open('/api/v1/toolbox/admin/config', '_blank')">⚙ Config TOML</button> <button class="tab active" data-tab="overview" onclick="switchTab('overview')">📊 Vue d'ensemble</button>
</div> <button class="tab" data-tab="clients" onclick="switchTab('clients')">👥 Clients</button>
<button class="tab" data-tab="filtres" onclick="switchTab('filtres')">🚦 Filtres MITM</button>
<button class="tab" data-tab="social" onclick="switchTab('social')">🕸️ Cartographie sociale</button>
<button class="tab" data-tab="config" onclick="switchTab('config')">⚙ Config</button>
</nav>
<div class="grid"> <!-- Overview -->
<section class="panel active" id="panel-overview">
<div class="toolbar">
<button onclick="refreshAll()">🔁 Refresh</button>
</div>
<div class="grid">
<div class="card">
<h2>📊 Live metrics (24h)</h2>
<div class="kv" id="metrics"><span class="k">loading…</span><span class="v"></span></div>
</div>
<div class="card">
<h2>♥ Liveness</h2>
<div class="kv" id="health"><span class="k">loading…</span><span class="v"></span></div>
</div>
</div>
</section>
<!-- Clients (folds in the kbin /admin/ level switcher) -->
<section class="panel" id="panel-clients">
<div class="toolbar">
<button onclick="loadClients()">🔁 Refresh</button>
</div>
<div class="card" style="margin-bottom:1rem">
<h2>👥 Clients actifs</h2>
<div id="clients"><div class="empty">loading…</div></div>
</div>
<div class="card" style="display:none" id="client-detail-card">
<h2 id="detail-title">🔍 Détails client</h2>
<div id="client-detail"></div>
</div>
</section>
<!-- Filtres MITM (bypass list) -->
<section class="panel" id="panel-filtres">
<div class="toolbar">
<button onclick="loadFilters()">🔁 Refresh</button>
</div>
<div class="card"> <div class="card">
<h2>📊 Live metrics (24h)</h2> <h2>🚦 Hosts bypassés (mitm ignore_hosts)</h2>
<div class="kv" id="metrics"><span class="k">loading…</span><span class="v"></span></div> <p style="font-size:0.82rem;color:var(--p31-dim);margin-bottom:0.6rem">
Hosts exclus de l'inspection TLS (cert-pinning détecté ou whitelist statique).
</p>
<ul class="filterlist" id="filters"><li class="empty">loading…</li></ul>
</div>
</section>
<!-- Cartographie sociale (Phase 11 operator aggregate) -->
<section class="panel" id="panel-social">
<div class="toolbar">
<button onclick="loadSocial()">🔁 Refresh</button>
</div>
<div class="grid">
<div class="card">
<h2>🕸️ Agrégat trackers (24h)</h2>
<div class="kv" id="social-kpi"><span class="k">loading…</span><span class="v"></span></div>
</div>
<div class="card">
<h2>🛰️ Réseaux CDN / edge</h2>
<div id="social-cdn"><div class="empty">loading…</div></div>
</div>
<div class="card">
<h2>🤖 Anti-bot / "prouvez que vous êtes humain"</h2>
<div id="social-antibot"><div class="empty">loading…</div></div>
</div>
<div class="card" style="grid-column:1/-1">
<h2>🎯 Top tracker domains</h2>
<div id="social-trackers"><div class="empty">loading…</div></div>
</div>
<div class="card" style="grid-column:1/-1">
<h2>👤 Clients anonymisés</h2>
<div id="social-clients"><div class="empty">loading…</div></div>
</div>
</div>
</section>
<!-- Config -->
<section class="panel" id="panel-config">
<div class="toolbar">
<button onclick="window.open('/api/v1/toolbox/admin/config', '_blank')">⚙ Config TOML brute</button>
</div> </div>
<div class="card"> <div class="card">
<h2>⚙ Configuration</h2> <h2>⚙ Configuration</h2>
<div class="kv" id="cfg-summary"><span class="k">loading…</span><span class="v"></span></div> <div class="kv" id="cfg-summary"><span class="k">loading…</span><span class="v"></span></div>
</div> </div>
<div class="card"> </section>
<h2>♥ Liveness</h2>
<div class="kv" id="health"><span class="k">loading…</span><span class="v"></span></div>
</div>
<div class="card" style="grid-column:1/-1">
<h2>👥 Clients actifs</h2>
<div id="clients"><div class="empty">loading…</div></div>
</div>
<div class="card" style="grid-column:1/-1;display:none" id="client-detail-card">
<h2 id="detail-title">🔍 Détails client</h2>
<div id="client-detail"></div>
</div>
</div>
</div> </div>
<script src="/shared/sidebar.js"></script> <script src="/shared/sidebar.js"></script>
<script> <script>
const API = '/api/v1/toolbox'; const API = '/api/v1/toolbox';
function switchTab(name) {
document.querySelectorAll('.tab').forEach(t => t.classList.toggle('active', t.dataset.tab === name));
document.querySelectorAll('.panel').forEach(p => p.classList.toggle('active', p.id === 'panel-' + name));
// Lazy-load the heavier tabs only when first opened.
if (name === 'filtres') loadFilters();
if (name === 'social') loadSocial();
location.hash = name;
}
async function J(path) { async function J(path) {
try { try {
const r = await fetch(API + path, { credentials: 'same-origin' }); const r = await fetch(API + path, { credentials: 'same-origin' });
@ -111,24 +205,54 @@ async function loadCfg() {
document.getElementById('r2-banner').style.display = c.r2.enabled ? '' : 'none'; document.getElementById('r2-banner').style.display = c.r2.enabled ? '' : 'none';
} }
function levelChip(lvl) {
const l = (lvl || 'r1').toLowerCase();
return `<span class="chip ${l}">${l.toUpperCase()}</span>`;
}
function levelSwitcher(macHash, current) {
const cur = (current || 'r1').toLowerCase();
return ['r0','r1','r2','r3'].map(l =>
l === cur ? '' :
`<button class="lvlbtn" onclick="setLevel('${macHash}','${l}')">${l.toUpperCase()}</button>`
).join('');
}
async function setLevel(macHash, level) {
try {
const fd = new FormData(); fd.append('level', level);
const r = await fetch(`${API}/admin/clients/${macHash}/level`, {
method: 'POST', body: fd, credentials: 'same-origin'
});
if (!r.ok) throw new Error('HTTP ' + r.status);
loadClients();
} catch (e) { alert('Échec changement niveau : ' + e.message); }
}
async function loadClients() { async function loadClients() {
const d = await J('/admin/clients'); const d = await J('/admin/clients/rich');
const el = document.getElementById('clients'); const el = document.getElementById('clients');
if (d.__error) { el.innerHTML = `<div class="empty">${d.__error}</div>`; return; } const rows = (d && d.clients) ? d.clients : (Array.isArray(d) ? d : null);
if (!d.length) { el.innerHTML = '<div class="empty">no clients</div>'; return; } if (!rows) { el.innerHTML = `<div class="empty">${(d && d.__error) || 'no data'}</div>`; return; }
let html = '<table><thead><tr><th>MAC (hash)</th><th>IP</th><th>state</th><th>score</th><th>last</th><th>Actions</th></tr></thead><tbody>'; if (!rows.length) { el.innerHTML = '<div class="empty">no clients</div>'; return; }
for (const c of d) { let html = '<table><thead><tr><th>MAC (hash)</th><th>IP</th><th>state</th><th>niveau</th><th>score</th><th>last</th><th>Actions</th></tr></thead><tbody>';
const ago = Math.round((Date.now()/1000 - c.last_seen) / 60); for (const c of rows) {
const ago = c.last_seen ? Math.round((Date.now()/1000 - c.last_seen) / 60) + 'm' : '—';
html += `<tr> html += `<tr>
<td><code>${c.mac_hash}</code></td> <td><code>${c.mac_hash}</code></td>
<td>${c.ip}</td> <td>${c.ip || '—'}</td>
<td><span class="state-${c.state}">${c.state}</span></td> <td><span class="state-${c.state}">${c.state || '—'}</span></td>
<td>${c.score}</td> <td>${levelChip(c.level)} ${levelSwitcher(c.mac_hash, c.level)}</td>
<td>${ago}m</td> <td>${c.score ?? '—'}</td>
<td>${ago}</td>
<td> <td>
<a href="${API}/admin/clients/${c.mac_hash}/report" target="_blank" style="color:var(--p31-peak);text-decoration:underline;font-size:0.8rem">PDF</a> <a class="link" href="${API}/admin/clients/${c.mac_hash}/social" target="_blank" style="font-size:0.8rem">🕸️ Carto</a>
· ·
<a href="javascript:void(0)" onclick="loadClientDetail('${c.mac_hash}')" style="color:var(--p31-peak);text-decoration:underline;font-size:0.8rem">Events</a> <a class="link" href="${API}/admin/clients/${c.mac_hash}/report" target="_blank" style="font-size:0.8rem">PDF</a>
·
<a class="link" href="javascript:void(0)" onclick="loadClientDetail('${c.mac_hash}')" style="font-size:0.8rem">Events</a>
·
<a class="link" href="javascript:void(0)" onclick="resetClient('${c.mac_hash}')" style="font-size:0.8rem;color:var(--red)">↺ Reset</a>
</td> </td>
</tr>`; </tr>`;
} }
@ -136,6 +260,21 @@ async function loadClients() {
el.innerHTML = html; el.innerHTML = html;
} }
// Phase 12.B — RAZ a specific client's accumulated statistics
// (social mapping graph + toolbox events). Operator-gated.
async function resetClient(macHash) {
if (!confirm(`Remettre à zéro les statistiques du client ${macHash.slice(0,16)}… ?\n\nCela efface sa cartographie sociale + ses events. Irréversible.`)) return;
try {
const r = await fetch(`${API}/admin/clients/${macHash}/reset`, {
method: 'POST', credentials: 'same-origin'
});
if (!r.ok) throw new Error('HTTP ' + r.status);
const j = await r.json();
alert(`RAZ effectuée : ${j.rows_deleted || 0} enregistrements supprimés.`);
loadClients();
} catch (e) { alert('Échec RAZ : ' + e.message); }
}
async function loadMetrics() { async function loadMetrics() {
const m = await J('/admin/metrics'); const m = await J('/admin/metrics');
const el = document.getElementById('metrics'); const el = document.getElementById('metrics');
@ -162,6 +301,7 @@ async function loadClientDetail(macHash) {
const div = document.getElementById('client-detail'); const div = document.getElementById('client-detail');
title.textContent = `🔍 Détails client — ${macHash}`; title.textContent = `🔍 Détails client — ${macHash}`;
card.style.display = ''; card.style.display = '';
card.scrollIntoView({behavior:'smooth', block:'nearest'});
div.innerHTML = '<div class="empty">loading…</div>'; div.innerHTML = '<div class="empty">loading…</div>';
const d = await J(`/admin/clients/${macHash}/events`); const d = await J(`/admin/clients/${macHash}/events`);
if (d.__error) { div.innerHTML = `<div class="empty">${d.__error}</div>`; return; } if (d.__error) { div.innerHTML = `<div class="empty">${d.__error}</div>`; return; }
@ -175,7 +315,7 @@ async function loadClientDetail(macHash) {
for (const e of d.recent) { for (const e of d.recent) {
const t = new Date(e.ts*1000).toLocaleTimeString(); const t = new Date(e.ts*1000).toLocaleTimeString();
const detail = e.host || e.url || e.sni || e.kind || '?'; const detail = e.host || e.url || e.sni || e.kind || '?';
recentHtml += `<tr><td>${t}</td><td><b>${e.source}</b></td><td><code style="font-size:0.78rem">${detail}</code></td></tr>`; recentHtml += `<tr><td>${t}</td><td><b>${e.source}</b></td><td><code>${detail}</code></td></tr>`;
} }
recentHtml += '</tbody></table>'; recentHtml += '</tbody></table>';
} }
@ -184,7 +324,7 @@ async function loadClientDetail(macHash) {
<p style="margin-bottom:0.5rem;font-size:0.85rem;color:var(--p31-dim)">Récents :</p> <p style="margin-bottom:0.5rem;font-size:0.85rem;color:var(--p31-dim)">Récents :</p>
${recentHtml} ${recentHtml}
<p style="margin-top:0.8rem"> <p style="margin-top:0.8rem">
<a href="${API}/admin/clients/${macHash}/report" target="_blank" style="color:var(--p31-peak);text-decoration:underline">⬇ Télécharger PDF complet</a> <a class="link" href="${API}/admin/clients/${macHash}/report" target="_blank">⬇ Télécharger PDF complet</a>
</p> </p>
`; `;
} }
@ -201,11 +341,88 @@ async function loadHealth() {
document.getElementById('version-badge').textContent = `v${h.version}`; document.getElementById('version-badge').textContent = `v${h.version}`;
} }
async function loadFilters() {
const el = document.getElementById('filters');
const d = await J('/admin/filter-control/list');
const items = (d && d.hosts) ? d.hosts : (Array.isArray(d) ? d : null);
if (!items) { el.innerHTML = `<li class="empty">${(d && d.__error) || 'no data'}</li>`; return; }
if (!items.length) { el.innerHTML = '<li class="empty">aucun host bypassé</li>'; return; }
el.innerHTML = items.map(h => {
const host = typeof h === 'string' ? h : (h.host || h.pattern || JSON.stringify(h));
const src = (typeof h === 'object' && h.source) ? h.source : '';
return `<li><code>${host}</code>${src ? `<span style="color:var(--p31-dim);font-size:0.72rem">${src}</span>` : ''}</li>`;
}).join('');
}
async function loadSocial() {
const agg = await J('/admin/social-aggregate?hours=24');
const kpi = document.getElementById('social-kpi');
const trk = document.getElementById('social-trackers');
const cli = document.getElementById('social-clients');
const cdn = document.getElementById('social-cdn');
if (agg.__error) {
kpi.innerHTML = `<span class="k">err</span><span class="v">${agg.__error}</span>`;
trk.innerHTML = cli.innerHTML = '';
if (cdn) cdn.innerHTML = '';
return;
}
kpi.innerHTML = `
<span class="k">Clients actifs</span> <span class="v">${agg.active_clients || 0}</span>
<span class="k">Trackers vus</span> <span class="v">${agg.total_trackers_seen || 0}</span>
<span class="k">Hors-UE (Phase C)</span> <span class="v">${agg.extra_eu_trackers || 0}</span>
<span class="k">Fenêtre</span> <span class="v">${agg.window_hours || 24}h</span>
`;
const td = agg.by_tracker_domain || [];
trk.innerHTML = td.length
? '<table><thead><tr><th>Tracker domain</th><th>hits</th><th>clients</th></tr></thead><tbody>' +
td.map(r => `<tr><td><code>${r.tracker_domain}</code></td><td>${r.hits}</td><td>${r.clients}</td></tr>`).join('') +
'</tbody></table>'
: '<div class="empty">aucun tracker dans la fenêtre</div>';
const bc = agg.by_client || [];
cli.innerHTML = bc.length
? '<table><thead><tr><th>Client (hash)</th><th>sites</th><th>trackers</th><th>last</th></tr></thead><tbody>' +
bc.map(r => {
const ago = r.last_seen ? Math.round((Date.now()/1000 - r.last_seen)/60) + 'm' : '—';
return `<tr><td><code>${(r.client_mac_hash||'').slice(0,16)}</code></td><td>${r.sites}</td><td>${r.trackers}</td><td>${ago}</td></tr>`;
}).join('') +
'</tbody></table>'
: '<div class="empty">aucun client</div>';
if (cdn) {
const bv = agg.by_cdn || [];
cdn.innerHTML = bv.length
? '<table><thead><tr><th>CDN / edge</th><th>hôtes</th></tr></thead><tbody>' +
bv.map(r => `<tr><td>${r.cdn_vendor}</td><td>${r.hosts}</td></tr>`).join('') +
'</tbody></table>'
: '<div class="empty">aucun CDN détecté</div>';
}
const ab = document.getElementById('social-antibot');
if (ab) {
const ba = agg.by_antibot || [];
ab.innerHTML = ba.length
? `<p style="font-size:0.8rem;color:var(--p31-dim);margin-bottom:0.5rem">${agg.antibot_clients||0} client(s) défié(s)</p>` +
'<table><thead><tr><th>Vendor</th><th>sites</th><th>clients</th><th>défis</th></tr></thead><tbody>' +
ba.map(r => `<tr><td>🤖 ${r.antibot_vendor}</td><td>${r.sites}</td><td>${r.clients}</td><td>${r.challenges}</td></tr>`).join('') +
'</tbody></table>'
: '<div class="empty">aucun anti-bot détecté</div>';
}
}
async function refreshAll() { async function refreshAll() {
await Promise.all([loadCfg(), loadClients(), loadHealth(), loadMetrics()]); await Promise.all([loadCfg(), loadClients(), loadHealth(), loadMetrics()]);
} }
// Deep-link : open the tab named in the URL hash on load.
const initial = (location.hash || '').replace('#', '');
if (['overview','clients','filtres','social','config'].includes(initial)) switchTab(initial);
refreshAll(); refreshAll();
setInterval(refreshAll, 10000); setInterval(() => {
// Only refresh the currently-visible tab's live data.
const active = document.querySelector('.tab.active')?.dataset.tab;
if (active === 'overview') { loadHealth(); loadMetrics(); }
else if (active === 'clients') loadClients();
else if (active === 'social') loadSocial();
}, 10000);
</script> </script>
</body> </body>
</html> </html>

View File

@ -0,0 +1,329 @@
/* SPDX-License-Identifier: LicenseRef-CMSD-1.0 */
/* Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr> */
/* Phase 11.B (#507) Social mapping per-client view stylesheet.
Palette: cosmos-black / gold-hermetic / cinnabar / matrix-green /
void-purple / cyber-cyan / text-primary / text-muted.
Typography: Cinzel (headers) + IM Fell English (body) + JetBrains Mono (code/data). */
:root {
--cosmos-black: #0a0a0f;
--gold-hermetic: #c9a84c;
--cinnabar: #e63946;
--matrix-green: #00ff41;
--void-purple: #6e40c9;
--cyber-cyan: #00d4ff;
--text-primary: #e8e6d9;
--text-muted: #6b6b7a;
}
* { box-sizing: border-box; }
html, body { margin: 0; padding: 0; background: var(--cosmos-black); color: var(--text-primary); }
html, body { height: 100%; }
body {
font-family: 'IM Fell English', Georgia, serif;
font-size: 15px;
line-height: 1.5;
/* Full-viewport flex layout : header (sticky), main (flex 1, owns the
graph), node-detail (slides over). Cards live inside main as a
compact bottom row so the graph stays hero. */
display: flex;
flex-direction: column;
height: 100vh;
overflow: hidden;
padding-bottom: env(safe-area-inset-bottom);
}
/* Header — compact, sticky, doesn't steal vertical space from the graph. */
.social-header {
flex: 0 0 auto;
display: flex; align-items: center; justify-content: space-between;
padding: 8px 14px;
border-bottom: 1px solid var(--gold-hermetic);
background: linear-gradient(180deg, rgba(201,168,76,.06), transparent);
}
.brand-title {
font-family: 'Cinzel', serif;
font-weight: 600;
font-size: 18px;
letter-spacing: 0.05em;
color: var(--gold-hermetic);
}
.brand-divider { color: var(--text-muted); margin: 0 8px; }
.brand-subtitle { color: var(--text-primary); font-style: italic; }
.lang {
font-family: 'JetBrains Mono', monospace;
font-size: 11px;
color: var(--text-muted);
border: 1px solid var(--text-muted);
border-radius: 3px;
padding: 2px 6px;
}
/* Main : flex column, takes all remaining height after the header.
Graph fills the body of main ; stats + cards float over the corners. */
.social-main {
flex: 1;
position: relative;
display: flex;
flex-direction: column;
min-height: 0; /* allow children to shrink past content size */
overflow: hidden;
}
/* Phase 11.B v2 intro paragraph hidden in full-viewport mode ; the
graph itself + the hover hint communicate the same thing. */
.social-intro { display: none; }
/* Stats tiles compact horizontal strip floating top-left over the
graph. Glassy background so the graph stays visually dominant. */
.stats {
position: absolute; top: 8px; left: 8px;
display: flex; gap: 6px;
z-index: 5;
pointer-events: none;
}
.stat-tile {
background: rgba(10, 10, 15, .65);
backdrop-filter: blur(6px);
-webkit-backdrop-filter: blur(6px);
border: 1px solid rgba(0, 212, 255, .35);
border-radius: 3px;
padding: 4px 9px;
display: flex; align-items: baseline; gap: 5px;
}
.stat-n {
font-family: 'JetBrains Mono', monospace;
font-size: 16px;
color: var(--cyber-cyan);
line-height: 1;
}
.stat-l { font-size: 10px; color: var(--text-muted); }
/* Graph fills all remaining vertical space inside .social-main. */
.graph-wrap {
position: relative;
flex: 1;
min-height: 0;
background: rgba(110, 64, 201, .03);
overflow: hidden;
}
.graph-hint {
position: absolute; top: 8px; right: 12px;
font-size: 10px; color: var(--text-muted);
font-family: 'JetBrains Mono', monospace;
pointer-events: none;
}
#social-graph { display: block; width: 100%; height: 100%; touch-action: none; }
/* d3 node + edge styling — colors come from the data class on each element. */
.node circle { stroke: var(--cosmos-black); stroke-width: 1.5; }
.node text {
font-family: 'JetBrains Mono', monospace;
font-size: 10px;
fill: var(--text-primary);
pointer-events: none;
}
.edge { fill: none; stroke: var(--cyber-cyan); stroke-opacity: .55; }
.edge.spoke { stroke: var(--cinnabar); stroke-opacity: .28; }
.edge.focused {
stroke: var(--gold-hermetic);
stroke-opacity: 1;
animation: pulse 1.4s ease-in-out infinite;
}
@keyframes pulse {
0%, 100% { stroke-opacity: .35; stroke-width: 1.2; }
50% { stroke-opacity: 1; stroke-width: 3; }
}
/* ── Round-Eye central hotspot (Phase 12.A) ── */
.node-eye circle { stroke: none; }
.eye-halo {
fill: var(--cinnabar);
opacity: .12;
transform-origin: center;
transform-box: fill-box;
animation: eye-breathe 2.6s ease-in-out infinite;
}
@keyframes eye-breathe {
0%, 100% { opacity: .08; r: 22px; }
50% { opacity: .22; r: 30px; }
}
.eye-sclera { fill: #e8e6d9; }
.eye-iris { fill: var(--cyber-cyan); }
.eye-pupil { fill: var(--cosmos-black); }
/* Node detail panel — bottom sheet on mobile, side panel on desktop */
.node-detail {
position: fixed; left: 0; right: 0; bottom: 0;
background: var(--cosmos-black);
border-top: 2px solid var(--gold-hermetic);
padding: 12px 16px env(safe-area-inset-bottom);
max-height: 50vh;
overflow-y: auto;
z-index: 100;
}
.node-detail header {
display: flex; justify-content: space-between; align-items: center;
margin-bottom: 8px;
}
.nd-domain {
font-family: 'JetBrains Mono', monospace;
color: var(--gold-hermetic);
font-size: 14px;
}
.nd-close {
background: transparent; border: 1px solid var(--text-muted);
color: var(--text-primary); border-radius: 3px;
padding: 2px 10px; cursor: pointer;
font-size: 18px; line-height: 1;
}
.node-detail dl { margin: 0; display: grid; grid-template-columns: max-content 1fr; gap: 4px 12px; font-size: 13px; }
.node-detail dt { color: var(--text-muted); font-style: italic; }
.node-detail dd { margin: 0; color: var(--text-primary); font-family: 'JetBrains Mono', monospace; font-size: 12px; word-break: break-word; }
/* Cards row floats at the bottom of the viewport as a 3-cell
horizontal strip ; each <details> opens upward as a popover so the
graph stays visible while you interact with a card. */
.card {
background: rgba(201, 168, 76, .04);
border: 1px solid rgba(201, 168, 76, .25);
border-radius: 4px;
}
.cards-row {
flex: 0 0 auto;
display: grid;
grid-template-columns: repeat(3, 1fr);
gap: 6px;
padding: 6px 8px env(safe-area-inset-bottom);
background: rgba(10, 10, 15, .85);
border-top: 1px solid var(--gold-hermetic);
}
.cards-row .card {
margin: 0;
padding: 6px 8px;
font-size: 12px;
}
.card summary {
cursor: pointer;
font-family: 'Cinzel', serif;
font-size: 12px;
letter-spacing: 0.04em;
color: var(--gold-hermetic);
list-style: none;
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}
/* Open-card popover : opens upward so cards stay readable above the
bottom strip. */
.cards-row .card[open] {
position: absolute;
left: 8px; right: 8px;
bottom: calc(50px + env(safe-area-inset-bottom));
background: rgba(10, 10, 15, .96);
border-color: var(--gold-hermetic);
padding: 12px 16px;
z-index: 8;
}
.card summary::-webkit-details-marker { display: none; }
.card summary::after { content: ' ▾'; color: var(--text-muted); font-size: 12px; }
.card[open] summary::after { content: ' ▴'; }
.card-pending { color: var(--text-muted); font-style: italic; margin: 8px 0 4px; }
.card-wipe { border-color: rgba(230, 57, 70, .4); background: rgba(230, 57, 70, .03); }
.card-wipe summary { color: var(--cinnabar); }
/* Buttons */
.wipe-btn {
display: block; width: 100%; margin-top: 8px;
background: transparent; color: var(--cinnabar);
border: 1px solid var(--cinnabar); border-radius: 3px;
padding: 10px;
font-family: 'Cinzel', serif; letter-spacing: 0.06em; cursor: pointer;
}
.wipe-btn:hover { background: rgba(230, 57, 70, .1); }
/* Phase 11.C — PDF report download button */
.pdf-btn {
display: block; width: 100%; margin-top: 8px;
text-align: center; text-decoration: none;
background: transparent; color: var(--gold-hermetic);
border: 1px solid var(--gold-hermetic); border-radius: 3px;
padding: 10px;
font-family: 'Cinzel', serif; letter-spacing: 0.06em;
}
.pdf-btn:hover { background: rgba(201, 168, 76, .12); }
/* Modal */
dialog#wipe-modal {
background: var(--cosmos-black);
color: var(--text-primary);
border: 2px solid var(--cinnabar);
border-radius: 4px;
max-width: 420px;
padding: 20px;
}
dialog::backdrop { background: rgba(10, 10, 15, .85); }
dialog h2 {
font-family: 'Cinzel', serif;
color: var(--cinnabar);
font-size: 18px;
letter-spacing: 0.05em;
margin: 0 0 12px;
}
.modal-actions { display: flex; gap: 10px; margin-top: 16px; }
.btn-secondary, .btn-danger {
flex: 1;
padding: 10px;
border-radius: 3px;
font-family: 'Cinzel', serif;
letter-spacing: 0.06em;
cursor: pointer;
font-size: 13px;
}
.btn-secondary { background: transparent; color: var(--text-primary); border: 1px solid var(--text-muted); }
.btn-danger { background: var(--cinnabar); color: var(--cosmos-black); border: 1px solid var(--cinnabar); font-weight: bold; }
.btn-danger:disabled { background: rgba(230, 57, 70, .25); color: var(--text-muted); cursor: not-allowed; border-color: var(--text-muted); }
.wipe-countdown {
color: var(--gold-hermetic);
font-family: 'JetBrains Mono', monospace;
font-size: 13px;
text-align: center;
margin: 8px 0;
}
@media (min-width: 720px) {
/* Desktop : node detail slides in from the right rail instead of bottom sheet */
.node-detail { top: 70px; bottom: auto; right: 14px; left: auto; width: 320px; max-height: 60vh; border-top: 0; border-left: 2px solid var(--gold-hermetic); border-radius: 4px; }
}
/* ── Anti-bot / "prove you're human" (Phase 12.B) ── */
.antibot-ring {
fill: none;
stroke: var(--cinnabar);
stroke-width: 1.5;
stroke-dasharray: 3 2;
opacity: .8;
animation: antibot-spin 6s linear infinite;
transform-origin: center;
transform-box: fill-box;
}
@keyframes antibot-spin { to { transform: rotate(360deg); } }
.antibot-alert {
position: absolute; top: 8px; left: 50%; transform: translateX(-50%);
z-index: 6;
background: rgba(230, 57, 70, .92);
color: #0a0a0f;
font-family: 'JetBrains Mono', monospace;
font-size: 11px;
font-weight: bold;
padding: 5px 12px;
border-radius: 3px;
max-width: 90%;
text-align: center;
box-shadow: 0 2px 10px rgba(230,57,70,.4);
}
/* ── Round-Eye ring guides (Phase 12.A/B) ── */
.ring-guides circle { fill: none; stroke-width: 1; pointer-events: none; }
.ring-inner { stroke: var(--gold-hermetic); stroke-opacity: .18; stroke-dasharray: 2 4; }
.ring-outer { stroke: var(--cyber-cyan); stroke-opacity: .14; stroke-dasharray: 2 6; }

View File

@ -0,0 +1,401 @@
// SPDX-License-Identifier: LicenseRef-CMSD-1.0
// Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
// Phase 11.B (#507) — Social mapping per-client view.
// Consumes the Phase A JSON contract (GET /social/graph/{token}) and
// renders a d3.js force-directed graph with:
// - site nodes (favicons fetched via /social/favicon/{domain})
// - tracker nodes (default cyber-cyan; family taxonomy in Phase C)
// - edges with stroke-width = log(reuse_count+1)*2 (locked in
// design lock round 2)
// - tap-to-focus on tracker nodes → bottom-sheet detail
// - wipe modal with 3-second countdown before confirm enable
// - empty state when no data
// No CDN dependency: d3 v7 is self-hosted at /toolbox/d3.v7.min.js.
(() => {
'use strict';
if (typeof d3 === 'undefined') {
console.error('[social] d3 missing — abort');
return;
}
const body = document.body;
const token = body.dataset.token;
// i18n is injected via <script>window.__SOCIAL_I18N__ = { … }</script>
// in the template head — keeps FR apostrophes intact (was a JSON.parse
// crash when inlined as a data-* attribute).
const i18n = window.__SOCIAL_I18N__ || {};
// ─── i18n helper ───
function t(key, vars = {}) {
let s = i18n[key] || key;
for (const [k, v] of Object.entries(vars)) {
s = s.replace(`{${k}}`, v);
}
return s;
}
// ─── DOM refs ───
const svgEl = document.getElementById('social-graph');
const svg = d3.select(svgEl);
const ndEl = document.getElementById('node-detail');
const wipeModal = document.getElementById('wipe-modal');
// ─── data binding helper ───
function bind(key, value) {
const el = document.querySelector(`[data-bind="${key}"]`);
if (el) el.textContent = value;
}
// Phase 12.B — show/hide the "challenged your humanity" banner.
function updateAntibotTile(sites, vendors) {
const el = document.getElementById('antibot-alert');
if (!el) return;
if (!sites) { el.hidden = true; return; }
const v = (vendors || []).join(', ');
el.textContent = t('antibot_alert', { n: sites }) + (v ? ' — ' + v : '');
el.hidden = false;
}
// ─── graph state ───
let simulation = null;
function svgSize() {
// Measure actual rendered size so the force center scales with the
// viewport. Falls back to a sane default if the layout hasn't
// settled yet.
const r = svgEl.getBoundingClientRect();
return { W: Math.max(r.width, 320), H: Math.max(r.height, 320) };
}
function clearGraph() {
svg.selectAll('*').remove();
if (simulation) simulation.stop();
}
function render(graph) {
clearGraph();
const { W, H } = svgSize();
svg.attr('viewBox', `0 0 ${W} ${H}`);
bind('total_trackers', graph.stats.total_trackers || 0);
bind('total_sites', graph.stats.total_sites || 0);
// Phase 12.B — "challenged your humanity" alert tile.
updateAntibotTile(graph.stats.antibot_sites || 0, graph.stats.antibot_vendors || []);
// Empty graph → just return ; the stats tiles already show 0/0 and
// the user knows. No persistent overlay message.
if (!graph.nodes.length) return;
// ── Round-Eye central hotspot (Phase 12.A) ──────────────────────
// The user's device is the EYE at the centre of the storm : every
// visited site orbits it, and every tracker that recognises the
// device reaches in toward the eye. Sites link to the eye, trackers
// link to their sites — so the eye is the gravitational centre and
// the densest tracker clusters become the visible "hot spots".
const EYE_ID = 'eye:device';
// Build d3 dataset: sites are union of all node.sites + tracker nodes themselves.
const siteSet = new Set();
for (const n of graph.nodes) for (const s of (n.sites || [])) siteSet.add(s);
const nodes = [];
const idx = new Map();
// The eye node, pinned to centre.
idx.set(EYE_ID, nodes.length);
nodes.push({ id: EYE_ID, label: '', kind: 'eye', fx: W / 2, fy: H / 2 });
for (const s of siteSet) {
idx.set('site:' + s, nodes.length);
nodes.push({ id: 'site:' + s, label: s, kind: 'site' });
}
for (const n of graph.nodes) {
idx.set('tracker:' + n.domain, nodes.length);
nodes.push({
id: 'tracker:' + n.domain,
label: n.domain,
kind: 'tracker',
hits: n.hits,
sites: n.sites,
first_seen: n.first_seen,
last_seen: n.last_seen,
cdn_vendor: n.cdn_vendor || null,
cache_status: n.cache_status || null,
antibot_vendor: n.antibot_vendor || null,
});
}
// Edges: eye → each site (gravity), then tracker → each of its sites.
const links = [];
for (const s of siteSet) {
links.push({ source: EYE_ID, target: 'site:' + s, reuse: 1, spoke: true });
}
for (const n of graph.nodes) {
const trackerKey = 'tracker:' + n.domain;
for (const s of (n.sites || [])) {
links.push({
source: trackerKey,
target: 'site:' + s,
reuse: n.hits,
});
}
}
// Phase A edges (cross-site shared trackers) also stamped as
// dashed accent links for emphasis.
const accentLinks = [];
for (const e of (graph.edges || [])) {
const a = 'site:' + e.src;
const b = 'site:' + e.dst;
if (idx.has(a) && idx.has(b)) {
accentLinks.push({
source: a, target: b, reuse: e.reuse_count, accent: true,
});
}
}
// Phase 11.B v4 — when we have many nodes (last test: 86 trackers +
// 60 sites = 146 nodes) the default force layout spreads them far
// outside the viewport, and the first autoFit caught the simulation
// mid-flight so only a single node was visible. Scale the forces
// with node count and pre-warm the simulation synchronously before
// first render so layout is already settled.
const N = nodes.length;
const chargeStr = N > 80 ? -28 : N > 30 ? -55 : -90;
const R = Math.min(W, H) / 2;
// Three concentric rings : eye (centre) → sites (inner) → trackers
// (outer). The radial force is now the DOMINANT force (strong pull
// to the ring), charge is weak (just spreads nodes along the ring),
// and links are weak springs so they don't yank nodes off-ring.
const ringR = d => d.kind === 'eye' ? 0 : d.kind === 'site' ? R * 0.40 : R * 0.80;
simulation = d3.forceSimulation(nodes)
.force('link', d3.forceLink([...links, ...accentLinks]).id(d => d.id)
.distance(d => d.spoke ? R * 0.40 : 24).strength(0.08))
.force('charge', d3.forceManyBody().strength(chargeStr).distanceMax(R * 0.6))
.force('radial', d3.forceRadial(ringR, W / 2, H / 2)
.strength(d => d.kind === 'eye' ? 0 : 0.9))
.force('collide', d3.forceCollide().radius(N > 120 ? 9 : N > 60 ? 13 : 20))
.alphaDecay(0.04);
// Phase 11.B v3 — content group that owns links + nodes ; the
// d3.zoom behavior applies its transform here so pan/pinch don't
// move the SVG itself (or its viewBox).
const content = svg.append('g').attr('class', 'content');
// Visible ring guides — the "Round-Eye" levels : inner = your sites,
// outer = the trackers reaching in. Drawn first so they sit behind.
const guides = content.append('g').attr('class', 'ring-guides');
[['ring-inner', R * 0.40], ['ring-outer', R * 0.80]].forEach(([cls, rad]) => {
guides.append('circle').attr('class', cls)
.attr('cx', W / 2).attr('cy', H / 2).attr('r', rad);
});
const linkSel = content.append('g').attr('class', 'links')
.selectAll('line').data([...links, ...accentLinks]).join('line')
.attr('class', d => d.accent ? 'edge accent' : 'edge')
.attr('stroke-width', d => Math.max(1, Math.log(1 + (d.reuse || 0)) * 1.8))
.attr('stroke-dasharray', d => d.accent ? '4,3' : null);
const nodeG = content.append('g').attr('class', 'nodes')
.selectAll('g').data(nodes).join('g')
.attr('class', d => 'node node-' + d.kind)
.call(d3.drag()
.on('start', (ev, d) => { if (d.kind === 'eye') return; if (!ev.active) simulation.alphaTarget(0.3).restart(); d.fx = d.x; d.fy = d.y; })
.on('drag', (ev, d) => { if (d.kind === 'eye') return; d.fx = ev.x; d.fy = ev.y; })
.on('end', (ev, d) => { if (d.kind === 'eye') return; if (!ev.active) simulation.alphaTarget(0); d.fx = null; d.fy = null; }))
.on('click', (ev, d) => focusNode(d, linkSel));
// CDN palette — distinct lens per edge-network vendor (Phase 12.A).
const CDN_COLORS = {
Cloudflare: '#f48120', Fastly: '#ff282d', Akamai: '#0099cc',
CloudFront: '#ff9900', Google: '#4285f4', Vercel: '#ffffff',
Netlify: '#00c7b7', BunnyCDN: '#ffb347', KeyCDN: '#3686ff',
Sucuri: '#00a651', 'Imperva/Incapsula': '#ff5a00', 'edge-cache': '#9aa0a6',
};
function nodeColor(d) {
if (d.kind === 'eye') return 'var(--cinnabar)';
if (d.kind === 'site') return 'var(--gold-hermetic)';
// Phase 12.B — anti-bot hosts get the highest-severity lens.
if (d.antibot_vendor) return 'var(--cinnabar)';
if (d.cdn_vendor && CDN_COLORS[d.cdn_vendor]) return CDN_COLORS[d.cdn_vendor];
return 'var(--cyber-cyan)';
}
// The central eye : concentric pulse rings + iris.
const eyeSel = nodeG.filter(d => d.kind === 'eye');
eyeSel.append('circle').attr('class', 'eye-halo').attr('r', 26);
eyeSel.append('circle').attr('class', 'eye-sclera').attr('r', 15);
eyeSel.append('circle').attr('class', 'eye-iris').attr('r', 7);
eyeSel.append('circle').attr('class', 'eye-pupil').attr('r', 3);
// Phase 12.B — anti-bot hosts get a severe pulsing warning ring.
nodeG.filter(d => d.kind === 'tracker' && d.antibot_vendor)
.append('circle').attr('class', 'antibot-ring').attr('r', 12);
// Site + tracker nodes.
nodeG.filter(d => d.kind !== 'eye').append('circle')
.attr('r', d => d.kind === 'tracker' ? 7 : 10)
.attr('fill', nodeColor)
.attr('stroke', d => (d.kind === 'tracker' && (d.cdn_vendor || d.antibot_vendor)) ? '#0a0a0f' : null)
.attr('stroke-width', d => (d.kind === 'tracker' && (d.cdn_vendor || d.antibot_vendor)) ? 1.5 : 0);
nodeG.filter(d => d.kind !== 'eye').append('text')
.attr('x', 12).attr('y', 4)
.text(d => (d.antibot_vendor ? '🤖 ' : '') + (d.label.length > 22 ? d.label.slice(0, 21) + '…' : d.label));
// ─── pan + pinch-zoom on the SVG (transform applies to content) ──
// Drag on a node calls d3.drag, drag on empty SVG calls d3.zoom's
// pan ; pinch and wheel always zoom. Touch-action: none on the
// svg (css) keeps the browser from intercepting these gestures.
const zoom = d3.zoom()
.scaleExtent([0.2, 6])
.filter((ev) => {
// Allow pan when the gesture didn't start on a node element.
// Allow all wheel + touch (multi-finger pinch).
if (ev.type === 'wheel' || (ev.touches && ev.touches.length > 1)) return true;
return !ev.target.closest('.node');
})
.on('zoom', (ev) => content.attr('transform', ev.transform));
svg.call(zoom).on('dblclick.zoom', () => autoFit(800));
// Auto-fit using node data positions (not getBBox — which can be
// skewed by text labels far outside the actual node cluster).
function autoFit(duration = 600) {
if (!nodes.length) return;
const xs = nodes.map(n => n.x).filter(Number.isFinite);
const ys = nodes.map(n => n.y).filter(Number.isFinite);
if (!xs.length) return;
const x0 = Math.min(...xs), x1 = Math.max(...xs);
const y0 = Math.min(...ys), y1 = Math.max(...ys);
const bw = Math.max(x1 - x0, 100);
const bh = Math.max(y1 - y0, 100);
const pad = 60;
const scale = Math.min(
(W - pad * 2) / bw,
(H - pad * 2) / bh,
2.5,
);
const cx = (x0 + x1) / 2;
const cy = (y0 + y1) / 2;
const tx = W / 2 - cx * scale;
const ty = H / 2 - cy * scale;
svg.transition().duration(duration).call(
zoom.transform,
d3.zoomIdentity.translate(tx, ty).scale(scale),
);
}
// Pre-warm the simulation synchronously so the layout is already
// settled before the user sees the first frame. 300 ticks is
// enough for ~150 nodes to find their resting positions.
for (let i = 0; i < 300; i++) simulation.tick();
// Now bind the live tick so subsequent micro-drift updates the DOM.
simulation.on('tick', () => {
linkSel
.attr('x1', d => d.source.x).attr('y1', d => d.source.y)
.attr('x2', d => d.target.x).attr('y2', d => d.target.y);
nodeG.attr('transform', d => `translate(${d.x},${d.y})`);
});
// Render once immediately with the pre-warmed positions.
linkSel
.attr('x1', d => d.source.x).attr('y1', d => d.source.y)
.attr('x2', d => d.target.x).attr('y2', d => d.target.y);
nodeG.attr('transform', d => `translate(${d.x},${d.y})`);
// Allow a gentle re-settle for visual polish (low alpha so it
// barely moves) and fit-to-viewport immediately.
simulation.alpha(0.05).restart();
requestAnimationFrame(() => autoFit(0));
// Re-fit on viewport resize.
let resizeTimer;
window.addEventListener('resize', () => {
clearTimeout(resizeTimer);
resizeTimer = setTimeout(() => {
const sz = svgSize();
svg.attr('viewBox', `0 0 ${sz.W} ${sz.H}`);
autoFit(400);
}, 150);
});
}
// ─── focus / detail panel ───
function focusNode(node, linkSel) {
if (node.kind !== 'tracker') { ndEl.hidden = true; return; }
bind('nd_domain', node.label);
bind('nd_country', '—'); // Phase C dependency (GeoIP)
bind('nd_asn', '—');
bind('nd_cdn', node.cdn_vendor ? (node.cdn_vendor + (node.cache_status ? ' · ' + node.cache_status : '')) : '—');
bind('nd_antibot', node.antibot_vendor ? ('🤖 ' + node.antibot_vendor) : '—');
bind('nd_sites', (node.sites || []).join(', ') || '—');
bind('nd_first_seen', node.first_seen ? new Date(node.first_seen * 1000).toISOString().slice(0, 16).replace('T', ' ') : '—');
bind('nd_last_seen', node.last_seen ? new Date(node.last_seen * 1000).toISOString().slice(0, 16).replace('T', ' ') : '—');
ndEl.hidden = false;
// Pulse the edges that touch this node
linkSel.classed('focused', l => l.source.id === node.id || l.target.id === node.id);
}
// ─── wipe modal ───
function openWipe() {
if (!wipeModal) return;
const confirmBtn = wipeModal.querySelector('[data-action="confirm-wipe"]');
const countdown = wipeModal.querySelector('[data-bind="wipe_countdown"]');
confirmBtn.disabled = true;
let n = 3;
countdown.hidden = false;
countdown.textContent = t('wipe_modal_countdown', { n });
const iv = setInterval(() => {
n--;
if (n <= 0) {
clearInterval(iv);
countdown.hidden = true;
confirmBtn.disabled = false;
} else {
countdown.textContent = t('wipe_modal_countdown', { n });
}
}, 1000);
wipeModal.showModal();
}
function cancelWipe() {
if (wipeModal) wipeModal.close();
}
async function confirmWipe() {
try {
const r = await fetch(`/social/wipe/${encodeURIComponent(token)}`, { method: 'POST' });
if (!r.ok) throw new Error('http ' + r.status);
const j = await r.json();
wipeModal.close();
alert(t('wipe_success', { n: j.rows_deleted || 0 }));
fetchGraph();
} catch (e) {
console.error('[social] wipe failed', e);
alert(t('error'));
}
}
// ─── event delegation ───
document.addEventListener('click', (ev) => {
const a = ev.target.closest('[data-action]');
if (!a) return;
switch (a.dataset.action) {
case 'close-nd': ndEl.hidden = true; break;
case 'open-wipe': openWipe(); break;
case 'cancel-wipe': cancelWipe(); break;
case 'confirm-wipe': confirmWipe(); break;
}
});
// ─── fetch + bootstrap ───
async function fetchGraph() {
try {
const r = await fetch(`/social/graph/${encodeURIComponent(token)}?since=86400`);
if (!r.ok) throw new Error('http ' + r.status);
const g = await r.json();
render(g);
} catch (e) {
console.error('[social] fetch failed', e);
}
}
fetchGraph();
})();