mirror of
https://github.com/CyberMind-FR/secubox-deb.git
synced 2026-07-01 11:47:31 +00:00
Compare commits
1 Commits
3064fb61fd
...
55626e510b
| Author | SHA1 | Date | |
|---|---|---|---|
| 55626e510b |
|
|
@ -3,79 +3,6 @@
|
|||
|
||||
---
|
||||
|
||||
## 2026-06-10 — Phase 11 social mapping (A+B) + system triage + v2.13.14 (ref #502-#509)
|
||||
|
||||
### Package bumps
|
||||
|
||||
| Package | from → to |
|
||||
|---|---|
|
||||
| secubox-toolbox | 2.5.2 → **2.6.0** (#505 Phase 11.A backend) |
|
||||
| secubox-toolbox | 2.6.0 → **2.6.1** (#507 Phase 11.B frontend) |
|
||||
| secubox-waf | 1.2.1 → **1.2.2** (#509 double-buffer cache) |
|
||||
| Release tag | **v2.13.14** |
|
||||
|
||||
### Phase 11 — Social mapping per device (#502)
|
||||
|
||||
**11.A backend** (`secubox-toolbox 2.6.0`, PR #506) — `social.py`
|
||||
correlation engine + 3 SQLite tables (`social_edges` / `social_nodes`
|
||||
/ `social_links`), `social_graph.py` mitm addon (cookie_id_hash =
|
||||
sha256, never persists raw values), `/social/graph/{token}` +
|
||||
`/social/wipe/{token}` (RGPD art. 17) + `/admin/social-aggregate`
|
||||
endpoints, fold + purge background tasks.
|
||||
|
||||
**11.B frontend** (`secubox-toolbox 2.6.1`, #507) — d3 force-directed
|
||||
graph view at `/social/{token}`, FR/EN i18n, server-side favicon proxy
|
||||
(7d cache), wipe modal with 3s countdown, full-viewport layout with
|
||||
pan/pinch-zoom + pre-warm + autoFit. Splash menu link `/social/me`
|
||||
(🕸️ Ma carto) resolving R3 peers via X-R3-Peer sentinel.
|
||||
|
||||
**Live result** : graph renders real cross-site tracking on gk2 — the
|
||||
ad-tech relay `35.214.136.108` bridging 360yield + seedtag +
|
||||
smartadserver + smilewanted publishers, surfacing exactly the
|
||||
fingerprint reuse Phase 11 targets.
|
||||
|
||||
**Critical live-deploy fixes** : addon relative-import never resolved
|
||||
(mitmproxy loads addons top-level) → inlined; **PYTHONPATH missing in
|
||||
mitm-wg launcher** silently degraded every addon's `secubox_toolbox`
|
||||
imports → fixed globally (also un-degraded inject_banner's host
|
||||
classification + GeoIP); i18n moved to `<script>` block (FR
|
||||
apostrophes broke JSON.parse); StaticFiles mount + chmod 0755 www
|
||||
(kbin HAProxy path bypasses nginx).
|
||||
|
||||
**11.C** (#508) — WIP checkpoint `55626e51` : schema (consent_state +
|
||||
GeoIP columns), EU/EEA whitelist, GeoIP fold enrichment, evidence()
|
||||
helper. PDF generator + consent-probe addon + frontend wire pending.
|
||||
|
||||
### System triage on gk2
|
||||
|
||||
- **CrowdSec firewall** — bouncer ran healthy but had no nft tables
|
||||
(external flush). Restart recreated `ip crowdsec` + `ip6 crowdsec6`,
|
||||
100 live decisions.
|
||||
- **WAF + SOC empty cards** — `/var/log/secubox` was 0750
|
||||
secubox-toolbox, blocking the aggregator (user `secubox`) from
|
||||
traversing to read `waf-threats.log`. chmod 0755 live.
|
||||
- **WAF /stats 30s+ timeout** — `_get_threat_stats()` re-parsed the full
|
||||
110 MB / 332k-entry JSONL on every request (89% aggregator CPU).
|
||||
Fixed via #509 double-buffered cache : disk-persisted counters +
|
||||
byte-position incremental tail reading. `/waf/stats` now 30-37 ms.
|
||||
- **PeerTube + PhotoPrism 502** — LXCs were STOPPED; `lxc-start` → live.
|
||||
|
||||
### CI + release
|
||||
|
||||
- #503/PR #504 — drop espressobin-v7 + ultra from the scheduled
|
||||
build-image matrix (cause of the v2.13.9-12 release failures).
|
||||
- #509/PR #510 — double-buffer WAF cache.
|
||||
- Merged both to master (`3ebb4477`, `a6f44807`), tagged **v2.13.14**.
|
||||
|
||||
### Carried forward
|
||||
|
||||
- Round Eye gadget remote-link to gk2 (shows local metrics only) —
|
||||
needs Pi-side investigation.
|
||||
- admin.gk2/toolbox/ tab surfacing decision (proxy/iframe/sub-tab).
|
||||
- `/var/log/secubox` 0755 source-side postinst patch (live-only for now).
|
||||
|
||||
---
|
||||
|
||||
## 2026-06-09 — Phase 10 banner injection perf quick wins + postinst regression fix (ref #501)
|
||||
|
||||
### Package bumps
|
||||
|
|
|
|||
|
|
@ -1,40 +1,10 @@
|
|||
# TODO — SecuBox-DEB Backlog
|
||||
*Mis à jour : 2026-06-10*
|
||||
*Mis à jour : 2026-06-09*
|
||||
|
||||
---
|
||||
|
||||
## 🔥 P0 — Immediate (in flight)
|
||||
|
||||
### Phase 11 — Social mapping per device (#502)
|
||||
|
||||
- [x] **11.A backend** (#505 / PR #506, `secubox-toolbox 2.6.0`) — correlation
|
||||
engine + SQLite + API. Déployé live gk2.
|
||||
- [x] **11.B frontend** (#507, `2.6.1`) — d3 graph + i18n FR/EN + favicon
|
||||
proxy + wipe modal + full-viewport pan/zoom. Live `/social/me`.
|
||||
- [ ] **11.C evidence + PDF** (#508) — reprendre depuis checkpoint
|
||||
`55626e51` : consent-probe addon (OneTrust/Didomi/Quantcast/Sourcepoint)
|
||||
+ extra-EU flag + PDF bilingue FR/EN + wire frontend (remplacer le
|
||||
placeholder "Rapport PDF arrive en Phase 11.C").
|
||||
- [ ] **Merger PR #506** (11.A backend) → master quand prêt.
|
||||
- [ ] **11.D opérateur** (futur) — dashboard agrégat `/admin/social/` HTML
|
||||
(l'endpoint JSON `/admin/social-aggregate` existe déjà depuis 2.6.0).
|
||||
|
||||
### Système — bugs gk2 (2026-06-10)
|
||||
|
||||
- [x] **CrowdSec firewall** — restart bouncer → tables nft recréées.
|
||||
- [x] **WAF /var/log/secubox traversal** — chmod 0755 live.
|
||||
- [x] **WAF /stats perf** (#509 / PR #510, `secubox-waf 1.2.2`) — double-buffer
|
||||
cache. Mergé + `v2.13.14`.
|
||||
- [x] **PeerTube + PhotoPrism** — LXC redémarrés.
|
||||
- [ ] **Round Eye gadget** — ne voit plus gk2, métriques locales only.
|
||||
Investigation Pi Zero (dashboard `localhost:8000` proxie vers gk2 via OTG).
|
||||
- [ ] **admin.gk2/toolbox/ tab** — toolbox déjà wiré (`/toolbox/` alias +
|
||||
sidebar). User veut surfacer l'UI kbin/admin dedans — décision en
|
||||
attente : proxy_pass `10.99.0.1:8088/admin/` (cleanest) / iframe (CSP) /
|
||||
sous-tab frontend-only.
|
||||
- [ ] **Postinst `/var/log/secubox` 0755** — porter le fix live en source
|
||||
(même pattern que `/etc/secubox` parent + `/usr/share/secubox/www`).
|
||||
|
||||
### Phase 10 — Banner injection perf (#501) — ✅ shipped 2026-06-09
|
||||
|
||||
- [x] **Banner perf quick wins** (`secubox-toolbox` 2.5.1, commit `ce059d0f`)
|
||||
|
|
|
|||
|
|
@ -1,73 +1,5 @@
|
|||
# WIP — Work In Progress
|
||||
*Mis à jour : 2026-06-10*
|
||||
|
||||
---
|
||||
|
||||
## 🔄 2026-06-10 : Phase 11 social mapping (A+B) + system triage round (ref #502-#509)
|
||||
|
||||
Grosse journée : Phase 11 social mapping shippé jusqu'au frontend live,
|
||||
puis une cascade de fixes système découverts par l'utilisateur en
|
||||
production sur gk2.
|
||||
|
||||
### ✅ Done — Phase 11 social mapping (#502 parent)
|
||||
|
||||
| Issue | Phase | État |
|
||||
|---|---|---|
|
||||
| #505 / PR #506 | **11.A backend** : correlation engine + SQLite + API | ✅ mergeable, déployé live `secubox-toolbox 2.6.0` |
|
||||
| #507 | **11.B frontend** : d3 graph + i18n FR/EN + favicon proxy + wipe modal | ✅ déployé live `2.6.1`, branche poussée |
|
||||
| #508 | **11.C evidence + PDF** | 🔄 WIP checkpoint `55626e51` (schema + GeoIP fold + evidence helper) |
|
||||
|
||||
**Design** : 2 rounds de design lock sur #502 (Gemini + GPT mockups),
|
||||
edge-thickness + animated-pulse + tracker bottom-sheet + 3s wipe
|
||||
countdown verrouillés.
|
||||
|
||||
**Live URL** : `https://kbin.gk2.secubox.in/social/me` (splash → 🕸️ Ma carto).
|
||||
Le graphe montre les trackers cross-site réels (relais ad-tech
|
||||
`35.214.136.108` reliant 360yield + seedtag + smartadserver + smilewanted).
|
||||
|
||||
**Fixes live-deploy critiques découverts** :
|
||||
- `social_graph.py` : `from . import local_store` ne résolvait jamais
|
||||
(mitmproxy charge les addons en top-level) → inliné le WG peer hash.
|
||||
- **PYTHONPATH manquant dans le launcher mitm-wg** : TOUS les addons
|
||||
(`inject_banner` dpi/geo/store, `social_graph`) avaient leurs
|
||||
`from secubox_toolbox import …` silencieusement dégradés. Fix global.
|
||||
- i18n déplacé de `data-*` attr vers `<script>` (apostrophes FR
|
||||
cassaient `JSON.parse`).
|
||||
- StaticFiles mount + chmod 0755 `/usr/share/secubox/www` (kbin passe
|
||||
par HAProxy direct uvicorn, bypass nginx).
|
||||
- d3 : full-viewport + pan/pinch-zoom + pre-warm 300 ticks + autoFit
|
||||
data-based (146 nodes spread off-screen avant).
|
||||
|
||||
### ✅ Done — triage système gk2 (2026-06-10)
|
||||
|
||||
| Bug | Cause racine | Fix |
|
||||
|---|---|---|
|
||||
| CrowdSec firewall status faux | bouncer tournait mais sans tables nft (flush externe) | restart bouncer → `ip crowdsec` + `ip6 crowdsec6` recréées, 100 décisions live |
|
||||
| WAF /threats + tracked attackers vides | `/var/log/secubox` 0750 secubox-toolbox bloquait traversal aggregator (user `secubox`) | chmod 0755 live |
|
||||
| WAF /stats timeout 30s+ | `_get_threat_stats()` re-parsait 110 MB / 332k JSONL à CHAQUE requête (CPU 89%) | **#509 double-buffer cache** (disk + byte-position incrémental) `secubox-waf 1.2.2` |
|
||||
| SOC /soc/ status WAF+firewall faux | consommait les mêmes endpoints WAF cassés | résolu en cascade par le fix WAF |
|
||||
| PeerTube + PhotoPrism 502 | LXC STOPPED | `lxc-start` → 200 / 307 |
|
||||
|
||||
### ✅ Done — CI + release
|
||||
|
||||
- **#503 / PR #504** : drop espressobin-v7 + ultra du matrix build-image
|
||||
scheduled (faisaient échouer le pipeline release v2.13.9-12).
|
||||
- **#509 / PR #510** : double-buffer WAF cache.
|
||||
- **Merge #504 + #510 → master** (`3ebb4477`, `a6f44807`).
|
||||
- **Tag `v2.13.14`** poussé.
|
||||
|
||||
### ⬜ Next up
|
||||
|
||||
- **Round Eye gadget** : ne voit plus le lien gk2, montre ses métriques
|
||||
locales. iface `eye-remote` UP côté gk2, route `/api/v1/eye-remote/*`
|
||||
renvoie page erreur. Investigation côté Pi Zero nécessaire.
|
||||
- **admin.gk2/toolbox/ tab** : le toolbox est DÉJÀ wiré (`/toolbox/`
|
||||
alias + sidebar). User veut surfacer l'UI kbin/admin dedans —
|
||||
décision en attente : proxy_pass / iframe / sous-tab.
|
||||
- **Phase 11.C** : reprendre depuis `55626e51` (consent probe addon +
|
||||
extra-EU flag + PDF bilingue + wire frontend).
|
||||
- **Postinst patch** : `/var/log/secubox` 0755 en source (pour l'instant
|
||||
fix live uniquement) — même pattern que `/etc/secubox` + `www`.
|
||||
*Mis à jour : 2026-06-09*
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
19
.github/workflows/build-image.yml
vendored
19
.github/workflows/build-image.yml
vendored
|
|
@ -24,8 +24,8 @@ on:
|
|||
type: choice
|
||||
options:
|
||||
- mochabin
|
||||
- espressobin-v7 # on-demand only — disabled in scheduled CI, ref #503
|
||||
- espressobin-ultra # on-demand only — disabled in scheduled CI, ref #503
|
||||
- espressobin-v7
|
||||
- espressobin-ultra
|
||||
- vm-x64
|
||||
- vm-arm64
|
||||
- rpi400
|
||||
|
|
@ -50,12 +50,7 @@ jobs:
|
|||
fail-fast: false
|
||||
matrix:
|
||||
# Handle all event types: push (tags), workflow_call, workflow_dispatch
|
||||
# Scheduled / tag-push matrix excludes espressobin-v7 + espressobin-ultra (#503) :
|
||||
# those board builds fail in the cross-arm64 chroot stage and block the
|
||||
# downstream release.yml job for every image even though fail-fast is off.
|
||||
# Operators can still build them on-demand via workflow_dispatch (the
|
||||
# choice list above retains the entries).
|
||||
board: ${{ (github.event_name == 'push' || (inputs.board == 'all' || inputs.board == '')) && fromJson('["mochabin","vm-x64","rpi400"]') || (github.event.inputs.board == 'all' && fromJson('["mochabin","vm-x64","rpi400"]') || fromJson(format('["{0}"]', inputs.board || github.event.inputs.board || 'vm-x64'))) }}
|
||||
board: ${{ (github.event_name == 'push' || (inputs.board == 'all' || inputs.board == '')) && fromJson('["mochabin","espressobin-v7","espressobin-ultra","vm-x64","rpi400"]') || (github.event.inputs.board == 'all' && fromJson('["mochabin","espressobin-v7","espressobin-ultra","vm-x64","rpi400"]') || fromJson(format('["{0}"]', inputs.board || github.event.inputs.board || 'vm-x64'))) }}
|
||||
|
||||
steps:
|
||||
- name: Checkout
|
||||
|
|
@ -226,17 +221,15 @@ jobs:
|
|||
| Image | Board | Architecture | Description |
|
||||
|-------|-------|--------------|-------------|
|
||||
| `secubox-mochabin-bookworm.img.gz` | MOCHAbin | arm64 | Marvell Armada 7040 (Pro) |
|
||||
| `secubox-espressobin-v7-bookworm.img.gz` | ESPRESSObin v7 | arm64 | Marvell Armada 3720 (Lite) |
|
||||
| `secubox-espressobin-ultra-bookworm.img.gz` | ESPRESSObin Ultra | arm64 | Marvell Armada 3720 (Lite+) |
|
||||
| `secubox-rpi400-bookworm.img.gz` | Raspberry Pi 400 | arm64 | Pi 400 / Pi 4 |
|
||||
| `secubox-vm-x64-bookworm.img.gz` | VirtualBox/QEMU | amd64 | VM for testing |
|
||||
| `create-qemu-arm64-vm.sh` | QEMU ARM64 | script | Run ARM64 on x86 hosts |
|
||||
|
||||
*ESPRESSObin v7 and Ultra board images are no longer published in
|
||||
scheduled releases (see #503). Board support remains in tree and
|
||||
on-demand builds are available via workflow_dispatch.*
|
||||
|
||||
### Installation
|
||||
|
||||
**ARM64 boards (MOCHAbin, Raspberry Pi 400):**
|
||||
**ARM64 boards (MOCHAbin, ESPRESSObin):**
|
||||
```bash
|
||||
# Flash to SD card or eMMC
|
||||
gunzip -c secubox-mochabin-bookworm.img.gz | sudo dd of=/dev/sdX bs=4M status=progress
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@ case "$1" in
|
|||
install -d -o secubox -g secubox -m 750 /run/secubox
|
||||
install -d -o secubox -g secubox -m 750 /var/lib/secubox
|
||||
install -d -o secubox -g secubox -m 750 /var/lib/secubox/admin
|
||||
install -d -o root -g secubox -m 0755 /var/log/secubox
|
||||
install -d -o root -g secubox -m 750 /var/log/secubox
|
||||
systemctl daemon-reload
|
||||
systemctl enable secubox-admin.service
|
||||
systemctl start secubox-admin.service || true
|
||||
|
|
|
|||
|
|
@ -20,7 +20,7 @@ case "$1" in
|
|||
# NE PAS le toucher ici — l'écraser bloque la traversée nginx (www-data) et
|
||||
# casse tous les /api/v1/<module>/* en 502 (cf. #471). Si besoin d'un
|
||||
# sous-dossier privé, utiliser /run/secubox/mesh/ (et non le parent).
|
||||
install -d -m 0755 -o secubox-mesh -g secubox-mesh /var/log/secubox
|
||||
install -d -m 0750 -o secubox-mesh -g secubox-mesh /var/log/secubox
|
||||
|
||||
# 4. Verrou régulatoire FR (idempotent ; ne pas planter si iw absent)
|
||||
if command -v iw >/dev/null 2>&1; then
|
||||
|
|
|
|||
|
|
@ -28,7 +28,5 @@
|
|||
"wipe_success": "Your data has been erased. {n} rows deleted.",
|
||||
"loading": "Loading…",
|
||||
"error": "Loading error.",
|
||||
"lang_label": "EN",
|
||||
"card_pdf_download": "⬇ Download PDF report (FR/EN)",
|
||||
"card_evidence_active": "Compliance analysis: trackers before consent (GDPR art. 6.1.a + 7) and extra-EU transfers (art. 44). See the PDF report for details."
|
||||
}
|
||||
"lang_label": "EN"
|
||||
}
|
||||
|
|
|
|||
|
|
@ -28,7 +28,5 @@
|
|||
"wipe_success": "Vos données ont été effacées. {n} enregistrements supprimés.",
|
||||
"loading": "Chargement…",
|
||||
"error": "Erreur de chargement.",
|
||||
"lang_label": "FR",
|
||||
"card_pdf_download": "⬇ Télécharger le rapport PDF (FR/EN)",
|
||||
"card_evidence_active": "Analyse de conformité : traqueurs avant consentement (RGPD art. 6.1.a + 7) et transferts hors UE (art. 44). Voir le rapport PDF pour le détail."
|
||||
}
|
||||
"lang_label": "FR"
|
||||
}
|
||||
|
|
|
|||
|
|
@ -86,6 +86,9 @@ a:hover{text-decoration:underline}
|
|||
<a href="/wg/qr.png" class=qi title="QR profil WireGuard">
|
||||
<span class=qi-emoji>📱</span><span class=qi-label>QR profil</span>
|
||||
</a>
|
||||
<a href="/admin/" class=qi title="Admin webui">
|
||||
<span class=qi-emoji>🛠️</span><span class=qi-label>Admin</span>
|
||||
</a>
|
||||
<a href="https://github.com/CyberMind-FR/secubox-deb/wiki/R3-WireGuard-install" class=qi title="Wiki R3 multi-OS">
|
||||
<span class=qi-emoji>📖</span><span class=qi-label>Wiki</span>
|
||||
</a>
|
||||
|
|
|
|||
|
|
@ -59,7 +59,7 @@
|
|||
<nav class="cards-row">
|
||||
<details class="card">
|
||||
<summary>{{ t.card_evidence }}</summary>
|
||||
<p class="card-pending">{{ t.card_evidence_active }}</p>
|
||||
<p class="card-pending">{{ t.card_evidence_pending }}</p>
|
||||
</details>
|
||||
<details class="card card-wipe">
|
||||
<summary>{{ t.card_wipe }}</summary>
|
||||
|
|
@ -67,7 +67,7 @@
|
|||
</details>
|
||||
<details class="card">
|
||||
<summary>{{ t.card_pdf }}</summary>
|
||||
<a class="pdf-btn" href="/social/report/{{ token }}.pdf" target="_blank" rel="noopener">{{ t.card_pdf_download }}</a>
|
||||
<p class="card-pending">{{ t.card_pdf_pending }}</p>
|
||||
</details>
|
||||
</nav>
|
||||
</main>
|
||||
|
|
|
|||
|
|
@ -1,53 +1,3 @@
|
|||
secubox-toolbox (2.6.3-1~bookworm1) bookworm; urgency=medium
|
||||
|
||||
* Phase 11.C (#508, parent #502) — social mapping evidence + bilingual
|
||||
FR/EN PDF report.
|
||||
- mitmproxy_addons/social_graph.py : consent-platform probe.
|
||||
Detects OneTrust / Didomi / Quantcast / Sourcepoint cookies +
|
||||
loader URLs per (peer, site). Each recorded edge is stamped
|
||||
consent_state in {none_seen, pre_consent, post_consent}.
|
||||
pre_consent = a tracker fired while the site runs a CMP but no
|
||||
consent cookie was seen yet (RGPD art. 6.1.a + 7 evidence).
|
||||
- secubox_toolbox/social.py : schema (consent_state + GeoIP
|
||||
columns) + EU/EEA whitelist + GeoIP fold + evidence() helper.
|
||||
- secubox_toolbox/social_report.py : NEW — bilingual FR/EN
|
||||
evidence PDF via fpdf2 (same engine as reports.py). Fact-only
|
||||
cover + summary + pre-consent table + extra-EU table + RGPD
|
||||
article references. Text fallback when fpdf2 is absent.
|
||||
- secubox_toolbox/api.py : GET /social/report/{token}.pdf, same
|
||||
HMAC-token gate as /social/graph/{token}.
|
||||
- conf/social_view.html.j2 + i18n : the "Rapport PDF" card now
|
||||
has a real download button ; the "Évidence juridique" card
|
||||
switched to the active compliance summary.
|
||||
Phase 11 (social mapping per device) is now feature-complete :
|
||||
A (backend) + B (d3 view) + C (evidence + PDF).
|
||||
|
||||
-- Gerald KERMA <devel@cybermind.fr> Wed, 10 Jun 2026 09:30:00 +0200
|
||||
|
||||
secubox-toolbox (2.6.2-1~bookworm1) bookworm; urgency=medium
|
||||
|
||||
* ToolBox WebUI sub-tab navigation + kbin /admin/ removal (#513).
|
||||
- www/toolbox/index.html : rebuilt with a 5-tab nav
|
||||
(Vue d'ensemble / Clients / Filtres MITM / Cartographie
|
||||
sociale / Config). Each tab lazy-loads its data; the live
|
||||
refresh interval only polls the visible tab. URL hash
|
||||
deep-links a tab (e.g. /toolbox/#social).
|
||||
- Clients tab folds in the R0/R1/R2/R3 level switcher that
|
||||
used to live only in the kbin inline admin UI (calls
|
||||
/admin/clients/{mac}/level).
|
||||
- Cartographie sociale tab surfaces the Phase 11 operator
|
||||
aggregate (/admin/social-aggregate) : KPI tiles + top
|
||||
tracker domains + anonymized client table.
|
||||
- api.py : DELETED the inline kbin /admin/ HTML route
|
||||
(admin_index(), ~230 lines). The canonical operator
|
||||
dashboard is now admin.gk2.secubox.in/toolbox/. All
|
||||
/admin/* JSON API routes are untouched.
|
||||
- conf/landing.html.j2 : removed the 🛠️ Admin quicknav icon
|
||||
(the page it pointed at no longer exists).
|
||||
kbin.gk2.secubox.in/admin/ now 404s by design.
|
||||
|
||||
-- Gerald KERMA <devel@cybermind.fr> Wed, 10 Jun 2026 09:15:00 +0200
|
||||
|
||||
secubox-toolbox (2.6.1-1~bookworm1) bookworm; urgency=medium
|
||||
|
||||
* Phase 11.B (#507, parent #502) — social mapping per-client view +
|
||||
|
|
|
|||
|
|
@ -44,13 +44,7 @@ case "$1" in
|
|||
|
||||
# 4. Storage dir (SQLite + future PDF reports)
|
||||
install -d -m 0750 -o secubox-toolbox -g secubox-toolbox /var/lib/secubox/toolbox
|
||||
# /var/log/secubox is a SHARED parent traversed by many service users
|
||||
# (the aggregator runs as `secubox` and reads waf-threats.log under
|
||||
# here). It MUST be 0755 — a 0750 owned by secubox-toolbox silently
|
||||
# breaks WAF + SOC dashboards for the `secubox` user (#511, regressed
|
||||
# the /waf/ + /soc/ pages on gk2 2026-06-10). Per-module log files +
|
||||
# subdirs inside keep their own restricted perms.
|
||||
install -d -m 0755 -o secubox-toolbox -g secubox-toolbox /var/log/secubox
|
||||
install -d -m 0750 -o secubox-toolbox -g secubox-toolbox /var/log/secubox
|
||||
|
||||
# 4a. Phase 11.B (#507) — make /usr/share/secubox/www traversable so
|
||||
# the FastAPI StaticFiles mount can serve /toolbox/social.{css,js} +
|
||||
|
|
|
|||
|
|
@ -171,105 +171,6 @@ def _ja4_hash(flow) -> Optional[str]:
|
|||
return None
|
||||
|
||||
|
||||
# ─── consent-platform detection (Phase 11.C #508) ───
|
||||
#
|
||||
# We detect the four dominant CMP (Consent Management Platform) cookies.
|
||||
# Their PRESENCE on a flow means the user has interacted with the consent
|
||||
# banner on this site at least once. We track, per (peer, site) :
|
||||
# - whether the site is KNOWN to run a CMP (we've seen the cookie OR the
|
||||
# loader path), and
|
||||
# - whether a consent cookie has been observed yet.
|
||||
# This lets us classify each tracker edge as pre/post/none-consent.
|
||||
#
|
||||
# Names are matched case-insensitively as a prefix (OneTrust appends
|
||||
# region suffixes, Didomi rotates tokens, etc.).
|
||||
_CMP_COOKIE_PREFIXES = (
|
||||
"optanonconsent", "onetrustconsent", "optanonalertboxclosed", # OneTrust
|
||||
"didomi_token", "euconsent-v2", # Didomi / IAB TCF
|
||||
"__qca", "quantcast", # Quantcast
|
||||
"sp_choice", "consentuid", "_sp_", # Sourcepoint
|
||||
)
|
||||
# CMP loader URL fragments — seeing the site REQUEST one of these proves
|
||||
# the site runs a consent platform even before the cookie is set.
|
||||
_CMP_LOADER_FRAGMENTS = (
|
||||
"cdn.cookielaw.org", "onetrust.com", # OneTrust
|
||||
"sdk.privacy-center.org", "didomi.io", # Didomi
|
||||
"quantcast.mgr.consensu.org", "quantcast.com/choice", # Quantcast
|
||||
"sourcepoint.mgr.consensu.org", "sp-prod.net", # Sourcepoint
|
||||
)
|
||||
|
||||
# Per-(peer, site) consent observation log. Bounded soft-cap : if it
|
||||
# grows past 20k entries we drop it wholesale (a fresh session rebuild is
|
||||
# cheap and the salt rotates daily anyway).
|
||||
_consent_log: dict = {}
|
||||
|
||||
|
||||
def _consent_key(mac_hash: str, site: str) -> tuple:
|
||||
return (mac_hash, site)
|
||||
|
||||
|
||||
def _update_consent_log(mac_hash: str, src_site: str, flow) -> None:
|
||||
"""Observe whether this flow reveals a CMP cookie or loader for the
|
||||
(peer, site) pair, and update the in-memory log."""
|
||||
try:
|
||||
if len(_consent_log) > 20000:
|
||||
_consent_log.clear()
|
||||
key = _consent_key(mac_hash, src_site)
|
||||
state = _consent_log.get(key, {"has_cmp": False, "consented": False})
|
||||
|
||||
# 1) CMP loader request → the site runs a consent platform.
|
||||
url = (flow.request.pretty_url or "").lower()
|
||||
if any(frag in url for frag in _CMP_LOADER_FRAGMENTS):
|
||||
state["has_cmp"] = True
|
||||
|
||||
# 2) CMP cookie present (either direction) → consent recorded.
|
||||
cookie_blobs = []
|
||||
cookie_blobs.extend(flow.request.headers.get_all("cookie") or [])
|
||||
cookie_blobs.extend(flow.response.headers.get_all("set-cookie") or [])
|
||||
for blob in cookie_blobs:
|
||||
low = blob.lower()
|
||||
for pref in _CMP_COOKIE_PREFIXES:
|
||||
if pref in low:
|
||||
state["has_cmp"] = True
|
||||
state["consented"] = True
|
||||
break
|
||||
_consent_log[key] = state
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def _consent_state_for(mac_hash: str, site: str) -> str:
|
||||
"""Classify the current consent state for the (peer, site) pair.
|
||||
|
||||
Evidence-only semantics (no interpretation beyond the observation) :
|
||||
post_consent — a CMP cookie has been seen here for this peer.
|
||||
pre_consent — the site runs a CMP (loader/cookie seen) but no
|
||||
consent cookie yet : a tracker firing now fires
|
||||
before consent.
|
||||
none_seen — no CMP detected at all ; we make no claim.
|
||||
"""
|
||||
st = _consent_log.get(_consent_key(mac_hash, site))
|
||||
if not st:
|
||||
return "none_seen"
|
||||
if st.get("consented"):
|
||||
return "post_consent"
|
||||
if st.get("has_cmp"):
|
||||
return "pre_consent"
|
||||
return "none_seen"
|
||||
|
||||
|
||||
# ─── JA4 lookup ───
|
||||
def _ja4_hash(flow) -> Optional[str]:
|
||||
"""Pull the JA4 fingerprint set by the ja4 addon, if present."""
|
||||
try:
|
||||
ja4 = (flow.metadata or {}).get("ja4")
|
||||
if ja4:
|
||||
return str(ja4)[:32]
|
||||
except Exception:
|
||||
pass
|
||||
return None
|
||||
|
||||
|
||||
# ─── main ───
|
||||
class SocialGraph:
|
||||
"""mitmproxy addon : record cookie-bearing edges per R2/R3 peer."""
|
||||
|
|
@ -285,20 +186,6 @@ class SocialGraph:
|
|||
if not src_site:
|
||||
return
|
||||
|
||||
# Phase 11.C (#508) — consent-state detection. Before recording
|
||||
# any edge, update our per-peer × per-site consent log : has a
|
||||
# consent-platform cookie (OneTrust / Didomi / Quantcast /
|
||||
# Sourcepoint) been observed for this peer on this site yet ?
|
||||
# The edge's consent_state is then :
|
||||
# post_consent — a consent cookie was already seen here
|
||||
# pre_consent — NOT yet seen, AND the site DOES run a consent
|
||||
# platform somewhere (so a tracker firing now is
|
||||
# firing before consent — RGPD art. 6.1.a + 7)
|
||||
# none_seen — no consent platform detected for this site at
|
||||
# all (we make no claim ; baseline)
|
||||
_update_consent_log(mac_hash, src_site, flow)
|
||||
consent_state = _consent_state_for(mac_hash, src_site)
|
||||
|
||||
# Set-Cookie headers : the 3rd-party server hands a new identifier.
|
||||
# The Set-Cookie domain may differ from flow.request.host (Set-Cookie
|
||||
# `Domain=` attribute) — when present, we trust that for the
|
||||
|
|
@ -324,7 +211,6 @@ class SocialGraph:
|
|||
tracker_domain=tracker_domain,
|
||||
cookie_id_hash_val=cid,
|
||||
ja4_hash=ja4,
|
||||
consent_state=consent_state,
|
||||
)
|
||||
|
||||
# Request-side Cookie headers (only meaningful when the
|
||||
|
|
@ -347,7 +233,6 @@ class SocialGraph:
|
|||
# called itself).
|
||||
return
|
||||
|
||||
ctx_consent = _consent_state_for(mac_hash, ctx_site)
|
||||
for hdr in cookie_hdrs[:5]:
|
||||
for name, value in _parse_cookie_header(hdr)[:50]:
|
||||
if _social.is_deny_listed(name):
|
||||
|
|
@ -359,7 +244,6 @@ class SocialGraph:
|
|||
tracker_domain=tracker_domain,
|
||||
cookie_id_hash_val=cid,
|
||||
ja4_hash=ja4,
|
||||
consent_state=ctx_consent,
|
||||
)
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -2085,30 +2085,6 @@ async def admin_social_aggregate(hours: int = 24) -> dict:
|
|||
return _s.aggregate(hours=hours)
|
||||
|
||||
|
||||
@router.get("/social/report/{token}.pdf")
|
||||
async def social_report_pdf(token: str) -> Response:
|
||||
"""Phase 11.C (#508) — bilingual FR/EN evidence PDF for a peer.
|
||||
|
||||
Same HMAC-token gate as /social/graph/{token}. Fact-only report :
|
||||
cross-site tracker reuse, trackers fired before consent (RGPD art.
|
||||
6.1.a + 7), extra-EU transfers (art. 44+).
|
||||
"""
|
||||
from . import social_report as _sr
|
||||
salt = _get_salt()
|
||||
ok, mac_hash = reports.verify_token(token, salt)
|
||||
if not ok:
|
||||
raise HTTPException(404, "report not found or expired")
|
||||
data = _sr.build_social_report(mac_hash, since_seconds=7 * 86400)
|
||||
data["generated_at"] = time.strftime("%Y-%m-%d %H:%M UTC", time.gmtime())
|
||||
pdf_bytes = _sr.render_social_pdf(data)
|
||||
fname = f"village3b-carto-{mac_hash[:8]}-{int(time.time())}.pdf"
|
||||
return Response(
|
||||
content=pdf_bytes,
|
||||
media_type="application/pdf",
|
||||
headers={"Content-Disposition": f'attachment; filename="{fname}"'},
|
||||
)
|
||||
|
||||
|
||||
# ───────────────── Phase 11.B — per-client view + favicon proxy (#507) ─────────────────
|
||||
|
||||
import json as _json_b
|
||||
|
|
@ -2376,9 +2352,235 @@ async def admin_override_level(mac_hash: str, request: Request) -> dict:
|
|||
"note": "nft sets not auto-updated; client must reload or operator manually adjusts nft"}
|
||||
|
||||
|
||||
# Phase 11.B+ (#513) — the inline kbin /admin/ HTML admin UI was removed.
|
||||
# The canonical operator dashboard is admin.gk2.secubox.in/toolbox/ (sub-tab
|
||||
# WebUI in www/toolbox/index.html). All /admin/* JSON API routes below stay.
|
||||
@router.get("/admin/", response_class=HTMLResponse)
|
||||
@router.get("/admin", response_class=HTMLResponse)
|
||||
async def admin_index() -> HTMLResponse:
|
||||
"""Operator admin webUI with client list + level switcher."""
|
||||
html = """<!DOCTYPE html><html lang=fr><head><meta charset=UTF-8>
|
||||
<meta name=viewport content="width=device-width,initial-scale=1">
|
||||
<title>🛡 Admin — Gondwana ToolBoX</title>
|
||||
<style>:root{--bg:#0a0a0f;--bg2:#0e0e15;--phos:#00dd44;--phos-hot:#00ff55;--dim:#006622;--text:#e8e6d9;--purple:#9e76ff;--amber:#ffb347;--red:#ff4466}
|
||||
*{box-sizing:border-box;margin:0;padding:0}
|
||||
body{font-family:Menlo,Consolas,monospace;background:var(--bg);color:var(--text);padding:1.2rem;max-width:1100px;margin:auto;line-height:1.55}
|
||||
h1{color:var(--phos-hot);text-shadow:0 0 6px var(--phos);font-size:1.6rem;margin-bottom:0.4rem;letter-spacing:0.05em}
|
||||
.sub{color:var(--dim);font-size:0.85rem;margin-bottom:1.2rem}
|
||||
table{width:100%;border-collapse:collapse;font-size:0.85rem;background:var(--bg2);border:1px solid var(--dim)}
|
||||
th,td{padding:0.5rem 0.6rem;text-align:left;border-bottom:1px solid var(--dim)}
|
||||
th{color:var(--phos-hot);text-shadow:0 0 3px var(--phos);background:rgba(0,221,68,0.08)}
|
||||
tr:hover{background:rgba(0,221,68,0.04)}
|
||||
.chip{display:inline-block;padding:0.15rem 0.5rem;border-radius:99px;font-size:0.72rem;font-weight:bold}
|
||||
.chip.r0{background:#222;color:#999}
|
||||
.chip.r1{background:rgba(0,221,68,0.2);color:var(--phos-hot)}
|
||||
.chip.r2{background:rgba(255,179,71,0.2);color:#ffd6a0}
|
||||
.chip.r3{background:rgba(158,118,255,0.2);color:#cbb6ff}
|
||||
.btn{background:var(--purple);color:#0a0a0f;padding:0.3rem 0.6rem;border:none;border-radius:3px;cursor:pointer;font-family:inherit;font-size:0.72rem;font-weight:bold;margin-right:0.2rem}
|
||||
.btn:hover{background:#b598ff}
|
||||
.btn.outline{background:transparent;color:var(--phos);border:1px solid var(--phos)}
|
||||
code{background:#222;padding:0.1rem 0.3rem;border-radius:2px;font-size:0.75rem}
|
||||
.cards{display:grid;grid-template-columns:repeat(auto-fit,minmax(180px,1fr));gap:0.8rem;margin-bottom:1.5rem}
|
||||
.card{background:var(--bg2);border:1px solid var(--dim);padding:0.8rem;border-radius:4px;text-align:center}
|
||||
.card .v{font-size:1.6rem;color:var(--phos-hot);font-weight:bold;display:block}
|
||||
.card .l{font-size:0.7rem;color:var(--dim)}
|
||||
.modal-bg{position:fixed;top:0;left:0;right:0;bottom:0;background:rgba(0,0,0,0.8);display:none;align-items:center;justify-content:center;z-index:100}
|
||||
.modal-bg.show{display:flex}
|
||||
.modal{background:var(--bg2);border:1px solid var(--phos);padding:1.5rem;border-radius:4px;max-width:400px;width:90%}
|
||||
.modal h2{color:var(--phos-hot);font-size:1rem;margin-bottom:0.8rem}
|
||||
.modal .lvl-row{display:grid;grid-template-columns:repeat(4,1fr);gap:0.4rem;margin-bottom:0.8rem}
|
||||
.modal .lvl-row button{padding:0.5rem;cursor:pointer;font-family:inherit;font-size:0.75rem;border-radius:3px}
|
||||
.modal .lvl-row .r0{background:#222;color:#999;border:1px solid #444}
|
||||
.modal .lvl-row .r1{background:rgba(0,221,68,0.15);color:var(--phos-hot);border:1px solid var(--phos)}
|
||||
.modal .lvl-row .r2{background:rgba(255,179,71,0.15);color:#ffd6a0;border:1px solid var(--amber)}
|
||||
.modal .lvl-row .r3{background:rgba(158,118,255,0.15);color:#cbb6ff;border:1px solid var(--purple)}
|
||||
.adm-tabs{display:flex;gap:2px;border-bottom:1px solid var(--dim);margin-bottom:0.8rem}
|
||||
.adm-tab{background:transparent;border:0;color:#888;padding:0.5rem 1rem;font-family:inherit;font-size:0.85rem;cursor:pointer;border-bottom:2px solid transparent;transition:all 0.15s}
|
||||
.adm-tab.active{color:var(--phos-hot);border-bottom-color:var(--phos);background:rgba(0,221,68,0.05)}
|
||||
.adm-tab:hover{color:var(--text)}
|
||||
.adm-content{display:none}
|
||||
.adm-content.active{display:block}
|
||||
button.del-pattern{background:var(--red);color:#fff;border:0;padding:0.2rem 0.5rem;border-radius:3px;cursor:pointer;font-weight:bold;font-size:0.75rem}
|
||||
</style></head><body>
|
||||
<h1>🛡 Admin — Gondwana ToolBoX</h1>
|
||||
<p class=sub>// Console opérateur · client management · level override · mitm filtering</p>
|
||||
|
||||
{# Phase 6.I : tabbed sections #}
|
||||
<div class=adm-tabs>
|
||||
<button class="adm-tab active" data-tab=clients>👥 Clients</button>
|
||||
<button class="adm-tab" data-tab=filter>🛡 Mitm filtering</button>
|
||||
</div>
|
||||
|
||||
<div class="adm-content active" data-content=clients>
|
||||
<div id=cards class=cards></div>
|
||||
|
||||
<table id=clients-table>
|
||||
<thead><tr>
|
||||
<th>Status</th><th>Device</th><th>Hash</th><th>IP</th>
|
||||
<th>Level</th><th>Risk</th><th>Last seen</th><th>Actions</th>
|
||||
</tr></thead>
|
||||
<tbody id=clients-tbody><tr><td colspan=8>chargement…</td></tr></tbody>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
<div class="adm-content" data-content=filter>
|
||||
<h2 style="color:var(--purple);font-size:1.1rem;margin-bottom:0.4rem">🛡 Mitm bypass whitelist</h2>
|
||||
<p style="font-size:0.78rem;color:var(--dim);margin-bottom:0.8rem">
|
||||
Hosts/domains qui ne sont JAMAIS déchiffrés par mitm (TLS passthrough).
|
||||
Pour apps cert-pinned ou E2E. Redémarre <code>secubox-toolbox-mitm-wg</code>
|
||||
après modif.
|
||||
</p>
|
||||
|
||||
<form id=add-pattern-form style="display:flex;gap:0.5rem;margin-bottom:0.8rem">
|
||||
<input type=text name=entry placeholder="ex: (.+\\.)?example\\.com" required
|
||||
style="flex:1;background:#111;color:var(--text);border:1px solid #2a2a3f;padding:0.5rem;border-radius:3px;font-family:monospace;font-size:0.8rem">
|
||||
<button type=submit class=btn style="background:var(--phos);padding:0.5rem 1rem">➕ Ajouter</button>
|
||||
</form>
|
||||
|
||||
<table id=filter-table style="font-size:0.78rem">
|
||||
<thead><tr><th>Pattern (regex)</th><th width=60>×</th></tr></thead>
|
||||
<tbody id=filter-tbody><tr><td colspan=2>chargement…</td></tr></tbody>
|
||||
</table>
|
||||
|
||||
<p style="font-size:0.72rem;color:var(--dim);margin-top:0.6rem;border-left:2px solid var(--amber);padding-left:0.6rem">
|
||||
📁 <code>/var/lib/secubox/toolbox/mitm-bypass.conf</code> · Source de vérité
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div id=modal class=modal-bg>
|
||||
<div class=modal>
|
||||
<h2>🔀 Change level — <code id=modal-mh></code></h2>
|
||||
<p style="font-size:0.78rem;color:var(--dim);margin-bottom:0.5rem">⚠ Admin override : updates store only. Client must reload to sync nft.</p>
|
||||
<div class=lvl-row>
|
||||
<button class=r0 onclick="setLevel('r0')">🌐 R0</button>
|
||||
<button class=r1 onclick="setLevel('r1')">🛡 R1</button>
|
||||
<button class=r2 onclick="setLevel('r2')">🔍 R2</button>
|
||||
<button class=r3 onclick="setLevel('r3')">🌐 R3</button>
|
||||
</div>
|
||||
<button onclick="closeModal()" class=btn style="background:transparent;color:var(--dim);border:1px solid var(--dim)">Annuler</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
var selectedMh = null;
|
||||
function openModal(mh) {
|
||||
selectedMh = mh;
|
||||
document.getElementById('modal-mh').textContent = mh;
|
||||
document.getElementById('modal').classList.add('show');
|
||||
}
|
||||
function closeModal() {
|
||||
document.getElementById('modal').classList.remove('show');
|
||||
}
|
||||
async function setLevel(lvl) {
|
||||
if (!selectedMh) return;
|
||||
var fd = new FormData();
|
||||
fd.append('level', lvl);
|
||||
var r = await fetch('/admin/clients/'+selectedMh+'/level', {method:'POST', body:fd});
|
||||
if (r.ok) {
|
||||
closeModal();
|
||||
loadClients();
|
||||
} else {
|
||||
alert('Erreur: '+r.status);
|
||||
}
|
||||
}
|
||||
async function loadClients() {
|
||||
var r = await fetch('/admin/clients/rich');
|
||||
if (!r.ok) {
|
||||
document.getElementById('clients-tbody').innerHTML = '<tr><td colspan=8>Erreur '+r.status+'</td></tr>';
|
||||
return;
|
||||
}
|
||||
var data = await r.json();
|
||||
var tbody = document.getElementById('clients-tbody');
|
||||
tbody.innerHTML = '';
|
||||
data.clients.forEach(function(c){
|
||||
var lvlChip = '<span class="chip '+c.level+'">'+c.level_emoji+' '+c.level.toUpperCase()+'</span>';
|
||||
var dt = new Date(c.last_seen*1000).toISOString().substring(11,16);
|
||||
var tr = document.createElement('tr');
|
||||
tr.innerHTML = '<td>'+c.status_emoji+' '+c.status_label+'</td>'+
|
||||
'<td>'+c.device_emoji+'</td>'+
|
||||
'<td><code>'+c.mac_hash.substring(0,12)+'…</code></td>'+
|
||||
'<td>'+(c.ip||'?')+'</td>'+
|
||||
'<td>'+lvlChip+'</td>'+
|
||||
'<td>'+c.risk_emoji+' '+c.score+'</td>'+
|
||||
'<td>'+dt+'</td>'+
|
||||
'<td><button class=btn onclick="openModal(\\''+c.mac_hash+'\\')">🔀 Override</button>'+
|
||||
'<a href="/admin/clients/'+c.mac_hash+'/report" class="btn outline">📄 PDF</a></td>';
|
||||
tbody.appendChild(tr);
|
||||
});
|
||||
// KPI cards
|
||||
var counts = {r0:0,r1:0,r2:0,r3:0,actif:0,risk_low:0,risk_mh:0};
|
||||
data.clients.forEach(function(c){
|
||||
counts[c.level] = (counts[c.level]||0)+1;
|
||||
if (c.status_label==='actif') counts.actif++;
|
||||
if (c.score < 30) counts.risk_low++;
|
||||
if (c.score >= 30) counts.risk_mh++;
|
||||
});
|
||||
document.getElementById('cards').innerHTML =
|
||||
'<div class=card><span class=v>'+data.count+'</span><span class=l>Total clients</span></div>'+
|
||||
'<div class=card><span class=v>'+counts.actif+'</span><span class=l>🟢 Actifs (5min)</span></div>'+
|
||||
'<div class=card><span class=v>'+counts.r1+'</span><span class=l>🛡 R1</span></div>'+
|
||||
'<div class=card><span class=v>'+counts.r2+'</span><span class=l>🔍 R2</span></div>'+
|
||||
'<div class=card><span class=v>'+counts.r3+'</span><span class=l>🌐 R3 WG</span></div>'+
|
||||
'<div class=card><span class=v>'+counts.risk_low+'</span><span class=l>🟢 Risque LOW</span></div>';
|
||||
}
|
||||
loadClients();
|
||||
setInterval(loadClients, 15000);
|
||||
|
||||
// ── Tabs (Phase 6.I) ──
|
||||
document.querySelectorAll('.adm-tab').forEach(function(t){
|
||||
t.addEventListener('click', function(){
|
||||
var tn = this.dataset.tab;
|
||||
document.querySelectorAll('.adm-tab').forEach(function(x){x.classList.remove('active');});
|
||||
this.classList.add('active');
|
||||
document.querySelectorAll('.adm-content').forEach(function(x){x.classList.remove('active');});
|
||||
document.querySelector('[data-content='+tn+']').classList.add('active');
|
||||
if (tn === 'filter') loadFilter();
|
||||
});
|
||||
});
|
||||
|
||||
// ── Filter Control (integrated tab) ──
|
||||
async function loadFilter(){
|
||||
var r = await fetch('/admin/filter-control/list');
|
||||
var tb = document.getElementById('filter-tbody');
|
||||
if (!r.ok) { tb.innerHTML = '<tr><td colspan=2>Erreur '+r.status+'</td></tr>'; return; }
|
||||
var d = await r.json();
|
||||
if (!d.patterns || !d.patterns.length) {
|
||||
tb.innerHTML = '<tr><td colspan=2 style="color:#666;text-align:center">Aucune entrée</td></tr>';
|
||||
return;
|
||||
}
|
||||
tb.innerHTML = '';
|
||||
d.patterns.forEach(function(p){
|
||||
var tr = document.createElement('tr');
|
||||
var td1 = document.createElement('td');
|
||||
var code = document.createElement('code');
|
||||
code.textContent = p;
|
||||
td1.appendChild(code);
|
||||
var td2 = document.createElement('td');
|
||||
var btn = document.createElement('button');
|
||||
btn.className = 'del-pattern';
|
||||
btn.textContent = '×';
|
||||
btn.onclick = function(){ delPattern(p); };
|
||||
td2.appendChild(btn);
|
||||
tr.appendChild(td1); tr.appendChild(td2);
|
||||
tb.appendChild(tr);
|
||||
});
|
||||
}
|
||||
async function delPattern(p){
|
||||
var fd = new FormData(); fd.append('entry', p);
|
||||
var r = await fetch('/admin/filter-control/remove', {method:'POST', body:fd});
|
||||
if (r.ok || r.status === 303) loadFilter();
|
||||
}
|
||||
document.getElementById('add-pattern-form').addEventListener('submit', async function(ev){
|
||||
ev.preventDefault();
|
||||
var entry = ev.target.entry.value.trim();
|
||||
if (!entry) return;
|
||||
var fd = new FormData(); fd.append('entry', entry);
|
||||
var r = await fetch('/admin/filter-control/add', {method:'POST', body:fd});
|
||||
if (r.ok || r.status === 303) {
|
||||
ev.target.entry.value = '';
|
||||
loadFilter();
|
||||
}
|
||||
});
|
||||
</script>
|
||||
</body></html>"""
|
||||
return HTMLResponse(html, headers={"Cache-Control": "no-store"})
|
||||
|
||||
|
||||
@router.get("/admin/filter-control/list")
|
||||
|
|
|
|||
|
|
@ -1,239 +0,0 @@
|
|||
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
|
||||
# Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
|
||||
|
||||
"""
|
||||
SecuBox-Deb :: ToolBoX Social Mapping — bilingual FR/EN PDF report
|
||||
|
||||
Phase 11.C (#508, parent #502) — evidence-only legal report consumed by
|
||||
the per-client view's "Rapport PDF" card. Renders via fpdf2 (same
|
||||
engine as reports.py) so no new dependency.
|
||||
|
||||
The report is FACT-ONLY : it documents what we observed (cross-site
|
||||
cookie reuse, trackers fired before consent, extra-EU transfers) with
|
||||
the relevant RGPD article references. It makes NO interpretation about
|
||||
SCC presence/absence or ToS contradiction.
|
||||
|
||||
FR is the source-of-truth ; EN is a secondary summary line per field.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import time
|
||||
from typing import Dict, List
|
||||
|
||||
from . import social as _social
|
||||
|
||||
log = logging.getLogger("secubox.toolbox.social.report")
|
||||
|
||||
|
||||
# Bilingual label pairs (FR primary, EN secondary).
|
||||
_L = {
|
||||
"title": ("Cartographie sociale — Rapport d'évidence", "Social mapping — Evidence report"),
|
||||
"subtitle": ("Cabine numérique VILLAGE3B · Analyseur R3", "VILLAGE3B digital booth · R3 analyzer"),
|
||||
"anon_id": ("Identifiant anonyme", "Anonymous identifier"),
|
||||
"hash": ("Hash session (sel rotatif 24h)", "Session hash (24h rotating salt)"),
|
||||
"window": ("Fenêtre d'analyse", "Analysis window"),
|
||||
"generated": ("Généré le", "Generated"),
|
||||
"summary": ("Synthèse", "Summary"),
|
||||
"n_trackers": ("Traqueurs distincts", "Distinct trackers"),
|
||||
"n_sites": ("Sites visités", "Sites visited"),
|
||||
"n_preconsent": ("Traqueurs avant consentement", "Trackers before consent"),
|
||||
"n_extraeu": ("Transferts hors UE/EEE", "Extra-EU/EEA transfers"),
|
||||
"ev_preconsent": ("Évidence : traqueurs déclenchés AVANT consentement",
|
||||
"Evidence: trackers fired BEFORE consent"),
|
||||
"ev_preconsent_basis": ("Base légale : RGPD art. 6.1.a (consentement) + art. 7 (preuve)",
|
||||
"Legal basis: GDPR art. 6.1.a (consent) + art. 7 (proof)"),
|
||||
"ev_extraeu": ("Évidence : transferts vers des pays hors UE/EEE",
|
||||
"Evidence: transfers to non-EU/EEA countries"),
|
||||
"ev_extraeu_basis": ("Base légale : RGPD art. 44+ (transferts internationaux). "
|
||||
"Nous rapportons le fait observé, pas l'absence de garanties (SCC).",
|
||||
"Legal basis: GDPR art. 44+ (international transfers). We report the "
|
||||
"observed fact, not the absence of safeguards (SCC)."),
|
||||
"col_tracker": ("Traqueur", "Tracker"),
|
||||
"col_sites": ("Sites", "Sites"),
|
||||
"col_hits": ("Occurrences", "Hits"),
|
||||
"col_country": ("Pays", "Country"),
|
||||
"col_asn": ("Hébergeur (ASN)", "Host (ASN)"),
|
||||
"none_pre": ("Aucun traqueur déclenché avant consentement détecté.",
|
||||
"No tracker fired before consent detected."),
|
||||
"none_eu": ("Aucun transfert hors UE/EEE détecté.",
|
||||
"No extra-EU/EEA transfer detected."),
|
||||
"footer": ("Rapport factuel — données calculées localement, aucune donnée externe. "
|
||||
"Anonyme : aucune valeur de cookie brute conservée. Droit à l'effacement : RGPD art. 17.",
|
||||
"Factual report — computed locally, no external data. Anonymous: no raw cookie "
|
||||
"value stored. Right to erasure: GDPR art. 17."),
|
||||
"ca_disclaimer": ("Évidence d'observation réseau via cabine R3 consentie. "
|
||||
"Ne constitue pas un avis juridique.",
|
||||
"Network observation evidence via consented R3 booth. "
|
||||
"Not legal advice."),
|
||||
}
|
||||
|
||||
|
||||
def _bi(key: str) -> str:
|
||||
fr, en = _L.get(key, (key, key))
|
||||
return fr
|
||||
|
||||
|
||||
def build_social_report(mac_hash: str, since_seconds: int = 7 * 86400) -> Dict:
|
||||
"""Assemble the data structure the PDF renderer consumes."""
|
||||
graph = _social.fetch_graph(mac_hash, since_seconds=since_seconds)
|
||||
evidence = _social.evidence(mac_hash, since_seconds=since_seconds)
|
||||
stats = graph.get("stats", {})
|
||||
return {
|
||||
"mac_hash": mac_hash,
|
||||
"window_days": max(1, since_seconds // 86400),
|
||||
"generated_at": None, # stamped by the API layer (Date.* unavailable here is fine; time.time ok)
|
||||
"total_trackers": stats.get("total_trackers", 0),
|
||||
"total_sites": stats.get("total_sites", 0),
|
||||
"pre_consent": evidence.get("pre_consent", []),
|
||||
"extra_eu": evidence.get("extra_eu", []),
|
||||
}
|
||||
|
||||
|
||||
def render_social_pdf(report: Dict) -> bytes:
|
||||
"""Render the bilingual evidence report as PDF (fpdf2)."""
|
||||
try:
|
||||
from fpdf import FPDF
|
||||
except ImportError:
|
||||
return _text_fallback(report).encode()
|
||||
|
||||
# Reuse the font setup from reports.py for emoji/Unicode support.
|
||||
try:
|
||||
from .reports import _setup_fonts # type: ignore
|
||||
except Exception:
|
||||
_setup_fonts = None
|
||||
|
||||
pdf = FPDF(orientation="P", unit="mm", format="A4")
|
||||
pdf.add_page()
|
||||
pdf.set_auto_page_break(auto=True, margin=15)
|
||||
family = _setup_fonts(pdf) if _setup_fonts else "helvetica"
|
||||
|
||||
def _mc(h, text):
|
||||
# Robust multi_cell : reset X to the left margin and use the
|
||||
# effective page width so fpdf2 never hits "not enough
|
||||
# horizontal space" when the cursor has drifted to the edge.
|
||||
pdf.set_x(pdf.l_margin)
|
||||
pdf.multi_cell(pdf.epw, h, text)
|
||||
|
||||
def bi(key, size=10, style="", r=0, g=0, b=0, gap=1.0):
|
||||
fr, en = _L.get(key, (key, key))
|
||||
pdf.set_text_color(r, g, b)
|
||||
pdf.set_font(family, "B" if "B" in style else "", size)
|
||||
_mc(size * 0.5, fr)
|
||||
pdf.set_text_color(120, 120, 120)
|
||||
pdf.set_font(family, "", max(7, size - 2))
|
||||
_mc((size - 2) * 0.5, en)
|
||||
pdf.ln(gap)
|
||||
|
||||
# ── Cover ──
|
||||
pdf.set_font(family, "B", 20)
|
||||
pdf.set_text_color(10, 90, 64)
|
||||
pdf.cell(0, 12, "🕸️ VILLAGE3B", ln=True, align="C")
|
||||
bi("title", 13, "B", 110, 64, 201, gap=0.5)
|
||||
bi("subtitle", 9, "", 110, 110, 110, gap=3)
|
||||
|
||||
# ── Anonymous id ──
|
||||
bi("anon_id", 12, "B", 0, 90, 64, gap=0.5)
|
||||
pdf.set_font(family, "", 9)
|
||||
pdf.set_text_color(0)
|
||||
pdf.cell(0, 5, f" {_bi('hash')}: {report.get('mac_hash', '?')[:16]}…", ln=True)
|
||||
pdf.cell(0, 5, f" {_bi('window')}: {report.get('window_days', 7)} j / days", ln=True)
|
||||
pdf.cell(0, 5, f" {_bi('generated')}: {report.get('generated_at', '?')}", ln=True)
|
||||
pdf.ln(3)
|
||||
|
||||
# ── Summary tiles ──
|
||||
bi("summary", 12, "B", 0, 90, 64, gap=0.5)
|
||||
pdf.set_font(family, "", 10)
|
||||
pdf.set_text_color(0)
|
||||
pdf.cell(0, 6, f" {_bi('n_trackers')}: {report.get('total_trackers', 0)}", ln=True)
|
||||
pdf.cell(0, 6, f" {_bi('n_sites')}: {report.get('total_sites', 0)}", ln=True)
|
||||
pdf.cell(0, 6, f" {_bi('n_preconsent')}: {len(report.get('pre_consent', []))}", ln=True)
|
||||
pdf.cell(0, 6, f" {_bi('n_extraeu')}: {len(report.get('extra_eu', []))}", ln=True)
|
||||
pdf.ln(3)
|
||||
|
||||
# ── Evidence : pre-consent ──
|
||||
bi("ev_preconsent", 12, "B", 230, 57, 70, gap=0.3)
|
||||
bi("ev_preconsent_basis", 8, "", 120, 120, 120, gap=1)
|
||||
pre = report.get("pre_consent", [])
|
||||
if pre:
|
||||
_table(pdf, family, pre, [
|
||||
(_bi("col_tracker"), "tracker_domain", 70),
|
||||
(_bi("col_sites"), lambda r: str(len(r.get("sites", []))), 25),
|
||||
("pre", "pre_consent_hits", 25),
|
||||
(_bi("col_country"), "country_iso", 25),
|
||||
(_bi("col_asn"), "asn_org", 45),
|
||||
])
|
||||
else:
|
||||
_note(pdf, family, _bi("none_pre"))
|
||||
pdf.ln(2)
|
||||
|
||||
# ── Evidence : extra-EU ──
|
||||
bi("ev_extraeu", 12, "B", 110, 64, 201, gap=0.3)
|
||||
bi("ev_extraeu_basis", 8, "", 120, 120, 120, gap=1)
|
||||
eu = report.get("extra_eu", [])
|
||||
if eu:
|
||||
_table(pdf, family, eu, [
|
||||
(_bi("col_tracker"), "tracker_domain", 70),
|
||||
(_bi("col_sites"), lambda r: str(len(r.get("sites", []))), 25),
|
||||
(_bi("col_hits"), "hits", 25),
|
||||
(_bi("col_country"), "country_iso", 25),
|
||||
(_bi("col_asn"), "asn_org", 45),
|
||||
])
|
||||
else:
|
||||
_note(pdf, family, _bi("none_eu"))
|
||||
pdf.ln(4)
|
||||
|
||||
# ── Footer ──
|
||||
pdf.set_font(family, "", 7)
|
||||
pdf.set_text_color(120, 120, 120)
|
||||
fr, en = _L["footer"]
|
||||
_mc(3.2, fr)
|
||||
_mc(3.0, en)
|
||||
pdf.ln(1)
|
||||
fr, en = _L["ca_disclaimer"]
|
||||
_mc(3.2, fr + " / " + en)
|
||||
|
||||
out = pdf.output(dest="S")
|
||||
return bytes(out) if isinstance(out, (bytes, bytearray)) else out.encode("latin-1")
|
||||
|
||||
|
||||
def _table(pdf, family, rows: List[Dict], cols) -> None:
|
||||
pdf.set_font(family, "B", 8)
|
||||
pdf.set_text_color(0, 90, 64)
|
||||
for label, _key, w in cols:
|
||||
pdf.cell(w, 5, str(label)[:24], border="B")
|
||||
pdf.ln()
|
||||
pdf.set_font(family, "", 8)
|
||||
pdf.set_text_color(0)
|
||||
for r in rows[:40]:
|
||||
for label, key, w in cols:
|
||||
if callable(key):
|
||||
val = key(r)
|
||||
else:
|
||||
val = r.get(key, "")
|
||||
pdf.cell(w, 4.5, str(val if val is not None else "—")[:int(w / 1.7)])
|
||||
pdf.ln()
|
||||
|
||||
|
||||
def _note(pdf, family, text: str) -> None:
|
||||
pdf.set_font(family, "I", 9)
|
||||
pdf.set_text_color(0, 120, 80)
|
||||
pdf.cell(0, 6, " " + text, ln=True)
|
||||
|
||||
|
||||
def _text_fallback(report: Dict) -> str:
|
||||
lines = [
|
||||
"VILLAGE3B — Social mapping evidence report",
|
||||
f"hash: {report.get('mac_hash', '?')[:16]}",
|
||||
f"trackers: {report.get('total_trackers', 0)} sites: {report.get('total_sites', 0)}",
|
||||
f"pre-consent: {len(report.get('pre_consent', []))} extra-EU: {len(report.get('extra_eu', []))}",
|
||||
"",
|
||||
"Pre-consent trackers (GDPR art. 6.1.a + 7):",
|
||||
]
|
||||
for e in report.get("pre_consent", [])[:40]:
|
||||
lines.append(f" - {e.get('tracker_domain')} ({e.get('pre_consent_hits')} hits, {e.get('country_iso')})")
|
||||
lines.append("")
|
||||
lines.append("Extra-EU transfers (GDPR art. 44+):")
|
||||
for e in report.get("extra_eu", [])[:40]:
|
||||
lines.append(f" - {e.get('tracker_domain')} ({e.get('country_iso')}, {e.get('asn_org')})")
|
||||
return "\n".join(lines)
|
||||
|
|
@ -2,7 +2,7 @@
|
|||
<!--
|
||||
SPDX-License-Identifier: LicenseRef-CMSD-1.0
|
||||
Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
|
||||
SecuBox-Deb :: ToolBoX admin dashboard (P31 light skin) — sub-tab nav (#513)
|
||||
SecuBox-Deb :: ToolBoX admin dashboard (P31 light skin)
|
||||
-->
|
||||
<html lang="fr">
|
||||
<head>
|
||||
|
|
@ -14,22 +14,14 @@
|
|||
<link rel="stylesheet" href="/shared/crt-light.css">
|
||||
<link rel="stylesheet" href="/shared/sidebar-light.css">
|
||||
<style>
|
||||
:root{--p31-peak:#00dd44;--p31-hot:#00ff55;--p31-mid:#009933;--p31-dim:#006622;--p31-decay:#ffb347;--tube-light:#e8f5e9;--tube-pale:#c8e6c9;--tube-soft:#a5d6a7;--bg-card:var(--tube-pale);--border:var(--tube-soft);--text-dim:var(--p31-dim);--red:#ff4466;--purple:#9e76ff;--bloom-text:0 0 2px var(--p31-peak),0 0 6px var(--p31-peak);--bloom-soft:0 0 6px var(--p31-peak)}
|
||||
:root{--p31-peak:#00dd44;--p31-hot:#00ff55;--p31-mid:#009933;--p31-dim:#006622;--p31-decay:#ffb347;--tube-light:#e8f5e9;--tube-pale:#c8e6c9;--tube-soft:#a5d6a7;--bg-card:var(--tube-pale);--border:var(--tube-soft);--text-dim:var(--p31-dim);--red:#ff4466;--bloom-text:0 0 2px var(--p31-peak),0 0 6px var(--p31-peak);--bloom-soft:0 0 6px var(--p31-peak)}
|
||||
*{box-sizing:border-box;margin:0;padding:0}
|
||||
body{font-family:'Courier Prime',monospace;background:var(--tube-light);color:var(--p31-mid);display:flex;min-height:100vh}
|
||||
.sidebar{width:220px;position:fixed;height:100vh;overflow-y:auto}
|
||||
.main{flex:1;margin-left:220px;padding:1.5rem}
|
||||
.header{display:flex;justify-content:space-between;align-items:center;padding:1rem 1.5rem;border:1px solid var(--border);background:var(--bg-card);margin-bottom:1rem}
|
||||
.header{display:flex;justify-content:space-between;align-items:center;padding:1rem 1.5rem;border:1px solid var(--border);background:var(--bg-card);margin-bottom:1.5rem}
|
||||
.header h1{font-size:1.4rem;color:var(--p31-hot);text-shadow:var(--bloom-text)}
|
||||
.badge{font-size:0.85rem;color:var(--p31-dim);padding:0.2rem 0.6rem;border:1px solid var(--border);border-radius:3px}
|
||||
/* Sub-tab nav (#513) */
|
||||
.tabs{display:flex;gap:0.3rem;margin-bottom:1.2rem;border-bottom:1px solid var(--border);flex-wrap:wrap}
|
||||
.tab{font-family:inherit;font-size:0.9rem;padding:0.5rem 1rem;background:transparent;color:var(--p31-dim);border:1px solid var(--border);border-bottom:none;cursor:pointer;border-radius:3px 3px 0 0}
|
||||
.tab:hover{color:var(--p31-peak);background:rgba(0,221,68,0.05)}
|
||||
.tab.active{color:var(--p31-hot);text-shadow:var(--bloom-text);background:var(--bg-card);border-color:var(--p31-mid);font-weight:bold}
|
||||
.panel{display:none}
|
||||
.panel.active{display:block;animation:fade .15s ease-in}
|
||||
@keyframes fade{from{opacity:0}to{opacity:1}}
|
||||
.grid{display:grid;grid-template-columns:repeat(auto-fit,minmax(380px,1fr));gap:1rem}
|
||||
.card{background:var(--bg-card);border:1px solid var(--border);padding:1rem;box-shadow:var(--bloom-soft)}
|
||||
.card h2{font-size:1rem;color:var(--p31-hot);text-shadow:var(--bloom-text);margin-bottom:0.6rem;padding-bottom:0.4rem;border-bottom:1px solid var(--border)}
|
||||
|
|
@ -40,23 +32,13 @@
|
|||
th,td{text-align:left;padding:0.4rem 0.5rem;border-bottom:1px solid var(--border)}
|
||||
th{color:var(--p31-hot);text-shadow:var(--bloom-text)}
|
||||
.empty{color:var(--text-dim);font-style:italic;padding:1rem;text-align:center}
|
||||
.toolbar{display:flex;gap:0.5rem;margin-bottom:1rem;flex-wrap:wrap}
|
||||
.toolbar{display:flex;gap:0.5rem;margin-bottom:1rem}
|
||||
button{font-family:inherit;font-size:0.9rem;padding:0.4rem 0.9rem;background:transparent;color:var(--p31-peak);border:1px solid var(--p31-mid);cursor:pointer;text-shadow:var(--bloom-text)}
|
||||
button:hover{background:rgba(0,221,68,0.1)}
|
||||
.r2-banner{background:rgba(255,179,71,0.08);border-left:2px solid var(--p31-decay);padding:0.6rem 0.8rem;margin-bottom:1rem;font-size:0.85rem;color:#996b1a}
|
||||
.state-validated{color:var(--p31-peak);text-shadow:var(--bloom-text)}
|
||||
.state-throttle{color:var(--p31-decay)}
|
||||
.state-quarantine{color:var(--red)}
|
||||
/* level chips + switcher (folded from kbin /admin/) */
|
||||
.chip{display:inline-block;padding:0.1rem 0.45rem;border-radius:99px;font-size:0.7rem;font-weight:bold}
|
||||
.chip.r0{background:#cfd8cf;color:#555}
|
||||
.chip.r1{background:rgba(0,221,68,0.25);color:var(--p31-mid)}
|
||||
.chip.r2{background:rgba(255,179,71,0.3);color:#996b1a}
|
||||
.chip.r3{background:rgba(158,118,255,0.25);color:#6a4fd0}
|
||||
.lvlbtn{font-size:0.68rem;padding:0.12rem 0.4rem;margin-right:0.15rem;border:1px solid var(--p31-dim)}
|
||||
code{background:rgba(0,0,0,0.05);padding:0.1rem 0.3rem;border-radius:2px;font-size:0.78rem}
|
||||
a.link{color:var(--p31-mid);text-decoration:underline}
|
||||
.filterlist li{font-size:0.82rem;padding:0.2rem 0;border-bottom:1px solid var(--border);list-style:none;display:flex;justify-content:space-between;align-items:center}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
|
|
@ -71,107 +53,39 @@
|
|||
<strong>R2 actif :</strong> TLS-break opt-in. Voir #475 / #474 pour la doctrine.
|
||||
</div>
|
||||
|
||||
<!-- Sub-tab navigation (#513) -->
|
||||
<nav class="tabs" id="tabs">
|
||||
<button class="tab active" data-tab="overview" onclick="switchTab('overview')">📊 Vue d'ensemble</button>
|
||||
<button class="tab" data-tab="clients" onclick="switchTab('clients')">👥 Clients</button>
|
||||
<button class="tab" data-tab="filtres" onclick="switchTab('filtres')">🚦 Filtres MITM</button>
|
||||
<button class="tab" data-tab="social" onclick="switchTab('social')">🕸️ Cartographie sociale</button>
|
||||
<button class="tab" data-tab="config" onclick="switchTab('config')">⚙ Config</button>
|
||||
</nav>
|
||||
<div class="toolbar">
|
||||
<button onclick="refreshAll()">🔁 Refresh</button>
|
||||
<button onclick="window.open('/api/v1/toolbox/admin/config', '_blank')">⚙ Config TOML</button>
|
||||
</div>
|
||||
|
||||
<!-- Overview -->
|
||||
<section class="panel active" id="panel-overview">
|
||||
<div class="toolbar">
|
||||
<button onclick="refreshAll()">🔁 Refresh</button>
|
||||
</div>
|
||||
<div class="grid">
|
||||
<div class="card">
|
||||
<h2>📊 Live metrics (24h)</h2>
|
||||
<div class="kv" id="metrics"><span class="k">loading…</span><span class="v"></span></div>
|
||||
</div>
|
||||
<div class="card">
|
||||
<h2>♥ Liveness</h2>
|
||||
<div class="kv" id="health"><span class="k">loading…</span><span class="v"></span></div>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<!-- Clients (folds in the kbin /admin/ level switcher) -->
|
||||
<section class="panel" id="panel-clients">
|
||||
<div class="toolbar">
|
||||
<button onclick="loadClients()">🔁 Refresh</button>
|
||||
</div>
|
||||
<div class="card" style="margin-bottom:1rem">
|
||||
<h2>👥 Clients actifs</h2>
|
||||
<div id="clients"><div class="empty">loading…</div></div>
|
||||
</div>
|
||||
<div class="card" style="display:none" id="client-detail-card">
|
||||
<h2 id="detail-title">🔍 Détails client</h2>
|
||||
<div id="client-detail"></div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<!-- Filtres MITM (bypass list) -->
|
||||
<section class="panel" id="panel-filtres">
|
||||
<div class="toolbar">
|
||||
<button onclick="loadFilters()">🔁 Refresh</button>
|
||||
</div>
|
||||
<div class="grid">
|
||||
<div class="card">
|
||||
<h2>🚦 Hosts bypassés (mitm ignore_hosts)</h2>
|
||||
<p style="font-size:0.82rem;color:var(--p31-dim);margin-bottom:0.6rem">
|
||||
Hosts exclus de l'inspection TLS (cert-pinning détecté ou whitelist statique).
|
||||
</p>
|
||||
<ul class="filterlist" id="filters"><li class="empty">loading…</li></ul>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<!-- Cartographie sociale (Phase 11 operator aggregate) -->
|
||||
<section class="panel" id="panel-social">
|
||||
<div class="toolbar">
|
||||
<button onclick="loadSocial()">🔁 Refresh</button>
|
||||
</div>
|
||||
<div class="grid">
|
||||
<div class="card">
|
||||
<h2>🕸️ Agrégat trackers (24h)</h2>
|
||||
<div class="kv" id="social-kpi"><span class="k">loading…</span><span class="v"></span></div>
|
||||
</div>
|
||||
<div class="card" style="grid-column:1/-1">
|
||||
<h2>🎯 Top tracker domains</h2>
|
||||
<div id="social-trackers"><div class="empty">loading…</div></div>
|
||||
</div>
|
||||
<div class="card" style="grid-column:1/-1">
|
||||
<h2>👤 Clients anonymisés</h2>
|
||||
<div id="social-clients"><div class="empty">loading…</div></div>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<!-- Config -->
|
||||
<section class="panel" id="panel-config">
|
||||
<div class="toolbar">
|
||||
<button onclick="window.open('/api/v1/toolbox/admin/config', '_blank')">⚙ Config TOML brute</button>
|
||||
<h2>📊 Live metrics (24h)</h2>
|
||||
<div class="kv" id="metrics"><span class="k">loading…</span><span class="v"></span></div>
|
||||
</div>
|
||||
<div class="card">
|
||||
<h2>⚙ Configuration</h2>
|
||||
<div class="kv" id="cfg-summary"><span class="k">loading…</span><span class="v"></span></div>
|
||||
</div>
|
||||
</section>
|
||||
<div class="card">
|
||||
<h2>♥ Liveness</h2>
|
||||
<div class="kv" id="health"><span class="k">loading…</span><span class="v"></span></div>
|
||||
</div>
|
||||
<div class="card" style="grid-column:1/-1">
|
||||
<h2>👥 Clients actifs</h2>
|
||||
<div id="clients"><div class="empty">loading…</div></div>
|
||||
</div>
|
||||
<div class="card" style="grid-column:1/-1;display:none" id="client-detail-card">
|
||||
<h2 id="detail-title">🔍 Détails client</h2>
|
||||
<div id="client-detail"></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script src="/shared/sidebar.js"></script>
|
||||
<script>
|
||||
const API = '/api/v1/toolbox';
|
||||
|
||||
function switchTab(name) {
|
||||
document.querySelectorAll('.tab').forEach(t => t.classList.toggle('active', t.dataset.tab === name));
|
||||
document.querySelectorAll('.panel').forEach(p => p.classList.toggle('active', p.id === 'panel-' + name));
|
||||
// Lazy-load the heavier tabs only when first opened.
|
||||
if (name === 'filtres') loadFilters();
|
||||
if (name === 'social') loadSocial();
|
||||
location.hash = name;
|
||||
}
|
||||
|
||||
async function J(path) {
|
||||
try {
|
||||
const r = await fetch(API + path, { credentials: 'same-origin' });
|
||||
|
|
@ -197,50 +111,24 @@ async function loadCfg() {
|
|||
document.getElementById('r2-banner').style.display = c.r2.enabled ? '' : 'none';
|
||||
}
|
||||
|
||||
function levelChip(lvl) {
|
||||
const l = (lvl || 'r1').toLowerCase();
|
||||
return `<span class="chip ${l}">${l.toUpperCase()}</span>`;
|
||||
}
|
||||
|
||||
function levelSwitcher(macHash, current) {
|
||||
const cur = (current || 'r1').toLowerCase();
|
||||
return ['r0','r1','r2','r3'].map(l =>
|
||||
l === cur ? '' :
|
||||
`<button class="lvlbtn" onclick="setLevel('${macHash}','${l}')">${l.toUpperCase()}</button>`
|
||||
).join('');
|
||||
}
|
||||
|
||||
async function setLevel(macHash, level) {
|
||||
try {
|
||||
const fd = new FormData(); fd.append('level', level);
|
||||
const r = await fetch(`${API}/admin/clients/${macHash}/level`, {
|
||||
method: 'POST', body: fd, credentials: 'same-origin'
|
||||
});
|
||||
if (!r.ok) throw new Error('HTTP ' + r.status);
|
||||
loadClients();
|
||||
} catch (e) { alert('Échec changement niveau : ' + e.message); }
|
||||
}
|
||||
|
||||
async function loadClients() {
|
||||
const d = await J('/admin/clients/rich');
|
||||
const d = await J('/admin/clients');
|
||||
const el = document.getElementById('clients');
|
||||
const rows = (d && d.clients) ? d.clients : (Array.isArray(d) ? d : null);
|
||||
if (!rows) { el.innerHTML = `<div class="empty">${(d && d.__error) || 'no data'}</div>`; return; }
|
||||
if (!rows.length) { el.innerHTML = '<div class="empty">no clients</div>'; return; }
|
||||
let html = '<table><thead><tr><th>MAC (hash)</th><th>IP</th><th>state</th><th>niveau</th><th>score</th><th>last</th><th>Actions</th></tr></thead><tbody>';
|
||||
for (const c of rows) {
|
||||
const ago = c.last_seen ? Math.round((Date.now()/1000 - c.last_seen) / 60) + 'm' : '—';
|
||||
if (d.__error) { el.innerHTML = `<div class="empty">${d.__error}</div>`; return; }
|
||||
if (!d.length) { el.innerHTML = '<div class="empty">no clients</div>'; return; }
|
||||
let html = '<table><thead><tr><th>MAC (hash)</th><th>IP</th><th>state</th><th>score</th><th>last</th><th>Actions</th></tr></thead><tbody>';
|
||||
for (const c of d) {
|
||||
const ago = Math.round((Date.now()/1000 - c.last_seen) / 60);
|
||||
html += `<tr>
|
||||
<td><code>${c.mac_hash}</code></td>
|
||||
<td>${c.ip || '—'}</td>
|
||||
<td><span class="state-${c.state}">${c.state || '—'}</span></td>
|
||||
<td>${levelChip(c.level)} ${levelSwitcher(c.mac_hash, c.level)}</td>
|
||||
<td>${c.score ?? '—'}</td>
|
||||
<td>${ago}</td>
|
||||
<td>${c.ip}</td>
|
||||
<td><span class="state-${c.state}">${c.state}</span></td>
|
||||
<td>${c.score}</td>
|
||||
<td>${ago}m</td>
|
||||
<td>
|
||||
<a class="link" href="${API}/admin/clients/${c.mac_hash}/report" target="_blank" style="font-size:0.8rem">PDF</a>
|
||||
<a href="${API}/admin/clients/${c.mac_hash}/report" target="_blank" style="color:var(--p31-peak);text-decoration:underline;font-size:0.8rem">PDF</a>
|
||||
·
|
||||
<a class="link" href="javascript:void(0)" onclick="loadClientDetail('${c.mac_hash}')" style="font-size:0.8rem">Events</a>
|
||||
<a href="javascript:void(0)" onclick="loadClientDetail('${c.mac_hash}')" style="color:var(--p31-peak);text-decoration:underline;font-size:0.8rem">Events</a>
|
||||
</td>
|
||||
</tr>`;
|
||||
}
|
||||
|
|
@ -274,7 +162,6 @@ async function loadClientDetail(macHash) {
|
|||
const div = document.getElementById('client-detail');
|
||||
title.textContent = `🔍 Détails client — ${macHash}`;
|
||||
card.style.display = '';
|
||||
card.scrollIntoView({behavior:'smooth', block:'nearest'});
|
||||
div.innerHTML = '<div class="empty">loading…</div>';
|
||||
const d = await J(`/admin/clients/${macHash}/events`);
|
||||
if (d.__error) { div.innerHTML = `<div class="empty">${d.__error}</div>`; return; }
|
||||
|
|
@ -288,7 +175,7 @@ async function loadClientDetail(macHash) {
|
|||
for (const e of d.recent) {
|
||||
const t = new Date(e.ts*1000).toLocaleTimeString();
|
||||
const detail = e.host || e.url || e.sni || e.kind || '?';
|
||||
recentHtml += `<tr><td>${t}</td><td><b>${e.source}</b></td><td><code>${detail}</code></td></tr>`;
|
||||
recentHtml += `<tr><td>${t}</td><td><b>${e.source}</b></td><td><code style="font-size:0.78rem">${detail}</code></td></tr>`;
|
||||
}
|
||||
recentHtml += '</tbody></table>';
|
||||
}
|
||||
|
|
@ -297,7 +184,7 @@ async function loadClientDetail(macHash) {
|
|||
<p style="margin-bottom:0.5rem;font-size:0.85rem;color:var(--p31-dim)">Récents :</p>
|
||||
${recentHtml}
|
||||
<p style="margin-top:0.8rem">
|
||||
<a class="link" href="${API}/admin/clients/${macHash}/report" target="_blank">⬇ Télécharger PDF complet</a>
|
||||
<a href="${API}/admin/clients/${macHash}/report" target="_blank" style="color:var(--p31-peak);text-decoration:underline">⬇ Télécharger PDF complet</a>
|
||||
</p>
|
||||
`;
|
||||
}
|
||||
|
|
@ -314,68 +201,11 @@ async function loadHealth() {
|
|||
document.getElementById('version-badge').textContent = `v${h.version}`;
|
||||
}
|
||||
|
||||
async function loadFilters() {
|
||||
const el = document.getElementById('filters');
|
||||
const d = await J('/admin/filter-control/list');
|
||||
const items = (d && d.hosts) ? d.hosts : (Array.isArray(d) ? d : null);
|
||||
if (!items) { el.innerHTML = `<li class="empty">${(d && d.__error) || 'no data'}</li>`; return; }
|
||||
if (!items.length) { el.innerHTML = '<li class="empty">aucun host bypassé</li>'; return; }
|
||||
el.innerHTML = items.map(h => {
|
||||
const host = typeof h === 'string' ? h : (h.host || h.pattern || JSON.stringify(h));
|
||||
const src = (typeof h === 'object' && h.source) ? h.source : '';
|
||||
return `<li><code>${host}</code>${src ? `<span style="color:var(--p31-dim);font-size:0.72rem">${src}</span>` : ''}</li>`;
|
||||
}).join('');
|
||||
}
|
||||
|
||||
async function loadSocial() {
|
||||
const agg = await J('/admin/social-aggregate?hours=24');
|
||||
const kpi = document.getElementById('social-kpi');
|
||||
const trk = document.getElementById('social-trackers');
|
||||
const cli = document.getElementById('social-clients');
|
||||
if (agg.__error) {
|
||||
kpi.innerHTML = `<span class="k">err</span><span class="v">${agg.__error}</span>`;
|
||||
trk.innerHTML = cli.innerHTML = '';
|
||||
return;
|
||||
}
|
||||
kpi.innerHTML = `
|
||||
<span class="k">Clients actifs</span> <span class="v">${agg.active_clients || 0}</span>
|
||||
<span class="k">Trackers vus</span> <span class="v">${agg.total_trackers_seen || 0}</span>
|
||||
<span class="k">Hors-UE (Phase C)</span> <span class="v">${agg.extra_eu_trackers || 0}</span>
|
||||
<span class="k">Fenêtre</span> <span class="v">${agg.window_hours || 24}h</span>
|
||||
`;
|
||||
const td = agg.by_tracker_domain || [];
|
||||
trk.innerHTML = td.length
|
||||
? '<table><thead><tr><th>Tracker domain</th><th>hits</th><th>clients</th></tr></thead><tbody>' +
|
||||
td.map(r => `<tr><td><code>${r.tracker_domain}</code></td><td>${r.hits}</td><td>${r.clients}</td></tr>`).join('') +
|
||||
'</tbody></table>'
|
||||
: '<div class="empty">aucun tracker dans la fenêtre</div>';
|
||||
const bc = agg.by_client || [];
|
||||
cli.innerHTML = bc.length
|
||||
? '<table><thead><tr><th>Client (hash)</th><th>sites</th><th>trackers</th><th>last</th></tr></thead><tbody>' +
|
||||
bc.map(r => {
|
||||
const ago = r.last_seen ? Math.round((Date.now()/1000 - r.last_seen)/60) + 'm' : '—';
|
||||
return `<tr><td><code>${(r.client_mac_hash||'').slice(0,16)}</code></td><td>${r.sites}</td><td>${r.trackers}</td><td>${ago}</td></tr>`;
|
||||
}).join('') +
|
||||
'</tbody></table>'
|
||||
: '<div class="empty">aucun client</div>';
|
||||
}
|
||||
|
||||
async function refreshAll() {
|
||||
await Promise.all([loadCfg(), loadClients(), loadHealth(), loadMetrics()]);
|
||||
}
|
||||
|
||||
// Deep-link : open the tab named in the URL hash on load.
|
||||
const initial = (location.hash || '').replace('#', '');
|
||||
if (['overview','clients','filtres','social','config'].includes(initial)) switchTab(initial);
|
||||
|
||||
refreshAll();
|
||||
setInterval(() => {
|
||||
// Only refresh the currently-visible tab's live data.
|
||||
const active = document.querySelector('.tab.active')?.dataset.tab;
|
||||
if (active === 'overview') { loadHealth(); loadMetrics(); }
|
||||
else if (active === 'clients') loadClients();
|
||||
else if (active === 'social') loadSocial();
|
||||
}, 10000);
|
||||
setInterval(refreshAll, 10000);
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
|
|
|
|||
|
|
@ -224,17 +224,6 @@ body {
|
|||
}
|
||||
.wipe-btn:hover { background: rgba(230, 57, 70, .1); }
|
||||
|
||||
/* Phase 11.C — PDF report download button */
|
||||
.pdf-btn {
|
||||
display: block; width: 100%; margin-top: 8px;
|
||||
text-align: center; text-decoration: none;
|
||||
background: transparent; color: var(--gold-hermetic);
|
||||
border: 1px solid var(--gold-hermetic); border-radius: 3px;
|
||||
padding: 10px;
|
||||
font-family: 'Cinzel', serif; letter-spacing: 0.06em;
|
||||
}
|
||||
.pdf-btn:hover { background: rgba(201, 168, 76, .12); }
|
||||
|
||||
/* Modal */
|
||||
dialog#wipe-modal {
|
||||
background: var(--cosmos-black);
|
||||
|
|
|
|||
|
|
@ -26,10 +26,6 @@ import geoip2.errors
|
|||
RULES_PATH = "/usr/share/secubox/waf/waf-rules.json"
|
||||
THREATS_LOG = "/var/log/secubox/waf-threats.log"
|
||||
STATS_CACHE = "/tmp/secubox/waf-stats.json"
|
||||
# Phase 7+ (#509) — disk-persisted counters + log byte position for
|
||||
# the double-buffered cache. Survives aggregator restart, populated
|
||||
# incrementally by the warm refresh loop.
|
||||
STATS_DISK_CACHE = "/var/lib/secubox/waf/stats-disk-cache.json"
|
||||
|
||||
# Runtime state
|
||||
_compiled_patterns: Dict[str, List[dict]] = {}
|
||||
|
|
@ -295,170 +291,70 @@ def _get_bans() -> List[dict]:
|
|||
return []
|
||||
|
||||
|
||||
def _load_stats_disk_cache() -> dict:
|
||||
"""Load the persisted counter state + last-read byte position.
|
||||
|
||||
Schema : {byte_position: int, counters: {...}, ip_countries: {...},
|
||||
today_iso: 'YYYY-MM-DD', threats_today: int,
|
||||
last_updated: int}
|
||||
|
||||
Counters are full-history accumulators ; `threats_today` is reset
|
||||
at the day rollover.
|
||||
"""
|
||||
p = Path(STATS_DISK_CACHE)
|
||||
if not p.exists():
|
||||
return {}
|
||||
try:
|
||||
return json.loads(p.read_text())
|
||||
except Exception:
|
||||
return {}
|
||||
|
||||
|
||||
def _save_stats_disk_cache(state: dict) -> None:
|
||||
try:
|
||||
p = Path(STATS_DISK_CACHE)
|
||||
p.parent.mkdir(parents=True, exist_ok=True)
|
||||
# Atomic write : tmp → rename so a half-written file never
|
||||
# corrupts the cache on the next load.
|
||||
tmp = p.with_suffix(".tmp")
|
||||
tmp.write_text(json.dumps(state))
|
||||
tmp.replace(p)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def _get_threat_stats() -> dict:
|
||||
"""Get threat statistics from log with GeoIP country lookup.
|
||||
|
||||
Phase 7+ (#509) — double-buffered cache : the first call after a
|
||||
cold start does ONE full-log pass and persists counters + the byte
|
||||
position to disk. Subsequent calls only read the new tail since
|
||||
the last position. Log rotation / truncation is detected via the
|
||||
file shrinking ; counters are reset cleanly.
|
||||
"""
|
||||
state = _load_stats_disk_cache()
|
||||
today = datetime.now().date().isoformat()
|
||||
|
||||
# Reset accumulators if the day has rolled over.
|
||||
if state.get("today_iso") != today:
|
||||
state["today_iso"] = today
|
||||
state["threats_today"] = 0
|
||||
|
||||
counters = state.get("counters", {})
|
||||
by_category = defaultdict(int, counters.get("by_category", {}))
|
||||
by_severity = defaultdict(int, counters.get("by_severity", {}))
|
||||
top_ips = defaultdict(int, counters.get("top_ips", {}))
|
||||
top_countries = defaultdict(int, counters.get("top_countries", {}))
|
||||
top_vhosts = defaultdict(int, counters.get("top_vhosts", {}))
|
||||
total_threats = counters.get("total_threats", 0)
|
||||
threats_today = state.get("threats_today", 0)
|
||||
ip_countries: Dict[str, str] = dict(state.get("ip_countries", {}))
|
||||
"""Get threat statistics from log with GeoIP country lookup."""
|
||||
stats = {
|
||||
"total_threats": 0,
|
||||
"threats_today": 0,
|
||||
"by_category": defaultdict(int),
|
||||
"by_severity": defaultdict(int),
|
||||
"top_ips": defaultdict(int),
|
||||
"top_countries": defaultdict(int),
|
||||
"top_vhosts": defaultdict(int),
|
||||
}
|
||||
ip_countries: Dict[str, str] = {} # IP → country mapping
|
||||
|
||||
log_path = Path(THREATS_LOG)
|
||||
if not log_path.exists():
|
||||
# Return whatever's cached.
|
||||
return _finalize_stats(
|
||||
total_threats, threats_today, by_category, by_severity,
|
||||
top_ips, top_countries, top_vhosts, ip_countries,
|
||||
)
|
||||
return stats
|
||||
|
||||
today = datetime.now().date().isoformat()
|
||||
geoip_reader = _get_geoip_reader()
|
||||
|
||||
try:
|
||||
size_now = log_path.stat().st_size
|
||||
byte_position = state.get("byte_position", 0)
|
||||
with open(log_path) as f:
|
||||
for line in f:
|
||||
try:
|
||||
entry = json.loads(line.strip())
|
||||
stats["total_threats"] += 1
|
||||
|
||||
# Log rotation / truncation : the file shrank since last read.
|
||||
# Drop accumulators ; we'll rebuild from the new (smaller) file.
|
||||
if size_now < byte_position:
|
||||
by_category.clear(); by_severity.clear(); top_ips.clear()
|
||||
top_countries.clear(); top_vhosts.clear()
|
||||
total_threats = 0
|
||||
threats_today = 0
|
||||
byte_position = 0
|
||||
ip_countries.clear()
|
||||
if entry.get("timestamp", "").startswith(today):
|
||||
stats["threats_today"] += 1
|
||||
|
||||
if size_now > byte_position:
|
||||
with open(log_path) as f:
|
||||
f.seek(byte_position)
|
||||
for line in f:
|
||||
try:
|
||||
entry = json.loads(line.strip())
|
||||
total_threats += 1
|
||||
stats["by_category"][entry.get("category", "unknown")] += 1
|
||||
stats["by_severity"][entry.get("severity", "unknown")] += 1
|
||||
|
||||
if entry.get("timestamp", "").startswith(today):
|
||||
threats_today += 1
|
||||
# IP tracking - try both field names for compatibility
|
||||
ip = entry.get("client_ip") or entry.get("ip", "unknown")
|
||||
stats["top_ips"][ip] += 1
|
||||
|
||||
by_category[entry.get("category", "unknown")] += 1
|
||||
by_severity[entry.get("severity", "unknown")] += 1
|
||||
# Country lookup via GeoIP (cache per IP)
|
||||
if ip not in ip_countries:
|
||||
ip_countries[ip] = _lookup_country(ip, geoip_reader)
|
||||
country = ip_countries[ip]
|
||||
stats["top_countries"][country] += 1
|
||||
|
||||
ip = entry.get("client_ip") or entry.get("ip", "unknown")
|
||||
top_ips[ip] += 1
|
||||
|
||||
if ip not in ip_countries:
|
||||
ip_countries[ip] = _lookup_country(ip, geoip_reader)
|
||||
top_countries[ip_countries[ip]] += 1
|
||||
|
||||
vhost = entry.get("host") or entry.get("vhost", "unknown")
|
||||
top_vhosts[vhost] += 1
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
byte_position = f.tell()
|
||||
# Vhost tracking
|
||||
vhost = entry.get("host") or entry.get("vhost", "unknown")
|
||||
stats["top_vhosts"][vhost] += 1
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Cap the ip_countries dict so it doesn't grow without bound.
|
||||
# Top-1000 most recently seen IPs is plenty for the dashboard.
|
||||
if len(ip_countries) > 1200:
|
||||
ip_countries = dict(
|
||||
sorted(ip_countries.items(), key=lambda kv: -top_ips.get(kv[0], 0))[:1000]
|
||||
)
|
||||
# Convert defaultdicts and get top 10
|
||||
stats["by_category"] = dict(stats["by_category"])
|
||||
stats["by_severity"] = dict(stats["by_severity"])
|
||||
|
||||
# Persist before returning so the next call starts from here.
|
||||
_save_stats_disk_cache({
|
||||
"today_iso": today,
|
||||
"threats_today": threats_today,
|
||||
"byte_position": byte_position,
|
||||
"counters": {
|
||||
"total_threats": total_threats,
|
||||
"by_category": dict(by_category),
|
||||
"by_severity": dict(by_severity),
|
||||
"top_ips": dict(top_ips),
|
||||
"top_countries": dict(top_countries),
|
||||
"top_vhosts": dict(top_vhosts),
|
||||
},
|
||||
"ip_countries": ip_countries,
|
||||
"last_updated": int(time.time()),
|
||||
})
|
||||
# Top IPs with country codes included
|
||||
top_ips_sorted = sorted(stats["top_ips"].items(), key=lambda x: -x[1])[:10]
|
||||
stats["top_ips"] = {ip: count for ip, count in top_ips_sorted}
|
||||
stats["top_ips_countries"] = {ip: ip_countries.get(ip, "??") for ip, _ in top_ips_sorted}
|
||||
|
||||
return _finalize_stats(
|
||||
total_threats, threats_today, by_category, by_severity,
|
||||
top_ips, top_countries, top_vhosts, ip_countries,
|
||||
)
|
||||
stats["top_countries"] = dict(sorted(stats["top_countries"].items(), key=lambda x: -x[1])[:10])
|
||||
stats["top_vhosts"] = dict(sorted(stats["top_vhosts"].items(), key=lambda x: -x[1])[:10])
|
||||
|
||||
|
||||
def _finalize_stats(
|
||||
total_threats: int, threats_today: int,
|
||||
by_category, by_severity, top_ips, top_countries, top_vhosts,
|
||||
ip_countries: dict,
|
||||
) -> dict:
|
||||
"""Shape the dashboard-friendly result : top-10 lists + plain dicts."""
|
||||
top_ips_sorted = sorted(top_ips.items(), key=lambda x: -x[1])[:10]
|
||||
return {
|
||||
"total_threats": total_threats,
|
||||
"threats_today": threats_today,
|
||||
"by_category": dict(by_category),
|
||||
"by_severity": dict(by_severity),
|
||||
"top_ips": {ip: count for ip, count in top_ips_sorted},
|
||||
"top_ips_countries": {
|
||||
ip: ip_countries.get(ip, "??") for ip, _ in top_ips_sorted
|
||||
},
|
||||
"top_countries": dict(
|
||||
sorted(top_countries.items(), key=lambda x: -x[1])[:10]
|
||||
),
|
||||
"top_vhosts": dict(
|
||||
sorted(top_vhosts.items(), key=lambda x: -x[1])[:10]
|
||||
),
|
||||
}
|
||||
return stats
|
||||
|
||||
|
||||
# ───────────────────────────────────────────────────────────────────────
|
||||
|
|
|
|||
|
|
@ -1,29 +1,3 @@
|
|||
secubox-waf (1.2.2-1~bookworm1) bookworm; urgency=medium
|
||||
|
||||
* Phase 11+ (#509) — double-buffered cache for WAF stats consumed by
|
||||
both /waf/ and /soc/ dashboards.
|
||||
Live triage on gk2 (2026-06-10) found the aggregator pinned at
|
||||
89 % CPU with 8+ concurrent open file descriptors on the 110 MB
|
||||
waf-threats.log because _get_threat_stats() iterated the full
|
||||
JSONL on every request. Both dashboards showed empty cards.
|
||||
Fix : incremental log reader with persisted byte position.
|
||||
- New disk cache : /var/lib/secubox/waf/stats-disk-cache.json
|
||||
stores counters + byte_position + ip_countries. Atomic
|
||||
write (.tmp -> rename) so a crash mid-write never corrupts.
|
||||
- _get_threat_stats() reloads from disk, seeks to the last
|
||||
position, reads only the new tail since then.
|
||||
- Log rotation / truncation detected via size shrink ; counters
|
||||
reset cleanly.
|
||||
- Day rollover resets threats_today only ; full-history
|
||||
counters keep accumulating.
|
||||
- ip_countries dict capped at 1200 entries (most-active 1000
|
||||
retained on overflow).
|
||||
Net effect : /waf/stats steady-state under 100 ms ; warm refresh
|
||||
cycle under 1 s per tick instead of 30 s ; aggregator CPU drops
|
||||
back to idle.
|
||||
|
||||
-- Gerald KERMA <devel@cybermind.fr> Wed, 10 Jun 2026 08:40:00 +0200
|
||||
|
||||
secubox-waf (1.2.1-1~bookworm1) bookworm; urgency=medium
|
||||
|
||||
* Phase 7 follow-up (#498) — LXC mitmproxy.service memory hygiene :
|
||||
|
|
|
|||
Loading…
Reference in New Issue
Block a user