Compare commits

...

17 Commits

Author SHA1 Message Date
47076b24d3 fix(netplan): mochabin — copper eth2 as 2nd DHCP WAN candidate, out of br-lan
Some checks are pending
License Headers / check (push) Waiting to run
The image labelled eth0 (SFP+) as the only WAN; the single copper RJ45
(eth2/mvpp2-2) was trapped in br-lan, so an operator/uplink arriving on
copper got no lease. Make both eth0 (SFP+) and eth2 (copper) DHCP WAN
candidates (whichever is cabled gets the lease); br-lan keeps the switch
LAN ports. Verified live on the c3box reference install.
2026-06-27 16:27:05 +02:00
cb6eee4b0b docs: netboot validation + c3box first install to Debian + #748 (ref #748 #737)
- HISTORY: 2026-06-27 milestone — gk2→c3box netboot proven, first SecuBox Debian
  install on a physical MOCHAbin, #748 wget scissor documented, boot.scr workaround
- WIP: update to 2026-06-27; c3box=reference install DONE; #748 deferred; rig teardown pending
- TODO: T5 section — proper bootloader (extlinux addr fix + #748 paths), package the
  netboot signed-install flow, rig teardown checklist
- wiki/Netboot-Install.md: new reference page — prereqs, TFTP steps, signed install,
  boot.scr persistence fix, gotchas table, eMMC layout
- wiki/_Sidebar.md: link Netboot-Install under BOOT section
2026-06-27 16:27:05 +02:00
CyberMind
5d505cae16
Merge PR #759 — nft-based network stats dashboard (#758)
nft-based network stats — log/counter drops, attacks, ad-blocks, in/out traffic into the dashboard
2026-06-27 16:25:52 +02:00
1c6a978748 feat(toolbox): add Réseau dashboard tab with nft throughput sparklines (ref #758) 2026-06-27 11:35:52 +02:00
9696cdad13 feat(toolbox): network_drops sourced from hub netstats snapshot (ref #758) 2026-06-27 11:31:16 +02:00
fd9f535e63 fix(hub): require JWT on netstats endpoints (ref #758)
Add user=Depends(require_jwt) to netstats_summary and netstats_series,
matching the sibling pattern used throughout the same router.
Drop redundant int() casts on already-typed window/step params.
2026-06-27 11:29:00 +02:00
d91f6a8b67 feat(hub): /netstats/summary + /netstats/series endpoints (ref #758) 2026-06-27 11:26:06 +02:00
c1c4810f3e feat(hub): package netstats collector, timer, nft tap deploy, sudoers (ref #758) 2026-06-27 11:22:04 +02:00
d05deee7ff feat(hub): netstats snapshot builder + privileged collector + root wrapper (ref #758) 2026-06-27 11:16:38 +02:00
b30a69316a fix(hub): robust max_ts guard + multi-name category aggregation test (ref #758)
- Replace truthiness guard (max_ts_c or max_ts_i) with explicit None-check
  any(t is not None ...) so a sole sample at ts=0 (boot edge) is handled by
  intent, not coincidence.
- Add test_query_drops_sums_multiple_names_per_category: proves blacklist_v4
  and blacklist_v6 (same category) are summed (10+5=15) in the same bucket,
  not overwritten.
2026-06-27 11:13:58 +02:00
851af7d172 feat(hub): netstats SQLite store + reset-aware series query (ref #758) 2026-06-27 11:10:43 +02:00
55814ac3a0 feat(hub): netstats pure parsers (proc/net/dev, nft counters, reset-aware delta) (ref #758) 2026-06-27 11:07:27 +02:00
e6260215a4 feat(hub): inet filter input policy-drop nft tap counter (ref #758) 2026-06-27 11:04:27 +02:00
6ae107db91 feat(mitmproxy): named nft counter on WAF rate-limit drops (ref #758) 2026-06-27 11:02:06 +02:00
60f876ad08 feat(toolbox): named nft counters in blacklist spine + named-counter reader (ref #758) 2026-06-27 10:58:58 +02:00
bdafced25a docs(netstats): implementation plan — 11 TDD tasks (ref #758) 2026-06-27 10:54:47 +02:00
5b283b6bff docs(netstats): design spec — nft-based network stats dashboard (ref #758) 2026-06-27 10:14:23 +02:00
34 changed files with 3095 additions and 34 deletions

View File

@ -3,6 +3,63 @@
--- ---
## 2026-06-27 — Netboot live PROUVÉ + première install SecuBox Debian sur c3box (second MOCHAbin) (#748 #737)
Grande session hardware : netboot gk2→c3box validé de bout en bout, premier SecuBox Debian installé
sur un vrai MOCHAbin, et le blocage U-Boot qui empêche #748 de fermer est formellement documenté.
### A. Netboot gk2 → c3box : validé en prod
- **c3box** (second MOCHAbin, Armada 7040) a booté l'installeur SecuBox Debian servi par gk2 via
TFTP : factory U-Boot 2020.10 → `tftpboot Image/dtb/initrd``booti` → rescue shell installeur,
kernel custom 6.12.85 #5secubox. Le FIT signé (49 Mo) était servi en HTTP sur `:8099`.
- Le long détour cabling était une impasse LAB (prouvé via gk2 bridge-FDB + test DHCP) — aucun
bug logiciel.
- **Learnings opérationnels réutilisables** (documentés dans `wiki/Netboot-Install.md`) :
- Factory U-Boot 2020.10 s'interrompt sur **Enter** (pas Ctrl-C), `bootdelay=2`.
- Son env n'est PAS dans SPI mtd2 (env étranger fossile) → `fw_setenv` depuis Linux n'a aucun
effet ; seule la config U-Boot interne compte.
- Seul le port cuivre RJ45 unique = `mvpp2-2` est bootable par le factory U-Boot (les 4 ports
switch nécessitent le driver MV88E6XXX DSA, absent au boot).
- Kernel load à `0x02080000` = adresse mémoire réservée → crash immédiat ; utiliser `0x0a000000`.
- `setenv tftpblocksize 1468` pour TFTP rapide.
### B. #748 enhanced Tow-Boot (HTTP/wget bootloader) — DIFFÉRÉ, bloquant documenté
Branche `feature/748-enhanced-tow-boot-http-netboot-serial-fl` (stackée sur #737) :
spec+plan (`docs/superpowers/`), Kconfig Tow-Boot, `build-uboot-overlay.sh --tow-boot`,
plan serial-flasher, CI `.github/workflows/build-tow-boot.yml` (push-triggered).
**Bloquant dur (ciseau)** : le board MOCHAbin n'existe que dans le fork U-Boot 2022.07 de
Tow-Boot (pas de `wget`) ; `wget` n'existe que dans U-Boot stock ≥2023.07 (pas de board
mochabin/DTS). Bump à stock 2023.07 = `wget` compile mais build sans DTS. Pour débloquer :
backporter wget/TCP dans le fork Tow-Boot 2022.07, OU porter le board mochabin vers mainline
≥2023.07. Pas un tweak de config.
### C. PREMIÈRE INSTALL — c3box → SecuBox Debian (la headline)
- **Image** : artefact CI `secubox-mochabin-bookworm` (run 27426515472, 1,8 Go gzip / 8,0 Gio
décompressé), téléchargée sur gk2 `/data`, SHA256SUMS vérifié.
- **Signature** : clé `secubox-netboot.key` de gk2. Vérifié : cette clé FIT == `netboot-image.pub`
embarquée dans l'installeur (modulus match + roundtrip sign/verify). `sbx.img.gz` + `.sig`
publiés dans le root HTTP netboot, servis sur `:8099` (symlink depuis `/data`).
- **Install automatisé depuis le rescue shell** :
`wget sbx.img.gz` (en RAM, c3box a 8 Go) →
`openssl dgst -verify` contre `netboot-image.pub` (résultat : Verified OK) →
`gunzip | dd of=/dev/mmcblk0 bs=4M conv=fsync` (8 Gio, progression 32→62→94→100%) → sync.
- **c3box démarre SecuBox Debian v1.9.0** — hostname `secubox-mochabin`, kernel Debian
6.1.0-47-arm64, stack complète : secuboxd, hub, grafana, zigbee, mqtt, authelia,
sentinel/rogue-BTS (layers WALL+MIND). Creds root/secubox, Web UI `:9443`.
- **Fix auto-boot persistant** : l'image utilise `extlinux.conf` à `0x02080000` (adresse réservée
factory U-Boot → reset immédiat) et ne livre pas de `boot.scr` compilé. Construit
`/boot/boot.scr` (kernel@`0x0a000000`, initrd@`0x10000000`, `console=ttyS0` + earlycon,
`root=LABEL=rootfs`) : le factory U-Boot charge `boot.scr` depuis mmc et démarre Debian sans
intervention. **VÉRIFIÉ** : reboot sans intervention → login Debian.
- **Layout eMMC installé** : GPT p1=boot (FAT, `/boot`) p2=ROOT (`/`) p3=DATA. c3box était
OpenWrt ; eMMC écrasé (install RAM-only, pas de risque sur l'OS tournant avant le `dd`).
- **Rig netboot temporaire gk2 encore actif** : `lan1=192.168.77.1/24`, dnsmasq test (DHCP) sur
`lan1`, `nft iif lan1 accept`, nginx boot-vhost extra listen `192.168.77.1:8099`.
## 2026-06-24 (cont.) — R4 analyst mode: MITM-everything + media reverse-catcher + clone (#736) ## 2026-06-24 (cont.) — R4 analyst mode: MITM-everything + media reverse-catcher + clone (#736)
New "R4" doctrine — visibility over performance. Delivered + live on gk2: New "R4" doctrine — visibility over performance. Delivered + live on gk2:

View File

@ -1,5 +1,53 @@
# TODO — SecuBox-DEB Backlog # TODO — SecuBox-DEB Backlog
*Mis à jour : 2026-06-22* *Mis à jour : 2026-06-27*
---
## ⚪ T5 — Images / OS variants / Hardware (ajouts 2026-06-27)
### ⬜ MOCHAbin — bootloader propre (adresses réservées + extlinux)
> Workaround actif : `/boot/boot.scr` compilé forçant le kernel à `0x0a000000`. Fix durable requis.
- [ ] **Option A — Corriger l'image** : patcher `extlinux.conf` généré par le CI pour utiliser
`0x0a000000` (kernel) et `0x10000000` (initrd) au lieu de `0x02080000` (adresse réservée
factory U-Boot 2020.10 → reset immédiat). Boot.scr deviendrait redondant.
- [ ] **Option B — Enhanced Tow-Boot (#748)** : bloqué par le ciseau U-Boot (voir ci-dessous) ;
déverrouille wget/HTTP natif dans U-Boot, supprime le besoin de TFTP pour les futures installs.
- [ ] **Valider** que le fix d'adresse tient sur les deux MOCHAbin (gk2 + c3box).
### ⬜ #748 — wget dans U-Boot pour MOCHAbin (bloquant documenté)
> Bloquant dur (ciseau) confirmé 2026-06-27. Branche
> `feature/748-enhanced-tow-boot-http-netboot-serial-fl` : spec + plan + Kconfig +
> `build-uboot-overlay.sh --tow-boot` + CI `.github/workflows/build-tow-boot.yml` en place.
> Problème : board mochabin UNIQUEMENT dans fork Tow-Boot U-Boot 2022.07 (pas de `wget`) ;
> `wget`/TCP UNIQUEMENT dans stock U-Boot ≥2023.07 (pas de board mochabin/DTS).
- [ ] **Voie 1** : backporter le stack TCP + `wget` de U-Boot ≥2023.07 dans le fork Tow-Boot
2022.07 (mochabin board natif). Diff TCP = `net/wget.c` + dépendances `CONFIG_NET_WGET`.
- [ ] **Voie 2** : porter le board mochabin (DTS Armada 7040 + PHY + eMMC) vers U-Boot mainline
≥2023.07 (sans Tow-Boot). Plus long mais durable.
- [ ] Choisir une voie, débloquer #748.
### ⬜ Packager le flow netboot + install signé (rig temporaire → procédure reproductible)
> Actuellement rig manuel sur gk2 : `lan1=192.168.77.1/24`, dnsmasq DHCP, nft, nginx `:8099`.
- [ ] Scripter la publication de l'image signée dans le root HTTP netboot (wget + sha256 + sig).
- [ ] Documenter / packager la config dnsmasq + nft + nginx pour un segment `lan1` dédié.
- [ ] Intégrer dans `scripts/deploy-netboot.sh` ou équivalent.
### ⬜ Teardown rig netboot temporaire gk2
> Le rig (lan1 bridge, dnsmasq, nft iif lan1 accept, nginx extra listen) reste actif jusqu'à
> ce que c3box soit autonome en prod.
- [ ] Retirer la règle nft `iif lan1 accept` (risque : tout le segment lan1 est accepté sans filtrage).
- [ ] Désactiver / retirer dnsmasq test sur lan1.
- [ ] Retirer le extra listen `192.168.77.1:8099` du vhost nginx netboot (ou couper le vhost si
plus nécessaire).
- [ ] Vérifier que c3box auto-boot sans rig (boot.scr en place → OK).
--- ---

View File

@ -1,5 +1,34 @@
# WIP — Work In Progress # WIP — Work In Progress
*Mis à jour : 2026-06-22* *Mis à jour : 2026-06-27*
---
## ✅ 2026-06-27 : c3box → SecuBox Debian — première install réussie · netboot prouvé (#748 #737)
### ✅ Fait (session 2026-06-27)
- **Netboot gk2→c3box prouvé** — factory U-Boot 2020.10 → TFTP → rescue shell installeur
(kernel 6.12.85 #5secubox). Détour cabling résolu (impasse LAB, pas logiciel).
- **Première install SecuBox Debian sur un MOCHAbin physique (c3box)** — image CI artefact
`secubox-mochabin-bookworm` (run 27426515472, 8 Gio), SHA256 + signature vérifiés,
`gunzip|dd` en RAM → eMMC. c3box boot Debian v1.9.0 avec stack complète.
- **boot.scr workaround déployé** — extlinux.conf charge le kernel à `0x02080000` (réservé
factory U-Boot → reset). Construit `/boot/boot.scr` (kernel@`0x0a000000`) ; auto-boot
Debian sans intervention vérifié après reboot.
- **#748 bloquant documenté** — ciseau U-Boot : mochabin board UNIQUEMENT dans fork Tow-Boot
2022.07 (pas de `wget`) ↔ `wget` UNIQUEMENT dans stock ≥2023.07 (pas de board mochabin).
Branche `feature/748-enhanced-tow-boot-http-netboot-serial-fl` parkée (spec+CI+Kconfig en
place, dépend du backport wget OU port board mainline).
### ⬜ Rig netboot temporaire gk2 à démonter (quand c3box autonome)
- `lan1=192.168.77.1/24` avec dnsmasq DHCP + `nft iif lan1 accept` + nginx `:8099` encore actifs.
- À retirer une fois c3box en prod (voir TODO T5 — teardown rig).
### ⬜ Bootloader propre à faire (#748 ou alternative)
- boot.scr = workaround ; fix durable = enhanced Tow-Boot (#748, bloqué ciseau) OU corriger
les adresses de boot dans l'image (extlinux.conf → `0x0a000000`). Voir TODO T5.
--- ---

View File

@ -6,16 +6,21 @@ network:
renderer: networkd renderer: networkd
ethernets: ethernets:
# WAN — connecté à l'opérateur # WAN candidate (SFP+, eth0) — connecté à l'opérateur via fibre/module SFP.
eth0: eth0:
dhcp4: true dhcp4: true
dhcp6: false dhcp6: false
optional: true optional: true
# LAN — ports GbE (DSA ou directs selon la config switch) # LAN — port GbE switch (DSA 88E6341)
eth1: eth1:
optional: true optional: true
# WAN candidate (RJ45 cuivre, eth2 = mvpp2-2). Sur MOCHAbin le seul RJ45
# direct ; sert d'uplink quand l'opérateur arrive en cuivre. Le port WAN
# câblé (eth0 SFP+ OU eth2 cuivre) obtient le bail DHCP ; l'autre reste idle.
eth2: eth2:
dhcp4: true
dhcp6: false
optional: true optional: true
eth3: eth3:
optional: true optional: true
@ -31,7 +36,7 @@ network:
bridges: bridges:
# Bridge LAN # Bridge LAN
br-lan: br-lan:
interfaces: [eth1, eth2, eth3, eth4] interfaces: [eth1, eth3, eth4]
addresses: [192.168.1.1/24] addresses: [192.168.1.1/24]
dhcp4: false dhcp4: false
parameters: parameters:

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,211 @@
# Design — nft-based network stats into the dashboard (#758)
**Date:** 2026-06-27
**Issue:** [#758](https://github.com/CyberMind-FR/secubox-deb/issues/758)
**Status:** Approved design, pending implementation plan
**Branch:** `feature/758-nft-based-network-stats-log-counter-drop`
---
## 1. Problem
The `#ads` dashboard breakdown (issue #755) exposes a **"Drops réseau"** KPI fed by
`network_drops`, which currently always reads **0**. Its source is an anonymous
inline counter in the `inet secubox_blacklist` `enforce` chain, read via
`nft -j list table inet secubox_blacklist` — and it only covers blacklist /
quarantine drops, which sit at 0 until the blacklist-sync daemon populates the
sets. There is no view of general firewall drops, attack/blocked-traffic
categories, or in/out interface throughput.
This work feeds **real network-layer stats from nftables** (plus interface
counters) into the dashboard: categorized drops, attack/blocked traffic,
ad-blocks (cross-referenced), and in/out throughput — with a 24h time-series and
charts.
## 2. Goals / Non-goals
### Goals
- Real, categorized **drops** and **attacks** sourced from nft **named counters**.
- **In/out** throughput per interface.
- **Time-series** retention (SQLite) powering 24h charts.
- A new **"Réseau"** tab in the toolbox dashboard.
- The existing `#ads` "Drops réseau" KPI becomes real.
### Non-goals (v1)
- No per-flow / per-connection accounting (that is nDPId / metrics territory).
- No injection of named counters into the externally-managed `inet crowdsec`
table (read-only best-effort instead).
- No new heavy charting dependency if the dashboard already ships one.
## 3. Decisions (from brainstorming)
| Decision | Choice |
|---|---|
| v1 scope | Full dashboard with time-series |
| Ownership | **Hub collects** (reuses its root nft poller + sudoers), **toolbox renders** |
| Instrumentation | **Hybrid** — named nft counters (drops/attacks) + `/proc/net/dev` (in/out) + app-layer `ad_block_stats` (ads) |
| Store | **SQLite** time-series (`/var/lib/secubox/hub/netstats.db`) |
| UI surface | New **"Réseau"** tab in the toolbox dashboard |
## 4. Architecture
```
[kernel nft named counters] ──┐
[/proc/net/dev interfaces] ──┼─► secubox-netstats-collect (root oneshot, 30s timer, in secubox-hub)
[inet crowdsec counters (RO)]─┘ │
├─► SQLite /var/lib/secubox/hub/netstats.db (cumulative samples, time-series)
└─► snapshot /var/lib/secubox/hub/netstats.json (latest + instantaneous rates, 0644)
hub api (user secubox, under aggregator) reads DB/json ──► GET /api/v1/hub/netstats/{summary,series}
toolbox "Réseau" tab JS ──► fetch /api/v1/hub/netstats/* (charts + tiles)
toolbox /admin/ad-stats ──► reads netstats.json → real network_drops (#ads KPI)
```
**Root/user split.** The collector runs as **root** (oneshot + timer, mirrors the
existing `secubox-nft-cache.timer` pattern in hub) because `nft -j list counters`
needs `CAP_NET_ADMIN`. The hub API and toolbox API run as user `secubox` inside
the aggregator process and only **read** the DB / snapshot file. A new dedicated
collector is used rather than overloading `secubox-nft-cache`, because this one
also does SQLite inserts, `/proc` reads, and crowdsec reads.
## 5. nft instrumentation — named counters
Add **declared named counter objects** to the owning tables and reference them
with `counter name "…"` on the existing drop rules. Naming convention:
`sbx_<category>[_<af>]`.
| Category | Counter(s) | File (owner package) | Change |
|---|---|---|---|
| C2 blacklist drops | `sbx_drop_blacklist_v4`, `sbx_drop_blacklist_v6` | `secubox-blacklist.nft` (secubox-toolbox) | add |
| Quarantine drops | `sbx_drop_quarantine_v4`, `sbx_drop_quarantine_v6` | `secubox-blacklist.nft` (secubox-toolbox) | add |
| DoH/DoT detect | `sbx_doh_detect_v4`, `sbx_doh_detect_v6` | `secubox-blacklist.nft` (secubox-toolbox, `doh_watch`) | add |
| WAF rate-limit (scanners/bots) | `sbx_drop_wafrl` | `secubox-waf-ratelimit.nft` (secubox-mitmproxy) | add |
| Unsolicited inbound (policy drop) | `sbx_drop_input_policy` | **new** `zz-secubox-netstats-tap.nft` (secubox-hub) | add |
| CrowdSec bouncer drops | existing counters in `inet crowdsec` | — | **read-only, best-effort** |
Notes:
- Named counter objects **must be declared in the same table** as the rules that
reference them (no cross-table counter refs). Each owning `.nft` declares its
own counters at table scope, then references them inline:
`ip daddr @blacklist_v4 limit rate 20/second log prefix "SBX-BL-DROP " counter name "sbx_drop_blacklist_v4" drop`.
- The **unsolicited-inbound** tap is a hub-owned drop-in that declares
`counter inet filter sbx_drop_input_policy` and appends a tail
`add rule inet filter input counter name "sbx_drop_input_policy"` — a bare
counter at the end of the default-drop `input` chain counts exactly what the
policy would drop. It **must load after** all accept rules, hence the `zz-`
prefix (same ordering precedent as `zz-secubox-toolbox-wg-fanout.nft`).
- `inet crowdsec` is created/regenerated by `crowdsec-firewall-bouncer`; we must
**not** rewrite it. The collector reads its counters if present and treats them
as best-effort (0 when absent).
- **Counters reset to 0 on every `nft -f` reload.** The collector and the series
math are reset-aware (§8).
## 6. Collector + store
**Script:** `/usr/lib/secubox/hub/netstats-collect.py` (Python, shipped by secubox-hub).
**Units:** `secubox-netstats.service` (`Type=oneshot`, root) + `secubox-netstats.timer`
(`OnBootSec=15s`, `OnUnitActiveSec=30s`, `AccuracySec=5s`).
Each tick:
1. `nft -j list counters` → map counter-name → cumulative `{packets, bytes}` per category.
2. `/proc/net/dev` → per-interface `rx_bytes, rx_packets, tx_bytes, tx_packets`.
Interfaces from hub config (default: all non-`lo`).
3. Best-effort read of `inet crowdsec` counters.
4. INSERT samples into SQLite (cumulative values + `ts`).
5. Write `netstats.json` snapshot: latest cumulative + **instantaneous rates**
computed vs the previous sample (reset-aware) + `updated` ts.
6. Retention: `DELETE FROM … WHERE ts < now - 7d` (keep a week; charts default 24h).
**SQLite schema** (long-and-narrow, easy to query/downsample; avoids a wide
dynamic-interface table):
```sql
CREATE TABLE IF NOT EXISTS counter_samples (
ts INTEGER NOT NULL,
name TEXT NOT NULL, -- e.g. sbx_drop_blacklist_v4
packets INTEGER NOT NULL,
bytes INTEGER NOT NULL,
PRIMARY KEY (ts, name)
);
CREATE TABLE IF NOT EXISTS iface_samples (
ts INTEGER NOT NULL,
iface TEXT NOT NULL, -- e.g. eth0, br-lan
rx_bytes INTEGER NOT NULL, rx_packets INTEGER NOT NULL,
tx_bytes INTEGER NOT NULL, tx_packets INTEGER NOT NULL,
PRIMARY KEY (ts, iface)
);
CREATE INDEX IF NOT EXISTS idx_counter_ts ON counter_samples(ts);
CREATE INDEX IF NOT EXISTS idx_iface_ts ON iface_samples(ts);
```
File perms: DB + json written by root, dir `0755`, files `0644` (user `secubox`
reads). DB opened read-only by the API.
## 7. API (secubox-hub, read-only)
- `GET /api/v1/hub/netstats/summary`
Latest snapshot: per-category cumulative + current rate (pkt/s, bit/s),
per-interface in/out throughput (bit/s), `updated` ts, `stale` flag.
- `GET /api/v1/hub/netstats/series?window=24h&step=5m&metric=<cat|iface>`
Downsampled buckets; deltas/rates computed server-side from cumulative
samples, **reset-aware**.
`network_drops` for the `#ads` KPI = sum of `sbx_drop_*` categories (+ crowdsec
best-effort), read by toolbox from `netstats.json`.
## 8. Correctness / error handling
- **Counter resets:** if a cumulative value is **lower** than the previous
sample, an `nft -f` reload occurred → that interval's delta = the current value
(never negative). Same logic for `/proc` interface counters (rare 64-bit wrap /
iface re-create).
- Each source is wrapped independently: missing counter/table/iface ⇒ 0, logged,
others proceed.
- Collector down ⇒ API serves last snapshot with `stale: true` + `updated` age;
frontend renders a staleness badge.
- SQLite write failure ⇒ snapshot still written; error logged to journald.
- Read/write privilege split enforced by file ownership (root writes, secubox
reads).
## 9. Frontend — new "Réseau" tab (`packages/secubox-toolbox/www/toolbox/index.html`)
- **Throughput chart:** in/out bit/s per interface, 24h (from `/series`).
- **Drops/attacks trend:** stacked-by-category, 24h (from `/series`).
- **Breakdown tiles:** total drops, by category, top sources (from `/summary`).
- **#ads KPI:** "Drops réseau" repointed to the real `network_drops`.
- **Charts:** reuse the dashboard's existing charting lib if present; otherwise a
lightweight inline SVG sparkline — **no new heavy dependency**. (Confirmed
during implementation.)
- Staleness badge when `stale: true`.
## 10. Packages touched + deployment
| Package | Change | Bump |
|---|---|---|
| **secubox-hub** | collector script, `.service`+`.timer`, `zz-secubox-netstats-tap.nft`, SQLite schema, API endpoints (sudoers already grants `nft -j list *`) | yes |
| **secubox-toolbox** | `secubox-blacklist.nft` named counters, new "Réseau" tab, repoint `network_drops` | yes |
| **secubox-mitmproxy** | `secubox-waf-ratelimit.nft` named counters | yes |
| secubox-crowdsec | none (read-only) | no |
- Each package's `postinst` reloads its own `.nft` (existing idempotent
delete+recreate pattern). `nft -c -f` syntax check added in CI for changed/new
`.nft` files.
- Postinst must preserve runtime state (try-restart for the timer; redeploy
operator drop-ins) per existing project guidance.
## 11. Testing
- **Unit:** counter-name→category map; `/proc/net/dev` parser; reset-aware
delta/rate math; SQLite insert + downsample query; snapshot staleness logic.
- **Integration:** one collector run yields rows + valid `/summary` and `/series`
shapes; graceful when DB / counters / crowdsec table are absent.
- **nft:** `nft -c -f` syntax check on each modified/new `.nft`.
## 12. Open items (resolve during implementation)
- Confirm the dashboard's existing charting approach before choosing chart impl.
- Confirm whether to expose a `forward` policy-drop tap in addition to `input`
(v1: input only).
- Interface selection: config-driven allow-list vs all-non-`lo` (v1: all-non-`lo`,
config override available).

View File

@ -14,6 +14,7 @@ import asyncio
import os import os
import time import time
from pathlib import Path from pathlib import Path
import netstats # #758 — shared collector/reader module
app = FastAPI(title="secubox-hub", version="1.7.0", root_path="/api/v1/hub") app = FastAPI(title="secubox-hub", version="1.7.0", root_path="/api/v1/hub")
@ -756,6 +757,29 @@ async def network_summary(user=Depends(require_jwt)):
} }
@router.get("/netstats/summary")
async def netstats_summary(user=Depends(require_jwt)) -> dict:
"""#758 — latest network-stats snapshot (categories, interfaces, drops).
Read-only; served from the collector's JSON snapshot (cheap)."""
return netstats.read_snapshot()
@router.get("/netstats/series")
async def netstats_series(window: int = 86400, step: int = 300, user=Depends(require_jwt)) -> dict:
"""#758 — reset-aware drops/throughput time-series for the dashboard charts.
Read-only over the collector's SQLite DB."""
import sqlite3 as _sql
w = max(300, min(window, 7 * 86400))
s = max(30, min(step, 3600))
if not netstats.DB_PATH.exists():
return {"window_s": w, "step_s": s, "drops": {}, "in_bps": {}, "out_bps": {}}
conn = _sql.connect(f"file:{netstats.DB_PATH}?mode=ro", uri=True)
try:
return netstats.query_series(conn, window_s=w, step_s=s)
finally:
conn.close()
@router.get("/quick_actions") @router.get("/quick_actions")
async def quick_actions(user=Depends(require_jwt)): async def quick_actions(user=Depends(require_jwt)):
"""Actions rapides disponibles.""" """Actions rapides disponibles."""

View File

@ -0,0 +1,336 @@
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
# Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
"""SecuBox-Deb :: secubox-hub network-stats (#758).
Shared by the root collector (write path: collect_once/main) and the FastAPI
app (read path: read_snapshot/query_series). Pure functions are unit-tested;
the privileged collect path is integration-tested with monkeypatched sources.
"""
from __future__ import annotations
import json
import sqlite3
import subprocess
import time
from pathlib import Path
DB_PATH = Path("/var/lib/secubox/hub/netstats.db")
SNAP_PATH = Path("/var/lib/secubox/hub/netstats.json")
DATA_DIR = DB_PATH.parent
STALE_AFTER_S = 120 # snapshot older than this is flagged stale
# counter-name → category. Named counters live in the owning packages' tables.
CATEGORY_MAP = {
"sbx_drop_blacklist_v4": "blacklist", "sbx_drop_blacklist_v6": "blacklist",
"sbx_drop_quarantine_v4": "quarantine", "sbx_drop_quarantine_v6": "quarantine",
"sbx_doh_detect_v4": "doh", "sbx_doh_detect_v6": "doh",
"sbx_drop_wafrl": "waf_ratelimit",
"sbx_drop_input_policy": "input_policy",
"sbx_drop_crowdsec": "crowdsec",
}
# Categories that count toward "network_drops" (doh is detect-only, excluded).
DROP_CATEGORIES = {"blacklist", "quarantine", "waf_ratelimit", "input_policy", "crowdsec"}
def category_for(name: str) -> str | None:
return CATEGORY_MAP.get(name)
def parse_proc_net_dev(text: str) -> dict[str, dict]:
"""Parse /proc/net/dev → {iface: {rx_bytes,rx_packets,tx_bytes,tx_packets}}.
Skips the two header lines and the loopback interface.
"""
out: dict[str, dict] = {}
for line in text.splitlines():
if ":" not in line:
continue
name, _, rest = line.partition(":")
name = name.strip()
if name == "lo" or not name:
continue
f = rest.split()
if len(f) < 16:
continue
out[name] = {
"rx_bytes": int(f[0]), "rx_packets": int(f[1]),
"tx_bytes": int(f[8]), "tx_packets": int(f[9]),
}
return out
def parse_nft_counters_json(data: dict) -> dict[str, dict]:
"""Parse `nft -j list counters` (or list table) → {name: {packets,bytes}}."""
out: dict[str, dict] = {}
for item in data.get("nftables", []):
c = item.get("counter")
if isinstance(c, dict) and "name" in c:
out[c["name"]] = {
"packets": int(c.get("packets", 0) or 0),
"bytes": int(c.get("bytes", 0) or 0),
}
return out
def reset_aware_delta(prev: int, cur: int) -> int:
"""Monotonic-counter delta that tolerates resets (nft reload → cur < prev)."""
if cur < prev:
return cur
return cur - prev
# ---------------------------------------------------------------------------
# SQLite store (Task 5)
# ---------------------------------------------------------------------------
def init_db(conn: sqlite3.Connection) -> None:
conn.executescript(
"""
CREATE TABLE IF NOT EXISTS counter_samples (
ts INTEGER NOT NULL, name TEXT NOT NULL,
packets INTEGER NOT NULL, bytes INTEGER NOT NULL,
PRIMARY KEY (ts, name)
);
CREATE TABLE IF NOT EXISTS iface_samples (
ts INTEGER NOT NULL, iface TEXT NOT NULL,
rx_bytes INTEGER NOT NULL, rx_packets INTEGER NOT NULL,
tx_bytes INTEGER NOT NULL, tx_packets INTEGER NOT NULL,
PRIMARY KEY (ts, iface)
);
CREATE INDEX IF NOT EXISTS idx_counter_ts ON counter_samples(ts);
CREATE INDEX IF NOT EXISTS idx_iface_ts ON iface_samples(ts);
"""
)
conn.commit()
def insert_sample(conn: sqlite3.Connection, ts: int, counters: dict, ifaces: dict) -> None:
for name, v in counters.items():
conn.execute(
"INSERT OR REPLACE INTO counter_samples(ts,name,packets,bytes) VALUES(?,?,?,?)",
(ts, name, int(v.get("packets", 0)), int(v.get("bytes", 0))),
)
for iface, v in ifaces.items():
conn.execute(
"INSERT OR REPLACE INTO iface_samples(ts,iface,rx_bytes,rx_packets,tx_bytes,tx_packets) "
"VALUES(?,?,?,?,?,?)",
(ts, iface, int(v["rx_bytes"]), int(v["rx_packets"]),
int(v["tx_bytes"]), int(v["tx_packets"])),
)
conn.commit()
def _bucket(ts: int, step_s: int) -> int:
return ts - (ts % step_s)
def query_series(conn: sqlite3.Connection, window_s: int, step_s: int) -> dict:
"""Reset-aware deltas/rates bucketed to step_s over the last window_s.
Rates are attributed to the bucket of the later sample in each pair.
"""
now_row = conn.execute("SELECT MAX(ts) FROM counter_samples").fetchone()
max_ts_c = now_row[0] if now_row and now_row[0] is not None else None
now_row2 = conn.execute("SELECT MAX(ts) FROM iface_samples").fetchone()
max_ts_i = now_row2[0] if now_row2 and now_row2[0] is not None else None
max_ts = max(t for t in (max_ts_c, max_ts_i) if t is not None) if any(t is not None for t in (max_ts_c, max_ts_i)) else 0
floor = max_ts - window_s
drops: dict[str, dict[int, int]] = {}
rows = conn.execute(
"SELECT ts,name,packets FROM counter_samples WHERE ts>=? ORDER BY name,ts",
(floor - step_s,),
).fetchall()
prev: dict[str, tuple[int, int]] = {}
for ts, name, pk in rows:
cat = category_for(name)
if cat is None:
continue
if name in prev:
d = reset_aware_delta(prev[name][1], pk)
b = _bucket(ts, step_s)
if b >= _bucket(floor, step_s):
drops.setdefault(cat, {}).setdefault(b, 0)
drops[cat][b] += d
prev[name] = (ts, pk)
in_bps: dict[str, dict[int, int]] = {}
out_bps: dict[str, dict[int, int]] = {}
irows = conn.execute(
"SELECT ts,iface,rx_bytes,tx_bytes FROM iface_samples WHERE ts>=? ORDER BY iface,ts",
(floor - step_s,),
).fetchall()
iprev: dict[str, tuple[int, int, int]] = {}
for ts, iface, rx, tx in irows:
if iface in iprev:
pts, prx, ptx = iprev[iface]
dt = ts - pts
if dt > 0:
b = _bucket(ts, step_s)
if b >= _bucket(floor, step_s):
in_bps.setdefault(iface, {})[b] = reset_aware_delta(prx, rx) * 8 // dt
out_bps.setdefault(iface, {})[b] = reset_aware_delta(ptx, tx) * 8 // dt
iprev[iface] = (ts, rx, tx)
def _flatten(d: dict[str, dict[int, int]]) -> dict[str, list]:
return {k: [[b, v[b]] for b in sorted(v)] for k, v in d.items()}
return {
"window_s": window_s, "step_s": step_s,
"drops": _flatten(drops), "in_bps": _flatten(in_bps), "out_bps": _flatten(out_bps),
}
def prune(conn: sqlite3.Connection, keep_s: int) -> None:
for tbl in ("counter_samples", "iface_samples"):
row = conn.execute(f"SELECT MAX(ts) FROM {tbl}").fetchone()
if row and row[0] is not None:
conn.execute(f"DELETE FROM {tbl} WHERE ts < ?", (row[0] - keep_s,))
conn.commit()
# ---------------------------------------------------------------------------
# Snapshot builder + privileged collector (Task 6)
# ---------------------------------------------------------------------------
def build_snapshot(conn: sqlite3.Connection, now: int) -> dict:
"""Latest cumulative per category + instantaneous rates vs the previous
sample. network_drops = sum of DROP_CATEGORIES packet counts (doh excluded).
"""
cats: dict[str, dict] = {}
# latest cumulative per counter-name → fold into categories
rows = conn.execute(
"SELECT name, packets, bytes, ts FROM counter_samples "
"WHERE ts=(SELECT MAX(ts) FROM counter_samples)"
).fetchall()
for name, pk, by, _ts in rows:
cat = category_for(name)
if cat is None:
continue
c = cats.setdefault(cat, {"packets": 0, "bytes": 0})
c["packets"] += int(pk)
c["bytes"] += int(by)
ser = query_series(conn, window_s=120, step_s=30)
for cat, pts in ser["drops"].items():
if pts:
cats.setdefault(cat, {"packets": 0, "bytes": 0})["pps"] = pts[-1][1] / 30.0
ifaces: dict[str, dict] = {}
irow = conn.execute(
"SELECT iface, rx_bytes, tx_bytes FROM iface_samples "
"WHERE ts=(SELECT MAX(ts) FROM iface_samples)"
).fetchall()
for iface, rx, tx in irow:
ifaces[iface] = {"rx_bytes": int(rx), "tx_bytes": int(tx)}
for iface, pts in ser["in_bps"].items():
if pts:
ifaces.setdefault(iface, {})["rx_bps"] = pts[-1][1]
for iface, pts in ser["out_bps"].items():
if pts:
ifaces.setdefault(iface, {})["tx_bps"] = pts[-1][1]
network_drops = sum(
cats.get(cat, {}).get("packets", 0) for cat in DROP_CATEGORIES
)
return {
"updated": now,
"stale": False,
"categories": cats,
"interfaces": ifaces,
"network_drops": int(network_drops),
}
def read_snapshot() -> dict:
"""Read the latest snapshot (API path). Flags stale by `updated` age."""
try:
d = json.loads(SNAP_PATH.read_text())
except Exception:
return {"updated": 0, "stale": True, "categories": {}, "interfaces": {}, "network_drops": 0}
age = max(0, int(time.time()) - int(d.get("updated", 0)))
d["stale"] = age > STALE_AFTER_S
return d
def _run_nft_json(args: list[str]) -> dict:
try:
r = subprocess.run(["/usr/sbin/nft", "-j", "list"] + args,
capture_output=True, text=True, timeout=5)
if r.returncode == 0:
return json.loads(r.stdout or "{}")
except Exception:
pass
return {}
def _read_nft_counters() -> dict:
return parse_nft_counters_json(_run_nft_json(["counters"]))
def _read_crowdsec() -> dict:
"""Best-effort: sum the externally-managed inet crowdsec table counters
into a single synthetic counter mapped to the 'crowdsec' category.
"""
data = _run_nft_json(["table", "inet", "crowdsec"])
total = 0
for item in data.get("nftables", []):
c = item.get("counter")
if isinstance(c, dict):
total += int(c.get("packets", 0) or 0)
rule = item.get("rule")
if isinstance(rule, dict):
for ex in rule.get("expr", []):
cc = ex.get("counter")
if isinstance(cc, dict):
total += int(cc.get("packets", 0) or 0)
if total:
# synthetic name so category_for resolves to 'crowdsec'
return {"sbx_drop_crowdsec": {"packets": total, "bytes": 0}}
return {}
def _read_ifaces() -> dict:
try:
return parse_proc_net_dev(Path("/proc/net/dev").read_text())
except Exception:
return {}
def _open_db(conn: sqlite3.Connection | None):
if conn is not None:
return conn, False
DATA_DIR.mkdir(parents=True, exist_ok=True)
try:
DATA_DIR.chmod(0o755) # our subdir only — NEVER the /var/lib/secubox parent
except Exception:
pass
c = sqlite3.connect(str(DB_PATH))
init_db(c)
return c, True
def collect_once(now: int | None = None, conn: sqlite3.Connection | None = None) -> dict:
if now is None:
now = int(time.time())
c, owns = _open_db(conn)
try:
counters = dict(_read_nft_counters())
counters.update(_read_crowdsec())
ifaces = _read_ifaces()
insert_sample(c, now, counters, ifaces)
prune(c, keep_s=7 * 86400)
snap = build_snapshot(c, now)
try:
SNAP_PATH.write_text(json.dumps(snap))
SNAP_PATH.chmod(0o644)
except Exception:
pass
return snap
finally:
if owns:
c.close()
def main() -> None:
collect_once()
if __name__ == "__main__":
main()

View File

@ -1,3 +1,12 @@
secubox-hub (1.5.0-1~bookworm1) bookworm; urgency=medium
* feat(#758): nft-based network-stats collector — root oneshot+timer samples
nft named counters + /proc/net/dev into /var/lib/secubox/hub/netstats.{db,json};
new read-only endpoints /api/v1/hub/netstats/{summary,series}; inet filter
input policy-drop tap drop-in (zz-secubox-netstats-tap.nft).
-- Gerald KERMA <devel@cybermind.fr> Sat, 27 Jun 2026 12:00:00 +0200
secubox-hub (1.4.7-1~bookworm1) bookworm; urgency=medium secubox-hub (1.4.7-1~bookworm1) bookworm; urgency=medium
* #494: drop the ExecStartPre chown/chmod of the shared /run/secubox parent. * #494: drop the ExecStartPre chown/chmod of the shared /run/secubox parent.

View File

@ -12,6 +12,19 @@ case "$1" in
systemctl start secubox-hub.service || true systemctl start secubox-hub.service || true
install -d -m 755 /etc/nginx/secubox.d install -d -m 755 /etc/nginx/secubox.d
systemctl reload nginx 2>/dev/null || true systemctl reload nginx 2>/dev/null || true
# #758 — deploy + load the network-stats input policy-drop tap
if [ -f /usr/share/secubox/hub/nftables.d/zz-secubox-netstats-tap.nft ]; then
install -d -m 0755 /etc/nftables.d
install -m 0644 /usr/share/secubox/hub/nftables.d/zz-secubox-netstats-tap.nft \
/etc/nftables.d/zz-secubox-netstats-tap.nft
if systemctl is-active --quiet nftables.service; then
systemctl reload nftables.service 2>/dev/null \
|| /usr/sbin/nft -f /etc/nftables.d/zz-secubox-netstats-tap.nft 2>/dev/null || true
fi
fi
# #758 — enable the collector timer (idempotent)
systemctl daemon-reload 2>/dev/null || true
systemctl enable --now secubox-netstats.timer 2>/dev/null || true
;; ;;
esac esac
#DEBHELPER# #DEBHELPER#

View File

@ -24,14 +24,27 @@ override_dh_auto_install:
install -d debian/secubox-hub/etc/systemd/system/timers.target.wants install -d debian/secubox-hub/etc/systemd/system/timers.target.wants
ln -sf /lib/systemd/system/secubox-nft-cache.timer \ ln -sf /lib/systemd/system/secubox-nft-cache.timer \
debian/secubox-hub/etc/systemd/system/timers.target.wants/secubox-nft-cache.timer debian/secubox-hub/etc/systemd/system/timers.target.wants/secubox-nft-cache.timer
# #758 — network-stats collector + timer
install -m 0755 sbin/secubox-netstats-collect debian/secubox-hub/usr/sbin/secubox-netstats-collect
install -m 0644 debian/secubox-netstats.service debian/secubox-hub/lib/systemd/system/
install -m 0644 debian/secubox-netstats.timer debian/secubox-hub/lib/systemd/system/
ln -sf /lib/systemd/system/secubox-netstats.timer \
debian/secubox-hub/etc/systemd/system/timers.target.wants/secubox-netstats.timer
# #758 — ship the inet filter input policy-drop tap drop-in
install -d debian/secubox-hub/usr/share/secubox/hub/nftables.d
install -m 0644 nftables.d/zz-secubox-netstats-tap.nft \
debian/secubox-hub/usr/share/secubox/hub/nftables.d/
# sudoers fragment so the hub service (User=secubox) can fall back to # sudoers fragment so the hub service (User=secubox) can fall back to
# realtime `nft list` when the cache is stale (read-only nft, no risk). # realtime `nft list` when the cache is stale (read-only nft, no risk).
# Two patterns because `-j` is a global option for nft and must precede # Two patterns because `-j` is a global option for nft and must precede
# the command (`nft -j list ruleset`, not `nft list ruleset -j`). # the command (`nft -j list ruleset`, not `nft list ruleset -j`).
install -d debian/secubox-hub/etc/sudoers.d install -d debian/secubox-hub/etc/sudoers.d
printf '%s\n%s\n%s\n' \ # #758 — netstats collector reads `nft -j list table inet crowdsec`
# (best-effort, covered by the existing `nft -j list *` grant above).
printf '%s\n%s\n%s\n%s\n' \
'secubox ALL=(root) NOPASSWD: /usr/sbin/nft list *' \ 'secubox ALL=(root) NOPASSWD: /usr/sbin/nft list *' \
'secubox ALL=(root) NOPASSWD: /usr/sbin/nft -j list *' \ 'secubox ALL=(root) NOPASSWD: /usr/sbin/nft -j list *' \
'secubox ALL=(root) NOPASSWD: /usr/bin/systemctl --no-block start secubox-nft-cache.service' \ 'secubox ALL=(root) NOPASSWD: /usr/bin/systemctl --no-block start secubox-nft-cache.service' \
'secubox ALL=(root) NOPASSWD: /usr/bin/systemctl --no-block start secubox-netstats.service' \
> debian/secubox-hub/etc/sudoers.d/secubox-hub-nft > debian/secubox-hub/etc/sudoers.d/secubox-hub-nft
chmod 0440 debian/secubox-hub/etc/sudoers.d/secubox-hub-nft chmod 0440 debian/secubox-hub/etc/sudoers.d/secubox-hub-nft

View File

@ -0,0 +1,12 @@
[Unit]
Description=SecuBox network-stats collector (nft named counters + /proc/net/dev → SQLite)
Documentation=https://github.com/CyberMind-FR/secubox-deb/issues/758
After=nftables.service
ConditionPathExists=/usr/sbin/nft
[Service]
Type=oneshot
ExecStart=/usr/sbin/secubox-netstats-collect
# Runs as root via systemd — nft list + /proc throughput need privilege.
StandardOutput=journal
StandardError=journal

View File

@ -0,0 +1,12 @@
[Unit]
Description=SecuBox network-stats collector timer (every 30s)
Documentation=https://github.com/CyberMind-FR/secubox-deb/issues/758
[Timer]
OnBootSec=15s
OnUnitActiveSec=30s
AccuracySec=5s
Unit=secubox-netstats.service
[Install]
WantedBy=timers.target

View File

@ -0,0 +1,14 @@
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
# Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
#
# #758 — network-stats tap. Counts packets that fall through to the
# default-drop policy of the base inet filter input chain = "unsolicited
# inbound" volume. ADDITIVE ONLY: `table inet filter { counter … }` adds the
# counter to the pre-existing base table without deleting it; the tail
# `add rule` appends a bare counter AFTER every accept rule. The zz- filename
# guarantees this loads last among /etc/nftables.d/*.nft. Counter resets on
# reload — the collector is reset-aware. Never drops or accepts; pure observe.
table inet filter {
counter sbx_drop_input_policy {}
}
add rule inet filter input counter name "sbx_drop_input_policy" comment "sbx-netstats-input-policy-tap"

View File

@ -0,0 +1,6 @@
#!/bin/sh
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
# Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
# #758 — root oneshot: sample nft named counters + /proc/net/dev into
# /var/lib/secubox/hub/netstats.{db,json}. Runs as root via systemd timer.
exec /usr/bin/python3 -c "import sys; sys.path.insert(0, '/usr/lib/secubox/hub/api'); import netstats; netstats.main()"

View File

@ -0,0 +1,39 @@
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
# Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
# Source-Disclosed License — All rights reserved except as expressly granted.
# See LICENCE-CMSD-1.0.md for terms.
"""Hub netstats endpoints (ref #758)."""
import asyncio
import importlib
import json
import sqlite3
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "api"))
main = importlib.import_module("main")
netstats = importlib.import_module("netstats")
def test_summary_reads_snapshot(tmp_path, monkeypatch):
snap = tmp_path / "netstats.json"
snap.write_text(json.dumps({
"updated": 0, "categories": {"blacklist": {"packets": 4}},
"interfaces": {}, "network_drops": 4,
}))
monkeypatch.setattr(netstats, "SNAP_PATH", snap)
out = asyncio.run(main.netstats_summary())
assert out["network_drops"] == 4
assert out["stale"] is True
def test_series_queries_db(tmp_path, monkeypatch):
db = tmp_path / "netstats.db"
monkeypatch.setattr(netstats, "DB_PATH", db)
c = sqlite3.connect(str(db))
netstats.init_db(c)
netstats.insert_sample(c, 0, {"sbx_drop_wafrl": {"packets": 0, "bytes": 0}}, {})
netstats.insert_sample(c, 30, {"sbx_drop_wafrl": {"packets": 12, "bytes": 0}}, {})
c.close()
out = asyncio.run(main.netstats_series(window=3600, step=30))
assert out["drops"]["waf_ratelimit"][-1][1] == 12

View File

@ -0,0 +1,29 @@
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
# Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
# Source-Disclosed License — All rights reserved except as expressly granted.
# See LICENCE-CMSD-1.0.md for terms.
"""Packaging wires the collector, timer, nft tap, and sudoers (ref #758)."""
from pathlib import Path
PKG = Path(__file__).resolve().parents[1]
def test_units_exist():
assert (PKG / "debian" / "secubox-netstats.service").exists()
assert (PKG / "debian" / "secubox-netstats.timer").exists()
def test_rules_installs_collector_and_units_and_tap():
rules = (PKG / "debian" / "rules").read_text()
assert "sbin/secubox-netstats-collect" in rules
assert "secubox-netstats.service" in rules
assert "secubox-netstats.timer" in rules
assert "zz-secubox-netstats-tap.nft" in rules
# crowdsec read grant added to the sudoers fragment
assert "inet crowdsec" in rules
def test_postinst_deploys_tap_and_enables_timer():
post = (PKG / "debian" / "postinst").read_text()
assert "zz-secubox-netstats-tap.nft" in post
assert "secubox-netstats.timer" in post

View File

@ -0,0 +1,48 @@
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
# Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
"""Pure parsers for the network-stats collector (ref #758)."""
import importlib
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "api"))
netstats = importlib.import_module("netstats")
PROC = (
"Inter-| Receive | Transmit\n"
" face |bytes packets errs drop fifo frame compressed multicast|bytes packets errs drop fifo colls carrier compressed\n"
" lo: 123 4 0 0 0 0 0 0 123 4 0 0 0 0 0 0\n"
" eth0: 1000 10 0 0 0 0 0 0 2000 20 0 0 0 0 0 0\n"
)
def test_parse_proc_net_dev_skips_lo_header():
out = netstats.parse_proc_net_dev(PROC)
assert "lo" not in out
assert out["eth0"] == {"rx_bytes": 1000, "rx_packets": 10, "tx_bytes": 2000, "tx_packets": 20}
def test_parse_nft_counters_json():
data = {"nftables": [
{"counter": {"name": "sbx_drop_blacklist_v4", "packets": 5, "bytes": 500}},
{"counter": {"name": "irrelevant", "packets": 9, "bytes": 9}},
{"rule": {"chain": "x"}},
]}
out = netstats.parse_nft_counters_json(data)
assert out["sbx_drop_blacklist_v4"] == {"packets": 5, "bytes": 500}
assert "irrelevant" in out # parser keeps all; mapping happens via category_for
def test_category_for():
assert netstats.category_for("sbx_drop_blacklist_v6") == "blacklist"
assert netstats.category_for("sbx_drop_quarantine_v4") == "quarantine"
assert netstats.category_for("sbx_drop_wafrl") == "waf_ratelimit"
assert netstats.category_for("sbx_drop_input_policy") == "input_policy"
assert netstats.category_for("sbx_doh_detect_v4") == "doh"
assert netstats.category_for("nope") is None
def test_reset_aware_delta():
assert netstats.reset_aware_delta(100, 150) == 50 # normal
assert netstats.reset_aware_delta(150, 10) == 10 # reset → treat cur as delta
assert netstats.reset_aware_delta(0, 0) == 0

View File

@ -0,0 +1,54 @@
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
"""Snapshot builder + collect_once with monkeypatched privileged sources (#758)."""
import importlib
import json
import sqlite3
import sys
import time
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "api"))
netstats = importlib.import_module("netstats")
def test_build_snapshot_network_drops_excludes_doh():
c = sqlite3.connect(":memory:")
netstats.init_db(c)
counters = {
"sbx_drop_blacklist_v4": {"packets": 4, "bytes": 0},
"sbx_drop_wafrl": {"packets": 6, "bytes": 0},
"sbx_doh_detect_v4": {"packets": 99, "bytes": 0},
}
netstats.insert_sample(c, 1000, counters, {})
snap = netstats.build_snapshot(c, now=1000)
assert snap["network_drops"] == 10 # 4 + 6, doh excluded
assert snap["categories"]["doh"]["packets"] == 99
assert snap["updated"] == 1000
def test_collect_once_writes_row_and_snapshot(tmp_path, monkeypatch):
db = tmp_path / "netstats.db"
snap = tmp_path / "netstats.json"
monkeypatch.setattr(netstats, "DB_PATH", db)
monkeypatch.setattr(netstats, "SNAP_PATH", snap)
monkeypatch.setattr(netstats, "DATA_DIR", tmp_path)
monkeypatch.setattr(netstats, "_read_nft_counters",
lambda: {"sbx_drop_wafrl": {"packets": 3, "bytes": 30}})
monkeypatch.setattr(netstats, "_read_crowdsec", lambda: {})
monkeypatch.setattr(netstats, "_read_ifaces",
lambda: {"eth0": {"rx_bytes": 1, "rx_packets": 1, "tx_bytes": 1, "tx_packets": 1}})
out = netstats.collect_once(now=1234)
assert out["network_drops"] == 3
assert snap.exists()
written = json.loads(snap.read_text())
assert written["updated"] == 1234
# a second tick must not raise (DB reused)
netstats.collect_once(now=1264)
def test_read_snapshot_marks_stale(tmp_path, monkeypatch):
snap = tmp_path / "netstats.json"
snap.write_text(json.dumps({"updated": 0, "categories": {}, "interfaces": {}, "network_drops": 0}))
monkeypatch.setattr(netstats, "SNAP_PATH", snap)
out = netstats.read_snapshot()
assert out["stale"] is True

View File

@ -0,0 +1,63 @@
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
# Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
"""SQLite store + reset-aware series for network-stats (ref #758)."""
import importlib
import sqlite3
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "api"))
netstats = importlib.import_module("netstats")
def _conn():
c = sqlite3.connect(":memory:")
netstats.init_db(c)
return c
def test_insert_and_query_drops_delta():
c = _conn()
# t=0 blacklist=100 ; t=30 blacklist=160 → delta 60 in the bucket at t=30
netstats.insert_sample(c, 0, {"sbx_drop_blacklist_v4": {"packets": 100, "bytes": 0}}, {})
netstats.insert_sample(c, 30, {"sbx_drop_blacklist_v4": {"packets": 160, "bytes": 0}}, {})
out = netstats.query_series(c, window_s=3600, step_s=30)
pts = out["drops"]["blacklist"]
assert pts[-1][1] == 60
def test_query_drops_reset_aware():
c = _conn()
netstats.insert_sample(c, 0, {"sbx_drop_blacklist_v4": {"packets": 100, "bytes": 0}}, {})
netstats.insert_sample(c, 30, {"sbx_drop_blacklist_v4": {"packets": 5, "bytes": 0}}, {}) # reload
out = netstats.query_series(c, window_s=3600, step_s=30)
assert out["drops"]["blacklist"][-1][1] == 5 # not negative
def test_query_throughput_bps():
c = _conn()
# 30s apart, +30000 rx bytes → 30000*8/30 = 8000 bps
netstats.insert_sample(c, 0, {}, {"eth0": {"rx_bytes": 0, "rx_packets": 0, "tx_bytes": 0, "tx_packets": 0}})
netstats.insert_sample(c, 30, {}, {"eth0": {"rx_bytes": 30000, "rx_packets": 0, "tx_bytes": 0, "tx_packets": 0}})
out = netstats.query_series(c, window_s=3600, step_s=30)
assert out["in_bps"]["eth0"][-1][1] == 8000
def test_query_drops_sums_multiple_names_per_category():
c = _conn()
# blacklist_v4 and blacklist_v6 both map to category "blacklist"
netstats.insert_sample(c, 0, {"sbx_drop_blacklist_v4": {"packets": 0, "bytes": 0},
"sbx_drop_blacklist_v6": {"packets": 0, "bytes": 0}}, {})
netstats.insert_sample(c, 30, {"sbx_drop_blacklist_v4": {"packets": 10, "bytes": 0},
"sbx_drop_blacklist_v6": {"packets": 5, "bytes": 0}}, {})
out = netstats.query_series(c, window_s=3600, step_s=30)
assert out["drops"]["blacklist"][-1][1] == 15 # 10 + 5 summed, not overwritten
def test_prune_drops_old_rows():
c = _conn()
netstats.insert_sample(c, 0, {"sbx_drop_wafrl": {"packets": 1, "bytes": 0}}, {})
netstats.insert_sample(c, 1_000_000, {"sbx_drop_wafrl": {"packets": 2, "bytes": 0}}, {})
netstats.prune(c, keep_s=10) # relative to max ts
rows = c.execute("SELECT COUNT(*) FROM counter_samples").fetchone()[0]
assert rows == 1

View File

@ -0,0 +1,16 @@
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
"""The hub input policy-drop tap declares + references its counter (ref #758)."""
import re
from pathlib import Path
NFT = Path(__file__).resolve().parents[1] / "nftables.d" / "zz-secubox-netstats-tap.nft"
def test_tap_counter_present_and_zz_ordered():
assert NFT.name.startswith("zz-"), "tap must sort after accept rules"
text = NFT.read_text()
assert re.search(r'counter\s+sbx_drop_input_policy\s*\{', text)
assert re.search(r'add rule inet filter input .*counter name "sbx_drop_input_policy"', text)
# additive only — must NOT delete or flush the base filter table
assert "delete table inet filter" not in text
assert "flush ruleset" not in text

View File

@ -1,3 +1,10 @@
secubox-mitmproxy (1.0.10-1~bookworm1) bookworm; urgency=medium
* feat(nft): named counter sbx_drop_wafrl on the WAF rate-limit drops so the
secubox-hub network-stats collector can attribute attack drops (ref #758).
-- Gerald KERMA <devel@cybermind.fr> Sat, 27 Jun 2026 12:00:00 +0200
secubox-mitmproxy (1.0.9-1~bookworm1) bookworm; urgency=medium secubox-mitmproxy (1.0.9-1~bookworm1) bookworm; urgency=medium
* feat(robustness): self-healing WAF inspector watchdog (closes #624). * feat(robustness): self-healing WAF inspector watchdog (closes #624).

View File

@ -37,6 +37,9 @@ table inet secubox_waf_ratelimit {
elements = { 127.0.0.0/8, 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16 } elements = { 127.0.0.0/8, 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16 }
} }
# #758 — named counter for the network-stats collector (attacks family).
counter sbx_drop_wafrl {}
chain input { chain input {
type filter hook input priority -10; policy accept; type filter hook input priority -10; policy accept;
@ -44,14 +47,14 @@ table inet secubox_waf_ratelimit {
ip saddr @whitelist_v4 accept ip saddr @whitelist_v4 accept
# Drop active offenders fast (no further matching) # Drop active offenders fast (no further matching)
ip saddr @offenders_v4 counter drop ip saddr @offenders_v4 counter name "sbx_drop_wafrl" drop
ip6 saddr @offenders_v6 counter drop ip6 saddr @offenders_v6 counter name "sbx_drop_wafrl" drop
# Rate-limit only on TCP SYN to 80/443 (new public conn attempts) # Rate-limit only on TCP SYN to 80/443 (new public conn attempts)
# nft 'limit rate over X' syntax matches packets exceeding the rate. # nft 'limit rate over X' syntax matches packets exceeding the rate.
tcp flags syn tcp dport { 80, 443 } \ tcp flags syn tcp dport { 80, 443 } \
limit rate over 30/second burst 50 packets \ limit rate over 30/second burst 50 packets \
add @offenders_v4 { ip saddr timeout 5m } \ add @offenders_v4 { ip saddr timeout 5m } \
log prefix "[secubox-rl] " level info counter drop log prefix "[secubox-rl] " level info counter name "sbx_drop_wafrl" drop
} }
} }

View File

@ -0,0 +1,18 @@
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
# Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
# Source-Disclosed License — All rights reserved except as expressly granted.
# See LICENCE-CMSD-1.0.md for terms.
"""WAF rate-limit drops carry a named counter (ref #758)."""
import re
from pathlib import Path
NFT = Path(__file__).resolve().parents[1] / "nftables" / "secubox-waf-ratelimit.nft"
def test_wafrl_counter_declared_and_referenced():
text = NFT.read_text()
decls = set(re.findall(r'counter\s+([a-z0-9_]+)\s*\{', text))
refs = set(re.findall(r'counter name "([a-z0-9_]+)"', text))
assert "sbx_drop_wafrl" in refs
assert "sbx_drop_wafrl" in decls

View File

@ -1,3 +1,12 @@
secubox-toolbox (2.8.0-1~bookworm1) bookworm; urgency=medium
* feat(#758): new "Réseau" dashboard tab — interface throughput sparklines +
per-category drop/attack trends from the hub netstats collector; the #ads
"Drops réseau" KPI now reads real data. Named nft counters added to the
blacklist spine.
-- Gerald KERMA <devel@cybermind.fr> Sat, 27 Jun 2026 12:00:00 +0200
secubox-toolbox (2.7.24-1~bookworm1) bookworm; urgency=medium secubox-toolbox (2.7.24-1~bookworm1) bookworm; urgency=medium
* ui: remove the redundant ♥ Liveness dashboard card (generic status/version); * ui: remove the redundant ♥ Liveness dashboard card (generic status/version);

View File

@ -53,18 +53,29 @@ table inet secubox_blacklist {
flags interval, timeout flags interval, timeout
} }
# #758 — named counters for the network-stats collector. Declared at
# table scope so `nft list counters` exposes them by name; referenced
# by the enforce/doh_watch rules below. Reset to 0 on reload (collector
# is reset-aware).
counter sbx_drop_quarantine_v4 {}
counter sbx_drop_quarantine_v6 {}
counter sbx_drop_blacklist_v4 {}
counter sbx_drop_blacklist_v6 {}
counter sbx_doh_detect_v4 {}
counter sbx_doh_detect_v6 {}
chain enforce { chain enforce {
type filter hook forward priority -10; policy accept; type filter hook forward priority -10; policy accept;
# Quarantine first : a quarantined device is fully cut off. # Quarantine first : a quarantined device is fully cut off.
ip saddr @quarantine_v4 counter drop ip saddr @quarantine_v4 counter name "sbx_drop_quarantine_v4" drop
ip6 saddr @quarantine_v6 counter drop ip6 saddr @quarantine_v6 counter name "sbx_drop_quarantine_v6" drop
# C2 blacklist (13.A). Phase 13.C : rate-limited log so the # C2 blacklist (13.A). Phase 13.C : rate-limited log so the
# attribution tailer can bucket drops per source device without # attribution tailer can bucket drops per source device without
# flooding the kernel log. # flooding the kernel log.
ip daddr @blacklist_v4 limit rate 20/second log prefix "SBX-BL-DROP " counter drop ip daddr @blacklist_v4 limit rate 20/second log prefix "SBX-BL-DROP " counter name "sbx_drop_blacklist_v4" drop
ip saddr @blacklist_v4 counter drop ip saddr @blacklist_v4 counter name "sbx_drop_blacklist_v4" drop
ip6 daddr @blacklist_v6 limit rate 20/second log prefix "SBX-BL-DROP " counter drop ip6 daddr @blacklist_v6 limit rate 20/second log prefix "SBX-BL-DROP " counter name "sbx_drop_blacklist_v6" drop
ip6 saddr @blacklist_v6 counter drop ip6 saddr @blacklist_v6 counter name "sbx_drop_blacklist_v6" drop
} }
# Phase 13.B (#522) — DoH/DoT detection. COUNT-ONLY by default : a # Phase 13.B (#522) — DoH/DoT detection. COUNT-ONLY by default : a
@ -86,7 +97,7 @@ table inet secubox_blacklist {
type filter hook forward priority -11; policy accept; type filter hook forward priority -11; policy accept;
# Phase 13.C : rate-limited log (new connections only) so the # Phase 13.C : rate-limited log (new connections only) so the
# attribution tailer can bucket DoH attempts per device. # attribution tailer can bucket DoH attempts per device.
ip daddr @doh_detect_v4 tcp dport { 443, 853 } ct state new limit rate 5/second log prefix "SBX-DOH " counter ip daddr @doh_detect_v4 tcp dport { 443, 853 } ct state new limit rate 5/second log prefix "SBX-DOH " counter name "sbx_doh_detect_v4"
ip6 daddr @doh_detect_v6 tcp dport { 443, 853 } ct state new limit rate 5/second log prefix "SBX-DOH " counter ip6 daddr @doh_detect_v6 tcp dport { 443, 853 } ct state new limit rate 5/second log prefix "SBX-DOH " counter name "sbx_doh_detect_v6"
} }
} }

View File

@ -45,6 +45,8 @@ try:
_HAS_TRANSPARENCY = True _HAS_TRANSPARENCY = True
except ImportError: except ImportError:
_HAS_TRANSPARENCY = False _HAS_TRANSPARENCY = False
from pathlib import Path as _Path
NETSTATS_SNAPSHOT = _Path("/var/lib/secubox/hub/netstats.json")
from .config import load_config, resolve_secret from .config import load_config, resolve_secret
from .models import AcceptResp, ClientRow, Config, StatusResp from .models import AcceptResp, ClientRow, Config, StatusResp
@ -2971,16 +2973,13 @@ async def admin_blacklist() -> dict:
out["doh_detect_v4"] = n out["doh_detect_v4"] = n
elif name == "doh_detect_v6": elif name == "doh_detect_v6":
out["doh_detect_v6"] = n out["doh_detect_v6"] = n
if "rule" in item: if "counter" in item and isinstance(item["counter"], dict):
chain = item["rule"].get("chain", "") cobj = item["counter"]
for ex in item["rule"].get("expr", []): cname = cobj.get("name", "")
c = ex.get("counter") pk = int(cobj.get("packets", 0) or 0)
if not c: if cname.startswith("sbx_doh_detect"):
continue
pk = int(c.get("packets", 0) or 0)
if chain == "doh_watch":
out["doh_hits"] += pk out["doh_hits"] += pk
else: elif cname.startswith(("sbx_drop_blacklist", "sbx_drop_quarantine")):
out["drops"] += pk out["drops"] += pk
out["active"] = True out["active"] = True
except Exception as e: # noqa: BLE001 except Exception as e: # noqa: BLE001
@ -3096,13 +3095,22 @@ async def admin_ad_stats(hours: int = 24) -> dict:
"""Contextual ad-block metrics for the #ads tab (read-only, kbin-safe).""" """Contextual ad-block metrics for the #ads tab (read-only, kbin-safe)."""
h = max(1, min(int(hours if hours is not None else 24), 168)) h = max(1, min(int(hours if hours is not None else 24), 168))
out = store.ad_stats(hours=h) out = store.ad_stats(hours=h)
# #755 — network-layer drops (blacklist nft sets). Best-effort; 0 when the # #758 — real network-layer drops from the hub netstats collector snapshot.
# blacklist is inert or unreadable. Reuses the admin_blacklist parse. # Fall back to the legacy blacklist nft parse when the snapshot is absent.
nd = None
try:
import json as _json
snap = _json.loads(NETSTATS_SNAPSHOT.read_text())
nd = int(snap.get("network_drops", 0) or 0)
except Exception:
nd = None
if nd is None:
try: try:
bl = await admin_blacklist() bl = await admin_blacklist()
out["network_drops"] = int(bl.get("drops", 0) or 0) nd = int(bl.get("drops", 0) or 0)
except Exception: except Exception:
out["network_drops"] = 0 nd = 0
out["network_drops"] = nd
return out return out

View File

@ -0,0 +1,26 @@
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
# Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
"""admin_blacklist must sum named counter OBJECTS (ref #758)."""
import asyncio
import json
import types
from secubox_toolbox import api
def _fake_nft_json():
return json.dumps({"nftables": [
{"set": {"name": "blacklist_v4", "elem": ["1.2.3.4"]}},
{"counter": {"name": "sbx_drop_blacklist_v4", "packets": 7, "bytes": 700}},
{"counter": {"name": "sbx_drop_quarantine_v4", "packets": 3, "bytes": 300}},
{"counter": {"name": "sbx_doh_detect_v4", "packets": 5, "bytes": 500}},
]})
def test_admin_blacklist_sums_named_counters(monkeypatch):
def fake_run(cmd, **kw):
return types.SimpleNamespace(returncode=0, stdout=_fake_nft_json(), stderr="")
monkeypatch.setattr("subprocess.run", fake_run)
out = asyncio.run(api.admin_blacklist())
assert out["drops"] == 10 # blacklist 7 + quarantine 3 (NOT doh)
assert out["doh_hits"] == 5
assert out["v4_count"] == 1

View File

@ -0,0 +1,25 @@
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
# Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
"""Every named counter referenced in the blacklist spine is declared (ref #758)."""
import re
from pathlib import Path
NFT = Path(__file__).resolve().parents[1] / "nftables.d" / "secubox-blacklist.nft"
EXPECTED = {
"sbx_drop_blacklist_v4", "sbx_drop_blacklist_v6",
"sbx_drop_quarantine_v4", "sbx_drop_quarantine_v6",
"sbx_doh_detect_v4", "sbx_doh_detect_v6",
}
def _decls_and_refs(text):
decls = set(re.findall(r'counter\s+([a-z0-9_]+)\s*\{', text))
refs = set(re.findall(r'counter name "([a-z0-9_]+)"', text))
return decls, refs
def test_named_counters_declared_and_referenced():
text = NFT.read_text()
decls, refs = _decls_and_refs(text)
assert EXPECTED <= refs, f"missing refs: {EXPECTED - refs}"
assert refs <= decls, f"undeclared counters referenced: {refs - decls}"

View File

@ -0,0 +1,32 @@
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
# Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
"""network_drops comes from the hub netstats snapshot (ref #758)."""
import asyncio
import json
from secubox_toolbox import api
def test_network_drops_from_snapshot(tmp_path, monkeypatch):
snap = tmp_path / "netstats.json"
snap.write_text(json.dumps({"network_drops": 42, "updated": 9_999_999_999}))
monkeypatch.setattr(api, "NETSTATS_SNAPSHOT", snap)
monkeypatch.setattr(api.store, "ad_stats", lambda hours: {"total_blocked": 0})
async def _boom():
raise AssertionError("must not fall back to nft when snapshot present")
monkeypatch.setattr(api, "admin_blacklist", _boom)
out = asyncio.run(api.admin_ad_stats(hours=24))
assert out["network_drops"] == 42
def test_network_drops_fallback_to_blacklist(tmp_path, monkeypatch):
monkeypatch.setattr(api, "NETSTATS_SNAPSHOT", tmp_path / "missing.json")
monkeypatch.setattr(api.store, "ad_stats", lambda hours: {"total_blocked": 0})
async def _bl():
return {"drops": 7}
monkeypatch.setattr(api, "admin_blacklist", _bl)
out = asyncio.run(api.admin_ad_stats(hours=24))
assert out["network_drops"] == 7

View File

@ -0,0 +1,16 @@
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
# Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
"""The Réseau tab is wired into the toolbox dashboard (ref #758)."""
from pathlib import Path
HTML = Path(__file__).resolve().parents[1] / "www" / "toolbox" / "index.html"
def test_reseau_tab_button_panel_and_loader():
t = HTML.read_text()
assert 'data-tab="reseau"' in t
assert 'id="panel-reseau"' in t
assert "loadNetstats" in t
# talks to the hub netstats endpoints
assert "/api/v1/hub/netstats/summary" in t
assert "/api/v1/hub/netstats/series" in t

View File

@ -83,6 +83,7 @@
<button class="tab" data-tab="filtres" onclick="switchTab('filtres')">🚦 Filtres MITM</button> <button class="tab" data-tab="filtres" onclick="switchTab('filtres')">🚦 Filtres MITM</button>
<button class="tab" data-tab="social" onclick="switchTab('social')">🕸️ Cartographie sociale</button> <button class="tab" data-tab="social" onclick="switchTab('social')">🕸️ Cartographie sociale</button>
<button class="tab" data-tab="ads" onclick="switchTab('ads')">🛑 Pubs</button> <button class="tab" data-tab="ads" onclick="switchTab('ads')">🛑 Pubs</button>
<button class="tab" data-tab="reseau" onclick="switchTab('reseau')">📡 Réseau</button>
<button class="tab" data-tab="tor" onclick="switchTab('tor')">🧅 Tor</button> <button class="tab" data-tab="tor" onclick="switchTab('tor')">🧅 Tor</button>
<button class="tab" data-tab="config" onclick="switchTab('config')">⚙ Config</button> <button class="tab" data-tab="config" onclick="switchTab('config')">⚙ Config</button>
</nav> </nav>
@ -194,6 +195,17 @@
</div> </div>
</section> </section>
<!-- #758 — network-layer stats (nft named counters + interface throughput) -->
<section class="panel" id="panel-reseau">
<h2>📡 Statistiques réseau (nftables)</h2>
<div id="ns-stale" style="display:none;color:var(--cinnabar,#e63946);font-size:.85em">⚠ données périmées (collecteur arrêté ?)</div>
<div id="ns-kpi" class="kpis"></div>
<h3>Débit interfaces (24h)</h3>
<div id="ns-throughput"></div>
<h3>Drops &amp; attaques par catégorie (24h)</h3>
<div id="ns-drops"></div>
</section>
<!-- Tor egress quick-switch (#683) --> <!-- Tor egress quick-switch (#683) -->
<section class="panel" id="panel-tor"> <section class="panel" id="panel-tor">
<div class="toolbar"> <div class="toolbar">
@ -248,6 +260,7 @@ function switchTab(name) {
if (name === 'filtres') loadFilters(); if (name === 'filtres') loadFilters();
if (name === 'social') loadSocial(); if (name === 'social') loadSocial();
if (name === 'ads') loadAds(); if (name === 'ads') loadAds();
if (name === 'reseau') loadNetstats();
if (name === 'tor') loadTor(); if (name === 'tor') loadTor();
location.hash = name; location.hash = name;
} }
@ -643,13 +656,62 @@ async function loadAdsClient(mh) {
: '<div class="empty">aucune pub bloquée pour ce visiteur</div>'); : '<div class="empty">aucune pub bloquée pour ce visiteur</div>');
} }
// ── Réseau tab (#758) — hub netstats endpoints ──
async function Jhub(path) {
try {
const r = await fetch('/api/v1/hub' + path, { credentials: 'same-origin' });
if (!r.ok) return { __error: 'HTTP ' + r.status };
return await r.json();
} catch (e) { return { __error: String(e) }; }
}
function sparkline(points, color) {
// points: [[ts, value], …] → inline SVG polyline, auto-scaled.
if (!points || !points.length) return '<span style="opacity:.5"></span>';
const W = 240, H = 40, vals = points.map(p => p[1]);
const max = Math.max(1, ...vals), n = points.length;
const path = points.map((p, i) =>
`${(i / Math.max(1, n - 1) * W).toFixed(1)},${(H - p[1] / max * H).toFixed(1)}`).join(' ');
return `<svg width="${W}" height="${H}" viewBox="0 0 ${W} ${H}">`
+ `<polyline fill="none" stroke="${color}" stroke-width="1.5" points="${path}"/></svg>`;
}
function fmtBps(b) {
if (b > 1e6) return (b / 1e6).toFixed(1) + ' Mb/s';
if (b > 1e3) return (b / 1e3).toFixed(1) + ' kb/s';
return (b || 0) + ' b/s';
}
// Endpoints: /api/v1/hub/netstats/summary /api/v1/hub/netstats/series
async function loadNetstats() {
const sum = await Jhub('/netstats/summary');
const kpi = document.getElementById('ns-kpi');
document.getElementById('ns-stale').style.display = (sum && sum.stale) ? 'block' : 'none';
if (!sum || sum.__error) { kpi.innerHTML = `<span class="k">err</span><span class="v">${(sum && sum.__error) || 'no data'}</span>`; return; }
const cats = sum.categories || {};
kpi.innerHTML = `<span class="k">Drops réseau (total)</span> <span class="v">${sum.network_drops || 0}</span>`
+ Object.keys(cats).map(c => ` <span class="k">${c}</span> <span class="v">${(cats[c].packets) || 0}</span>`).join('');
const ser = await Jhub('/netstats/series?window=86400&step=300');
const tp = document.getElementById('ns-throughput');
const dr = document.getElementById('ns-drops');
if (!ser || ser.__error) { tp.textContent = 'no data'; dr.textContent = ''; return; }
const inb = ser.in_bps || {}, outb = ser.out_bps || {};
tp.innerHTML = Object.keys(inb).map(ifc =>
`<div class="row"><b>${ifc}</b> ↓ ${fmtBps((inb[ifc].slice(-1)[0] || [0, 0])[1])} ${sparkline(inb[ifc], '#00d4ff')}`
+ ` ↑ ${fmtBps(((outb[ifc] || []).slice(-1)[0] || [0, 0])[1])} ${sparkline(outb[ifc] || [], '#c9a84c')}</div>`).join('') || '<span style="opacity:.5"></span>';
const drops = ser.drops || {};
dr.innerHTML = Object.keys(drops).map(c =>
`<div class="row"><b>${c}</b> ${sparkline(drops[c], '#e63946')}</div>`).join('') || '<span style="opacity:.5"></span>';
}
async function refreshAll() { async function refreshAll() {
await Promise.all([loadCfg(), loadClients(), loadHealth(), loadMetrics()]); await Promise.all([loadCfg(), loadClients(), loadHealth(), loadMetrics()]);
} }
// Deep-link : open the tab named in the URL hash on load. // Deep-link : open the tab named in the URL hash on load.
const initial = (location.hash || '').replace('#', ''); const initial = (location.hash || '').replace('#', '');
if (['overview','clients','filtres','social','ads','config'].includes(initial)) switchTab(initial); if (['overview','clients','filtres','social','ads','reseau','config'].includes(initial)) switchTab(initial);
refreshAll(); refreshAll();
setInterval(() => { setInterval(() => {

167
wiki/Netboot-Install.md Normal file
View File

@ -0,0 +1,167 @@
# Netboot Install — MOCHAbin from gk2
Network-boot a second MOCHAbin (c3box) from gk2's netboot rig and perform a
fully automated, cryptographically verified SecuBox Debian install to eMMC.
Validated 2026-06-27 (factory U-Boot 2020.10, custom kernel 6.12.85 #5secubox,
image `secubox-mochabin-bookworm` CI run 27426515472).
---
## Prerequisites
| Item | Detail |
|------|--------|
| Cable | Straight copper RJ45 from gk2 **lan1** to c3box **mvpp2-2** (the single copper port, not the 4-port switch) |
| Signing key | `secubox-netboot.key` on gk2 (matches `netboot-image.pub` embedded in the installer FIT) |
| Image | `secubox-mochabin-bookworm` artifact from CI (latest successful build-packages + build-image run) |
| gk2 services | dnsmasq DHCP on `192.168.77.0/24`, TFTP root `/srv/tftp`, nginx `:8099` serving signed image |
| Serial console | USB-UART to c3box debug header (115200 8N1) to catch factory U-Boot |
> **Port note**: Only `mvpp2-2` (the single copper RJ45) is reachable by the factory U-Boot.
> The 4 switch ports require the MV88E6XXX DSA driver which is absent at early boot.
---
## Step 1 — Publish the signed image on gk2
```bash
# Sign the image
openssl dgst -sha256 -sign /etc/secubox/netboot/secubox-netboot.key \
-out /srv/netboot/sbx.img.gz.sig /data/sbx.img.gz
# Publish (symlink or copy into the nginx root)
ln -sf /data/sbx.img.gz /srv/netboot/sbx.img.gz
ln -sf /data/sbx.img.gz.sig /srv/netboot/sbx.img.gz.sig
# Verify the public key matches what the installer FIT embeds
openssl rsa -in /etc/secubox/netboot/secubox-netboot.key -pubout \
| diff - /etc/secubox/netboot/netboot-image.pub
```
---
## Step 2 — Catch the factory U-Boot on c3box
Power-cycle c3box. On the serial console you have **2 seconds** (`bootdelay=2`).
> **Important**: press **Enter** to interrupt autoboot — NOT Ctrl-C.
```
Hit any key to stop autoboot: 2
Marvell>>
```
---
## Step 3 — Configure TFTP environment
```
# Set TFTP block size for fast transfer
setenv tftpblocksize 1468
# Set the TFTP server (gk2 lan1 address)
setenv serverip 192.168.77.1
setenv ipaddr 192.168.77.100
# Save (optional — env is NOT in SPI mtd2 on this board, lost on power cycle)
# saveenv
```
> **U-Boot env note**: `fw_setenv` from Linux has no effect on this board — the factory
> U-Boot does not store its env in SPI mtd2 (a stale foreign env sits there). All env
> changes must be made interactively in the U-Boot shell.
---
## Step 4 — Load installer via TFTP and boot
```
# Load kernel, DTB, initrd into free RAM addresses (NOT 0x02080000 — reserved)
tftpboot 0x0a000000 Image
tftpboot 0x09000000 armada-7040-mochabin.dtb
tftpboot 0x10000000 initrd.img
# Boot
booti 0x0a000000 0x10000000 0x09000000
```
The installer rescue shell appears. The 49 MB signed FIT is fetched automatically
from `http://192.168.77.1:8099/` during installer init.
---
## Step 5 — Automated verified install from the rescue shell
```bash
# Download image to RAM (c3box has 8 GB — fits comfortably)
wget http://192.168.77.1:8099/sbx.img.gz -O /tmp/sbx.img.gz
wget http://192.168.77.1:8099/sbx.img.gz.sig -O /tmp/sbx.img.gz.sig
# Verify signature against the embedded public key
openssl dgst -sha256 -verify /etc/secubox/installer/netboot-image.pub \
-signature /tmp/sbx.img.gz.sig /tmp/sbx.img.gz
# Expected output: Verified OK
# Stream to eMMC (progress tracked via status=progress)
gunzip -c /tmp/sbx.img.gz | dd of=/dev/mmcblk0 bs=4M conv=fsync status=progress
sync
```
---
## Step 6 — Fix auto-boot persistence (boot.scr)
The image ships `extlinux.conf` pointing to `0x02080000` (reserved on factory U-Boot —
causes an immediate reset). It also ships no compiled `boot.scr`. Build one before rebooting:
```bash
# From the rescue shell, after dd completes, mount the boot partition
mount /dev/mmcblk0p1 /mnt
cat > /tmp/boot.cmd << 'EOF'
setenv bootargs "console=ttyS0,115200 earlycon root=LABEL=rootfs rw"
load mmc 0:1 0x0a000000 Image
load mmc 0:1 0x10000000 initrd.img
load mmc 0:1 0x09000000 armada-7040-mochabin.dtb
booti 0x0a000000 0x10000000:${filesize} 0x09000000
EOF
mkimage -C none -A arm64 -T script -d /tmp/boot.cmd /mnt/boot.scr
sync
umount /mnt
```
Reboot. The factory U-Boot picks up `boot.scr` from mmc and boots Debian unattended.
---
## Gotchas
| Symptom | Root cause | Fix |
|---------|-----------|-----|
| Board resets immediately after kernel load | Kernel loaded at `0x02080000` (reserved memory region) | Use `0x0a000000` for kernel, `0x10000000` for initrd |
| TFTP very slow | Default block size too small | `setenv tftpblocksize 1468` |
| `fw_setenv` from Linux has no effect | Factory U-Boot env is NOT in SPI mtd2 (stale foreign env there) | Reconfigure U-Boot env interactively each time, or use boot.scr |
| Autoboot not interrupted | Interrupt key is **Enter**, not Ctrl-C | Press Enter within 2 seconds of power-on |
| Switch ports (4-port block) unreachable at boot | MV88E6XXX DSA driver absent in factory U-Boot | Cable must go to `mvpp2-2` (single copper RJ45) |
| Image boots but SecuBox stack not starting | First boot — wait ~90 s for systemd units to settle | Check `journalctl -u secubox-aggregator` |
---
## eMMC layout after install
| Partition | Type | Mount | Purpose |
|-----------|------|-------|---------|
| mmcblk0p1 | FAT32 | `/boot` | Kernel, DTB, initrd, boot.scr, extlinux.conf |
| mmcblk0p2 | ext4 | `/` | Root filesystem |
| mmcblk0p3 | ext4 | `/data` | Persistent data |
---
## Related
- [[ARM-Installation]] — general ARM installation guide
- [[Hardware-Matrix]] — board support matrix
- `feature/748-enhanced-tow-boot-http-netboot-serial-fl` — enhanced Tow-Boot with native HTTP/wget (blocked, see #748)
- GitHub: [#748](https://github.com/CyberMind-FR/secubox-deb/issues/748) · [#737](https://github.com/CyberMind-FR/secubox-deb/issues/737)

View File

@ -27,6 +27,7 @@
* [[Live-USB]] | [FR](Live-USB-FR) | [DE](Live-USB-DE) | [中文](Live-USB-ZH) * [[Live-USB]] | [FR](Live-USB-FR) | [DE](Live-USB-DE) | [中文](Live-USB-ZH)
* [[Installation]] | [FR](Installation-FR) | [DE](Installation-DE) | [中文](Installation-ZH) * [[Installation]] | [FR](Installation-FR) | [DE](Installation-DE) | [中文](Installation-ZH)
* [[ARM-Installation]] | [FR](ARM-Installation-FR) | [DE](ARM-Installation-DE) | [中文](ARM-Installation-ZH) * [[ARM-Installation]] | [FR](ARM-Installation-FR) | [DE](ARM-Installation-DE) | [中文](ARM-Installation-ZH)
* [[Netboot-Install]] 🔧 MOCHAbin netboot + signed install
* [[ESPRESSObin]] | [FR](ESPRESSObin-FR) | [DE](ESPRESSObin-DE) | [中文](ESPRESSObin-ZH) * [[ESPRESSObin]] | [FR](ESPRESSObin-FR) | [DE](ESPRESSObin-DE) | [中文](ESPRESSObin-ZH)
* [[Eye-Remote]] 📡 * [[Eye-Remote]] 📡
* [[Android-ToolBox]] 📱 one-tap R3 * [[Android-ToolBox]] 📱 one-tap R3