Compare commits

...

6 Commits

Author SHA1 Message Date
4ef6d3aa76 docs: record #662 R3 cutover to Go engine + banner port (PR #670)
Some checks are pending
License Headers / check (push) Waiting to run
2026-06-18 19:20:21 +02:00
af76e33b45 docs: record #662 P5-prep + P6-prep (PRs #668, #669) in HISTORY 2026-06-18 19:19:38 +02:00
CyberMind
8df8f4d181
Merge pull request #670 from CyberMind-FR/feat/662-cutover-fix
feat(#662): R3 cutover to the Go MITM engine — unit fix, R3-CA loadCA, banner port
2026-06-18 19:19:23 +02:00
70d35eb7f2 feat(toolbox-ng): port real banner inject + /__toolbox portal reverse-proxy (ref #662) 2026-06-18 19:16:27 +02:00
73795bb3c3 feat(toolbox-ng): port transparency-banner loader inject + /__toolbox/* portal proxy (ref #662)
The Go MITM engine now injects the REAL visible transparency-banner loader
(replacing the invisible `<!-- sbx-ng banner -->` marker regression), mirroring
the authoritative Python inject_banner.py with stream_inject ON.

- banner.go: injectLoader() builds the guarded loader <script src="/__toolbox/
  loader.js" data-mh=.. data-wg=.. async> exactly like Python _loader_script;
  placement mirrors _LoaderInjector (after <head>'s '>', else before <body>,
  else unchanged); bannerGuard idempotency matches _GUARD; data-mh ascii-stripped.
- /__toolbox/loader.js + /__toolbox/bundle short-circuited in BOTH the CONNECT
  mitmPipeline and the transparent path, reverse-proxied to the portal
  (--portal, default http://127.0.0.1:8088). Startswith match (query-aware),
  fail-open to 204 so a banner asset never 502s the navigation.
- mitmPipeline threads `wg bool`: transparent path derives it from the
  10.99.1.0/24 peer IP (R3 WG), CONNECT passes false. Injection tightened to
  2xx text/html (Python skips non-200). injectMarker/Policy.Inject kept for the
  existing PoC tests.
- banner_test.go: guard idempotency, <head>/<body>/neither placement, wg + mh
  attributes, non-ascii stripping, path-detection + portal URL construction.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 19:12:28 +02:00
03fdc8fe14 fix(toolbox-ng): cutover-ready worker unit — live R3 CA, transparent 10.99.1.1 bind, combined-PEM loadCA (ref #662) 2026-06-18 18:54:41 +02:00
8 changed files with 509 additions and 22 deletions

View File

@ -3,6 +3,51 @@
---
## 2026-06-18 — #662 R3 CUTOVER to the Go MITM engine (PR #670) — LIVE + banner ported
- **Cutover executed and live.** The Go engine now serves **100% of R3 traffic**,
replacing the Python mitmproxy workers. Found + fixed 4 blockers that made the dark
package unable to serve the live path: (1) it forged with the wrong CA (ca-wg "WG CA"
vs the "R3 CA" clients trust) → now uses the mitmproxy confdir bundle; (2) root-only
key vs non-root user → R3 CA bundle is group-readable; (3) bound 127.0.0.1 vs the
10.99.1.1 DNAT target → now binds 10.99.1.1; (4) ran CONNECT vs transparent → now
`--transparent`. `loadCA` scans PEM blocks by type (combined cert+key bundle).
- **Validated on real arm64 hardware** then rolled out gated: localhost forge against
the real R3 CA → scoped-DNAT transparent capture → **canary slot 3 (~25%, dead-man
armed)** → **widen to 100%**. At 100%: 0 restarts, 0 errors, ~64MB total
(vs Python ~280-470MB), even round-robin, 142 distinct SNIs/75s.
- **Banner ported** (the one regression the user caught — "no more banner but fast").
Go now injects the real loader `<script src="/__toolbox/loader.js" data-mh=.. data-wg=..>`
(guard-idempotent, R3 wg flag, mac_hash identity) and reverse-proxies
`/__toolbox/loader.js`+`/__toolbox/bundle` to the portal (127.0.0.1:8088, fail-open),
keeping bundle/level logic in Python. Verified live: loader injected + assets 200.
- **Rollback** = one `nft replace` (Python workers kept warm). **Persistence gap**: the
nft flip is a live edit, not yet in the drift-managed generator → reboot safely falls
back to Python (workers enabled, banner intact). Phase 7 (decommission Python +
persist nft) deferred to a soak'd follow-up.
## 2026-06-18 — #662 MITM engine migration: P5-prep + P6-prep (PRs #668, #669, all DARK)
- **P5-prep (PR #668).** Wired the ported `Decide`+jar into the Go engine's request/
response handlers: `handleConnect` runs allow/splice/block/mitm; `anonymizeRequest`
(strip operator/re-id headers + DNT/GPC) on every MITM'd flow; cookie-poison gated
to mitm+tracker only (never allow/own-infra; fail-closed-to-clean; benign cookies +
Set-Cookie attrs preserved). New `secubox-toolbox-ng` debian pkg builds an arm64
`.deb` shipping `/usr/sbin/sbxmitm` + a **DISABLED** `worker@.service` on `:809%i`
(no enable/start, no nft). 22 Go tests, reviewed APPROVED.
- **P6-prep (PR #669).** No-traffic build-out of the live transparent path, still DARK.
`machash.go` ports `mac_hash_of`/`_wg_hash_of` (WG peers → `sha256(pubkey)[:16]`,
mtime-cached, fail-open) wired into `clientHashFromConn`, cross-engine parity vs
Python (anti-rig verified). Transparent `SO_ORIGINAL_DST` accept (`--transparent`,
default off): peeks ClientHello SNI WITHOUT decrypting → Decide → **splice = true raw
passthrough** (never `tls.Server`) / else forge via replayable `prefixConn`; upstream
TLS verifies by SNI, pins captured ip:port. Two-stage review caught + fixed a
splice-decrypt defect. Builds linux/arm64+amd64+darwin, vet clean, race green, Python
parity 10 passed. CONNECT path + poison gate byte-unchanged.
- **Engine now functionally complete + packaged, entirely DARK.** Remaining work =
the production DEPLOYMENT phases (shadow → cutover → decommission), which touch live
R3 traffic and are deferred to a deliberate watched session — NOT chained off "go".
## 2026-06-18 — #656 Ad Intelligence (PR #657, toolbox 2.6.56) + splice reverted
- **Ad Intelligence — learn/act/measure.** `ad_ghost` now records every

View File

@ -0,0 +1,167 @@
// SPDX-License-Identifier: LicenseRef-CMSD-1.0
// Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
//
// SecuBox-Deb :: toolbox-ng :: transparency-banner loader inject (#662)
//
// Ports the LIVE transparency-banner injection from the authoritative Python
// addon (../secubox-toolbox/mitmproxy_addons/inject_banner.py) into the Go
// engine. With stream_inject ON the Python addon injects a tiny LOADER
// <script src="/__toolbox/loader.js" data-mh=.. data-wg=.. async></script> and
// SERVES /__toolbox/loader.js + /__toolbox/bundle itself for ANY origin (the
// injected same-origin URL resolves to whatever MITM'd host the client is on).
//
// To avoid re-porting the bundle/level business logic to Go, this engine
// REVERSE-PROXIES /__toolbox/* to the portal (default http://127.0.0.1:8088),
// which already serves both endpoints. The injection (injectLoader) mirrors the
// Python _loader_script + _LoaderInjector byte-for-byte on the tag shape and
// placement; the guard makes it idempotent (matches Python _GUARD).
//
// Pure standard library — no external modules.
package main
import (
"bytes"
"io"
"log"
"net/http"
"strings"
"time"
)
// bannerGuard matches the Python _GUARD ("__GONDWANA_MITM_BANNER__"): an HTML
// comment marker that makes injection idempotent across stream chunks / repeat
// passes. If the body already contains it, we never inject again.
const bannerGuard = "__GONDWANA_MITM_BANNER__"
// asciiOnly drops every non-ASCII byte from s, mirroring the Python
// `s.encode("ascii", "ignore")` used on the client hash before it lands in the
// data-mh attribute. The clientHash is normally a hex mac_hash (already ASCII),
// but a non-WG fallback could carry odd bytes — strip defensively.
func asciiOnly(s string) string {
var b strings.Builder
b.Grow(len(s))
for i := 0; i < len(s); i++ {
if s[i] < 0x80 {
b.WriteByte(s[i])
}
}
return b.String()
}
// loaderScript builds the loader <script> tag EXACTLY like the Python
// _loader_script: a guard comment followed by the same-origin loader.js tag
// carrying the client identity (data-mh) + WG flag (data-wg). wg → "1" else "0";
// clientHash is ascii-sanitised. The src is same-origin so it resolves to the
// MITM'd host and is intercepted by the /__toolbox/* short-circuit.
func loaderScript(clientHash string, wg bool) []byte {
wgVal := "0"
if wg {
wgVal = "1"
}
mh := asciiOnly(clientHash)
tag := `<script src="/__toolbox/loader.js" data-mh="` + mh +
`" data-wg="` + wgVal + `" async></script>`
return []byte("<!-- " + bannerGuard + " -->" + tag)
}
// injectLoader inserts the loader <script> into an HTML body once. Placement
// mirrors the Python _LoaderInjector.__call__:
// - guard idempotency: if the body already contains bannerGuard → unchanged.
// - find the first (case-insensitive) "<head"; if present, find the next ">"
// after it and insert the tag right after that ">".
// - else find the first "<body" and insert the tag right BEFORE it.
// - if neither is present → return the body unchanged (no inject).
func injectLoader(body []byte, clientHash string, wg bool) []byte {
if bytes.Contains(body, []byte(bannerGuard)) {
return body
}
script := loaderScript(clientHash, wg)
low := bytes.ToLower(body)
if h := bytes.Index(low, []byte("<head")); h >= 0 {
if j := bytes.IndexByte(body[h:], '>'); j >= 0 {
at := h + j + 1
out := make([]byte, 0, len(body)+len(script))
out = append(out, body[:at]...)
out = append(out, script...)
out = append(out, body[at:]...)
return out
}
}
if b := bytes.Index(low, []byte("<body")); b >= 0 {
out := make([]byte, 0, len(body)+len(script))
out = append(out, body[:b]...)
out = append(out, script...)
out = append(out, body[b:]...)
return out
}
return body
}
// ── /__toolbox/* reverse-proxy to the portal ─────────────────────────────────
// isToolboxAssetPath reports whether a request path is one of the banner assets
// the engine must serve itself (by reverse-proxying to the portal) for ANY
// origin. STARTSWITH (not exact) is REQUIRED: the path includes the query
// string and the bundle is fetched as /__toolbox/bundle?mh=..&wg=.. — an exact
// match would never fire. Mirrors the Python request() p.startswith(...) checks.
func isToolboxAssetPath(path string) bool {
return strings.HasPrefix(path, "/__toolbox/loader.js") ||
strings.HasPrefix(path, "/__toolbox/bundle")
}
// portalTargetURL builds the absolute portal URL for an intercepted asset
// request: <portal-base> + the original request path (which already includes
// the query string). The portal base's trailing slash is trimmed so the result
// never doubles the leading "/" of the path.
func portalTargetURL(portal, pathWithQuery string) string {
return strings.TrimRight(portal, "/") + pathWithQuery
}
// portalClient is the short-timeout HTTP client used to fetch banner assets from
// the portal. Shared (stdlib http.Client is goroutine-safe) so we don't churn
// connections per request.
var portalClient = &http.Client{
Timeout: 5 * time.Second,
// Never follow redirects: the portal is a fixed loopback base, so not
// following 3xx means a misbehaving/compromised portal can't steer the
// worker into fetching an arbitrary outbound host (SSRF hygiene). The 3xx
// is relayed to the client as-is.
CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse },
}
// servePortalAsset reverse-proxies a /__toolbox/* request to the portal and
// writes the portal's response (status + Content-Type + Cache-Control + body)
// back to the client over the already-established (TLS) conn. It returns true
// once it has written a response — the caller MUST NOT then forward upstream.
//
// Fail-open: if the portal request errors (portal down, timeout, non-2xx read
// failure) we serve a minimal 204 No Content so the navigation is never broken,
// and log at most a warning. We never 502 the whole page over a banner asset.
func servePortalAsset(w io.Writer, portal, pathWithQuery string) bool {
target := portalTargetURL(portal, pathWithQuery)
resp, err := portalClient.Get(target)
if err != nil {
log.Printf("portal asset fetch failed for %s: %v", target, err)
writeRaw(w, 204, "No Content", nil, nil)
return true
}
defer resp.Body.Close()
body, rerr := io.ReadAll(io.LimitReader(resp.Body, 8<<20))
if rerr != nil {
log.Printf("portal asset read failed for %s: %v", target, rerr)
writeRaw(w, 204, "No Content", nil, nil)
return true
}
headers := map[string]string{}
if ct := resp.Header.Get("Content-Type"); ct != "" {
headers["Content-Type"] = ct
}
if cc := resp.Header.Get("Cache-Control"); cc != "" {
headers["Cache-Control"] = cc
}
// writeRaw formats "HTTP/1.1 <code> <status>"; pass only the reason phrase
// (not resp.Status, which already embeds the code → would double it).
writeRaw(w, resp.StatusCode, http.StatusText(resp.StatusCode), headers, body)
return true
}

View File

@ -0,0 +1,132 @@
// SPDX-License-Identifier: LicenseRef-CMSD-1.0
// Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
//
// SecuBox-Deb :: toolbox-ng :: transparency-banner loader inject tests (#662)
//
// Mirrors the authoritative Python tests of inject_banner._loader_script /
// _LoaderInjector / the /__toolbox/* request() short-circuit. The portal
// reverse-proxy integration (a live portal) is validated on-board, NOT here;
// these unit tests cover the pure injection logic + the path/url helpers.
package main
import (
"strings"
"testing"
)
func TestInjectLoaderGuardIdempotent(t *testing.T) {
// Body already carrying the guard → returned byte-for-byte unchanged.
body := []byte("<html><head><!-- " + bannerGuard + " --><script></script></head><body>hi</body></html>")
out := injectLoader(body, "abc123", false)
if string(out) != string(body) {
t.Fatalf("guarded body must be unchanged.\n got: %s", out)
}
}
func TestInjectLoaderHeadInsertion(t *testing.T) {
body := []byte(`<html><head lang="en"><title>x</title></head><body>hi</body></html>`)
out := string(injectLoader(body, "deadbeef", true))
// The tag must land right AFTER the first <head ...>'s closing '>'.
headOpen := `<head lang="en">`
idx := strings.Index(out, headOpen)
if idx < 0 {
t.Fatalf("head open lost: %s", out)
}
after := out[idx+len(headOpen):]
wantTag := `<!-- ` + bannerGuard + ` --><script src="/__toolbox/loader.js" data-mh="deadbeef" data-wg="1" async></script>`
if !strings.HasPrefix(after, wantTag) {
t.Fatalf("tag not inserted right after <head>'s '>'.\n got: %s", after)
}
// <title> must still follow the injected tag (we inserted, not replaced).
if !strings.Contains(out, wantTag+`<title>x</title>`) {
t.Fatalf("original head content displaced: %s", out)
}
}
func TestInjectLoaderBodyFallback(t *testing.T) {
// No <head> → insert right BEFORE the first <body>.
body := []byte(`<html><body class="x">hi</body></html>`)
out := string(injectLoader(body, "cafe", false))
wantTag := `<!-- ` + bannerGuard + ` --><script src="/__toolbox/loader.js" data-mh="cafe" data-wg="0" async></script>`
if !strings.Contains(out, wantTag+`<body class="x">`) {
t.Fatalf("tag not inserted right before <body>.\n got: %s", out)
}
}
func TestInjectLoaderNeitherHeadNorBody(t *testing.T) {
body := []byte(`<p>just a fragment</p>`)
out := injectLoader(body, "x", true)
if string(out) != string(body) {
t.Fatalf("no head/body → must be unchanged.\n got: %s", out)
}
}
func TestInjectLoaderWGAttr(t *testing.T) {
cases := []struct {
wg bool
want string
}{
{true, `data-wg="1"`},
{false, `data-wg="0"`},
}
for _, c := range cases {
out := string(injectLoader([]byte(`<head></head>`), "mh1", c.wg))
if !strings.Contains(out, c.want) {
t.Fatalf("wg=%v: want %q in %s", c.wg, c.want, out)
}
}
}
func TestInjectLoaderNonASCIIHashStripped(t *testing.T) {
// Non-ascii bytes in the client hash are dropped (Python .encode("ascii","ignore")).
out := string(injectLoader([]byte(`<head></head>`), "abécÿ12", false))
if !strings.Contains(out, `data-mh="abc12"`) {
t.Fatalf("non-ascii bytes not stripped: %s", out)
}
}
func TestInjectLoaderHeadCaseInsensitive(t *testing.T) {
body := []byte(`<HTML><HEAD></HEAD><BODY>hi</BODY></HTML>`)
out := string(injectLoader(body, "z", false))
if !strings.Contains(out, `<HEAD><!-- `+bannerGuard) {
t.Fatalf("case-insensitive <HEAD> match failed: %s", out)
}
}
func TestIsToolboxAssetPath(t *testing.T) {
cases := []struct {
path string
want bool
}{
{"/__toolbox/loader.js", true},
{"/__toolbox/loader.js?v=2", true},
{"/__toolbox/bundle", true},
{"/__toolbox/bundle?mh=abc&wg=1", true},
{"/__toolbox/other", false},
{"/index.html", false},
{"/", false},
{"", false},
{"/__toolboxbundle", false},
}
for _, c := range cases {
if got := isToolboxAssetPath(c.path); got != c.want {
t.Errorf("isToolboxAssetPath(%q) = %v, want %v", c.path, got, c.want)
}
}
}
func TestPortalTargetURL(t *testing.T) {
cases := []struct {
portal, path, want string
}{
{"http://127.0.0.1:8088", "/__toolbox/loader.js", "http://127.0.0.1:8088/__toolbox/loader.js"},
{"http://127.0.0.1:8088", "/__toolbox/bundle?mh=abc&wg=1", "http://127.0.0.1:8088/__toolbox/bundle?mh=abc&wg=1"},
// Trailing slash on the portal base must not double up.
{"http://127.0.0.1:8088/", "/__toolbox/loader.js", "http://127.0.0.1:8088/__toolbox/loader.js"},
}
for _, c := range cases {
if got := portalTargetURL(c.portal, c.path); got != c.want {
t.Errorf("portalTargetURL(%q,%q) = %q, want %q", c.portal, c.path, got, c.want)
}
}
}

View File

@ -61,17 +61,21 @@ func loadCA(certPath, keyPath string) (*CA, error) {
if err != nil {
return nil, fmt.Errorf("read ca key: %w", err)
}
cblk, _ := pem.Decode(cpem)
// Scan for the right block TYPE rather than assuming position: the live R3
// CA the toolbox forges with (mitmproxy confdir `mitmproxy-ca.pem`) is a
// COMBINED cert+key bundle, and --ca-key may point at it. Tolerate cert and
// key co-residing in either file, in any order.
cblk := firstPEMBlock(cpem, func(b *pem.Block) bool { return b.Type == "CERTIFICATE" })
if cblk == nil {
return nil, fmt.Errorf("ca cert: no PEM block")
return nil, fmt.Errorf("ca cert: no CERTIFICATE PEM block")
}
cert, err := x509.ParseCertificate(cblk.Bytes)
if err != nil {
return nil, fmt.Errorf("parse ca cert: %w", err)
}
kblk, _ := pem.Decode(kpem)
kblk := firstPEMBlock(kpem, func(b *pem.Block) bool { return strings.Contains(b.Type, "PRIVATE KEY") })
if kblk == nil {
return nil, fmt.Errorf("ca key: no PEM block")
return nil, fmt.Errorf("ca key: no PRIVATE KEY PEM block")
}
key, err := parseKey(kblk.Bytes)
if err != nil {
@ -80,6 +84,22 @@ func loadCA(certPath, keyPath string) (*CA, error) {
return &CA{cert: cert, key: key, cache: map[string]*tls.Certificate{}}, nil
}
// firstPEMBlock returns the first PEM block in data satisfying want, or nil.
// Used to pull a specific block (CERTIFICATE / PRIVATE KEY) out of a file that
// may hold several (e.g. mitmproxy's combined CA bundle).
func firstPEMBlock(data []byte, want func(*pem.Block) bool) *pem.Block {
for {
blk, rest := pem.Decode(data)
if blk == nil {
return nil
}
if want(blk) {
return blk
}
data = rest
}
}
func parseKey(der []byte) (crypto.Signer, error) {
if k, err := x509.ParsePKCS8PrivateKey(der); err == nil {
if s, ok := k.(crypto.Signer); ok {
@ -182,6 +202,7 @@ type Proxy struct {
jaSink func(string) // JA4 observations (logged; a sidecar in prod)
jarKey []byte // anti-track HMAC fake-identity seed (nil → poison off)
poison bool // master gate: poison tracker Set-Cookies (default on when jarKey present)
portal string // portal base URL for /__toolbox/* reverse-proxy (banner assets)
}
func (px *Proxy) serverTLSConfig() *tls.Config {
@ -238,7 +259,8 @@ func (px *Proxy) handleConnect(w http.ResponseWriter, r *http.Request) {
// Shared post-TLS pipeline. CONNECT dials upstream by the request URL host
// (req.URL.Host set inside), so dialHost is "" → mitmPipeline derives it.
px.mitmPipeline(tconn, client, host, verdict, "")
// CONNECT PoC is never an R3 WG client → wg=false.
px.mitmPipeline(tconn, client, host, verdict, "", false)
}
// mitmPipeline runs the shared post-TLS-handshake MITM logic used by BOTH the
@ -256,7 +278,9 @@ func (px *Proxy) handleConnect(w http.ResponseWriter, r *http.Request) {
// CONNECT semantics: dial by req.URL.Host (the request URL / host). Non-""
// → transparent: TCP-connect the captured original-dst while doing TLS with
// ServerName=host and verifying the cert against host (not the bare IP).
func (px *Proxy) mitmPipeline(tconn *tls.Conn, rawClient net.Conn, host, verdict, dialHost string) {
// - wg : the client is an R3 WireGuard peer (10.99.1.0/24); threaded
// into the injected loader's data-wg attribute. CONNECT path passes false.
func (px *Proxy) mitmPipeline(tconn *tls.Conn, rawClient net.Conn, host, verdict, dialHost string, wg bool) {
br := newReader(tconn)
req, err := http.ReadRequest(br)
if err != nil {
@ -266,6 +290,16 @@ func (px *Proxy) mitmPipeline(tconn *tls.Conn, rawClient net.Conn, host, verdict
if req.URL.Host == "" {
req.URL.Host = host
}
// #636/#662 — serve the banner loader + bundle for ANY origin so the injected
// <script src="/__toolbox/loader.js"> resolves (R3 clients hit arbitrary
// hosts whose origin can't serve /__toolbox/*). Short-circuit BEFORE dialing
// the real upstream by reverse-proxying to the portal. Mirrors the Python
// InjectBanner.request() startswith checks (path includes the query string).
if isToolboxAssetPath(req.URL.RequestURI()) {
servePortalAsset(tconn, px.portal, req.URL.RequestURI())
return
}
// Transparent: the upstream request must carry the SNI host (for Host header,
// SNI, and cert verification); the actual TCP dial is pinned to the captured
// original-dst by transparentTransport. We do NOT put the bare ip:port in
@ -319,8 +353,12 @@ func (px *Proxy) mitmPipeline(tconn *tls.Conn, rawClient net.Conn, host, verdict
}
body, _ := io.ReadAll(io.LimitReader(resp.Body, 8<<20))
if strings.Contains(resp.Header.Get("Content-Type"), "text/html") {
body = px.pol.injectMarker(body)
// Inject the transparency-banner loader only on 2xx text/html responses
// (mirrors the Python addon, which skips non-200). The loader's same-origin
// <script src="/__toolbox/loader.js"> is served by the short-circuit above.
if resp.StatusCode >= 200 && resp.StatusCode < 300 &&
strings.Contains(resp.Header.Get("Content-Type"), "text/html") {
body = injectLoader(body, clientHash, wg)
}
writeResponse(tconn, resp, body)
}
@ -355,6 +393,8 @@ func main() {
"poison tracking Set-Cookies on MITM'd tracker flows (needs --jar-key; never touches allow/own-infra)")
transparent := flag.Bool("transparent", false,
"transparent mode: accept nft-DNAT'd conns + recover SO_ORIGINAL_DST (live R3); default is the CONNECT proxy PoC")
portal := flag.String("portal", "http://127.0.0.1:8088",
"portal base URL; /__toolbox/loader.js + /__toolbox/bundle are reverse-proxied here (banner assets, served for any MITM'd origin)")
flag.Parse()
ca, err := loadCA(*caCert, *caKey)
if err != nil {
@ -380,6 +420,7 @@ func main() {
jaSink: func(s string) { log.Printf("ja4 %s", s) },
jarKey: jarKey,
poison: *poison,
portal: *portal,
}
if *transparent {
// Transparent R3 mode: raw accept loop, each conn carries its pre-DNAT

View File

@ -72,6 +72,65 @@ func TestForgeChainsToCA(t *testing.T) {
}
}
// TestLoadCACombinedPEM proves loadCA pulls the right blocks out of a COMBINED
// cert+key bundle — the real shape of mitmproxy's confdir `mitmproxy-ca.pem`,
// which the live R3 CA uses and the worker unit points --ca-key at. mitmproxy
// writes the PRIVATE KEY block first, then the CERTIFICATE; loadCA must scan by
// type, not position.
func TestLoadCACombinedPEM(t *testing.T) {
dir := t.TempDir()
key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
if err != nil {
t.Fatal(err)
}
tmpl := &x509.Certificate{
SerialNumber: big.NewInt(7),
Subject: pkix.Name{CommonName: "Gondwana ToolBoX R3 CA (test)"},
NotBefore: time.Now().Add(-time.Hour),
NotAfter: time.Now().Add(24 * time.Hour),
IsCA: true,
KeyUsage: x509.KeyUsageCertSign | x509.KeyUsageDigitalSignature,
BasicConstraintsValid: true,
}
der, err := x509.CreateCertificate(rand.Reader, tmpl, tmpl, key.Public(), key)
if err != nil {
t.Fatal(err)
}
kder, _ := x509.MarshalPKCS8PrivateKey(key)
keyPEM := pem.EncodeToMemory(&pem.Block{Type: "PRIVATE KEY", Bytes: kder})
certPEM := pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: der})
// mitmproxy-ca.pem layout: key THEN cert in one file.
combined := filepath.Join(dir, "mitmproxy-ca.pem")
if err := os.WriteFile(combined, append(append([]byte{}, keyPEM...), certPEM...), 0o600); err != nil {
t.Fatal(err)
}
// mitmproxy-ca-cert.pem: cert only.
certOnly := filepath.Join(dir, "mitmproxy-ca-cert.pem")
if err := os.WriteFile(certOnly, certPEM, 0o644); err != nil {
t.Fatal(err)
}
// The unit's exact arg shape: --ca-cert <cert-only> --ca-key <combined>.
ca, err := loadCA(certOnly, combined)
if err != nil {
t.Fatalf("loadCA(cert-only, combined): %v", err)
}
leaf, err := ca.forge("ads.example.com")
if err != nil {
t.Fatalf("forge: %v", err)
}
pool := x509.NewCertPool()
pool.AddCert(ca.cert)
if _, err := leaf.Leaf.Verify(x509.VerifyOptions{Roots: pool, DNSName: "ads.example.com"}); err != nil {
t.Fatalf("forged leaf does not chain to combined-PEM CA: %v", err)
}
// Belt-and-braces: the combined file works as BOTH cert and key source.
if _, err := loadCA(combined, combined); err != nil {
t.Fatalf("loadCA(combined, combined): %v", err)
}
}
// NOTE (#662 Phase 3): the old TestActionDecision drove the removed hardcoded
// Policy{AdHosts, SpliceHosts} fields. The decision surface now loads from
// disk (LoadPolicy) and mirrors the Python addons; coverage moved to

View File

@ -29,6 +29,7 @@ import (
"io"
"log"
"net"
"strings"
"syscall"
"unsafe"
)
@ -339,6 +340,13 @@ func (px *Proxy) handleTransparent(client net.Conn) {
if !ok {
return // transparent mode only accepts raw TCP conns
}
// R3 WG client? The data-wg attribute of the injected loader mirrors the
// Python _loader_script (ip.startswith("10.99.1.")) — derived from the same
// client conn peer IP that feeds clientHashFromConn.
wg := false
if peer, _, perr := net.SplitHostPort(client.RemoteAddr().String()); perr == nil {
wg = strings.HasPrefix(peer, "10.99.1.")
}
dstHost, dstPort, err := origDst(tcp)
if err != nil {
return // no original-dst (not DNAT'd) → drop; nothing safe to do
@ -386,5 +394,5 @@ func (px *Proxy) handleTransparent(client net.Conn) {
return
}
defer tconn.Close()
px.mitmPipeline(tconn, client, decisionHost, verdict, dialAddr)
px.mitmPipeline(tconn, client, decisionHost, verdict, dialAddr, wg)
}

View File

@ -1,3 +1,24 @@
secubox-toolbox-ng (0.1.2-1~bookworm1) bookworm; urgency=medium
* banner: port the real transparency-banner inject — inject the loader
<script src="/__toolbox/loader.js" data-mh=.. data-wg=..> (guard-idempotent,
R3 wg flag, mac_hash identity) and reverse-proxy /__toolbox/loader.js +
/__toolbox/bundle to the portal (127.0.0.1:8088), replacing the invisible
marker comment. Fail-open to 204. (ref #662)
-- Gerald KERMA <devel@cybermind.fr> Wed, 18 Jun 2026 19:20:00 +0000
secubox-toolbox-ng (0.1.1-1~bookworm1) bookworm; urgency=medium
* worker@ unit: forge with the LIVE R3 CA clients trust (mitmproxy confdir
bundle, group-readable) instead of the root-only ca-wg WG-CA key; bind
transparent on 10.99.1.1:809%i (the nft R3 DNAT target) instead of CONNECT
on 127.0.0.1; add wg-quick@wg-toolbox dependency. (ref #662)
* loadCA: scan PEM blocks by type so a combined cert+key bundle
(mitmproxy-ca.pem) is accepted for --ca-key. (ref #662)
-- Gerald KERMA <devel@cybermind.fr> Wed, 18 Jun 2026 19:00:00 +0000
secubox-toolbox-ng (0.1.0-1~bookworm1) bookworm; urgency=medium
* Initial packaging of the Go MITM engine migration target (#662 Phase 5-prep).

View File

@ -8,31 +8,45 @@
# (secubox-toolbox-mitm-wg-worker@{1..4}, ports 8081-8084) until the cutover is
# performed manually.
#
# Mirrors the Python worker@ fanout: each %i ∈ {1..4} listens on 127.0.0.1:809%i
# (distinct from the Python 808%i ports so both fleets can coexist during a
# side-by-side cutover validation). Enable ONLY at Phase 6:
# Mirrors the Python worker@ fanout: each %i ∈ {1..4} listens TRANSPARENT on
# 10.99.1.1:809%i — the SAME wg-toolbox interface IP the nft R3 DNAT targets
# (`iif wg-toolbox tcp dport 443/80 → 10.99.1.1:numgen inc mod 4 → 808{1..4}`),
# on 809%i ports so the Go and Python fleets coexist during a side-by-side
# canary. The engine recovers the original destination via SO_ORIGINAL_DST
# (works for this non-root user under NoNewPrivileges, same as mitmdump).
#
# systemctl enable --now secubox-toolbox-ng-worker@{1,2,3,4}.service
# # then re-point the nft DNAT at 809%i and retire the Python workers
# Forges with the LIVE R3 CA clients already trust — mitmproxy's confdir bundle
# (CN "Gondwana ToolBoX R3 CA"), group-readable by secubox-toolbox — NOT the
# root-only ca-wg key.pem (CN "WG CA"), which clients do NOT trust.
#
# Rollback: disable these, re-point DNAT at the Python 808%i workers.
# Enable ONLY at Phase 6 canary:
#
# systemctl enable --now secubox-toolbox-ng-worker@1.service # one slot first
# # canary: nft ... map { ... 3 : 8091 } (was 3:8084), watch, then widen
#
# Rollback: re-point the nft DNAT map slot back at the Python 808%i worker,
# then disable this unit.
[Unit]
Description=SecuBox ToolBoX-NG Go MITM worker %i (migration target, port 809%i)
Description=SecuBox ToolBoX-NG Go MITM worker %i (migration target, transparent 10.99.1.1:809%i)
Documentation=https://github.com/CyberMind-FR/secubox-deb/issues/662
After=network.target
After=network.target wg-quick@wg-toolbox.service
Wants=wg-quick@wg-toolbox.service
[Service]
Type=simple
User=secubox-toolbox
Group=secubox-toolbox
# Reuse the EXISTING ca-wg CA (R3 clients already trust it — no re-enroll).
# The anti-track jar key is best-effort: absent → poison stays off.
# Forge with the LIVE R3 CA the clients trust: cert = mitmproxy-ca-cert.pem,
# key = mitmproxy-ca.pem (a combined cert+key bundle — loadCA scans for the
# PRIVATE KEY block). Both are group-readable by secubox-toolbox. The anti-track
# jar key is best-effort: absent → poison stays off.
ExecStart=/usr/sbin/sbxmitm \
--listen 127.0.0.1:809%i \
--ca-cert /etc/secubox/toolbox/ca-wg/ca.pem \
--ca-key /etc/secubox/toolbox/ca-wg/key.pem \
--transparent \
--listen 10.99.1.1:809%i \
--ca-cert /etc/secubox/toolbox/ca-wg/mitmproxy-ca-cert.pem \
--ca-key /etc/secubox/toolbox/ca-wg/mitmproxy-ca.pem \
--jar-key /etc/secubox/secrets/privacy-jar.key
Restart=on-failure