Commit Graph

21 Commits

Author SHA1 Message Date
ddf480e6ed fix(droplet,dpi): Resolve publish hang and broken pipe errors
- dropletctl: Remove pipe to grep that blocked on background children
- metablogizerctl: Background HAProxy generate/reload (~90s with 95 certs)
- dpi-lan-collector: Pre-compute flow counts in single pass instead of
  spawning grep per client (eliminates broken pipe errors)

Publish time reduced from ~2 min to ~35 seconds.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-17 07:39:09 +01:00
e17c73e343 feat(nfo): Add Module Manifest system with batch generation
Introduce flat-file UCI-style NFO manifest format for Streamlit apps
and MetaBlog sites. Enables AI context integration, search indexing,
and mesh publishing metadata.

New features:
- NFO parser library with parse/validate/update/export functions
- NFO validator with type-specific schema validation (streamlit/metablog)
- Batch generation: slforge nfo init-all, metablogizerctl nfo init-all
- RPCD methods: nfo_read, nfo_write, nfo_validate
- Reusable LuCI NFO viewer component with collapsible sections
- LuCI editor modal in Streamlit Forge overview
- Hub generator enhanced with NFO metadata (descriptions, capabilities)
- Metacatalog search with --category and --capability filters

New files:
- nfo-parser.sh, nfo-validator.sh (shell libraries)
- nfo-viewer.js (LuCI component)
- NFO-SPEC.md (specification)
- install.sh (universal NFO-based installer)
- nfo-template.nfo (templates for streamlit/metablog)

Deployed and tested: 136 NFO files generated (107 MetaBlogs, 29 Streamlit)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-14 08:51:09 +01:00
f8367fc0a3 feat(metablogizer): Add duplicate port detection and auto-fix
- Fix get_next_port() to check both uhttpd and metablogizer configs
- Add check-ports command to scan for duplicate port assignments
- Add fix-ports command to auto-assign new ports to duplicates
- Update WIP.md with 2026-03-10 changes

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-10 18:41:31 +01:00
2bb40d9419 fix(users,routing): Add gitea/jellyfin support and fix mitmproxy routes
secubox-users:
- Add gitea and jellyfin to supported services list
- Add create/update/delete handlers for gitea (via API) and jellyfin
- Update CLI help and status display to include new services

luci-app-secubox-users:
- Add jellyfin service checkbox and badge in frontend
- Update RPCD handler to check jellyfin service status

mitmproxy routing fix:
- nextcloudctl: Use host LAN IP instead of 127.0.0.1 for WAF routes
  (mitmproxy runs in container, can't reach host's localhost)
- metablogizerctl: Same fix for mitmproxy route registration
- mitmproxyctl: Fix sync_metablogizer_routes to use host IP

This fixes 502/403 errors when accessing services through HAProxy->mitmproxy
because the mitmproxy container couldn't route to 127.0.0.1 on the host.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-04 10:16:07 +01:00
684673d714 fix(emancipate): Direct mitmproxy route registration on emancipation
Previously, emancipation relied on secubox-route or mitmproxyctl sync-routes
which didn't reliably add routes to haproxy-routes.json. This caused newly
emancipated services to return 404 from mitmproxy.

Changes:
- streamlitctl: Direct JSON write as primary method for route registration
- metablogizerctl: Direct JSON write as primary method
- peertubectl: Direct JSON write as primary method
- pinaforectl: Direct JSON write + route through mitmproxy_inspector for WAF

All emancipation flows now directly write to /srv/mitmproxy-in/haproxy-routes.json
using Python, with secubox-route and mitmproxyctl as fallbacks.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-03 16:38:30 +01:00
19632e38e0 feat(routing): Centralize mitmproxy route management in secubox-core
Add centralized route registry (`secubox-route`) in secubox-core to eliminate
route management duplication across metablogizerctl, streamlitctl, and
mitmproxyctl.

New features:
- `/etc/config/secubox-routes` - UCI config for central route registry
- `/usr/sbin/secubox-route` - CLI for route management (add, remove, sync)
- Import routes from HAProxy, MetaBlogizer, Streamlit with source tracking
- Auto-sync to all mitmproxy instances on route changes
- Skip wildcard domains and LuCI (port 8081) routes

Updated services to use centralized registry:
- metablogizerctl: Use secubox-route add instead of mitmproxyctl sync
- streamlitctl: Use secubox-route add with domain/port params
- peertubectl: Use secubox-route add for emancipation
- vhost-manager/mitmproxy.sh: Prefer secubox-route when available
- mitmproxyctl: Delegate to secubox-route import-all for sync-routes

This prevents route mixups between services and provides a single
source of truth for all WAF routing configuration.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-28 08:56:04 +01:00
3b32d631cb fix(metablogizer): Auto-republish and Gitea push on file upload
- Add auto-republish to method_upload_file (was only in upload_finalize)
- Add Gitea push to cmd_publish when gitea.enabled=1
- Use haproxyctl reload instead of init.d (container-aware)
- Uploaded content now triggers full republish flow for emancipated sites

This fixes the issue where uploading new HTML content didn't update
the live site because republish wasn't triggered.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-25 12:08:58 +01:00
d267474ba3 fix(publish): Ensure uhttpd instances created on publish/emancipate
Streamlit RPCD:
- Fix backend address: 127.0.0.1 -> 192.168.255.1 (host network)
- Remove waf_bypass=1 (all traffic through mitmproxy)
- Add mitmproxy sync-routes call
- Use wildcard cert for *.gk2.secubox.in domains
- Restart HAProxy instead of just reload

MetaBlogizer:
- Add uhttpd instance creation check in cmd_publish()
- Add uhttpd instance creation check in _emancipate_haproxy()
- Sites now auto-start on publish/emancipate

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-25 09:46:51 +01:00
9eaa16171d fix(mitmproxy): Fix wildcard route priority matching
- Support both "*.domain" and ".domain" wildcard formats in haproxy_router.py
- Sort wildcards by length (longest first) for correct specificity matching
- Add auto-reload: check routes file mtime every 10 requests
- Update metablogizerctl to use mitmproxyctl sync-routes

Also fix luci-app-wazuh api.js to use baseclass.extend

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-14 16:03:43 +01:00
f38df2b319 feat(metablogizer): Enhance emancipate with WAF and path ACL integration
- Add _emancipate_mitmproxy() to register domain in WAF routes
- Add _emancipate_path_acl() to create secubox.in/gk2/{name} path routing
- Auto-detect wildcard SSL coverage for *.gk2.secubox.in domains
- Restart mitmproxy-in container after adding routes
- Update help text with 7-step workflow

Emancipate now handles full deployment:
1. DNS A record (Gandi/OVH)
2. Vortex DNS mesh publication
3. HAProxy vhost + backend
4. WAF/mitmproxy integration
5. Path ACL (secubox.in/gk2/{name})
6. SSL certificate (or wildcard)
7. Zero-downtime reload

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-14 13:21:27 +01:00
9ad45d6b1d feat(haproxy,metablogizer,streamlit): Add GK2 Hub landing page regeneration hooks
Add hooks to auto-regenerate the GK2 Hub landing page when services change:
- haproxyctl: vhost add/remove, reload
- metablogizerctl: publish, delete, emancipate
- streamlitctl: instance add/remove, app delete, emancipate

The gk2hub-generate script dynamically builds the landing page from
HAProxy vhosts, Streamlit instances, and MetaBlogizer sites.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-12 14:35:51 +01:00
54113d8454 feat(gitea): Create repositories as private by default
Changed default visibility from public to private for new Gitea
repositories created by metablogizerctl and streamlitctl.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-10 08:22:33 +01:00
7c1c3993e0 feat(gitea): Auto-push to Gitea after file uploads
- Add automatic Gitea push after upload_finalize in Streamlit RPCD
- Add automatic Gitea push after upload_finalize in MetaBlogizer RPCD
- Fix MetaBlogizer to use site name instead of UCI section ID for push
- Fix metablogizerctl to read Gitea config from dedicated gitea section

Uploaded files via LuCI are now automatically synced to Gitea repos.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-10 07:13:47 +01:00
c1c91f1227 feat: Auto-Gitea integration for apps and sites
Streamlit:
- App create/deploy now auto-pushes to Gitea when enabled
- Add 'gitea init-all' command to initialize repos for all existing apps
- Scans all app directories and creates Gitea repos

MetaBlogizer:
- Site create now auto-pushes to Gitea when token configured
- Add 'gitea init-all' command to initialize repos for all existing sites
- Iterates over UCI site configs and syncs to Gitea

Usage:
  # Configure Gitea once
  uci set streamlit.gitea.enabled=1
  uci set streamlit.gitea.url='http://192.168.255.1:3000'
  uci set streamlit.gitea.user='admin'
  uci set streamlit.gitea.token='<token>'
  uci commit streamlit

  # Initialize all existing apps/sites
  streamlitctl gitea init-all
  metablogizerctl gitea init-all

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-10 06:47:09 +01:00
364f19d421 feat: Add Gitea auto-push and fix Tor Shield server mode
Streamlit/MetaBlogizer:
- Add 'gitea push <name>' command to both streamlitctl and metablogizerctl
- Auto-creates Gitea repo via API if it doesn't exist
- Initializes git, commits all files, and pushes to Gitea
- Stores repo reference in UCI for future syncs

Tor Shield:
- Add 'wan_input_allow' option for server preset
- Server mode now properly allows WAN inbound (ports 80, 443, 8443)
- Uses nftables rules to integrate with OpenWrt firewall4
- Outbound traffic still routed through Tor (kill_switch)
- Cleanup nftables rules on stop/disable

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-10 06:42:50 +01:00
e3784537c3 fix(emancipate): Multi-zone DNS and hyphenated site names
- Add -z/--zone option to dnsctl for zone override
- Detect correct DNS zone from domain suffix (secubox.in, maegia.tv, cybermind.fr)
- Register on both published domain zone AND vortex node subdomain
- Fix hyphenated site names (e.g., bazi-weekly) in UCI lookups

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-06 09:20:39 +01:00
df34698acb feat(metablogizer): Add LuCI Emancipate button with async workflow
- Add Emancipate button to dashboard sites table
- Implement async RPC with job polling to avoid XHR timeout
- Add emancipate + emancipate_status RPCD methods
- Add ACL permissions for new RPC methods
- Change HAProxy reload to restart for clean state
- Document RPCD ACL requirements in CLAUDE.md

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-06 08:52:53 +01:00
e21ca8a060 feat(metablogizer): Add KISS ULTIME MODE emancipate workflow
Add `metablogizerctl emancipate <name>` command for one-command full
exposure workflow:
1. DNS A record via dnsctl (Gandi/OVH based on availability)
2. Vortex DNS mesh publication
3. HAProxy vhost with SSL/ACME enabled
4. SSL certificate request (webroot mode)
5. Zero-downtime HAProxy reload

Usage:
  metablogizerctl create myblog blog.example.com
  metablogizerctl emancipate myblog

Bump version to 1.1.0.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-06 08:30:58 +01:00
304ac7b9a1 feat: P2P App Store, Remote Access & Mesh Media packages
P2P App Store Emancipation:
- secubox-p2p: Package distribution via mesh peers (CGI API, RPCD, CLI)
- packages.js: LuCI view with LOCAL/PEER badges, fetch/install actions
- devstatus.js: Dev Status widget with Gitea commits, v1.0 progress tracking
- secubox-feed: sync-content command for auto-installing content packages
- ACL fix for P2P feed RPCD methods

Remote Access:
- secubox-app-rustdesk: Native hbbs/hbbr relay server from GitHub releases
- secubox-app-guacamole: LXC Debian container with guacd + Tomcat (partial)

Content Distribution:
- secubox-content-pkg: Auto-package Metablogizer/Streamlit as IPKs
- Auto-publish hooks in metablogizerctl and streamlitctl

Mesh Media:
- secubox-app-ksmbd: In-kernel SMB3 server with ksmbdctl CLI
- Pre-configured shares for Jellyfin, Lyrion, Backup

UI Consistency:
- client-guardian: Ported to sh-page-header chip layout
- auth-guardian: Ported to sh-page-header chip layout

Fixes:
- services.js: RPC expect unwrapping bug fix
- metablogizer: Chunked upload for uhttpd 64KB limit

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-05 00:33:53 +01:00
888ad50f79 2026-01-27 15:56:27 +01:00
fa5d573755 feat(multi): New LuCI apps, MetaBlogizer dual-runtime, service watchdog
- Add luci-app-lyrion: Music server dashboard
- Add luci-app-mailinabox: Email server management
- Add luci-app-nextcloud: Cloud storage dashboard
- Add luci-app-mitmproxy: Security proxy in security section
- Add luci-app-magicmirror2: Smart display dashboard
- Add secubox-app-metablogizer: CLI tool with uhttpd/nginx support
- Update luci-app-metablogizer: Runtime selection, QR codes, social share
- Update secubox-core v0.8.1: Service watchdog (auto-restart crashed services)
- Update haproxyctl: Hostname validation to prevent config errors
- Fix portal.js app discovery

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 15:23:53 +01:00