Implement comprehensive multi-source catalog system with automatic fallback,
advanced version tracking, and rich update management interfaces.
## Phase 1: Backend Infrastructure (COMPLETE)
### UCI Configuration
- **New**: `/etc/config/secubox-appstore`
- 4 source types: GitHub (remote), local web (remote), USB (local), embedded (fallback)
- Priority-based fallback (1=highest, 999=embedded last resort)
- Settings: auto_sync, force_source, check_updates_on_boot, notify_updates
- Update checking with configurable intervals
### Catalog Sync Script
- **New**: `/usr/sbin/secubox-catalog-sync` (364 lines)
- Automatic multi-source fallback by priority
- Download tools: uclient-fetch, wget, curl (auto-detect)
- HTTP caching: ETag support, 304 Not Modified handling
- JSON validation before use
- Metadata tracking with jq
- Logging to syslog
- Source types: remote (HTTPS/HTTP), local (filesystem), embedded (ROM)
### CLI Enhancement
- **Modified**: `/usr/sbin/secubox-appstore`
- New commands: `sync [source]`, `check-updates [--json]`, `changelog <app> [version]`
- `get_active_catalog()`: Reads from cache or embedded
- `sync_catalog()`: Wrapper for secubox-catalog-sync
- `check_updates()`: Version comparison with opkg
- `get_changelog()`: Extracts from catalog JSON
### Metadata Structure
- **New**: `/usr/share/secubox/catalog-metadata.json.example`
- Active source tracking
- Source status (online/offline/error)
- ETag cache for HTTP sources
- Installed apps version tracking
- Update statistics
### Makefile Updates
- **Modified**: `secubox-core/Makefile`
- PKG_RELEASE: 5 → 6
- Added conffiles: `/etc/config/secubox-appstore`
- Install secubox-catalog-sync binary
- Install catalog-metadata.json.example
- Added dependency: +jq
- postinst: Create cache directories (/var/cache/secubox/catalogs, /var/lib/secubox)
## Phase 2: RPCD Backend (COMPLETE)
### New RPC Methods
- **Modified**: `/usr/libexec/rpcd/luci.secubox`
- `get_catalog_sources()`: List configured sources from UCI, status from metadata
- `set_catalog_source(source)`: Configure force_source in UCI
- `sync_catalog([source])`: Trigger catalog sync (auto-fallback or specific)
- `check_updates()`: Compare installed vs catalog versions
- `get_app_versions(app_id)`: Detailed version info (pkg, app, installed, catalog)
- `get_changelog(app_id, from, to)`: Extract changelog from catalog
- `get_widget_data(app_id)`: Widget metrics (Phase 5 prep)
All methods integrate with:
- UCI config parsing (`config_load`, `config_foreach`)
- Metadata file reading (`/var/lib/secubox/catalog-metadata.json`)
- Catalog reading (`/var/cache/secubox/catalogs/*.json` or embedded)
- opkg version checking
## Phase 3: Frontend LuCI Views (COMPLETE)
### API Module Enhancement
- **Modified**: `secubox-admin/api.js`
- New RPC declarations: 7 new methods
- Exports: `getCatalogSources`, `setCatalogSource`, `syncCatalog`,
`checkUpdates`, `getAppVersions`, `getChangelog`, `getWidgetData`
### Catalog Sources Management
- **New**: `view/secubox-admin/catalog-sources.js` (370 lines)
- Live source status display (online/offline/error)
- Priority-based ordering
- Active source indicator
- Per-source actions: Sync, Test, Set Active, Enable/Disable
- Summary stats: Total sources, active source, updates available
- Auto-refresh every 30 seconds
- Timestamp formatting (relative: "5 minutes ago", "2 days ago")
### Updates Manager
- **New**: `view/secubox-admin/updates.js` (380 lines)
- Available updates list with version comparison
- Changelog preview in update cards
- Version arrows: "0.3.0-1 → 0.4.0-2"
- Per-app actions: Update Now, View Full Changelog, Skip Version
- Batch update: "Update All" button
- Check for Updates: Sync + check flow
- Auto-refresh every 60 seconds
- No updates state: Checkmark with message
### Apps Manager Enhancement
- **Modified**: `view/secubox-admin/apps.js`
- Load update info on page load
- Update available badges (warning style)
- Version display with tooltip (installed → available)
- Visual indicators: `.has-update`, `.version-outdated` classes
- New filter: "Updates Available" / "Installed" / "Not Installed"
- Changelog button on all apps (installed or not)
- Update button for apps with available updates
- `updateApp()`: Shows changelog before update
- `viewChangelog()`: Modal with version history
- `filterByStatus()`: Filter by update/install status
### Menu Integration
- **Modified**: `menu.d/luci-app-secubox-admin.json`
- New entries:
- "Updates" (order: 25) → `/admin/secubox/admin/updates`
- "Catalog Sources" (order: 27) → `/admin/secubox/admin/catalog-sources`
- Placed between Apps Manager and App Settings
## Data Flow Architecture
```
User Action (Web UI)
↓
LuCI View (catalog-sources.js, updates.js, apps.js)
↓
API Module (api.js RPC calls)
↓
RPCD Backend (luci.secubox)
↓
CLI Scripts (secubox-appstore, secubox-catalog-sync)
↓
Data Layer
├── UCI Config (/etc/config/secubox-appstore)
├── Cache (/var/cache/secubox/catalogs/*.json)
├── Metadata (/var/lib/secubox/catalog-metadata.json)
└── Embedded (/usr/share/secubox/catalog.json)
```
## Fallback Logic
1. User triggers sync (or auto-sync)
2. secubox-catalog-sync reads UCI config
3. Sorts sources by priority (1 = GitHub, 2 = Local Web, 3 = USB, 999 = Embedded)
4. Attempts each source in order:
- GitHub HTTPS → timeout/fail → Next
- Local Web → unreachable → Next
- USB → not mounted → Next
- Embedded → Always succeeds (ROM)
5. First successful source becomes active
6. Metadata updated with status, ETag, timestamp
7. Cache written to `/var/cache/secubox/catalogs/<source>.json`
## Version Tracking
- **PKG_VERSION**: OpenWrt package version (e.g., "0.4.0")
- **PKG_RELEASE**: Build release number (e.g., "2")
- **pkg_version**: Full package string "0.4.0-2" (in catalog)
- **app_version**: Underlying app version (e.g., "0.4.0")
- **installed_version**: From `opkg list-installed`
- **catalog_version**: From active catalog JSON
- **Comparison**: Uses `opkg compare-versions` for semantic versioning
## Storage Layout
```
/etc/config/secubox-appstore # UCI configuration
/var/cache/secubox/catalogs/ # Downloaded catalogs (755/644)
├── github.json
├── local_web.json
└── usb.json
/var/lib/secubox/ # Runtime metadata (700/600)
└── catalog-metadata.json
/usr/share/secubox/catalog.json # Embedded fallback (ROM)
```
## Key Features
✅ **Multi-source support**: GitHub + Web + USB + Embedded
✅ **Automatic fallback**: Priority-based with retry logic
✅ **HTTP optimization**: ETag caching, 304 Not Modified
✅ **Version management**: PKG + App versions, changelog tracking
✅ **Update notifications**: Badges, filters, dedicated updates page
✅ **Offline capable**: USB and embedded sources work without internet
✅ **Live status**: Auto-refresh, real-time source health
✅ **User control**: Manual sync, force specific source, enable/disable sources
## Files Modified (8)
- package/secubox/secubox-core/Makefile
- package/secubox/secubox-core/root/usr/libexec/rpcd/luci.secubox
- package/secubox/secubox-core/root/usr/sbin/secubox-appstore
- package/secubox/luci-app-secubox-admin/htdocs/luci-static/resources/secubox-admin/api.js
- package/secubox/luci-app-secubox-admin/htdocs/luci-static/resources/view/secubox-admin/apps.js
- package/secubox/luci-app-secubox-admin/root/usr/share/luci/menu.d/luci-app-secubox-admin.json
## Files Created (4)
- package/secubox/secubox-core/root/etc/config/secubox-appstore
- package/secubox/secubox-core/root/usr/sbin/secubox-catalog-sync
- package/secubox/secubox-core/root/usr/share/secubox/catalog-metadata.json.example
- package/secubox/luci-app-secubox-admin/htdocs/luci-static/resources/view/secubox-admin/catalog-sources.js
- package/secubox/luci-app-secubox-admin/htdocs/luci-static/resources/view/secubox-admin/updates.js
## Next Steps (Phase 4-5)
- Phase 4: Enrich catalog.json with changelog sections
- Phase 5: Widget system (renderer + templates for security/network/monitoring)
- Phase 6: Auto-sync service with cron
- Phase 7: Optimizations (signature validation, compression, CDN)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
||
|---|---|---|
| .. | ||
| root | ||
| IMPLEMENTATION.md | ||
| Makefile | ||
| README.md | ||
SecuBox Core Framework
Version: 0.8.0 License: GPL-2.0 Category: Administration
Overview
SecuBox Core is the foundational framework for the modular SecuBox system. It provides a unified infrastructure for managing OpenWrt-based security appliances through a plugin-based architecture.
Features
Core Capabilities
- Modular AppStore: Plugin-based module discovery, installation, and management
- Profile System: Declarative configuration profiles, templates, and macros
- Unified CLI: Single
secuboxcommand for all operations - Health Monitoring: Comprehensive diagnostics and health checks
- Recovery System: Automatic snapshots, rollback, and disaster recovery
- ubus Integration: Full RPC API for LuCI and third-party integration
Architecture
secubox-core
├── Core Services
│ ├── secubox-core daemon (procd)
│ ├── ubus RPC interface
│ └── Health monitoring
│
├── Module Management
│ ├── AppStore catalog
│ ├── Module discovery
│ ├── Dependency resolution
│ └── Lifecycle hooks
│
├── Configuration
│ ├── Profile engine
│ ├── Template rendering
│ └── Macro execution
│
└── Operations
├── Diagnostics
├── Snapshot/recovery
└── Verification
Installation
From Package
opkg update
opkg install secubox-core
From Source
# In OpenWrt buildroot
make package/secubox/secubox-core/compile
make package/secubox/secubox-core/install
Quick Start
1. Check System Status
secubox device status
Output:
Version: 0.8.0
Uptime: 1 day, 3:42
CPU Load: 0.45
Memory: 45%
Storage: 12%
WAN: 192.0.2.1 (eth0)
LAN: 192.168.1.1
2. Browse Available Modules
secubox app list
3. Install a Module
secubox app install wireguard-vpn
4. Run Health Check
secubox diag health
CLI Reference
Main Commands
secubox <command> [subcommand] [options]
| Command | Description |
|---|---|
app |
Manage modules and AppStore |
profile |
Manage profiles and templates |
device |
Device information and management |
net |
Network management |
diag |
Diagnostics and health checks |
ai |
AI copilot (optional, experimental) |
App Commands
secubox app list # List all modules
secubox app search <query> # Search for modules
secubox app info <module> # Show module details
secubox app install <module> # Install a module
secubox app remove <module> # Remove a module
secubox app update [module] # Update module(s)
secubox app health # Check module health
Profile Commands
secubox profile list # List available profiles
secubox profile show <profile> # Show profile details
secubox profile apply <profile> # Apply a profile
secubox profile validate <profile> # Validate profile syntax
secubox profile export [file] # Export current config
Device Commands
secubox device info # Show device information
secubox device status # Show system status
secubox device reboot # Reboot device
secubox device factory-reset # Factory reset
secubox device backup [file] # Backup configuration
Diagnostics Commands
secubox diag health # Run health check
secubox diag logs [service] # View system logs
secubox diag trace <target> # Network trace
secubox diag report # Generate diagnostic report
Configuration
UCI Configuration
File: /etc/config/secubox
config core 'main'
option enabled '1'
option log_level 'info'
option appstore_url 'https://repo.secubox.org/catalog'
option health_check_interval '300'
option ai_enabled '0'
config security 'enforcement'
option sandboxing '1'
option module_signature_check '0'
option auto_update_check '1'
config diagnostics 'settings'
option health_threshold_cpu '80'
option health_threshold_memory '90'
option health_threshold_storage '85'
Directories
| Path | Purpose |
|---|---|
/etc/config/secubox |
UCI configuration |
/etc/secubox/profiles/ |
Profile definitions |
/etc/secubox/templates/ |
Configuration templates |
/etc/secubox/macros/ |
Reusable macros |
/usr/share/secubox/plugins/catalog/ |
Module catalog |
/usr/share/secubox/modules/ |
Module metadata |
/var/run/secubox/ |
Runtime state |
/var/log/secubox/ |
Log files |
/overlay/secubox-backups/ |
Configuration snapshots |
Module System
Module Catalog
Modules are discovered through catalog entries in JSON format:
Location: /usr/share/secubox/plugins/catalog/<module-id>.json
Example:
{
"id": "wireguard-vpn",
"name": "WireGuard VPN Manager",
"version": "1.0.0",
"category": "networking",
"runtime": "native",
"packages": {
"required": ["luci-app-wireguard-vpn", "wireguard-tools"]
},
"capabilities": ["vpn-server", "vpn-client"],
"requirements": {
"min_ram_mb": 64,
"min_storage_mb": 10
}
}
Module Lifecycle
- Discovery: Catalog scanned for available modules
- Validation: Manifest and dependencies checked
- Pre-install: Pre-install hooks executed
- Installation: opkg packages installed
- Post-install: Post-install configuration
- Health Check: Module health verified
Hooks
Modules can define lifecycle hooks:
pre_install: Run before installationpost_install: Run after installationpre_remove: Run before removalpost_remove: Run after removal
Profile System
Profile Structure
Profiles are declarative YAML/JSON configurations:
profile:
id: home-office
name: "Home Office Network"
modules:
required:
- wireguard-vpn
- dns-filter
- bandwidth-manager
uci_overrides:
network:
lan:
ipaddr: "192.168.10.1"
netmask: "255.255.255.0"
Applying Profiles
# Dry-run first
secubox profile apply home-office --dryrun
# Apply profile
secubox profile apply home-office
Recovery and Snapshots
Automatic Snapshots
Snapshots are automatically created:
- Before profile application
- Before module installation
- On first boot
Manual Snapshots
# Create snapshot
secubox-recovery snapshot "my-snapshot"
# List snapshots
secubox-recovery list
# Restore from snapshot
secubox-recovery restore my-snapshot
Recovery Mode
secubox-recovery enter
ubus API
Available Objects
ubus list luci.secubox
Objects:
luci.secubox- Core operationsluci.secubox.appstore- Module management (legacy, use luci.secubox)luci.secubox.profile- Profile management (legacy, use luci.secubox)luci.secubox.diagnostics- Health checks (legacy, use luci.secubox)
Example Usage
# Get system status
ubus call luci.secubox getStatus
# List modules
ubus call luci.secubox getModules
# Install module
ubus call luci.secubox installModule '{"module":"wireguard-vpn"}'
# Run diagnostics
ubus call luci.secubox runDiagnostics '{"target":"all"}'
Health Monitoring
Health Checks
The system monitors:
- CPU load
- Memory usage
- Storage capacity
- Network connectivity
- Module status
- Service health
Thresholds
Configure in /etc/config/secubox:
config diagnostics 'settings'
option health_threshold_cpu '80'
option health_threshold_memory '90'
option health_threshold_storage '85'
Automated Checks
Health checks run automatically every 5 minutes (configurable):
uci set secubox.main.health_check_interval='300'
uci commit secubox
Security
Module Verification
Enable signature verification:
uci set secubox.enforcement.module_signature_check='1'
uci commit secubox
Sandboxing
Modules run with resource limits (when supported by kernel):
procd_set_param cgroup.memory.limit_in_bytes 134217728 # 128 MB
ACL Integration
All ubus methods are protected by LuCI ACL system.
Troubleshooting
Check Service Status
/etc/init.d/secubox-core status
View Logs
logread | grep secubox
or
tail -f /var/log/secubox/core.log
Restart Service
/etc/init.d/secubox-core restart
Reset to Defaults
uci revert secubox
/etc/init.d/secubox-core restart
Recovery
If system becomes unresponsive:
secubox-recovery enter
Development
Adding New Modules
-
Create module catalog entry:
/usr/share/secubox/plugins/catalog/my-module.json -
Define manifest with required fields:
id,name,versioncategory,runtimepackages,capabilities
-
(Optional) Add lifecycle hooks
-
Test installation:
secubox app install my-module
Custom Profiles
-
Create profile YAML/JSON in
/etc/secubox/profiles/ -
Validate:
secubox profile validate my-profile -
Test with dry-run:
secubox profile apply my-profile --dryrun -
Apply:
secubox profile apply my-profile
Dependencies
Required:
libuboxlibubuslibucirpcdbashcoreutils-base64jsonfilter
Optional:
python3(for YAML profile support)signify-openbsdoropenssl(for signature verification)
Files
Executables
/usr/sbin/secubox- Main CLI entrypoint/usr/sbin/secubox-core- Core daemon/usr/sbin/secubox-appstore- AppStore manager/usr/sbin/secubox-profile- Profile engine/usr/sbin/secubox-diagnostics- Diagnostics system/usr/sbin/secubox-recovery- Recovery tools/usr/sbin/secubox-verify- Verification tools
RPCD Scripts
/usr/libexec/rpcd/luci.secubox- Main ubus interface
Init Scripts
/etc/init.d/secubox-core- procd service/etc/uci-defaults/99-secubox-firstboot- First-boot provisioning
License
GPL-2.0
Support
- Documentation: https://docs.secubox.org
- Issues: https://github.com/gkerma/secubox-openwrt/issues
- Community: https://forum.secubox.org
Version History
0.8.0 (Current)
- Initial framework implementation
- Core module system
- Profile engine
- Health monitoring
- Recovery system
- CLI interface
- ubus API
Roadmap
0.9.0
- LuCI WebUI integration
- Enhanced profile templating
- Remote catalog support
- AI copilot integration
1.0.0
- Production-ready release
- Complete module ecosystem
- Advanced security features
- Performance optimizations