diff --git a/VALIDATION-GUIDE.md b/VALIDATION-GUIDE.md new file mode 100644 index 00000000..fb8875fd --- /dev/null +++ b/VALIDATION-GUIDE.md @@ -0,0 +1,495 @@ +# SecuBox Module Validation Guide + +This guide explains the validation checks performed on SecuBox modules during generation and before git push. + +## Overview + +SecuBox uses a multi-layered validation approach: + +1. **Module Generation Validation** - Validates newly created/modified modules +2. **Pre-Push Validation** - Blocks git push if critical issues are found +3. **Runtime Validation** - Continuous checks on deployed modules + +## Validation Tools + +### 1. validate-module-generation.sh + +Comprehensive validation for a single module during/after generation. + +**Usage:** +```bash +./secubox-tools/validate-module-generation.sh luci-app-cdn-cache +``` + +**Checks performed:** +- ✅ Makefile completeness and correctness +- ✅ RPCD script naming convention (must use `luci.` prefix) +- ✅ RPCD script executable permissions +- ✅ RPCD script structure (list/call handlers) +- ✅ ACL file JSON validity +- ✅ ACL permissions cover RPCD methods +- ✅ Menu file JSON validity +- ✅ Menu paths match view file locations +- ✅ JavaScript view files exist +- ✅ JavaScript strict mode usage +- ✅ RPC method calls match RPCD methods +- ✅ ubus object names match RPCD script names +- ✅ UCI config validity (if present) +- ✅ Security scan (hardcoded credentials, dangerous commands) +- ✅ Documentation presence + +**Exit codes:** +- `0` - All checks passed +- `1` - Critical errors found (module should not be deployed) + +### 2. pre-push-validation.sh + +Validates all modules before allowing git push. + +**Usage:** +```bash +# Automatic (via git hook): +git push # validation runs automatically + +# Manual: +./secubox-tools/pre-push-validation.sh +``` + +**Checks performed:** +- ✅ Git staged changes check +- ✅ RPCD naming conventions across all modules +- ✅ Menu path validation across all modules +- ✅ JSON syntax validation +- ✅ RPCD executable permissions +- ✅ ACL method coverage +- ✅ Makefile validation +- ✅ Security scans +- ✅ Full module validation on modified modules + +**Exit codes:** +- `0` - Push allowed +- `1` - Push blocked (critical errors) + +### 3. validate-modules.sh + +Fast validation of all modules (existing tool). + +**Usage:** +```bash +./secubox-tools/validate-modules.sh +``` + +See `secubox-tools/README.md` for details. + +## Installing Git Hooks + +To enable automatic validation before git push: + +```bash +./secubox-tools/install-git-hooks.sh +``` + +This creates a symbolic link from `.git/hooks/pre-push` to `secubox-tools/pre-push-validation.sh`. + +## Critical Naming Conventions + +### 1. RPCD Script MUST Match ubus Object + +**Rule:** The RPCD script filename MUST exactly match the ubus object name declared in JavaScript. + +**Why:** LuCI's RPC system looks for RPCD scripts by their filename. If the name doesn't match, you get: +- `RPC call failed with error -32000: Object not found` +- `Command failed: Method not found` + +**Example:** + +```javascript +// JavaScript (htdocs/luci-static/resources/view/cdn-cache/overview.js) +var callStatus = rpc.declare({ + object: 'luci.cdn-cache', // ← This must match RPCD filename + method: 'status' +}); +``` + +```bash +# RPCD script filename MUST be: +root/usr/libexec/rpcd/luci.cdn-cache # ← Exactly 'luci.cdn-cache' +``` + +**Common mistakes:** +- ❌ `root/usr/libexec/rpcd/cdn-cache` (missing `luci.` prefix) +- ❌ `root/usr/libexec/rpcd/luci-cdn-cache` (using dash instead of dot) +- ❌ `root/usr/libexec/rpcd/cdn_cache` (using underscore) + +**Validation:** +```bash +# Check naming: +./secubox-tools/validate-module-generation.sh luci-app-cdn-cache + +# Look for: +# ✓ RPCD script follows naming convention (luci.* prefix) +# ✓ CRITICAL: RPCD script name matches ACL ubus object +``` + +### 2. Menu Paths MUST Match View File Locations + +**Rule:** Menu JSON `path` entries MUST correspond to actual view file paths. + +**Why:** LuCI loads views based on the path in the menu. Wrong path = HTTP 404. + +**Example:** + +```json +// Menu (root/usr/share/luci/menu.d/luci-app-netifyd-dashboard.json) +{ + "action": { + "type": "view", + "path": "netifyd-dashboard/overview" // ← Must match file location + } +} +``` + +```bash +# View file MUST exist at: +htdocs/luci-static/resources/view/netifyd-dashboard/overview.js +# ↑ Same path as menu ↑ +``` + +**Common mistakes:** +- ❌ Menu: `"path": "netifyd/overview"` but file at `view/netifyd-dashboard/overview.js` +- ❌ Menu: `"path": "overview"` but file at `view/netifyd-dashboard/overview.js` + +**Validation:** +```bash +# Check paths: +./secubox-tools/validate-module-generation.sh luci-app-netifyd-dashboard + +# Look for: +# ✓ Menu path 'netifyd-dashboard/overview' → view file EXISTS +``` + +### 3. All ubus Objects MUST Use `luci.` Prefix + +**Rule:** Every ubus object declaration must start with `luci.` + +**Why:** Consistent naming convention for LuCI applications. ACL system expects it. + +**Example:** + +```javascript +// ✅ Correct: +object: 'luci.cdn-cache' +object: 'luci.system-hub' +object: 'luci.wireguard-dashboard' + +// ❌ Wrong: +object: 'cdn-cache' // Missing luci. prefix +object: 'systemhub' // Missing luci. prefix +``` + +**Validation:** +```bash +# Check convention: +./secubox-tools/validate-modules.sh + +# Look for: +# ✓ ubus object 'luci.cdn-cache' follows naming convention +``` + +## Module Generation Checklist + +Use this checklist when generating a new module: + +### Phase 1: Initial Generation + +- [ ] Create module directory: `luci-app-/` +- [ ] Generate Makefile with all required fields +- [ ] Create RPCD script at `root/usr/libexec/rpcd/luci.` +- [ ] Make RPCD script executable: `chmod +x` +- [ ] Add shebang to RPCD: `#!/bin/sh` +- [ ] Implement RPCD methods (list, call, status, etc.) +- [ ] Create ACL file with read/write permissions +- [ ] Create menu JSON with correct paths +- [ ] Create view JavaScript files +- [ ] Add `'use strict';` to all JS files + +### Phase 2: Validation + +- [ ] Run module generation validation: + ```bash + ./secubox-tools/validate-module-generation.sh luci-app- + ``` + +- [ ] Fix all ERRORS (critical) +- [ ] Review all WARNINGS (recommended) + +### Phase 3: Integration Validation + +- [ ] Verify RPCD script name matches ubus object: + ```bash + grep -r "object:" luci-app-/htdocs --include="*.js" + ls -la luci-app-/root/usr/libexec/rpcd/ + # Names must match exactly + ``` + +- [ ] Verify menu paths match view files: + ```bash + grep '"path":' luci-app-/root/usr/share/luci/menu.d/*.json + ls -R luci-app-/htdocs/luci-static/resources/view/ + # Paths must align + ``` + +- [ ] Verify ACL permissions cover all RPCD methods: + ```bash + grep 'case "$2"' luci-app-/root/usr/libexec/rpcd/* + grep -A 20 '"ubus":' luci-app-/root/usr/share/rpcd/acl.d/*.json + # All methods should be in ACL + ``` + +### Phase 4: Pre-Commit + +- [ ] Run comprehensive validation: + ```bash + ./secubox-tools/validate-modules.sh + ``` + +- [ ] Review security scan results +- [ ] Check JSON validity: + ```bash + find luci-app- -name "*.json" -exec python3 -m json.tool {} \; > /dev/null + ``` + +- [ ] Optional: Run shellcheck on RPCD: + ```bash + shellcheck luci-app-/root/usr/libexec/rpcd/* + ``` + +### Phase 5: Git Commit + +- [ ] Stage changes: + ```bash + git add luci-app- + ``` + +- [ ] Commit with descriptive message: + ```bash + git commit -m "feat: implement module + + - Add RPCD backend with methods: status, get_*, set_* + - Create views for overview, settings, etc. + - Configure ACL permissions + - Add menu entries + + 🤖 Generated with [Claude Code](https://claude.com/claude-code) + + Co-Authored-By: Claude Sonnet 4.5 " + ``` + +- [ ] Push (validation runs automatically): + ```bash + git push + ``` + +## Common Validation Errors and Fixes + +### Error: RPCD script name doesn't match ubus object + +``` +✗ ERROR: luci-app-cdn-cache: RPCD script 'cdn-cache' does NOT match ubus object 'luci.cdn-cache' +``` + +**Fix:** +```bash +cd luci-app-cdn-cache/root/usr/libexec/rpcd +mv cdn-cache luci.cdn-cache +``` + +### Error: Menu path → file NOT FOUND + +``` +✗ ERROR: luci-app-netifyd: Menu path 'netifyd/overview' → file NOT FOUND +Expected: htdocs/luci-static/resources/view/netifyd/overview.js +``` + +**Fix Option 1:** Update menu path to match file: +```bash +# Edit root/usr/share/luci/menu.d/luci-app-netifyd-dashboard.json +# Change: "path": "netifyd/overview" +# To: "path": "netifyd-dashboard/overview" +``` + +**Fix Option 2:** Move view file to match menu: +```bash +mv htdocs/luci-static/resources/view/netifyd-dashboard \ + htdocs/luci-static/resources/view/netifyd +``` + +### Error: RPCD script is NOT executable + +``` +✗ ERROR: luci-app-cdn-cache: luci.cdn-cache is NOT executable +``` + +**Fix:** +```bash +chmod +x luci-app-cdn-cache/root/usr/libexec/rpcd/luci.cdn-cache +``` + +### Error: Method 'get_stats' from RPCD not found in ACL + +``` +⚠ WARNING: luci-app-cdn-cache: Method 'get_stats' from RPCD not in ACL +``` + +**Fix:** +```bash +# Edit root/usr/share/rpcd/acl.d/luci-app-cdn-cache.json +# Add 'get_stats' to the read.ubus array: +{ + "luci-app-cdn-cache": { + "read": { + "ubus": { + "luci.cdn-cache": ["status", "get_config", "get_stats"] + ↑ Add here + } + } + } +} +``` + +### Error: Invalid JSON syntax + +``` +✗ ERROR: luci-app-cdn-cache: acl.d JSON is INVALID - syntax error +``` + +**Fix:** +```bash +# Validate JSON: +python3 -m json.tool root/usr/share/rpcd/acl.d/luci-app-cdn-cache.json + +# Common issues: +# - Missing comma between array elements +# - Trailing comma after last element +# - Unescaped quotes in strings +``` + +## Bypassing Validation (NOT RECOMMENDED) + +In rare cases, you may need to bypass validation: + +```bash +# Skip pre-push validation: +git push --no-verify + +# Skip module generation validation: +# (can't bypass - it's informational only) +``` + +**⚠️ WARNING:** Bypassing validation can lead to broken modules in production! + +## Integration with CI/CD + +### GitHub Actions + +Add validation to your workflow: + +```yaml +name: Validate Modules + +on: [push, pull_request] + +jobs: + validate: + runs-on: ubuntu-latest + + steps: + - uses: actions/checkout@v4 + + - name: Install dependencies + run: | + sudo apt-get update + sudo apt-get install -y python3 shellcheck + + - name: Run module validation + run: | + chmod +x secubox-tools/validate-modules.sh + ./secubox-tools/validate-modules.sh + + - name: Run pre-push validation + run: | + chmod +x secubox-tools/pre-push-validation.sh + ./secubox-tools/pre-push-validation.sh +``` + +## Best Practices + +1. **Always validate before committing** + ```bash + ./secubox-tools/validate-module-generation.sh luci-app- + ``` + +2. **Install git hooks for automatic validation** + ```bash + ./secubox-tools/install-git-hooks.sh + ``` + +3. **Fix errors immediately** - Don't accumulate validation debt + +4. **Review warnings** - They often indicate real issues + +5. **Test on OpenWrt** before marking complete: + ```bash + scp bin/packages/*/base/luci-app-*.ipk root@192.168.1.1:/tmp/ + ssh root@192.168.1.1 + opkg install /tmp/luci-app-*.ipk + /etc/init.d/rpcd restart + /etc/init.d/uhttpd restart + ``` + +6. **Document module-specific requirements** in module README + +## Troubleshooting + +### Validation script fails to run + +```bash +# Make sure scripts are executable: +chmod +x secubox-tools/*.sh + +# Check dependencies: +which python3 # For JSON validation +which shellcheck # For shell script validation +``` + +### Git hook not running + +```bash +# Check hook is installed: +ls -la .git/hooks/pre-push + +# Reinstall hooks: +./secubox-tools/install-git-hooks.sh +``` + +### False positives in validation + +If validation incorrectly reports an error, please report it: +- Create issue with full validation output +- Include module name and specific check that failed +- We'll update validation logic + +## Additional Resources + +- [CLAUDE.md](CLAUDE.md) - Main project documentation +- [secubox-tools/README.md](secubox-tools/README.md) - Tools documentation +- [.claude/module-prompts.md](.claude/module-prompts.md) - Module generation prompts + +## Support + +If you encounter validation issues: + +1. Check this guide for common errors +2. Run validation with verbose output +3. Review CLAUDE.md for naming conventions +4. Create issue on GitHub with validation output diff --git a/secubox-tools/README.md b/secubox-tools/README.md index a5e64e9d..845598ca 100644 --- a/secubox-tools/README.md +++ b/secubox-tools/README.md @@ -2,11 +2,13 @@ This directory contains utilities for validating, debugging, and maintaining SecuBox modules. -## Tools +## Tools Overview -### validate-modules.sh +### Validation Tools -Comprehensive validation script that checks all critical naming conventions and module structure. +#### validate-modules.sh + +Fast validation of all modules in the repository. **Usage:** ```bash @@ -32,7 +34,63 @@ Comprehensive validation script that checks all critical naming conventions and ❌ ERROR: luci-app-example: RPCD script 'example' does not match ubus object 'luci.example' ``` -### secubox-repair.sh +#### validate-module-generation.sh + +**NEW!** Comprehensive validation for a single module during/after generation. + +**Usage:** +```bash +./secubox-tools/validate-module-generation.sh luci-app-cdn-cache +``` + +**Checks performed:** +- Makefile completeness (all required fields) +- RPCD script naming convention (luci.* prefix) +- RPCD script structure and handlers +- ACL permissions coverage +- Menu path validation +- JavaScript view validation +- RPC method declarations vs RPCD implementations +- Security scans (hardcoded credentials, dangerous commands) +- Documentation presence + +**When to use:** +- After generating a new module +- Before committing changes to a module +- When debugging module integration issues + +#### pre-push-validation.sh + +**NEW!** Git pre-push hook that validates all modules before allowing push. + +**Usage:** +```bash +# Automatic (via git hook): +git push # validation runs automatically + +# Manual: +./secubox-tools/pre-push-validation.sh +``` + +**Checks performed:** +- All validation from validate-modules.sh +- Git staged changes analysis +- Modified module detection +- Comprehensive security scans +- Full module validation on modified modules + +**Exit codes:** +- `0` - Push allowed +- `1` - Push blocked (critical errors found) + +**Installation:** +```bash +./secubox-tools/install-git-hooks.sh +``` + +### Maintenance Tools + +#### secubox-repair.sh Auto-repair tool that fixes common issues in Makefiles and RPCD scripts. @@ -41,18 +99,67 @@ Auto-repair tool that fixes common issues in Makefiles and RPCD scripts. ./secubox-tools/secubox-repair.sh ``` -### secubox-debug.sh +#### secubox-debug.sh -Validates individual package structure and dependencies. +Validates individual package structure and dependencies on OpenWrt device. **Usage:** ```bash ./secubox-tools/secubox-debug.sh luci-app- ``` +#### install-git-hooks.sh + +**NEW!** Installs git hooks for automatic validation. + +**Usage:** +```bash +./secubox-tools/install-git-hooks.sh +``` + +This creates a symbolic link from `.git/hooks/pre-push` to `pre-push-validation.sh`. + ## Recommended Workflow -Before committing changes: +### When Generating a New Module + +1. **Generate module files** (use Claude with module-prompts.md) + +2. **Validate the module:** + ```bash + ./secubox-tools/validate-module-generation.sh luci-app- + ``` + +3. **Fix all ERRORS** (critical) + +4. **Review and fix WARNINGS** (recommended) + +5. **Commit changes:** + ```bash + git add luci-app- + git commit -m "feat: implement module" + git push # Pre-push validation runs automatically + ``` + +### When Modifying Existing Modules + +1. **Make your changes** + +2. **Run quick validation:** + ```bash + ./secubox-tools/validate-modules.sh + ``` + +3. **For complex changes, run full validation:** + ```bash + ./secubox-tools/validate-module-generation.sh luci-app- + ``` + +4. **Commit and push** (validation runs automatically) + +### Before Committing Changes + +Always run at least one validation tool before committing: 1. **Run validation** (CRITICAL): ```bash diff --git a/secubox-tools/install-git-hooks.sh b/secubox-tools/install-git-hooks.sh new file mode 100755 index 00000000..73eed5d3 --- /dev/null +++ b/secubox-tools/install-git-hooks.sh @@ -0,0 +1,35 @@ +#!/bin/bash +# +# install-git-hooks.sh +# ==================== +# Installs SecuBox validation git hooks +# + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +REPO_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)" +GIT_HOOKS_DIR="$REPO_ROOT/.git/hooks" + +echo "Installing SecuBox git hooks..." + +# Create hooks directory if it doesn't exist +mkdir -p "$GIT_HOOKS_DIR" + +# Install pre-push hook +if [ -f "$SCRIPT_DIR/pre-push-validation.sh" ]; then + ln -sf ../../secubox-tools/pre-push-validation.sh "$GIT_HOOKS_DIR/pre-push" + chmod +x "$SCRIPT_DIR/pre-push-validation.sh" + echo "✓ Installed pre-push hook" +else + echo "✗ pre-push-validation.sh not found" + exit 1 +fi + +echo "" +echo "Git hooks installed successfully!" +echo "" +echo "The pre-push hook will run automatically before every 'git push'" +echo "to validate your modules." +echo "" +echo "To bypass validation (NOT RECOMMENDED):" +echo " git push --no-verify" +echo "" diff --git a/secubox-tools/pre-push-validation.sh b/secubox-tools/pre-push-validation.sh new file mode 100755 index 00000000..fc381186 --- /dev/null +++ b/secubox-tools/pre-push-validation.sh @@ -0,0 +1,379 @@ +#!/bin/bash +# +# pre-push-validation.sh +# ====================== +# Git pre-push hook for SecuBox repository +# Validates all modules before allowing push to remote +# +# Installation: +# ln -s ../../secubox-tools/pre-push-validation.sh .git/hooks/pre-push +# chmod +x .git/hooks/pre-push +# +# Or run manually: +# ./secubox-tools/pre-push-validation.sh +# + +set -e + +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +CYAN='\033[0;36m' +BOLD='\033[1m' +NC='\033[0m' + +ERRORS=0 +WARNINGS=0 + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +REPO_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)" + +echo "" +echo -e "${CYAN}╔══════════════════════════════════════════════════════════════════════╗${NC}" +echo -e "${CYAN}║${NC} ${BOLD}SecuBox Pre-Push Validation${NC}" +echo -e "${CYAN}╚══════════════════════════════════════════════════════════════════════╝${NC}" +echo "" + +error() { + echo -e "${RED}✗ $1${NC}" + ((ERRORS++)) +} + +warn() { + echo -e "${YELLOW}⚠ $1${NC}" + ((WARNINGS++)) +} + +success() { + echo -e "${GREEN}✓ $1${NC}" +} + +info() { + echo -e "${CYAN}→ $1${NC}" +} + +section() { + echo "" + echo -e "${BLUE}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}" + echo -e "${BLUE} $1${NC}" + echo -e "${BLUE}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}" +} + +cd "$REPO_ROOT" + +# ============================================ +# 1. Check Git Status +# ============================================ +section "1. Git Status Check" + +if git diff --cached --quiet; then + warn "No staged changes to commit" +else + success "Staged changes detected" +fi + +# Get list of modified module directories +MODIFIED_MODULES=$(git diff --cached --name-only | grep '^luci-app-' | cut -d'/' -f1 | sort -u) + +if [ -z "$MODIFIED_MODULES" ]; then + info "No module files modified in this commit" +else + info "Modified modules in this commit:" + echo "$MODIFIED_MODULES" | while read -r mod; do + echo " - $mod" + done +fi + +# ============================================ +# 2. Critical Naming Convention Checks +# ============================================ +section "2. Critical Naming Conventions" + +info "Checking RPCD script names match ubus object names..." +echo "" + +for module_dir in luci-app-*/; do + [ -d "$module_dir" ] || continue + + module_name=$(basename "$module_dir") + rpcd_dir="$module_dir/root/usr/libexec/rpcd" + + if [ -d "$rpcd_dir" ]; then + rpcd_script=$(find "$rpcd_dir" -type f ! -name "*.md" 2>/dev/null | head -1) + + if [ -n "$rpcd_script" ]; then + rpcd_name=$(basename "$rpcd_script") + + # Extract ubus object from JavaScript + js_objects=$(find "$module_dir/htdocs" -name "*.js" -type f 2>/dev/null | \ + xargs grep -h "object:" 2>/dev/null | \ + grep -o "'[^']*'" | grep "luci\." | tr -d "'" | sort -u) + + if [ -n "$js_objects" ]; then + match_found=false + + for obj in $js_objects; do + if [ "$rpcd_name" = "$obj" ]; then + success "$module_name: RPCD '$rpcd_name' matches ubus object '$obj'" + match_found=true + break + fi + done + + if [ "$match_found" = false ]; then + error "$module_name: RPCD '$rpcd_name' does NOT match ubus objects: $js_objects" + info " Fix: Rename $rpcd_script to one of: $js_objects" + fi + fi + + # Check naming convention + if [[ ! $rpcd_name == luci.* ]]; then + error "$module_name: RPCD script '$rpcd_name' missing 'luci.' prefix" + info " All RPCD scripts MUST start with 'luci.'" + fi + fi + fi +done + +# ============================================ +# 3. Menu Path Validation +# ============================================ +section "3. Menu Path vs View File Validation" + +info "Checking menu paths match view file locations..." +echo "" + +for module_dir in luci-app-*/; do + [ -d "$module_dir" ] || continue + + module_name=$(basename "$module_dir") + menu_file="$module_dir/root/usr/share/luci/menu.d/${module_name}.json" + + if [ -f "$menu_file" ]; then + # Extract paths from menu + menu_paths=$(grep -o '"path":\s*"[^"]*"' "$menu_file" | cut -d'"' -f4) + + for path in $menu_paths; do + view_file="$module_dir/htdocs/luci-static/resources/view/${path}.js" + + if [ -f "$view_file" ]; then + success "$module_name: Menu path '$path' → file exists" + else + error "$module_name: Menu path '$path' → file NOT FOUND" + info " Expected: $view_file" + fi + done + fi +done + +# ============================================ +# 4. JSON Validation +# ============================================ +section "4. JSON Syntax Validation" + +info "Validating all JSON files..." +echo "" + +find . -name "*.json" -path "*/luci-app-*" ! -path "*/.git/*" | while read -r json_file; do + if python3 -m json.tool "$json_file" > /dev/null 2>&1; then + success "$(echo "$json_file" | cut -d'/' -f1-2): $(basename "$json_file") is valid" + else + error "Invalid JSON: $json_file" + fi +done + +# ============================================ +# 5. RPCD Executable Check +# ============================================ +section "5. RPCD Script Permissions" + +info "Checking RPCD scripts are executable..." +echo "" + +for module_dir in luci-app-*/; do + [ -d "$module_dir" ] || continue + + module_name=$(basename "$module_dir") + rpcd_dir="$module_dir/root/usr/libexec/rpcd" + + if [ -d "$rpcd_dir" ]; then + find "$rpcd_dir" -type f ! -name "*.md" 2>/dev/null | while read -r script; do + if [ -x "$script" ]; then + success "$module_name: $(basename "$script") is executable" + else + error "$module_name: $(basename "$script") is NOT executable" + info " Fix: chmod +x $script" + fi + done + fi +done + +# ============================================ +# 6. ACL Method Coverage +# ============================================ +section "6. ACL Method Coverage" + +info "Checking ACL permissions cover RPCD methods..." +echo "" + +for module_dir in luci-app-*/; do + [ -d "$module_dir" ] || continue + + module_name=$(basename "$module_dir") + rpcd_dir="$module_dir/root/usr/libexec/rpcd" + acl_file="$module_dir/root/usr/share/rpcd/acl.d/${module_name}.json" + + if [ -d "$rpcd_dir" ] && [ -f "$acl_file" ]; then + rpcd_script=$(find "$rpcd_dir" -type f ! -name "*.md" 2>/dev/null | head -1) + + if [ -n "$rpcd_script" ]; then + # Extract methods from RPCD + rpcd_methods=$(grep -A 50 'case "\$2" in' "$rpcd_script" 2>/dev/null | \ + grep -E '^\s+[a-z_]+\)' | \ + sed 's/[[:space:]]*\(.*\))/\1/' | \ + grep -v '^\*') + + # Check each method is in ACL + for method in $rpcd_methods; do + if grep -q "\"$method\"" "$acl_file"; then + success "$module_name: Method '$method' is in ACL" + else + warn "$module_name: Method '$method' from RPCD not in ACL" + fi + done + fi + fi +done + +# ============================================ +# 7. Makefile Validation +# ============================================ +section "7. Makefile Validation" + +info "Checking Makefiles have required fields..." +echo "" + +for module_dir in luci-app-*/; do + [ -d "$module_dir" ] || continue + + module_name=$(basename "$module_dir") + makefile="$module_dir/Makefile" + + if [ ! -f "$makefile" ]; then + error "$module_name: Makefile missing" + continue + fi + + # Check required fields + required_fields=("PKG_NAME" "PKG_VERSION" "PKG_RELEASE" "PKG_LICENSE" "LUCI_TITLE") + missing=() + + for field in "${required_fields[@]}"; do + if ! grep -q "^${field}" "$makefile"; then + missing+=("$field") + fi + done + + if [ ${#missing[@]} -eq 0 ]; then + success "$module_name: Makefile has all required fields" + else + error "$module_name: Makefile missing fields: ${missing[*]}" + fi + + # Check PKG_NAME matches directory + if grep -q "^PKG_NAME.*${module_name}" "$makefile"; then + success "$module_name: PKG_NAME matches directory" + else + warn "$module_name: PKG_NAME might not match directory" + fi +done + +# ============================================ +# 8. Security Checks +# ============================================ +section "8. Security Checks" + +info "Scanning for potential security issues..." +echo "" + +# Check for hardcoded secrets +SECRET_PATTERNS="password\s*=\s*['\"][^'\"]*['\"]|api_key\s*=|secret\s*=\s*['\"][^'\"]*['\"]|token\s*=\s*['\"][^'\"]*['\"]" + +for module_dir in luci-app-*/; do + [ -d "$module_dir" ] || continue + + module_name=$(basename "$module_dir") + + # Check RPCD scripts + if [ -d "$module_dir/root/usr/libexec/rpcd" ]; then + if find "$module_dir/root/usr/libexec/rpcd" -type f -exec grep -iE "$SECRET_PATTERNS" {} \; 2>/dev/null | grep -v '^#' | head -1 > /dev/null; then + warn "$module_name: Possible hardcoded credentials in RPCD scripts" + fi + fi + + # Check JavaScript files + if [ -d "$module_dir/htdocs" ]; then + if find "$module_dir/htdocs" -name "*.js" -type f -exec grep -iE "$SECRET_PATTERNS" {} \; 2>/dev/null | grep -v '^//' | grep -v '^\*' | head -1 > /dev/null; then + warn "$module_name: Possible hardcoded credentials in JS files" + fi + fi +done + +success "Security scan complete" + +# ============================================ +# 9. Run Full Module Validation (if modified) +# ============================================ +if [ -n "$MODIFIED_MODULES" ] && [ -f "$SCRIPT_DIR/validate-modules.sh" ]; then + section "9. Full Module Validation" + + info "Running comprehensive validation on modified modules..." + echo "" + + if "$SCRIPT_DIR/validate-modules.sh" 2>&1 | tail -20; then + success "Full validation passed" + else + error "Full validation found issues" + fi +fi + +# ============================================ +# Summary +# ============================================ +section "Validation Summary" + +echo "" +if [ $ERRORS -eq 0 ] && [ $WARNINGS -eq 0 ]; then + echo -e "${GREEN}╔══════════════════════════════════════════════════════════════════════╗${NC}" + echo -e "${GREEN}║ ✓ ALL CHECKS PASSED - Push is allowed ║${NC}" + echo -e "${GREEN}╚══════════════════════════════════════════════════════════════════════╝${NC}" + echo "" + exit 0 +elif [ $ERRORS -eq 0 ]; then + echo -e "${YELLOW}╔══════════════════════════════════════════════════════════════════════╗${NC}" + echo -e "${YELLOW}║ ⚠ Passed with $WARNINGS warning(s) - Push is allowed ║${NC}" + echo -e "${YELLOW}╚══════════════════════════════════════════════════════════════════════╝${NC}" + echo "" + echo -e "${CYAN}Consider fixing warnings before pushing.${NC}" + echo "" + exit 0 +else + echo -e "${RED}╔══════════════════════════════════════════════════════════════════════╗${NC}" + echo -e "${RED}║ ✗ VALIDATION FAILED - Push is BLOCKED ║${NC}" + echo -e "${RED}║ Found $ERRORS error(s) and $WARNINGS warning(s) ║${NC}" + echo -e "${RED}╚══════════════════════════════════════════════════════════════════════╝${NC}" + echo "" + echo -e "${CYAN}Fix the errors above before pushing:${NC}" + echo "" + echo " 1. Rename RPCD scripts to match ubus objects (use luci. prefix)" + echo " 2. Update menu paths to match view file locations" + echo " 3. Add missing RPCD methods to ACL permissions" + echo " 4. Fix Makefile missing fields" + echo " 5. Make RPCD scripts executable: chmod +x" + echo "" + echo "To bypass this check (NOT RECOMMENDED):" + echo " git push --no-verify" + echo "" + exit 1 +fi diff --git a/secubox-tools/validate-module-generation.sh b/secubox-tools/validate-module-generation.sh new file mode 100755 index 00000000..f9a82693 --- /dev/null +++ b/secubox-tools/validate-module-generation.sh @@ -0,0 +1,507 @@ +#!/bin/bash +# +# validate-module-generation.sh +# ============================ +# Validates a newly generated or modified SecuBox module +# Ensures all naming conventions, file structures, and integrations are correct +# +# Usage: +# ./validate-module-generation.sh +# ./validate-module-generation.sh luci-app-cdn-cache +# + +set -e + +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +CYAN='\033[0;36m' +MAGENTA='\033[0;35m' +BOLD='\033[1m' +NC='\033[0m' + +ERRORS=0 +WARNINGS=0 +CHECKS=0 + +MODULE_DIR="$1" + +if [ -z "$MODULE_DIR" ]; then + echo "Usage: $0 " + echo "Example: $0 luci-app-cdn-cache" + exit 1 +fi + +if [ ! -d "$MODULE_DIR" ]; then + echo -e "${RED}Error: Directory $MODULE_DIR not found${NC}" + exit 1 +fi + +MODULE_NAME=$(basename "$MODULE_DIR" | sed 's/^luci-app-//') +PKG_NAME=$(basename "$MODULE_DIR") + +echo "" +echo -e "${CYAN}╔══════════════════════════════════════════════════════════════════════╗${NC}" +echo -e "${CYAN}║${NC} ${BOLD}Module Generation Validation: $MODULE_NAME${NC}" +echo -e "${CYAN}╚══════════════════════════════════════════════════════════════════════╝${NC}" +echo "" + +error() { + echo -e " ${RED}✗ ERROR: $1${NC}" + ((ERRORS++)) +} + +warn() { + echo -e " ${YELLOW}⚠ WARNING: $1${NC}" + ((WARNINGS++)) +} + +success() { + echo -e " ${GREEN}✓ $1${NC}" + ((CHECKS++)) +} + +info() { + echo -e " ${CYAN}→ $1${NC}" +} + +section() { + echo "" + echo -e "${BLUE}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}" + echo -e "${BLUE} $1${NC}" + echo -e "${BLUE}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}" +} + +# ============================================ +# 1. Check Makefile +# ============================================ +section "1. Validating Makefile" + +MAKEFILE="$MODULE_DIR/Makefile" + +if [ ! -f "$MAKEFILE" ]; then + error "Makefile is missing" +else + success "Makefile exists" + + # Check required fields + REQUIRED_FIELDS=("PKG_NAME" "PKG_VERSION" "PKG_RELEASE" "PKG_LICENSE" "LUCI_TITLE" "LUCI_DEPENDS") + + for field in "${REQUIRED_FIELDS[@]}"; do + if grep -q "^${field}" "$MAKEFILE"; then + success "Makefile has $field" + else + error "Makefile missing $field" + fi + done + + # Check PKG_NAME matches directory + if grep -q "^PKG_NAME.*${PKG_NAME}" "$MAKEFILE"; then + success "PKG_NAME matches directory name" + else + error "PKG_NAME in Makefile doesn't match directory name" + fi + + # Check luci.mk include + if grep -q 'include.*luci\.mk' "$MAKEFILE"; then + success "Includes luci.mk" + else + error "Missing include for luci.mk" + fi + + # Check for proper dependencies + if grep -q 'LUCI_DEPENDS.*+luci-base.*+rpcd' "$MAKEFILE"; then + success "Has required dependencies (luci-base, rpcd)" + else + warn "Missing standard dependencies (luci-base, rpcd)" + fi +fi + +# ============================================ +# 2. Check RPCD Script +# ============================================ +section "2. Validating RPCD Backend" + +RPCD_DIR="$MODULE_DIR/root/usr/libexec/rpcd" +RPCD_SCRIPT="" + +if [ ! -d "$RPCD_DIR" ]; then + error "RPCD directory missing: $RPCD_DIR" +else + # Find RPCD script + RPCD_COUNT=$(find "$RPCD_DIR" -type f ! -name "*.md" 2>/dev/null | wc -l) + + if [ "$RPCD_COUNT" -eq 0 ]; then + error "No RPCD script found in $RPCD_DIR" + elif [ "$RPCD_COUNT" -gt 1 ]; then + warn "Multiple RPCD scripts found (expected 1)" + find "$RPCD_DIR" -type f ! -name "*.md" -exec basename {} \; + else + RPCD_SCRIPT=$(find "$RPCD_DIR" -type f ! -name "*.md" 2>/dev/null | head -1) + RPCD_NAME=$(basename "$RPCD_SCRIPT") + + success "Found RPCD script: $RPCD_NAME" + + # CRITICAL: Check naming convention + if [[ $RPCD_NAME == luci.* ]]; then + success "RPCD script follows naming convention (luci.* prefix)" + else + error "RPCD script MUST start with 'luci.' prefix (found: $RPCD_NAME)" + info "Expected: luci.$MODULE_NAME or similar" + fi + + # Check executable + if [ -x "$RPCD_SCRIPT" ]; then + success "RPCD script is executable" + else + error "RPCD script is NOT executable" + info "Fix: chmod +x $RPCD_SCRIPT" + fi + + # Check shebang + if head -1 "$RPCD_SCRIPT" | grep -q '^#!/bin/sh\|^#!/bin/bash'; then + success "RPCD script has valid shebang" + else + error "RPCD script missing or invalid shebang" + fi + + # Check required structure + if grep -q 'case "\$1" in' "$RPCD_SCRIPT"; then + success "RPCD script has case structure for method routing" + else + warn "RPCD script might be missing case structure" + fi + + if grep -q 'list)' "$RPCD_SCRIPT" && grep -q 'call)' "$RPCD_SCRIPT"; then + success "RPCD script has list and call handlers" + else + error "RPCD script missing list or call handlers" + fi + + # Extract RPCD methods + info "Extracting RPCD methods..." + RPCD_METHODS=$(grep -A 50 'case "\$2" in' "$RPCD_SCRIPT" 2>/dev/null | grep -E '^\s+[a-z_]+\)' | sed 's/[[:space:]]*\(.*\))/\1/' | tr '\n' ' ') + + if [ -n "$RPCD_METHODS" ]; then + info "Found methods: $RPCD_METHODS" + else + warn "Could not extract RPCD methods" + fi + fi +fi + +# ============================================ +# 3. Check ACL File +# ============================================ +section "3. Validating ACL Permissions" + +ACL_FILE="$MODULE_DIR/root/usr/share/rpcd/acl.d/${PKG_NAME}.json" + +if [ ! -f "$ACL_FILE" ]; then + error "ACL file missing: $ACL_FILE" +else + success "ACL file exists" + + # Validate JSON syntax + if python3 -m json.tool "$ACL_FILE" > /dev/null 2>&1; then + success "ACL JSON is valid" + else + error "ACL JSON is INVALID - syntax error" + fi + + # Check structure + if grep -q '"read":' "$ACL_FILE" && grep -q '"write":' "$ACL_FILE"; then + success "ACL has read and write sections" + else + warn "ACL might be missing read or write sections" + fi + + # Extract ubus object from ACL + ACL_UBUS_OBJECTS=$(grep -o '"luci\.[^"]*"' "$ACL_FILE" | sort -u | tr -d '"' | tr '\n' ' ') + + if [ -n "$ACL_UBUS_OBJECTS" ]; then + success "ACL defines ubus objects: $ACL_UBUS_OBJECTS" + + # CRITICAL: Check if RPCD script name matches ACL ubus object + if [ -n "$RPCD_SCRIPT" ]; then + RPCD_NAME=$(basename "$RPCD_SCRIPT") + + MATCH_FOUND=false + for obj in $ACL_UBUS_OBJECTS; do + if [ "$RPCD_NAME" = "$obj" ]; then + success "CRITICAL: RPCD script name '$RPCD_NAME' matches ACL ubus object '$obj'" + MATCH_FOUND=true + break + fi + done + + if [ "$MATCH_FOUND" = false ]; then + error "CRITICAL: RPCD script name '$RPCD_NAME' does NOT match any ACL ubus object" + info "ACL objects: $ACL_UBUS_OBJECTS" + info "This will cause RPC errors: 'Object not found'" + fi + fi + else + error "No ubus objects defined in ACL (must have luci.* objects)" + fi + + # Check if RPCD methods are in ACL + if [ -n "$RPCD_METHODS" ] && [ -n "$ACL_UBUS_OBJECTS" ]; then + info "Checking if RPCD methods are in ACL permissions..." + + for method in $RPCD_METHODS; do + if grep -q "\"$method\"" "$ACL_FILE"; then + success "Method '$method' is in ACL" + else + warn "Method '$method' from RPCD not found in ACL" + fi + done + fi +fi + +# ============================================ +# 4. Check Menu File +# ============================================ +section "4. Validating Menu Definition" + +MENU_FILE="$MODULE_DIR/root/usr/share/luci/menu.d/${PKG_NAME}.json" + +if [ ! -f "$MENU_FILE" ]; then + error "Menu file missing: $MENU_FILE" +else + success "Menu file exists" + + # Validate JSON + if python3 -m json.tool "$MENU_FILE" > /dev/null 2>&1; then + success "Menu JSON is valid" + else + error "Menu JSON is INVALID - syntax error" + fi + + # Extract view paths from menu + MENU_PATHS=$(grep -o '"path":\s*"[^"]*"' "$MENU_FILE" | cut -d'"' -f4) + + if [ -n "$MENU_PATHS" ]; then + info "Found menu paths:" + echo "$MENU_PATHS" | while read -r path; do + info " - $path" + + # CRITICAL: Check if view file exists + VIEW_FILE="$MODULE_DIR/htdocs/luci-static/resources/view/${path}.js" + + if [ -f "$VIEW_FILE" ]; then + success "Menu path '$path' → view file EXISTS" + else + error "Menu path '$path' → view file NOT FOUND: $VIEW_FILE" + info "This will cause HTTP 404 errors" + fi + done + else + warn "No view paths found in menu (might use different structure)" + fi + + # Check if menu references ACL + if grep -q "\"acl\".*${PKG_NAME}" "$MENU_FILE"; then + success "Menu references ACL: $PKG_NAME" + else + warn "Menu might not reference ACL properly" + fi +fi + +# ============================================ +# 5. Check JavaScript Views +# ============================================ +section "5. Validating JavaScript Views" + +VIEW_DIR="$MODULE_DIR/htdocs/luci-static/resources/view" + +if [ ! -d "$VIEW_DIR" ]; then + warn "View directory missing: $VIEW_DIR" +else + VIEW_COUNT=$(find "$VIEW_DIR" -name "*.js" -type f 2>/dev/null | wc -l) + + if [ "$VIEW_COUNT" -eq 0 ]; then + warn "No JavaScript view files found" + else + success "Found $VIEW_COUNT view file(s)" + + # Check each view file + find "$VIEW_DIR" -name "*.js" -type f 2>/dev/null | while read -r view_file; do + rel_path=$(echo "$view_file" | sed "s|$MODULE_DIR/htdocs/luci-static/resources/view/||" | sed 's|.js$||') + + # Check strict mode + if head -5 "$view_file" | grep -q "'use strict'"; then + success "View '$rel_path' has 'use strict'" + else + warn "View '$rel_path' missing 'use strict'" + fi + + # Extract ubus object declarations + VIEW_UBUS_OBJECTS=$(grep -o "object:\s*['\"][^'\"]*['\"]" "$view_file" | grep -o "['\"]luci\.[^'\"]*['\"]" | tr -d "\"'" | sort -u | tr '\n' ' ') + + if [ -n "$VIEW_UBUS_OBJECTS" ]; then + info "View '$rel_path' uses ubus objects: $VIEW_UBUS_OBJECTS" + + # CRITICAL: Check if ubus object matches RPCD script + if [ -n "$RPCD_SCRIPT" ]; then + RPCD_NAME=$(basename "$RPCD_SCRIPT") + + for obj in $VIEW_UBUS_OBJECTS; do + if [ "$RPCD_NAME" = "$obj" ]; then + success "View ubus object '$obj' matches RPCD script '$RPCD_NAME'" + else + if [ -n "$ACL_UBUS_OBJECTS" ] && echo "$ACL_UBUS_OBJECTS" | grep -q "$obj"; then + success "View ubus object '$obj' is in ACL" + else + error "View ubus object '$obj' does NOT match RPCD '$RPCD_NAME' or ACL objects" + fi + fi + done + fi + fi + + # Extract RPC method calls + VIEW_METHODS=$(grep -o "method:\s*['\"][^'\"]*['\"]" "$view_file" | cut -d"'" -f2 | cut -d'"' -f2 | sort -u | tr '\n' ' ') + + if [ -n "$VIEW_METHODS" ] && [ -n "$RPCD_METHODS" ]; then + info "Checking if view RPC methods exist in RPCD..." + + for method in $VIEW_METHODS; do + if echo "$RPCD_METHODS" | grep -qw "$method"; then + success "Method '$method' exists in RPCD" + else + error "Method '$method' called in view but NOT in RPCD" + fi + done + fi + done + fi +fi + +# ============================================ +# 6. Check UCI Config (Optional) +# ============================================ +section "6. Checking UCI Configuration (Optional)" + +MODULE_UNDERSCORE=$(echo "$MODULE_NAME" | tr '-' '_') +UCI_CONFIG="$MODULE_DIR/root/etc/config/$MODULE_UNDERSCORE" + +if [ -f "$UCI_CONFIG" ]; then + success "UCI config file exists: $MODULE_UNDERSCORE" + + # Basic validation + if grep -q '^config ' "$UCI_CONFIG"; then + success "UCI config has valid structure" + else + warn "UCI config might have invalid structure" + fi +else + info "No UCI config file (optional for many modules)" +fi + +# ============================================ +# 7. Check File Permissions +# ============================================ +section "7. Validating File Permissions" + +# All RPCD scripts must be executable +if [ -d "$RPCD_DIR" ]; then + NON_EXEC_RPCD=$(find "$RPCD_DIR" -type f ! -executable ! -name "*.md" 2>/dev/null) + + if [ -n "$NON_EXEC_RPCD" ]; then + error "Found non-executable RPCD scripts:" + echo "$NON_EXEC_RPCD" | while read -r f; do + error " - $f (need chmod +x)" + done + else + success "All RPCD scripts are executable" + fi +fi + +# ============================================ +# 8. Security Checks +# ============================================ +section "8. Security Checks" + +# Check for hardcoded credentials +if [ -n "$RPCD_SCRIPT" ]; then + if grep -qi 'password\s*=\|secret\s*=' "$RPCD_SCRIPT"; then + warn "RPCD script might contain hardcoded credentials" + else + success "No obvious hardcoded credentials in RPCD" + fi +fi + +# Check for dangerous commands +if [ -n "$RPCD_SCRIPT" ]; then + DANGEROUS_CMDS="rm -rf|mkfs|dd if=|format|shutdown -h" + + if grep -E "$DANGEROUS_CMDS" "$RPCD_SCRIPT" > /dev/null 2>&1; then + warn "RPCD script contains potentially dangerous commands - review carefully" + else + success "No obviously dangerous commands in RPCD" + fi +fi + +# Check for SQL injection patterns (if using sqlite/mysql) +if [ -n "$RPCD_SCRIPT" ]; then + if grep -q 'sqlite3\|mysql' "$RPCD_SCRIPT"; then + if grep -q '\$[A-Za-z_]*' "$RPCD_SCRIPT" | grep -q 'SELECT\|INSERT\|UPDATE'; then + warn "Possible SQL injection risk - ensure proper escaping" + fi + fi +fi + +# ============================================ +# 9. Documentation Check +# ============================================ +section "9. Documentation Check" + +README="$MODULE_DIR/README.md" + +if [ -f "$README" ]; then + success "README.md exists" + + if wc -l "$README" | awk '{print $1}' | grep -q '^[0-9]\+$' && [ "$(wc -l < "$README")" -gt 10 ]; then + success "README has substantial content ($(wc -l < "$README") lines)" + else + warn "README seems short or empty" + fi +else + warn "README.md missing (recommended for documentation)" +fi + +# ============================================ +# Summary +# ============================================ +section "Validation Summary" + +echo "" +if [ $ERRORS -eq 0 ] && [ $WARNINGS -eq 0 ]; then + echo -e "${GREEN}╔══════════════════════════════════════════════════════════════════════╗${NC}" + echo -e "${GREEN}║ ✓ ALL CHECKS PASSED - Module is ready for deployment! ║${NC}" + echo -e "${GREEN}╚══════════════════════════════════════════════════════════════════════╝${NC}" + exit 0 +elif [ $ERRORS -eq 0 ]; then + echo -e "${YELLOW}╔══════════════════════════════════════════════════════════════════════╗${NC}" + echo -e "${YELLOW}║ ✓ Critical checks passed with $WARNINGS warning(s) ║${NC}" + echo -e "${YELLOW}╚══════════════════════════════════════════════════════════════════════╝${NC}" + echo "" + echo -e "${CYAN}Module can be deployed, but review warnings above.${NC}" + exit 0 +else + echo -e "${RED}╔══════════════════════════════════════════════════════════════════════╗${NC}" + echo -e "${RED}║ ✗ VALIDATION FAILED - Found $ERRORS error(s) and $WARNINGS warning(s) ║${NC}" + echo -e "${RED}╚══════════════════════════════════════════════════════════════════════╝${NC}" + echo "" + echo -e "${CYAN}Please fix the errors above before deploying this module.${NC}" + echo "" + echo -e "${CYAN}Common fixes:${NC}" + echo " 1. Rename RPCD script to match ubus object (must use luci. prefix)" + echo " 2. Update menu paths to match view file locations" + echo " 3. Add missing RPCD methods to ACL permissions" + echo " 4. Make RPCD script executable: chmod +x" + echo "" + exit 1 +fi