From bbf2b194153e21ae978bf2b02b05bb69e25d8803 Mon Sep 17 00:00:00 2001 From: CyberMind-FR Date: Sat, 21 Feb 2026 20:18:51 +0100 Subject: [PATCH] feat(peertube): Add video import with multi-track subtitle sync - New peertube-import script for importing from YouTube, Vimeo, 1000+ sites - CGI endpoints for portal integration (peertube-import, peertube-import-status) - Portal UI: Video Import card with progress tracking - Multi-language subtitle download and PeerTube caption upload - Fixed stdout/stderr separation for reliable function returns - UCI config: uses peertube.admin.username/password - Package version bumped to 1.2.0 - Added README.md with full documentation Co-Authored-By: Claude Opus 4.5 --- .claude/HISTORY.md | 43 ++ .claude/settings.local.json | 7 +- .../root/www/gk2-hub/portal.html | 277 ++++++++- package/secubox/secubox-app-peertube/Makefile | 6 +- .../secubox/secubox-app-peertube/README.md | 132 ++++ .../files/usr/sbin/peertube-import | 586 ++++++++++++++++++ .../files/www/cgi-bin/peertube-import | 164 +++++ .../files/www/cgi-bin/peertube-import-status | 69 +++ 8 files changed, 1281 insertions(+), 3 deletions(-) create mode 100644 package/secubox/secubox-app-peertube/README.md create mode 100644 package/secubox/secubox-app-peertube/files/usr/sbin/peertube-import create mode 100644 package/secubox/secubox-app-peertube/files/www/cgi-bin/peertube-import create mode 100644 package/secubox/secubox-app-peertube/files/www/cgi-bin/peertube-import-status diff --git a/.claude/HISTORY.md b/.claude/HISTORY.md index 6ab61d38..65a40e72 100644 --- a/.claude/HISTORY.md +++ b/.claude/HISTORY.md @@ -3045,3 +3045,46 @@ git checkout HEAD -- index.html - YouTube videos blocked by PO token requirement for subtitle access - PeerTube videos on tube.gk2 have no captions uploaded - Metadata extraction works; transcript step fails without subtitles/Whisper + +31. **PeerTube Video Import with Multi-Track Subtitles (2026-02-21)** + - New `peertube-import` CLI tool for importing videos from YouTube, Vimeo, and 1000+ sites. + - **Features:** + - Download video via yt-dlp (best quality MP4) + - Extract metadata (title, description, tags) + - Download subtitles in multiple languages (configurable) + - Upload video to PeerTube via API + - Upload each subtitle track via `/api/v1/videos/{id}/captions/{lang}` + - **CLI Interface:** + ```bash + peertube-import --lang fr,en,de,es https://youtube.com/watch?v=xxx + peertube-import --privacy 2 --channel 1 https://vimeo.com/xxx + ``` + - **Portal Integration:** + - New "Video Import" card in Intelligence & Analyse section + - Modal dialog with URL input, language selection, privacy options + - Progress bar with live status updates + - Direct link to imported video on completion + - **CGI Endpoints:** + - `POST /cgi-bin/peertube-import` — Start import job + - `GET /cgi-bin/peertube-import-status?job_id=xxx` — Poll status + - **Authentication:** + - Supports PEERTUBE_TOKEN env var + - UCI config: `peertube.api.username` / `peertube.api.password` + - OAuth client credential flow for token acquisition + - Package version bumped to 1.2.0 + - **Files:** + - `secubox-app-peertube/files/usr/sbin/peertube-import` (new) + - `secubox-app-peertube/files/www/cgi-bin/peertube-import` (new) + - `secubox-app-peertube/files/www/cgi-bin/peertube-import-status` (new) + - `luci-app-secubox-portal/root/www/gk2-hub/portal.html` (updated) + - `secubox-app-peertube/Makefile` (updated) + +31. **PeerTube Import Fixes (2026-02-21)** + - Fixed stdout/stderr separation in `peertube-import` script + - Changed UCI config path from `peertube.api.*` to `peertube.admin.*` + - Fixed yt-dlp output redirection to prevent mixing with function return values + - Fixed curl response handling in upload functions (use temp file, not 2>&1) + - Upgraded yt-dlp to 2026.2.4 for YouTube compatibility + - Installed Node.js (20.20.0) for yt-dlp JavaScript runtime support + - Verified end-to-end import flow: YouTube → download → subtitles → PeerTube upload + diff --git a/.claude/settings.local.json b/.claude/settings.local.json index 2c3effaf..12aba016 100644 --- a/.claude/settings.local.json +++ b/.claude/settings.local.json @@ -389,7 +389,12 @@ "Bash(__NEW_LINE_02bd2dd51e90cbf8__ echo \"\")", "Bash(__NEW_LINE_70eb6f3ae1c26753__ echo \"\")", "WebFetch(domain:radio.gk2.secubox.in)", - "WebFetch(domain:nextcloud-talk.readthedocs.io)" + "WebFetch(domain:nextcloud-talk.readthedocs.io)", + "Bash(__NEW_LINE_0334b7e65952251f__ rm -f \"$COOKIES\")", + "Bash(__NEW_LINE_d0f84baac9f3813d__ rm -f \"$COOKIES\")", + "Bash(__NEW_LINE_722c25da6bf58fe1__ rm -f \"$COOKIES\" /tmp/login.html)", + "WebFetch(domain:portal.nextcloud.com)", + "WebFetch(domain:arnowelzel.de)" ] } } diff --git a/package/secubox/luci-app-secubox-portal/root/www/gk2-hub/portal.html b/package/secubox/luci-app-secubox-portal/root/www/gk2-hub/portal.html index acf58b97..252d7295 100644 --- a/package/secubox/luci-app-secubox-portal/root/www/gk2-hub/portal.html +++ b/package/secubox/luci-app-secubox-portal/root/www/gk2-hub/portal.html @@ -108,6 +108,7 @@ text-decoration: none; color: inherit; transition: all 0.2s; + cursor: pointer; } .service-card:hover { border-color: var(--accent); @@ -126,6 +127,101 @@ font-size: 0.85rem; } .footer a { color: var(--accent); text-decoration: none; } + + /* Modal styles */ + .modal-overlay { + position: fixed; + inset: 0; + background: rgba(0,0,0,0.8); + display: flex; + align-items: center; + justify-content: center; + z-index: 1000; + } + .modal-box { + background: var(--surface); + border: 1px solid var(--border); + border-radius: 12px; + width: 90%; + max-width: 500px; + box-shadow: 0 20px 60px rgba(0,95,158,0.3); + } + .modal-header { + display: flex; + justify-content: space-between; + align-items: center; + padding: 20px; + border-bottom: 1px solid var(--border); + } + .modal-header h3 { margin: 0; color: #fff; } + .modal-close { + background: none; + border: none; + color: var(--muted); + font-size: 1.5rem; + cursor: pointer; + } + .modal-close:hover { color: #fff; } + .modal-body { padding: 20px; } + .modal-footer { + padding: 15px 20px; + border-top: 1px solid var(--border); + display: flex; + justify-content: flex-end; + gap: 10px; + } + .form-group { margin-bottom: 15px; } + .form-group label { display: block; margin-bottom: 5px; color: var(--text); font-size: 0.9rem; } + .modal-input { + width: 100%; + padding: 10px 12px; + background: var(--bg); + border: 1px solid var(--border); + border-radius: 6px; + color: var(--text); + font-size: 0.95rem; + } + .modal-input:focus { outline: none; border-color: var(--accent); } + .checkbox-group { display: flex; flex-wrap: wrap; gap: 15px; } + .checkbox-label { display: flex; align-items: center; gap: 5px; cursor: pointer; } + .checkbox-label input { accent-color: var(--accent); } + .btn-primary { + background: var(--accent); + border: none; + color: #fff; + padding: 10px 20px; + border-radius: 6px; + cursor: pointer; + font-weight: 600; + } + .btn-primary:hover { background: #007acc; } + .btn-primary:disabled { opacity: 0.5; cursor: not-allowed; } + .btn-secondary { + background: transparent; + border: 1px solid var(--border); + color: var(--text); + padding: 10px 20px; + border-radius: 6px; + cursor: pointer; + } + .btn-secondary:hover { border-color: var(--accent); } + .progress-bar { + background: var(--bg); + border-radius: 4px; + height: 8px; + overflow: hidden; + margin-bottom: 10px; + } + .progress-fill { + background: var(--accent); + height: 100%; + width: 0%; + transition: width 0.3s; + } + .progress-status { font-size: 0.85rem; color: var(--muted); } + .result-success { color: #2ecc71; padding: 15px; background: rgba(46,204,113,0.1); border-radius: 6px; } + .result-success a { color: var(--accent); } + .result-error { color: #e74c3c; padding: 15px; background: rgba(231,76,60,0.1); border-radius: 6px; } @@ -203,15 +299,25 @@

Intelligence & Analyse

-
🎬
+
🔍
PeerTube Analyse
analyse.gk2.secubox.in
 + +
📥
+
Video Import
+
YouTube, Vimeo, 1000+ sites
+
📻
Radio Stream
stream.gk2.secubox.in
 + +
🎵
+
Lyrion Radio
+
radio.gk2.secubox.in
+

Administration

@@ -243,6 +349,50 @@ + + + diff --git a/package/secubox/secubox-app-peertube/Makefile b/package/secubox/secubox-app-peertube/Makefile index 10318985..403404c0 100644 --- a/package/secubox/secubox-app-peertube/Makefile +++ b/package/secubox/secubox-app-peertube/Makefile @@ -2,7 +2,7 @@ include $(TOPDIR)/rules.mk PKG_NAME:=secubox-app-peertube PKG_RELEASE:=1 -PKG_VERSION:=1.1.0 +PKG_VERSION:=1.2.0 PKG_ARCH:=all PKG_MAINTAINER:=CyberMind Studio PKG_LICENSE:=AGPL-3.0 @@ -23,6 +23,7 @@ PeerTube federated video streaming platform. Runs in an LXC Debian container with PostgreSQL, Redis, and Node.js. Supports video hosting, live streaming, and ActivityPub federation. Includes peertube-analyse: transcript extraction and Claude AI analysis. +Includes peertube-import: auto-import from YouTube/Vimeo with multi-track subtitles. endef define Package/secubox-app-peertube/conffiles @@ -42,6 +43,7 @@ define Package/secubox-app-peertube/install $(INSTALL_DIR) $(1)/usr/sbin $(INSTALL_BIN) ./files/usr/sbin/peertubectl $(1)/usr/sbin/peertubectl $(INSTALL_BIN) ./files/usr/sbin/peertube-analyse $(1)/usr/sbin/peertube-analyse + $(INSTALL_BIN) ./files/usr/sbin/peertube-import $(1)/usr/sbin/peertube-import $(INSTALL_DIR) $(1)/www/peertube-analyse $(INSTALL_DATA) ./files/www/peertube-analyse/index.html $(1)/www/peertube-analyse/ @@ -49,6 +51,8 @@ define Package/secubox-app-peertube/install $(INSTALL_DIR) $(1)/www/cgi-bin $(INSTALL_BIN) ./files/www/cgi-bin/peertube-analyse $(1)/www/cgi-bin/ $(INSTALL_BIN) ./files/www/cgi-bin/peertube-analyse-status $(1)/www/cgi-bin/ + $(INSTALL_BIN) ./files/www/cgi-bin/peertube-import $(1)/www/cgi-bin/ + $(INSTALL_BIN) ./files/www/cgi-bin/peertube-import-status $(1)/www/cgi-bin/ endef $(eval $(call BuildPackage,secubox-app-peertube)) diff --git a/package/secubox/secubox-app-peertube/README.md b/package/secubox/secubox-app-peertube/README.md new file mode 100644 index 00000000..47521efc --- /dev/null +++ b/package/secubox/secubox-app-peertube/README.md @@ -0,0 +1,132 @@ +# SecuBox PeerTube + +Federated video streaming platform running in an LXC Debian container. + +## Features + +- **PeerTube Instance**: Self-hosted video platform with ActivityPub federation +- **Video Import**: Import videos from YouTube, Vimeo, and 1000+ sites via yt-dlp +- **Multi-Track Subtitles**: Automatic subtitle download and sync in multiple languages +- **Video Analysis**: Transcript extraction and Claude AI analysis (peertube-analyse) +- **Live Streaming**: RTMP ingest with HLS output + +## Components + +| Component | Description | +|-----------|-------------| +| `peertubectl` | Main control script for container management | +| `peertube-import` | Video import with subtitle sync | +| `peertube-analyse` | Transcript extraction and AI analysis | + +## Video Import + +Import videos from external platforms with automatic subtitle synchronization. + +### CLI Usage + +```bash +# Basic import +peertube-import https://youtube.com/watch?v=xxx + +# Import with multiple subtitle languages +peertube-import --lang fr,en,de,es https://youtube.com/watch?v=xxx + +# Import as unlisted video +peertube-import --privacy 2 https://youtube.com/watch?v=xxx + +# Import to specific channel +peertube-import --channel 2 https://vimeo.com/xxx +``` + +### Options + +| Option | Description | Default | +|--------|-------------|---------| +| `--lang ` | Subtitle languages (comma-separated) | `fr,en` | +| `--channel ` | PeerTube channel ID | `1` | +| `--privacy ` | 1=public, 2=unlisted, 3=private | `1` | +| `--output ` | Temp directory for downloads | `/tmp/peertube-import` | +| `--peertube ` | PeerTube instance URL | from UCI config | + +### Portal Integration + +Access via SecuBox Portal → Intelligence & Analyse → Video Import + +The portal provides: +- URL input for video source +- Language selection checkboxes +- Privacy level selector +- Real-time progress tracking +- Direct link to imported video + +### CGI Endpoints + +```bash +# Start import job +curl -X POST http://192.168.255.1/cgi-bin/peertube-import \ + -H "Content-Type: application/json" \ + -d '{"url":"https://youtube.com/watch?v=xxx","languages":"fr,en"}' + +# Response: {"success": true, "job_id": "import_xxx"} + +# Check status +curl "http://192.168.255.1/cgi-bin/peertube-import-status?job_id=import_xxx" + +# Response (in progress): +# {"status": "downloading", "progress": 45, "job_id": "import_xxx"} + +# Response (completed): +# {"success": true, "video_url": "https://tube.example.com/w/uuid"} +``` + +## Configuration + +UCI config file: `/etc/config/peertube` + +``` +config peertube 'main' + option enabled '1' + option data_path '/srv/peertube' + +config peertube 'server' + option hostname 'tube.example.com' + option port '9001' + option https '1' + +config peertube 'admin' + option username 'root' + option password 'changeme' + +config peertube 'transcoding' + option enabled '1' + option threads '2' + list resolutions '480p' + list resolutions '720p' +``` + +## Dependencies + +- `lxc`, `lxc-common` - Container runtime +- `wget-ssl` - HTTPS downloads +- `tar`, `jsonfilter` - Archive and JSON handling +- `yt-dlp` - Video download (pip install) +- `node` - JavaScript runtime for yt-dlp (opkg install) + +## Supported Import Sources + +yt-dlp supports 1000+ sites including: +- YouTube, YouTube Music +- Vimeo +- Dailymotion +- Twitch (VODs) +- Twitter/X +- TikTok +- And many more... + +See: https://github.com/yt-dlp/yt-dlp/blob/master/supportedsites.md + +## Version + +- Package: 1.2.0 +- yt-dlp: 2026.2.4 (recommended) +- Node.js: 20.20.0 (for YouTube JS runtime) diff --git a/package/secubox/secubox-app-peertube/files/usr/sbin/peertube-import b/package/secubox/secubox-app-peertube/files/usr/sbin/peertube-import new file mode 100644 index 00000000..12c1d788 --- /dev/null +++ b/package/secubox/secubox-app-peertube/files/usr/sbin/peertube-import @@ -0,0 +1,586 @@ +#!/bin/sh +# PeerTube Video Import with Multi-Track Subtitle Sync +# SecuBox Intelligence Module +# Compatible: OpenWrt + +set -e + +#============================================================================= +# CONFIGURATION +#============================================================================= + +SCRIPT_VERSION="1.0.0" +PEERTUBE_URL="${PEERTUBE_URL:-https://tube.gk2.secubox.in}" +PEERTUBE_TOKEN="${PEERTUBE_TOKEN:-}" +OUTPUT_BASE="${OUTPUT_BASE:-/tmp/peertube-import}" +DEFAULT_CHANNEL_ID=1 +DEFAULT_PRIVACY=1 # 1=public, 2=unlisted, 3=private + +#============================================================================= +# LOGGING +#============================================================================= + +log_info() { echo >&2 "[INFO] $*"; } +log_ok() { echo >&2 "[OK] $*"; } +log_warn() { echo >&2 "[WARN] $*"; } +log_error() { echo >&2 "[ERROR] $*"; } +log_step() { echo >&2 ""; echo >&2 "==> $*"; } +log_progress() { echo >&2 "[PROGRESS] $*"; } + +#============================================================================= +# UTILITY FUNCTIONS +#============================================================================= + +# Generate slug from title +generate_slug() { + echo "$1" | tr '[:upper:]' '[:lower:]' | \ + sed -E 's/[àáâãäå]/a/g; s/[èéêë]/e/g; s/[ìíîï]/i/g; s/[òóôõö]/o/g; s/[ùúûü]/u/g; s/[ç]/c/g' | \ + sed -E 's/[^a-z0-9]+/-/g; s/^-+|-+$//g' | \ + cut -c1-50 +} + +# Get PeerTube authentication token +get_peertube_token() { + local username="$1" + local password="$2" + + if [ -n "$PEERTUBE_TOKEN" ]; then + echo "$PEERTUBE_TOKEN" + return 0 + fi + + # Read credentials from UCI config (admin section) + if [ -z "$username" ]; then + username=$(uci -q get peertube.admin.username) + password=$(uci -q get peertube.admin.password) + fi + + # Try to get OAuth client credentials + local client_id client_secret + local oauth_clients + oauth_clients=$(curl -s "${PEERTUBE_URL}/api/v1/oauth-clients/local") + + if command -v jq >/dev/null 2>&1; then + client_id=$(echo "$oauth_clients" | jq -r '.client_id // empty') + client_secret=$(echo "$oauth_clients" | jq -r '.client_secret // empty') + else + client_id=$(echo "$oauth_clients" | jsonfilter -e '@.client_id' 2>/dev/null) + client_secret=$(echo "$oauth_clients" | jsonfilter -e '@.client_secret' 2>/dev/null) + fi + + if [ -z "$client_id" ] || [ -z "$username" ]; then + log_error "Cannot get PeerTube token. Set PEERTUBE_TOKEN or configure api credentials." + return 1 + fi + + # Get access token + local token_response + token_response=$(curl -s -X POST "${PEERTUBE_URL}/api/v1/users/token" \ + -H "Content-Type: application/x-www-form-urlencoded" \ + -d "client_id=${client_id}" \ + -d "client_secret=${client_secret}" \ + -d "grant_type=password" \ + -d "response_type=code" \ + -d "username=${username}" \ + -d "password=${password}") + + local token + if command -v jq >/dev/null 2>&1; then + token=$(echo "$token_response" | jq -r '.access_token // empty') + else + token=$(echo "$token_response" | jsonfilter -e '@.access_token' 2>/dev/null) + fi + + if [ -z "$token" ]; then + log_error "Failed to get access token" + return 1 + fi + + echo "$token" +} + +#============================================================================= +# VIDEO DOWNLOAD +#============================================================================= + +download_video() { + local url="$1" + local output_dir="$2" + local slug="$3" + + log_step "Downloading video" + + local video_file="$output_dir/${slug}.%(ext)s" + + # Download best quality video - redirect output to stderr to keep stdout clean + if yt-dlp -f "bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]/best" \ + --merge-output-format mp4 \ + -o "$output_dir/${slug}.%(ext)s" \ + --no-playlist \ + "$url" >&2 2>&1; then + + # Find downloaded file + local found_video + found_video=$(find "$output_dir" -name "${slug}.*" -type f | grep -E '\.(mp4|webm|mkv)$' | head -1) + + if [ -n "$found_video" ] && [ -f "$found_video" ]; then + log_ok "Video downloaded: $found_video" + echo "$found_video" + return 0 + fi + fi + + log_error "Failed to download video" + return 1 +} + +#============================================================================= +# METADATA EXTRACTION +#============================================================================= + +extract_metadata() { + local url="$1" + local output_dir="$2" + local slug="$3" + + log_step "Extracting metadata" + + local meta_file="$output_dir/${slug}.meta.json" + + if yt-dlp --dump-json --no-warnings "$url" 2>/dev/null > "$meta_file.tmp"; then + if command -v jq >/dev/null 2>&1; then + jq '{ + id: .id, + title: .title, + description: .description, + duration: .duration, + upload_date: .upload_date, + uploader: .uploader, + channel: .channel, + tags: .tags, + webpage_url: .webpage_url, + thumbnail: .thumbnail, + subtitles: ((.subtitles // {}) | keys), + automatic_captions: ((.automatic_captions // {}) | keys) + }' "$meta_file.tmp" > "$meta_file" + else + mv "$meta_file.tmp" "$meta_file" + fi + rm -f "$meta_file.tmp" + log_ok "Metadata saved: $meta_file" + echo "$meta_file" + return 0 + fi + + log_error "Failed to extract metadata" + return 1 +} + +#============================================================================= +# SUBTITLE DOWNLOAD +#============================================================================= + +download_subtitles() { + local url="$1" + local output_dir="$2" + local slug="$3" + local languages="$4" # Comma-separated: fr,en,de + + log_step "Downloading subtitles" + + [ -z "$languages" ] && languages="fr,en" + + log_info "Requested languages: $languages" + + # Download subtitles with yt-dlp + yt-dlp --write-sub --write-auto-sub \ + --sub-lang "$languages" \ + --sub-format vtt \ + --convert-subs vtt \ + --skip-download \ + -o "$output_dir/${slug}" \ + "$url" 2>&1 || true + + # List downloaded subtitle files + local count=0 + for vtt in "$output_dir"/${slug}*.vtt; do + [ -f "$vtt" ] || continue + count=$((count + 1)) + log_ok "Downloaded: $(basename "$vtt")" + done + + if [ "$count" -eq 0 ]; then + log_warn "No subtitles found for requested languages" + return 1 + fi + + log_ok "Downloaded $count subtitle file(s)" + return 0 +} + +#============================================================================= +# PEERTUBE UPLOAD +#============================================================================= + +upload_video_to_peertube() { + local video_file="$1" + local meta_file="$2" + local channel_id="$3" + local privacy="$4" + local token="$5" + + log_step "Uploading video to PeerTube" + + [ -z "$channel_id" ] && channel_id=$DEFAULT_CHANNEL_ID + [ -z "$privacy" ] && privacy=$DEFAULT_PRIVACY + + # Get title and description from metadata + local title description + if command -v jq >/dev/null 2>&1 && [ -f "$meta_file" ]; then + title=$(jq -r '.title // "Imported Video"' "$meta_file") + description=$(jq -r '.description // ""' "$meta_file" | head -c 10000) + else + title="Imported Video" + description="" + fi + + log_info "Title: $title" + log_info "Channel ID: $channel_id" + log_info "Privacy: $privacy" + + # Sanitize description (escape newlines and quotes) + description=$(echo "$description" | tr '\n' ' ' | sed 's/"/\\"/g' | head -c 5000) + + # Upload video - use temp file to avoid mixing stderr with response + local tmpfile="/tmp/peertube_upload_$$.json" + local http_code + + http_code=$(curl -s -w "%{http_code}" -o "$tmpfile" -X POST "${PEERTUBE_URL}/api/v1/videos/upload" \ + -H "Authorization: Bearer ${token}" \ + -F "videofile=@${video_file}" \ + -F "channelId=${channel_id}" \ + -F "name=${title}" \ + -F "privacy=${privacy}" \ + -F "description=${description}" \ + 2>/dev/null) + + local body="" + [ -f "$tmpfile" ] && body=$(cat "$tmpfile") + rm -f "$tmpfile" + + if [ "$http_code" = "200" ] || [ "$http_code" = "201" ]; then + local video_id video_uuid + if command -v jq >/dev/null 2>&1; then + video_id=$(echo "$body" | jq -r '.video.id // empty') + video_uuid=$(echo "$body" | jq -r '.video.uuid // empty') + else + video_id=$(echo "$body" | jsonfilter -e '@.video.id' 2>/dev/null) + video_uuid=$(echo "$body" | jsonfilter -e '@.video.uuid' 2>/dev/null) + fi + + if [ -n "$video_id" ]; then + log_ok "Video uploaded: ID=$video_id, UUID=$video_uuid" + echo "${video_id}|${video_uuid}" + return 0 + fi + fi + + log_error "Upload failed (HTTP $http_code)" + log_error "$body" + return 1 +} + +upload_subtitles_to_peertube() { + local video_id="$1" + local output_dir="$2" + local slug="$3" + local token="$4" + + log_step "Uploading subtitles to PeerTube" + + local uploaded=0 + + for vtt in "$output_dir"/${slug}*.vtt; do + [ -f "$vtt" ] || continue + + # Extract language code from filename + # Format: slug.fr.vtt or slug.en-US.vtt + local filename + filename=$(basename "$vtt") + local lang + lang=$(echo "$filename" | sed -E "s/${slug}\.([a-zA-Z-]+)\.vtt/\1/") + + # Normalize language code (en-US -> en) + lang=$(echo "$lang" | cut -d'-' -f1) + + log_info "Uploading subtitle: $lang ($filename)" + + local http_code + http_code=$(curl -s -w "%{http_code}" -o /dev/null -X PUT \ + "${PEERTUBE_URL}/api/v1/videos/${video_id}/captions/${lang}" \ + -H "Authorization: Bearer ${token}" \ + -F "captionfile=@${vtt}" \ + 2>/dev/null) + + if [ "$http_code" = "200" ] || [ "$http_code" = "204" ] || [ "$http_code" = "201" ]; then + log_ok "Uploaded: $lang" + uploaded=$((uploaded + 1)) + else + log_warn "Failed to upload $lang (HTTP $http_code)" + fi + done + + log_ok "Uploaded $uploaded subtitle(s)" + return 0 +} + +#============================================================================= +# MAIN IMPORT FUNCTION +#============================================================================= + +import_video() { + local url="$1" + local languages="$2" + local channel_id="$3" + local privacy="$4" + local username="$5" + local password="$6" + + log_step "Starting video import" + log_info "URL: $url" + + # Create output directory + # If OUTPUT_BASE already contains import_ (called from CGI), use it as-is + local output_dir + if echo "$OUTPUT_BASE" | grep -q "/import_"; then + output_dir="$OUTPUT_BASE" + else + output_dir="$OUTPUT_BASE/import_$(date +%s)_$$" + fi + mkdir -p "$output_dir" + + # Extract metadata first to get title + local meta_file + meta_file=$(extract_metadata "$url" "$output_dir" "video") || { + log_error "Metadata extraction failed" + return 1 + } + + # Generate slug from title + local title slug + if command -v jq >/dev/null 2>&1; then + title=$(jq -r '.title // "video"' "$meta_file") + else + title=$(jsonfilter -i "$meta_file" -e '@.title' 2>/dev/null || echo "video") + fi + slug=$(generate_slug "$title") + [ -z "$slug" ] && slug="video_$$" + + # Rename metadata file + mv "$meta_file" "$output_dir/${slug}.meta.json" + meta_file="$output_dir/${slug}.meta.json" + + log_progress "downloading" + + # Download video + local video_file + video_file=$(download_video "$url" "$output_dir" "$slug") || { + log_error "Video download failed" + return 1 + } + + # Download subtitles + download_subtitles "$url" "$output_dir" "$slug" "$languages" || { + log_warn "Subtitle download failed (continuing without subtitles)" + } + + log_progress "uploading" + + # Get PeerTube token + local token + token=$(get_peertube_token "$username" "$password") || { + log_error "Authentication failed" + return 1 + } + + # Upload video + local upload_result + upload_result=$(upload_video_to_peertube "$video_file" "$meta_file" "$channel_id" "$privacy" "$token") || { + log_error "Video upload failed" + return 1 + } + + local video_id video_uuid + video_id=$(echo "$upload_result" | cut -d'|' -f1) + video_uuid=$(echo "$upload_result" | cut -d'|' -f2) + + # Upload subtitles + upload_subtitles_to_peertube "$video_id" "$output_dir" "$slug" "$token" || { + log_warn "Subtitle upload failed" + } + + log_progress "completed" + + # Output result + log_step "Import complete" + log_ok "Video ID: $video_id" + log_ok "Video UUID: $video_uuid" + log_ok "URL: ${PEERTUBE_URL}/w/${video_uuid}" + + # Output JSON result + cat << EOF +{ + "success": true, + "video_id": $video_id, + "video_uuid": "$video_uuid", + "video_url": "${PEERTUBE_URL}/w/${video_uuid}", + "title": $(echo "$title" | jq -Rs . 2>/dev/null || echo "\"$title\""), + "output_dir": "$output_dir" +} +EOF + + return 0 +} + +#============================================================================= +# CLI PARSING +#============================================================================= + +show_help() { + cat << EOF +PeerTube Video Import with Subtitle Sync +SecuBox Intelligence Module v${SCRIPT_VERSION} + +Usage: + $(basename "$0") [OPTIONS] + +Options: + --url Video URL (YouTube, Vimeo, etc.) + --lang Subtitle languages (comma-separated: fr,en,de) + Default: fr,en + --channel PeerTube channel ID (default: 1) + --privacy 1=public, 2=unlisted, 3=private (default: 1) + --username PeerTube username (or set via UCI) + --password PeerTube password (or set via UCI) + --output Output directory (default: /tmp/peertube-import) + --peertube PeerTube instance URL + -h, --help Show this help message + +Environment Variables: + PEERTUBE_URL PeerTube instance URL + PEERTUBE_TOKEN OAuth access token (skip authentication) + +Examples: + # Basic import + $(basename "$0") https://youtube.com/watch?v=xxx + + # Import with multiple subtitle languages + $(basename "$0") --lang fr,en,de,es https://youtube.com/watch?v=xxx + + # Import as unlisted video + $(basename "$0") --privacy 2 https://youtube.com/watch?v=xxx + +EOF +} + +parse_args() { + VIDEO_URL="" + LANGUAGES="fr,en" + CHANNEL_ID="" + PRIVACY="" + USERNAME="" + PASSWORD="" + + while [ $# -gt 0 ]; do + case "$1" in + --url) + VIDEO_URL="$2" + shift 2 + ;; + --lang|--languages) + LANGUAGES="$2" + shift 2 + ;; + --channel) + CHANNEL_ID="$2" + shift 2 + ;; + --privacy) + PRIVACY="$2" + shift 2 + ;; + --username) + USERNAME="$2" + shift 2 + ;; + --password) + PASSWORD="$2" + shift 2 + ;; + --output) + OUTPUT_BASE="$2" + shift 2 + ;; + --peertube) + PEERTUBE_URL="$2" + shift 2 + ;; + -h|--help) + show_help + exit 0 + ;; + -*) + log_error "Unknown option: $1" + show_help + exit 1 + ;; + *) + # Positional argument = URL + if [ -z "$VIDEO_URL" ]; then + VIDEO_URL="$1" + else + log_error "Multiple URLs not supported" + exit 1 + fi + shift + ;; + esac + done + + if [ -z "$VIDEO_URL" ]; then + log_error "No video URL provided" + show_help + exit 1 + fi +} + +#============================================================================= +# ENTRY POINT +#============================================================================= + +main() { + parse_args "$@" + + echo >&2 "" + echo >&2 "╔══════════════════════════════════════════════════════╗" + echo >&2 "║ PeerTube Video Import v${SCRIPT_VERSION} ║" + echo >&2 "║ SecuBox Intelligence Module ║" + echo >&2 "╚══════════════════════════════════════════════════════╝" + echo >&2 "" + + # Check dependencies + for dep in yt-dlp curl; do + if ! command -v "$dep" >/dev/null 2>&1; then + log_error "Required dependency not found: $dep" + exit 1 + fi + done + + import_video "$VIDEO_URL" "$LANGUAGES" "$CHANNEL_ID" "$PRIVACY" "$USERNAME" "$PASSWORD" + exit $? +} + +# Run if not sourced +if [ "${0##*/}" = "peertube-import" ] || [ "${0##*/}" = "sh" ]; then + main "$@" +fi diff --git a/package/secubox/secubox-app-peertube/files/www/cgi-bin/peertube-import b/package/secubox/secubox-app-peertube/files/www/cgi-bin/peertube-import new file mode 100644 index 00000000..21102b98 --- /dev/null +++ b/package/secubox/secubox-app-peertube/files/www/cgi-bin/peertube-import @@ -0,0 +1,164 @@ +#!/bin/sh +# CGI endpoint for PeerTube video import +# Returns JSON response + +# Set headers +printf "Content-Type: application/json\r\n" +printf "Access-Control-Allow-Origin: *\r\n" +printf "Access-Control-Allow-Methods: POST, OPTIONS\r\n" +printf "Access-Control-Allow-Headers: Content-Type\r\n" +printf "\r\n" + +# Handle OPTIONS (CORS preflight) +if [ "$REQUEST_METHOD" = "OPTIONS" ]; then + exit 0 +fi + +# Only allow POST +if [ "$REQUEST_METHOD" != "POST" ]; then + echo '{"error": "Method not allowed"}' + exit 0 +fi + +# Read input +INPUT=$(cat) + +# Parse JSON (use jq if available, else jsonfilter) +if command -v jq >/dev/null 2>&1; then + URL=$(echo "$INPUT" | jq -r '.url // empty') + LANGUAGES=$(echo "$INPUT" | jq -r '.languages // "fr,en"') + CHANNEL_ID=$(echo "$INPUT" | jq -r '.channel_id // "1"') + PRIVACY=$(echo "$INPUT" | jq -r '.privacy // "1"') +else + URL=$(echo "$INPUT" | jsonfilter -e '@.url' 2>/dev/null) + LANGUAGES=$(echo "$INPUT" | jsonfilter -e '@.languages' 2>/dev/null) + CHANNEL_ID=$(echo "$INPUT" | jsonfilter -e '@.channel_id' 2>/dev/null) + PRIVACY=$(echo "$INPUT" | jsonfilter -e '@.privacy' 2>/dev/null) +fi + +# Handle array of languages +if echo "$LANGUAGES" | grep -q '^\['; then + if command -v jq >/dev/null 2>&1; then + LANGUAGES=$(echo "$INPUT" | jq -r '.languages | if type == "array" then join(",") else . end') + else + LANGUAGES="fr,en" + fi +fi + +# Validate URL +if [ -z "$URL" ]; then + echo '{"error": "URL is required"}' + exit 0 +fi + +# Sanitize URL (basic security check) +case "$URL" in + http://*|https://*) + # Valid URL prefix + ;; + *) + echo '{"error": "Invalid URL format"}' + exit 0 + ;; +esac + +# Set defaults +[ -z "$LANGUAGES" ] && LANGUAGES="fr,en" +[ -z "$CHANNEL_ID" ] && CHANNEL_ID="1" +[ -z "$PRIVACY" ] && PRIVACY="1" + +# Generate job ID +JOB_ID="import_$(date +%s)_$$" +STATUS_FILE="/tmp/peertube-import-${JOB_ID}.status" +RESULT_FILE="/tmp/peertube-import-${JOB_ID}.json" +LOG_FILE="/tmp/peertube-import-${JOB_ID}.log" +PROGRESS_FILE="/tmp/peertube-import-${JOB_ID}.progress" + +# Check for import script +if [ ! -x "/usr/sbin/peertube-import" ]; then + echo '{"error": "peertube-import not installed"}' + exit 0 +fi + +# Initialize status +echo "starting" > "$STATUS_FILE" +echo "0" > "$PROGRESS_FILE" + +# Start import in background +( + echo "downloading" > "$STATUS_FILE" + echo "10" > "$PROGRESS_FILE" + + # Run the import + OUTPUT=$(/usr/sbin/peertube-import \ + --lang "$LANGUAGES" \ + --channel "$CHANNEL_ID" \ + --privacy "$PRIVACY" \ + "$URL" 2>&1) + RC=$? + + echo "$OUTPUT" >> "$LOG_FILE" + + if [ $RC -eq 0 ]; then + echo "completed" > "$STATUS_FILE" + echo "100" > "$PROGRESS_FILE" + + # Extract JSON result from output (last JSON block) + local result_json + result_json=$(echo "$OUTPUT" | grep -E '^\{' | tail -1) + + if [ -n "$result_json" ]; then + # Parse and rebuild result + if command -v jq >/dev/null 2>&1; then + VIDEO_ID=$(echo "$result_json" | jq -r '.video_id // empty') + VIDEO_UUID=$(echo "$result_json" | jq -r '.video_uuid // empty') + VIDEO_URL=$(echo "$result_json" | jq -r '.video_url // empty') + TITLE=$(echo "$result_json" | jq -r '.title // "Imported Video"') + else + VIDEO_ID=$(echo "$result_json" | jsonfilter -e '@.video_id' 2>/dev/null) + VIDEO_UUID=$(echo "$result_json" | jsonfilter -e '@.video_uuid' 2>/dev/null) + VIDEO_URL=$(echo "$result_json" | jsonfilter -e '@.video_url' 2>/dev/null) + TITLE="Imported Video" + fi + + cat > "$RESULT_FILE" << EOF +{ + "success": true, + "job_id": "$JOB_ID", + "video_id": $VIDEO_ID, + "video_uuid": "$VIDEO_UUID", + "video_url": "$VIDEO_URL", + "title": "$TITLE", + "status": "completed" +} +EOF + else + cat > "$RESULT_FILE" << EOF +{ + "success": true, + "job_id": "$JOB_ID", + "status": "completed", + "message": "Import completed but no result JSON found" +} +EOF + fi + else + echo "failed" > "$STATUS_FILE" + echo "0" > "$PROGRESS_FILE" + + # Get last error from log + ERROR_MSG=$(tail -5 "$LOG_FILE" 2>/dev/null | tr '\n' ' ' | sed 's/"/\\"/g' | head -c 500) + + cat > "$RESULT_FILE" << EOF +{ + "success": false, + "job_id": "$JOB_ID", + "status": "failed", + "error": "$ERROR_MSG" +} +EOF + fi +) & + +# Return job ID for polling +echo "{\"success\": true, \"message\": \"Import started\", \"job_id\": \"$JOB_ID\"}" diff --git a/package/secubox/secubox-app-peertube/files/www/cgi-bin/peertube-import-status b/package/secubox/secubox-app-peertube/files/www/cgi-bin/peertube-import-status new file mode 100644 index 00000000..13cd82ff --- /dev/null +++ b/package/secubox/secubox-app-peertube/files/www/cgi-bin/peertube-import-status @@ -0,0 +1,69 @@ +#!/bin/sh +# CGI endpoint for checking PeerTube import status +# Returns JSON response + +# Set headers +printf "Content-Type: application/json\r\n" +printf "Access-Control-Allow-Origin: *\r\n" +printf "\r\n" + +# Get job_id from query string or POST body +JOB_ID="" + +if [ -n "$QUERY_STRING" ]; then + JOB_ID=$(echo "$QUERY_STRING" | sed -n 's/.*job_id=\([^&]*\).*/\1/p') +fi + +if [ -z "$JOB_ID" ] && [ "$REQUEST_METHOD" = "POST" ]; then + INPUT=$(cat) + if command -v jq >/dev/null 2>&1; then + JOB_ID=$(echo "$INPUT" | jq -r '.job_id // empty') + else + JOB_ID=$(echo "$INPUT" | jsonfilter -e '@.job_id' 2>/dev/null) + fi +fi + +if [ -z "$JOB_ID" ]; then + echo '{"error": "job_id is required"}' + exit 0 +fi + +# Sanitize job_id (only allow alphanumeric and underscore) +JOB_ID=$(echo "$JOB_ID" | tr -cd 'a-zA-Z0-9_') + +STATUS_FILE="/tmp/peertube-import-${JOB_ID}.status" +RESULT_FILE="/tmp/peertube-import-${JOB_ID}.json" +LOG_FILE="/tmp/peertube-import-${JOB_ID}.log" +PROGRESS_FILE="/tmp/peertube-import-${JOB_ID}.progress" + +# Check if job exists +if [ ! -f "$STATUS_FILE" ]; then + echo '{"error": "Job not found", "job_id": "'"$JOB_ID"'"}' + exit 0 +fi + +STATUS=$(cat "$STATUS_FILE" 2>/dev/null || echo "unknown") +PROGRESS=$(cat "$PROGRESS_FILE" 2>/dev/null || echo "0") + +# If completed or failed, return full result +if [ -f "$RESULT_FILE" ]; then + if [ "$STATUS" = "completed" ] || [ "$STATUS" = "failed" ]; then + cat "$RESULT_FILE" + exit 0 + fi +fi + +# Otherwise return status with progress and logs +LOGS="" +if [ -f "$LOG_FILE" ]; then + LOGS=$(tail -3 "$LOG_FILE" 2>/dev/null | tr '\n' ' ' | sed 's/"/\\"/g' | head -c 300) +fi + +cat << EOF +{ + "status": "$STATUS", + "job_id": "$JOB_ID", + "progress": $PROGRESS, + "logs": "$LOGS" +} +EOF