joachim/FileRise
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

release(v1.8.8): background ZIP jobs w/ tokenized download + in‑modal progress bar; robust finalize; janitor cleanup — closes #60

Browse Source
This commit is contained in:
Ryan
2025-11-07 02:57:30 -05:00
committed by GitHub
parent dc45fed886
commit ca8788a694
9 changed files with 760 additions and 168 deletions
Download Patch File Download Diff File Expand all files Collapse all files

50
CHANGELOG.md
View File

@@ -1,5 +1,55 @@
# Changelog
## Changes 11/7/2025 (v1.8.8)
release(v1.8.8): background ZIP jobs w/ tokenized download + in‑modal progress bar; robust finalize; janitor cleanup — closes #60
**Summary**
This release moves ZIP creation off the request thread into a **background worker** and switches the client to a **queue > poll > tokenized GET** download flow. It fixes large multi‑GB ZIP failures caused by request timeouts or cross‑device renames, and provides a resilient in‑modal progress experience. It also adds a 6‑hour janitor for temporary tokens/logs.
**Backend** changes:
- Add **zip status** endpoint that returns progress and readiness, and **tokenized download** endpoint for one‑shot downloads.
- Update `FileController::downloadZip()` to enqueue a job and return `{ token, statusUrl, downloadUrl }` instead of streaming a blob in the POST response.
- Implement `spawnZipWorker()` to find a working PHP CLI, set `TMPDIR` on the same filesystem as the final ZIP, spawn with `nohup`, and persist PID/log metadata for diagnostics.
- Serve finished ZIPs via `downloadZipFile()` with strict token/user checks and streaming headers; unlink the ZIP after successful read.
New **Worker**:
- New `src/cli/zip_worker.php` builds the archive in the background.
- Writes progress fields (`pct`, `filesDone`, `filesTotal`, `bytesDone`, `bytesTotal`, `current`, `phase`, `startedAt`, `finalizeAt`) to the per‑token JSON.
- During **finalizing**, publishes `selectedFiles`/`selectedBytes` and clears incremental counters to avoid the confusing “N/N files” display before `close()` returns.
- Adds a **janitor**: purge `.tokens/*.json` and `.logs/WORKER-*.log` older than **6 hours** on each run.
New **API/Status Payload**:
- `zipStatus()` exposes `ready` (derived from `status=done` + existing `zipPath`), and includes `startedAt`/`finalizeAt` for UI timers.
- Returns a prebuilt `downloadUrl` for a direct handoff once the ZIP is ready.
**Frontend (UX)** changes:
- Replace blob POST download with **enqueue → poll → tokenized GET** flow.
- Native `<progress>` bar now renders **inside the modal** (no overflow/jitter).
- Shows determinate **0–98%** during enumeration, then **locks at 100%** with **“Finalizing… mm:ss — N files, ~Size”** until the download starts.
- Modal closes just before download; UI resets for the next operation.
Added **CSS**:
- Ensure the progress modal has a minimum height and hidden overflow; ellipsize the status line to prevent scrollbars.
**Why this closes #60**?
- ZIP creation no longer depends on the request lifetime (avoids proxy/Apache timeouts).
- Temporary files and final ZIP are created on the **same filesystem** (prevents “rename temp file failed” during `ZipArchive::close()`).
- Users get continuous, truthful feedback for large multi‑GB archives.
Additional **Notes**
- Download tokens are **one‑shot** and are deleted after the GET completes.
- Temporary artifacts (`META_DIR/ziptmp/.tokens`, `.logs`, and old ZIPs) are cleaned up automatically (≥6h).
---
## Changes 11/5/2025 (v1.8.7)
release(v1.8.7): fix(zip-download): stream clean ZIP response and purge stale temp archives

31
public/.htaccess
View File

@@ -1,12 +1,13 @@
# --------------------------------
# FileRise portable .htaccess
# --------------------------------
Options -Indexes
Options -Indexes -Multiviews
DirectoryIndex index.html
# ---------------- Security: dotfiles ----------------
<IfModule mod_authz_core.c>
# Block dotfiles like .env, .git, etc., but allow ACME under .well-known
<FilesMatch "^\.(?!well-known(?:/|$))">
# Block direct access to dotfiles like .env, .gitignore, etc.
<FilesMatch "^\..*">
Require all denied
</FilesMatch>
</IfModule>
@@ -15,15 +16,24 @@ DirectoryIndex index.html
<IfModule mod_rewrite.c>
RewriteEngine On
# Never redirect local/dev hosts
RewriteCond %{HTTP_HOST} ^(localhost|127\.0\.0\.1|fr\.local|192\.168\.[0-9]+\.[0-9]+)$ [NC]
RewriteRule ^ - [L]
# Let ACME http-01 pass BEFORE any redirect (needed for auto-renew)
# 0) Let ACME http-01 pass BEFORE any other rule (needed for auto-renew)
RewriteCond %{REQUEST_URI} ^/.well-known/acme-challenge/
RewriteRule - - [L]
# HTTPS redirect (enable ONE of these, comment the other)
# 1) Block hidden files/dirs anywhere EXCEPT .well-known (path-aware)
# Prevents requests like /.env, /.git/config, /.ssh/id_rsa, etc.
RewriteRule "(^|/)\.(?!well-known/)" - [F]
# 2) Deny direct access to PHP outside /api/
# This stops scanners from hitting /index.php, /admin.php, /wso.php, etc.
RewriteCond %{REQUEST_URI} !^/api/
RewriteRule \.php$ - [F]
# 3) Never redirect local/dev hosts
RewriteCond %{HTTP_HOST} ^(localhost|127\.0\.0\.1|fr\.local|192\.168\.[0-9]+\.[0-9]+)$ [NC]
RewriteRule ^ - [L]
# 4) HTTPS redirect (enable ONE of these, comment the other)
# A) Direct TLS on this server
#RewriteCond %{HTTPS} !=on
@@ -35,7 +45,7 @@ RewriteRule - - [L]
#RewriteCond %{HTTPS} !=on
#RewriteRule ^ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301]
# Mark versioned assets (?v=...) with env flag for caching rules below
# 5) Mark versioned assets (?v=...) with env flag for caching rules below
RewriteCond %{QUERY_STRING} (^|&)v= [NC]
RewriteRule ^ - [E=IS_VER:1]
</IfModule>
@@ -98,7 +108,6 @@ RewriteRule ^ - [E=IS_VER:1]
# ---------------- Compression ----------------
<IfModule mod_brotli.c>
# Do NOT set BrotliCompressionQuality in .htaccess (vhost/server only)
AddOutputFilterByType BROTLI_COMPRESS text/html text/css application/javascript application/json image/svg+xml
</IfModule>
<IfModule mod_deflate.c>

24
public/api/file/downloadZipFile.php Normal file
View File

@@ -0,0 +1,24 @@
<?php
// public/api/file/downloadZipFile.php
/**
* @OA\Get(
* path="/api/file/downloadZipFile.php",
* summary="Download a finished ZIP by token",
* description="Streams the zip once; token is one-shot.",
* operationId="downloadZipFile",
* tags={"Files"},
* security={{"cookieAuth": {}}},
* @OA\Parameter(name="k", in="query", required=true, @OA\Schema(type="string"), description="Job token"),
* @OA\Parameter(name="name", in="query", required=false, @OA\Schema(type="string"), description="Suggested filename"),
* @OA\Response(response=200, description="ZIP stream"),
* @OA\Response(response=401, description="Unauthorized"),
* @OA\Response(response=404, description="Not found")
* )
*/
require_once __DIR__ . '/../../../config/config.php';
require_once PROJECT_ROOT . '/src/controllers/FileController.php';
$controller = new FileController();
$controller->downloadZipFile();

23
public/api/file/zipStatus.php Normal file
View File

@@ -0,0 +1,23 @@
<?php
// public/api/file/zipStatus.php
/**
* @OA\Get(
* path="/api/file/zipStatus.php",
* summary="Check status of a background ZIP build",
* description="Returns status for the authenticated user's token.",
* operationId="zipStatus",
* tags={"Files"},
* security={{"cookieAuth": {}}},
* @OA\Parameter(name="k", in="query", required=true, @OA\Schema(type="string"), description="Job token"),
* @OA\Response(response=200, description="Status payload"),
* @OA\Response(response=401, description="Unauthorized"),
* @OA\Response(response=404, description="Not found")
* )
*/
require_once __DIR__ . '/../../../config/config.php';
require_once PROJECT_ROOT . '/src/controllers/FileController.php';
$controller = new FileController();
$controller->zipStatus();

16
public/css/styles.css
View File

@@ -1925,4 +1925,18 @@ body {
.status-badge.progress {
border-color: rgba(250,204,21,.35); /* amber-ish */
background: rgba(250,204,21,.15);
}
}
#downloadProgressModal .modal-body,
#downloadProgressModal .rise-modal-body,
#downloadProgressModal .modal-content {
min-height: 88px;
overflow: hidden;
}
#downloadProgressText {
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}
#downloadProgressBarOuter { height: 10px; }

238
public/js/fileActions.js
View File

@@ -119,7 +119,7 @@ export async function handleCreateFile(e) {
method: 'POST',
credentials: 'include',
headers: {
'Content-Type':'application/json',
'Content-Type': 'application/json',
'X-CSRF-Token': window.csrfToken
},
// ⚠️ must send `name`, not `filename`
@@ -139,7 +139,7 @@ export async function handleCreateFile(e) {
document.addEventListener('DOMContentLoaded', () => {
const cancel = document.getElementById('cancelCreateFile');
const confirm = document.getElementById('confirmCreateFile');
if (cancel) cancel.addEventListener('click', () => document.getElementById('createFileModal').style.display = 'none');
if (cancel) cancel.addEventListener('click', () => document.getElementById('createFileModal').style.display = 'none');
if (confirm) confirm.addEventListener('click', handleCreateFile);
});
@@ -265,7 +265,7 @@ document.addEventListener("DOMContentLoaded", () => {
const cancelZipBtn = document.getElementById("cancelDownloadZip");
const confirmZipBtn = document.getElementById("confirmDownloadZip");
const cancelCreate = document.getElementById('cancelCreateFile');
if (cancelCreate) {
cancelCreate.addEventListener('click', () => {
document.getElementById('createFileModal').style.display = 'none';
@@ -305,7 +305,7 @@ document.addEventListener("DOMContentLoaded", () => {
showToast(err.message || t('error_creating_file'));
}
});
attachEnterKeyListener('createFileModal','confirmCreateFile');
attachEnterKeyListener('createFileModal', 'confirmCreateFile');
}
// 1) Cancel button hides the name modal
@@ -321,63 +321,187 @@ document.addEventListener("DOMContentLoaded", () => {
confirmZipBtn.addEventListener("click", async () => {
// a) Validate ZIP filename
let zipName = document.getElementById("zipFileNameInput").value.trim();
if (!zipName) {
showToast("Please enter a name for the zip file.");
return;
}
if (!zipName.toLowerCase().endsWith(".zip")) {
zipName += ".zip";
}
if (!zipName) { showToast("Please enter a name for the zip file."); return; }
if (!zipName.toLowerCase().endsWith(".zip")) zipName += ".zip";
// b) Hide the name‐input modal, show the spinner modal
// b) Hide the name‐input modal, show the progress modal
zipNameModal.style.display = "none";
progressModal.style.display = "block";
// c) (Optional) update the “Preparing…” text if you gave it an ID
// c) Title text (optional)
const titleEl = document.getElementById("downloadProgressTitle");
if (titleEl) titleEl.textContent = `Preparing ${zipName}…`;
try {
// d) POST and await the ZIP blob
const res = await fetch("/api/file/downloadZip.php", {
method: "POST",
credentials: "include",
headers: {
"Content-Type": "application/json",
"X-CSRF-Token": window.csrfToken
},
body: JSON.stringify({
folder: window.currentFolder || "root",
files: window.filesToDownload
})
});
if (!res.ok) {
const txt = await res.text();
throw new Error(txt || `Status ${res.status}`);
}
const blob = await res.blob();
if (!blob || blob.size === 0) {
throw new Error("Received empty ZIP file.");
}
// e) Hand off to the browser’s download manager
const url = URL.createObjectURL(blob);
const a = document.createElement("a");
a.href = url;
a.download = zipName;
document.body.appendChild(a);
a.click();
URL.revokeObjectURL(url);
a.remove();
} catch (err) {
console.error("Error downloading ZIP:", err);
showToast("Error: " + err.message);
} finally {
// f) Always hide spinner modal
progressModal.style.display = "none";
// d) Queue the job
const res = await fetch("/api/file/downloadZip.php", {
method: "POST",
credentials: "include",
headers: { "Content-Type": "application/json", "X-CSRF-Token": window.csrfToken },
body: JSON.stringify({ folder: window.currentFolder || "root", files: window.filesToDownload })
});
const jsr = await res.json().catch(() => ({}));
if (!res.ok || !jsr.ok) {
const msg = (jsr && jsr.error) ? jsr.error : `Status ${res.status}`;
throw new Error(msg);
}
const token = jsr.token;
const statusUrl = jsr.statusUrl;
const downloadUrl = jsr.downloadUrl + "&name=" + encodeURIComponent(zipName);
// Ensure a progress UI exists in the modal
function ensureZipProgressUI() {
const modalEl = document.getElementById("downloadProgressModal");
if (!modalEl) {
// really shouldn't happen, but fall back to body
console.warn("downloadProgressModal not found; falling back to document.body");
}
// Prefer a dedicated content node inside the modal
let host =
(modalEl && modalEl.querySelector("#downloadProgressContent")) ||
(modalEl && modalEl.querySelector(".modal-body")) ||
(modalEl && modalEl.querySelector(".rise-modal-body")) ||
(modalEl && modalEl.querySelector(".modal-content")) ||
(modalEl && modalEl.querySelector(".content")) ||
null;
// If no suitable container, create one inside the modal
if (!host) {
host = document.createElement("div");
host.id = "downloadProgressContent";
(modalEl || document.body).appendChild(host);
}
// Helper: ensure/move an element with given id into host
function ensureInHost(id, tag, init) {
let el = document.getElementById(id);
if (el && el.parentElement !== host) host.appendChild(el); // move if it exists elsewhere
if (!el) {
el = document.createElement(tag);
el.id = id;
if (typeof init === "function") init(el);
host.appendChild(el);
}
return el;
}
// Title
const title = ensureInHost("downloadProgressTitle", "div", (el) => {
el.style.marginBottom = "8px";
el.textContent = "Preparing…";
});
// Progress bar (native <progress>)
const bar = (function () {
let el = document.getElementById("downloadProgressBar");
if (el && el.parentElement !== host) host.appendChild(el); // move into modal
if (!el) {
el = document.createElement("progress");
el.id = "downloadProgressBar";
host.appendChild(el);
}
el.max = 100;
el.value = 0;
el.style.display = ""; // override any inline display:none
el.style.width = "100%";
el.style.height = "1.1em";
return el;
})();
// Text line
const text = ensureInHost("downloadProgressText", "div", (el) => {
el.style.marginTop = "8px";
el.style.fontSize = "0.9rem";
el.style.whiteSpace = "nowrap";
el.style.overflow = "hidden";
el.style.textOverflow = "ellipsis";
});
// Optional spinner hider
const hideSpinner = () => {
const sp = document.getElementById("downloadSpinner");
if (sp) sp.style.display = "none";
};
return { bar, text, title, hideSpinner };
}
function humanBytes(n) {
if (!Number.isFinite(n) || n < 0) return "";
const u = ["B", "KB", "MB", "GB", "TB"]; let i = 0, x = n;
while (x >= 1024 && i < u.length - 1) { x /= 1024; i++; }
return x.toFixed(x >= 10 || i === 0 ? 0 : 1) + " " + u[i];
}
function mmss(sec) {
sec = Math.max(0, sec | 0);
const m = (sec / 60) | 0, s = sec % 60;
return `${m}:${s.toString().padStart(2, '0')}`;
}
const ui = ensureZipProgressUI();
const t0 = Date.now();
// e) Poll until ready
while (true) {
await new Promise(r => setTimeout(r, 1200));
const s = await fetch(`${statusUrl}&_=${Date.now()}`, {
credentials: "include", cache: "no-store",
}).then(r => r.json());
if (s.error) throw new Error(s.error);
if (ui.title) ui.title.textContent = `Preparing ${zipName}…`;
// --- RENDER PROGRESS ---
if (typeof s.pct === "number" && ui.bar && ui.text) {
if ((s.phase !== 'finalizing') && (s.pct < 99)) {
ui.hideSpinner && ui.hideSpinner();
const filesDone = s.filesDone ?? 0;
const filesTotal = s.filesTotal ?? 0;
const bytesDone = s.bytesDone ?? 0;
const bytesTotal = s.bytesTotal ?? 0;
// Determinate 0–98% while enumerating
const pct = Math.max(0, Math.min(98, s.pct | 0));
if (!ui.bar.hasAttribute("value")) ui.bar.value = 0;
ui.bar.value = pct;
ui.text.textContent =
`${pct}% — ${filesDone}/${filesTotal} files, ${humanBytes(bytesDone)} / ${humanBytes(bytesTotal)}`;
} else {
// FINALIZING: keep progress at 100% and show timer + selected totals
if (!ui.bar.hasAttribute("value")) ui.bar.value = 100;
ui.bar.value = 100; // lock at 100 during finalizing
const since = s.finalizeAt ? Math.max(0, (Date.now() / 1000 | 0) - (s.finalizeAt | 0)) : 0;
const selF = s.selectedFiles ?? s.filesTotal ?? 0;
const selB = s.selectedBytes ?? s.bytesTotal ?? 0;
ui.text.textContent = `Finalizing… ${mmss(since)} — ${selF} file${selF === 1 ? '' : 's'}, ~${humanBytes(selB)}`;
}
} else if (ui.text) {
ui.text.textContent = "Still preparing…";
}
// --- /RENDER ---
if (s.ready) {
// Snap to 100 and close modal just before download
if (ui.bar) { ui.bar.max = 100; ui.bar.value = 100; }
progressModal.style.display = "none";
await new Promise(r => setTimeout(r, 0));
break;
}
if (Date.now() - t0 > 15 * 60 * 1000) throw new Error("Timed out preparing ZIP");
}
// f) Trigger download
const a = document.createElement("a");
a.href = downloadUrl;
a.download = zipName;
a.style.display = "none";
document.body.appendChild(a);
a.click();
a.remove();
// g) Reset for next time
if (ui.bar) ui.bar.value = 0;
if (ui.text) ui.text.textContent = "";
if (Array.isArray(window.filesToDownload)) window.filesToDownload = [];
});
}
});
@@ -694,10 +818,10 @@ document.addEventListener("DOMContentLoaded", () => {
});
document.addEventListener('DOMContentLoaded', () => {
const btn = document.getElementById('createBtn');
const menu = document.getElementById('createMenu');
const fileOpt = document.getElementById('createFileOption');
const folderOpt= document.getElementById('createFolderOption');
const btn = document.getElementById('createBtn');
const menu = document.getElementById('createMenu');
const fileOpt = document.getElementById('createFileOption');
const folderOpt = document.getElementById('createFolderOption');
// Toggle dropdown on click
btn.addEventListener('click', (e) => {

179
src/cli/zip_worker.php Normal file
View File

@@ -0,0 +1,179 @@
#!/usr/bin/env php
<?php
declare(strict_types=1);
require __DIR__ . '/../../config/config.php';
require __DIR__ . '/../../src/models/FileModel.php';
$token = $argv[1] ?? '';
$token = preg_replace('/[^a-f0-9]/','',$token);
if ($token === '') { fwrite(STDERR, "No token\n"); exit(1); }
$root = rtrim((string)META_DIR, '/\\') . '/ziptmp';
$tokDir = $root . '/.tokens';
$logDir = $root . '/.logs';
@mkdir($tokDir, 0775, true);
@mkdir($logDir, 0775, true);
$tokFile = $tokDir . '/' . $token . '.json';
$logFile = $logDir . '/WORKER-' . $token . '.log';
file_put_contents($logFile, "[".date('c')."] worker start token={$token}\n", FILE_APPEND);
// Keep libzip temp files on same FS as final zip (prevents cross-device rename failures)
@mkdir($root, 0775, true);
@putenv('TMPDIR='.$root);
@ini_set('sys_temp_dir', $root);
// Small janitor: purge old tokens/logs (> 6h)
$now = time();
foreach (glob($tokDir.'/*.json') ?: [] as $f) { if (is_file($f) && ($now - @filemtime($f)) > 21600) @unlink($f); }
foreach (glob($logDir.'/WORKER-*.log') ?: [] as $f) { if (is_file($f) && ($now - @filemtime($f)) > 21600) @unlink($f); }
// Helpers to read/write the token file safely
$job = json_decode((string)@file_get_contents($tokFile), true) ?: [];
$save = function() use (&$job, $tokFile) {
@file_put_contents($tokFile, json_encode($job, JSON_PRETTY_PRINT), LOCK_EX);
@clearstatcache(true, $tokFile);
};
$touchPhase = function(string $phase) use (&$job, $save) {
$job['phase'] = $phase;
$save();
};
// Init timing
if (empty($job['startedAt'])) {
$job['startedAt'] = time();
}
$job['status'] = 'working';
$job['error'] = null;
$save();
// Build the list of files to zip using the model (same validation FileRise uses)
try {
// Reuse FileModel’s validation by calling it but not keeping the zip; we’ll enumerate sizes here.
$folder = (string)($job['folder'] ?? 'root');
$names = (array)($job['files'] ?? []);
// Resolve folder path similarly to createZipArchive
$baseDir = realpath(UPLOAD_DIR);
if ($baseDir === false) {
throw new RuntimeException('Uploads directory not configured correctly.');
}
if (strtolower($folder) === 'root' || $folder === "") {
$folderPathReal = $baseDir;
} else {
if (strpos($folder, '..') !== false) throw new RuntimeException('Invalid folder name.');
$parts = explode('/', trim($folder, "/\\ "));
foreach ($parts as $part) {
if ($part === '' || !preg_match(REGEX_FOLDER_NAME, $part)) {
throw new RuntimeException('Invalid folder name.');
}
}
$folderPath = rtrim(UPLOAD_DIR, '/\\') . DIRECTORY_SEPARATOR . implode(DIRECTORY_SEPARATOR, $parts);
$folderPathReal = realpath($folderPath);
if ($folderPathReal === false || strpos($folderPathReal, $baseDir) !== 0) {
throw new RuntimeException('Folder not found.');
}
}
// Collect files (only regular files)
$filesToZip = [];
foreach ($names as $nm) {
$bn = basename(trim((string)$nm));
if (!preg_match(REGEX_FILE_NAME, $bn)) continue;
$fp = $folderPathReal . DIRECTORY_SEPARATOR . $bn;
if (is_file($fp)) $filesToZip[] = $fp;
}
if (!$filesToZip) throw new RuntimeException('No valid files to zip.');
// Totals for progress
$filesTotal = count($filesToZip);
$bytesTotal = 0;
foreach ($filesToZip as $fp) {
$sz = @filesize($fp);
if ($sz !== false) $bytesTotal += (int)$sz;
}
$job['filesTotal'] = $filesTotal;
$job['bytesTotal'] = $bytesTotal;
$job['filesDone'] = 0;
$job['bytesDone'] = 0;
$job['pct'] = 0;
$job['current'] = null;
$job['phase'] = 'zipping';
$save();
// Create final zip path in META_DIR/ziptmp
$zipName = 'download-' . date('Ymd-His') . '-' . bin2hex(random_bytes(4)) . '.zip';
$zipPath = $root . DIRECTORY_SEPARATOR . $zipName;
$zip = new ZipArchive();
if ($zip->open($zipPath, ZipArchive::CREATE | ZipArchive::OVERWRITE) !== true) {
throw new RuntimeException('Could not create zip archive.');
}
// Enumerate files; report up to 98%
$bytesDone = 0;
$filesDone = 0;
foreach ($filesToZip as $fp) {
$bn = basename($fp);
$zip->addFile($fp, $bn);
$filesDone++;
$sz = @filesize($fp);
if ($sz !== false) $bytesDone += (int)$sz;
$job['filesDone'] = $filesDone;
$job['bytesDone'] = $bytesDone;
$job['current'] = $bn;
$pct = ($bytesTotal > 0) ? (int) floor(($bytesDone / $bytesTotal) * 98) : 0;
if ($pct < 0) $pct = 0;
if ($pct > 98) $pct = 98;
if ($pct > (int)($job['pct'] ?? 0)) $job['pct'] = $pct;
$save();
}
// Finalizing (this is where libzip writes & renames)
$job['pct'] = max((int)($job['pct'] ?? 0), 99);
$job['phase'] = 'finalizing';
$job['finalizeAt'] = time();
// Publish selected totals for a truthful UI during finalizing,
// and clear incremental fields so the UI doesn't show "7/7 14 GB / 14 GB" prematurely.
$job['selectedFiles'] = $filesTotal;
$job['selectedBytes'] = $bytesTotal;
$job['filesDone'] = null;
$job['bytesDone'] = null;
$job['current'] = null;
$save();
// ---- finalize the zip on disk ----
$ok = $zip->close();
$statusStr = method_exists($zip, 'getStatusString') ? $zip->getStatusString() : '';
if (!$ok || !is_file($zipPath)) {
$job['status'] = 'error';
$job['error'] = 'Failed to finalize ZIP' . ($statusStr ? " ($statusStr)" : '');
$save();
file_put_contents($logFile, "[".date('c')."] error: ".$job['error']."\n", FILE_APPEND);
exit(0);
}
$job['status'] = 'done';
$job['zipPath'] = $zipPath;
$job['pct'] = 100;
$job['phase'] = 'finalized';
$save();
file_put_contents($logFile, "[".date('c')."] done zip={$zipPath}\n", FILE_APPEND);
} catch (Throwable $e) {
$job['status'] = 'error';
$job['error'] = 'Worker exception: '.$e->getMessage();
$save();
file_put_contents($logFile, "[".date('c')."] exception: ".$e->getMessage()."\n", FILE_APPEND);
}

346
src/controllers/FileController.php
View File

@@ -190,6 +190,59 @@ class FileController
return $ok ? null : "Forbidden: folder scope violation.";
}
private function spawnZipWorker(string $token, string $tokFile, string $logDir): array
{
$worker = realpath(PROJECT_ROOT . '/src/cli/zip_worker.php');
if (!$worker || !is_file($worker)) {
return ['ok'=>false, 'error'=>'zip_worker.php not found'];
}
// Find a PHP CLI binary that actually works
$candidates = array_values(array_filter([
PHP_BINARY ?: null,
'/usr/local/bin/php',
'/usr/bin/php',
'/bin/php'
]));
$php = null;
foreach ($candidates as $bin) {
if (!$bin) continue;
$rc = 1;
@exec(escapeshellcmd($bin).' -v >/dev/null 2>&1', $o, $rc);
if ($rc === 0) { $php = $bin; break; }
}
if (!$php) {
return ['ok'=>false, 'error'=>'No working php CLI found'];
}
$logFile = $logDir . DIRECTORY_SEPARATOR . 'WORKER-' . $token . '.log';
// Ensure TMPDIR is on the same FS as the final zip; actually apply it to the child process.
$tmpDir = rtrim((string)META_DIR, '/\\') . '/ziptmp';
@mkdir($tmpDir, 0775, true);
// Build one sh -c string so env + nohup + echo $! are in the same shell
$cmdStr =
'export TMPDIR=' . escapeshellarg($tmpDir) . ' ; ' .
'nohup ' . escapeshellcmd($php) . ' ' . escapeshellarg($worker) . ' ' . escapeshellarg($token) .
' >> ' . escapeshellarg($logFile) . ' 2>&1 & echo $!';
$pid = @shell_exec('/bin/sh -c ' . escapeshellarg($cmdStr));
$pid = is_string($pid) ? (int)trim($pid) : 0;
// Persist spawn metadata into token (best-effort)
$job = json_decode((string)@file_get_contents($tokFile), true) ?: [];
$job['spawn'] = [
'ts' => time(),
'php' => $php,
'pid' => $pid,
'log' => $logFile
];
@file_put_contents($tokFile, json_encode($job, JSON_PRETTY_PRINT), LOCK_EX);
return $pid > 0 ? ['ok'=>true] : ['ok'=>false, 'error'=>'spawn returned no PID'];
}
// --- small helpers ---
private function _jsonStart(): void {
if (session_status() !== PHP_SESSION_ACTIVE) session_start();
@@ -665,99 +718,214 @@ public function deleteFiles()
exit;
}
public function downloadZip()
{
try {
if (!$this->_checkCsrf()) { http_response_code(400); echo "Bad CSRF"; return; }
if (!$this->_requireAuth()) { http_response_code(401); echo "Unauthorized"; return; }
$data = $this->_readJsonBody();
if (!is_array($data) || !isset($data['folder'], $data['files']) || !is_array($data['files'])) {
http_response_code(400); echo "Invalid input."; return;
}
$folder = $this->_normalizeFolder($data['folder']);
$files = $data['files'];
if (!$this->_validFolder($folder)) { http_response_code(400); echo "Invalid folder name."; return; }
$username = $_SESSION['username'] ?? '';
$perms = $this->loadPerms($username);
// Optional zip gate by account flag
if (!$this->isAdmin($perms) && !empty($perms['disableZip'])) {
http_response_code(403); echo "ZIP downloads are not allowed for your account."; return;
}
$ignoreOwnership = $this->isAdmin($perms)
|| ($perms['bypassOwnership'] ?? (defined('DEFAULT_BYPASS_OWNERSHIP') ? DEFAULT_BYPASS_OWNERSHIP : false));
// Ancestor-owner counts as full view
$fullView = $ignoreOwnership
|| ACL::canRead($username, $perms, $folder)
|| $this->ownsFolderOrAncestor($folder, $username, $perms);
$ownOnly = !$fullView && ACL::hasGrant($username, $folder, 'read_own');
if (!$fullView && !$ownOnly) { http_response_code(403); echo "Forbidden: no view access to this folder."; return; }
if ($ownOnly) {
$meta = $this->loadFolderMetadata($folder);
foreach ($files as $f) {
$bn = basename((string)$f);
if (!isset($meta[$bn]['uploader']) || strcasecmp((string)$meta[$bn]['uploader'], $username) !== 0) {
http_response_code(403); echo "Forbidden: you are not the owner of '{$bn}'."; return;
}
public function zipStatus()
{
if (!$this->_requireAuth()) { http_response_code(401); header('Content-Type: application/json'); echo json_encode(["error"=>"Unauthorized"]); return; }
$username = $_SESSION['username'] ?? '';
$token = isset($_GET['k']) ? preg_replace('/[^a-f0-9]/','',(string)$_GET['k']) : '';
if ($token === '' || strlen($token) < 8) { http_response_code(400); header('Content-Type: application/json'); echo json_encode(["error"=>"Bad token"]); return; }
$tokFile = rtrim((string)META_DIR, '/\\') . '/ziptmp/.tokens/' . $token . '.json';
if (!is_file($tokFile)) { http_response_code(404); header('Content-Type: application/json'); echo json_encode(["error"=>"Not found"]); return; }
$job = json_decode((string)@file_get_contents($tokFile), true) ?: [];
if (($job['user'] ?? '') !== $username) { http_response_code(403); header('Content-Type: application/json'); echo json_encode(["error"=>"Forbidden"]); return; }
$ready = (($job['status'] ?? '') === 'done') && !empty($job['zipPath']) && is_file($job['zipPath']);
$out = [
'status' => $job['status'] ?? 'unknown',
'error' => $job['error'] ?? null,
'ready' => $ready,
// progress (if present)
'pct' => $job['pct'] ?? null,
'filesDone' => $job['filesDone'] ?? null,
'filesTotal' => $job['filesTotal'] ?? null,
'bytesDone' => $job['bytesDone'] ?? null,
'bytesTotal' => $job['bytesTotal'] ?? null,
'current' => $job['current'] ?? null,
'phase' => $job['phase'] ?? null,
// timing (always include for UI)
'startedAt' => $job['startedAt'] ?? null,
'finalizeAt' => $job['finalizeAt'] ?? null,
];
if ($ready) {
$out['size'] = @filesize($job['zipPath']) ?: null;
$out['downloadUrl'] = '/api/file/downloadZipFile.php?k=' . urlencode($token);
}
header('Content-Type: application/json');
header('Cache-Control: no-store, no-cache, must-revalidate, max-age=0');
header('Pragma: no-cache');
header('Expires: 0');
echo json_encode($out);
}
public function downloadZipFile()
{
if (!isset($_SESSION['authenticated']) || $_SESSION['authenticated'] !== true) { http_response_code(401); echo "Unauthorized"; return; }
$username = $_SESSION['username'] ?? '';
$token = isset($_GET['k']) ? preg_replace('/[^a-f0-9]/','',(string)$_GET['k']) : '';
if ($token === '' || strlen($token) < 8) { http_response_code(400); echo "Bad token"; return; }
$tokFile = rtrim((string)META_DIR, '/\\') . '/ziptmp/.tokens/' . $token . '.json';
if (!is_file($tokFile)) { http_response_code(404); echo "Not found"; return; }
$job = json_decode((string)@file_get_contents($tokFile), true) ?: [];
@unlink($tokFile); // one-shot token
if (($job['user'] ?? '') !== $username) { http_response_code(403); echo "Forbidden"; return; }
$zip = (string)($job['zipPath'] ?? '');
$zipReal = realpath($zip);
$root = realpath(rtrim((string)META_DIR, '/\\') . '/ziptmp');
if (!$zipReal || !$root || strpos($zipReal, $root) !== 0 || !is_file($zipReal)) { http_response_code(404); echo "Not found"; return; }
@session_write_close();
@set_time_limit(0);
@ignore_user_abort(true);
if (function_exists('apache_setenv')) @apache_setenv('no-gzip','1');
@ini_set('zlib.output_compression','0');
@ini_set('output_buffering','off');
while (ob_get_level()>0) @ob_end_clean();
@clearstatcache(true, $zipReal);
$name = isset($_GET['name']) ? preg_replace('/[^A-Za-z0-9._-]/','_', (string)$_GET['name']) : 'files.zip';
if ($name === '' || str_ends_with($name,'.')) $name = 'files.zip';
$size = (int)@filesize($zipReal);
header('X-Accel-Buffering: no');
header('X-Content-Type-Options: nosniff');
header('Content-Type: application/zip');
header('Content-Disposition: attachment; filename="'.$name.'"');
if ($size>0) header('Content-Length: '.$size);
header('Cache-Control: no-store, no-cache, must-revalidate');
header('Pragma: no-cache');
readfile($zipReal);
@unlink($zipReal);
}
public function downloadZip()
{
try {
if (!$this->_checkCsrf()) { $this->_jsonOut(["error"=>"Bad CSRF"],400); return; }
if (!$this->_requireAuth()) { $this->_jsonOut(["error"=>"Unauthorized"],401); return; }
$data = $this->_readJsonBody();
if (!is_array($data) || !isset($data['folder'], $data['files']) || !is_array($data['files'])) {
$this->_jsonOut(["error" => "Invalid input."], 400); return;
}
$folder = $this->_normalizeFolder($data['folder']);
$files = $data['files'];
if (!$this->_validFolder($folder)) { $this->_jsonOut(["error"=>"Invalid folder name."], 400); return; }
$username = $_SESSION['username'] ?? '';
$perms = $this->loadPerms($username);
// Optional zip gate by account flag
if (!$this->isAdmin($perms) && !empty($perms['disableZip'])) {
$this->_jsonOut(["error" => "ZIP downloads are not allowed for your account."], 403); return;
}
$ignoreOwnership = $this->isAdmin($perms)
|| ($perms['bypassOwnership'] ?? (defined('DEFAULT_BYPASS_OWNERSHIP') ? DEFAULT_BYPASS_OWNERSHIP : false));
// Ancestor-owner counts as full view
$fullView = $ignoreOwnership
|| ACL::canRead($username, $perms, $folder)
|| $this->ownsFolderOrAncestor($folder, $username, $perms);
$ownOnly = !$fullView && ACL::hasGrant($username, $folder, 'read_own');
if (!$fullView && !$ownOnly) { $this->_jsonOut(["error" => "Forbidden: no view access to this folder."], 403); return; }
// If own-only, ensure all files are owned by the user
if ($ownOnly) {
$meta = $this->loadFolderMetadata($folder);
foreach ($files as $f) {
$bn = basename((string)$f);
if (!isset($meta[$bn]['uploader']) || strcasecmp((string)$meta[$bn]['uploader'], $username) !== 0) {
$this->_jsonOut(["error" => "Forbidden: you are not the owner of '{$bn}'."], 403); return;
}
}
$result = FileModel::createZipArchive($folder, $files);
if (isset($result['error'])) { http_response_code(400); echo $result['error']; return; }
$zipPath = $result['zipPath'] ?? null;
if (!$zipPath || !is_file($zipPath)) { http_response_code(500); echo "ZIP archive not found."; return; }
// ---- Clean binary stream setup ----
@session_write_close();
@set_time_limit(0);
@ignore_user_abort(true);
if (function_exists('apache_setenv')) { @apache_setenv('no-gzip', '1'); }
@ini_set('zlib.output_compression', '0');
@ini_set('output_buffering', 'off');
while (ob_get_level() > 0) { @ob_end_clean(); }
@clearstatcache(true, $zipPath);
$size = (int)@filesize($zipPath);
header('X-Accel-Buffering: no');
header_remove('Content-Type');
header('Content-Type: application/zip');
// Client sets the final name via a.download in your JS; server can be generic
header('Content-Disposition: attachment; filename="files.zip"');
if ($size > 0) header('Content-Length: ' . $size);
header('Cache-Control: no-store, no-cache, must-revalidate');
header('Pragma: no-cache');
$fp = fopen($zipPath, 'rb');
if ($fp === false) { http_response_code(500); echo "Failed to open ZIP."; return; }
$chunk = 1048576; // 1 MiB
while (!feof($fp)) {
$buf = fread($fp, $chunk);
if ($buf === false) break;
echo $buf;
flush();
}
fclose($fp);
@unlink($zipPath);
exit;
} catch (Throwable $e) {
error_log('FileController::downloadZip error: '.$e->getMessage().' @ '.$e->getFile().':'.$e->getLine());
if (!headers_sent()) http_response_code(500);
echo "Internal server error while preparing ZIP.";
}
$root = rtrim((string)META_DIR, '/\\') . DIRECTORY_SEPARATOR . 'ziptmp';
$tokDir = $root . DIRECTORY_SEPARATOR . '.tokens';
$logDir = $root . DIRECTORY_SEPARATOR . '.logs';
if (!is_dir($tokDir)) @mkdir($tokDir, 0700, true);
if (!is_dir($logDir)) @mkdir($logDir, 0700, true);
@chmod($tokDir, 0700);
@chmod($logDir, 0700);
if (!is_dir($tokDir) || !is_writable($tokDir)) {
$this->_jsonOut(["error"=>"ZIP token dir not writable."],500); return;
}
// Light janitor: purge old tokens/logs > 6h (best-effort)
$now = time();
foreach ((glob($tokDir . DIRECTORY_SEPARATOR . '*.json') ?: []) as $tf) {
if (is_file($tf) && ($now - (int)@filemtime($tf)) > 21600) { @unlink($tf); }
}
foreach ((glob($logDir . DIRECTORY_SEPARATOR . 'WORKER-*.log') ?: []) as $lf) {
if (is_file($lf) && ($now - (int)@filemtime($lf)) > 21600) { @unlink($lf); }
}
// Per-user and global caps (simple anti-DoS)
$perUserCap = 2; // tweak if desired
$globalCap = 8; // tweak if desired
$tokens = glob($tokDir . DIRECTORY_SEPARATOR . '*.json') ?: [];
$mine = 0; $all = 0;
foreach ($tokens as $tf) {
$job = json_decode((string)@file_get_contents($tf), true) ?: [];
$st = $job['status'] ?? 'unknown';
if ($st === 'queued' || $st === 'working' || $st === 'finalizing') {
$all++;
if (($job['user'] ?? '') === $username) $mine++;
}
}
if ($mine >= $perUserCap) { $this->_jsonOut(["error"=>"You already have ZIP jobs running. Try again shortly."], 429); return; }
if ($all >= $globalCap) { $this->_jsonOut(["error"=>"ZIP queue is busy. Try again shortly."], 429); return; }
// Create job token
$token = bin2hex(random_bytes(16));
$tokFile = $tokDir . DIRECTORY_SEPARATOR . $token . '.json';
$job = [
'user' => $username,
'folder' => $folder,
'files' => array_values($files),
'status' => 'queued',
'ctime' => time(),
'startedAt' => null,
'finalizeAt' => null,
'zipPath' => null,
'error' => null
];
if (file_put_contents($tokFile, json_encode($job, JSON_PRETTY_PRINT), LOCK_EX) === false) {
$this->_jsonOut(["error"=>"Failed to create zip job."],500); return;
}
// Robust spawn (detect php CLI, log, record PID)
$spawn = $this->spawnZipWorker($token, $tokFile, $logDir);
if (!$spawn['ok']) {
$job['status'] = 'error';
$job['error'] = 'Spawn failed: '.$spawn['error'];
@file_put_contents($tokFile, json_encode($job, JSON_PRETTY_PRINT), LOCK_EX);
$this->_jsonOut(["error"=>"Failed to enqueue ZIP: ".$spawn['error']], 500);
return;
}
$this->_jsonOut([
'ok' => true,
'token' => $token,
'status' => 'queued',
'statusUrl' => '/api/file/zipStatus.php?k=' . urlencode($token),
'downloadUrl' => '/api/file/downloadZipFile.php?k=' . urlencode($token)
]);
} catch (Throwable $e) {
error_log('FileController::downloadZip enqueue error: '.$e->getMessage().' @ '.$e->getFile().':'.$e->getLine());
$this->_jsonOut(['error' => 'Internal error while queuing ZIP.'], 500);
}
}
public function extractZip()
{

21
src/models/FileModel.php
View File

@@ -557,13 +557,13 @@ class FileModel {
* @return array An associative array with either an "error" key or a "zipPath" key.
*/
public static function createZipArchive($folder, $files) {
// (optional) purge old temp zips > 6h
// Purge old temp zips > 6h (best-effort)
$zipRoot = rtrim((string)META_DIR, '/\\') . DIRECTORY_SEPARATOR . 'ziptmp';
$now = time();
foreach (glob($zipRoot . DIRECTORY_SEPARATOR . 'download-*.zip') ?: [] as $zp) {
if (is_file($zp) && ($now - @filemtime($zp)) > 21600) { @unlink($zp); }
foreach ((glob($zipRoot . DIRECTORY_SEPARATOR . 'download-*.zip') ?: []) as $zp) {
if (is_file($zp) && ($now - (int)@filemtime($zp)) > 21600) { @unlink($zp); }
}
// Normalize and validate target folder
$folder = trim((string)$folder) ?: 'root';
$baseDir = realpath(UPLOAD_DIR);
@@ -574,7 +574,6 @@ class FileModel {
if (strtolower($folder) === 'root' || $folder === "") {
$folderPathReal = $baseDir;
} else {
// Prevent traversal and validate each segment against folder regex
if (strpos($folder, '..') !== false) {
return ["error" => "Invalid folder name."];
}
@@ -599,6 +598,10 @@ class FileModel {
continue;
}
$fullPath = $folderPathReal . DIRECTORY_SEPARATOR . $fileName;
// Skip symlinks (avoid archiving outside targets via links)
if (is_link($fullPath)) {
continue;
}
if (is_file($fullPath)) {
$filesToZip[] = $fullPath;
}
@@ -609,9 +612,7 @@ class FileModel {
// Workspace on the big disk: META_DIR/ziptmp
$work = rtrim((string)META_DIR, '/\\') . DIRECTORY_SEPARATOR . 'ziptmp';
if (!is_dir($work)) {
@mkdir($work, 0775, true);
}
if (!is_dir($work)) { @mkdir($work, 0775, true); }
if (!is_dir($work) || !is_writable($work)) {
return ["error" => "ZIP temp dir not writable: " . $work];
}
@@ -633,7 +634,7 @@ class FileModel {
@set_time_limit(0);
// Create the ZIP path inside META_DIR/ziptmp
// Create the ZIP path inside META_DIR/ziptmp (libzip temp stays on same FS)
$zipName = 'download-' . date('Ymd-His') . '-' . bin2hex(random_bytes(4)) . '.zip';
$zipPath = $work . DIRECTORY_SEPARATOR . $zipName;
@@ -643,7 +644,7 @@ class FileModel {
}
foreach ($filesToZip as $filePath) {
// Add using basename at the root of the zip (matches your current behavior)
// Add using basename at the root of the zip (matches current behavior)
$zip->addFile($filePath, basename($filePath));
}