left4me/docs/superpowers/plans/2026-05-08-overlay-file-tree.md

Overlay File Tree Implementation Plan

Approval status: User-approved 2026-05-08; implemented + deployed in the same session. This plan is committed retrospectively to record the work.

Goal: Build the overlay-detail "Files" section per docs/superpowers/specs/2026-05-08-overlay-file-tree-design.md — a server-rendered collapsible tree of ${LEFT4ME_ROOT}/overlays/{overlay.id}/ with HTMX lazy expansion and click-to-download for individual files. Read-only; same access rule as the rest of the overlay detail page.

Architecture: A new files_bp blueprint exposes two GETs: /overlays/<id>/files?path=<rel> returns the listing as an HTML fragment (used both for first paint at the root level via page_routes.overlay_detail context, and for HTMX swaps when a folder expands), and /overlays/<id>/files/download?path=<rel> streams a single file. Pure helpers live in l4d2web/services/overlay_files.py: safe_resolve_for_listing (refuses symlink escape from overlay root), safe_resolve_for_download (allows symlink targets anywhere under LEFT4ME_ROOT — workshop addons stream from the shared cache; absolute symlinks to /etc/passwd are still blocked), and list_directory (one-level scan, dirs-first sort, 500-entry cap, symlink + broken-symlink markers, resolved size for files). Two Jinja partials (_overlay_file_tree.html, _overlay_file_node.html) plus a 12-line event-delegated static/js/file-tree.js for collapse/re-expand handle the UI; styles append to static/css/components.css against existing tokens.

---

Locked Decisions

See the design doc for rationale. Implementation-relevant summary:

• New blueprint files_bp registered in l4d2web/app.py next to overlay_bp.
• Path resolution chains through l4d2host.paths.overlay_path() (already validates the overlay ref + resolves under LEFT4ME_ROOT/overlays/) and l4d2web.services.security.validate_overlay_ref (rejects empty/./../absolute/whitespace/backslash for the sub-path component).
• Listing rule: target must be a descendant of overlay_root after Path.resolve(). Download rule: real path must be a descendant of LEFT4ME_ROOT after os.path.realpath().
• Tree shape: single recursive partial. _overlay_file_tree.html renders <ul>; _overlay_file_node.html renders one folder or file <li>. Folder buttons carry data-files-url="/overlays/{id}/files?path=…". static/js/file-tree.js handles every click — toggles aria-expanded + hidden, fetches once on first expand, dedupes rapid clicks via dataset.loaded.
• DEFAULT_MAX_ENTRIES = 500 in the helper module; re-resolved per call so tests can monkeypatch.
• No changes to l4d2host, builders, or workshop/script edit flows.

---

Task 1: Pure helpers — path safety + directory listing

Files:

• Create: l4d2web/services/overlay_files.py — safe_resolve_for_listing, safe_resolve_for_download, list_directory, _format_size, DEFAULT_MAX_ENTRIES.
• Create: l4d2web/tests/test_overlay_files.py — 20 tests (path safety, listing semantics, symlink + broken-symlink handling, sort order, truncation cap, human-size formatting).

Test plan (RED first):

1. Listing returns overlay root for empty sub-path; joins under root for nested sub-path; rejects .., absolute path, empty component (foo//bar); rejects symlink escaping the overlay root even when target sits in workshop_cache/.
2. Download rejects empty path; returns real path for a regular file; follows a symlink into workshop_cache/; rejects a symlink to a path outside LEFT4ME_ROOT; rejects .. and absolute paths.
3. list_directory: empty dir → empty list, truncated 0; dirs-first then files, both case-insensitive alphabetical; kind ∈ {"dir", "file"}; rel is forward-slash relative to overlay root; symlinks marked with is_symlink=True and resolved-target size; broken symlinks marked broken=True with size=None; truncation at supplied cap returns first N + truncated_count; size_human formats 5 B and 3.0 MB correctly.

Implementation:

• safe_resolve_for_listing calls l4d2host.paths.overlay_path(overlay_path_value).resolve() for the overlay root, short-circuits on empty sub_path, validates the sub-path via validate_overlay_ref, then (overlay_root / sub_path).resolve(strict=False) and asserts the result is the overlay root or a descendant.
• safe_resolve_for_download rejects empty sub_path, validates, builds overlay_root / sub_path, applies os.path.realpath(), asserts the result is under get_left4me_root().resolve().
• list_directory(target, overlay_root, *, max_entries=None) uses os.scandir (free stat cache, follow_symlinks toggle). Per entry: is_symlink = entry.is_symlink(); is_dir = entry.is_dir(follow_symlinks=True) inside a try (OSError → broken=True, kind=file, size=None); regular files use entry.stat(follow_symlinks=True).st_size. rel is "/".join(Path(entry.path).relative_to(overlay_root).parts). Sort by (0 if dir else 1, name.casefold()). Truncate to max_entries or DEFAULT_MAX_ENTRIES.
• _format_size: bytes (N B, no decimal) up to 1024, then KB/MB/GB/TB at one decimal place.

Verification:

pytest l4d2web/tests/test_overlay_files.py -q

Commit: part of Task 4's bundled feat commit.

---

Task 2: HTTP routes — files_bp blueprint

Files:

• Create: l4d2web/routes/files_routes.py — files_bp with GET /overlays/<id>/files (fragment) and GET /overlays/<id>/files/download (stream).
• Modify: l4d2web/app.py — from l4d2web.routes.files_routes import bp as files_bp and app.register_blueprint(files_bp) next to overlay_bp.
• Create: l4d2web/tests/test_overlay_files_routes.py — 16 HTTP-level tests at this stage (3 more added in Task 4).

Test plan (RED first):

• Fragment: 200 + entries for root listing; 200 + entries for sub-directory; 400 on .., absolute path, empty component; 404 on unknown overlay; 404 on missing sub-dir; 403 on foreign user's overlay; 200 for admin viewing foreign overlay; truncation cap exposes "+ N more" footer (monkeypatch DEFAULT_MAX_ENTRIES); broken symlink rendered with broken badge and no <a> link.
• Download: 200 + Content-Disposition: attachment + exact byte match for regular file; 200 + cache content for workshop-cache symlink; 400 for symlink resolving outside LEFT4ME_ROOT; 400 for directory target; 404 for missing file; 403 for foreign user's overlay.

Implementation:

• Decorator stack: @files_bp.get(...) + @require_login. Auth gate inside the handler mirrors page_routes.overlay_detail:194 (g.user.admin or overlay.user_id is None or overlay.user_id == g.user.id).
• Shared _load_overlay_for_user(overlay_id, user) does the lookup, the auth gate, and db.expunge(overlay) so the route can read scalar attributes after the session closes.
• ValueError from either resolver → Response("invalid path", status=400). target.is_dir() failure on the listing route → 404. real.exists() / real.is_dir() failure on the download route → 404 / 400.
• send_file(str(real), as_attachment=True, download_name=os.path.basename(real)).
• The fragment renders _overlay_file_tree.html only — no base.html shell — so HTMX swaps inject just the <ul> content.

Verification:

pytest l4d2web/tests/test_overlay_files_routes.py -q

Commit: part of Task 4's bundled feat commit.

---

Task 3: Templates + page-routes integration

Files:

• Create: l4d2web/templates/_overlay_file_tree.html — <ul class="file-tree" role="group"> + per-entry _overlay_file_node.html include + optional truncated-footer <li>.
• Create: l4d2web/templates/_overlay_file_node.html — folder row (button + HTMX attrs + empty <div class="file-tree-children" hidden>) or file row (<a> for regular/symlink files; <span> for broken symlinks; link / broken link badges; size_human).
• Modify: l4d2web/templates/overlay_detail.html — add <section class="panel"><h2>Files</h2>…</section> between the type-specific sections and the existing "Used by" section. Renders empty-state <p class="muted">No files yet — build this overlay to populate it.</p> when file_tree_root_entries is none, else includes the partial.
• Modify: l4d2web/routes/page_routes.py — import the helpers, add _root_file_tree(overlay) (returns (entries, truncated_count) or (None, 0) on ValueError / missing dir / legacy absolute overlay.path), pass file_tree_root_entries + file_tree_truncated + file_tree_truncated_count into render_template("overlay_detail.html", …).

Test plan (RED first, added to test_overlay_files_routes.py):

• test_overlay_detail_renders_files_section_with_tree — page contains "Files" header + entry names.
• test_overlay_detail_shows_empty_state_when_overlay_dir_missing — wipe directory, page shows "No files yet".
• test_overlay_detail_files_section_present_for_workshop_overlays — workshop type also gets the section.

Implementation:

• Section placement matters: <section><h2>Files</h2>…</section> is inserted before the existing "Used by" <section>.
• The partial uses {% set entries = file_tree_root_entries %} etc. so the same partial works whether called from the page (with full context) or from the HTMX route (rendering directly with named kwargs).
• _root_file_tree swallows ValueError and missing-dir cases into (None, 0), and the template's {% if file_tree_root_entries is none %} renders the empty state.
• Use overlay.path (not str(overlay.id)) so legacy/seeded rows whose path differs still work correctly when resolvable.

Verification:

pytest l4d2web/tests/test_overlay_files_routes.py -q -k overlay_detail
pytest l4d2web/tests/ -q              # no regressions across the full suite

Commit: part of Task 4's bundled feat commit.

---

Task 4: CSS + JS + base.html script wiring

Files:

• Create: l4d2web/static/js/file-tree.js — event-delegated click handler that toggles aria-expanded on .file-tree-toggle and hidden on the next .file-tree-children sibling, and on first expand fires fetch(button.dataset.filesUrl) and innerHTMLs the response. dataset.loaded flag dedupes rapid clicks; cleared on error to allow retry.
• Modify: l4d2web/templates/base.html — <script src="{{ url_for('static', filename='js/file-tree.js') }}"></script> next to the existing csrf.js / sse.js / modal.js lines.
• Modify: l4d2web/static/css/components.css — append ~50 lines: .file-tree, .file-tree-row, .file-tree-toggle (transparent button, inherits color), .file-tree-toggle .chevron rotation transform on aria-expanded="true", .file-tree-children[hidden], .file-tree-badge + .file-tree-badge-warn. All against existing tokens (--space-xs, --space-l, --color-surface-muted, --color-muted, --color-danger, --radius-s).

Implementation:

• The JS handler fires on every click. First-expand path: read button.dataset.filesUrl, set dataset.loaded="1" optimistically, fetch(url, {credentials: "same-origin"}), replace .file-tree-children innerHTML with the response. Subsequent clicks just toggle aria-expanded + hidden — no re-fetch since dataset.loaded is set. On fetch error: delete dataset.loaded so a future click retries.
• The CSS chevron is a Unicode › inside a <span class="chevron">; rotated 90° on expanded via transform: rotate(90deg) with a 120ms transition.

Verification:

pytest l4d2web/tests/ -q            # 293 passed, 1 skipped

Manual smoke (post-deploy on ckn@10.0.4.128):

• Navigate to an overlay detail page with a populated runtime directory.
• Confirm the "Files" section renders the root level.
• Click a folder: HTMX request fires once, children appear, chevron rotates.
• Click again: children hide; no second request in DevTools network tab.
• Click a file: browser downloads it with the correct filename.
• Visit another user's overlay as a non-admin: 403.

Commit: feat(l4d2-web): overlay detail Files section with HTMX file tree + downloads — covers all four tasks (services helper + routes + templates + CSS/JS), since the feature is small and the tasks share a single set of integration tests.

---

End-to-end verification

After all tasks committed:

pytest l4d2web/tests/ -q                 # 293 passed, 1 skipped
deploy/deploy-test-server.sh ckn@10.0.4.128
ssh ckn@10.0.4.128 'systemctl status left4me-web --no-pager | head -10'
curl -s http://10.0.4.128:8000/health    # {"status":"ok"}

Then exercise the manual smoke checklist from Task 4 against the deployed instance.