76bd6e8d4d
Captures the design rationale for the new overlay-detail Files section (verify build output, click-to-download for individual files via Flask send_file, HTMX-driven lazy folder expansion) and the paired implementation plan that produced it. Adds .superpowers/ to .gitignore so brainstorm session artifacts never sneak into a future commit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
161 lines
12 KiB
Markdown
161 lines
12 KiB
Markdown
# Overlay File Tree Implementation Plan
|
||
|
||
> **Approval status:** User-approved 2026-05-08; implemented + deployed in the same session. This plan is committed retrospectively to record the work.
|
||
|
||
**Goal:** Build the overlay-detail "Files" section per `docs/superpowers/specs/2026-05-08-overlay-file-tree-design.md` — a server-rendered collapsible tree of `${LEFT4ME_ROOT}/overlays/{overlay.id}/` with HTMX lazy expansion and click-to-download for individual files. Read-only; same access rule as the rest of the overlay detail page.
|
||
|
||
**Architecture:** A new `files_bp` blueprint exposes two GETs: `/overlays/<id>/files?path=<rel>` returns the listing as an HTML fragment (used both for first paint at the root level via `page_routes.overlay_detail` context, and for HTMX swaps when a folder expands), and `/overlays/<id>/files/download?path=<rel>` streams a single file. Pure helpers live in `l4d2web/services/overlay_files.py`: `safe_resolve_for_listing` (refuses symlink escape from overlay root), `safe_resolve_for_download` (allows symlink targets anywhere under `LEFT4ME_ROOT` — workshop addons stream from the shared cache; absolute symlinks to `/etc/passwd` are still blocked), and `list_directory` (one-level scan, dirs-first sort, 500-entry cap, symlink + broken-symlink markers, resolved size for files). Two Jinja partials (`_overlay_file_tree.html`, `_overlay_file_node.html`) plus a 12-line event-delegated `static/js/file-tree.js` for collapse/re-expand handle the UI; styles append to `static/css/components.css` against existing tokens.
|
||
|
||
---
|
||
|
||
## Locked Decisions
|
||
|
||
See the design doc for rationale. Implementation-relevant summary:
|
||
|
||
- New blueprint `files_bp` registered in `l4d2web/app.py` next to `overlay_bp`.
|
||
- Path resolution chains through `l4d2host.paths.overlay_path()` (already validates the overlay ref + resolves under `LEFT4ME_ROOT/overlays/`) and `l4d2web.services.security.validate_overlay_ref` (rejects empty/`.`/`..`/absolute/whitespace/backslash for the sub-path component).
|
||
- Listing rule: target must be a descendant of `overlay_root` after `Path.resolve()`. Download rule: real path must be a descendant of `LEFT4ME_ROOT` after `os.path.realpath()`.
|
||
- Tree shape: single recursive partial. `_overlay_file_tree.html` renders `<ul>`; `_overlay_file_node.html` renders one folder or file `<li>`. Folder buttons carry `hx-get`, `hx-target="next .file-tree-children"`, `hx-swap="innerHTML"`, `hx-trigger="click once"`. Subsequent toggles handled by `static/js/file-tree.js`.
|
||
- `DEFAULT_MAX_ENTRIES = 500` in the helper module; re-resolved per call so tests can monkeypatch.
|
||
- No changes to `l4d2host`, builders, or workshop/script edit flows.
|
||
|
||
---
|
||
|
||
## Task 1: Pure helpers — path safety + directory listing
|
||
|
||
**Files:**
|
||
|
||
- Create: `l4d2web/services/overlay_files.py` — `safe_resolve_for_listing`, `safe_resolve_for_download`, `list_directory`, `_format_size`, `DEFAULT_MAX_ENTRIES`.
|
||
- Create: `l4d2web/tests/test_overlay_files.py` — 20 tests (path safety, listing semantics, symlink + broken-symlink handling, sort order, truncation cap, human-size formatting).
|
||
|
||
Test plan (RED first):
|
||
|
||
1. Listing returns overlay root for empty sub-path; joins under root for nested sub-path; rejects `..`, absolute path, empty component (`foo//bar`); rejects symlink escaping the overlay root even when target sits in `workshop_cache/`.
|
||
2. Download rejects empty path; returns real path for a regular file; follows a symlink into `workshop_cache/`; rejects a symlink to a path outside `LEFT4ME_ROOT`; rejects `..` and absolute paths.
|
||
3. `list_directory`: empty dir → empty list, truncated 0; dirs-first then files, both case-insensitive alphabetical; `kind ∈ {"dir", "file"}`; `rel` is forward-slash relative to overlay root; symlinks marked with `is_symlink=True` and resolved-target size; broken symlinks marked `broken=True` with `size=None`; truncation at supplied cap returns first N + `truncated_count`; `size_human` formats `5 B` and `3.0 MB` correctly.
|
||
|
||
**Implementation:**
|
||
|
||
- `safe_resolve_for_listing` calls `l4d2host.paths.overlay_path(overlay_path_value).resolve()` for the overlay root, short-circuits on empty `sub_path`, validates the sub-path via `validate_overlay_ref`, then `(overlay_root / sub_path).resolve(strict=False)` and asserts the result is the overlay root or a descendant.
|
||
- `safe_resolve_for_download` rejects empty `sub_path`, validates, builds `overlay_root / sub_path`, applies `os.path.realpath()`, asserts the result is under `get_left4me_root().resolve()`.
|
||
- `list_directory(target, overlay_root, *, max_entries=None)` uses `os.scandir` (free `stat` cache, `follow_symlinks` toggle). Per entry: `is_symlink = entry.is_symlink()`; `is_dir = entry.is_dir(follow_symlinks=True)` inside a try (OSError → broken=True, kind=file, size=None); regular files use `entry.stat(follow_symlinks=True).st_size`. `rel` is `"/".join(Path(entry.path).relative_to(overlay_root).parts)`. Sort by `(0 if dir else 1, name.casefold())`. Truncate to `max_entries or DEFAULT_MAX_ENTRIES`.
|
||
- `_format_size`: bytes (`N B`, no decimal) up to 1024, then KB/MB/GB/TB at one decimal place.
|
||
|
||
**Verification:**
|
||
|
||
```
|
||
pytest l4d2web/tests/test_overlay_files.py -q
|
||
```
|
||
|
||
**Commit:** part of Task 4's bundled `feat` commit.
|
||
|
||
---
|
||
|
||
## Task 2: HTTP routes — files_bp blueprint
|
||
|
||
**Files:**
|
||
|
||
- Create: `l4d2web/routes/files_routes.py` — `files_bp` with `GET /overlays/<id>/files` (fragment) and `GET /overlays/<id>/files/download` (stream).
|
||
- Modify: `l4d2web/app.py` — `from l4d2web.routes.files_routes import bp as files_bp` and `app.register_blueprint(files_bp)` next to `overlay_bp`.
|
||
- Create: `l4d2web/tests/test_overlay_files_routes.py` — 16 HTTP-level tests at this stage (3 more added in Task 4).
|
||
|
||
Test plan (RED first):
|
||
|
||
- Fragment: 200 + entries for root listing; 200 + entries for sub-directory; 400 on `..`, absolute path, empty component; 404 on unknown overlay; 404 on missing sub-dir; 403 on foreign user's overlay; 200 for admin viewing foreign overlay; truncation cap exposes "+ N more" footer (monkeypatch `DEFAULT_MAX_ENTRIES`); broken symlink rendered with `broken` badge and no `<a>` link.
|
||
- Download: 200 + `Content-Disposition: attachment` + exact byte match for regular file; 200 + cache content for workshop-cache symlink; 400 for symlink resolving outside `LEFT4ME_ROOT`; 400 for directory target; 404 for missing file; 403 for foreign user's overlay.
|
||
|
||
**Implementation:**
|
||
|
||
- Decorator stack: `@files_bp.get(...)` + `@require_login`. Auth gate inside the handler mirrors `page_routes.overlay_detail:194` (`g.user.admin or overlay.user_id is None or overlay.user_id == g.user.id`).
|
||
- Shared `_load_overlay_for_user(overlay_id, user)` does the lookup, the auth gate, and `db.expunge(overlay)` so the route can read scalar attributes after the session closes.
|
||
- `ValueError` from either resolver → `Response("invalid path", status=400)`. `target.is_dir()` failure on the listing route → 404. `real.exists()` / `real.is_dir()` failure on the download route → 404 / 400.
|
||
- `send_file(str(real), as_attachment=True, download_name=os.path.basename(real))`.
|
||
- The fragment renders `_overlay_file_tree.html` only — no `base.html` shell — so HTMX swaps inject just the `<ul>` content.
|
||
|
||
**Verification:**
|
||
|
||
```
|
||
pytest l4d2web/tests/test_overlay_files_routes.py -q
|
||
```
|
||
|
||
**Commit:** part of Task 4's bundled `feat` commit.
|
||
|
||
---
|
||
|
||
## Task 3: Templates + page-routes integration
|
||
|
||
**Files:**
|
||
|
||
- Create: `l4d2web/templates/_overlay_file_tree.html` — `<ul class="file-tree" role="group">` + per-entry `_overlay_file_node.html` include + optional truncated-footer `<li>`.
|
||
- Create: `l4d2web/templates/_overlay_file_node.html` — folder row (button + HTMX attrs + empty `<div class="file-tree-children" hidden>`) or file row (`<a>` for regular/symlink files; `<span>` for broken symlinks; `link` / `broken link` badges; `size_human`).
|
||
- Modify: `l4d2web/templates/overlay_detail.html` — add `<section class="panel"><h2>Files</h2>…</section>` between the type-specific sections and the existing "Used by" section. Renders empty-state `<p class="muted">No files yet — build this overlay to populate it.</p>` when `file_tree_root_entries is none`, else includes the partial.
|
||
- Modify: `l4d2web/routes/page_routes.py` — import the helpers, add `_root_file_tree(overlay)` (returns `(entries, truncated_count)` or `(None, 0)` on `ValueError` / missing dir / legacy absolute `overlay.path`), pass `file_tree_root_entries` + `file_tree_truncated` + `file_tree_truncated_count` into `render_template("overlay_detail.html", …)`.
|
||
|
||
Test plan (RED first, added to `test_overlay_files_routes.py`):
|
||
|
||
- `test_overlay_detail_renders_files_section_with_tree` — page contains "Files" header + entry names.
|
||
- `test_overlay_detail_shows_empty_state_when_overlay_dir_missing` — wipe directory, page shows "No files yet".
|
||
- `test_overlay_detail_files_section_present_for_workshop_overlays` — workshop type also gets the section.
|
||
|
||
**Implementation:**
|
||
|
||
- Section placement matters: `<section><h2>Files</h2>…</section>` is inserted before the existing "Used by" `<section>`.
|
||
- The partial uses `{% set entries = file_tree_root_entries %}` etc. so the same partial works whether called from the page (with full context) or from the HTMX route (rendering directly with named kwargs).
|
||
- `_root_file_tree` swallows `ValueError` and missing-dir cases into `(None, 0)`, and the template's `{% if file_tree_root_entries is none %}` renders the empty state.
|
||
- Use `overlay.path` (not `str(overlay.id)`) so legacy/seeded rows whose path differs still work correctly when resolvable.
|
||
|
||
**Verification:**
|
||
|
||
```
|
||
pytest l4d2web/tests/test_overlay_files_routes.py -q -k overlay_detail
|
||
pytest l4d2web/tests/ -q # no regressions across the full suite
|
||
```
|
||
|
||
**Commit:** part of Task 4's bundled `feat` commit.
|
||
|
||
---
|
||
|
||
## Task 4: CSS + JS + base.html script wiring
|
||
|
||
**Files:**
|
||
|
||
- Create: `l4d2web/static/js/file-tree.js` — 12-line event-delegated `click` handler that toggles `aria-expanded` on `.file-tree-toggle` and `hidden` on the next `.file-tree-children` sibling.
|
||
- Modify: `l4d2web/templates/base.html` — `<script src="{{ url_for('static', filename='js/file-tree.js') }}"></script>` next to the existing `csrf.js` / `sse.js` / `modal.js` lines.
|
||
- Modify: `l4d2web/static/css/components.css` — append `~50` lines: `.file-tree`, `.file-tree-row`, `.file-tree-toggle` (transparent button, inherits color), `.file-tree-toggle .chevron` rotation transform on `aria-expanded="true"`, `.file-tree-children[hidden]`, `.file-tree-badge` + `.file-tree-badge-warn`. All against existing tokens (`--space-xs`, `--space-l`, `--color-surface-muted`, `--color-muted`, `--color-danger`, `--radius-s`).
|
||
|
||
**Implementation:**
|
||
|
||
- `hx-trigger="click once"` makes HTMX fire the GET only on the first expansion; the JS handler fires on every click and toggles aria/hidden state. First click: HTMX populates `.file-tree-children`, JS toggles to expanded. Subsequent clicks: only JS fires, no re-fetch.
|
||
- The CSS chevron is a Unicode `›` inside a `<span class="chevron">`; rotated 90° on expanded via `transform: rotate(90deg)` with a 120ms transition.
|
||
|
||
**Verification:**
|
||
|
||
```
|
||
pytest l4d2web/tests/ -q # 293 passed, 1 skipped
|
||
```
|
||
|
||
Manual smoke (post-deploy on `ckn@10.0.4.128`):
|
||
|
||
- Navigate to an overlay detail page with a populated runtime directory.
|
||
- Confirm the "Files" section renders the root level.
|
||
- Click a folder: HTMX request fires once, children appear, chevron rotates.
|
||
- Click again: children hide; no second request in DevTools network tab.
|
||
- Click a file: browser downloads it with the correct filename.
|
||
- Visit another user's overlay as a non-admin: 403.
|
||
|
||
**Commit:** `feat(l4d2-web): overlay detail Files section with HTMX file tree + downloads` — covers all four tasks (services helper + routes + templates + CSS/JS), since the feature is small and the tasks share a single set of integration tests.
|
||
|
||
---
|
||
|
||
## End-to-end verification
|
||
|
||
After all tasks committed:
|
||
|
||
```
|
||
pytest l4d2web/tests/ -q # 293 passed, 1 skipped
|
||
deploy/deploy-test-server.sh ckn@10.0.4.128
|
||
ssh ckn@10.0.4.128 'systemctl status left4me-web --no-pager | head -10'
|
||
curl -s http://10.0.4.128:8000/health # {"status":"ok"}
|
||
```
|
||
|
||
Then exercise the manual smoke checklist from Task 4 against the deployed instance.
|