left4me/docs/superpowers/specs/2026-05-09-files-overlay-design.md

# Files overlay (user-managed file content)

## Context

In the prior `ckn-bw` setup, per-server config-style files (`admins.txt`, `motd.txt`, mapcycle, etc.) lived under `bundles/left4dead2/files/scripts/overlays/standard`. `left4me` has no equivalent: today an overlay's contents come from either Steam Workshop (`workshop` type) or a user-authored bash build script (`script` type). Both have an external source-of-truth, so neither is the right home for files the user owns directly. The user wants both online editing of text files *and* arbitrary file upload, and we unify them into a single mechanism.

## Goal

Add a third overlay type `files` whose source-of-truth IS the overlay directory itself. Provide a web UI to:

- **Upload** any file or whole folder by dragging it onto a folder row in the tree (drag from the OS).
- **Move** files and folders by dragging rows inside the tree (internal drag).
- **Create / edit / rename / replace** files through a single modal editor, opened from row buttons. Modal adapts to text or binary content.
- **Download** files (or zip an entire folder).
- **Delete** files and empty folders.
- **Create new folders** explicitly (including nested intermediates in one shot).

Reuse the existing overlayfs / spec / mount / `expose_server_cfg` pipeline unchanged: a `files` overlay is a normal overlay attached to blueprints.

## Non-goals (v1)

- Per-server overrides (servers still bind to a blueprint without per-instance file changes).
- Concurrency policing when an overlay is in use by a running server. Overlayfs technically calls lower-layer mutation undefined behavior, but L4D2 reads most config at boot, so "edits visible on next start" is acceptable.
- Versioning / undo / history.
- Syntax highlighting (CodeMirror-style). Plain `<textarea>`; can add later.
- "Save As" copy. The filename input *is* Save-As.
- Recursive directory delete from the UI.
- Multi-file drop into the binary "replace" zone (single file only).

## Approach

### Data model

`Overlay.type` accepts a new value: `"files"` (in addition to `"workshop"` and `"script"`). No schema change needed — `Overlay.type` is already `String(16)`. The `script` column stays empty for files overlays; `last_build_status` is set to `"ok"` on creation and not otherwise managed. Privacy follows the existing `user_id` rules unchanged.

`BlueprintOverlay` and the `expose_server_cfg` checkbox keep working as-is: a `files` overlay containing a `server.cfg` is exposed via the same alias mechanism the 2026-05-08 plan introduced.

### Filesystem layout

A files overlay lives at `${LEFT4ME_ROOT}/overlays/{overlay.path}/` like every other overlay. Example contents:

```
overlays/{id}/
  left4dead2/
    cfg/
      server.cfg
      motd.txt
      mapcycle.txt
    addons/
      sourcemod/configs/admins_simple.ini
      custom_map.vpk
```

The `InstanceSpec` / `OverlayRef` shape already supports this. The spec builder in `l4d2web/services/l4d2_facade.py` doesn't need to learn about overlay types, only to keep emitting `path` (and `alias` when `expose_server_cfg` is set).

### Builder registration

`l4d2web/services/overlay_builders.py::BUILDERS` gains a `"files"` entry whose `build()` is a no-op that ensures `_overlay_root(overlay)` exists. The route layer also short-circuits: there is no "rebuild" concept for a files overlay — every save / upload / move / mkdir / delete is immediately authoritative.

### Safety helpers

`l4d2web/services/overlay_files.py` already has `safe_resolve_for_listing` and `safe_resolve_for_download` (anchor-and-resolve, refuse `..` traversal and symlink-target escapes). Add three siblings using the same pattern:

- `safe_resolve_for_write(overlay_path_value, sub_path) -> Path` — destination path. Refuses empty `sub_path`, refuses any escape, refuses to overwrite an existing symlink, refuses a path whose parent resolves to a non-directory.
- `safe_resolve_for_delete(overlay_path_value, sub_path) -> Path` — same root-escape rules; allows deleting files and empty directories. Non-empty directory delete returns an error.
- `safe_resolve_for_move(overlay_path_value, src, dst) -> tuple[Path, Path]` — both endpoints inside the overlay root. Refuses `dst` inside `src` (cycle). Refuses if `src` doesn't exist. Refuses if `dst` parent is missing or not a directory. Refuses overwriting a symlink at `dst`.

Plus a small predicate:

- `is_editable(path: Path) -> bool` — true iff `path` is a regular file (not symlink), size ≤ 1 MiB, and first 8 KiB decodes as strict UTF-8. Surfaced via `_entry_dict` in listings as `editable: bool`.

### UI design

The file-manager lives inside the existing overlay detail page, only when `overlay.type == "files"`. Layout follows the existing `<ul class="file-tree">` pattern, extended as below.

#### Tree row buttons (hover-reveal, CSS `:hover`)

| Row | Buttons (left-to-right) | Click on row body | Draggable |
|---|---|---|---|
| Folder (incl. overlay root) | `+ new file` · `+ new folder` · `⬇ zip` · `✕` | toggle expand/collapse | yes (move subtree) |
| File (any) | `edit` · `⬇` · `✕` | nothing | yes (move file) |

Files always show `edit` regardless of editability — the modal adapts. Touch devices fall back to always-visible buttons via a `(hover: none)` media query.

#### Drag-and-drop on tree rows — single gesture, source distinguishes

| Drag source | Action | Visual on hovered row | Endpoint |
|---|---|---|---|
| OS file/folder (`dataTransfer.files` / `webkitGetAsEntry`) | upload | green outline + `↑ Release to upload N items here` | `POST /overlays/{id}/files/upload` |
| Tree row (file or folder) | move | green outline + `↦ Move {name} here` | `POST /overlays/{id}/files/move` |

Refused drops (UI rejects without server round-trip): drop on self, drop on own ancestor (cycle), drop where parent doesn't exist. Conflict at destination → server returns 409 → overwrite/keep-both modal.

#### Upload progress panel

Each dropped item becomes one `POST /files/upload` request (one file part, `target_path` set to the dropped row's path, `webkitRelativePath` preserved). A floating "Uploads" panel docks to the bottom-right of the page while there is at least one in-flight or queued upload, and auto-collapses when the queue is empty.

- **Per-file rows** in the panel: filename, target path (subtle), progress bar driven by `XMLHttpRequest.upload.onprogress`, queue position, per-file cancel button.
- **Concurrency:** at most 3 uploads in flight; remainder queue. Drop-while-uploading appends to the queue with no special UI.
- **Cancel mid-flight:** aborts the XHR; server cleans up any partial file in a `finally` block.
- **Conflicts:** a 409 on an individual file pauses just that upload (panel row shows "conflict — overwrite / keep both") and opens the existing overwrite/keep-both modal scoped to that one path. The rest of the queue keeps running.
- **Errors:** per-file error states (413 too large, 415 bad content, 422 path validation, 5xx) stay sticky in the panel until the user dismisses them. The panel has a "clear done" toggle.
- **Tree refresh:** when an upload finishes, the affected parent folder's listing partial is re-fetched (`hx-get` on the folder row). Debounced (50 ms) so many siblings finishing in one tick coalesce into one fetch.

#### Editor modal — single `<dialog>` with two flavors

The editor modal opens via the row's `edit` button or the folder's `+ new file` button.

**Common chrome (both flavors):**
- **Title** = full path (e.g. `left4dead2/cfg/motd.txt`). For new files: `addons/sourcemod/configs/…new file`.
- **Filename input** — single line, slashes rejected. Diverging from the original shows an inline `↻ Save will rename foo.txt → bar.txt` hint.
- **Footer** — `Delete` on the left (only for existing files), then `⬇ Download`, `Cancel`, `Save`/`Create` on the right.

**Text flavor** (file is editable, or new file):
- Content `<textarea>`, 1 MiB cap on save, UTF-8 only.
- Footer hint: `UTF-8 · {n} bytes` + `Ctrl+S to save`.

**Binary flavor** (existing file is not editable):
- Replaces the textarea with a "Replace file" panel: a label noting `⛌ Inline editing not available · {size} · binary content`, plus a drop zone (`↑ Drop a file here to replace`) with a `browse` link as fallback. Single file only.
- Once a replacement is queued, the drop zone shows `↻ {newName} · {size} · queued` with an `✕` to clear the queue.

**Save semantics** (atomic per call; rename + content change happen in one server operation):

| Mode | Filename unchanged | Filename changed |
|---|---|---|
| Text | write content | rename + write content |
| Binary, no replacement queued | (Save disabled) | rename only |
| Binary, replacement queued | overwrite content | rename + overwrite content |

Rename target collision → 409 → overwrite/keep-both modal (same modal as upload conflicts).

#### `+ new folder` dialog

A small dedicated `<dialog>` separate from the editor. Single text input for the folder name. Slashes allowed → creates intermediate dirs (`mkdir(parents=True, exist_ok=False)`).

#### `+ new file` flow

Reuses the editor modal in text flavor with empty content; the filename input is empty and focused, the title shows the source folder + `…new file`.

### Web routes

In `l4d2web/routes/files_routes.py` (alongside the existing `overlay_files_fragment` and `download` endpoints):

| Method | Path | Body | Purpose |
|---|---|---|---|
| GET | `/overlays/{id}/files/content` | `?path=` | Returns `{path, content}` for an editable file. 415 if not editable. |
| POST | `/overlays/{id}/files/save` | JSON `{path, content, new_path?}` | Text-mode save. Optional `new_path` performs rename atomically with the write. |
| POST | `/overlays/{id}/files/replace` | multipart `path`, `file`, optional `new_path` | Binary-mode replace. Optional `new_path` performs rename atomically. |
| POST | `/overlays/{id}/files/upload` | multipart `target_path`, single `file` part (carrying `webkitRelativePath`) | OS-drag upload, one file per request. Creates intermediate dirs via `mkdir(parents=True)`. Cleans up partial writes on cancel via `finally`. 200 on success, 409 on conflict, 413/415/422 on validation failure. |
| POST | `/overlays/{id}/files/move` | JSON `{src, dst}` | Internal drag move (and plain rename when same parent). |
| POST | `/overlays/{id}/files/mkdir` | JSON `{path}` | Create empty folder; slashes in `path` produce nested intermediates. |
| POST | `/overlays/{id}/files/delete` | form `path` | Delete file or empty folder. |
| GET | `/overlays/{id}/files/download_zip` | `?path=` | Stream a zip of the folder's contents. |

Existing `GET /overlays/{id}/files?path=...` and `GET /overlays/{id}/files/download?path=...` stay as-is. The listing endpoint additionally returns `editable` per file row.

All new routes:
- 404 when `overlay.type != "files"`.
- Require `overlay.user_id == current_user.id` (or admin).
- Use the new safe-resolve helpers.
- CSRF via the existing `csrf.js` injection (multipart endpoints included).

### Tech stack

Stay inside the project's established stack — Flask + Jinja2 + HTMX + tiny vanilla JS in `static/js/` + custom CSS with tokens, no build step:

- **Templates:** Jinja2 partials, returned as HTMX swaps where appropriate (subtree refresh after upload/move/mkdir/delete).
- **Modals:** native `<dialog>` with the existing `data-modal-open` / `data-modal-close` event-delegated handlers.
- **JS:** vanilla. Extend `static/js/file-tree.js` (or add a sibling `files-overlay.js`) covering: `dragstart` on rows, `dragover` highlight + source-discrimination (`dataTransfer.types.includes("Files")` vs internal MIME), `webkitGetAsEntry()` walk for whole-folder OS drops, editor modal open/save (Ctrl+S, fetch POST), binary replace-zone drop handler, conflict-modal flow, new-folder dialog, upload queue + floating progress panel (XHR per file, concurrency 3, abort on cancel, debounced tree-refresh on completion).
- **CSS:** extend `tokens.css` and `components.css` with file-manager-specific rules — drop-target outline, hover-reveal action column, editor modal sizing, replace-zone styling.

No external libraries (no Dropzone, no jsTree, no CodeMirror) — adding one would be a meaningful departure from the project's "no build step, vendored libs only" posture.

### Creation flow for new overlays

The "create overlay" UI gains a third radio option: `Files`. Selecting it skips the type-specific fields (no Steam Workshop selector, no script editor) and creates an empty `Overlay` row with `type="files"`, `last_build_status="ok"`, and an empty directory.

### Host-side

No changes. The mount helper, instance lifecycle, and srcds startup don't care what produced the contents of an overlay directory.

### Migration / Alembic

None. `Overlay.type` already stores arbitrary strings; introducing a new value is data-only.

## Critical files

| Layer | File | Change |
|---|---|---|
| Models | `l4d2web/models.py` | None (Overlay.type already String) |
| Builders | `l4d2web/services/overlay_builders.py` | Register `FilesBuilder` (no-op `build`) |
| Safety | `l4d2web/services/overlay_files.py` | Add `safe_resolve_for_write`, `safe_resolve_for_delete`, `safe_resolve_for_move`; add `is_editable` and surface it via `_entry_dict` |
| Routes | `l4d2web/routes/files_routes.py` | Add `content`, `save`, `replace`, `upload`, `move`, `mkdir`, `delete`, `download_zip` endpoints |
| Templates | `l4d2web/templates/overlay_detail.html`, `l4d2web/templates/_overlay_file_tree.html` | Hover-reveal action buttons; `data-target-path` on folder rows; `draggable="true"` on file/folder rows; editor modal `<dialog>` with both flavors; new-folder modal `<dialog>`; conflict modal `<dialog>` |
| Static JS | `l4d2web/static/js/file-tree.js` (extend) or new `files-overlay.js` | Drag-drop wiring, modal save, binary replace, mkdir, conflict flow, upload queue + panel |
| Static CSS | `l4d2web/static/css/components.css` | Drop-target outline, hover action column, editor modal sizing, replace-zone, upload panel |
| Create form | overlay creation template + route | Add `files` option to the type radio |
| Spec / facade | `l4d2web/services/l4d2_facade.py` | None — already type-agnostic |
| Host spec | `l4d2host/spec.py`, `l4d2host/instances.py` | None |
| Tests | adjacent to each touched module | safe-resolve refusals; `is_editable` heuristic; CRUD round-trip; ownership; non-files-type 404s; multipart with `webkitRelativePath`; move refuses cycles; conflict (409); zip stream; mkdir parents |

## Verification

1. **Safety unit tests** — `safe_resolve_for_write`, `_for_delete`, `_for_move` reject `..` traversal, absolute paths, symlink-target escapes, attempts to overwrite a symlink, non-empty-dir delete, and `dst` inside `src`.
2. **Editability heuristic** — `is_editable` returns false for files > 1 MiB, symlinks, files with non-UTF-8 bytes in their first 8 KiB.
3. **Editor round-trip (text)** — from a folder row, "+ new file" → modal → save creates `left4dead2/cfg/admins.txt`; row appears with `edit` button; edit; rename via filename input; delete.
4. **Editor round-trip (binary)** — upload a `.vpk`, click `edit`, queue a replacement file via drop, change filename, Save → rename + replace happen atomically.
5. **Upload single file** — drag a file from the OS onto `left4dead2/cfg/`; appears with size and download link.
6. **Upload whole folder** — drag `addons/sourcemod/` from the OS onto the overlay root; nested structure preserved; intermediate directories auto-created.
7. **Conflict on upload** — drop a file with a colliding name; overwrite/keep-both modal; both choices behave correctly.
8. **Move within tree** — drag `motd.txt` onto `addons/`; file moves; tree refreshes.
9. **Move refusals** — drag a folder onto itself or a descendant; UI rejects without server round-trip.
10. **mkdir** — `+ new folder` with name `sourcemod/configs` creates both intermediates; collision returns 409.
11. **Zip download** — `⬇ zip` on `addons/` streams a valid zip containing the subtree.
12. **Mount integration** — attach the files overlay to a blueprint, start a server, confirm the files appear under `runtime/{server_id}/merged/...`.
13. **server.cfg alias** — with `expose_server_cfg=true` and a `server.cfg` in the files overlay, `exec server_overlay_{id}` is auto-injected into the merged `server.cfg`.
14. **Type isolation** — every new endpoint returns 404 for `workshop` and `script` overlays.
15. **Browser smoke test** — Chromium and Firefox: drag a folder containing nested files into a row; confirm `webkitRelativePath` arrives correctly.
16. **Upload progress panel** — drop 5 files of mixed sizes; panel shows 3 in flight, 2 queued; per-file progress bars advance; canceling one file aborts that XHR cleanly without affecting the others; partial file is removed server-side; tree refreshes once per parent folder (debounced) when uploads finish.
17. **End-to-end on the real test box** — deploy the branch to `ckn@10.0.4.128` via the project's deploy path, then drive the running web UI through the `claude-in-chrome` MCP tools end-to-end: create a `files` overlay, attach to a blueprint, exercise every CRUD path, boot a server, confirm the files materialize in the merged mount. Iterate until all paths work without errors.