feat(deploy): workshop_cache provisioning

Adds /var/lib/left4me/workshop_cache to the deploy mkdir list (owned by
the left4me runtime user). Updates deploy/README.md to document the new
directory and the workshop overlay layout: web app downloads VPKs into
the cache and symlinks them into overlays/{overlay_id}/left4dead2/addons/.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

deploy/README.md

@@ -12,7 +12,8 @@ The deployment uses these paths:
 - `/opt/left4me`: deployed repository contents.
 - `/var/lib/left4me/left4me.db`: SQLite database used by the web app.
 - `/var/lib/left4me/installation`: shared L4D2 installation.
-- `/var/lib/left4me/overlays`: externally managed overlay directories.
+- `/var/lib/left4me/overlays`: overlay directories. External (admin-managed) overlays still live at any relative path under here; new overlays created through the web UI use `${overlay_id}` as their path.
+- `/var/lib/left4me/workshop_cache`: deduplicated cache of `.vpk` files downloaded for workshop overlays. One file per Steam item, named `{steam_id}.vpk`. Workshop overlays symlink into this tree.
 - `/var/lib/left4me/instances`: rendered instance specifications and per-instance state.
 - `/var/lib/left4me/runtime`: per-instance runtime mount directories.
 - `/var/lib/left4me/tmp`: temporary files used by deployment/runtime operations.
@@ -69,4 +70,6 @@ Invalid references are rejected:
 - Empty path components such as `competitive//base`.
 - Symlink escapes that resolve outside `${LEFT4ME_ROOT}/overlays`.
 
-Overlay content is external to the host library and deployment contract. Populate overlay directories separately before referencing them from blueprints or instance specs.
+Overlay content for `external` (admin-managed) overlays is populated outside the host library — typically via SFTP. The web app does not write into them.
+
+`workshop` overlays are populated by the web app: it downloads `.vpk` files from the public Steam Web API into `${LEFT4ME_ROOT}/workshop_cache/{steam_id}.vpk` and creates absolute symlinks under `${LEFT4ME_ROOT}/overlays/{overlay_id}/left4dead2/addons/{steam_id}.vpk`. Both the cache and the overlay directory are owned by the `left4me` runtime user; if the web service ever runs as a different uid, ensure it shares a group with the host process and that both trees are group-readable.

deploy/deploy-test-server.sh

@@ -95,6 +95,7 @@ $sudo_cmd mkdir -p \
     /var/lib/left4me/overlays \
     /var/lib/left4me/instances \
     /var/lib/left4me/runtime \
+    /var/lib/left4me/workshop_cache \
     /var/lib/left4me/tmp
 
 $sudo_cmd chown -R left4me:left4me /var/lib/left4me /opt/left4me

docs/superpowers/plans/2026-05-07-l4d2-workshop-overlays.md

@@ -0,0 +1,557 @@
+# L4D2 Workshop Overlays Implementation Plan
+
+> **Approval gate:** This plan may be written and refined without further approval. Do not implement code changes from this plan until the user explicitly approves implementation.
+
+**Goal:** Implement the workshop overlay feature per `docs/superpowers/specs/2026-05-07-l4d2-workshop-overlays-design.md`. Add a `WorkshopItem` registry, a typed `Overlay.type` column with a builder registry, a workshop builder that downloads from the Steam Web API and manages symlinks into a deduplicated cache, and the supporting routes, templates, jobs, and tests.
+
+**Architecture:** Keep the v1 single-process Flask architecture. New code is additive: a `WorkshopBuilder` class registered in a builder dispatcher, a `steam_workshop` service module for the Steam Web API and downloader, two new database tables and one extended one, and two new job operations on the existing in-process worker. fuse-overlayfs mount handling in `l4d2host` is unchanged — workshop content arrives at overlay paths the same way externals do today.
+
+---
+
+## Locked Decisions
+
+See `docs/superpowers/specs/2026-05-07-l4d2-workshop-overlays-design.md` for the design rationale. Implementation-relevant decisions:
+
+- Typed overlays: `external` (existing rows; no-op builder) and `workshop` (new); future types deferred.
+- No JSON `source_config` blob; per-type structured data in proper tables.
+- `WorkshopItem` is a global deduplicated registry keyed on `steam_id`. Cache at `/var/lib/left4me/workshop_cache/{steam_id}.vpk`.
+- Overlay symlinks are absolute, named `{steam_id}.vpk`; no Steam filename in any on-disk path.
+- `overlay_workshop_items` is a pure association; toggle = remove/re-add.
+- Collections are atomic UI bulk-imports; DB never tracks collection attribution.
+- Single global admin "Refresh all workshop items" button.
+- No cache GC in v1.
+- `Overlay.user_id` is the scope (NULL = system, set = private); independent of `type`.
+- Workshop overlays default to private; existing externals stay system-wide.
+- One unified Create-overlay button with type radio; no path field — paths are always `str(overlay_id)`.
+- `consumer_app_id == 550` validated at fetch/add; not stored.
+- Input field accepts numeric ID, full Workshop URL, or multi-line batch.
+- Auto-rebuild after add/remove with build coalescing.
+- HTTPS for all Steam Web API calls.
+- `Overlay.id` uses `AUTOINCREMENT`; `create_overlay_directory` uses `exist_ok=False`.
+- Two partial unique indexes for overlay names: `(name) WHERE user_id IS NULL` and `(name, user_id) WHERE user_id IS NOT NULL`.
+
+---
+
+## Current Gap
+
+- `Overlay` rows have `id`, `name`, `path`, no type, no scope.
+- The web app cannot download anything from Steam; users must SFTP `.vpk` files into prepared overlay directories.
+- The job worker has no operations for overlay builds or workshop refreshes.
+- The mount/build pipeline assumes overlay directories are externally populated.
+- There is no UI affordance to add or list workshop content.
+
+---
+
+## Task 1: Extend Tests First — Schema Migration And Models
+
+**Files:**
+- Create: `l4d2web/tests/test_workshop_overlay_models.py`
+- Modify: `l4d2web/tests/test_models.py` (extend) — partial unique index behavior
+
+Write tests against fresh SQLite schemas asserting:
+
+- An `Overlay` migration round-trip: existing rows acquire `type='external'` and `user_id=NULL`; their `name` values remain unique by partial index.
+- After migration, two externals (both `user_id=NULL`) with the same name are rejected by the system partial unique index.
+- After migration, two users may both own a workshop overlay named `"my-maps"` (per-user partial unique index).
+- `WorkshopItem.steam_id` is unique; concurrent inserts of the same `steam_id` raise integrity errors.
+- `overlay_workshop_items` enforces `UNIQUE(overlay_id, workshop_item_id)`.
+- `Overlay` deletion cascades `overlay_workshop_items` rows but does not delete `WorkshopItem` rows (`ON DELETE RESTRICT`).
+- `Job.overlay_id` is nullable and references `overlays(id)`.
+- `Overlay.id` does not reuse a deleted ID after the migration (AUTOINCREMENT).
+
+Verification command:
+
+```bash
+pytest l4d2web/tests/test_workshop_overlay_models.py l4d2web/tests/test_models.py -q
+```
+
+Expected before implementation: FAIL.
+
+---
+
+## Task 2: Schema Migration And ORM Mappings
+
+**Files:**
+- Create: `l4d2web/alembic/versions/0002_workshop_overlays.py`
+- Modify: `l4d2web/models.py`
+
+Migration `0002_workshop_overlays` (`down_revision = "b2c684fddbd3"`):
+
+1. `op.batch_alter_table("overlays")`:
+   - Add `type VARCHAR(16) NOT NULL DEFAULT 'external'` (server_default during migration; remove after backfill).
+   - Add `user_id INTEGER NULL REFERENCES users(id)`.
+   - Drop the existing `unique=True` on `name`.
+   - Add index `ix_overlays_type_user_id` on `(type, user_id)`.
+   - Switch `id` to `AUTOINCREMENT`.
+2. After batch alter, create the two partial unique indexes via raw `op.create_index(..., postgresql_where=..., sqlite_where=...)`:
+   - `uq_overlay_name_system` on `(name)` `WHERE user_id IS NULL`.
+   - `uq_overlay_name_per_user` on `(name, user_id)` `WHERE user_id IS NOT NULL`.
+3. `op.create_table("workshop_items", ...)` per spec data-model section.
+4. `op.create_table("overlay_workshop_items", ...)` with the unique constraint and the reverse-lookup index.
+5. `op.batch_alter_table("jobs")`: add `overlay_id INTEGER NULL REFERENCES overlays(id)`.
+
+ORM (`models.py`):
+
+- Extend `Overlay`: add `type`, `user_id`. Drop `unique=True` on `name`. Set `__table_args__` with the two partial indexes and `ix_overlays_type_user_id`.
+- Extend `Job`: add `overlay_id` mapped column with FK.
+- New `WorkshopItem` and `OverlayWorkshopItem` classes per spec. Set up `Overlay.workshop_items` relationship through the association.
+
+Verification command:
+
+```bash
+pytest l4d2web/tests/test_workshop_overlay_models.py l4d2web/tests/test_models.py -q
+```
+
+Expected after implementation: PASS.
+
+Run alembic against a fresh test DB to verify upgrade and downgrade succeed.
+
+---
+
+## Task 3: Tests First — Steam Web API And Downloader
+
+**Files:**
+- Create: `l4d2web/tests/test_steam_workshop.py`
+
+Mock HTTP with `responses` or `pytest-httpserver`. Cover:
+
+- `parse_workshop_input` accepts a single numeric ID, a single Workshop URL (`steamcommunity.com/sharedfiles/filedetails/?id=N`), and a multi-line whitespace-separated batch of either; returns deduplicated ordered list of digit-only IDs.
+- `parse_workshop_input` rejects garbage, paths outside `?id=`, non-digit IDs.
+- `resolve_collection` POSTs to the HTTPS endpoint with the form-encoded payload and returns `publishedfileid` children.
+- `fetch_metadata_batch` POSTs once with `itemcount=N`; returns parsed `WorkshopMetadata` per item; captures `result != 1` into `last_error`; raises `WorkshopValidationError` when any `consumer_app_id != 550` during user-add; logs and skips during refresh-mode.
+- `WorkshopMetadata.preview_url` is captured.
+- `download_to_cache` writes `cache_root/{steam_id}.vpk.partial`, then `os.replace` to the final name; sets `os.utime(file, (time_updated, time_updated))`.
+- `download_to_cache` is idempotent: a second call where on-disk `(mtime, size)` matches `(time_updated, file_size)` is a no-op (no HTTP request issued).
+- `refresh_all` runs downloads via `ThreadPoolExecutor(max_workers=8)` and reports per-item errors without aborting the batch.
+- All Steam API URLs use `https://`.
+
+Verification command:
+
+```bash
+pytest l4d2web/tests/test_steam_workshop.py -q
+```
+
+Expected before implementation: FAIL.
+
+---
+
+## Task 4: Steam Workshop Service Module
+
+**Files:**
+- Create: `l4d2web/services/steam_workshop.py`
+
+Public surface:
+
+```python
+def parse_workshop_input(raw: str) -> list[str]: ...
+def resolve_collection(collection_id: str) -> list[str]: ...
+def fetch_metadata_batch(steam_ids: list[str], *, mode: Literal["add","refresh"]) -> list[WorkshopMetadata]: ...
+def download_to_cache(meta: WorkshopMetadata, cache_root: Path, *, on_progress=None, should_cancel=None) -> Path: ...
+def refresh_all(items: list[WorkshopItem], cache_root: Path, executor_workers: int = 8) -> RefreshReport: ...
+```
+
+Implementation rules:
+
+- Endpoints are HTTPS:
+  - `https://api.steampowered.com/ISteamRemoteStorage/GetCollectionDetails/v1/`
+  - `https://api.steampowered.com/ISteamRemoteStorage/GetPublishedFileDetails/v1/`
+- Form-encoded POSTs with `itemcount=N` / `collectioncount=N` and `publishedfileids[i]=…` per index.
+- Per-request timeout 30s; per-item ceiling 5min. No retry or backoff in v1.
+- `consumer_app_id != 550`:
+  - In `mode="add"`: raise `WorkshopValidationError` with the offending `steam_id`.
+  - In `mode="refresh"`: log and skip; do not abort other items.
+- `result != 1`: capture Steam's result code in the item's `last_error`; do not download; do not abort siblings.
+- Cooperative cancellation: `download_to_cache` checks `should_cancel()` between chunked reads; `refresh_all`'s executor checks before each task.
+- `WorkshopMetadata` is a dataclass with `steam_id, title, filename, file_url, file_size, time_updated, preview_url, consumer_app_id, result`.
+- `RefreshReport` aggregates per-item outcomes for the caller's job log.
+- Use a single `requests.Session` per call site for connection reuse.
+
+Verification command:
+
+```bash
+pytest l4d2web/tests/test_steam_workshop.py -q
+```
+
+Expected after implementation: PASS.
+
+---
+
+## Task 5: Tests First — Path Helpers And Overlay Creation
+
+**Files:**
+- Create: `l4d2web/tests/test_workshop_paths.py`
+- Create: `l4d2web/tests/test_overlay_creation.py`
+
+Cover:
+
+- `workshop_cache_root()` returns `LEFT4ME_ROOT/workshop_cache`.
+- `cache_path(steam_id)` returns `cache_root / f"{steam_id}.vpk"` for valid digit strings; rejects non-digits, slashes, dot-dot.
+- `generate_overlay_path(overlay_id)` returns `str(overlay_id)`; passes `validate_overlay_ref` from `l4d2host.paths`.
+- `create_overlay_directory(overlay)` creates `LEFT4ME_ROOT/overlays/{path}/` with `exist_ok=False`. Calling twice raises (DB/disk drift surfaced loudly).
+
+Verification command:
+
+```bash
+pytest l4d2web/tests/test_workshop_paths.py l4d2web/tests/test_overlay_creation.py -q
+```
+
+Expected before implementation: FAIL.
+
+---
+
+## Task 6: Path Helpers And Overlay Creation
+
+**Files:**
+- Create: `l4d2web/services/workshop_paths.py`
+- Create: `l4d2web/services/overlay_creation.py`
+
+`workshop_paths`:
+
+```python
+def workshop_cache_root() -> Path: ...        # LEFT4ME_ROOT/workshop_cache
+def cache_path(steam_id: str) -> Path: ...    # validates digits-only; returns cache_root/{steam_id}.vpk
+```
+
+`overlay_creation`:
+
+```python
+def generate_overlay_path(overlay_id: int) -> str: ...   # str(overlay_id) + validate_overlay_ref
+def create_overlay_directory(overlay: Overlay) -> None:  # makedirs(..., exist_ok=False)
+    ...
+```
+
+Verification command:
+
+```bash
+pytest l4d2web/tests/test_workshop_paths.py l4d2web/tests/test_overlay_creation.py -q
+```
+
+Expected after implementation: PASS.
+
+---
+
+## Task 7: Tests First — Overlay Builders
+
+**Files:**
+- Create: `l4d2web/tests/test_overlay_builders.py`
+
+Cover with `tmp_path`:
+
+- `BUILDERS` dict resolves `"external"` and `"workshop"` to instances; unknown types raise `KeyError` (caller's error).
+- `ExternalBuilder.build()` is a no-op: makes the overlay directory if missing, writes one log line, returns. Existing files in the directory are untouched.
+- `WorkshopBuilder.build()` against a fixture overlay with three associated `WorkshopItem` rows (two with cache files present, one without):
+  - Creates `left4dead2/addons/` if missing.
+  - Creates symlinks `addons/{steam_id_a}.vpk → cache_root/{steam_id_a}.vpk` for items with cache files. Symlinks are absolute.
+  - Skips the uncached item; emits a warning log line. Does not create a dangling symlink.
+  - On a re-run with the same associations: no FS changes; logs report `unchanged=2 skipped(uncached)=1`.
+  - On a re-run after one association is removed: removes the obsolete symlink only; leaves cache files alone.
+  - On a re-run after one item is added: adds only the new symlink.
+  - Files in `addons/` that aren't symlinks into the cache are left untouched.
+- `should_cancel` mid-build: stops between filesystem ops; partial state is consistent and a re-run heals.
+
+Verification command:
+
+```bash
+pytest l4d2web/tests/test_overlay_builders.py -q
+```
+
+Expected before implementation: FAIL.
+
+---
+
+## Task 8: Overlay Builders And Dispatcher
+
+**Files:**
+- Create: `l4d2web/services/overlay_builders.py`
+
+```python
+class OverlayBuilder(Protocol):
+    def build(self, overlay: Overlay, *, on_stdout, on_stderr, should_cancel) -> None: ...
+
+class ExternalBuilder: ...
+class WorkshopBuilder: ...
+
+BUILDERS: dict[str, OverlayBuilder] = {
+    "external": ExternalBuilder(),
+    "workshop": WorkshopBuilder(),
+}
+```
+
+`WorkshopBuilder.build()`:
+
+1. Load the overlay's `WorkshopItem` rows.
+2. `os.makedirs(overlay_root / "left4dead2/addons", exist_ok=True)`.
+3. Compute `desired = {f"{steam_id}.vpk": cache_path(steam_id)}` for items where `last_downloaded_at IS NOT NULL` and the cache file exists. Skip and warn for items missing a cache file.
+4. Inspect existing entries in `addons/` via `os.scandir`: keep entries that are not symlinks into `workshop_cache`; otherwise diff against `desired` and apply changes via `os.unlink` and `os.symlink(absolute_target, link_path)`.
+5. Emit `created N, removed M, unchanged K, skipped (uncached) S` log line.
+6. Check `should_cancel()` between filesystem ops.
+
+Verification command:
+
+```bash
+pytest l4d2web/tests/test_overlay_builders.py -q
+```
+
+Expected after implementation: PASS.
+
+---
+
+## Task 9: Tests First — Worker Scheduler Truth Table And Coalescing
+
+**Files:**
+- Modify: `l4d2web/tests/test_job_worker.py`
+
+Add coverage:
+
+- Truth table for `can_start`:
+  - `install` not claimed while `refresh_workshop_items`, any `build_overlay`, or any server job is running.
+  - `refresh_workshop_items` not claimed while `install`, any `build_overlay`, or any server job is running.
+  - `build_overlay(N)` not claimed while `install`, `refresh_workshop_items`, or another `build_overlay(N)` is running. Two `build_overlay` jobs for **different** overlay IDs claim concurrently.
+  - Server start/init blocks if `refresh_workshop_items` runs or if any `build_overlay(N)` runs where N ∈ overlays of the server's blueprint.
+- `enqueue_build_overlay(overlay_id)`:
+  - Inserts a new queued job when no pending job exists.
+  - Returns the existing pending job when one is already queued (coalescing).
+  - Does not coalesce against running jobs (a new add after build start gets a fresh queued job).
+- `refresh_workshop_items` post-completion enqueues `build_overlay` only for overlays whose items had `time_updated` advance or `filename` change; each such enqueue uses the coalescing helper.
+
+Verification command:
+
+```bash
+pytest l4d2web/tests/test_job_worker.py -q
+```
+
+Expected before implementation: FAIL.
+
+---
+
+## Task 10: Worker Scheduler And New Operations
+
+**Files:**
+- Modify: `l4d2web/services/job_worker.py`
+
+Changes:
+
+- Define `OVERLAY_OPERATIONS = {"build_overlay"}` and `GLOBAL_OPERATIONS = {"install", "refresh_workshop_items"}`. Update `malformed_server_job` to allow `server_id IS NULL` for these.
+- Extend `SchedulerState` with `running_overlays: set[int]` and `refresh_running: bool`.
+- Update `claim_next_job()`:
+  - Compute `running_overlays` from queries against `running` jobs of operation `build_overlay`.
+  - Apply the truth-table rules above.
+  - Continue using `created_at, id` ordering for deterministic claim.
+- Add `enqueue_build_overlay(overlay_id: int) -> Job` helper:
+  - Look for `queued` `build_overlay` job with same `overlay_id`. Return it if present.
+  - Otherwise insert a new queued job with `overlay_id` set, `server_id=None`, `operation="build_overlay"`.
+- Update `run_job` dispatch:
+  - `build_overlay` → load `Overlay`, dispatch to `BUILDERS[overlay.type].build(overlay, on_stdout, on_stderr, should_cancel)`.
+  - `refresh_workshop_items` → call `steam_workshop.refresh_all(...)`. After completion, for each affected overlay, call `enqueue_build_overlay(overlay_id)`.
+
+Verification command:
+
+```bash
+pytest l4d2web/tests/test_job_worker.py -q
+```
+
+Expected after implementation: PASS.
+
+---
+
+## Task 11: Tests First — Routes, Permissions, And Auto-Rebuild
+
+**Files:**
+- Modify: `l4d2web/tests/test_overlays.py`
+- Create: `l4d2web/tests/test_workshop_routes.py`
+
+Cover:
+
+- `POST /overlays` with `type='workshop'` and `name` succeeds for any logged-in user; `path` is auto-generated; `user_id` is set; the directory exists at `LEFT4ME_ROOT/overlays/{id}`.
+- `POST /overlays` with `type='external'` succeeds only for admins; `user_id` is NULL.
+- Duplicate workshop name within the same user is rejected; duplicate names across users are accepted.
+- Duplicate external name is rejected.
+- Non-admins see `type='external' OR user_id=current_user.id` only when listing overlays.
+- `POST /overlays/{id}/items` with one numeric ID adds an association and enqueues a coalesced `build_overlay`. The response is an HTMX fragment of the updated item table.
+- `POST /overlays/{id}/items` with a multi-line batch (mix of IDs and URLs) adds all and enqueues one coalesced job for the batch.
+- `POST /overlays/{id}/items` with a collection ID resolves members and adds N associations.
+- Adding a non-L4D2 item (`consumer_app_id != 550`) returns HTTP 400 with a useful message; no association is created.
+- Adding an item already in the overlay returns "already in overlay" (no 500).
+- `POST /overlays/{id}/items/{item_id}/delete` removes the association and enqueues a coalesced build.
+- `POST /overlays/{id}/build` enqueues the manual rebuild and redirects to the job page.
+- `POST /admin/workshop/refresh` is admin-only; non-admins receive 403.
+
+Mock `steam_workshop` HTTP layer for these tests.
+
+Verification command:
+
+```bash
+pytest l4d2web/tests/test_overlays.py l4d2web/tests/test_workshop_routes.py -q
+```
+
+Expected before implementation: FAIL.
+
+---
+
+## Task 12: Routes And Templates
+
+**Files:**
+- Modify: `l4d2web/routes/overlay_routes.py`
+- Create: `l4d2web/routes/workshop_routes.py`
+- Modify: `l4d2web/routes/page_routes.py`
+- Modify: `l4d2web/templates/overlays.html`
+- Modify: `l4d2web/templates/overlay_detail.html`
+- Create: `l4d2web/templates/_overlay_item_table.html`
+- Modify: `l4d2web/templates/admin.html`
+- Modify: `l4d2web/app.py` (register the workshop blueprint)
+
+`overlay_routes.py`:
+
+- `create_overlay`: read `type` and `name` from form. No `path` field accepted.
+  - `type='external'`: admin-only; `user_id=NULL`. After insert, set `path = generate_overlay_path(id)`; call `create_overlay_directory(overlay)`.
+  - `type='workshop'`: any logged-in user; `user_id=current_user.id`. After insert, set `path = generate_overlay_path(id)`; call `create_overlay_directory(overlay)`.
+- `update_overlay`: forbid changing `type` and `path`. Workshop: owner or admin can edit `name`. External: admin-only `name` edits.
+- `delete_overlay`: after the row deletes, `shutil.rmtree(LEFT4ME_ROOT/overlays/{path})` only if `overlay.path == str(overlay.id)` (legacy externals are left alone). Cache untouched.
+
+`workshop_routes.py`:
+
+- `POST /overlays/{id}/items`: parse input via `parse_workshop_input`; if a collection ID, resolve members; batch-fetch metadata in `mode="add"`; reject non-550 with HTTP 400; upsert `WorkshopItem` via SQLite `INSERT ... ON CONFLICT DO UPDATE` on `steam_id`; bulk-add associations catching `(overlay_id, workshop_item_id)` unique violations; call `enqueue_build_overlay(overlay_id)`; return rendered `_overlay_item_table.html` fragment.
+- `POST /overlays/{id}/items/{item_id}/delete`: ownership check; remove association; call `enqueue_build_overlay(overlay_id)`; return updated fragment.
+- `POST /overlays/{id}/build`: ownership check; enqueue (coalesced); redirect to `/jobs/{job_id}`.
+- `POST /admin/workshop/refresh`: `@require_admin`; insert a `refresh_workshop_items` queued job; redirect to `/admin/jobs`.
+
+`page_routes.py`:
+
+- `overlays()`: admins see all; non-admins see `type='external' OR user_id=current_user.id`.
+- `overlay_detail()`: load `WorkshopItem` rows for workshop-type overlays.
+
+Templates:
+
+- `overlays.html`: add Type column. Modal has type radio (External | Workshop) and name field. No path field.
+- `overlay_detail.html`: branch on `overlay.type`.
+  - External view: read-only path display, name edit (admin only).
+  - Workshop view: an `<textarea>` accepting one or many IDs/URLs plus a radio (Items | Collection); item table with thumbnail (`preview_url`), `steam_id` linked to Steam, title, filename, time_updated, file_size, last_error, Remove; Rebuild button; small status indicator showing the latest related job.
+- `_overlay_item_table.html`: renderable standalone for HTMX swaps.
+- `admin.html`: add a CSRF-protected "Refresh all workshop items" button.
+
+Verification command:
+
+```bash
+pytest l4d2web/tests/test_overlays.py l4d2web/tests/test_workshop_routes.py -q
+```
+
+Expected after implementation: PASS.
+
+---
+
+## Task 13: Tests First — Initialize-Time Guard
+
+**Files:**
+- Modify: `l4d2web/tests/test_l4d2_facade.py` (or create if missing)
+
+Cover:
+
+- `initialize_server(server_id)` calls `BUILDERS[overlay.type].build()` for each overlay in the blueprint before writing the spec.
+- For workshop overlays, when an associated `WorkshopItem` lacks a cache file (`workshop_cache/{steam_id}.vpk` missing), `initialize_server` raises a clear error containing the missing `steam_id`s and the overlay name; the spec is not written; `l4d2ctl initialize` is not invoked.
+- For workshop overlays where all items have cache files, the symlinks are present and `l4d2ctl initialize` runs.
+
+Verification command:
+
+```bash
+pytest l4d2web/tests/test_l4d2_facade.py -q
+```
+
+Expected before implementation: FAIL.
+
+---
+
+## Task 14: Initialize-Time Guard
+
+**Files:**
+- Modify: `l4d2web/services/l4d2_facade.py`
+
+Implementation:
+
+- Before writing the temp spec, iterate over the blueprint's overlays and call `BUILDERS[overlay.type].build(...)`.
+- For workshop overlays, the builder logs and skips uncached items rather than failing. After all builders run, perform a second pass: query the blueprint's workshop overlays for any associated `WorkshopItem` with no cache file. If any are found, raise an exception whose message names the missing `steam_id`s and points at the overlay page (`Open overlay {name} ({id}) and click Build`).
+
+Verification command:
+
+```bash
+pytest l4d2web/tests/test_l4d2_facade.py -q
+```
+
+Expected after implementation: PASS.
+
+---
+
+## Task 15: Deploy Provisioning
+
+**Files:**
+- Modify: `deploy/install.sh` (or whichever provisioning script creates `/var/lib/left4me/`)
+- Modify: `deploy/README.md`
+
+Behavior:
+
+- Provisioning creates `/var/lib/left4me/workshop_cache/` (mode 0755), owned by the web user.
+- `deploy/README.md` documents:
+  - The new directory and its purpose.
+  - Permission requirement: web user owns; host user reads (shared group with `g+r` if uids differ).
+  - `LEFT4ME_ROOT` layout updated with the new subtree.
+
+No tests; verify via test deploy.
+
+---
+
+## Task 16: Full Verification And Manual Test Plan
+
+Run focused suites first:
+
+```bash
+pytest l4d2web/tests/test_workshop_overlay_models.py -q
+pytest l4d2web/tests/test_models.py -q
+pytest l4d2web/tests/test_steam_workshop.py -q
+pytest l4d2web/tests/test_workshop_paths.py l4d2web/tests/test_overlay_creation.py -q
+pytest l4d2web/tests/test_overlay_builders.py -q
+pytest l4d2web/tests/test_job_worker.py -q
+pytest l4d2web/tests/test_overlays.py l4d2web/tests/test_workshop_routes.py -q
+pytest l4d2web/tests/test_l4d2_facade.py -q
+```
+
+Then run the full web suite:
+
+```bash
+pytest l4d2web/tests -q
+```
+
+Manual test plan on the test deploy:
+
+1. Apply migration on a copy of the prod DB; verify all existing overlays read as `type='external'`, `user_id=NULL`; names still unique by partial index; two externals with the same name are rejected.
+2. As non-admin, create a workshop overlay. Add a known popular L4D2 addon by URL. Verify the build job auto-enqueues. Verify symlink + cache file. Confirm web UI shows metadata and thumbnail.
+3. Paste a multi-line block of item IDs and URLs. Verify all are parsed and added; verify coalescing (only one `build_overlay` job runs).
+4. Add a 50-item collection. Verify all 50 metadata rows appear and no UI mention of "from collection". Verify single coalesced build job.
+5. Remove an item. Verify auto-rebuild removes the symlink while the cache file remains.
+6. As admin, click Refresh All. Verify only items with newer `time_updated` re-download. Verify affected overlays get coalesced `build_overlay` jobs enqueued.
+7. Boot an L4D2 server with a workshop overlay attached. Connect locally and confirm the maps appear in the map vote and load.
+8. Concurrency probe: enqueue Refresh All while a `build_overlay` is queued; verify scheduler waits per truth table.
+9. Initialize-time guard: manually delete a cache file for an item that's in an overlay attached to a server's blueprint. Try to start the server; verify clear error mentioning the missing `steam_id`.
+10. Negative: paste a non-L4D2 workshop ID (e.g., a Skyrim mod). Expect HTTP 400 with a clear message; no row inserted.
+11. Negative: simulate Steam API down (block egress). Verify add fails with clean error, not 500. Verify refresh job logs the failure.
+
+---
+
+## Commit Strategy
+
+Use small commits after passing relevant tests:
+
+1. `feat(l4d2-web): typed overlays + workshop schema migration`
+2. `feat(l4d2-web): steam workshop API client and downloader`
+3. `feat(l4d2-web): overlay path helpers and creation`
+4. `feat(l4d2-web): overlay builder registry with workshop builder`
+5. `feat(l4d2-web): worker support for build_overlay and refresh_workshop_items`
+6. `feat(l4d2-web): workshop overlay UI (routes + templates)`
+7. `feat(l4d2-web): initialize-time guard for uncached workshop items`
+8. `feat(deploy): workshop_cache provisioning`
+
+Do not commit unless the user explicitly asks for commits.
+
+---
+
+## Open Approval Gate
+
+Before modifying implementation files, ask the user for explicit approval to proceed with the workshop-overlays implementation.

docs/superpowers/specs/2026-05-07-l4d2-workshop-overlays-design.md

@@ -0,0 +1,226 @@
+# L4D2 Workshop Overlays Design
+
+**Goal:** Let users add Steam Workshop content (.vpk addons and maps) to L4D2 servers from the web UI. Workshop downloads run as a new typed overlay that fits the existing `Overlay` + `BlueprintOverlay` model, downloaded via the public Steam Web API and exposed through the existing fuse-overlayfs mount layer.
+
+**Approval status:** User-approved design direction. Implementation proceeds in lockstep with the companion plan at `docs/superpowers/plans/2026-05-07-l4d2-workshop-overlays.md`.
+
+## Context
+
+`left4me` users today add `.vpk` content to a server only by SFTP-ing files into a manually-prepared overlay directory or by maintaining shell scripts (`competitive_rework`, `workshop_maps`, `tickrate`, etc.) that wrap `curl`/`steamcmd`. The web app exposes overlay rows but offers no way for users to populate them.
+
+This spec adds **workshop overlays**: a user-private overlay type that downloads `.vpk` files via the public `ISteamRemoteStorage` API and surfaces them through the existing mount layer. Users keep composing blueprints by stacking overlays — workshop overlays become another row alongside today's externally-managed ones.
+
+This is the first *typed* overlay. The design adds a `type` column and a builder-registry so future overlay types (tarball, inline, manual upload) plug in without schema churn or workflow changes.
+
+Steam Workshop content for L4D2 (consumer_app_id 550) is downloadable via two anonymous-POST endpoints with no Steam Web API key required: `GetCollectionDetails` resolves a collection ID to its child item IDs, and `GetPublishedFileDetails` returns per-item metadata including a public `file_url` for the `.vpk`. This is the same API the user's existing `steam-workshop-download` script uses.
+
+L4D2-specific player-side pain points (sv_consistency / RestrictAddons configuration gotchas, the inability to push workshop content via `sv_downloadurl`) are documented in **Out of scope** and tracked as separate follow-ups. This spec stays strictly on workshop content acquisition.
+
+## Locked Decisions
+
+1. **Typed overlays.** `Overlay.type` joins `external` (existing rows; admin-managed; no-op builder) and `workshop` (new). Future types — tarball, inline, manual upload — slot in via the same builder registry without schema churn.
+2. **No JSON `source_config` blob.** Per-type structured data lives in proper relational tables. JSON is reserved for genuinely opaque diagnostic payloads.
+3. **Central deduplicated `WorkshopItem` registry** keyed on `steam_id`. Cache lives at `/var/lib/left4me/workshop_cache/{steam_id}.vpk`. Multiple overlays referencing the same Steam item share the same cache file.
+4. **Symlinks, not copies.** Overlay directories contain `left4dead2/addons/{steam_id}.vpk` symlinks pointing into the cache. Both the cache file and the symlink are named by `{steam_id}` only — no Steam filename in any on-disk path, so Steam can rename the upstream `.vpk` without breaking lookup.
+5. **Many-to-many association is pure** (no `enabled` flag). Toggle a workshop item by removing or re-adding the association. The shared cache makes this cheap.
+6. **Collections are atomic UI bulk-imports.** Pasting a collection URL/ID resolves member items and creates N item associations. The DB never tracks "this came from a collection." Re-importing a collection is idempotent on existing items and additive for new ones.
+7. **Single global admin "Refresh all workshop items" button.** One Steam metadata batch call, then re-download items whose `time_updated` advanced. No per-item, per-overlay, or scheduled refresh in v1.
+8. **No cache GC in v1.** Cache grows monotonically. Reference-counted cleanup is a follow-up.
+9. **Globality is independent of overlay type.** `Overlay.user_id` is the scope (NULL = system-wide, set = private to that user). v1 defaults newly-created workshop overlays to private and leaves existing external overlays as system-wide. A future "publish/share" button will let owners toggle `user_id` without changing type.
+10. **One unified "Create overlay" UI button.** Modal has a type radio (External | Workshop). No path field — the web app generates the path for every new overlay.
+11. **Strict scope.** v1 ships only the workshop type. L4D2 server-config gotchas, client-subscription helpers, other recipe types — all deferred to follow-up specs.
+12. **`consumer_app_id == 550` validation** at every Steam API response at fetch/add time; non-L4D2 items are rejected and never reach the row. The value is a fixed precondition, not data.
+13. **Input field accepts numeric ID, full Workshop URL, or a multi-line batch** of either. Pasting `123456` and pasting `steamcommunity.com/sharedfiles/filedetails/?id=123456` produce the same result; pasting many of either at once works too.
+14. **Web-managed overlay paths.** All new overlays (any type) get `path = str(overlay_id)` at insert time. The user never picks a path. Existing legacy external overlay rows keep their current path values; migrating them to the ID-based scheme is a follow-up. `Overlay.id` uses SQLite `AUTOINCREMENT` so deleted IDs are never reused.
+15. **Auto-rebuild on item change.** Adding or removing items from a workshop overlay automatically enqueues a `build_overlay` job. The "Rebuild" button on the detail page is for manual recovery only. New build jobs for an overlay coalesce with any pending one for the same overlay (don't queue duplicates).
+16. **HTTPS** for all Steam Web API calls. The reference downloader uses HTTP; we don't.
+
+## Architecture
+
+```
+Overlay row (type=workshop)
+   └─refs─▶ overlay_workshop_items
+                  └─▶ WorkshopItem (global, by steam_id)
+                          │
+                          ▼ download (Steam GetPublishedFileDetails + HTTP GET)
+                     workshop_cache/{steam_id}.vpk
+                          ▲
+overlay_dir/left4dead2/addons/{steam_id}.vpk ─symlink─┘
+```
+
+Build dispatch via a registry:
+
+```python
+BUILDERS = {"external": ExternalBuilder(), "workshop": WorkshopBuilder()}
+
+def build_overlay(overlay_id):
+    overlay = db.get(Overlay, overlay_id)
+    BUILDERS[overlay.type].build(overlay, on_stdout, on_stderr, should_cancel)
+```
+
+`ExternalBuilder` is a no-op for legacy admin-managed dirs. `WorkshopBuilder` performs an idempotent diff-apply of `addons/` symlinks against the current associations. Future types add their own builders without changing the dispatcher, the mount layer, or the blueprint editor.
+
+## Data Model
+
+### `Overlay` (extended)
+
+```
+id INTEGER PK AUTOINCREMENT
+name VARCHAR(255) NOT NULL
+path VARCHAR(255) NOT NULL                 -- new overlays: str(id); legacy externals: existing values
+type VARCHAR(16) NOT NULL                  -- 'external' | 'workshop' (extensible)
+user_id INTEGER NULL REFERENCES users(id)  -- NULL = system-wide
+created_at, updated_at
+
+UNIQUE INDEX on (name) WHERE user_id IS NULL    -- system overlays globally unique by name
+UNIQUE INDEX on (name, user_id) WHERE user_id IS NOT NULL  -- per-user namespace
+INDEX on (type, user_id)
+```
+
+Two partial unique indexes are required because a naive composite `UNIQUE(name, user_id)` doesn't constrain externals — SQLite treats NULL as distinct in unique constraints, so two externals could share a name. Partial indexes preserve the prior global-uniqueness invariant for system rows.
+
+### `WorkshopItem` (new)
+
+```
+id INTEGER PK
+steam_id VARCHAR(20) NOT NULL UNIQUE        -- 64-bit, store as text
+title VARCHAR(255) NOT NULL DEFAULT ''
+filename VARCHAR(255) NOT NULL DEFAULT ''   -- upstream Steam filename, display only
+file_url TEXT NOT NULL DEFAULT ''
+file_size BIGINT NOT NULL DEFAULT 0
+time_updated INTEGER NOT NULL DEFAULT 0     -- Steam epoch
+preview_url TEXT NOT NULL DEFAULT ''        -- thumbnail URL hot-linked from Steam
+last_downloaded_at DATETIME NULL
+last_error TEXT NOT NULL DEFAULT ''
+created_at, updated_at
+```
+
+`consumer_app_id` is **not** stored. It's validated at fetch time and the row never exists for non-L4D2 items.
+
+### `overlay_workshop_items` (new, pure association)
+
+```
+id INTEGER PK
+overlay_id INTEGER NOT NULL REFERENCES overlays(id) ON DELETE CASCADE
+workshop_item_id INTEGER NOT NULL REFERENCES workshop_items(id) ON DELETE RESTRICT
+UNIQUE (overlay_id, workshop_item_id)
+INDEX (workshop_item_id)                    -- reverse lookup for refresh
+```
+
+No `enabled` column — toggle is remove/add, which is cheap because the cache survives.
+
+### `Job` (extended)
+
+Add `overlay_id INTEGER NULL REFERENCES overlays(id)` for `build_overlay` jobs.
+
+## Filesystem Layout
+
+```
+/var/lib/left4me/
+  overlays/
+    {overlay_id}/                                   # flat — same shape for every type
+      left4dead2/addons/
+        {steam_id}.vpk -> /var/lib/left4me/workshop_cache/{steam_id}.vpk
+  workshop_cache/
+    {steam_id}.vpk                                  # one file per Steam item
+```
+
+- Every new overlay (workshop, future tarball/inline/manual) lives at `overlays/{overlay_id}/`. Legacy external overlays keep their pre-migration paths (e.g. `overlays/standard/`).
+- `workshop_cache/` is created during deploy provisioning, not lazily — avoids races between concurrent first downloads.
+- Web user owns both trees (mode 0755). Host user (`l4d2ctl`) needs read on both. If web and host are different users, they share a group.
+- Symlink targets are absolute. Relative targets resolve in the merged-mount namespace and break across the host/web boundary.
+- The builder never creates a dangling symlink. If a `WorkshopItem` lacks a cache file at build time, the builder logs a warning and skips it — fuse-overlayfs surfaces broken links to L4D2 as opaque addon-scan failures.
+
+## UI
+
+A single "Create overlay" button on `/overlays` opens a modal with type radio (External | Workshop) and a name field. No path field. The web app generates `path = str(overlay_id)` after insert.
+
+Workshop overlay detail page (`/overlays/{id}` when `type='workshop'`) shows:
+
+- A multi-line input plus a radio (Items | Collection). Pasting one or many IDs/URLs adds them in order; pasting a collection ID resolves its members.
+- An item table with: thumbnail (`preview_url`), `steam_id` linking to Steam, title, filename, last-updated, size, last-error if any, Remove.
+- A manual "Rebuild" button (for recovery only — every add/remove auto-enqueues a coalesced `build_overlay` job).
+- Status indicator pulled from the latest related `Job` row.
+
+External overlay detail page is unchanged in shape: read-only path display, name edit (admin only). The "External" type retains the existing admin-only SFTP-to-disk workflow until a future "manual upload" type replaces it.
+
+The blueprint editor is unchanged in structure. Workshop overlays appear alongside externals in the user's overlay picker; ordering and stacking semantics are identical.
+
+Admin section gets one new control: "Refresh all workshop items" button on the admin landing or workshop subsection. Pressing it enqueues a single `refresh_workshop_items` job.
+
+### Routes
+
+| Method | Path | Purpose |
+|---|---|---|
+| GET | `/overlays` | List with Type column, filtered by user permissions |
+| POST | `/overlays` | Create; reads `type` and `name` only |
+| GET | `/overlays/{id}` | Type-aware detail page |
+| POST | `/overlays/{id}/items` | Add items or collection; auto-enqueues coalesced `build_overlay` |
+| POST | `/overlays/{id}/items/{item_id}/delete` | Remove association; auto-enqueues coalesced `build_overlay` |
+| POST | `/overlays/{id}/build` | Manual rebuild (recovery) |
+| POST | `/admin/workshop/refresh` | Admin only; enqueue `refresh_workshop_items` |
+
+HTMX usage stays minimal: only the add-item form and per-row delete swap a fragment. Everything else is full-page POST/redirect/GET.
+
+## Job Operations
+
+Two new operations join the existing job worker:
+
+- **`build_overlay(overlay_id)`** — `Job.overlay_id` is set; `server_id` is NULL. Dispatches to `BUILDERS[overlay.type].build(...)`. Cancellation between filesystem operations.
+- **`refresh_workshop_items()`** — admin-only. Both `server_id` and `overlay_id` are NULL. Phases: fetch all metadata in one batched call, download items where `time_updated` advanced, enqueue (coalesced) `build_overlay` for affected overlays. v1 doesn't wait on child builds; the admin sees them in the jobs list.
+
+### Scheduler rules
+
+- `install` and `refresh_workshop_items` are mutually exclusive with each other, with all `build_overlay`s, and with all server jobs.
+- `build_overlay(overlay_id=N)` blocks if `install_running`, `refresh_running`, or another build for the same `overlay_id` is running. Builds for *different* overlays may run concurrently.
+- Server start/init blocks if `refresh_running` or any `build_overlay` for an overlay referenced by the server's blueprint is running.
+
+Coalescing: a new `build_overlay` for an overlay that already has a queued (not-yet-running) build returns the existing job instead of inserting a new row.
+
+`initialize_server` synchronously calls each overlay's builder before writing the spec for `l4d2ctl initialize`. If a workshop overlay references uncached items (no file in `workshop_cache/`), `initialize_server` fails fast with a clear error naming the missing IDs and pointing the user at the overlay page. It never silently mounts a partial overlay.
+
+## Permissions
+
+- **External overlays**: admin-only create/edit. Visible to all authenticated users (system-wide).
+- **Workshop overlays**: any logged-in user can create. Owner or admin can edit and delete. Visible to the owner and admins.
+- **Admin refresh**: admin-only.
+
+The `Overlay` listing query for non-admins becomes: `type='external' OR user_id=current_user.id`.
+
+## Risks
+
+- **Broken symlinks across host/web boundary** — mitigated by absolute targets, build-time pre-check skipping uncached items, and `deploy/` documenting permission requirements.
+- **Initialize against uncached items** — would silently mount overlays missing maps. Mitigated by `initialize_server`'s fail-fast check; tested.
+- **Steam API rate limits** — refresh of 100 items is one metadata POST plus 100 downloads at 8-way parallelism. No retry/backoff in v1; 429s surface verbatim in the job log.
+- **Partial failure during refresh** — each item is independent; per-item errors land on the row. Re-running refresh retries failures.
+- **Concurrent same-ID adds** — `WorkshopItem.steam_id` unique handles cache dedup. `(overlay_id, workshop_item_id)` unique catches double-association; the route returns "already in overlay" rather than 500.
+- **Build coalescing missed** — would enqueue dozens of redundant builds during multi-item adds. Mitigated by the `enqueue_build_overlay` helper; tested.
+- **Worker concurrency rule miss** — the truth-table test in `test_job_worker.py` is the only way to trust the new scheduler logic; written before dispatch.
+- **DB/disk drift** — a stray directory left by a prior failed delete could shadow a fresh overlay. Mitigated by `AUTOINCREMENT` (no ID reuse) and `os.makedirs(exist_ok=False)` (loud failure on collision).
+- **Partial unique gap on SQLite** — naive composite `UNIQUE(name, user_id)` doesn't constrain externals because NULL is distinct. Mitigated by two partial unique indexes; tested explicitly.
+- **Cache growth without GC** — accepted v1 trade-off.
+- **Item removed from Steam** — refresh marks `result != 1`; row keeps last good cache file; UI surfaces error string. Operator decides removal.
+- **L4D2 containerized run** — symlink absolute targets break if the server runs in a different mount namespace. Re-evaluate when containerization comes up.
+
+## Out Of Scope
+
+These came up in research and dialog but stay out of v1:
+
+- **Publish / share button on overlays.** Lets owners flip `Overlay.user_id` between their own ID and NULL without changing type. The schema already supports it; only the UI is deferred.
+- **Migrate legacy external overlay paths to the ID-based scheme.** Existing external rows keep their pre-migration paths in v1; a follow-up migration moves the directories on disk and updates the rows.
+- **Switch from fuse-overlayfs to kernel overlayfs via a privileged helper.** Matches the existing systemd / steam-install sudoers helper pattern under `/usr/local/libexec/left4me/`. Workshop overlays would work identically under either mount engine — symlinks resolve through normal VFS in both.
+- **`sv_consistency` / `addonconfig.cfg RestrictAddons` auto-handling.** When a workshop overlay attaches to a blueprint, surface a banner with a one-click fix. Most-cited L4D2 player pain.
+- **Shareable Steam Workshop collection link for clients.** Server cannot push workshop content via `sv_downloadurl`; clients must subscribe themselves. A panel-generated collection makes that one click for players. Requires Steam OAuth.
+- **Other overlay types.** `tarball` (covers the old `competitive_rework` GitHub-tarball recipe), `inline` (covers `tickrate`'s inline `server.cfg`), `manual` (file manager / upload, replaces the admin-SFTP external workflow). All slot in via the builder registry without schema churn.
+- **Cache GC.** Reference-counted delete or admin "Clear unreferenced" page.
+- **Per-item / per-overlay / scheduled refresh.** v1 has one global admin button; revisit if users want finer control.
+- **Update-aware server restart UX.** Notify users when a running server's overlay content has been refreshed underneath it.
+
+## Implementation Boundaries
+
+- The host library contract is unchanged. Workshop content arrives in overlay directories the same way externals do today; `l4d2host` doesn't know overlays have types.
+- The job-execution model is preserved: same workers, same logs, same cancel callbacks. Only the operations table grows.
+- The blueprint privacy model and desired-vs-actual server state model are unchanged.
+- No new frontend dependencies. Vendored HTMX + custom CSS + small inline JS.
+- No new Steam Web API key required; both endpoints used accept anonymous POSTs.
+- The companion implementation plan governs task ordering and verification commands. Implementation must not start without explicit user approval per that plan's gate.

l4d2web/alembic/versions/0002_workshop_overlays.py

@@ -0,0 +1,174 @@
+"""workshop overlays
+
+Revision ID: 0002_workshop_overlays
+Revises: b2c684fddbd3
+Create Date: 2026-05-07
+"""
+from typing import Sequence, Union
+
+from alembic import op
+import sqlalchemy as sa
+
+
+revision: str = "0002_workshop_overlays"
+down_revision: Union[str, Sequence[str], None] = "b2c684fddbd3"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def _existing_overlays_table() -> sa.Table:
+    """Pre-migration shape used as `copy_from` so batch_alter_table rebuilds
+    overlays without the inline UNIQUE on `name` (replaced by partial unique
+    indexes after the recreate)."""
+    metadata = sa.MetaData()
+    return sa.Table(
+        "overlays",
+        metadata,
+        sa.Column("id", sa.Integer(), primary_key=True),
+        sa.Column("name", sa.String(length=128), nullable=False),
+        sa.Column("path", sa.String(length=512), nullable=False),
+        sa.Column("created_at", sa.DateTime(), nullable=False),
+        sa.Column("updated_at", sa.DateTime(), nullable=False),
+    )
+
+
+def upgrade() -> None:
+    # 1. Recreate `overlays` with `type`, `user_id`, autoincrement, and no inline UNIQUE on name.
+    with op.batch_alter_table(
+        "overlays",
+        recreate="always",
+        copy_from=_existing_overlays_table(),
+        table_kwargs={"sqlite_autoincrement": True},
+    ) as batch_op:
+        batch_op.add_column(
+            sa.Column(
+                "type",
+                sa.String(length=16),
+                nullable=False,
+                server_default="external",
+            )
+        )
+        batch_op.add_column(
+            sa.Column(
+                "user_id",
+                sa.Integer(),
+                sa.ForeignKey("users.id", name="fk_overlays_user_id_users"),
+                nullable=True,
+            )
+        )
+        batch_op.create_index("ix_overlays_type_user_id", ["type", "user_id"])
+
+    # Drop the temporary server_default once existing rows are backfilled.
+    with op.batch_alter_table("overlays") as batch_op:
+        batch_op.alter_column("type", server_default=None)
+
+    # 2. Partial unique indexes for name uniqueness:
+    #    - system overlays (user_id IS NULL): globally unique by name
+    #    - user overlays (user_id IS NOT NULL): unique per user by name
+    op.create_index(
+        "uq_overlay_name_system",
+        "overlays",
+        ["name"],
+        unique=True,
+        sqlite_where=sa.text("user_id IS NULL"),
+    )
+    op.create_index(
+        "uq_overlay_name_per_user",
+        "overlays",
+        ["name", "user_id"],
+        unique=True,
+        sqlite_where=sa.text("user_id IS NOT NULL"),
+    )
+
+    # 3. workshop_items registry (global, deduplicated by steam_id).
+    op.create_table(
+        "workshop_items",
+        sa.Column("id", sa.Integer(), primary_key=True),
+        sa.Column("steam_id", sa.String(length=20), nullable=False, unique=True),
+        sa.Column("title", sa.String(length=255), nullable=False, server_default=""),
+        sa.Column("filename", sa.String(length=255), nullable=False, server_default=""),
+        sa.Column("file_url", sa.Text(), nullable=False, server_default=""),
+        sa.Column("file_size", sa.BigInteger(), nullable=False, server_default="0"),
+        sa.Column("time_updated", sa.Integer(), nullable=False, server_default="0"),
+        sa.Column("preview_url", sa.Text(), nullable=False, server_default=""),
+        sa.Column("last_downloaded_at", sa.DateTime(), nullable=True),
+        sa.Column("last_error", sa.Text(), nullable=False, server_default=""),
+        sa.Column("created_at", sa.DateTime(), nullable=False),
+        sa.Column("updated_at", sa.DateTime(), nullable=False),
+    )
+
+    # 4. overlay_workshop_items association.
+    op.create_table(
+        "overlay_workshop_items",
+        sa.Column("id", sa.Integer(), primary_key=True),
+        sa.Column(
+            "overlay_id",
+            sa.Integer(),
+            sa.ForeignKey("overlays.id", ondelete="CASCADE"),
+            nullable=False,
+        ),
+        sa.Column(
+            "workshop_item_id",
+            sa.Integer(),
+            sa.ForeignKey("workshop_items.id", ondelete="RESTRICT"),
+            nullable=False,
+        ),
+        sa.UniqueConstraint(
+            "overlay_id", "workshop_item_id", name="uq_overlay_workshop_item"
+        ),
+    )
+    op.create_index(
+        "ix_owi_workshop_item",
+        "overlay_workshop_items",
+        ["workshop_item_id"],
+    )
+
+    # 5. Add overlay_id to jobs for build_overlay tracking.
+    with op.batch_alter_table("jobs") as batch_op:
+        batch_op.add_column(
+            sa.Column(
+                "overlay_id",
+                sa.Integer(),
+                sa.ForeignKey("overlays.id", name="fk_jobs_overlay_id_overlays"),
+                nullable=True,
+            )
+        )
+
+
+def downgrade() -> None:
+    with op.batch_alter_table("jobs") as batch_op:
+        batch_op.drop_column("overlay_id")
+
+    op.drop_index("ix_owi_workshop_item", table_name="overlay_workshop_items")
+    op.drop_table("overlay_workshop_items")
+    op.drop_table("workshop_items")
+
+    op.drop_index("uq_overlay_name_per_user", table_name="overlays")
+    op.drop_index("uq_overlay_name_system", table_name="overlays")
+    op.drop_index("ix_overlays_type_user_id", table_name="overlays")
+
+    # Recreate `overlays` to drop type/user_id and restore single-column UNIQUE on name.
+    current_overlays = sa.Table(
+        "overlays",
+        sa.MetaData(),
+        sa.Column("id", sa.Integer(), primary_key=True),
+        sa.Column("name", sa.String(length=128), nullable=False),
+        sa.Column("path", sa.String(length=512), nullable=False),
+        sa.Column("type", sa.String(length=16), nullable=False),
+        sa.Column(
+            "user_id",
+            sa.Integer(),
+            sa.ForeignKey("users.id", name="fk_overlays_user_id_users"),
+            nullable=True,
+        ),
+        sa.Column("created_at", sa.DateTime(), nullable=False),
+        sa.Column("updated_at", sa.DateTime(), nullable=False),
+    )
+    with op.batch_alter_table(
+        "overlays",
+        recreate="always",
+        copy_from=current_overlays,
+    ) as batch_op:
+        batch_op.drop_column("user_id")
+        batch_op.drop_column("type")
+        batch_op.create_unique_constraint("uq_overlays_name", ["name"])

l4d2web/app.py

@@ -16,6 +16,7 @@ from l4d2web.routes.log_routes import bp as log_bp
 from l4d2web.routes.overlay_routes import bp as overlay_bp
 from l4d2web.routes.page_routes import bp as page_bp
 from l4d2web.routes.server_routes import bp as server_bp
+from l4d2web.routes.workshop_routes import bp as workshop_bp
 from l4d2web.services.job_worker import recover_stale_jobs, start_job_workers
 
 
@@ -69,6 +70,7 @@ def create_app(test_config: dict[str, object] | None = None) -> Flask:
     app.before_request(load_current_user)
     app.register_blueprint(auth_bp)
     app.register_blueprint(overlay_bp)
+    app.register_blueprint(workshop_bp)
     app.register_blueprint(blueprint_bp)
     app.register_blueprint(server_bp)
     app.register_blueprint(job_bp)

l4d2web/models.py

@@ -1,6 +1,17 @@
 from datetime import UTC, datetime
 
-from sqlalchemy import Boolean, DateTime, ForeignKey, Integer, String, Text
+from sqlalchemy import (
+    BigInteger,
+    Boolean,
+    DateTime,
+    ForeignKey,
+    Index,
+    Integer,
+    String,
+    Text,
+    UniqueConstraint,
+    text,
+)
 from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column
 
 
@@ -25,14 +36,66 @@ class User(Base):
 
 class Overlay(Base):
     __tablename__ = "overlays"
+    __table_args__ = (
+        Index(
+            "uq_overlay_name_system",
+            "name",
+            unique=True,
+            sqlite_where=text("user_id IS NULL"),
+        ),
+        Index(
+            "uq_overlay_name_per_user",
+            "name",
+            "user_id",
+            unique=True,
+            sqlite_where=text("user_id IS NOT NULL"),
+        ),
+        Index("ix_overlays_type_user_id", "type", "user_id"),
+        {"sqlite_autoincrement": True},
+    )
 
-    id: Mapped[int] = mapped_column(Integer, primary_key=True)
-    name: Mapped[str] = mapped_column(String(128), unique=True, nullable=False)
+    id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
+    name: Mapped[str] = mapped_column(String(128), nullable=False)
     path: Mapped[str] = mapped_column(String(512), nullable=False)
+    type: Mapped[str] = mapped_column(String(16), nullable=False, default="external")
+    user_id: Mapped[int | None] = mapped_column(ForeignKey("users.id"), nullable=True)
     created_at: Mapped[datetime] = mapped_column(DateTime, default=now_utc, nullable=False)
     updated_at: Mapped[datetime] = mapped_column(DateTime, default=now_utc, nullable=False)
 
 
+class WorkshopItem(Base):
+    __tablename__ = "workshop_items"
+
+    id: Mapped[int] = mapped_column(Integer, primary_key=True)
+    steam_id: Mapped[str] = mapped_column(String(20), unique=True, nullable=False)
+    title: Mapped[str] = mapped_column(String(255), default="", nullable=False)
+    filename: Mapped[str] = mapped_column(String(255), default="", nullable=False)
+    file_url: Mapped[str] = mapped_column(Text, default="", nullable=False)
+    file_size: Mapped[int] = mapped_column(BigInteger, default=0, nullable=False)
+    time_updated: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
+    preview_url: Mapped[str] = mapped_column(Text, default="", nullable=False)
+    last_downloaded_at: Mapped[datetime | None] = mapped_column(DateTime, nullable=True)
+    last_error: Mapped[str] = mapped_column(Text, default="", nullable=False)
+    created_at: Mapped[datetime] = mapped_column(DateTime, default=now_utc, nullable=False)
+    updated_at: Mapped[datetime] = mapped_column(DateTime, default=now_utc, nullable=False)
+
+
+class OverlayWorkshopItem(Base):
+    __tablename__ = "overlay_workshop_items"
+    __table_args__ = (
+        UniqueConstraint("overlay_id", "workshop_item_id", name="uq_overlay_workshop_item"),
+        Index("ix_owi_workshop_item", "workshop_item_id"),
+    )
+
+    id: Mapped[int] = mapped_column(Integer, primary_key=True)
+    overlay_id: Mapped[int] = mapped_column(
+        ForeignKey("overlays.id", ondelete="CASCADE"), nullable=False
+    )
+    workshop_item_id: Mapped[int] = mapped_column(
+        ForeignKey("workshop_items.id", ondelete="RESTRICT"), nullable=False
+    )
+
+
 class Blueprint(Base):
     __tablename__ = "blueprints"
 
@@ -78,6 +141,7 @@ class Job(Base):
     id: Mapped[int] = mapped_column(Integer, primary_key=True)
     user_id: Mapped[int] = mapped_column(ForeignKey("users.id"), nullable=False)
     server_id: Mapped[int | None] = mapped_column(ForeignKey("servers.id"), nullable=True)
+    overlay_id: Mapped[int | None] = mapped_column(ForeignKey("overlays.id"), nullable=True)
     operation: Mapped[str] = mapped_column(String(32), nullable=False)
     state: Mapped[str] = mapped_column(String(16), default="queued", nullable=False)
     exit_code: Mapped[int | None] = mapped_column(Integer, nullable=True)

l4d2web/pyproject.toml

@@ -14,6 +14,7 @@ dependencies = [
   "alembic>=1.13",
   "PyYAML>=6.0",
   "gunicorn>=22.0",
+  "requests>=2.31",
 ]
 
 [tool.setuptools]

l4d2web/routes/overlay_routes.py

@@ -1,72 +1,132 @@
+import shutil
+
 from flask import Blueprint, Response, redirect, request
 from sqlalchemy import select
 
-from l4d2web.auth import require_admin
+from l4d2host.paths import get_left4me_root
+
+from l4d2web.auth import current_user, require_login
 from l4d2web.db import session_scope
 from l4d2web.models import BlueprintOverlay, Overlay
-from l4d2web.services.security import validate_overlay_ref
+from l4d2web.services.overlay_creation import (
+    create_overlay_directory,
+    generate_overlay_path,
+)
 
 
 bp = Blueprint("overlay", __name__)
 
 
-@bp.post("/overlays")
-@require_admin
-def create_overlay() -> Response:
-    name = request.form.get("name", "").strip()
-    raw_path = request.form.get("path", "")
-    if not name or not raw_path:
-        return Response("missing fields", status=400)
+VALID_TYPES = {"external", "workshop"}
 
-    try:
-        overlay_ref = validate_overlay_ref(raw_path)
-    except ValueError as exc:
-        return Response(str(exc), status=400)
+
+def _is_managed_path(overlay: Overlay) -> bool:
+    return overlay.path == str(overlay.id)
+
+
+def _can_edit_overlay(overlay: Overlay, user) -> bool:
+    if user is None:
+        return False
+    if user.admin:
+        return True
+    if overlay.type == "external":
+        return False
+    if overlay.type == "workshop":
+        return overlay.user_id == user.id
+    return False
+
+
+def _name_already_taken(db, name: str, scope_user_id: int | None, *, except_id: int | None = None) -> bool:
+    query = select(Overlay).where(Overlay.name == name)
+    if scope_user_id is None:
+        query = query.where(Overlay.user_id.is_(None))
+    else:
+        query = query.where(Overlay.user_id == scope_user_id)
+    if except_id is not None:
+        query = query.where(Overlay.id != except_id)
+    return db.scalar(query) is not None
+
+
+@bp.post("/overlays")
+@require_login
+def create_overlay() -> Response:
+    user = current_user()
+    assert user is not None
+
+    name = request.form.get("name", "").strip()
+    overlay_type = request.form.get("type", "external").strip().lower()
+    if not name:
+        return Response("missing fields", status=400)
+    if overlay_type not in VALID_TYPES:
+        return Response(f"unknown overlay type: {overlay_type}", status=400)
+
+    if overlay_type == "external":
+        if not user.admin:
+            return Response("admin only", status=403)
+        scope_user_id: int | None = None
+    else:  # workshop
+        scope_user_id = user.id
 
     with session_scope() as db:
-        existing = db.scalar(select(Overlay).where(Overlay.name == name))
-        if existing is not None:
+        if _name_already_taken(db, name, scope_user_id):
             return Response("overlay already exists", status=409)
-        db.add(Overlay(name=name, path=overlay_ref))
 
-    return redirect("/overlays")
+        overlay = Overlay(name=name, path="", type=overlay_type, user_id=scope_user_id)
+        db.add(overlay)
+        db.flush()
+        overlay.path = generate_overlay_path(overlay.id)
+        db.flush()
+        create_overlay_directory(overlay)
+        new_id = overlay.id
+
+    return redirect(f"/overlays/{new_id}")
 
 
 @bp.post("/overlays/<int:overlay_id>")
-@require_admin
+@require_login
 def update_overlay(overlay_id: int) -> Response:
-    name = request.form.get("name", "").strip()
-    raw_path = request.form.get("path", "")
-    if not name or not raw_path:
-        return Response("missing fields", status=400)
+    user = current_user()
+    assert user is not None
 
-    try:
-        overlay_ref = validate_overlay_ref(raw_path)
-    except ValueError as exc:
-        return Response(str(exc), status=400)
+    name = request.form.get("name", "").strip()
+    if not name:
+        return Response("missing fields", status=400)
 
     with session_scope() as db:
         overlay = db.scalar(select(Overlay).where(Overlay.id == overlay_id))
         if overlay is None:
             return Response(status=404)
-        duplicate = db.scalar(select(Overlay).where(Overlay.name == name, Overlay.id != overlay_id))
-        if duplicate is not None:
+        if not _can_edit_overlay(overlay, user):
+            return Response(status=403)
+        if _name_already_taken(db, name, overlay.user_id, except_id=overlay_id):
             return Response("overlay already exists", status=409)
         overlay.name = name
-        overlay.path = overlay_ref
 
     return redirect(f"/overlays/{overlay_id}")
 
 
 @bp.post("/overlays/<int:overlay_id>/delete")
-@require_admin
+@require_login
 def delete_overlay(overlay_id: int) -> Response:
+    user = current_user()
+    assert user is not None
+
     with session_scope() as db:
         overlay = db.scalar(select(Overlay).where(Overlay.id == overlay_id))
         if overlay is None:
             return Response(status=404)
+        if not _can_edit_overlay(overlay, user):
+            return Response(status=403)
         in_use = db.scalar(select(BlueprintOverlay).where(BlueprintOverlay.overlay_id == overlay_id))
         if in_use is not None:
             return Response("overlay is in use", status=409)
+        path_value = overlay.path
+        path_is_managed = _is_managed_path(overlay)
         db.delete(overlay)
+
+    if path_is_managed and path_value:
+        target = get_left4me_root() / "overlays" / path_value
+        if target.exists():
+            shutil.rmtree(target)
+
     return redirect("/overlays")

l4d2web/routes/page_routes.py

@@ -6,7 +6,15 @@ from sqlalchemy import select
 from l4d2web.auth import current_user, require_admin, require_login
 from l4d2web.db import session_scope
 from l4d2web.models import Blueprint as BlueprintModel
-from l4d2web.models import BlueprintOverlay, Job, Overlay, Server, User
+from l4d2web.models import (
+    BlueprintOverlay,
+    Job,
+    Overlay,
+    OverlayWorkshopItem,
+    Server,
+    User,
+    WorkshopItem,
+)
 
 
 bp = Blueprint("pages", __name__)
@@ -141,28 +149,60 @@ def server_jobs_page(server_id: int):
 @bp.get("/overlays")
 @require_login
 def overlays() -> str:
+    user = current_user()
+    assert user is not None
     with session_scope() as db:
-        overlays = db.scalars(select(Overlay).order_by(Overlay.name)).all()
+        query = select(Overlay).order_by(Overlay.name)
+        if not user.admin:
+            query = query.where(
+                (Overlay.type == "external") | (Overlay.user_id == user.id)
+            )
+        overlays = db.scalars(query).all()
     return render_template("overlays.html", overlays=overlays)
 
 
 @bp.get("/overlays/<int:overlay_id>")
 @require_login
 def overlay_detail(overlay_id: int):
+    user = current_user()
+    assert user is not None
     with session_scope() as db:
         overlay = db.scalar(select(Overlay).where(Overlay.id == overlay_id))
         if overlay is None:
             return Response(status=404)
+        # Visibility: externals are visible to all; workshop overlays are
+        # visible to the owner and admins.
+        if overlay.type == "workshop" and not user.admin and overlay.user_id != user.id:
+            return Response(status=403)
         using_blueprints = db.scalars(
             select(BlueprintModel)
             .join(BlueprintOverlay, BlueprintOverlay.blueprint_id == BlueprintModel.id)
             .where(BlueprintOverlay.overlay_id == overlay.id)
             .order_by(BlueprintModel.name)
         ).all()
+        workshop_items = []
+        if overlay.type == "workshop":
+            workshop_items = db.scalars(
+                select(WorkshopItem)
+                .join(
+                    OverlayWorkshopItem,
+                    OverlayWorkshopItem.workshop_item_id == WorkshopItem.id,
+                )
+                .where(OverlayWorkshopItem.overlay_id == overlay.id)
+                .order_by(WorkshopItem.created_at)
+            ).all()
+        latest_build_job = db.scalar(
+            select(Job)
+            .where(Job.operation == "build_overlay", Job.overlay_id == overlay.id)
+            .order_by(Job.created_at.desc())
+            .limit(1)
+        )
     return render_template(
         "overlay_detail.html",
         overlay=overlay,
         using_blueprints=using_blueprints,
+        workshop_items=workshop_items,
+        latest_build_job=latest_build_job,
     )
 
 

l4d2web/routes/workshop_routes.py

@@ -0,0 +1,173 @@
+"""Routes for the workshop overlay type (add/remove items, manual rebuild,
+admin global refresh)."""
+from __future__ import annotations
+
+from flask import Blueprint, Response, redirect, render_template, request
+from sqlalchemy import delete as sa_delete
+from sqlalchemy import select
+
+from l4d2web.auth import current_user, require_admin, require_login
+from l4d2web.db import session_scope
+from l4d2web.models import (
+    Job,
+    Overlay,
+    OverlayWorkshopItem,
+    WorkshopItem,
+)
+from l4d2web.services import steam_workshop
+from l4d2web.services.job_worker import enqueue_build_overlay
+
+
+bp = Blueprint("workshop", __name__)
+
+
+def _check_workshop_overlay_access(overlay_id: int, user, db):
+    overlay = db.scalar(select(Overlay).where(Overlay.id == overlay_id))
+    if overlay is None:
+        return None, Response(status=404)
+    if overlay.type != "workshop":
+        return None, Response("not a workshop overlay", status=400)
+    if overlay.user_id != user.id and not user.admin:
+        return None, Response(status=403)
+    return overlay, None
+
+
+def _render_item_table(overlay_id: int):
+    with session_scope() as db:
+        overlay = db.scalar(select(Overlay).where(Overlay.id == overlay_id))
+        items = db.scalars(
+            select(WorkshopItem)
+            .join(
+                OverlayWorkshopItem,
+                OverlayWorkshopItem.workshop_item_id == WorkshopItem.id,
+            )
+            .where(OverlayWorkshopItem.overlay_id == overlay_id)
+            .order_by(WorkshopItem.created_at)
+        ).all()
+        # Detach so attributes survive after the session closes.
+        for item in items:
+            db.expunge(item)
+        if overlay is not None:
+            db.expunge(overlay)
+    return render_template(
+        "_overlay_item_table.html",
+        overlay=overlay,
+        workshop_items=items,
+    )
+
+
+@bp.post("/overlays/<int:overlay_id>/items")
+@require_login
+def add_items(overlay_id: int) -> Response:
+    user = current_user()
+    assert user is not None
+
+    raw_input = request.form.get("input", "").strip()
+    mode = request.form.get("input_mode", "items")
+    if not raw_input:
+        return Response("missing input", status=400)
+
+    try:
+        ids = steam_workshop.parse_workshop_input(raw_input)
+    except ValueError as exc:
+        return Response(str(exc), status=400)
+
+    if mode == "collection":
+        if len(ids) != 1:
+            return Response("collection mode expects exactly one id or url", status=400)
+        try:
+            ids = steam_workshop.resolve_collection(ids[0])
+        except Exception as exc:
+            return Response(f"failed to resolve collection: {exc}", status=502)
+        if not ids:
+            return Response("collection has no items", status=400)
+
+    try:
+        metas = steam_workshop.fetch_metadata_batch(ids, mode="add")
+    except steam_workshop.WorkshopValidationError as exc:
+        return Response(str(exc), status=400)
+    except Exception as exc:
+        return Response(f"steam api error: {exc}", status=502)
+
+    with session_scope() as db:
+        overlay, err = _check_workshop_overlay_access(overlay_id, user, db)
+        if err is not None:
+            return err
+        for meta in metas:
+            wi = db.scalar(select(WorkshopItem).where(WorkshopItem.steam_id == meta.steam_id))
+            if wi is None:
+                wi = WorkshopItem(steam_id=meta.steam_id)
+                db.add(wi)
+            wi.title = meta.title
+            wi.filename = meta.filename
+            wi.file_url = meta.file_url
+            wi.file_size = meta.file_size
+            wi.time_updated = meta.time_updated
+            wi.preview_url = meta.preview_url
+            wi.last_error = "" if meta.result == 1 else f"steam result {meta.result}"
+            db.flush()
+
+            existing = db.scalar(
+                select(OverlayWorkshopItem).where(
+                    OverlayWorkshopItem.overlay_id == overlay_id,
+                    OverlayWorkshopItem.workshop_item_id == wi.id,
+                )
+            )
+            if existing is None:
+                db.add(OverlayWorkshopItem(overlay_id=overlay_id, workshop_item_id=wi.id))
+
+        enqueue_build_overlay(db, overlay_id=overlay_id, user_id=user.id)
+
+    return _render_item_table(overlay_id)
+
+
+@bp.post("/overlays/<int:overlay_id>/items/<int:item_id>/delete")
+@require_login
+def remove_item(overlay_id: int, item_id: int) -> Response:
+    user = current_user()
+    assert user is not None
+    with session_scope() as db:
+        overlay, err = _check_workshop_overlay_access(overlay_id, user, db)
+        if err is not None:
+            return err
+        result = db.execute(
+            sa_delete(OverlayWorkshopItem).where(
+                OverlayWorkshopItem.overlay_id == overlay_id,
+                OverlayWorkshopItem.workshop_item_id == item_id,
+            )
+        )
+        if result.rowcount == 0:
+            return Response(status=404)
+        enqueue_build_overlay(db, overlay_id=overlay_id, user_id=user.id)
+    return _render_item_table(overlay_id)
+
+
+@bp.post("/overlays/<int:overlay_id>/build")
+@require_login
+def manual_build(overlay_id: int) -> Response:
+    user = current_user()
+    assert user is not None
+    with session_scope() as db:
+        overlay, err = _check_workshop_overlay_access(overlay_id, user, db)
+        if err is not None:
+            return err
+        job = enqueue_build_overlay(db, overlay_id=overlay_id, user_id=user.id)
+        job_id = job.id
+    return redirect(f"/jobs/{job_id}")
+
+
+@bp.post("/admin/workshop/refresh")
+@require_admin
+def admin_refresh() -> Response:
+    user = current_user()
+    assert user is not None
+    with session_scope() as db:
+        db.add(
+            Job(
+                user_id=user.id,
+                server_id=None,
+                operation="refresh_workshop_items",
+                state="queued",
+            )
+        )
+    return redirect("/admin/jobs")

l4d2web/services/job_worker.py

@@ -10,13 +10,24 @@ from sqlalchemy.orm import Session
 from typing import Callable
 
 from l4d2web.db import session_scope
-from l4d2web.models import Job, JobLog, Server
+from l4d2web.models import (
+    Blueprint,
+    BlueprintOverlay,
+    Job,
+    JobLog,
+    Overlay,
+    OverlayWorkshopItem,
+    Server,
+    WorkshopItem,
+)
 from l4d2web.services.host_commands import CommandCancelledError
 
 
 TERMINAL_JOB_STATES = {"succeeded", "failed", "cancelled"}
 ACTIVE_JOB_STATES = {"running", "cancelling"}
 SERVER_OPERATIONS = {"initialize", "start", "stop", "delete"}
+OVERLAY_OPERATIONS = {"build_overlay"}
+GLOBAL_OPERATIONS = {"install", "refresh_workshop_items"}
 
 _claim_lock = threading.Lock()
 _log_lock = threading.RLock()
@@ -27,17 +38,55 @@ _workers_started = False
 @dataclass
 class SchedulerState:
     install_running: bool = False
+    refresh_running: bool = False
     running_servers: set[int] = field(default_factory=set)
+    running_overlays: set[int] = field(default_factory=set)
+    blocked_servers_by_overlay: set[int] = field(default_factory=set)
 
 
 def can_start(job, state: SchedulerState) -> bool:
+    """Truth table for the worker's claim policy.
+
+    install / refresh_workshop_items are global mutexes — they block each
+    other, all build_overlay jobs, and all server jobs.
+
+    build_overlay(overlay_id=N) is per-overlay: blocks on install/refresh, and
+    on another build for the same overlay. Different overlays may build
+    concurrently.
+
+    Server start/init blocks on install/refresh and on a build_overlay for any
+    overlay referenced by the server's blueprint.
+    """
     if job.operation == "install":
-        return (not state.install_running) and (len(state.running_servers) == 0)
-    if state.install_running:
+        return (
+            not state.install_running
+            and not state.refresh_running
+            and len(state.running_servers) == 0
+            and len(state.running_overlays) == 0
+        )
+    if job.operation == "refresh_workshop_items":
+        return (
+            not state.install_running
+            and not state.refresh_running
+            and len(state.running_servers) == 0
+            and len(state.running_overlays) == 0
+        )
+    if job.operation == "build_overlay":
+        if state.install_running or state.refresh_running:
+            return False
+        if job.overlay_id is None:
+            return False
+        return job.overlay_id not in state.running_overlays
+    # Server operations from here on.
+    if state.install_running or state.refresh_running:
         return False
     if job.server_id is None:
         return False
-    return job.server_id not in state.running_servers
+    if job.server_id in state.running_servers:
+        return False
+    if job.server_id in state.blocked_servers_by_overlay:
+        return False
+    return True
 
 
 def build_scheduler_state(session: Session) -> SchedulerState:
@@ -46,8 +95,22 @@ def build_scheduler_state(session: Session) -> SchedulerState:
     for job in running_jobs:
         if job.operation == "install":
             state.install_running = True
+        elif job.operation == "refresh_workshop_items":
+            state.refresh_running = True
+        elif job.operation == "build_overlay" and job.overlay_id is not None:
+            state.running_overlays.add(job.overlay_id)
         elif job.server_id is not None:
             state.running_servers.add(job.server_id)
+
+    if state.running_overlays:
+        rows = session.execute(
+            select(Server.id)
+            .join(Blueprint, Blueprint.id == Server.blueprint_id)
+            .join(BlueprintOverlay, BlueprintOverlay.blueprint_id == Blueprint.id)
+            .where(BlueprintOverlay.overlay_id.in_(state.running_overlays))
+        ).all()
+        state.blocked_servers_by_overlay = {row[0] for row in rows}
+
     return state
 
 
@@ -65,8 +128,22 @@ def claim_next_job() -> int | None:
             jobs = db.scalars(select(Job).where(Job.state == "queued").order_by(Job.created_at, Job.id)).all()
             now = datetime.now(UTC)
             for job in jobs:
-                malformed_server_job = job.operation != "install" and job.server_id is None
-                if not malformed_server_job and not can_start(job, state):
+                malformed_server_job = (
+                    job.operation in SERVER_OPERATIONS and job.server_id is None
+                )
+                malformed_overlay_job = (
+                    job.operation in OVERLAY_OPERATIONS and job.overlay_id is None
+                )
+                if malformed_server_job or malformed_overlay_job:
+                    # Mark malformed jobs failed immediately so the scheduler can move on.
+                    job.state = "failed"
+                    job.exit_code = 1
+                    job.started_at = now
+                    job.finished_at = now
+                    job.updated_at = now
+                    db.flush()
+                    continue
+                if not can_start(job, state):
                     continue
 
                 job.state = "running"
@@ -78,6 +155,31 @@ def claim_next_job() -> int | None:
     return None
 
 
+def enqueue_build_overlay(session: Session, *, overlay_id: int, user_id: int) -> Job:
+    """Insert a `build_overlay` job, coalescing against any already-queued
+    (not-yet-running) build for the same overlay. Running jobs are NOT
+    coalesced — a fresh add after a build started gets its own job."""
+    existing = session.scalar(
+        select(Job).where(
+            Job.operation == "build_overlay",
+            Job.overlay_id == overlay_id,
+            Job.state == "queued",
+        )
+    )
+    if existing is not None:
+        return existing
+    job = Job(
+        user_id=user_id,
+        server_id=None,
+        overlay_id=overlay_id,
+        operation="build_overlay",
+        state="queued",
+    )
+    session.add(job)
+    session.flush()
+    return job
+
+
 def run_worker_once() -> bool:
     job_id = claim_next_job()
     if job_id is None:
@@ -90,12 +192,14 @@ def run_job(job_id: int) -> None:
     from l4d2web.services import l4d2_facade
 
     server_name = "unknown"
+    overlay_id_for_job: int | None = None
     with session_scope() as db:
         job = db.scalar(select(Job).where(Job.id == job_id))
         if job is None:
             return
         operation = job.operation
         server_id = job.server_id
+        overlay_id_for_job = job.overlay_id
         if server_id is not None:
             server = db.scalar(select(Server).where(Server.id == server_id))
             if server is not None:
@@ -133,6 +237,27 @@ def run_job(job_id: int) -> None:
                 on_stderr=on_stderr,
                 should_cancel=should_cancel,
             )
+        elif operation == "refresh_workshop_items":
+            _run_with_boundaries(
+                "refresh",
+                "workshop items",
+                _run_refresh_workshop_items,
+                on_stdout=on_stdout,
+                on_stderr=on_stderr,
+                should_cancel=should_cancel,
+            )
+        elif operation == "build_overlay":
+            if overlay_id_for_job is None:
+                raise ValueError("build_overlay job has no overlay_id")
+            _run_with_boundaries(
+                "build",
+                f"overlay {overlay_id_for_job}",
+                _run_build_overlay,
+                overlay_id_for_job,
+                on_stdout=on_stdout,
+                on_stderr=on_stderr,
+                should_cancel=should_cancel,
+            )
         elif operation in SERVER_OPERATIONS and server_id is None:
             raise ValueError(f"{operation} job has no server_id")
         elif operation == "initialize":
@@ -213,6 +338,153 @@ def run_job(job_id: int) -> None:
         finish_job(job_id, "failed", 1, error=error)
 
 
+def _run_build_overlay(
+    overlay_id: int,
+    *,
+    on_stdout: Callable[[str], None],
+    on_stderr: Callable[[str], None],
+    should_cancel: Callable[[], bool],
+) -> None:
+    """Dispatch a build_overlay job through the builder registry."""
+    from l4d2web.services.overlay_builders import BUILDERS
+
+    with session_scope() as db:
+        overlay = db.scalar(select(Overlay).where(Overlay.id == overlay_id))
+        if overlay is None:
+            raise ValueError(f"overlay {overlay_id} not found")
+        builder = BUILDERS.get(overlay.type)
+        if builder is None:
+            raise ValueError(f"no builder registered for overlay type {overlay.type!r}")
+        # Detach overlay before leaving the session so the builder can read its
+        # attributes without a stale connection.
+        db.expunge(overlay)
+
+    builder.build(
+        overlay,
+        on_stdout=on_stdout,
+        on_stderr=on_stderr,
+        should_cancel=should_cancel,
+    )
+
+
+def _run_refresh_workshop_items(
+    *,
+    on_stdout: Callable[[str], None],
+    on_stderr: Callable[[str], None],
+    should_cancel: Callable[[], bool],
+) -> list[int]:
+    """Refresh metadata for every WorkshopItem, redownload changed items, and
+    enqueue (coalesced) build_overlay jobs for any overlay whose items had
+    `time_updated` advance or `filename` change. Returns the affected
+    overlay_ids for testability."""
+    from l4d2web.services import steam_workshop
+    from l4d2web.services.workshop_paths import workshop_cache_root
+
+    # Snapshot all WorkshopItems for the metadata batch.
+    with session_scope() as db:
+        items = db.scalars(select(WorkshopItem)).all()
+        snapshot = [
+            (it.id, it.steam_id, it.time_updated, it.filename) for it in items
+        ]
+
+    if not snapshot:
+        on_stdout("no workshop items registered; nothing to refresh")
+        return []
+
+    steam_ids = [s for _, s, _, _ in snapshot]
+    on_stdout(f"fetching metadata for {len(steam_ids)} items")
+    metas = steam_workshop.fetch_metadata_batch(steam_ids, mode="refresh")
+    metas_by_id = {m.steam_id: m for m in metas}
+    on_stdout(f"metadata phase complete (received {len(metas)} entries)")
+
+    if should_cancel():
+        on_stderr("refresh cancelled after metadata phase")
+        return []
+
+    # Update DB rows + collect items that need (re)download.
+    affected_workshop_item_ids: set[int] = set()
+    download_metas: list[steam_workshop.WorkshopMetadata] = []
+    with session_scope() as db:
+        for item_id, steam_id, prior_time_updated, prior_filename in snapshot:
+            meta = metas_by_id.get(steam_id)
+            wi = db.scalar(select(WorkshopItem).where(WorkshopItem.id == item_id))
+            if wi is None:
+                continue
+            if meta is None:
+                # Steam dropped this item from the response.
+                wi.last_error = "steam returned no entry for this item"
+                continue
+            wi.title = meta.title
+            wi.filename = meta.filename
+            wi.file_url = meta.file_url
+            wi.file_size = meta.file_size
+            wi.preview_url = meta.preview_url
+            wi.last_error = "" if meta.result == 1 else f"steam result {meta.result}"
+            if meta.result != 1 or not meta.file_url:
+                continue
+            if (
+                meta.time_updated > prior_time_updated
+                or meta.filename != prior_filename
+                or wi.last_downloaded_at is None
+            ):
+                wi.time_updated = meta.time_updated
+                affected_workshop_item_ids.add(item_id)
+                download_metas.append(meta)
+
+    on_stdout(f"downloading {len(download_metas)} items")
+    if download_metas:
+        report = steam_workshop.refresh_all(
+            download_metas, workshop_cache_root(), should_cancel=should_cancel
+        )
+        on_stdout(
+            f"download phase complete (downloaded={report.downloaded} errors={report.errors})"
+        )
+        if report.errors:
+            for steam_id, err in report.per_item_errors.items():
+                on_stderr(f"download {steam_id}: {err}")
+        # Mark successfully downloaded items.
+        with session_scope() as db:
+            for meta in download_metas:
+                if meta.steam_id in report.per_item_errors:
+                    continue
+                wi = db.scalar(select(WorkshopItem).where(WorkshopItem.steam_id == meta.steam_id))
+                if wi is not None:
+                    wi.last_downloaded_at = datetime.now(UTC)
+
+    # Enqueue (coalesced) build_overlay for affected overlays.
+    if not affected_workshop_item_ids:
+        on_stdout("no overlays needed rebuilding")
+        return []
+
+    with session_scope() as db:
+        overlay_rows = db.execute(
+            select(OverlayWorkshopItem.overlay_id)
+            .where(OverlayWorkshopItem.workshop_item_id.in_(affected_workshop_item_ids))
+            .distinct()
+        ).all()
+        affected_overlay_ids = [row[0] for row in overlay_rows]
+        for ov_id in affected_overlay_ids:
+            # Find a sensible owner for the auto-enqueued job: the overlay's
+            # user_id if private, else any admin (best effort) — fall back to
+            # the most recent existing job's user_id.
+            overlay = db.scalar(select(Overlay).where(Overlay.id == ov_id))
+            if overlay is None:
+                continue
+            user_id = overlay.user_id
+            if user_id is None:
+                # System overlay — pick any admin user; fall back to first user.
+                user_id = db.scalar(
+                    select(Job.user_id).order_by(Job.created_at.desc()).limit(1)
+                )
+                if user_id is None:
+                    continue
+            enqueue_build_overlay(db, overlay_id=ov_id, user_id=user_id)
+        on_stdout(
+            f"enqueued build_overlay for {len(affected_overlay_ids)} overlay(s)"
+        )
+        return list(affected_overlay_ids)
+
+
 def finish_job(job_id: int, state: str, exit_code: int | None, error: str = "") -> None:
     now = datetime.now(UTC)
     with session_scope() as db:

l4d2web/services/l4d2_facade.py

@@ -5,9 +5,17 @@ from pathlib import Path
 from sqlalchemy import select
 
 from l4d2web.db import session_scope
-from l4d2web.models import Blueprint, BlueprintOverlay, Overlay, Server
+from l4d2web.models import (
+    Blueprint,
+    BlueprintOverlay,
+    Overlay,
+    OverlayWorkshopItem,
+    Server,
+    WorkshopItem,
+)
 from l4d2web.services import host_commands
 from l4d2web.services.spec_yaml import write_temp_spec
+from l4d2web.services.workshop_paths import cache_path
 
 
 @dataclass(slots=True)
@@ -57,6 +65,21 @@ def install_runtime(on_stdout=None, on_stderr=None, should_cancel=None) -> None:
 
 def initialize_server(server_id: int, on_stdout=None, on_stderr=None, should_cancel=None) -> None:
     server, blueprint, overlay_refs = load_server_blueprint_bundle(server_id)
+
+    # Run each overlay's builder synchronously so symlinks/dirs are present
+    # before l4d2ctl initialize composes the lowerdirs.
+    _run_blueprint_builders(
+        blueprint_id=blueprint.id,
+        on_stdout=on_stdout,
+        on_stderr=on_stderr,
+        should_cancel=should_cancel,
+    )
+
+    # Workshop overlays may have items not yet downloaded. The builders skip
+    # them, but we don't want to mount a partial overlay silently — fail
+    # loudly with the missing IDs.
+    _check_workshop_overlay_caches(blueprint_id=blueprint.id)
+
     spec_path = write_temp_spec(build_server_spec_payload(server, blueprint, overlay_refs))
     try:
         host_commands.run_command(
@@ -69,6 +92,87 @@ def initialize_server(server_id: int, on_stdout=None, on_stderr=None, should_can
         spec_path.unlink(missing_ok=True)
 
 
+def _run_blueprint_builders(
+    *,
+    blueprint_id: int,
+    on_stdout=None,
+    on_stderr=None,
+    should_cancel=None,
+) -> None:
+    """Synchronously invoke each overlay's builder for the given blueprint."""
+    from l4d2web.services.overlay_builders import BUILDERS
+
+    with session_scope() as db:
+        overlays = db.scalars(
+            select(Overlay)
+            .join(BlueprintOverlay, BlueprintOverlay.overlay_id == Overlay.id)
+            .where(BlueprintOverlay.blueprint_id == blueprint_id)
+            .order_by(BlueprintOverlay.position)
+        ).all()
+        for overlay in overlays:
+            db.expunge(overlay)
+
+    log_stdout = on_stdout if on_stdout is not None else (lambda _line: None)
+    log_stderr = on_stderr if on_stderr is not None else (lambda _line: None)
+    cancel = should_cancel if should_cancel is not None else (lambda: False)
+
+    for overlay in overlays:
+        builder = BUILDERS.get(overlay.type)
+        if builder is None:
+            raise ValueError(f"no builder registered for overlay type {overlay.type!r}")
+        builder.build(
+            overlay,
+            on_stdout=log_stdout,
+            on_stderr=log_stderr,
+            should_cancel=cancel,
+        )
+
+
+def _check_workshop_overlay_caches(*, blueprint_id: int) -> None:
+    """Raise if any workshop overlay attached to this blueprint has items
+    that aren't yet in the workshop_cache. Mounting a partial overlay would
+    leave maps mysteriously missing in-game; surface the issue here instead.
+    """
+    with session_scope() as db:
+        rows = db.execute(
+            select(Overlay.id, Overlay.name, WorkshopItem.steam_id)
+            .join(BlueprintOverlay, BlueprintOverlay.overlay_id == Overlay.id)
+            .join(
+                OverlayWorkshopItem,
+                OverlayWorkshopItem.overlay_id == Overlay.id,
+            )
+            .join(
+                WorkshopItem,
+                WorkshopItem.id == OverlayWorkshopItem.workshop_item_id,
+            )
+            .where(
+                BlueprintOverlay.blueprint_id == blueprint_id,
+                Overlay.type == "workshop",
+            )
+        ).all()
+
+    missing: dict[tuple[int, str], list[str]] = {}
+    for overlay_id, overlay_name, steam_id in rows:
+        if not cache_path(steam_id).exists():
+            missing.setdefault((overlay_id, overlay_name), []).append(steam_id)
+
+    if not missing:
+        return
+
+    parts = []
+    for (overlay_id, overlay_name), steam_ids in missing.items():
+        ids = ", ".join(steam_ids)
+        parts.append(
+            f"overlay {overlay_name!r} (id={overlay_id}): items {ids} not yet downloaded"
+        )
+    detail = "; ".join(parts)
+    raise RuntimeError(
+        f"workshop content missing — {detail}. "
+        f"Open the overlay page and click Build (or wait for the auto-rebuild job), "
+        f"then retry."
+    )
+
+
 def start_server(server_id: int, on_stdout=None, on_stderr=None, should_cancel=None) -> None:
     server, _, _ = load_server_blueprint_bundle(server_id)
     host_commands.run_command(

l4d2web/services/overlay_builders.py

@@ -0,0 +1,193 @@
+"""Overlay builder registry.
+
+Each `Overlay.type` maps to a builder. `build_overlay(overlay_id)` jobs (and
+the synchronous `initialize_server` hook) dispatch through `BUILDERS`. Adding
+a new overlay type means writing a new builder and registering it here — no
+changes to the worker, the mount layer, or the blueprint editor.
+"""
+from __future__ import annotations
+
+import os
+from pathlib import Path
+from typing import Callable, Protocol
+
+from sqlalchemy import select
+
+from l4d2host.paths import get_left4me_root
+
+from l4d2web.db import session_scope
+from l4d2web.models import Overlay, OverlayWorkshopItem, WorkshopItem
+from l4d2web.services.workshop_paths import cache_path, workshop_cache_root
+
+
+CancelCheck = Callable[[], bool]
+LogSink = Callable[[str], None]
+
+
+class OverlayBuilder(Protocol):
+    def build(
+        self,
+        overlay: Overlay,
+        *,
+        on_stdout: LogSink,
+        on_stderr: LogSink,
+        should_cancel: CancelCheck,
+    ) -> None: ...
+
+
+def _overlay_root(overlay: Overlay) -> Path:
+    return get_left4me_root() / "overlays" / overlay.path
+
+
+class ExternalBuilder:
+    """No-op builder for admin-managed overlays. Ensures the overlay directory
+    exists; everything inside it is the admin's responsibility (SFTP, etc.)."""
+
+    def build(
+        self,
+        overlay: Overlay,
+        *,
+        on_stdout: LogSink,
+        on_stderr: LogSink,
+        should_cancel: CancelCheck,
+    ) -> None:
+        root = _overlay_root(overlay)
+        root.mkdir(parents=True, exist_ok=True)
+        on_stdout(f"external overlay {overlay.name!r} ready at {root}")
+
+
+class WorkshopBuilder:
+    """Diff-apply symlinks under `left4dead2/addons/` against the overlay's
+    current `WorkshopItem` associations. Cached items get an absolute symlink
+    into `workshop_cache/{steam_id}.vpk`. Items missing from cache are
+    skipped with a warning rather than turned into broken symlinks."""
+
+    def build(
+        self,
+        overlay: Overlay,
+        *,
+        on_stdout: LogSink,
+        on_stderr: LogSink,
+        should_cancel: CancelCheck,
+    ) -> None:
+        addons_dir = _overlay_root(overlay) / "left4dead2" / "addons"
+        addons_dir.mkdir(parents=True, exist_ok=True)
+
+        with session_scope() as db:
+            items = db.scalars(
+                select(WorkshopItem)
+                .join(
+                    OverlayWorkshopItem,
+                    OverlayWorkshopItem.workshop_item_id == WorkshopItem.id,
+                )
+                .where(OverlayWorkshopItem.overlay_id == overlay.id)
+            ).all()
+            # Detach items so we can use them outside the session.
+            items_data = [
+                (it.steam_id, it.last_downloaded_at) for it in items
+            ]
+
+        cache_root = workshop_cache_root()
+        # desired: symlink-name -> absolute target path (only for cached items)
+        desired: dict[str, Path] = {}
+        skipped: list[str] = []
+        for steam_id, last_downloaded_at in items_data:
+            target = cache_path(steam_id)
+            if last_downloaded_at is None or not target.exists():
+                skipped.append(steam_id)
+                continue
+            desired[f"{steam_id}.vpk"] = target.resolve()
+
+        if should_cancel():
+            on_stderr("workshop build cancelled before applying symlinks")
+            return
+
+        # existing: symlink-name -> link target (only for symlinks pointing at our cache)
+        existing: dict[str, Path] = {}
+        for entry in os.scandir(addons_dir):
+            if not entry.is_symlink():
+                continue
+            try:
+                target = Path(os.readlink(entry.path))
+            except OSError:
+                continue
+            try:
+                resolved = target.resolve(strict=False)
+            except OSError:
+                continue
+            if not _is_under(resolved, cache_root):
+                continue
+            existing[entry.name] = resolved
+
+        created = 0
+        removed = 0
+        unchanged = 0
+
+        # Remove obsolete or stale symlinks first.
+        for name, current_target in existing.items():
+            if should_cancel():
+                on_stderr("workshop build cancelled mid-removal")
+                return
+            desired_target = desired.get(name)
+            if desired_target is None:
+                os.unlink(addons_dir / name)
+                removed += 1
+            elif current_target != desired_target:
+                os.unlink(addons_dir / name)
+                # will be recreated below
+            else:
+                unchanged += 1
+
+        # Recompute existing post-removal so the create loop knows what's left.
+        post_removal_existing = {
+            name for name in existing if name in desired and existing[name] == desired[name]
+        }
+
+        # Create new symlinks.
+        for name, target in desired.items():
+            if should_cancel():
+                on_stderr("workshop build cancelled mid-creation")
+                return
+            if name in post_removal_existing:
+                continue
+            link_path = addons_dir / name
+            # Defensive: if a non-symlink file collides with our name, leave it.
+            if link_path.exists() and not link_path.is_symlink():
+                on_stderr(
+                    f"refusing to overwrite non-symlink at {link_path}; manual intervention required"
+                )
+                continue
+            if link_path.is_symlink():
+                # An obsolete symlink not in `existing` (target outside cache).
+                # We don't manage these — leave alone.
+                on_stderr(
+                    f"refusing to overwrite foreign symlink at {link_path}"
+                )
+                continue
+            os.symlink(str(target), str(link_path))
+            created += 1
+
+        on_stdout(
+            f"workshop overlay {overlay.name!r}: created={created} "
+            f"removed={removed} unchanged={unchanged} "
+            f"skipped(uncached)={len(skipped)}"
+        )
+        for steam_id in skipped:
+            on_stderr(
+                f"workshop item {steam_id} skipped: not yet downloaded "
+                f"(refresh required before this overlay can mount it)"
+            )
+
+
+def _is_under(path: Path, root: Path) -> bool:
+    try:
+        path.relative_to(root)
+    except ValueError:
+        return False
+    return True
+
+
+BUILDERS: dict[str, OverlayBuilder] = {
+    "external": ExternalBuilder(),
+    "workshop": WorkshopBuilder(),
+}

l4d2web/services/overlay_creation.py

@@ -0,0 +1,35 @@
+"""Overlay path generation and on-disk directory bootstrap.
+
+All new overlays (any type) get `path = str(overlay_id)`. The directory is
+created with `exist_ok=False` so a stray folder from a prior failed delete
+surfaces loudly instead of silently shadowing fresh content. Combined with
+SQLite AUTOINCREMENT on `overlays.id`, that catches DB/disk drift.
+"""
+from __future__ import annotations
+
+import os
+
+from l4d2host.paths import get_left4me_root, validate_overlay_ref
+
+from l4d2web.models import Overlay
+
+
+def generate_overlay_path(overlay_id: int) -> str:
+    """Return the canonical relative path for an overlay row.
+
+    Validates the result through l4d2host's overlay-ref guard. Pure numeric IDs
+    always pass — this is just a belt-and-suspenders check that surfaces
+    immediately if someone changes the scheme.
+    """
+    candidate = str(overlay_id)
+    return validate_overlay_ref(candidate)
+
+
+def create_overlay_directory(overlay: Overlay) -> None:
+    """Create `LEFT4ME_ROOT/overlays/{overlay.path}/` with `exist_ok=False`.
+
+    Raises `FileExistsError` if the directory already exists, surfacing the
+    rare DB/disk-drift state where a stray directory matches a fresh ID.
+    """
+    target = get_left4me_root() / "overlays" / overlay.path
+    os.makedirs(target, exist_ok=False)

l4d2web/services/steam_workshop.py

@@ -0,0 +1,295 @@
+"""Steam Workshop API client + downloader.
+
+Pure HTTP/file logic — no DB writes, no Flask, no job-worker integration.
+Used by the workshop overlay builder and the admin refresh job.
+
+Endpoints:
+- GetCollectionDetails: resolve a collection ID to its child item IDs.
+- GetPublishedFileDetails: batch-fetch metadata for items, including a public
+  file_url for the .vpk.
+
+Both endpoints accept anonymous POSTs; no Steam Web API key required.
+"""
+from __future__ import annotations
+
+import os
+import re
+import threading
+from concurrent.futures import ThreadPoolExecutor, as_completed
+from dataclasses import dataclass, field
+from pathlib import Path
+from typing import Callable, Iterable, Literal
+
+import requests
+
+
+# HTTPS only (decision 16). The reference downloader uses HTTP — we don't.
+GET_PUBLISHED_FILE_DETAILS_URL = (
+    "https://api.steampowered.com/ISteamRemoteStorage/GetPublishedFileDetails/v1/"
+)
+GET_COLLECTION_DETAILS_URL = (
+    "https://api.steampowered.com/ISteamRemoteStorage/GetCollectionDetails/v1/"
+)
+
+L4D2_APP_ID = 550
+
+REQUEST_TIMEOUT_SECONDS = 30
+DOWNLOAD_CHUNK_BYTES = 1_048_576
+
+_NUMERIC_ID_RE = re.compile(r"^\d+$")
+_URL_ID_RE = re.compile(r"^https?://([a-z0-9.-]*\.)?steamcommunity\.com/.*[?&]id=(\d+)", re.IGNORECASE)
+_BARE_URL_ID_RE = re.compile(r"^([a-z0-9.-]*\.)?steamcommunity\.com/.*[?&]id=(\d+)", re.IGNORECASE)
+
+_session_local = threading.local()
+
+
+def _session() -> requests.Session:
+    """Per-thread session for connection reuse without cross-thread leakage."""
+    sess = getattr(_session_local, "session", None)
+    if sess is None:
+        sess = requests.Session()
+        _session_local.session = sess
+    return sess
+
+
+class WorkshopValidationError(ValueError):
+    """Raised during user-add when an item fails a fixed precondition
+    (e.g. consumer_app_id != 550)."""
+
+
+@dataclass(slots=True)
+class WorkshopMetadata:
+    steam_id: str
+    title: str
+    filename: str
+    file_url: str
+    file_size: int
+    time_updated: int
+    preview_url: str
+    consumer_app_id: int
+    result: int
+
+
+@dataclass(slots=True)
+class RefreshReport:
+    downloaded: int = 0
+    skipped: int = 0
+    errors: int = 0
+    per_item_errors: dict[str, str] = field(default_factory=dict)
+
+
+def parse_workshop_input(raw: str) -> list[str]:
+    """Parse a single ID, a single workshop URL, or a multi-line / whitespace-
+    separated batch of either. Returns deduplicated digit-only IDs in order.
+    Raises ValueError on garbage."""
+    if not raw or not raw.strip():
+        raise ValueError("input is empty")
+
+    tokens: list[str] = []
+    for token in re.split(r"\s+", raw.strip()):
+        if not token:
+            continue
+        tokens.append(_extract_id(token))
+
+    seen: set[str] = set()
+    deduped: list[str] = []
+    for tok in tokens:
+        if tok not in seen:
+            seen.add(tok)
+            deduped.append(tok)
+    return deduped
+
+
+def _extract_id(token: str) -> str:
+    if _NUMERIC_ID_RE.fullmatch(token):
+        return token
+    m = _URL_ID_RE.match(token)
+    if m:
+        return m.group(2)
+    m = _BARE_URL_ID_RE.match(token)
+    if m:
+        return m.group(2)
+    raise ValueError(f"could not parse a Steam workshop id from: {token!r}")
+
+
+def resolve_collection(collection_id: str) -> list[str]:
+    """POST GetCollectionDetails for one collection; return its non-collection
+    child publishedfileids in order. Nested collections (filetype != 0) are
+    skipped."""
+    if not _NUMERIC_ID_RE.fullmatch(collection_id):
+        raise ValueError("collection_id must be digits only")
+
+    response = _session().post(
+        GET_COLLECTION_DETAILS_URL,
+        data={
+            "collectioncount": 1,
+            "publishedfileids[0]": collection_id,
+        },
+        timeout=REQUEST_TIMEOUT_SECONDS,
+    )
+    response.raise_for_status()
+    payload = response.json()
+    children: list[str] = []
+    for collection in payload.get("response", {}).get("collectiondetails", []):
+        for child in collection.get("children", []):
+            if child.get("filetype", 0) != 0:
+                continue  # nested collection, skip
+            child_id = child.get("publishedfileid")
+            if child_id is not None:
+                children.append(str(child_id))
+    return children
+
+
+def fetch_metadata_batch(
+    steam_ids: list[str], *, mode: Literal["add", "refresh"]
+) -> list[WorkshopMetadata]:
+    """One POST to GetPublishedFileDetails covering all ids.
+
+    In `mode="add"`, any non-L4D2 (`consumer_app_id != 550`) raises
+    WorkshopValidationError so the user-add request fails cleanly.
+
+    In `mode="refresh"`, non-L4D2 entries are skipped from the result.
+
+    Items with `result != 1` are returned as-is (the caller persists the result
+    code into `WorkshopItem.last_error`).
+    """
+    if not steam_ids:
+        return []
+    for sid in steam_ids:
+        if not _NUMERIC_ID_RE.fullmatch(sid):
+            raise ValueError(f"steam id must be digits only: {sid!r}")
+
+    payload: dict[str, str | int] = {"itemcount": len(steam_ids)}
+    for index, sid in enumerate(steam_ids):
+        payload[f"publishedfileids[{index}]"] = sid
+
+    response = _session().post(
+        GET_PUBLISHED_FILE_DETAILS_URL,
+        data=payload,
+        timeout=REQUEST_TIMEOUT_SECONDS,
+    )
+    response.raise_for_status()
+    body = response.json()
+
+    metas: list[WorkshopMetadata] = []
+    for entry in body.get("response", {}).get("publishedfiledetails", []):
+        meta = WorkshopMetadata(
+            steam_id=str(entry.get("publishedfileid", "")),
+            title=str(entry.get("title", "") or ""),
+            filename=str(entry.get("filename", "") or ""),
+            file_url=str(entry.get("file_url", "") or ""),
+            file_size=int(entry.get("file_size") or 0),
+            time_updated=int(entry.get("time_updated") or 0),
+            preview_url=str(entry.get("preview_url", "") or ""),
+            consumer_app_id=int(entry.get("consumer_app_id") or 0),
+            result=int(entry.get("result") or 0),
+        )
+
+        # consumer_app_id is only meaningful when the lookup itself succeeded.
+        if meta.result == 1 and meta.consumer_app_id != L4D2_APP_ID:
+            if mode == "add":
+                raise WorkshopValidationError(
+                    f"item {meta.steam_id} is not a Left 4 Dead 2 workshop "
+                    f"item (consumer_app_id={meta.consumer_app_id})"
+                )
+            # refresh mode: drop the entry silently from the batch
+            continue
+
+        metas.append(meta)
+    return metas
+
+
+def download_to_cache(
+    meta: WorkshopMetadata,
+    cache_root: Path,
+    *,
+    on_progress: Callable[[int, int], None] | None = None,
+    should_cancel: Callable[[], bool] | None = None,
+) -> Path:
+    """Download `meta.file_url` to `cache_root/{steam_id}.vpk`.
+
+    Atomic via `*.partial` + `os.replace`. Idempotent: a no-op when the
+    existing file's `(mtime, size)` already matches `(time_updated, file_size)`.
+    Sets `os.utime(target, (time_updated, time_updated))` so the next run
+    short-circuits.
+    """
+    if not _NUMERIC_ID_RE.fullmatch(meta.steam_id):
+        raise ValueError("meta.steam_id must be digits only")
+    cache_root.mkdir(parents=True, exist_ok=True)
+    target = cache_root / f"{meta.steam_id}.vpk"
+
+    if (
+        target.exists()
+        and int(target.stat().st_mtime) == int(meta.time_updated)
+        and int(target.stat().st_size) == int(meta.file_size)
+    ):
+        return target
+
+    if not meta.file_url:
+        raise ValueError(f"item {meta.steam_id} has no file_url; cannot download")
+
+    partial = target.with_suffix(target.suffix + ".partial")
+    response = _session().get(meta.file_url, stream=True, timeout=REQUEST_TIMEOUT_SECONDS)
+    response.raise_for_status()
+
+    written = 0
+    try:
+        with open(partial, "wb") as f:
+            for chunk in response.iter_content(chunk_size=DOWNLOAD_CHUNK_BYTES):
+                if should_cancel is not None and should_cancel():
+                    raise InterruptedError("download cancelled")
+                if not chunk:
+                    continue
+                f.write(chunk)
+                written += len(chunk)
+                if on_progress is not None:
+                    on_progress(written, int(meta.file_size))
+        os.replace(partial, target)
+    except BaseException:
+        partial.unlink(missing_ok=True)
+        raise
+
+    os.utime(target, (meta.time_updated, meta.time_updated))
+    return target
+
+
+def refresh_all(
+    metas: Iterable[WorkshopMetadata],
+    cache_root: Path,
+    *,
+    executor_workers: int = 8,
+    should_cancel: Callable[[], bool] | None = None,
+) -> RefreshReport:
+    """Download (or skip-as-cached) every metadata item using a thread pool.
+    Per-item errors are collected; sibling items continue."""
+    metas_list = list(metas)
+    report = RefreshReport()
+    if not metas_list:
+        return report
+
+    cache_root.mkdir(parents=True, exist_ok=True)
+
+    with ThreadPoolExecutor(max_workers=executor_workers) as executor:
+        futures = {}
+        for meta in metas_list:
+            if should_cancel is not None and should_cancel():
+                break
+            future = executor.submit(
+                download_to_cache,
+                meta,
+                cache_root,
+                should_cancel=should_cancel,
+            )
+            futures[future] = meta
+
+        for future in as_completed(futures):
+            meta = futures[future]
+            try:
+                future.result()
+            except Exception as exc:
+                report.errors += 1
+                report.per_item_errors[meta.steam_id] = str(exc)
+                continue
+            report.downloaded += 1
+
+    return report

l4d2web/services/workshop_paths.py

@@ -0,0 +1,24 @@
+"""Cache-path helpers for workshop content.
+
+The cache lives at `$LEFT4ME_ROOT/workshop_cache/{steam_id}.vpk`. Steam IDs
+are validated digit-only here so callers don't need to guard separately.
+"""
+from __future__ import annotations
+
+import re
+from pathlib import Path
+
+from l4d2host.paths import get_left4me_root
+
+
+_NUMERIC_ID_RE = re.compile(r"^\d+$")
+
+
+def workshop_cache_root() -> Path:
+    return get_left4me_root() / "workshop_cache"
+
+
+def cache_path(steam_id: str) -> Path:
+    if not isinstance(steam_id, str) or not _NUMERIC_ID_RE.fullmatch(steam_id):
+        raise ValueError(f"steam_id must be digits only: {steam_id!r}")
+    return workshop_cache_root() / f"{steam_id}.vpk"

l4d2web/templates/_overlay_item_table.html

@@ -0,0 +1,50 @@
+{% set can_edit = g.user.admin or (overlay and overlay.type == 'workshop' and overlay.user_id == g.user.id) %}
+<table class="table">
+  <thead>
+    <tr>
+      <th></th>
+      <th>Steam ID</th>
+      <th>Title</th>
+      <th>Filename</th>
+      <th>Size</th>
+      <th>Updated</th>
+      <th>Status</th>
+      {% if can_edit %}<th></th>{% endif %}
+    </tr>
+  </thead>
+  <tbody>
+    {% for item in workshop_items %}
+    <tr>
+      <td>
+        {% if item.preview_url %}
+        <img src="{{ item.preview_url }}" alt="" width="48" height="48" loading="lazy">
+        {% endif %}
+      </td>
+      <td><a href="https://steamcommunity.com/sharedfiles/filedetails/?id={{ item.steam_id }}" target="_blank" rel="noopener">{{ item.steam_id }}</a></td>
+      <td>{{ item.title }}</td>
+      <td class="muted">{{ item.filename }}</td>
+      <td class="muted">{{ item.file_size }}</td>
+      <td class="muted">{{ item.time_updated }}</td>
+      <td>
+        {% if item.last_error %}
+        <span class="warning">{{ item.last_error }}</span>
+        {% elif item.last_downloaded_at %}
+        cached
+        {% else %}
+        <span class="muted">pending</span>
+        {% endif %}
+      </td>
+      {% if can_edit %}
+      <td>
+        <form method="post" action="/overlays/{{ overlay.id }}/items/{{ item.id }}/delete" class="inline-form">
+          <input type="hidden" name="csrf_token" value="{{ session.get('csrf_token', '') }}">
+          <button type="submit" class="button-secondary">Remove</button>
+        </form>
+      </td>
+      {% endif %}
+    </tr>
+    {% else %}
+    <tr><td colspan="{% if can_edit %}8{% else %}7{% endif %}" class="muted">No workshop items yet.</td></tr>
+    {% endfor %}
+  </tbody>
+</table>

l4d2web/templates/admin.html

@@ -19,4 +19,13 @@
     <button type="submit">Install or update runtime</button>
   </form>
 </section>
+
+<section class="panel">
+  <h2>Workshop</h2>
+  <p class="muted">Re-fetch metadata for every workshop item; re-download those with newer versions on Steam, then enqueue rebuilds for the affected overlays.</p>
+  <form method="post" action="/admin/workshop/refresh">
+    <input type="hidden" name="csrf_token" value="{{ session.get('csrf_token', '') }}">
+    <button type="submit">Refresh all workshop items</button>
+  </form>
+</section>
 {% endblock %}

l4d2web/templates/overlay_detail.html

@@ -6,30 +6,74 @@
 <section class="panel">
   <div class="page-heading">
     <h1>Overlay: {{ overlay.name }}</h1>
-    {% if g.user.admin %}
+    {% set can_edit = g.user.admin or (overlay.type == 'workshop' and overlay.user_id == g.user.id) %}
+    {% if can_edit %}
     <button type="button" class="danger" data-modal-open="delete-overlay-modal">Delete</button>
     {% endif %}
   </div>
 
-  {% if g.user.admin %}
+  {% if can_edit %}
   <form method="post" action="/overlays/{{ overlay.id }}" class="stack">
     <input type="hidden" name="csrf_token" value="{{ session.get('csrf_token', '') }}">
     <label>Name <input name="name" value="{{ overlay.name }}" required></label>
-    <label>Path <input name="path" value="{{ overlay.path }}" required></label>
     <div>
       <button type="submit">Save</button>
     </div>
   </form>
-  {% else %}
+  {% endif %}
+
   <table class="definition-table">
     <tbody>
-      <tr><th>Name</th><td>{{ overlay.name }}</td></tr>
-      <tr><th>Path</th><td>{{ overlay.path }}</td></tr>
+      <tr><th>Type</th><td>{{ overlay.type }}</td></tr>
+      <tr><th>Scope</th><td>{% if overlay.user_id %}private{% else %}system{% endif %}</td></tr>
+      <tr><th>Path</th><td class="muted">{{ overlay.path }}</td></tr>
     </tbody>
   </table>
-  {% endif %}
 </section>
 
+{% if overlay.type == 'workshop' %}
+<section class="panel">
+  <div class="page-heading">
+    <h2>Workshop items</h2>
+    {% if can_edit %}
+    <form method="post" action="/overlays/{{ overlay.id }}/build" class="inline-form">
+      <input type="hidden" name="csrf_token" value="{{ session.get('csrf_token', '') }}">
+      <button type="submit" class="button-secondary">Rebuild</button>
+    </form>
+    {% endif %}
+  </div>
+
+  {% if can_edit %}
+  <form method="post" action="/overlays/{{ overlay.id }}/items" class="stack">
+    <input type="hidden" name="csrf_token" value="{{ session.get('csrf_token', '') }}">
+    <fieldset class="workshop-input-mode">
+      <legend>Input mode</legend>
+      <label><input type="radio" name="input_mode" value="items" checked> Items (paste IDs or URLs; one or many)</label>
+      <label><input type="radio" name="input_mode" value="collection"> Collection (one ID or URL)</label>
+    </fieldset>
+    <label>Workshop input <textarea name="input" rows="3" placeholder="123456789&#10;https://steamcommunity.com/sharedfiles/filedetails/?id=987654321"></textarea></label>
+    <div>
+      <button type="submit">Add</button>
+    </div>
+  </form>
+  {% endif %}
+
+  <div id="overlay-item-table">
+    {% include "_overlay_item_table.html" with context %}
+  </div>
+</section>
+
+{% if latest_build_job %}
+<section class="panel">
+  <h2>Latest build</h2>
+  <p>
+    <a href="/jobs/{{ latest_build_job.id }}">job #{{ latest_build_job.id }}</a>
+    — state: <strong>{{ latest_build_job.state }}</strong>
+  </p>
+</section>
+{% endif %}
+{% endif %}
+
 <section class="panel">
   <h2>Used by</h2>
   {% if using_blueprints %}
@@ -43,7 +87,7 @@
   {% endif %}
 </section>
 
-{% if g.user.admin %}
+{% if can_edit %}
 <dialog id="delete-overlay-modal" class="modal" aria-labelledby="delete-overlay-title">
   <div class="modal-header">
     <h2 id="delete-overlay-title">Delete overlay "{{ overlay.name }}"?</h2>

l4d2web/templates/overlays.html

@@ -6,43 +6,48 @@
 <section class="panel">
   <div class="page-heading">
     <h1>Overlays</h1>
-    {% if g.user.admin %}
     <button type="button" data-modal-open="create-overlay-modal">+ Create</button>
-    {% endif %}
   </div>
 
   <table class="table">
-    <thead><tr><th>Name</th><th>Path</th></tr></thead>
+    <thead><tr><th>Name</th><th>Type</th><th>Scope</th><th>Path</th></tr></thead>
     <tbody>
       {% for overlay in overlays %}
       <tr>
         <td><a href="/overlays/{{ overlay.id }}">{{ overlay.name }}</a></td>
+        <td>{{ overlay.type }}</td>
+        <td class="muted">{% if overlay.user_id %}private{% else %}system{% endif %}</td>
         <td class="muted">{{ overlay.path }}</td>
       </tr>
       {% else %}
-      <tr><td colspan="2" class="muted">No overlays configured.</td></tr>
+      <tr><td colspan="4" class="muted">No overlays yet.</td></tr>
       {% endfor %}
     </tbody>
   </table>
 </section>
 
-{% if g.user.admin %}
 <dialog id="create-overlay-modal" class="modal" aria-labelledby="create-overlay-title">
   <form method="post" action="/overlays" class="stack">
     <div class="modal-header">
-      <h2 id="create-overlay-title">Add overlay</h2>
+      <h2 id="create-overlay-title">Create overlay</h2>
       <button type="button" class="modal-close" data-modal-close aria-label="Close">&times;</button>
     </div>
     <div class="modal-body">
       <input type="hidden" name="csrf_token" value="{{ session.get('csrf_token', '') }}">
+      <fieldset class="overlay-type-radio">
+        <legend>Type</legend>
+        <label><input type="radio" name="type" value="workshop" checked> Workshop (downloads from Steam)</label>
+        {% if g.user.admin %}
+        <label><input type="radio" name="type" value="external"> External (admin-managed; populated via filesystem)</label>
+        {% endif %}
+      </fieldset>
       <label>Name <input name="name" required></label>
-      <label>Path <input name="path" required placeholder="standard"></label>
+      <p class="muted">The path is generated automatically.</p>
     </div>
     <div class="modal-footer">
       <button type="button" class="button-secondary" data-modal-close>Cancel</button>
-      <button type="submit">Add overlay</button>
+      <button type="submit">Create</button>
     </div>
   </form>
 </dialog>
-{% endif %}
 {% endblock %}

l4d2web/tests/test_job_worker.py

@@ -8,16 +8,33 @@ from sqlalchemy import select
 
 from l4d2web.auth import hash_password
 from l4d2web.db import init_db, session_scope
-from l4d2web.models import Blueprint, Job, Server, User
+from l4d2web.models import (
+    Blueprint,
+    BlueprintOverlay,
+    Job,
+    Overlay,
+    OverlayWorkshopItem,
+    Server,
+    User,
+    WorkshopItem,
+)
 from l4d2web.services import l4d2_facade
 from l4d2web.services.host_commands import CommandCancelledError
-from l4d2web.services.job_worker import SchedulerState, can_start, recover_stale_jobs, run_worker_once
+from l4d2web.services.job_worker import (
+    SchedulerState,
+    build_scheduler_state,
+    can_start,
+    enqueue_build_overlay,
+    recover_stale_jobs,
+    run_worker_once,
+)
 
 
 @dataclass
 class DummyJob:
     operation: str
     server_id: int | None = None
+    overlay_id: int | None = None
 
 
 @pytest.fixture
@@ -65,12 +82,14 @@ def add_job(
     server_id: int | None,
     state: str = "queued",
     created_at: datetime | None = None,
+    overlay_id: int | None = None,
 ) -> int:
     now = datetime.now(UTC)
     with session_scope() as session:
         job = Job(
             user_id=user_id,
             server_id=server_id,
+            overlay_id=overlay_id,
             operation=operation,
             state=state,
             created_at=created_at or now,
@@ -379,3 +398,192 @@ def test_worker_startup_when_enabled_outside_testing(monkeypatch, tmp_path) -> N
     app = app_module.create_app({"TESTING": False, "DATABASE_URL": db_url, "SECRET_KEY": "test"})
 
     assert called == [app]
+
+
+# ---------------------------------------------------------------------------
+# Scheduler truth table for the new operations (build_overlay,
+# refresh_workshop_items) and their interaction with existing ops.
+# ---------------------------------------------------------------------------
+
+
+def test_install_blocks_build_overlay_and_refresh() -> None:
+    state = SchedulerState(install_running=True)
+    assert can_start(DummyJob(operation="build_overlay", overlay_id=1), state) is False
+    assert can_start(DummyJob(operation="refresh_workshop_items"), state) is False
+
+
+def test_refresh_blocks_install_build_overlay_and_servers() -> None:
+    state = SchedulerState(refresh_running=True)
+    assert can_start(DummyJob(operation="install"), state) is False
+    assert can_start(DummyJob(operation="build_overlay", overlay_id=1), state) is False
+    assert can_start(DummyJob(operation="start", server_id=1), state) is False
+
+
+def test_build_overlay_blocks_same_overlay_only() -> None:
+    state = SchedulerState()
+    state.running_overlays.add(7)
+    assert can_start(DummyJob(operation="build_overlay", overlay_id=7), state) is False
+    assert can_start(DummyJob(operation="build_overlay", overlay_id=8), state) is True
+
+
+def test_install_blocked_by_active_build_overlay() -> None:
+    state = SchedulerState()
+    state.running_overlays.add(7)
+    assert can_start(DummyJob(operation="install"), state) is False
+
+
+def test_refresh_blocked_by_active_build_overlay() -> None:
+    state = SchedulerState()
+    state.running_overlays.add(7)
+    assert can_start(DummyJob(operation="refresh_workshop_items"), state) is False
+
+
+def test_server_job_blocked_when_blueprint_overlay_is_building() -> None:
+    state = SchedulerState()
+    state.running_overlays.add(7)
+    state.blocked_servers_by_overlay.add(42)
+    assert can_start(DummyJob(operation="start", server_id=42), state) is False
+    # Other servers (whose blueprints don't reference overlay 7) are NOT blocked.
+    assert can_start(DummyJob(operation="start", server_id=43), state) is True
+
+
+@pytest.fixture
+def overlay_seeded_worker(seeded_worker):
+    app, ids = seeded_worker
+    with session_scope() as s:
+        overlay = Overlay(name="ws", path="9", type="workshop", user_id=ids.user)
+        s.add(overlay)
+        s.flush()
+        # Move server_two onto a different blueprint with NO workshop overlay,
+        # so the test can distinguish "blocked by overlay build" from "any
+        # server is blocked".
+        bp_with_overlay = s.scalar(select(Blueprint).where(Blueprint.user_id == ids.user))
+        s.add(BlueprintOverlay(blueprint_id=bp_with_overlay.id, overlay_id=overlay.id, position=0))
+        bp_without = Blueprint(user_id=ids.user, name="no-overlay", arguments="[]", config="[]")
+        s.add(bp_without)
+        s.flush()
+        server_two = s.scalar(select(Server).where(Server.id == ids.server_two))
+        server_two.blueprint_id = bp_without.id
+        ids.overlay = overlay.id
+    return app, ids
+
+
+def test_scheduler_state_finds_servers_blocked_by_running_build(overlay_seeded_worker) -> None:
+    app, ids = overlay_seeded_worker
+    add_job(ids.user, "build_overlay", server_id=None, state="running", overlay_id=ids.overlay)
+
+    with session_scope() as s:
+        state = build_scheduler_state(s)
+
+    assert ids.overlay in state.running_overlays
+    assert ids.server_one in state.blocked_servers_by_overlay
+    assert ids.server_two not in state.blocked_servers_by_overlay
+
+
+def test_enqueue_build_overlay_creates_new_job_when_none_pending(overlay_seeded_worker) -> None:
+    app, ids = overlay_seeded_worker
+
+    with session_scope() as s:
+        job = enqueue_build_overlay(s, overlay_id=ids.overlay, user_id=ids.user)
+        assert job.operation == "build_overlay"
+        assert job.overlay_id == ids.overlay
+        assert job.server_id is None
+        assert job.state == "queued"
+
+
+def test_enqueue_build_overlay_coalesces_against_pending(overlay_seeded_worker) -> None:
+    app, ids = overlay_seeded_worker
+
+    with session_scope() as s:
+        first = enqueue_build_overlay(s, overlay_id=ids.overlay, user_id=ids.user)
+        first_id = first.id
+
+    with session_scope() as s:
+        second = enqueue_build_overlay(s, overlay_id=ids.overlay, user_id=ids.user)
+        assert second.id == first_id, "should coalesce against the pending job"
+
+    with session_scope() as s:
+        n = s.query(Job).filter_by(operation="build_overlay", overlay_id=ids.overlay).count()
+        assert n == 1
+
+
+def test_enqueue_build_overlay_does_not_coalesce_against_running(overlay_seeded_worker) -> None:
+    app, ids = overlay_seeded_worker
+    add_job(ids.user, "build_overlay", server_id=None, state="running", overlay_id=ids.overlay)
+
+    with session_scope() as s:
+        new_job = enqueue_build_overlay(s, overlay_id=ids.overlay, user_id=ids.user)
+        assert new_job.state == "queued"
+
+    with session_scope() as s:
+        running = s.scalars(
+            select(Job).where(
+                Job.operation == "build_overlay",
+                Job.overlay_id == ids.overlay,
+                Job.state == "running",
+            )
+        ).all()
+        queued = s.scalars(
+            select(Job).where(
+                Job.operation == "build_overlay",
+                Job.overlay_id == ids.overlay,
+                Job.state == "queued",
+            )
+        ).all()
+        assert len(running) == 1
+        assert len(queued) == 1
+
+
+def test_run_worker_once_dispatches_build_overlay(overlay_seeded_worker, monkeypatch, tmp_path) -> None:
+    app, ids = overlay_seeded_worker
+    monkeypatch.setenv("LEFT4ME_ROOT", str(tmp_path))
+
+    with session_scope() as s:
+        wi = WorkshopItem(steam_id="1001", title="A", filename="a.vpk", file_url="u", file_size=3, time_updated=1)
+        s.add(wi)
+        s.flush()
+        s.add(OverlayWorkshopItem(overlay_id=ids.overlay, workshop_item_id=wi.id))
+    cache = tmp_path / "workshop_cache"
+    cache.mkdir()
+    (cache / "1001.vpk").write_bytes(b"abc")
+    import os
+    os.utime(cache / "1001.vpk", (1, 1))
+    # Mark item as downloaded.
+    with session_scope() as s:
+        wi = s.query(WorkshopItem).filter_by(steam_id="1001").one()
+        wi.last_downloaded_at = datetime.now(UTC)
+
+    job_id = add_job(ids.user, "build_overlay", server_id=None, overlay_id=ids.overlay)
+
+    with app.app_context():
+        assert run_worker_once() is True
+
+    job = load_job(job_id)
+    assert job.state == "succeeded", job.exit_code
+    addons = tmp_path / "overlays" / "9" / "left4dead2" / "addons"
+    assert (addons / "1001.vpk").is_symlink()
+
+
+def test_run_worker_once_dispatches_refresh(overlay_seeded_worker, monkeypatch, tmp_path) -> None:
+    app, ids = overlay_seeded_worker
+    monkeypatch.setenv("LEFT4ME_ROOT", str(tmp_path))
+
+    from l4d2web.services import steam_workshop, job_worker
+
+    refresh_calls = []
+
+    def fake_refresh_workshop_items(*, on_stdout, on_stderr, should_cancel):
+        refresh_calls.append(True)
+        on_stdout("refresh phase complete (downloaded=0 errors=0)")
+        return []  # no overlays affected
+
+    monkeypatch.setattr(job_worker, "_run_refresh_workshop_items", fake_refresh_workshop_items)
+
+    job_id = add_job(ids.user, "refresh_workshop_items", server_id=None)
+
+    with app.app_context():
+        assert run_worker_once() is True
+
+    assert refresh_calls == [True]
+    job = load_job(job_id)
+    assert job.state == "succeeded"

l4d2web/tests/test_l4d2_facade.py

@@ -1,3 +1,4 @@
+from datetime import UTC, datetime
 from pathlib import Path
 
 import pytest
@@ -5,7 +6,15 @@ import pytest
 from l4d2web.app import create_app
 from l4d2web.auth import hash_password
 from l4d2web.db import init_db, session_scope
-from l4d2web.models import Blueprint, BlueprintOverlay, Overlay, Server, User
+from l4d2web.models import (
+    Blueprint,
+    BlueprintOverlay,
+    Overlay,
+    OverlayWorkshopItem,
+    Server,
+    User,
+    WorkshopItem,
+)
 from l4d2web.services.host_commands import CommandResult
 
 
@@ -13,6 +22,7 @@ from l4d2web.services.host_commands import CommandResult
 def server_with_blueprint(tmp_path, monkeypatch):
     db_url = f"sqlite:///{tmp_path/'facade.db'}"
     monkeypatch.setenv("DATABASE_URL", db_url)
+    monkeypatch.setenv("LEFT4ME_ROOT", str(tmp_path))
     app = create_app({"TESTING": True, "DATABASE_URL": db_url, "SECRET_KEY": "test"})
     init_db()
 
@@ -22,7 +32,7 @@ def server_with_blueprint(tmp_path, monkeypatch):
             session.add(user)
             session.flush()
 
-            overlay = Overlay(name="Standard Overlay", path="standard")
+            overlay = Overlay(name="Standard Overlay", path="standard", type="external", user_id=None)
             session.add(overlay)
             session.flush()
 
@@ -41,8 +51,9 @@ def server_with_blueprint(tmp_path, monkeypatch):
             session.add(server)
             session.flush()
             server_id = server.id
+            user_id = user.id
 
-    return server_id
+    return server_id, user_id
 
 
 def test_initialize_uses_l4d2ctl_with_latest_blueprint_data(
@@ -70,7 +81,8 @@ def test_initialize_uses_l4d2ctl_with_latest_blueprint_data(
 
     from l4d2web.services.l4d2_facade import initialize_server
 
-    initialize_server(server_with_blueprint)
+    server_id, _ = server_with_blueprint
+    initialize_server(server_id)
 
     assert calls[0][:3] == ["l4d2ctl", "initialize", "alpha"]
     assert calls[0][3] == "-f"
@@ -97,10 +109,11 @@ def test_install_and_lifecycle_commands_use_l4d2ctl(
 
     from l4d2web.services.l4d2_facade import delete_server, install_runtime, start_server, stop_server
 
+    server_id, _ = server_with_blueprint
     install_runtime()
-    start_server(server_with_blueprint)
-    stop_server(server_with_blueprint)
-    delete_server(server_with_blueprint)
+    start_server(server_id)
+    stop_server(server_id)
+    delete_server(server_id)
 
     assert calls == [
         ["l4d2ctl", "install"],
@@ -159,3 +172,89 @@ def test_server_logs_stream_l4d2ctl_logs(monkeypatch: pytest.MonkeyPatch) -> Non
 
     assert calls == [["l4d2ctl", "logs", "alpha", "--lines", "10", "--no-follow"]]
     assert lines == ["one", "two"]
+
+
+def _attach_workshop_overlay_to_blueprint(
+    server_id: int, user_id: int, *, item_cached: bool, tmp_path: Path
+) -> tuple[int, str]:
+    """Add a workshop overlay with a single workshop item to the server's
+    blueprint. Returns (overlay_id, steam_id)."""
+    with session_scope() as session:
+        server = session.query(Server).filter_by(id=server_id).one()
+        overlay = Overlay(name="ws", path="placeholder", type="workshop", user_id=user_id)
+        session.add(overlay)
+        session.flush()
+        # Path matches id, like the production create_overlay flow does.
+        overlay.path = str(overlay.id)
+        wi = WorkshopItem(
+            steam_id="1001",
+            title="A",
+            filename="a.vpk",
+            file_url="https://example.com/a.vpk",
+            file_size=3,
+            time_updated=1700000000,
+            last_downloaded_at=datetime.now(UTC) if item_cached else None,
+        )
+        session.add(wi)
+        session.flush()
+        session.add(
+            BlueprintOverlay(blueprint_id=server.blueprint_id, overlay_id=overlay.id, position=1)
+        )
+        session.add(OverlayWorkshopItem(overlay_id=overlay.id, workshop_item_id=wi.id))
+        overlay_id = overlay.id
+
+    if item_cached:
+        cache_root = tmp_path / "workshop_cache"
+        cache_root.mkdir(exist_ok=True)
+        (cache_root / "1001.vpk").write_bytes(b"abc")
+
+    return overlay_id, "1001"
+
+
+def test_initialize_runs_overlay_builders_synchronously(
+    monkeypatch: pytest.MonkeyPatch, server_with_blueprint, tmp_path
+) -> None:
+    server_id, user_id = server_with_blueprint
+    overlay_id, _steam_id = _attach_workshop_overlay_to_blueprint(
+        server_id, user_id, item_cached=True, tmp_path=tmp_path
+    )
+
+    monkeypatch.setattr(
+        "l4d2web.services.host_commands.run_command",
+        lambda *args, **kwargs: CommandResult(returncode=0, stdout="", stderr=""),
+    )
+
+    from l4d2web.services.l4d2_facade import initialize_server
+
+    initialize_server(server_id)
+
+    addons = tmp_path / "overlays" / str(overlay_id) / "left4dead2" / "addons"
+    assert (addons / "1001.vpk").is_symlink(), "workshop builder must run before l4d2ctl initialize"
+
+
+def test_initialize_fails_fast_on_uncached_workshop_items(
+    monkeypatch: pytest.MonkeyPatch, server_with_blueprint, tmp_path
+) -> None:
+    server_id, user_id = server_with_blueprint
+    overlay_id, steam_id = _attach_workshop_overlay_to_blueprint(
+        server_id, user_id, item_cached=False, tmp_path=tmp_path
+    )
+
+    invocations: list[list[str]] = []
+
+    def fake_run_command(cmd, **kwargs):
+        invocations.append(list(cmd))
+        return CommandResult(returncode=0, stdout="", stderr="")
+
+    monkeypatch.setattr("l4d2web.services.host_commands.run_command", fake_run_command)
+
+    from l4d2web.services.l4d2_facade import initialize_server
+
+    with pytest.raises(Exception) as excinfo:
+        initialize_server(server_id)
+
+    msg = str(excinfo.value)
+    assert steam_id in msg
+    assert str(overlay_id) in msg or "ws" in msg
+    # l4d2ctl initialize MUST NOT run when uncached items are present.
+    assert all("initialize" not in cmd for cmd in invocations), invocations

l4d2web/tests/test_overlay_builders.py

@@ -0,0 +1,238 @@
+"""Tests for overlay builders (registry, ExternalBuilder, WorkshopBuilder)."""
+from __future__ import annotations
+
+import os
+from datetime import UTC, datetime
+from pathlib import Path
+
+import pytest
+
+from l4d2web.db import init_db, session_scope
+from l4d2web.models import Overlay, OverlayWorkshopItem, User, WorkshopItem
+from l4d2web.services import overlay_builders
+
+
+@pytest.fixture
+def env(tmp_path, monkeypatch):
+    monkeypatch.setenv("DATABASE_URL", f"sqlite:///{tmp_path/'b.db'}")
+    monkeypatch.setenv("LEFT4ME_ROOT", str(tmp_path))
+    init_db()
+    yield tmp_path
+
+
+def _create_user_and_overlay(name: str, type_: str) -> tuple[int, int]:
+    with session_scope() as s:
+        user = User(username="alice", password_digest="x")
+        s.add(user)
+        s.flush()
+        overlay = Overlay(name=name, path=str(7), type=type_, user_id=user.id)
+        s.add(overlay)
+        s.flush()
+        return user.id, overlay.id
+
+
+def _add_workshop_item(steam_id: str, *, downloaded: bool, cache_root: Path, content: bytes = b"x") -> int:
+    if downloaded:
+        cache_root.mkdir(parents=True, exist_ok=True)
+        (cache_root / f"{steam_id}.vpk").write_bytes(content)
+    with session_scope() as s:
+        wi = WorkshopItem(
+            steam_id=steam_id,
+            title=f"item-{steam_id}",
+            filename=f"orig-{steam_id}.vpk",
+            file_url=f"https://example.com/{steam_id}.vpk",
+            file_size=len(content) if downloaded else 0,
+            time_updated=1700000000 if downloaded else 0,
+            last_downloaded_at=datetime.now(UTC) if downloaded else None,
+        )
+        s.add(wi)
+        s.flush()
+        return wi.id
+
+
+def _associate(overlay_id: int, item_id: int) -> None:
+    with session_scope() as s:
+        s.add(OverlayWorkshopItem(overlay_id=overlay_id, workshop_item_id=item_id))
+
+
+def _capture_logs():
+    out: list[str] = []
+    err: list[str] = []
+    return out, err, out.append, err.append
+
+
+def test_registry_has_external_and_workshop() -> None:
+    assert "external" in overlay_builders.BUILDERS
+    assert "workshop" in overlay_builders.BUILDERS
+
+
+def test_registry_unknown_type_raises_keyerror() -> None:
+    with pytest.raises(KeyError):
+        overlay_builders.BUILDERS["nope"]
+
+
+def test_external_builder_is_idempotent_noop_with_log(env: Path) -> None:
+    _, overlay_id = _create_user_and_overlay("ext", "external")
+    out, err, on_stdout, on_stderr = _capture_logs()
+
+    with session_scope() as s:
+        overlay = s.query(Overlay).filter_by(id=overlay_id).one()
+        overlay_builders.BUILDERS["external"].build(
+            overlay, on_stdout=on_stdout, on_stderr=on_stderr, should_cancel=lambda: False
+        )
+
+    assert (env / "overlays" / "7").is_dir()
+    assert any("external overlay" in line for line in out), out
+    # Existing files in the overlay dir are not touched on subsequent build.
+    (env / "overlays" / "7" / "untouched.txt").write_text("data")
+    with session_scope() as s:
+        overlay = s.query(Overlay).filter_by(id=overlay_id).one()
+        overlay_builders.BUILDERS["external"].build(
+            overlay, on_stdout=on_stdout, on_stderr=on_stderr, should_cancel=lambda: False
+        )
+    assert (env / "overlays" / "7" / "untouched.txt").read_text() == "data"
+
+
+def test_workshop_builder_creates_absolute_symlinks(env: Path) -> None:
+    _, overlay_id = _create_user_and_overlay("ws", "workshop")
+    cache_root = env / "workshop_cache"
+    item_a = _add_workshop_item("1001", downloaded=True, cache_root=cache_root, content=b"AAA")
+    item_b = _add_workshop_item("1002", downloaded=True, cache_root=cache_root, content=b"BBBB")
+    _associate(overlay_id, item_a)
+    _associate(overlay_id, item_b)
+
+    out, err, on_stdout, on_stderr = _capture_logs()
+    with session_scope() as s:
+        overlay = s.query(Overlay).filter_by(id=overlay_id).one()
+        overlay_builders.BUILDERS["workshop"].build(
+            overlay, on_stdout=on_stdout, on_stderr=on_stderr, should_cancel=lambda: False
+        )
+
+    addons = env / "overlays" / "7" / "left4dead2" / "addons"
+    link_a = addons / "1001.vpk"
+    link_b = addons / "1002.vpk"
+    assert link_a.is_symlink()
+    assert link_b.is_symlink()
+    # Targets must be ABSOLUTE so they resolve in the host's namespace.
+    assert os.path.isabs(os.readlink(link_a))
+    assert os.path.isabs(os.readlink(link_b))
+    # And they must resolve to the cache files.
+    assert link_a.resolve() == (cache_root / "1001.vpk").resolve()
+    assert link_b.resolve() == (cache_root / "1002.vpk").resolve()
+
+
+def test_workshop_builder_skips_uncached_items_with_warning(env: Path) -> None:
+    _, overlay_id = _create_user_and_overlay("ws", "workshop")
+    cache_root = env / "workshop_cache"
+
+    cached = _add_workshop_item("1001", downloaded=True, cache_root=cache_root)
+    uncached = _add_workshop_item("9999", downloaded=False, cache_root=cache_root)
+    _associate(overlay_id, cached)
+    _associate(overlay_id, uncached)
+
+    out, err, on_stdout, on_stderr = _capture_logs()
+    with session_scope() as s:
+        overlay = s.query(Overlay).filter_by(id=overlay_id).one()
+        overlay_builders.BUILDERS["workshop"].build(
+            overlay, on_stdout=on_stdout, on_stderr=on_stderr, should_cancel=lambda: False
+        )
+
+    addons = env / "overlays" / "7" / "left4dead2" / "addons"
+    assert (addons / "1001.vpk").is_symlink()
+    assert not (addons / "9999.vpk").exists(), "must NOT create dangling symlink"
+    assert any("9999" in line and ("skip" in line.lower() or "uncached" in line.lower()) for line in err + out), err + out
+
+
+def test_workshop_builder_rerun_is_idempotent(env: Path) -> None:
+    _, overlay_id = _create_user_and_overlay("ws", "workshop")
+    cache_root = env / "workshop_cache"
+    item = _add_workshop_item("1001", downloaded=True, cache_root=cache_root)
+    _associate(overlay_id, item)
+
+    addons = env / "overlays" / "7" / "left4dead2" / "addons"
+
+    # First run.
+    with session_scope() as s:
+        overlay = s.query(Overlay).filter_by(id=overlay_id).one()
+        overlay_builders.BUILDERS["workshop"].build(overlay, on_stdout=lambda _x: None, on_stderr=lambda _x: None, should_cancel=lambda: False)
+    first_inode = (addons / "1001.vpk").lstat().st_ino
+
+    # Second run — no-op.
+    out, err, on_stdout, on_stderr = _capture_logs()
+    with session_scope() as s:
+        overlay = s.query(Overlay).filter_by(id=overlay_id).one()
+        overlay_builders.BUILDERS["workshop"].build(overlay, on_stdout=on_stdout, on_stderr=on_stderr, should_cancel=lambda: False)
+
+    second_inode = (addons / "1001.vpk").lstat().st_ino
+    assert first_inode == second_inode, "symlink should be untouched on idempotent rebuild"
+    assert any("unchanged" in line.lower() for line in out), out
+
+
+def test_workshop_builder_removes_obsolete_symlinks(env: Path) -> None:
+    _, overlay_id = _create_user_and_overlay("ws", "workshop")
+    cache_root = env / "workshop_cache"
+    item_a = _add_workshop_item("1001", downloaded=True, cache_root=cache_root)
+    item_b = _add_workshop_item("1002", downloaded=True, cache_root=cache_root)
+    _associate(overlay_id, item_a)
+    _associate(overlay_id, item_b)
+
+    addons = env / "overlays" / "7" / "left4dead2" / "addons"
+    with session_scope() as s:
+        overlay = s.query(Overlay).filter_by(id=overlay_id).one()
+        overlay_builders.BUILDERS["workshop"].build(overlay, on_stdout=lambda _x: None, on_stderr=lambda _x: None, should_cancel=lambda: False)
+    assert (addons / "1002.vpk").is_symlink()
+
+    # Remove the association for 1002.
+    with session_scope() as s:
+        s.query(OverlayWorkshopItem).filter_by(workshop_item_id=item_b).delete()
+
+    out, err, on_stdout, on_stderr = _capture_logs()
+    with session_scope() as s:
+        overlay = s.query(Overlay).filter_by(id=overlay_id).one()
+        overlay_builders.BUILDERS["workshop"].build(overlay, on_stdout=on_stdout, on_stderr=on_stderr, should_cancel=lambda: False)
+
+    assert (addons / "1001.vpk").is_symlink()
+    assert not (addons / "1002.vpk").exists()
+    # Cache file must remain — overlays are diff-applied, cache is shared.
+    assert (cache_root / "1002.vpk").exists()
+
+
+def test_workshop_builder_leaves_unrelated_files_alone(env: Path) -> None:
+    _, overlay_id = _create_user_and_overlay("ws", "workshop")
+    cache_root = env / "workshop_cache"
+    item = _add_workshop_item("1001", downloaded=True, cache_root=cache_root)
+    _associate(overlay_id, item)
+
+    addons = env / "overlays" / "7" / "left4dead2" / "addons"
+    addons.mkdir(parents=True, exist_ok=True)
+    (addons / "manual_addon.vpk").write_bytes(b"hand-placed")
+
+    with session_scope() as s:
+        overlay = s.query(Overlay).filter_by(id=overlay_id).one()
+        overlay_builders.BUILDERS["workshop"].build(overlay, on_stdout=lambda _x: None, on_stderr=lambda _x: None, should_cancel=lambda: False)
+
+    # Manual file is preserved.
+    assert (addons / "manual_addon.vpk").read_bytes() == b"hand-placed"
+    # Workshop symlink is created alongside.
+    assert (addons / "1001.vpk").is_symlink()
+
+
+def test_workshop_builder_honors_should_cancel(env: Path) -> None:
+    _, overlay_id = _create_user_and_overlay("ws", "workshop")
+    cache_root = env / "workshop_cache"
+    items = [_add_workshop_item(f"100{i}", downloaded=True, cache_root=cache_root) for i in range(3)]
+    for it in items:
+        _associate(overlay_id, it)
+
+    cancel_calls = {"n": 0}
+
+    def cancel():
+        cancel_calls["n"] += 1
+        return cancel_calls["n"] > 0  # cancel immediately
+
+    with session_scope() as s:
+        overlay = s.query(Overlay).filter_by(id=overlay_id).one()
+        # Should not crash; partial state is consistent (re-run heals).
+        overlay_builders.BUILDERS["workshop"].build(
+            overlay, on_stdout=lambda _x: None, on_stderr=lambda _x: None, should_cancel=cancel
+        )

l4d2web/tests/test_overlay_creation.py

@@ -0,0 +1,35 @@
+"""Tests for overlay path generation and directory creation."""
+from pathlib import Path
+
+import pytest
+
+from l4d2web.models import Overlay
+from l4d2web.services import overlay_creation
+
+
+def test_generate_overlay_path_returns_str_id() -> None:
+    assert overlay_creation.generate_overlay_path(42) == "42"
+
+
+def test_generate_overlay_path_validates_through_overlay_ref(monkeypatch) -> None:
+    # Sanity: numeric paths pass validate_overlay_ref. Anything bizarre would raise.
+    assert overlay_creation.generate_overlay_path(1) == "1"
+
+
+def test_create_overlay_directory_makes_subtree(monkeypatch, tmp_path: Path) -> None:
+    monkeypatch.setenv("LEFT4ME_ROOT", str(tmp_path))
+    overlay = Overlay(id=7, name="test", path="7", type="workshop", user_id=None)
+    overlay_creation.create_overlay_directory(overlay)
+    expected = tmp_path / "overlays" / "7"
+    assert expected.is_dir()
+
+
+def test_create_overlay_directory_raises_if_already_exists(
+    monkeypatch, tmp_path: Path
+) -> None:
+    monkeypatch.setenv("LEFT4ME_ROOT", str(tmp_path))
+    overlay = Overlay(id=7, name="test", path="7", type="workshop", user_id=None)
+    (tmp_path / "overlays" / "7").mkdir(parents=True)
+    # exist_ok=False guards against a stray directory shadowing fresh content.
+    with pytest.raises(FileExistsError):
+        overlay_creation.create_overlay_directory(overlay)

l4d2web/tests/test_overlays.py

@@ -10,6 +10,7 @@ from l4d2web.services.security import validate_overlay_ref
 def admin_client(tmp_path, monkeypatch):
     db_url = f"sqlite:///{tmp_path/'admin_overlay.db'}"
     monkeypatch.setenv("DATABASE_URL", db_url)
+    monkeypatch.setenv("LEFT4ME_ROOT", str(tmp_path))
     app = create_app({"TESTING": True, "DATABASE_URL": db_url, "SECRET_KEY": "test"})
     init_db()
 
@@ -30,13 +31,15 @@ def admin_client(tmp_path, monkeypatch):
 def user_client_with_overlay(tmp_path, monkeypatch):
     db_url = f"sqlite:///{tmp_path/'user_overlay.db'}"
     monkeypatch.setenv("DATABASE_URL", db_url)
+    monkeypatch.setenv("LEFT4ME_ROOT", str(tmp_path))
     app = create_app({"TESTING": True, "DATABASE_URL": db_url, "SECRET_KEY": "test"})
     init_db()
 
     with session_scope() as session:
         user = User(username="alice", password_digest=hash_password("secret"), admin=False)
         session.add(user)
-        session.add(Overlay(name="standard", path="standard"))
+        # System external overlay (no user_id), pre-existing.
+        session.add(Overlay(name="standard", path="standard", type="external", user_id=None))
         session.flush()
         user_id = user.id
 
@@ -53,7 +56,8 @@ def test_user_can_view_overlay_catalog(user_client_with_overlay) -> None:
 
     assert response.status_code == 200
     assert "standard" in text
-    assert "Add overlay" not in text
+    # Non-admin users can create workshop overlays, so the Create button shows.
+    assert "Create overlay" in text
 
 
 def test_admin_can_view_overlay_edit_controls(admin_client) -> None:
@@ -61,27 +65,19 @@ def test_admin_can_view_overlay_edit_controls(admin_client) -> None:
     text = response.get_data(as_text=True)
 
     assert response.status_code == 200
-    assert "Add overlay" in text
+    assert "Create overlay" in text
     assert 'action="/overlays"' in text
 
 
-def test_admin_can_create_overlay(admin_client) -> None:
+def test_admin_can_create_external_overlay(admin_client) -> None:
     response = admin_client.post(
         "/overlays",
-        data={"name": "standard", "path": "standard"},
+        data={"name": "standard", "type": "external"},
         headers={"X-CSRF-Token": "test-token"},
     )
     assert response.status_code == 302
-    assert response.headers["Location"] == "/overlays"
-
-
-def test_overlay_ref_must_be_relative(admin_client) -> None:
-    response = admin_client.post(
-        "/overlays",
-        data={"name": "bad", "path": "/tmp/bad"},
-        headers={"X-CSRF-Token": "test-token"},
-    )
-    assert response.status_code == 400
+    # Redirect to the new detail page now that paths are auto-generated.
+    assert response.headers["Location"].startswith("/overlays/")
 
 
 @pytest.mark.parametrize("overlay_ref", [" standard", "standard ", "a//b", "a/", "./a", "a/.", "."])
@@ -90,71 +86,136 @@ def test_overlay_ref_rejects_unsafe_components(overlay_ref: str) -> None:
         validate_overlay_ref(overlay_ref)
 
 
-def test_overlay_route_rejects_whitespace_padded_ref(admin_client) -> None:
-    response = admin_client.post(
-        "/overlays",
-        data={"name": "bad", "path": " standard"},
-        headers={"X-CSRF-Token": "test-token"},
-    )
-    assert response.status_code == 400
-
-
-def test_non_admin_cannot_create_overlay(user_client_with_overlay) -> None:
+def test_non_admin_cannot_create_external_overlay(user_client_with_overlay) -> None:
     response = user_client_with_overlay.post(
         "/overlays",
-        data={"name": "bad", "path": "bad"},
+        data={"name": "bad", "type": "external"},
         headers={"X-CSRF-Token": "test-token"},
     )
     assert response.status_code == 403
 
 
+def test_user_can_create_workshop_overlay(user_client_with_overlay) -> None:
+    response = user_client_with_overlay.post(
+        "/overlays",
+        data={"name": "my-maps", "type": "workshop"},
+        headers={"X-CSRF-Token": "test-token"},
+    )
+    assert response.status_code == 302
+    with session_scope() as session:
+        overlay = session.query(Overlay).filter_by(name="my-maps").one()
+        assert overlay.type == "workshop"
+        assert overlay.user_id is not None
+        assert overlay.path == str(overlay.id)
+
+
+def test_workshop_overlay_directory_is_created_on_disk(user_client_with_overlay, tmp_path) -> None:
+    response = user_client_with_overlay.post(
+        "/overlays",
+        data={"name": "my-maps", "type": "workshop"},
+        headers={"X-CSRF-Token": "test-token"},
+    )
+    assert response.status_code == 302
+    with session_scope() as session:
+        overlay = session.query(Overlay).filter_by(name="my-maps").one()
+        overlay_id = overlay.id
+    assert (tmp_path / "overlays" / str(overlay_id)).is_dir()
+
+
+def test_two_users_can_have_workshop_overlay_with_same_name(tmp_path, monkeypatch) -> None:
+    # Set up a fresh app with two users.
+    db_url = f"sqlite:///{tmp_path/'shared.db'}"
+    monkeypatch.setenv("DATABASE_URL", db_url)
+    monkeypatch.setenv("LEFT4ME_ROOT", str(tmp_path))
+    app = create_app({"TESTING": True, "DATABASE_URL": db_url, "SECRET_KEY": "test"})
+    init_db()
+
+    with session_scope() as session:
+        for username in ("alice", "bob"):
+            session.add(User(username=username, password_digest=hash_password("x"), admin=False))
+        session.flush()
+        alice_id, bob_id = (
+            session.query(User).filter_by(username="alice").one().id,
+            session.query(User).filter_by(username="bob").one().id,
+        )
+
+    def client_for(uid):
+        c = app.test_client()
+        with c.session_transaction() as sess:
+            sess["user_id"] = uid
+            sess["csrf_token"] = "test-token"
+        return c
+
+    for uid in (alice_id, bob_id):
+        r = client_for(uid).post(
+            "/overlays",
+            data={"name": "my-maps", "type": "workshop"},
+            headers={"X-CSRF-Token": "test-token"},
+        )
+        assert r.status_code == 302
+
+    with session_scope() as session:
+        rows = session.query(Overlay).filter_by(name="my-maps").all()
+        assert {r.user_id for r in rows} == {alice_id, bob_id}
+
+
 def test_admin_can_update_and_delete_overlay(admin_client) -> None:
     create = admin_client.post(
         "/overlays",
-        data={"name": "standard", "path": "standard"},
+        data={"name": "standard", "type": "external"},
         headers={"X-CSRF-Token": "test-token"},
     )
     assert create.status_code == 302
 
+    with session_scope() as session:
+        overlay_id = session.query(Overlay).filter_by(name="standard").one().id
+
     update = admin_client.post(
-        "/overlays/1",
-        data={"name": "edited", "path": "edited"},
+        f"/overlays/{overlay_id}",
+        data={"name": "edited"},
         headers={"X-CSRF-Token": "test-token"},
     )
     assert update.status_code == 302
 
     delete = admin_client.post(
-        "/overlays/1/delete",
+        f"/overlays/{overlay_id}/delete",
         headers={"X-CSRF-Token": "test-token"},
     )
     assert delete.status_code == 302
 
 
 def test_update_overlay_rejects_duplicate_name(admin_client) -> None:
-    for name in ["standard", "competitive"]:
+    ids: list[int] = []
+    for name in ("standard", "competitive"):
         response = admin_client.post(
             "/overlays",
-            data={"name": name, "path": name},
+            data={"name": name, "type": "external"},
             headers={"X-CSRF-Token": "test-token"},
         )
         assert response.status_code == 302
+    with session_scope() as session:
+        ids = [
+            session.query(Overlay).filter_by(name="standard").one().id,
+            session.query(Overlay).filter_by(name="competitive").one().id,
+        ]
 
     response = admin_client.post(
-        "/overlays/2",
-        data={"name": "standard", "path": "competitive"},
+        f"/overlays/{ids[1]}",
+        data={"name": "standard"},
         headers={"X-CSRF-Token": "test-token"},
     )
-
     assert response.status_code == 409
 
 
 def test_overlay_detail_page_lists_using_blueprints(admin_client) -> None:
     create = admin_client.post(
         "/overlays",
-        data={"name": "shared", "path": "shared"},
+        data={"name": "shared", "type": "external"},
         headers={"X-CSRF-Token": "test-token"},
     )
     assert create.status_code == 302
+    with session_scope() as session:
+        overlay_id = session.query(Overlay).filter_by(name="shared").one().id
 
     with session_scope() as session:
         admin = session.query(User).filter_by(username="admin").one()
@@ -162,10 +223,10 @@ def test_overlay_detail_page_lists_using_blueprints(admin_client) -> None:
         bp_two = Blueprint(user_id=admin.id, name="beta-bp", arguments="[]", config="[]")
         session.add_all([bp_one, bp_two])
         session.flush()
-        session.add(BlueprintOverlay(blueprint_id=bp_one.id, overlay_id=1, position=0))
-        session.add(BlueprintOverlay(blueprint_id=bp_two.id, overlay_id=1, position=0))
+        session.add(BlueprintOverlay(blueprint_id=bp_one.id, overlay_id=overlay_id, position=0))
+        session.add(BlueprintOverlay(blueprint_id=bp_two.id, overlay_id=overlay_id, position=0))
 
-    response = admin_client.get("/overlays/1")
+    response = admin_client.get(f"/overlays/{overlay_id}")
     text = response.get_data(as_text=True)
 
     assert response.status_code == 200
@@ -179,7 +240,8 @@ def test_overlay_detail_page_404_when_missing(admin_client) -> None:
     assert response.status_code == 404
 
 
-def test_overlay_detail_hides_edit_for_non_admin(user_client_with_overlay) -> None:
+def test_overlay_detail_hides_edit_for_non_admin_external(user_client_with_overlay) -> None:
+    # The seeded "standard" external overlay (id=1, user_id=NULL) is admin-only edit.
     response = user_client_with_overlay.get("/overlays/1")
     text = response.get_data(as_text=True)
 
@@ -192,37 +254,41 @@ def test_overlay_detail_hides_edit_for_non_admin(user_client_with_overlay) -> No
 def test_overlay_update_redirects_to_detail(admin_client) -> None:
     create = admin_client.post(
         "/overlays",
-        data={"name": "standard", "path": "standard"},
+        data={"name": "standard", "type": "external"},
         headers={"X-CSRF-Token": "test-token"},
     )
     assert create.status_code == 302
+    with session_scope() as session:
+        overlay_id = session.query(Overlay).filter_by(name="standard").one().id
 
     response = admin_client.post(
-        "/overlays/1",
-        data={"name": "renamed", "path": "renamed"},
+        f"/overlays/{overlay_id}",
+        data={"name": "renamed"},
         headers={"X-CSRF-Token": "test-token"},
     )
     assert response.status_code == 302
-    assert response.headers["Location"] == "/overlays/1"
+    assert response.headers["Location"] == f"/overlays/{overlay_id}"
 
 
 def test_delete_overlay_rejects_in_use_overlay(admin_client) -> None:
     create = admin_client.post(
         "/overlays",
-        data={"name": "standard", "path": "standard"},
+        data={"name": "standard", "type": "external"},
         headers={"X-CSRF-Token": "test-token"},
     )
     assert create.status_code == 302
+    with session_scope() as session:
+        overlay_id = session.query(Overlay).filter_by(name="standard").one().id
 
     with session_scope() as session:
         user = session.query(User).filter_by(username="admin").one()
         blueprint = Blueprint(user_id=user.id, name="bp", arguments="[]", config="[]")
         session.add(blueprint)
         session.flush()
-        session.add(BlueprintOverlay(blueprint_id=blueprint.id, overlay_id=1, position=0))
+        session.add(BlueprintOverlay(blueprint_id=blueprint.id, overlay_id=overlay_id, position=0))
 
     response = admin_client.post(
-        "/overlays/1/delete",
+        f"/overlays/{overlay_id}/delete",
         headers={"X-CSRF-Token": "test-token"},
     )
 

l4d2web/tests/test_steam_workshop.py

@@ -0,0 +1,312 @@
+"""Tests for the Steam Workshop API client and downloader."""
+from __future__ import annotations
+
+import os
+from pathlib import Path
+from unittest.mock import MagicMock, patch
+
+import pytest
+
+from l4d2web.services import steam_workshop
+
+
+def test_parse_workshop_input_single_numeric() -> None:
+    assert steam_workshop.parse_workshop_input("12345") == ["12345"]
+
+
+def test_parse_workshop_input_single_url() -> None:
+    url = "https://steamcommunity.com/sharedfiles/filedetails/?id=98765"
+    assert steam_workshop.parse_workshop_input(url) == ["98765"]
+
+
+def test_parse_workshop_input_workshop_url_variant() -> None:
+    url = "steamcommunity.com/workshop/filedetails/?id=42"
+    assert steam_workshop.parse_workshop_input(url) == ["42"]
+
+
+def test_parse_workshop_input_multiline_batch() -> None:
+    raw = """
+    12345
+    https://steamcommunity.com/sharedfiles/filedetails/?id=67890
+    99999
+    """
+    assert steam_workshop.parse_workshop_input(raw) == ["12345", "67890", "99999"]
+
+
+def test_parse_workshop_input_deduplicates_preserving_order() -> None:
+    raw = "100\n200\n100\n300"
+    assert steam_workshop.parse_workshop_input(raw) == ["100", "200", "300"]
+
+
+def test_parse_workshop_input_rejects_garbage() -> None:
+    with pytest.raises(ValueError):
+        steam_workshop.parse_workshop_input("not-a-number")
+
+
+def test_parse_workshop_input_rejects_empty() -> None:
+    with pytest.raises(ValueError):
+        steam_workshop.parse_workshop_input("")
+
+
+def test_parse_workshop_input_rejects_non_steam_url() -> None:
+    with pytest.raises(ValueError):
+        steam_workshop.parse_workshop_input("https://example.com/?id=12345")
+
+
+def test_endpoints_are_https() -> None:
+    assert steam_workshop.GET_PUBLISHED_FILE_DETAILS_URL.startswith("https://")
+    assert steam_workshop.GET_COLLECTION_DETAILS_URL.startswith("https://")
+    assert "api.steampowered.com" in steam_workshop.GET_PUBLISHED_FILE_DETAILS_URL
+
+
+def test_resolve_collection_returns_child_ids() -> None:
+    fake_response = MagicMock(status_code=200)
+    fake_response.raise_for_status = MagicMock()
+    fake_response.json.return_value = {
+        "response": {
+            "collectiondetails": [
+                {
+                    "publishedfileid": "555",
+                    "result": 1,
+                    "children": [
+                        {"publishedfileid": "1001", "filetype": 0},
+                        {"publishedfileid": "1002", "filetype": 0},
+                        {"publishedfileid": "9999", "filetype": 1},  # nested collection — skip
+                    ],
+                }
+            ]
+        }
+    }
+    with patch.object(steam_workshop, "_session", return_value=MagicMock(post=MagicMock(return_value=fake_response))):
+        ids = steam_workshop.resolve_collection("555")
+    assert ids == ["1001", "1002"]
+
+
+def test_fetch_metadata_batch_parses_published_file_details() -> None:
+    fake_response = MagicMock(status_code=200)
+    fake_response.raise_for_status = MagicMock()
+    fake_response.json.return_value = {
+        "response": {
+            "publishedfiledetails": [
+                {
+                    "publishedfileid": "1001",
+                    "result": 1,
+                    "consumer_app_id": 550,
+                    "title": "Map A",
+                    "filename": "map_a.vpk",
+                    "file_url": "https://steamusercontent.com/abc/map_a.vpk",
+                    "file_size": "1024",
+                    "time_updated": 1700000000,
+                    "preview_url": "https://steamuserimages.com/preview_a.jpg",
+                }
+            ]
+        }
+    }
+    with patch.object(steam_workshop, "_session", return_value=MagicMock(post=MagicMock(return_value=fake_response))):
+        metas = steam_workshop.fetch_metadata_batch(["1001"], mode="add")
+    assert len(metas) == 1
+    m = metas[0]
+    assert m.steam_id == "1001"
+    assert m.title == "Map A"
+    assert m.filename == "map_a.vpk"
+    assert m.file_url == "https://steamusercontent.com/abc/map_a.vpk"
+    assert m.file_size == 1024
+    assert m.time_updated == 1700000000
+    assert m.preview_url == "https://steamuserimages.com/preview_a.jpg"
+    assert m.consumer_app_id == 550
+    assert m.result == 1
+
+
+def test_fetch_metadata_batch_rejects_non_l4d2_in_add_mode() -> None:
+    fake_response = MagicMock(status_code=200)
+    fake_response.raise_for_status = MagicMock()
+    fake_response.json.return_value = {
+        "response": {
+            "publishedfiledetails": [
+                {
+                    "publishedfileid": "1001",
+                    "result": 1,
+                    "consumer_app_id": 440,  # TF2
+                    "title": "Other",
+                    "filename": "x.vpk",
+                    "file_url": "https://example.com/x.vpk",
+                    "file_size": "0",
+                    "time_updated": 0,
+                }
+            ]
+        }
+    }
+    with patch.object(steam_workshop, "_session", return_value=MagicMock(post=MagicMock(return_value=fake_response))):
+        with pytest.raises(steam_workshop.WorkshopValidationError):
+            steam_workshop.fetch_metadata_batch(["1001"], mode="add")
+
+
+def test_fetch_metadata_batch_skips_non_l4d2_in_refresh_mode() -> None:
+    fake_response = MagicMock(status_code=200)
+    fake_response.raise_for_status = MagicMock()
+    fake_response.json.return_value = {
+        "response": {
+            "publishedfiledetails": [
+                {
+                    "publishedfileid": "1001",
+                    "result": 1,
+                    "consumer_app_id": 440,
+                    "title": "Other",
+                    "filename": "x.vpk",
+                    "file_url": "https://example.com/x.vpk",
+                    "file_size": "0",
+                    "time_updated": 0,
+                },
+                {
+                    "publishedfileid": "1002",
+                    "result": 1,
+                    "consumer_app_id": 550,
+                    "title": "Good",
+                    "filename": "g.vpk",
+                    "file_url": "https://example.com/g.vpk",
+                    "file_size": "100",
+                    "time_updated": 1,
+                },
+            ]
+        }
+    }
+    with patch.object(steam_workshop, "_session", return_value=MagicMock(post=MagicMock(return_value=fake_response))):
+        metas = steam_workshop.fetch_metadata_batch(["1001", "1002"], mode="refresh")
+    # The non-L4D2 item is dropped; the L4D2 item is kept.
+    assert [m.steam_id for m in metas] == ["1002"]
+
+
+def test_fetch_metadata_batch_captures_result_failure() -> None:
+    fake_response = MagicMock(status_code=200)
+    fake_response.raise_for_status = MagicMock()
+    fake_response.json.return_value = {
+        "response": {
+            "publishedfiledetails": [
+                {
+                    "publishedfileid": "999",
+                    "result": 9,  # not found / hidden / etc.
+                }
+            ]
+        }
+    }
+    with patch.object(steam_workshop, "_session", return_value=MagicMock(post=MagicMock(return_value=fake_response))):
+        metas = steam_workshop.fetch_metadata_batch(["999"], mode="refresh")
+    # Item is kept but marked with the failing result; consumer app id never validated.
+    assert len(metas) == 1
+    assert metas[0].result == 9
+
+
+def test_download_to_cache_writes_atomically_and_sets_mtime(tmp_path: Path) -> None:
+    cache_root = tmp_path / "workshop_cache"
+    cache_root.mkdir()
+    meta = steam_workshop.WorkshopMetadata(
+        steam_id="1001",
+        title="A",
+        filename="a.vpk",
+        file_url="https://example.com/a.vpk",
+        file_size=11,
+        time_updated=1700000000,
+        preview_url="",
+        consumer_app_id=550,
+        result=1,
+    )
+    fake_response = MagicMock(status_code=200)
+    fake_response.raise_for_status = MagicMock()
+    fake_response.iter_content.return_value = [b"hello world"]
+
+    with patch.object(steam_workshop, "_session", return_value=MagicMock(get=MagicMock(return_value=fake_response))):
+        path = steam_workshop.download_to_cache(meta, cache_root)
+
+    assert path == cache_root / "1001.vpk"
+    assert path.read_bytes() == b"hello world"
+    assert int(path.stat().st_mtime) == 1700000000
+    # No leftover .partial file.
+    assert not (cache_root / "1001.vpk.partial").exists()
+
+
+def test_download_to_cache_is_idempotent(tmp_path: Path) -> None:
+    cache_root = tmp_path / "workshop_cache"
+    cache_root.mkdir()
+    target = cache_root / "1001.vpk"
+    target.write_bytes(b"existing")
+    os.utime(target, (1700000000, 1700000000))
+
+    meta = steam_workshop.WorkshopMetadata(
+        steam_id="1001",
+        title="A",
+        filename="a.vpk",
+        file_url="https://example.com/a.vpk",
+        file_size=8,  # matches existing
+        time_updated=1700000000,  # matches existing mtime
+        preview_url="",
+        consumer_app_id=550,
+        result=1,
+    )
+
+    fake_session = MagicMock()
+    with patch.object(steam_workshop, "_session", return_value=fake_session):
+        steam_workshop.download_to_cache(meta, cache_root)
+
+    fake_session.get.assert_not_called()
+
+
+def test_download_to_cache_redownloads_when_mtime_or_size_differ(tmp_path: Path) -> None:
+    cache_root = tmp_path / "workshop_cache"
+    cache_root.mkdir()
+    target = cache_root / "1001.vpk"
+    target.write_bytes(b"old")
+    os.utime(target, (1500000000, 1500000000))
+
+    meta = steam_workshop.WorkshopMetadata(
+        steam_id="1001",
+        title="A",
+        filename="a.vpk",
+        file_url="https://example.com/a.vpk",
+        file_size=11,
+        time_updated=1700000000,
+        preview_url="",
+        consumer_app_id=550,
+        result=1,
+    )
+
+    fake_response = MagicMock(status_code=200)
+    fake_response.raise_for_status = MagicMock()
+    fake_response.iter_content.return_value = [b"hello world"]
+
+    with patch.object(steam_workshop, "_session", return_value=MagicMock(get=MagicMock(return_value=fake_response))):
+        steam_workshop.download_to_cache(meta, cache_root)
+
+    assert target.read_bytes() == b"hello world"
+    assert int(target.stat().st_mtime) == 1700000000
+
+
+def test_refresh_all_uses_thread_pool_and_collects_errors(tmp_path: Path) -> None:
+    cache_root = tmp_path / "workshop_cache"
+    cache_root.mkdir()
+
+    metas = [
+        steam_workshop.WorkshopMetadata(
+            steam_id=str(i),
+            title=f"M{i}",
+            filename=f"m{i}.vpk",
+            file_url=f"https://example.com/m{i}.vpk",
+            file_size=5,
+            time_updated=1700000000,
+            preview_url="",
+            consumer_app_id=550,
+            result=1,
+        )
+        for i in (1, 2, 3)
+    ]
+
+    def fake_download(meta, cache_root_arg, **kwargs):
+        if meta.steam_id == "2":
+            raise RuntimeError("simulated download failure")
+        return cache_root_arg / f"{meta.steam_id}.vpk"
+
+    with patch.object(steam_workshop, "download_to_cache", side_effect=fake_download):
+        report = steam_workshop.refresh_all(metas, cache_root, executor_workers=4)
+
+    assert report.downloaded == 2
+    assert report.errors == 1
+    assert "2" in report.per_item_errors

l4d2web/tests/test_workshop_overlay_models.py

@@ -0,0 +1,183 @@
+"""Tests for the workshop-overlay schema additions: typed Overlay, partial
+unique indexes, WorkshopItem registry, and overlay_workshop_items association.
+"""
+import pytest
+from sqlalchemy.exc import IntegrityError
+
+from l4d2web.db import init_db, session_scope
+from l4d2web.models import (
+    Job,
+    Overlay,
+    OverlayWorkshopItem,
+    User,
+    WorkshopItem,
+)
+
+
+@pytest.fixture
+def db(tmp_path, monkeypatch):
+    monkeypatch.setenv("DATABASE_URL", f"sqlite:///{tmp_path/'workshop.db'}")
+    init_db()
+    yield
+
+
+def _make_user(username: str) -> int:
+    with session_scope() as s:
+        user = User(username=username, password_digest="x")
+        s.add(user)
+        s.flush()
+        return user.id
+
+
+def test_overlay_has_type_and_user_id(db) -> None:
+    with session_scope() as s:
+        s.add(Overlay(name="standard", path="standard"))
+        s.flush()
+        row = s.query(Overlay).filter_by(name="standard").one()
+        assert row.type == "external"
+        assert row.user_id is None
+
+
+def test_two_externals_with_same_name_are_rejected(db) -> None:
+    with session_scope() as s:
+        s.add(Overlay(name="shared", path="shared", type="external", user_id=None))
+        s.flush()
+
+    with pytest.raises(IntegrityError):
+        with session_scope() as s:
+            s.add(Overlay(name="shared", path="other", type="external", user_id=None))
+            s.flush()
+
+
+def test_two_users_can_share_workshop_overlay_name(db) -> None:
+    alice_id = _make_user("alice")
+    bob_id = _make_user("bob")
+
+    with session_scope() as s:
+        s.add(Overlay(name="my-maps", path="1", type="workshop", user_id=alice_id))
+        s.add(Overlay(name="my-maps", path="2", type="workshop", user_id=bob_id))
+        s.flush()
+
+    with session_scope() as s:
+        rows = s.query(Overlay).filter_by(name="my-maps").all()
+        assert {r.user_id for r in rows} == {alice_id, bob_id}
+
+
+def test_same_user_cannot_have_duplicate_workshop_name(db) -> None:
+    user_id = _make_user("alice")
+
+    with session_scope() as s:
+        s.add(Overlay(name="dupe", path="1", type="workshop", user_id=user_id))
+        s.flush()
+
+    with pytest.raises(IntegrityError):
+        with session_scope() as s:
+            s.add(Overlay(name="dupe", path="2", type="workshop", user_id=user_id))
+            s.flush()
+
+
+def test_workshop_item_steam_id_is_unique(db) -> None:
+    with session_scope() as s:
+        s.add(WorkshopItem(steam_id="123", title="Map A"))
+        s.flush()
+
+    with pytest.raises(IntegrityError):
+        with session_scope() as s:
+            s.add(WorkshopItem(steam_id="123", title="Map A duplicate"))
+            s.flush()
+
+
+def test_overlay_workshop_item_unique_per_overlay(db) -> None:
+    user_id = _make_user("alice")
+
+    with session_scope() as s:
+        ov = Overlay(name="my-maps", path="1", type="workshop", user_id=user_id)
+        wi = WorkshopItem(steam_id="555", title="A")
+        s.add_all([ov, wi])
+        s.flush()
+        s.add(OverlayWorkshopItem(overlay_id=ov.id, workshop_item_id=wi.id))
+        s.flush()
+        overlay_id = ov.id
+        workshop_item_id = wi.id
+
+    with pytest.raises(IntegrityError):
+        with session_scope() as s:
+            s.add(
+                OverlayWorkshopItem(
+                    overlay_id=overlay_id, workshop_item_id=workshop_item_id
+                )
+            )
+            s.flush()
+
+
+def test_deleting_overlay_cascades_associations_but_not_workshop_items(db) -> None:
+    user_id = _make_user("alice")
+
+    with session_scope() as s:
+        ov = Overlay(name="my-maps", path="1", type="workshop", user_id=user_id)
+        wi = WorkshopItem(steam_id="777", title="A")
+        s.add_all([ov, wi])
+        s.flush()
+        s.add(OverlayWorkshopItem(overlay_id=ov.id, workshop_item_id=wi.id))
+        s.flush()
+        overlay_id = ov.id
+
+    # Delete via raw connection to actually exercise ON DELETE CASCADE / RESTRICT.
+    from l4d2web.db import get_engine
+
+    engine = get_engine()
+    with engine.begin() as conn:
+        conn.exec_driver_sql("PRAGMA foreign_keys=ON")
+        conn.exec_driver_sql(f"DELETE FROM overlays WHERE id = {overlay_id}")
+
+    with session_scope() as s:
+        assert s.query(OverlayWorkshopItem).count() == 0
+        assert s.query(WorkshopItem).filter_by(steam_id="777").count() == 1
+
+
+def test_job_has_overlay_id_column(db) -> None:
+    user_id = _make_user("alice")
+
+    with session_scope() as s:
+        ov = Overlay(name="my-maps", path="1", type="workshop", user_id=user_id)
+        s.add(ov)
+        s.flush()
+        s.add(
+            Job(
+                user_id=user_id,
+                server_id=None,
+                overlay_id=ov.id,
+                operation="build_overlay",
+                state="queued",
+            )
+        )
+        s.flush()
+
+    with session_scope() as s:
+        job = s.query(Job).filter_by(operation="build_overlay").one()
+        assert job.overlay_id is not None
+        assert job.server_id is None
+
+
+def test_overlay_id_does_not_reuse_after_delete(db) -> None:
+    """SQLite AUTOINCREMENT must guarantee deleted IDs are never reused."""
+    with session_scope() as s:
+        s.add(Overlay(name="first", path="1", type="external", user_id=None))
+        s.add(Overlay(name="second", path="2", type="external", user_id=None))
+        s.flush()
+        ids_before = sorted(o.id for o in s.query(Overlay).all())
+        last_id = ids_before[-1]
+
+    with session_scope() as s:
+        last = s.query(Overlay).filter_by(id=last_id).one()
+        s.delete(last)
+        s.flush()
+
+    with session_scope() as s:
+        s.add(Overlay(name="third", path="3", type="external", user_id=None))
+        s.flush()
+        new_id = s.query(Overlay).filter_by(name="third").one().id
+
+    assert new_id > last_id, (
+        f"AUTOINCREMENT should never reuse IDs, but got {new_id} after deleting {last_id}"
+    )

l4d2web/tests/test_workshop_paths.py

@@ -0,0 +1,23 @@
+"""Tests for workshop_paths cache-resolution helpers."""
+from pathlib import Path
+
+import pytest
+
+from l4d2web.services import workshop_paths
+
+
+def test_workshop_cache_root_uses_left4me_root(monkeypatch, tmp_path: Path) -> None:
+    monkeypatch.setenv("LEFT4ME_ROOT", str(tmp_path))
+    assert workshop_paths.workshop_cache_root() == tmp_path / "workshop_cache"
+
+
+def test_cache_path_returns_id_only_filename(monkeypatch, tmp_path: Path) -> None:
+    monkeypatch.setenv("LEFT4ME_ROOT", str(tmp_path))
+    assert workshop_paths.cache_path("12345") == tmp_path / "workshop_cache" / "12345.vpk"
+
+
+@pytest.mark.parametrize("bad", ["abc", "", "12/34", "..", "../etc", "1 2", " 1"])
+def test_cache_path_rejects_non_digit_steam_id(monkeypatch, tmp_path: Path, bad: str) -> None:
+    monkeypatch.setenv("LEFT4ME_ROOT", str(tmp_path))
+    with pytest.raises(ValueError):
+        workshop_paths.cache_path(bad)

l4d2web/tests/test_workshop_routes.py

@@ -0,0 +1,279 @@
+"""Tests for the workshop overlay routes (add items, remove items, build,
+admin refresh)."""
+from __future__ import annotations
+
+from typing import Iterable
+from unittest.mock import patch
+
+import pytest
+
+from l4d2web.app import create_app
+from l4d2web.auth import hash_password
+from l4d2web.db import init_db, session_scope
+from l4d2web.models import (
+    Job,
+    Overlay,
+    OverlayWorkshopItem,
+    User,
+    WorkshopItem,
+)
+from l4d2web.services import steam_workshop
+
+
+def _meta(steam_id: str, *, app_id: int = 550, result: int = 1) -> steam_workshop.WorkshopMetadata:
+    return steam_workshop.WorkshopMetadata(
+        steam_id=steam_id,
+        title=f"Item {steam_id}",
+        filename=f"{steam_id}.vpk",
+        file_url=f"https://example.com/{steam_id}.vpk",
+        file_size=42,
+        time_updated=1700000000,
+        preview_url=f"https://example.com/preview-{steam_id}.jpg",
+        consumer_app_id=app_id,
+        result=result,
+    )
+
+
+@pytest.fixture
+def env_user(tmp_path, monkeypatch):
+    db_url = f"sqlite:///{tmp_path/'wr.db'}"
+    monkeypatch.setenv("DATABASE_URL", db_url)
+    monkeypatch.setenv("LEFT4ME_ROOT", str(tmp_path))
+    app = create_app({"TESTING": True, "DATABASE_URL": db_url, "SECRET_KEY": "test"})
+    init_db()
+
+    with session_scope() as session:
+        user = User(username="alice", password_digest=hash_password("x"), admin=False)
+        admin = User(username="admin", password_digest=hash_password("x"), admin=True)
+        session.add_all([user, admin])
+        session.flush()
+        user_id = user.id
+        admin_id = admin.id
+
+    def login(uid):
+        c = app.test_client()
+        with c.session_transaction() as sess:
+            sess["user_id"] = uid
+            sess["csrf_token"] = "test-token"
+        return c
+
+    return app, login, user_id, admin_id
+
+
+@pytest.fixture
+def overlay_for(env_user):
+    app, login, user_id, admin_id = env_user
+
+    user_client = login(user_id)
+    response = user_client.post(
+        "/overlays",
+        data={"name": "my-maps", "type": "workshop"},
+        headers={"X-CSRF-Token": "test-token"},
+    )
+    assert response.status_code == 302, response.get_data(as_text=True)
+
+    with session_scope() as session:
+        overlay = session.query(Overlay).filter_by(name="my-maps").one()
+        overlay_id = overlay.id
+
+    return app, login, user_id, admin_id, overlay_id
+
+
+def _patch_steam(metas: Iterable[steam_workshop.WorkshopMetadata]):
+    return patch.object(steam_workshop, "fetch_metadata_batch", return_value=list(metas))
+
+
+def test_add_single_item_creates_association_and_enqueues_build(overlay_for):
+    app, login, user_id, _admin_id, overlay_id = overlay_for
+    user_client = login(user_id)
+
+    with _patch_steam([_meta("1001")]):
+        response = user_client.post(
+            f"/overlays/{overlay_id}/items",
+            data={"input": "1001", "input_mode": "items"},
+            headers={"X-CSRF-Token": "test-token"},
+        )
+    assert response.status_code == 200
+    assert b"1001" in response.data
+
+    with session_scope() as session:
+        n_assoc = session.query(OverlayWorkshopItem).count()
+        assert n_assoc == 1
+        wi = session.query(WorkshopItem).filter_by(steam_id="1001").one()
+        assert wi.title == "Item 1001"
+        assert wi.preview_url.endswith("preview-1001.jpg")
+        # Auto-enqueued build_overlay job.
+        jobs = session.query(Job).filter_by(operation="build_overlay", overlay_id=overlay_id).all()
+        assert len(jobs) == 1
+        assert jobs[0].state == "queued"
+
+
+def test_add_multiline_batch_coalesces_into_one_build_job(overlay_for):
+    app, login, user_id, _admin_id, overlay_id = overlay_for
+    user_client = login(user_id)
+
+    with _patch_steam([_meta(s) for s in ("1001", "1002", "1003")]):
+        response = user_client.post(
+            f"/overlays/{overlay_id}/items",
+            data={"input": "1001\n1002\n1003", "input_mode": "items"},
+            headers={"X-CSRF-Token": "test-token"},
+        )
+    assert response.status_code == 200
+
+    with session_scope() as session:
+        assert session.query(OverlayWorkshopItem).count() == 3
+        jobs = session.query(Job).filter_by(operation="build_overlay", overlay_id=overlay_id).all()
+        assert len(jobs) == 1, "multi-item add should coalesce into a single build job"
+
+
+def test_add_collection_resolves_members(overlay_for):
+    app, login, user_id, _admin_id, overlay_id = overlay_for
+    user_client = login(user_id)
+
+    with patch.object(steam_workshop, "resolve_collection", return_value=["1001", "1002"]) as resolve:
+        with _patch_steam([_meta("1001"), _meta("1002")]):
+            response = user_client.post(
+                f"/overlays/{overlay_id}/items",
+                data={"input": "555", "input_mode": "collection"},
+                headers={"X-CSRF-Token": "test-token"},
+            )
+    assert response.status_code == 200
+    resolve.assert_called_once_with("555")
+
+    with session_scope() as session:
+        assert session.query(OverlayWorkshopItem).count() == 2
+
+
+def test_add_non_l4d2_item_returns_400(overlay_for):
+    app, login, user_id, _admin_id, overlay_id = overlay_for
+    user_client = login(user_id)
+
+    def raise_validation(*args, **kwargs):
+        raise steam_workshop.WorkshopValidationError("not L4D2")
+
+    with patch.object(steam_workshop, "fetch_metadata_batch", side_effect=raise_validation):
+        response = user_client.post(
+            f"/overlays/{overlay_id}/items",
+            data={"input": "9999", "input_mode": "items"},
+            headers={"X-CSRF-Token": "test-token"},
+        )
+    assert response.status_code == 400
+    assert b"not L4D2" in response.data
+
+    with session_scope() as session:
+        assert session.query(WorkshopItem).count() == 0
+        assert session.query(OverlayWorkshopItem).count() == 0
+
+
+def test_add_duplicate_item_does_not_500(overlay_for):
+    app, login, user_id, _admin_id, overlay_id = overlay_for
+    user_client = login(user_id)
+
+    with _patch_steam([_meta("1001")]):
+        first = user_client.post(
+            f"/overlays/{overlay_id}/items",
+            data={"input": "1001", "input_mode": "items"},
+            headers={"X-CSRF-Token": "test-token"},
+        )
+        assert first.status_code == 200
+
+    with _patch_steam([_meta("1001")]):
+        second = user_client.post(
+            f"/overlays/{overlay_id}/items",
+            data={"input": "1001", "input_mode": "items"},
+            headers={"X-CSRF-Token": "test-token"},
+        )
+        assert second.status_code == 200
+
+    with session_scope() as session:
+        assert session.query(OverlayWorkshopItem).count() == 1
+
+
+def test_remove_item_drops_association_and_enqueues_rebuild(overlay_for):
+    app, login, user_id, _admin_id, overlay_id = overlay_for
+    user_client = login(user_id)
+
+    with _patch_steam([_meta("1001")]):
+        user_client.post(
+            f"/overlays/{overlay_id}/items",
+            data={"input": "1001", "input_mode": "items"},
+            headers={"X-CSRF-Token": "test-token"},
+        )
+
+    with session_scope() as session:
+        wi = session.query(WorkshopItem).filter_by(steam_id="1001").one()
+        item_id = wi.id
+
+    response = user_client.post(
+        f"/overlays/{overlay_id}/items/{item_id}/delete",
+        headers={"X-CSRF-Token": "test-token"},
+    )
+    assert response.status_code == 200
+
+    with session_scope() as session:
+        assert session.query(OverlayWorkshopItem).count() == 0
+        # WorkshopItem itself remains (cache survives the association removal).
+        assert session.query(WorkshopItem).filter_by(steam_id="1001").one() is not None
+        # Coalesced into the same queued build_overlay job.
+        jobs = session.query(Job).filter_by(operation="build_overlay", overlay_id=overlay_id).all()
+        assert len(jobs) == 1
+
+
+def test_manual_build_button_enqueues_job(overlay_for):
+    app, login, user_id, _admin_id, overlay_id = overlay_for
+    user_client = login(user_id)
+
+    response = user_client.post(
+        f"/overlays/{overlay_id}/build",
+        headers={"X-CSRF-Token": "test-token"},
+    )
+    assert response.status_code == 302
+    assert response.headers["Location"].startswith("/jobs/")
+    with session_scope() as session:
+        jobs = session.query(Job).filter_by(operation="build_overlay", overlay_id=overlay_id).all()
+        assert len(jobs) == 1
+
+
+def test_admin_refresh_enqueues_global_job(env_user):
+    app, login, user_id, admin_id = env_user
+    admin_client = login(admin_id)
+
+    response = admin_client.post(
+        "/admin/workshop/refresh",
+        headers={"X-CSRF-Token": "test-token"},
+    )
+    assert response.status_code == 302
+    assert response.headers["Location"] == "/admin/jobs"
+    with session_scope() as session:
+        jobs = session.query(Job).filter_by(operation="refresh_workshop_items").all()
+        assert len(jobs) == 1
+        assert jobs[0].state == "queued"
+
+
+def test_non_admin_cannot_refresh(env_user):
+    app, login, user_id, _admin_id = env_user
+    user_client = login(user_id)
+
+    response = user_client.post(
+        "/admin/workshop/refresh",
+        headers={"X-CSRF-Token": "test-token"},
+    )
+    assert response.status_code == 403
+
+
+def test_other_user_cannot_modify_workshop_overlay(overlay_for):
+    app, login, user_id, _admin_id, overlay_id = overlay_for
+
+    with session_scope() as session:
+        intruder = User(username="bob", password_digest=hash_password("x"), admin=False)
+        session.add(intruder)
+        session.flush()
+        intruder_id = intruder.id
+
+    intruder_client = login(intruder_id)
+    response = intruder_client.post(
+        f"/overlays/{overlay_id}/items",
+        data={"input": "1001", "input_mode": "items"},
+        headers={"X-CSRF-Token": "test-token"},
+    )
+    assert response.status_code == 403