Refine the host library plan with web-facing API boundaries and rewrite the web app plan around live-linked blueprints, async execution, and hardened logging/state workflows.
903 lines
30 KiB
Markdown
903 lines
30 KiB
Markdown
# L4D2 Web App v1 Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
**Goal:** Build a local Flask web app where users create blueprints and manage L4D2 servers derived from those blueprints, with async lifecycle jobs and live logs.
|
|
|
|
**Architecture:** Run a single Flask process with Jinja templates, vendored HTMX, custom CSS, and in-process worker threads. Persist app state in SQLite with Rails-style foreign-key naming (`user_id`, `server_id`, `blueprint_id`, `overlay_id`, `job_id`). Integrate directly with `l4d2host` write/read APIs: jobs call `install/initialize/start/stop/delete` with output callbacks, while status/logs use `get_instance_status` and `stream_instance_logs`.
|
|
|
|
**Tech Stack:** Python 3.12+, Flask, SQLAlchemy, Alembic, pytest, vendored HTMX, custom CSS, vanilla JS (SSE).
|
|
|
|
---
|
|
|
|
## Scope and Constraints
|
|
|
|
- In scope:
|
|
- public signup/login
|
|
- one-time CLI admin promotion
|
|
- admin-managed overlay catalog
|
|
- user-private blueprints with ordered overlays
|
|
- server creation from blueprint (live-linked, no server overrides)
|
|
- async lifecycle actions (`install`, `initialize`, `start`, `stop`, `delete`)
|
|
- desired-vs-actual state with UI status resolution
|
|
- live command logs (DB + SSE) and live server logs (journald + SSE)
|
|
- Out of scope (later phases):
|
|
- workshop mod management
|
|
- user-created overlays via web file manager
|
|
- shared/global blueprints
|
|
- Blueprint rules (locked):
|
|
- live-linked to servers
|
|
- no per-server overrides for overlays/arguments/config
|
|
- blueprint updates apply on next action
|
|
- delete blueprint blocked if linked servers exist
|
|
- server can change blueprint anytime
|
|
- Frontend rules:
|
|
- no external frontend dependencies
|
|
- HTMX vendored locally only
|
|
- custom CSS only
|
|
- consistent link color `#0F766E`
|
|
- Runtime rules:
|
|
- single-process deployment in v1
|
|
- periodic status refresh every 8 seconds
|
|
|
|
## Planned File Layout
|
|
|
|
- `components/l4d2-web-app/pyproject.toml`
|
|
- `components/l4d2-web-app/src/l4d2web/app.py`
|
|
- `components/l4d2-web-app/src/l4d2web/config.py`
|
|
- `components/l4d2-web-app/src/l4d2web/db.py`
|
|
- `components/l4d2-web-app/src/l4d2web/models.py`
|
|
- `components/l4d2-web-app/src/l4d2web/auth.py`
|
|
- `components/l4d2-web-app/src/l4d2web/cli.py`
|
|
- `components/l4d2-web-app/src/l4d2web/services/l4d2_facade.py`
|
|
- `components/l4d2-web-app/src/l4d2web/services/spec_yaml.py`
|
|
- `components/l4d2-web-app/src/l4d2web/services/job_worker.py`
|
|
- `components/l4d2-web-app/src/l4d2web/services/status.py`
|
|
- `components/l4d2-web-app/src/l4d2web/services/security.py`
|
|
- `components/l4d2-web-app/src/l4d2web/routes/auth_routes.py`
|
|
- `components/l4d2-web-app/src/l4d2web/routes/overlay_routes.py`
|
|
- `components/l4d2-web-app/src/l4d2web/routes/blueprint_routes.py`
|
|
- `components/l4d2-web-app/src/l4d2web/routes/server_routes.py`
|
|
- `components/l4d2-web-app/src/l4d2web/routes/job_routes.py`
|
|
- `components/l4d2-web-app/src/l4d2web/routes/log_routes.py`
|
|
- `components/l4d2-web-app/src/l4d2web/templates/*.html`
|
|
- `components/l4d2-web-app/src/l4d2web/static/vendor/htmx.min.js`
|
|
- `components/l4d2-web-app/src/l4d2web/static/css/{tokens,layout,components,logs}.css`
|
|
- `components/l4d2-web-app/src/l4d2web/static/js/{sse,csrf}.js`
|
|
- `components/l4d2-web-app/alembic.ini`
|
|
- `components/l4d2-web-app/alembic/env.py`
|
|
- `components/l4d2-web-app/alembic/versions/0001_initial.py`
|
|
- `components/l4d2-web-app/tests/*.py`
|
|
- `components/l4d2-web-app/README.md`
|
|
|
|
### Task 1: Scaffold Flask app and base wiring
|
|
|
|
**Files:**
|
|
- Create: `components/l4d2-web-app/pyproject.toml`
|
|
- Create: `components/l4d2-web-app/src/l4d2web/app.py`
|
|
- Create: `components/l4d2-web-app/src/l4d2web/config.py`
|
|
- Test: `components/l4d2-web-app/tests/test_health.py`
|
|
|
|
- [ ] **Step 1: Write failing health test**
|
|
|
|
```python
|
|
from l4d2web.app import create_app
|
|
|
|
|
|
def test_health_endpoint():
|
|
app = create_app({"TESTING": True})
|
|
client = app.test_client()
|
|
r = client.get("/health")
|
|
assert r.status_code == 200
|
|
assert r.get_json() == {"status": "ok"}
|
|
```
|
|
|
|
- [ ] **Step 2: Run test and verify failure**
|
|
|
|
Run: `pytest components/l4d2-web-app/tests/test_health.py -q`
|
|
Expected: FAIL.
|
|
|
|
- [ ] **Step 3: Implement app factory**
|
|
|
|
```python
|
|
from flask import Flask, jsonify
|
|
|
|
|
|
def create_app(test_config=None):
|
|
app = Flask(__name__)
|
|
app.config.from_mapping(
|
|
SECRET_KEY="dev",
|
|
DATABASE_URL="sqlite:///l4d2web.db",
|
|
STATUS_REFRESH_SECONDS=8,
|
|
JOB_WORKER_THREADS=4,
|
|
JOB_LOG_REPLAY_LIMIT=2000,
|
|
JOB_LOG_LINE_MAX_CHARS=4096,
|
|
)
|
|
if test_config:
|
|
app.config.update(test_config)
|
|
|
|
@app.get("/health")
|
|
def health():
|
|
return jsonify({"status": "ok"})
|
|
|
|
return app
|
|
```
|
|
|
|
- [ ] **Step 4: Run test and verify pass**
|
|
|
|
Run: `pytest components/l4d2-web-app/tests/test_health.py -q`
|
|
Expected: PASS.
|
|
|
|
- [ ] **Step 5: Commit scaffold**
|
|
|
|
```bash
|
|
git add components/l4d2-web-app
|
|
git commit -m "feat(l4d2-web): scaffold flask app and health endpoint"
|
|
```
|
|
|
|
### Task 2: Add database models and migration baseline
|
|
|
|
**Files:**
|
|
- Create: `components/l4d2-web-app/src/l4d2web/db.py`
|
|
- Create: `components/l4d2-web-app/src/l4d2web/models.py`
|
|
- Create: `components/l4d2-web-app/alembic.ini`
|
|
- Create: `components/l4d2-web-app/alembic/env.py`
|
|
- Create: `components/l4d2-web-app/alembic/versions/0001_initial.py`
|
|
- Test: `components/l4d2-web-app/tests/test_models.py`
|
|
|
|
- [ ] **Step 1: Write failing model tests**
|
|
|
|
```python
|
|
from l4d2web.db import init_db, session_scope
|
|
from l4d2web.models import User, Blueprint
|
|
|
|
|
|
def test_create_user_and_blueprint(tmp_path, monkeypatch):
|
|
monkeypatch.setenv("DATABASE_URL", f"sqlite:///{tmp_path/'app.db'}")
|
|
init_db()
|
|
with session_scope() as s:
|
|
user = User(username="alice", password_digest="digest", admin=False)
|
|
s.add(user)
|
|
s.flush()
|
|
bp = Blueprint(user_id=user.id, name="default", arguments="[]", config="[]")
|
|
s.add(bp)
|
|
s.flush()
|
|
assert user.id is not None
|
|
assert bp.id is not None
|
|
```
|
|
|
|
- [ ] **Step 2: Run test and verify failure**
|
|
|
|
Run: `pytest components/l4d2-web-app/tests/test_models.py -q`
|
|
Expected: FAIL.
|
|
|
|
- [ ] **Step 3: Implement schema**
|
|
|
|
```python
|
|
class User(Base):
|
|
__tablename__ = "users"
|
|
id = mapped_column(Integer, primary_key=True)
|
|
username = mapped_column(String(64), unique=True, nullable=False)
|
|
password_digest = mapped_column(String(255), nullable=False)
|
|
admin = mapped_column(Boolean, default=False, nullable=False)
|
|
created_at = mapped_column(DateTime, default=datetime.utcnow, nullable=False)
|
|
updated_at = mapped_column(DateTime, default=datetime.utcnow, nullable=False)
|
|
|
|
|
|
class Overlay(Base):
|
|
__tablename__ = "overlays"
|
|
id = mapped_column(Integer, primary_key=True)
|
|
name = mapped_column(String(128), unique=True, nullable=False)
|
|
path = mapped_column(String(512), nullable=False)
|
|
created_at = mapped_column(DateTime, default=datetime.utcnow, nullable=False)
|
|
updated_at = mapped_column(DateTime, default=datetime.utcnow, nullable=False)
|
|
|
|
|
|
class Blueprint(Base):
|
|
__tablename__ = "blueprints"
|
|
id = mapped_column(Integer, primary_key=True)
|
|
user_id = mapped_column(ForeignKey("users.id"), nullable=False)
|
|
name = mapped_column(String(128), nullable=False)
|
|
arguments = mapped_column(Text, nullable=False, default="[]")
|
|
config = mapped_column(Text, nullable=False, default="[]")
|
|
created_at = mapped_column(DateTime, default=datetime.utcnow, nullable=False)
|
|
updated_at = mapped_column(DateTime, default=datetime.utcnow, nullable=False)
|
|
|
|
|
|
class BlueprintOverlay(Base):
|
|
__tablename__ = "blueprint_overlays"
|
|
id = mapped_column(Integer, primary_key=True)
|
|
blueprint_id = mapped_column(ForeignKey("blueprints.id"), nullable=False)
|
|
overlay_id = mapped_column(ForeignKey("overlays.id"), nullable=False)
|
|
position = mapped_column(Integer, nullable=False)
|
|
created_at = mapped_column(DateTime, default=datetime.utcnow, nullable=False)
|
|
updated_at = mapped_column(DateTime, default=datetime.utcnow, nullable=False)
|
|
|
|
|
|
class Server(Base):
|
|
__tablename__ = "servers"
|
|
id = mapped_column(Integer, primary_key=True)
|
|
user_id = mapped_column(ForeignKey("users.id"), nullable=False)
|
|
blueprint_id = mapped_column(ForeignKey("blueprints.id"), nullable=False)
|
|
name = mapped_column(String(128), unique=True, nullable=False)
|
|
port = mapped_column(Integer, nullable=False)
|
|
desired_state = mapped_column(String(16), nullable=False, default="stopped")
|
|
actual_state = mapped_column(String(16), nullable=False, default="unknown")
|
|
actual_state_updated_at = mapped_column(DateTime, nullable=True)
|
|
last_error = mapped_column(Text, nullable=False, default="")
|
|
created_at = mapped_column(DateTime, default=datetime.utcnow, nullable=False)
|
|
updated_at = mapped_column(DateTime, default=datetime.utcnow, nullable=False)
|
|
|
|
|
|
class Job(Base):
|
|
__tablename__ = "jobs"
|
|
id = mapped_column(Integer, primary_key=True)
|
|
user_id = mapped_column(ForeignKey("users.id"), nullable=False)
|
|
server_id = mapped_column(ForeignKey("servers.id"), nullable=True)
|
|
operation = mapped_column(String(32), nullable=False)
|
|
state = mapped_column(String(16), nullable=False, default="queued")
|
|
exit_code = mapped_column(Integer, nullable=True)
|
|
started_at = mapped_column(DateTime, nullable=True)
|
|
finished_at = mapped_column(DateTime, nullable=True)
|
|
created_at = mapped_column(DateTime, default=datetime.utcnow, nullable=False)
|
|
updated_at = mapped_column(DateTime, default=datetime.utcnow, nullable=False)
|
|
|
|
|
|
class JobLog(Base):
|
|
__tablename__ = "job_logs"
|
|
id = mapped_column(Integer, primary_key=True)
|
|
job_id = mapped_column(ForeignKey("jobs.id"), nullable=False)
|
|
seq = mapped_column(Integer, nullable=False)
|
|
stream = mapped_column(String(8), nullable=False)
|
|
line = mapped_column(Text, nullable=False)
|
|
created_at = mapped_column(DateTime, default=datetime.utcnow, nullable=False)
|
|
```
|
|
|
|
- [ ] **Step 4: Run tests and verify pass**
|
|
|
|
Run: `pytest components/l4d2-web-app/tests/test_models.py -q`
|
|
Expected: PASS.
|
|
|
|
- [ ] **Step 5: Commit models and migration**
|
|
|
|
```bash
|
|
git add components/l4d2-web-app/src/l4d2web/{db.py,models.py} components/l4d2-web-app/alembic*
|
|
git commit -m "feat(l4d2-web): add sqlite schema including blueprints and job logs"
|
|
```
|
|
|
|
### Task 3: Implement auth and admin bootstrap
|
|
|
|
**Files:**
|
|
- Create: `components/l4d2-web-app/src/l4d2web/auth.py`
|
|
- Create: `components/l4d2-web-app/src/l4d2web/routes/auth_routes.py`
|
|
- Create: `components/l4d2-web-app/src/l4d2web/cli.py`
|
|
- Modify: `components/l4d2-web-app/src/l4d2web/app.py`
|
|
- Test: `components/l4d2-web-app/tests/test_auth.py`
|
|
|
|
- [ ] **Step 1: Write failing auth tests**
|
|
|
|
```python
|
|
def test_public_signup(client):
|
|
r = client.post("/signup", data={"username": "alice", "password": "secret"})
|
|
assert r.status_code == 302
|
|
|
|
|
|
def test_login_sets_session(client, seed_user):
|
|
r = client.post("/login", data={"username": "alice", "password": "secret"})
|
|
assert r.status_code == 302
|
|
with client.session_transaction() as sess:
|
|
assert sess["user_id"] == seed_user.id
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests and verify failure**
|
|
|
|
Run: `pytest components/l4d2-web-app/tests/test_auth.py -q`
|
|
Expected: FAIL.
|
|
|
|
- [ ] **Step 3: Implement auth and CLI command**
|
|
|
|
```python
|
|
from werkzeug.security import generate_password_hash, check_password_hash
|
|
|
|
|
|
def hash_password(raw: str) -> str:
|
|
return generate_password_hash(raw)
|
|
|
|
|
|
def verify_password(raw: str, digest: str) -> bool:
|
|
return check_password_hash(digest, raw)
|
|
```
|
|
|
|
```python
|
|
@click.command("promote-admin")
|
|
@click.argument("username")
|
|
def promote_admin(username: str):
|
|
with session_scope() as s:
|
|
user = s.scalar(select(User).where(User.username == username))
|
|
if user is None:
|
|
raise click.ClickException("user not found")
|
|
user.admin = True
|
|
s.commit()
|
|
```
|
|
|
|
- [ ] **Step 4: Run tests and verify pass**
|
|
|
|
Run: `pytest components/l4d2-web-app/tests/test_auth.py -q`
|
|
Expected: PASS.
|
|
|
|
- [ ] **Step 5: Commit auth features**
|
|
|
|
```bash
|
|
git add components/l4d2-web-app/src/l4d2web/{auth.py,cli.py,app.py} components/l4d2-web-app/src/l4d2web/routes/auth_routes.py components/l4d2-web-app/tests/test_auth.py
|
|
git commit -m "feat(l4d2-web): add public auth and admin bootstrap command"
|
|
```
|
|
|
|
### Task 4: Implement admin overlay catalog CRUD with path safety
|
|
|
|
**Files:**
|
|
- Create: `components/l4d2-web-app/src/l4d2web/routes/overlay_routes.py`
|
|
- Create: `components/l4d2-web-app/src/l4d2web/services/security.py`
|
|
- Test: `components/l4d2-web-app/tests/test_overlays.py`
|
|
|
|
- [ ] **Step 1: Write failing overlay tests**
|
|
|
|
```python
|
|
def test_admin_can_create_overlay(admin_client):
|
|
r = admin_client.post("/admin/overlays", data={"name": "standard", "path": "/opt/l4d2/overlays/standard"})
|
|
assert r.status_code == 302
|
|
|
|
|
|
def test_overlay_path_must_be_under_root(admin_client):
|
|
r = admin_client.post("/admin/overlays", data={"name": "bad", "path": "/tmp/bad"})
|
|
assert r.status_code == 400
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests and verify failure**
|
|
|
|
Run: `pytest components/l4d2-web-app/tests/test_overlays.py -q`
|
|
Expected: FAIL.
|
|
|
|
- [ ] **Step 3: Implement route and validator**
|
|
|
|
```python
|
|
OVERLAY_ROOT = Path("/opt/l4d2/overlays").resolve()
|
|
|
|
|
|
def validate_overlay_path(raw: str) -> Path:
|
|
p = Path(raw).resolve()
|
|
if OVERLAY_ROOT not in p.parents and p != OVERLAY_ROOT:
|
|
raise ValueError("overlay path must be under /opt/l4d2/overlays")
|
|
return p
|
|
```
|
|
|
|
- [ ] **Step 4: Run tests and verify pass**
|
|
|
|
Run: `pytest components/l4d2-web-app/tests/test_overlays.py -q`
|
|
Expected: PASS.
|
|
|
|
- [ ] **Step 5: Commit overlay CRUD**
|
|
|
|
```bash
|
|
git add components/l4d2-web-app/src/l4d2web/{routes/overlay_routes.py,services/security.py} components/l4d2-web-app/tests/test_overlays.py
|
|
git commit -m "feat(l4d2-web): add admin overlay catalog CRUD with path validation"
|
|
```
|
|
|
|
### Task 5: Implement blueprint CRUD and linkage rules
|
|
|
|
**Files:**
|
|
- Create: `components/l4d2-web-app/src/l4d2web/routes/blueprint_routes.py`
|
|
- Test: `components/l4d2-web-app/tests/test_blueprints.py`
|
|
|
|
- [ ] **Step 1: Write failing blueprint tests**
|
|
|
|
```python
|
|
def test_user_can_create_private_blueprint(user_client, seed_overlays):
|
|
payload = {
|
|
"name": "comp",
|
|
"arguments": ["-tickrate 100"],
|
|
"config": ["sv_consistency 1"],
|
|
"overlay_ids": [1, 2],
|
|
}
|
|
r = user_client.post("/blueprints", json=payload)
|
|
assert r.status_code == 201
|
|
|
|
|
|
def test_delete_blueprint_blocked_when_in_use(user_client, linked_blueprint):
|
|
r = user_client.delete(f"/blueprints/{linked_blueprint.id}")
|
|
assert r.status_code == 409
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests and verify failure**
|
|
|
|
Run: `pytest components/l4d2-web-app/tests/test_blueprints.py -q`
|
|
Expected: FAIL.
|
|
|
|
- [ ] **Step 3: Implement blueprint routes**
|
|
|
|
```python
|
|
@bp.post("/blueprints")
|
|
@require_login
|
|
def create_blueprint():
|
|
payload = request.get_json()
|
|
with session_scope() as s:
|
|
blueprint = Blueprint(
|
|
user_id=current_user().id,
|
|
name=payload["name"],
|
|
arguments=json.dumps(payload.get("arguments", [])),
|
|
config=json.dumps(payload.get("config", [])),
|
|
)
|
|
s.add(blueprint)
|
|
s.flush()
|
|
for position, overlay_id in enumerate(payload.get("overlay_ids", [])):
|
|
s.add(BlueprintOverlay(blueprint_id=blueprint.id, overlay_id=overlay_id, position=position))
|
|
s.commit()
|
|
return {"id": blueprint.id}, 201
|
|
```
|
|
|
|
- [ ] **Step 4: Run tests and verify pass**
|
|
|
|
Run: `pytest components/l4d2-web-app/tests/test_blueprints.py -q`
|
|
Expected: PASS.
|
|
|
|
- [ ] **Step 5: Commit blueprint feature**
|
|
|
|
```bash
|
|
git add components/l4d2-web-app/src/l4d2web/routes/blueprint_routes.py components/l4d2-web-app/tests/test_blueprints.py
|
|
git commit -m "feat(l4d2-web): add private blueprint CRUD with in-use deletion guard"
|
|
```
|
|
|
|
### Task 6: Implement server CRUD from blueprints (no overrides)
|
|
|
|
**Files:**
|
|
- Create: `components/l4d2-web-app/src/l4d2web/routes/server_routes.py`
|
|
- Test: `components/l4d2-web-app/tests/test_servers.py`
|
|
|
|
- [ ] **Step 1: Write failing server tests**
|
|
|
|
```python
|
|
def test_create_server_from_blueprint(user_client, owned_blueprint):
|
|
payload = {"name": "alpha", "port": 27015, "blueprint_id": owned_blueprint.id}
|
|
r = user_client.post("/servers", json=payload)
|
|
assert r.status_code == 201
|
|
|
|
|
|
def test_reassign_blueprint_anytime(user_client, server_and_two_blueprints):
|
|
r = user_client.patch(f"/servers/{server_and_two_blueprints.server_id}", json={"blueprint_id": server_and_two_blueprints.other_blueprint_id})
|
|
assert r.status_code == 200
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests and verify failure**
|
|
|
|
Run: `pytest components/l4d2-web-app/tests/test_servers.py -q`
|
|
Expected: FAIL.
|
|
|
|
- [ ] **Step 3: Implement server routes**
|
|
|
|
```python
|
|
@bp.post("/servers")
|
|
@require_login
|
|
def create_server():
|
|
payload = request.get_json()
|
|
with session_scope() as s:
|
|
server = Server(
|
|
user_id=current_user().id,
|
|
blueprint_id=int(payload["blueprint_id"]),
|
|
name=payload["name"],
|
|
port=int(payload["port"]),
|
|
desired_state="stopped",
|
|
actual_state="unknown",
|
|
last_error="",
|
|
)
|
|
s.add(server)
|
|
s.commit()
|
|
return {"id": server.id}, 201
|
|
```
|
|
|
|
- [ ] **Step 4: Run tests and verify pass**
|
|
|
|
Run: `pytest components/l4d2-web-app/tests/test_servers.py -q`
|
|
Expected: PASS.
|
|
|
|
- [ ] **Step 5: Commit server routes**
|
|
|
|
```bash
|
|
git add components/l4d2-web-app/src/l4d2web/routes/server_routes.py components/l4d2-web-app/tests/test_servers.py
|
|
git commit -m "feat(l4d2-web): add server creation and blueprint reassignment routes"
|
|
```
|
|
|
|
### Task 7: Add direct `l4d2host` facade and blueprint-to-spec generation
|
|
|
|
**Files:**
|
|
- Create: `components/l4d2-web-app/src/l4d2web/services/spec_yaml.py`
|
|
- Create: `components/l4d2-web-app/src/l4d2web/services/l4d2_facade.py`
|
|
- Test: `components/l4d2-web-app/tests/test_l4d2_facade.py`
|
|
|
|
- [ ] **Step 1: Write failing facade tests**
|
|
|
|
```python
|
|
def test_initialize_uses_latest_blueprint_data(monkeypatch, server_with_blueprint):
|
|
called = {}
|
|
|
|
def fake_initialize(name, spec_path, **kwargs):
|
|
called["name"] = name
|
|
called["spec"] = Path(spec_path).read_text()
|
|
|
|
monkeypatch.setattr("l4d2web.services.l4d2_facade.initialize_instance", fake_initialize)
|
|
from l4d2web.services.l4d2_facade import initialize_server
|
|
initialize_server(server_with_blueprint.id)
|
|
assert called["name"] == "alpha"
|
|
assert "sv_consistency" in called["spec"]
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests and verify failure**
|
|
|
|
Run: `pytest components/l4d2-web-app/tests/test_l4d2_facade.py -q`
|
|
Expected: FAIL.
|
|
|
|
- [ ] **Step 3: Implement facade/spec generation**
|
|
|
|
```python
|
|
def build_server_spec_payload(server, blueprint, overlay_names):
|
|
return {
|
|
"port": server.port,
|
|
"overlays": overlay_names,
|
|
"arguments": json.loads(blueprint.arguments),
|
|
"config": json.loads(blueprint.config),
|
|
}
|
|
```
|
|
|
|
```python
|
|
def initialize_server(server_id: int, on_stdout=None, on_stderr=None):
|
|
server, blueprint, overlay_names = load_server_blueprint_bundle(server_id)
|
|
spec_path = write_temp_spec(build_server_spec_payload(server, blueprint, overlay_names))
|
|
initialize_instance(server.name, spec_path, on_stdout=on_stdout, on_stderr=on_stderr)
|
|
```
|
|
|
|
- [ ] **Step 4: Run tests and verify pass**
|
|
|
|
Run: `pytest components/l4d2-web-app/tests/test_l4d2_facade.py -q`
|
|
Expected: PASS.
|
|
|
|
- [ ] **Step 5: Commit facade**
|
|
|
|
```bash
|
|
git add components/l4d2-web-app/src/l4d2web/services/{spec_yaml.py,l4d2_facade.py} components/l4d2-web-app/tests/test_l4d2_facade.py
|
|
git commit -m "feat(l4d2-web): resolve live-linked blueprints to runtime specs via l4d2host"
|
|
```
|
|
|
|
### Task 8: Implement scheduler, lock rules, and startup recovery
|
|
|
|
**Files:**
|
|
- Create: `components/l4d2-web-app/src/l4d2web/services/job_worker.py`
|
|
- Modify: `components/l4d2-web-app/src/l4d2web/app.py`
|
|
- Test: `components/l4d2-web-app/tests/test_job_worker.py`
|
|
|
|
- [ ] **Step 1: Write failing worker tests**
|
|
|
|
```python
|
|
def test_same_server_jobs_serialized(worker_fixture):
|
|
result = worker_fixture.run_once()
|
|
assert result["same_server_parallel"] is False
|
|
|
|
|
|
def test_different_servers_can_run_parallel(worker_fixture):
|
|
result = worker_fixture.run_once()
|
|
assert result["different_servers_parallel"] is True
|
|
|
|
|
|
def test_install_global_exclusive(worker_fixture):
|
|
result = worker_fixture.run_once()
|
|
assert result["install_parallel"] is False
|
|
|
|
|
|
def test_recover_stale_running_jobs(worker_fixture):
|
|
recovered = worker_fixture.recover_stale_jobs()
|
|
assert recovered >= 0
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests and verify failure**
|
|
|
|
Run: `pytest components/l4d2-web-app/tests/test_job_worker.py -q`
|
|
Expected: FAIL.
|
|
|
|
- [ ] **Step 3: Implement scheduler with in-memory guards**
|
|
|
|
```python
|
|
class SchedulerState:
|
|
def __init__(self):
|
|
self.install_running = False
|
|
self.running_servers = set()
|
|
|
|
|
|
def can_start(job, state: SchedulerState) -> bool:
|
|
if job.operation == "install":
|
|
return (not state.install_running) and (len(state.running_servers) == 0)
|
|
if state.install_running:
|
|
return False
|
|
return job.server_id not in state.running_servers
|
|
```
|
|
|
|
- [ ] **Step 4: Implement single-process guard + stale recovery**
|
|
|
|
Run: `pytest components/l4d2-web-app/tests/test_job_worker.py -q`
|
|
Expected: PASS.
|
|
|
|
- [ ] **Step 5: Commit scheduler**
|
|
|
|
```bash
|
|
git add components/l4d2-web-app/src/l4d2web/{services/job_worker.py,app.py} components/l4d2-web-app/tests/test_job_worker.py
|
|
git commit -m "feat(l4d2-web): add async scheduler with lock rules and crash recovery"
|
|
```
|
|
|
|
### Task 9: Persist command logs and add SSE job stream
|
|
|
|
**Files:**
|
|
- Create: `components/l4d2-web-app/src/l4d2web/routes/job_routes.py`
|
|
- Modify: `components/l4d2-web-app/src/l4d2web/services/job_worker.py`
|
|
- Test: `components/l4d2-web-app/tests/test_job_logs.py`
|
|
|
|
- [ ] **Step 1: Write failing job-log tests**
|
|
|
|
```python
|
|
def test_job_logs_seq_monotonic(db_session, completed_job):
|
|
rows = db_session.execute(text("select seq from job_logs where job_id=:id order by seq"), {"id": completed_job.id}).all()
|
|
values = [r[0] for r in rows]
|
|
assert values == sorted(values)
|
|
|
|
|
|
def test_sse_resume_from_last_seq(client, seeded_job_logs):
|
|
r = client.get(f"/jobs/{seeded_job_logs.job_id}/stream?last_seq=5")
|
|
assert r.status_code == 200
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests and verify failure**
|
|
|
|
Run: `pytest components/l4d2-web-app/tests/test_job_logs.py -q`
|
|
Expected: FAIL.
|
|
|
|
- [ ] **Step 3: Implement persistence and SSE route**
|
|
|
|
```python
|
|
def append_job_log(session, job_id: int, stream: str, line: str, max_chars: int = 4096):
|
|
last_seq = session.scalar(select(func.max(JobLog.seq)).where(JobLog.job_id == job_id)) or 0
|
|
session.add(JobLog(job_id=job_id, seq=last_seq + 1, stream=stream, line=line[:max_chars]))
|
|
session.flush()
|
|
```
|
|
|
|
```python
|
|
@bp.get("/jobs/<int:job_id>/stream")
|
|
@require_login
|
|
def stream_job(job_id: int):
|
|
last_seq = int(request.args.get("last_seq", "0"))
|
|
limit = current_app.config["JOB_LOG_REPLAY_LIMIT"]
|
|
|
|
def gen():
|
|
with session_scope() as s:
|
|
rows = s.execute(
|
|
select(JobLog)
|
|
.where(JobLog.job_id == job_id, JobLog.seq > last_seq)
|
|
.order_by(JobLog.seq)
|
|
.limit(limit)
|
|
).scalars()
|
|
for row in rows:
|
|
yield f"id: {row.seq}\n"
|
|
yield f"event: {row.stream}\n"
|
|
yield f"data: {row.line}\n\n"
|
|
|
|
return Response(gen(), mimetype="text/event-stream")
|
|
```
|
|
|
|
- [ ] **Step 4: Run tests and verify pass**
|
|
|
|
Run: `pytest components/l4d2-web-app/tests/test_job_logs.py -q`
|
|
Expected: PASS.
|
|
|
|
- [ ] **Step 5: Commit job log streaming**
|
|
|
|
```bash
|
|
git add components/l4d2-web-app/src/l4d2web/{routes/job_routes.py,services/job_worker.py} components/l4d2-web-app/tests/test_job_logs.py
|
|
git commit -m "feat(l4d2-web): persist command logs and stream them with sse"
|
|
```
|
|
|
|
### Task 10: Add runtime server log stream and status model
|
|
|
|
**Files:**
|
|
- Create: `components/l4d2-web-app/src/l4d2web/routes/log_routes.py`
|
|
- Create: `components/l4d2-web-app/src/l4d2web/services/status.py`
|
|
- Modify: `components/l4d2-web-app/src/l4d2web/services/job_worker.py`
|
|
- Test: `components/l4d2-web-app/tests/test_status_and_server_logs.py`
|
|
|
|
- [ ] **Step 1: Write failing tests**
|
|
|
|
```python
|
|
def test_owner_can_stream_server_logs(owner_client, owned_server):
|
|
r = owner_client.get(f"/servers/{owned_server.id}/logs/stream")
|
|
assert r.status_code == 200
|
|
|
|
|
|
def test_status_precedence():
|
|
from l4d2web.services.status import compute_display_state
|
|
assert compute_display_state("start", "stopped") == "starting"
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests and verify failure**
|
|
|
|
Run: `pytest components/l4d2-web-app/tests/test_status_and_server_logs.py -q`
|
|
Expected: FAIL.
|
|
|
|
- [ ] **Step 3: Implement log stream and status computation**
|
|
|
|
```python
|
|
def compute_display_state(active_operation, actual_state):
|
|
if active_operation == "delete":
|
|
return "deleting"
|
|
if active_operation == "start":
|
|
return "starting"
|
|
if active_operation == "stop":
|
|
return "stopping"
|
|
if active_operation == "initialize":
|
|
return "initializing"
|
|
return actual_state
|
|
|
|
|
|
def has_drift(desired_state, actual_state, has_active_job):
|
|
return (not has_active_job) and desired_state != actual_state
|
|
```
|
|
|
|
```python
|
|
@bp.get("/servers/<int:server_id>/logs/stream")
|
|
@require_login
|
|
def stream_server_logs(server_id: int):
|
|
server = load_authorized_server(server_id)
|
|
|
|
def gen():
|
|
for line in facade.stream_server_logs(server.name, lines=200, follow=True):
|
|
yield f"data: {line}\n\n"
|
|
|
|
return Response(gen(), mimetype="text/event-stream")
|
|
```
|
|
|
|
- [ ] **Step 4: Add actual-state refresh updates**
|
|
|
|
Run: `pytest components/l4d2-web-app/tests/test_status_and_server_logs.py -q`
|
|
Expected: PASS.
|
|
|
|
- [ ] **Step 5: Commit status/log features**
|
|
|
|
```bash
|
|
git add components/l4d2-web-app/src/l4d2web/{routes/log_routes.py,services/status.py,services/job_worker.py} components/l4d2-web-app/tests/test_status_and_server_logs.py
|
|
git commit -m "feat(l4d2-web): add live server logs and desired-vs-actual status model"
|
|
```
|
|
|
|
### Task 11: Apply security and reliability hardening
|
|
|
|
**Files:**
|
|
- Modify: `components/l4d2-web-app/src/l4d2web/app.py`
|
|
- Modify: `components/l4d2-web-app/src/l4d2web/db.py`
|
|
- Modify: `components/l4d2-web-app/src/l4d2web/routes/auth_routes.py`
|
|
- Create: `components/l4d2-web-app/tests/test_security.py`
|
|
|
|
- [ ] **Step 1: Write failing hardening tests**
|
|
|
|
```python
|
|
def test_csrf_required(client, seed_user):
|
|
r = client.post("/servers", data={"name": "x"})
|
|
assert r.status_code == 400
|
|
|
|
|
|
def test_login_rate_limit(client):
|
|
for _ in range(20):
|
|
client.post("/login", data={"username": "x", "password": "y"})
|
|
r = client.post("/login", data={"username": "x", "password": "y"})
|
|
assert r.status_code == 429
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests and verify failure**
|
|
|
|
Run: `pytest components/l4d2-web-app/tests/test_security.py -q`
|
|
Expected: FAIL.
|
|
|
|
- [ ] **Step 3: Implement hardening measures**
|
|
|
|
```python
|
|
app.config.update(
|
|
SESSION_COOKIE_HTTPONLY=True,
|
|
SESSION_COOKIE_SAMESITE="Lax",
|
|
)
|
|
```
|
|
|
|
```python
|
|
engine = create_engine(db_url, connect_args={"check_same_thread": False})
|
|
with engine.connect() as conn:
|
|
conn.exec_driver_sql("PRAGMA journal_mode=WAL;")
|
|
conn.exec_driver_sql("PRAGMA busy_timeout=5000;")
|
|
```
|
|
|
|
- [ ] **Step 4: Run tests and verify pass**
|
|
|
|
Run: `pytest components/l4d2-web-app/tests/test_security.py -q`
|
|
Expected: PASS.
|
|
|
|
- [ ] **Step 5: Commit hardening**
|
|
|
|
```bash
|
|
git add components/l4d2-web-app/src/l4d2web/{app.py,db.py} components/l4d2-web-app/src/l4d2web/routes/auth_routes.py components/l4d2-web-app/tests/test_security.py
|
|
git commit -m "feat(l4d2-web): add csrf, rate limiting, and sqlite reliability settings"
|
|
```
|
|
|
|
### Task 12: Build UI and finalize docs
|
|
|
|
**Files:**
|
|
- Create: `components/l4d2-web-app/src/l4d2web/templates/base.html`
|
|
- Create: `components/l4d2-web-app/src/l4d2web/templates/dashboard.html`
|
|
- Create: `components/l4d2-web-app/src/l4d2web/templates/blueprints.html`
|
|
- Create: `components/l4d2-web-app/src/l4d2web/templates/server_detail.html`
|
|
- Create: `components/l4d2-web-app/src/l4d2web/templates/admin_overlays.html`
|
|
- Create: `components/l4d2-web-app/src/l4d2web/static/vendor/htmx.min.js`
|
|
- Create: `components/l4d2-web-app/src/l4d2web/static/css/tokens.css`
|
|
- Create: `components/l4d2-web-app/src/l4d2web/static/css/layout.css`
|
|
- Create: `components/l4d2-web-app/src/l4d2web/static/css/components.css`
|
|
- Create: `components/l4d2-web-app/src/l4d2web/static/css/logs.css`
|
|
- Create: `components/l4d2-web-app/src/l4d2web/static/js/sse.js`
|
|
- Create: `components/l4d2-web-app/src/l4d2web/static/js/csrf.js`
|
|
- Create: `components/l4d2-web-app/README.md`
|
|
- Test: `components/l4d2-web-app/tests/test_pages.py`
|
|
|
|
- [ ] **Step 1: Write failing page tests**
|
|
|
|
```python
|
|
def test_dashboard_renders_server_and_status(auth_client_with_server):
|
|
r = auth_client_with_server.get("/dashboard")
|
|
text = r.get_data(as_text=True)
|
|
assert r.status_code == 200
|
|
assert "alpha" in text
|
|
|
|
|
|
def test_blueprint_page_private_visibility(user_client, other_users_blueprint):
|
|
r = user_client.get(f"/blueprints/{other_users_blueprint.id}")
|
|
assert r.status_code == 403
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests and verify failure**
|
|
|
|
Run: `pytest components/l4d2-web-app/tests/test_pages.py -q`
|
|
Expected: FAIL.
|
|
|
|
- [ ] **Step 3: Implement templates and style tokens**
|
|
|
|
```css
|
|
:root {
|
|
--color-link: #0F766E;
|
|
--color-bg: #F6FBFA;
|
|
--color-text: #12302B;
|
|
--color-card: #FFFFFF;
|
|
--space-2: 0.5rem;
|
|
--space-4: 1rem;
|
|
--radius: 8px;
|
|
}
|
|
|
|
a {
|
|
color: var(--color-link);
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 4: Run tests and full suite**
|
|
|
|
Run: `pytest components/l4d2-web-app/tests -q`
|
|
Expected: PASS.
|
|
|
|
- [ ] **Step 5: Commit UI and docs**
|
|
|
|
```bash
|
|
git add components/l4d2-web-app
|
|
git commit -m "docs(l4d2-web): finalize blueprint-driven ui and deployment contracts"
|
|
```
|
|
|
|
---
|
|
|
|
## Self-Review
|
|
|
|
- [ ] Spec coverage complete for locked blueprint semantics, auth model, scheduler rules, and logging behavior.
|
|
- [ ] No placeholders (`TODO`, `TBD`, vague directives).
|
|
- [ ] Type/name consistency (`user_id`, `server_id`, `blueprint_id`, `overlay_id`, `job_id`, `password_digest`, `admin`).
|
|
- [ ] Every task includes concrete tests and exact commands.
|