cronekorkn/left4me
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
left4me/docs/superpowers/plans/2026-05-06-left4me-deployment.md

1136 lines
38 KiB
Markdown
Raw Blame History

# Left4me Deployment Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Add a production-like test deployment path that installs the current working tree on a Linux host as `left4me`, with configurable runtime state, root-owned systemd units, and constrained sudo helpers for game-server control.
**Architecture:** Runtime state is rooted at `LEFT4ME_ROOT`, defaulting to `/var/lib/left4me`, while deployed code lives directly in `/opt/left4me` with its virtualenv at `/opt/left4me/.venv`. The web app runs as a global system service, and each game server runs as a root-owned global template unit `left4me-server@.service` under `/usr/local/lib/systemd/system/`. `l4d2ctl` manages server units through allowlisted helper commands invoked with `sudo -n`, not by writing user units.
**Tech Stack:** Python 3.12+, Typer, Flask, SQLAlchemy, pytest, POSIX shell, systemd, sudo, SteamCMD, fuse-overlayfs.
---
> **Approval gate:** This plan may be written and refined without further approval. Do not implement code changes from this plan until the user explicitly approves implementation.
## Source Design
- `docs/superpowers/specs/2026-05-06-left4me-deployment-design.md`
## File Map
- `l4d2host/paths.py`: central `LEFT4ME_ROOT` and safe overlay-ref resolution.
- `l4d2host/instances.py`: derive installation, overlays, instances, and runtime directories from `LEFT4ME_ROOT`; stop installing user units.
- `l4d2host/steam_install.py`: install SteamCMD app content into `${LEFT4ME_ROOT}/installation` by default.
- `l4d2host/service_control.py`: call constrained sudo helpers for start, stop, status, and logs.
- `l4d2host/status.py`: parse status from `left4me-systemctl show` helper output.
- `l4d2host/logs.py`: stream logs through `left4me-journalctl` helper.
- `l4d2host/pyproject.toml`: remove the user-unit template package data if the template file is removed.
- `l4d2host/tests/test_paths.py`: cover root config and overlay-ref validation.
- `l4d2host/tests/test_initialize.py`: cover initialized paths under the configured root.
- `l4d2host/tests/test_lifecycle.py`: cover helper-based start/stop/delete commands.
- `l4d2host/tests/test_status.py`: cover helper-based status parsing.
- `l4d2host/tests/test_logs.py`: cover helper-based log streaming command construction.
- `l4d2web/config.py`: read deployment config from environment.
- `l4d2web/cli.py`: add non-public user/admin bootstrap command.
- `l4d2web/services/security.py`: validate overlay refs rather than absolute overlay paths.
- `l4d2web/routes/overlay_routes.py`: store validated overlay refs in the existing `Overlay.path` column.
- `l4d2web/services/l4d2_facade.py`: emit overlay refs from `Overlay.path` into generated host specs.
- `l4d2web/pyproject.toml`: include `gunicorn` for the systemd web service.
- `l4d2web/tests/test_config.py`: cover environment-derived config.
- `l4d2web/tests/test_auth.py`: cover user/admin bootstrap CLI.
- `l4d2web/tests/test_overlays.py`: cover safe overlay refs.
- `l4d2web/tests/test_l4d2_facade.py`: cover generated specs using overlay refs.
- `deploy/files/usr/local/lib/systemd/system/left4me-web.service`: global web systemd service.
- `deploy/files/usr/local/lib/systemd/system/left4me-server@.service`: global per-server systemd template.
- `deploy/files/usr/local/libexec/left4me/left4me-systemctl`: root-run helper for validated service actions.
- `deploy/files/usr/local/libexec/left4me/left4me-journalctl`: root-run helper for validated journal reads.
- `deploy/files/etc/sudoers.d/left4me`: allowlist for the helper commands.
- `deploy/templates/etc/left4me/host.env`: default host env file.
- `deploy/templates/etc/left4me/web.env.template`: web env template without committed secrets.
- `deploy/deploy-test-server.sh`: idempotent SSH deployment script for a sudo-capable deploy user.
- `deploy/tests/test_deploy_artifacts.py`: static checks for deployment artifacts.
- `deploy/README.md`: operator documentation for the test deployment.
- `README.md`, `l4d2host/README.md`, `l4d2web/README.md`, `AGENTS.md`: document the deployment contract changes.
## Locked Deployment Layout
Remote runtime user:
```text
user: left4me
home: /var/lib/left4me
shell: /usr/sbin/nologin
```
Remote paths:
```text
/etc/left4me/host.env
/etc/left4me/web.env
/opt/left4me/.venv
/opt/left4me/<repository contents>
/var/lib/left4me/left4me.db
/var/lib/left4me/installation
/var/lib/left4me/overlays
/var/lib/left4me/instances
/var/lib/left4me/runtime
/var/lib/left4me/tmp
/usr/local/lib/systemd/system/left4me-web.service
/usr/local/lib/systemd/system/left4me-server@.service
/usr/local/libexec/left4me/left4me-systemctl
/usr/local/libexec/left4me/left4me-journalctl
/etc/sudoers.d/left4me
```
Overlay refs are relative refs under `${LEFT4ME_ROOT}/overlays`. Valid examples are `standard`, `competitive/base`, and `users/42/custom`. Absolute paths, `..`, empty components, and symlink escapes are rejected.
### Task 1: Add Configurable Host Root And Overlay Refs
**Files:**
- Create: `l4d2host/paths.py`
- Modify: `l4d2host/instances.py`
- Modify: `l4d2host/steam_install.py`
- Create: `l4d2host/tests/test_paths.py`
- Modify: `l4d2host/tests/test_initialize.py`
- Modify: `l4d2host/tests/test_install.py`
- [ ] **Step 1: Write failing host path tests**
Create `l4d2host/tests/test_paths.py`:
```python
from pathlib import Path
import pytest
from l4d2host.paths import get_left4me_root, overlay_path, validate_overlay_ref
def test_left4me_root_defaults_to_var_lib(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.delenv("LEFT4ME_ROOT", raising=False)
assert get_left4me_root() == Path("/var/lib/left4me")
def test_left4me_root_can_be_set_by_env(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("LEFT4ME_ROOT", "/srv/left4me")
assert get_left4me_root() == Path("/srv/left4me")
@pytest.mark.parametrize("ref", ["standard", "competitive/base", "users/42/custom"])
def test_overlay_ref_accepts_safe_relative_refs(ref: str) -> None:
assert validate_overlay_ref(ref) == ref
@pytest.mark.parametrize("ref", ["", "/tmp/bad", "../bad", "bad/../evil", "bad//evil", " bad"])
def test_overlay_ref_rejects_unsafe_refs(ref: str) -> None:
with pytest.raises(ValueError):
validate_overlay_ref(ref)
def test_overlay_path_resolves_under_root(tmp_path: Path) -> None:
assert overlay_path("standard", root=tmp_path) == tmp_path / "overlays" / "standard"
```
- [ ] **Step 2: Run path tests and verify failure**
Run: `pytest l4d2host/tests/test_paths.py -q`
Expected: FAIL because `l4d2host.paths` does not exist.
- [ ] **Step 3: Implement `l4d2host.paths`**
Create `l4d2host/paths.py`:
```python
from pathlib import Path
import os
DEFAULT_LEFT4ME_ROOT = Path("/var/lib/left4me")
def get_left4me_root() -> Path:
raw = os.environ.get("LEFT4ME_ROOT", str(DEFAULT_LEFT4ME_ROOT)).strip()
if not raw:
raise ValueError("LEFT4ME_ROOT must not be empty")
return Path(raw)
def validate_overlay_ref(raw: str) -> str:
ref = raw.strip()
if not ref:
raise ValueError("overlay ref must not be empty")
path = Path(ref)
if path.is_absolute():
raise ValueError("overlay ref must be relative")
if any(part in {"", ".", ".."} for part in path.parts):
raise ValueError("overlay ref must not contain empty, current, or parent components")
if ref != path.as_posix():
raise ValueError("overlay ref must use forward-slash path separators")
return ref
def overlay_path(ref: str, *, root: Path | None = None) -> Path:
base = root if root is not None else get_left4me_root()
safe_ref = validate_overlay_ref(ref)
candidate = (base / "overlays" / safe_ref).resolve(strict=False)
overlay_root = (base / "overlays").resolve(strict=False)
if candidate != overlay_root and overlay_root not in candidate.parents:
raise ValueError("overlay ref escapes overlay root")
return candidate
```
- [ ] **Step 4: Update host instance and install defaults**
Modify `l4d2host/instances.py` so functions take `root: Path | None = None`, resolve `root = get_left4me_root()` when omitted, and build lowerdirs with `overlay_path`:
```python
from l4d2host.paths import get_left4me_root, overlay_path
def _root_or_default(root: Path | None) -> Path:
return root if root is not None else get_left4me_root()
```
Use `_root_or_default(root)` at the top of `initialize_instance`, `start_instance`, `stop_instance`, and `delete_instance`. In `initialize_instance`, replace direct overlay path construction with:
```python
lowerdirs = [str(overlay_path(overlay, root=root)) for overlay in spec.overlays]
lowerdirs.append(str(root / "installation"))
```
Modify `l4d2host/steam_install.py` so `SteamInstaller()` defaults to `${LEFT4ME_ROOT}/installation`:
```python
from l4d2host.paths import get_left4me_root
class SteamInstaller:
def __init__(self, install_dir: Path | None = None, steamcmd: str = "steamcmd"):
self.install_dir = install_dir if install_dir is not None else get_left4me_root() / "installation"
self.steamcmd = steamcmd
```
- [ ] **Step 5: Update initialize and install tests**
In `l4d2host/tests/test_initialize.py`, add:
```python
def test_initialize_uses_configured_left4me_root(tmp_path: Path, monkeypatch) -> None:
monkeypatch.setenv("LEFT4ME_ROOT", str(tmp_path))
spec = tmp_path / "spec.yaml"
spec.write_text("port: 27015\noverlays: [standard]\n")
initialize_instance("alpha", spec)
env = (tmp_path / "instances/alpha/instance.env").read_text()
assert f"L4D2_LOWERDIRS={tmp_path}/overlays/standard:{tmp_path}/installation" in env
```
In `l4d2host/tests/test_install.py`, add:
```python
def test_default_install_dir_uses_left4me_root(tmp_path, monkeypatch):
monkeypatch.setenv("LEFT4ME_ROOT", str(tmp_path))
calls = []
monkeypatch.setattr("l4d2host.steam_install.run_command", lambda cmd, **kwargs: calls.append(cmd))
SteamInstaller().install_or_update()
assert str(tmp_path / "installation") in calls[0]
```
- [ ] **Step 6: Run host path tests and verify pass**
Run: `pytest l4d2host/tests/test_paths.py l4d2host/tests/test_initialize.py l4d2host/tests/test_install.py -q`
Expected: PASS.
- [ ] **Step 7: Commit configurable host root**
```bash
git add l4d2host/paths.py l4d2host/instances.py l4d2host/steam_install.py l4d2host/tests/test_paths.py l4d2host/tests/test_initialize.py l4d2host/tests/test_install.py
git commit -m "feat(l4d2): configure host state root"
```
### Task 2: Switch Host Control To Global Unit Helpers
**Files:**
- Create: `l4d2host/service_control.py`
- Modify: `l4d2host/instances.py`
- Modify: `l4d2host/status.py`
- Modify: `l4d2host/logs.py`
- Modify: `l4d2host/pyproject.toml`
- Delete: `l4d2host/systemd_user.py`
- Delete: `l4d2host/templates/l4d2@.service`
- Modify: `l4d2host/tests/test_lifecycle.py`
- Modify: `l4d2host/tests/test_status.py`
- Modify: `l4d2host/tests/test_logs.py`
- [ ] **Step 1: Write failing service-control tests**
Create or extend `l4d2host/tests/test_status.py` with:
```python
from l4d2host.status import get_instance_status
def test_status_uses_left4me_systemctl_helper(monkeypatch) -> None:
calls = []
def fake_run_command(cmd, **kwargs):
del kwargs
calls.append(cmd)
return type("Result", (), {"stdout": "ActiveState=active\nSubState=running\n"})()
monkeypatch.setattr("l4d2host.service_control.run_command", fake_run_command)
status = get_instance_status("alpha")
assert calls == [["sudo", "-n", "/usr/local/libexec/left4me/left4me-systemctl", "show", "alpha"]]
assert status.state == "running"
```
Extend `l4d2host/tests/test_logs.py` with:
```python
def test_logs_use_left4me_journalctl_helper(monkeypatch) -> None:
calls = []
def fake_stream(cmd):
calls.append(cmd)
return iter(["one"])
monkeypatch.setattr("l4d2host.service_control.stream_command", fake_stream)
from l4d2host.logs import stream_instance_logs
assert list(stream_instance_logs("alpha", lines=25, follow=False)) == ["one"]
assert calls == [["sudo", "-n", "/usr/local/libexec/left4me/left4me-journalctl", "alpha", "--lines", "25", "--no-follow"]]
```
Update `l4d2host/tests/test_lifecycle.py` start/stop expectations to use `left4me-systemctl` helper instead of `systemctl --user`.
- [ ] **Step 2: Run service-control tests and verify failure**
Run: `pytest l4d2host/tests/test_lifecycle.py l4d2host/tests/test_status.py l4d2host/tests/test_logs.py -q`
Expected: FAIL because `l4d2host.service_control` does not exist and lifecycle code still uses user units.
- [ ] **Step 3: Implement service-control module**
Create `l4d2host/service_control.py`:
```python
import subprocess
from typing import Iterator
from l4d2host.process import run_command
SYSTEMCTL_HELPER = "/usr/local/libexec/left4me/left4me-systemctl"
JOURNALCTL_HELPER = "/usr/local/libexec/left4me/left4me-journalctl"
def systemctl_command(action: str, name: str) -> list[str]:
return ["sudo", "-n", SYSTEMCTL_HELPER, action, name]
def journalctl_command(name: str, *, lines: int = 200, follow: bool = True) -> list[str]:
return [
"sudo",
"-n",
JOURNALCTL_HELPER,
name,
"--lines",
str(lines),
"--follow" if follow else "--no-follow",
]
def start_service(name: str, **kwargs) -> None:
run_command(systemctl_command("start", name), **kwargs)
def stop_service(name: str, **kwargs) -> None:
run_command(systemctl_command("stop", name), **kwargs)
def show_service(name: str):
return run_command(systemctl_command("show", name))
def stream_command(cmd: list[str]) -> Iterator[str]:
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, bufsize=1)
try:
if proc.stdout is None:
return
for raw in iter(proc.stdout.readline, ""):
yield raw.rstrip("\n")
finally:
if proc.poll() is None:
proc.terminate()
proc.wait(timeout=2)
def stream_journal(name: str, *, lines: int = 200, follow: bool = True) -> Iterator[str]:
return stream_command(journalctl_command(name, lines=lines, follow=follow))
```
- [ ] **Step 4: Update host lifecycle/status/log implementations**
In `l4d2host/instances.py`, remove `ensure_template_unit` and `daemon_reload` imports and calls. Replace service operations:
```python
from l4d2host.service_control import start_service, stop_service
```
Use:
```python
start_service(name, on_stdout=on_stdout, on_stderr=on_stderr, passthrough=passthrough, should_cancel=should_cancel)
```
and:
```python
stop_service(name, on_stdout=on_stdout, on_stderr=on_stderr, passthrough=passthrough, should_cancel=should_cancel)
```
In `delete_instance`, call `stop_service(...)` before unmount/removal when instance or runtime exists.
In `l4d2host/status.py`, replace direct `systemctl --user show` with:
```python
from l4d2host.service_control import show_service
```
and call `result = show_service(name)`.
In `l4d2host/logs.py`, replace direct `journalctl --user` with:
```python
from l4d2host.service_control import stream_journal
def stream_instance_logs(name: str, *, lines: int = 200, follow: bool = True) -> Iterator[str]:
return stream_journal(name, lines=lines, follow=follow)
```
Delete `l4d2host/systemd_user.py` and `l4d2host/templates/l4d2@.service`. In `l4d2host/pyproject.toml`, remove `"l4d2host.templates"` from `packages` and remove the `tool.setuptools.package-data` entry for templates.
- [ ] **Step 5: Run host service-control tests and verify pass**
Run: `pytest l4d2host/tests/test_lifecycle.py l4d2host/tests/test_status.py l4d2host/tests/test_logs.py -q`
Expected: PASS.
- [ ] **Step 6: Commit global helper service boundary**
```bash
git add l4d2host/service_control.py l4d2host/instances.py l4d2host/status.py l4d2host/logs.py l4d2host/pyproject.toml l4d2host/tests/test_lifecycle.py l4d2host/tests/test_status.py l4d2host/tests/test_logs.py
git rm l4d2host/systemd_user.py l4d2host/templates/l4d2@.service
git commit -m "feat(l4d2): manage servers through global unit helpers"
```
### Task 3: Add Deployment Artifacts
**Files:**
- Create: `deploy/files/usr/local/lib/systemd/system/left4me-web.service`
- Create: `deploy/files/usr/local/lib/systemd/system/left4me-server@.service`
- Create: `deploy/files/usr/local/libexec/left4me/left4me-systemctl`
- Create: `deploy/files/usr/local/libexec/left4me/left4me-journalctl`
- Create: `deploy/files/etc/sudoers.d/left4me`
- Create: `deploy/templates/etc/left4me/host.env`
- Create: `deploy/templates/etc/left4me/web.env.template`
- Create: `deploy/tests/test_deploy_artifacts.py`
- [ ] **Step 1: Write failing deployment artifact tests**
Create `deploy/tests/test_deploy_artifacts.py`:
```python
from pathlib import Path
import subprocess
ROOT = Path(__file__).resolve().parents[2]
def read(path: str) -> str:
return (ROOT / path).read_text()
def test_global_unit_names_are_product_level() -> None:
assert (ROOT / "deploy/files/usr/local/lib/systemd/system/left4me-web.service").exists()
assert (ROOT / "deploy/files/usr/local/lib/systemd/system/left4me-server@.service").exists()
def test_web_unit_uses_left4me_paths_and_env_files() -> None:
body = read("deploy/files/usr/local/lib/systemd/system/left4me-web.service")
assert "User=left4me" in body
assert "WorkingDirectory=/opt/left4me" in body
assert "EnvironmentFile=/etc/left4me/host.env" in body
assert "EnvironmentFile=/etc/left4me/web.env" in body
assert "ExecStart=/opt/left4me/.venv/bin/gunicorn" in body
def test_server_unit_runs_as_left4me_and_has_hardening() -> None:
body = read("deploy/files/usr/local/lib/systemd/system/left4me-server@.service")
assert "User=left4me" in body
assert "EnvironmentFile=/var/lib/left4me/instances/%i/instance.env" in body
assert "WorkingDirectory=/var/lib/left4me/runtime/%i/merged/left4dead2" in body
assert "NoNewPrivileges=true" in body
assert "ProtectSystem=strict" in body
assert "ReadWritePaths=/var/lib/left4me/runtime/%i" in body
def test_helper_scripts_reject_bad_arguments_without_systemctl() -> None:
for helper in [
ROOT / "deploy/files/usr/local/libexec/left4me/left4me-systemctl",
ROOT / "deploy/files/usr/local/libexec/left4me/left4me-journalctl",
]:
subprocess.run(["sh", "-n", str(helper)], check=True)
result = subprocess.run(["sh", str(helper), "bad/action", "../evil"], capture_output=True, text=True)
assert result.returncode != 0
def test_sudoers_allows_only_left4me_helpers() -> None:
body = read("deploy/files/etc/sudoers.d/left4me")
assert "/usr/local/libexec/left4me/left4me-systemctl *" in body
assert "/usr/local/libexec/left4me/left4me-journalctl *" in body
assert "/bin/systemctl" not in body
assert "/bin/journalctl" not in body
```
- [ ] **Step 2: Run deployment artifact tests and verify failure**
Run: `pytest deploy/tests/test_deploy_artifacts.py -q`
Expected: FAIL because deployment artifact files do not exist.
- [ ] **Step 3: Add web service unit**
Create `deploy/files/usr/local/lib/systemd/system/left4me-web.service`:
```ini
[Unit]
Description=left4me web application
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
User=left4me
Group=left4me
WorkingDirectory=/opt/left4me
Environment=HOME=/var/lib/left4me
EnvironmentFile=/etc/left4me/host.env
EnvironmentFile=/etc/left4me/web.env
ExecStart=/opt/left4me/.venv/bin/gunicorn --workers 1 --threads 8 --bind 0.0.0.0:8000 'l4d2web.app:create_app()'
Restart=on-failure
RestartSec=3
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=full
ReadWritePaths=/var/lib/left4me
[Install]
WantedBy=multi-user.target
```
- [ ] **Step 4: Add server service unit**
Create `deploy/files/usr/local/lib/systemd/system/left4me-server@.service`:
```ini
[Unit]
Description=left4me server instance %i
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
User=left4me
Group=left4me
EnvironmentFile=/etc/left4me/host.env
EnvironmentFile=/var/lib/left4me/instances/%i/instance.env
WorkingDirectory=/var/lib/left4me/runtime/%i/merged/left4dead2
ExecStart=/var/lib/left4me/installation/srcds_run -game left4dead2 +hostport ${L4D2_PORT} ${L4D2_ARGS}
Restart=on-failure
RestartSec=5
NoNewPrivileges=true
PrivateTmp=true
PrivateDevices=true
ProtectHome=true
ProtectSystem=strict
ReadOnlyPaths=/var/lib/left4me/installation /var/lib/left4me/overlays
ReadWritePaths=/var/lib/left4me/runtime/%i
RestrictSUIDSGID=true
LockPersonality=true
[Install]
WantedBy=multi-user.target
```
- [ ] **Step 5: Add sudo helper scripts**
Create `deploy/files/usr/local/libexec/left4me/left4me-systemctl`:
```sh
#!/bin/sh
set -eu
usage() {
printf '%s\n' 'usage: left4me-systemctl start|stop|show <server-name>' >&2
exit 64
}
validate_name() {
name=$1
case "$name" in
''|.*|*..*|*/*|*\\*|*[!A-Za-z0-9_.-]*) usage ;;
esac
}
[ "$#" -eq 2 ] || usage
action=$1
name=$2
validate_name "$name"
unit="left4me-server@${name}.service"
case "$action" in
start) exec /bin/systemctl start "$unit" ;;
stop) exec /bin/systemctl stop "$unit" ;;
show) exec /bin/systemctl show "$unit" --property=ActiveState --property=SubState --no-pager ;;
*) usage ;;
esac
```
Create `deploy/files/usr/local/libexec/left4me/left4me-journalctl`:
```sh
#!/bin/sh
set -eu
usage() {
printf '%s\n' 'usage: left4me-journalctl <server-name> --lines <n> --follow|--no-follow' >&2
exit 64
}
validate_name() {
name=$1
case "$name" in
''|.*|*..*|*/*|*\\*|*[!A-Za-z0-9_.-]*) usage ;;
esac
}
[ "$#" -eq 4 ] || usage
name=$1
lines_flag=$2
lines=$3
follow_flag=$4
validate_name "$name"
[ "$lines_flag" = '--lines' ] || usage
case "$lines" in ''|*[!0-9]*) usage ;; esac
unit="left4me-server@${name}.service"
case "$follow_flag" in
--follow) exec /bin/journalctl -u "$unit" -n "$lines" -o cat -f ;;
--no-follow) exec /bin/journalctl -u "$unit" -n "$lines" -o cat ;;
*) usage ;;
esac
```
- [ ] **Step 6: Add sudoers and env templates**
Create `deploy/files/etc/sudoers.d/left4me`:
```sudoers
Defaults:left4me !requiretty
left4me ALL=(root) NOPASSWD: /usr/local/libexec/left4me/left4me-systemctl *
left4me ALL=(root) NOPASSWD: /usr/local/libexec/left4me/left4me-journalctl *
```
Create `deploy/templates/etc/left4me/host.env`:
```sh
LEFT4ME_ROOT=/var/lib/left4me
```
Create `deploy/templates/etc/left4me/web.env.template`:
```sh
DATABASE_URL=sqlite:////var/lib/left4me/left4me.db
SECRET_KEY=replace-with-generated-secret
JOB_WORKER_THREADS=4
```
- [ ] **Step 7: Run deployment artifact tests and verify pass**
Run: `pytest deploy/tests/test_deploy_artifacts.py -q`
Expected: PASS.
- [ ] **Step 8: Commit deployment artifacts**
```bash
git add deploy/files deploy/templates deploy/tests/test_deploy_artifacts.py
git commit -m "feat(deploy): add left4me systemd and sudo artifacts"
```
### Task 4: Add Web Deployment Config And Bootstrap CLI
**Files:**
- Modify: `l4d2web/config.py`
- Modify: `l4d2web/cli.py`
- Modify: `l4d2web/services/security.py`
- Modify: `l4d2web/routes/overlay_routes.py`
- Modify: `l4d2web/services/l4d2_facade.py`
- Modify: `l4d2web/pyproject.toml`
- Create: `l4d2web/tests/test_config.py`
- Modify: `l4d2web/tests/test_auth.py`
- Modify: `l4d2web/tests/test_overlays.py`
- Modify: `l4d2web/tests/test_l4d2_facade.py`
- [ ] **Step 1: Write failing web config and bootstrap tests**
Create `l4d2web/tests/test_config.py`:
```python
from l4d2web.config import load_config
def test_load_config_uses_environment(monkeypatch) -> None:
monkeypatch.setenv("DATABASE_URL", "sqlite:////var/lib/left4me/left4me.db")
monkeypatch.setenv("SECRET_KEY", "secret-from-env")
monkeypatch.setenv("JOB_WORKER_THREADS", "2")
config = load_config()
assert config["DATABASE_URL"] == "sqlite:////var/lib/left4me/left4me.db"
assert config["SECRET_KEY"] == "secret-from-env"
assert config["JOB_WORKER_THREADS"] == 2
```
Extend `l4d2web/tests/test_auth.py`:
```python
def test_create_user_cli_creates_admin(app, monkeypatch):
monkeypatch.setenv("LEFT4ME_ADMIN_PASSWORD", "secret")
result = app.test_cli_runner().invoke(args=["create-user", "admin", "--admin"])
assert result.exit_code == 0
assert "created user admin" in result.output
```
Extend `l4d2web/tests/test_overlays.py`:
```python
def test_overlay_ref_must_be_relative(admin_client):
r = admin_client.post("/overlays", data={"name": "bad", "path": "/tmp/bad"})
assert r.status_code == 400
def test_admin_can_create_overlay_with_relative_ref(admin_client):
r = admin_client.post("/overlays", data={"name": "standard", "path": "standard"})
assert r.status_code == 302
```
Update `l4d2web/tests/test_l4d2_facade.py` seed data so `Overlay.path` is a relative ref, and assert generated specs contain the path ref rather than the display name.
- [ ] **Step 2: Run web config tests and verify failure**
Run: `pytest l4d2web/tests/test_config.py l4d2web/tests/test_auth.py l4d2web/tests/test_overlays.py l4d2web/tests/test_l4d2_facade.py -q`
Expected: FAIL because `load_config`, `create-user`, and overlay-ref validation are not implemented.
- [ ] **Step 3: Implement environment config**
Modify `l4d2web/config.py`:
```python
import os
def load_config() -> dict[str, object]:
return {
"SECRET_KEY": os.getenv("SECRET_KEY", "dev"),
"DATABASE_URL": os.getenv("DATABASE_URL", "sqlite:///l4d2web.db"),
"STATUS_REFRESH_SECONDS": int(os.getenv("STATUS_REFRESH_SECONDS", "8")),
"JOB_WORKER_THREADS": int(os.getenv("JOB_WORKER_THREADS", "4")),
"JOB_WORKER_ENABLED": os.getenv("JOB_WORKER_ENABLED", "true").lower() not in {"0", "false", "no"},
"JOB_WORKER_POLL_SECONDS": float(os.getenv("JOB_WORKER_POLL_SECONDS", "1")),
"JOB_LOG_REPLAY_LIMIT": int(os.getenv("JOB_LOG_REPLAY_LIMIT", "2000")),
"JOB_LOG_LINE_MAX_CHARS": int(os.getenv("JOB_LOG_LINE_MAX_CHARS", "4096")),
}
DEFAULT_CONFIG = load_config()
```
Modify `l4d2web/app.py` to call `load_config()` inside `create_app()` instead of reusing import-time `DEFAULT_CONFIG`:
```python
from l4d2web.config import load_config
app.config.from_mapping(load_config())
```
- [ ] **Step 4: Implement create-user CLI**
Modify `l4d2web/cli.py`:
```python
import os
from l4d2web.auth import hash_password
@click.command("create-user")
@click.argument("username")
@click.option("--admin", is_flag=True, default=False)
def create_user(username: str, admin: bool) -> None:
password = os.environ.get("LEFT4ME_ADMIN_PASSWORD")
if password is None:
password = click.prompt("Password", hide_input=True, confirmation_prompt=True)
with session_scope() as db:
existing = db.scalar(select(User).where(User.username == username))
if existing is not None:
raise click.ClickException("user already exists")
db.add(User(username=username, password_digest=hash_password(password), admin=admin))
click.echo(f"created user {username}")
```
Register it in `register_cli(app)`:
```python
app.cli.add_command(create_user)
```
- [ ] **Step 5: Implement web overlay refs**
Modify `l4d2web/services/security.py`:
```python
from pathlib import Path
def validate_overlay_ref(raw: str) -> str:
ref = raw.strip()
if not ref:
raise ValueError("overlay ref must not be empty")
path = Path(ref)
if path.is_absolute():
raise ValueError("overlay ref must be relative")
if any(part in {"", ".", ".."} for part in path.parts):
raise ValueError("overlay ref must not contain empty, current, or parent components")
if ref != path.as_posix():
raise ValueError("overlay ref must use forward-slash path separators")
return ref
```
Modify `l4d2web/routes/overlay_routes.py` to import `validate_overlay_ref` and store the returned ref in `Overlay.path`:
```python
from l4d2web.services.security import validate_overlay_ref
validated_ref = validate_overlay_ref(raw_path)
db.add(Overlay(name=name, path=validated_ref))
```
Apply the same assignment in `update_overlay`.
Modify `l4d2web/services/l4d2_facade.py` so `load_server_blueprint_bundle` selects `Overlay.path` instead of `Overlay.name`:
```python
select(Overlay.path)
```
- [ ] **Step 6: Add gunicorn dependency**
Modify `l4d2web/pyproject.toml` dependencies:
```toml
dependencies = [
"Flask>=3.0",
"SQLAlchemy>=2.0",
"alembic>=1.13",
"PyYAML>=6.0",
"gunicorn>=22.0",
]
```
- [ ] **Step 7: Run web config/bootstrap tests and verify pass**
Run: `pytest l4d2web/tests/test_config.py l4d2web/tests/test_auth.py l4d2web/tests/test_overlays.py l4d2web/tests/test_l4d2_facade.py -q`
Expected: PASS.
- [ ] **Step 8: Commit web deployment config**
```bash
git add l4d2web/config.py l4d2web/app.py l4d2web/cli.py l4d2web/services/security.py l4d2web/routes/overlay_routes.py l4d2web/services/l4d2_facade.py l4d2web/pyproject.toml l4d2web/tests/test_config.py l4d2web/tests/test_auth.py l4d2web/tests/test_overlays.py l4d2web/tests/test_l4d2_facade.py
git commit -m "feat(l4d2-web): add deployment config and bootstrap cli"
```
### Task 5: Add Test-Server Deployment Script
**Files:**
- Create: `deploy/deploy-test-server.sh`
- Modify: `deploy/tests/test_deploy_artifacts.py`
- [ ] **Step 1: Write failing deployment script tests**
Extend `deploy/tests/test_deploy_artifacts.py`:
```python
def test_deploy_script_has_safe_defaults_and_preserves_state() -> None:
script = read("deploy/deploy-test-server.sh")
assert "useradd --system --home-dir /var/lib/left4me" in script
assert "/var/lib/left4me/installation" in script
assert "/var/lib/left4me/overlays" in script
assert "/var/lib/left4me/instances" in script
assert "/var/lib/left4me/runtime" in script
assert "tar" in script
assert "pip install -e /opt/left4me/l4d2host -e /opt/left4me/l4d2web" in script
assert "systemctl enable --now left4me-web.service" in script
def test_deploy_script_shell_syntax() -> None:
subprocess.run(["sh", "-n", str(ROOT / "deploy/deploy-test-server.sh")], check=True)
```
- [ ] **Step 2: Run deployment script tests and verify failure**
Run: `pytest deploy/tests/test_deploy_artifacts.py -q`
Expected: FAIL because `deploy/deploy-test-server.sh` does not exist.
- [ ] **Step 3: Implement test deployment script**
Create `deploy/deploy-test-server.sh`:
```sh
#!/bin/sh
set -eu
usage() {
printf '%s\n' 'usage: deploy/deploy-test-server.sh <ssh-user@host>' >&2
exit 64
}
[ "$#" -eq 1 ] || usage
target=$1
repo_root=$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)
archive=$(mktemp -t left4me-deploy.XXXXXX.tar.gz)
remote_archive=/tmp/left4me-deploy.tar.gz
cleanup() {
rm -f "$archive"
}
trap cleanup EXIT INT TERM
tar \
--exclude='.git' \
--exclude='__pycache__' \
--exclude='.pytest_cache' \
--exclude='*.egg-info' \
--exclude='l4d2web.db' \
--exclude='l4d2web.db-shm' \
--exclude='l4d2web.db-wal' \
-C "$repo_root" \
-czf "$archive" \
.
scp "$archive" "$target:$remote_archive"
ssh "$target" 'set -eu
if ! id left4me >/dev/null 2>&1; then
sudo useradd --system --home-dir /var/lib/left4me --create-home --shell /usr/sbin/nologin left4me
fi
if command -v apt-get >/dev/null 2>&1; then
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y python3 python3-venv python3-pip curl ca-certificates tar gzip fuse-overlayfs fuse3 sudo
elif command -v dnf >/dev/null 2>&1; then
sudo dnf install -y python3 python3-pip curl ca-certificates tar gzip fuse-overlayfs fuse3 sudo
else
printf "%s\n" "unsupported package manager: install python3, venv tooling, fuse-overlayfs, fuse3, sudo, curl, tar, gzip" >&2
exit 1
fi
sudo mkdir -p /etc/left4me /opt/left4me /usr/local/lib/systemd/system /usr/local/libexec/left4me /var/lib/left4me/installation /var/lib/left4me/overlays /var/lib/left4me/instances /var/lib/left4me/runtime /var/lib/left4me/tmp
sudo chown -R left4me:left4me /var/lib/left4me /opt/left4me
sudo chmod 0750 /var/lib/left4me
sudo find /opt/left4me -mindepth 1 -maxdepth 1 ! -name .venv -exec rm -rf {} +
sudo tar -xzf /tmp/left4me-deploy.tar.gz -C /opt/left4me
sudo chown -R left4me:left4me /opt/left4me
sudo install -m 0644 /opt/left4me/deploy/files/usr/local/lib/systemd/system/left4me-web.service /usr/local/lib/systemd/system/left4me-web.service
sudo install -m 0644 /opt/left4me/deploy/files/usr/local/lib/systemd/system/left4me-server@.service /usr/local/lib/systemd/system/left4me-server@.service
sudo install -m 0755 /opt/left4me/deploy/files/usr/local/libexec/left4me/left4me-systemctl /usr/local/libexec/left4me/left4me-systemctl
sudo install -m 0755 /opt/left4me/deploy/files/usr/local/libexec/left4me/left4me-journalctl /usr/local/libexec/left4me/left4me-journalctl
sudo install -m 0440 /opt/left4me/deploy/files/etc/sudoers.d/left4me /etc/sudoers.d/left4me
sudo visudo -cf /etc/sudoers.d/left4me
sudo install -m 0644 /opt/left4me/deploy/templates/etc/left4me/host.env /etc/left4me/host.env
if ! sudo test -f /etc/left4me/web.env; then
secret=$(python3 -c "import secrets; print(secrets.token_hex(32))")
printf "DATABASE_URL=sqlite:////var/lib/left4me/left4me.db\nSECRET_KEY=%s\nJOB_WORKER_THREADS=4\n" "$secret" | sudo tee /etc/left4me/web.env >/dev/null
sudo chmod 0640 /etc/left4me/web.env
sudo chown root:left4me /etc/left4me/web.env
fi
sudo -u left4me python3 -m venv /opt/left4me/.venv
sudo -u left4me /opt/left4me/.venv/bin/python -m pip install --upgrade pip
sudo -u left4me /opt/left4me/.venv/bin/python -m pip install -e /opt/left4me/l4d2host -e /opt/left4me/l4d2web
sudo -u left4me sh -c "set -a; . /etc/left4me/host.env; . /etc/left4me/web.env; set +a; /opt/left4me/.venv/bin/python -c '"'"'from l4d2web.app import create_app; create_app()'"'"'"
if [ -n "${LEFT4ME_ADMIN_USERNAME:-}" ] && [ -n "${LEFT4ME_ADMIN_PASSWORD:-}" ]; then
sudo -u left4me sh -c "set -a; . /etc/left4me/host.env; . /etc/left4me/web.env; set +a; LEFT4ME_ADMIN_PASSWORD=\"$LEFT4ME_ADMIN_PASSWORD\" /opt/left4me/.venv/bin/flask --app '"'"'l4d2web.app:create_app'"'"' create-user \"$LEFT4ME_ADMIN_USERNAME\" --admin || true"
fi
sudo systemctl daemon-reload
sudo systemctl enable --now left4me-web.service
sudo systemctl restart left4me-web.service
curl -fsS http://127.0.0.1:8000/health
'
```
- [ ] **Step 4: Run deployment script tests and verify pass**
Run: `pytest deploy/tests/test_deploy_artifacts.py -q`
Expected: PASS.
- [ ] **Step 5: Commit deployment script**
```bash
git add deploy/deploy-test-server.sh deploy/tests/test_deploy_artifacts.py
git commit -m "feat(deploy): add ssh test deployment script"
```
### Task 6: Document Deployment Contract And Verify
**Files:**
- Create: `deploy/README.md`
- Modify: `README.md`
- Modify: `l4d2host/README.md`
- Modify: `l4d2web/README.md`
- Modify: `AGENTS.md`
- [ ] **Step 1: Write deployment README**
Create `deploy/README.md`:
```markdown
# left4me Deployment
This directory contains the production-like test deployment for a Linux server.
## Target Layout
- `/etc/left4me/host.env`
- `/etc/left4me/web.env`
- `/opt/left4me/.venv`
- `/opt/left4me/<repository contents>`
- `/var/lib/left4me/left4me.db`
- `/var/lib/left4me/installation`
- `/var/lib/left4me/overlays`
- `/var/lib/left4me/instances`
- `/var/lib/left4me/runtime`
- `/var/lib/left4me/tmp`
- `/usr/local/lib/systemd/system/left4me-web.service`
- `/usr/local/lib/systemd/system/left4me-server@.service`
- `/usr/local/libexec/left4me/left4me-systemctl`
- `/usr/local/libexec/left4me/left4me-journalctl`
- `/etc/sudoers.d/left4me`
## Runtime User
The deployment creates a system user named `left4me` with home directory `/var/lib/left4me` and shell `/usr/sbin/nologin`.
## Running A Test Deployment
Run from the repository root:
```bash
deploy/deploy-test-server.sh deploy-user@example-host
```
The SSH user must be able to run `sudo`. The script deploys the current local working tree, installs or updates the virtualenv, preserves `/var/lib/left4me`, installs systemd units and sudo helpers, and restarts `left4me-web.service`.
To bootstrap an admin user during deployment:
```bash
LEFT4ME_ADMIN_USERNAME=admin LEFT4ME_ADMIN_PASSWORD='change-me' deploy/deploy-test-server.sh deploy-user@example-host
```
## Overlay Refs
Overlay refs are relative paths under `${LEFT4ME_ROOT}/overlays`, for example `standard`, `competitive/base`, and `users/42/custom`. Absolute paths, `..`, empty components, and symlink escapes are rejected.
```
- [ ] **Step 2: Update component docs**
Update `README.md` with a link to `deploy/README.md` and state that production-like test deployment uses `/var/lib/left4me`, `/opt/left4me`, `/etc/left4me`, and global units under `/usr/local/lib/systemd/system`.
Update `l4d2host/README.md` to document:
```text
LEFT4ME_ROOT defaults to /var/lib/left4me.
l4d2ctl start/stop/status/logs use sudo -n /usr/local/libexec/left4me/left4me-systemctl and left4me-journalctl.
Deployment or config management owns left4me-server@.service.
```
Update `l4d2web/README.md` to document:
```text
DATABASE_URL, SECRET_KEY, and JOB_WORKER_THREADS are read from environment.
The systemd deployment loads /etc/left4me/host.env and /etc/left4me/web.env.
Use flask create-user <username> --admin with LEFT4ME_ADMIN_PASSWORD for bootstrap.
```
Update `AGENTS.md` so the host path rule references `LEFT4ME_ROOT` instead of hard-coded `/opt/l4d2`, and so global unit/helper ownership is recorded.
- [ ] **Step 3: Run focused verification**
Run:
```bash
pytest l4d2host/tests -q
pytest l4d2web/tests -q
pytest deploy/tests -q
```
Expected: PASS for all three commands.
- [ ] **Step 4: Run code index refresh**
Run: `ccc index`
Expected: command exits 0, or reports that `ccc` is unavailable. If `ccc` is unavailable, report that the index refresh was skipped because the command is missing.
- [ ] **Step 5: Commit docs and verification updates**
```bash
git add deploy/README.md README.md l4d2host/README.md l4d2web/README.md AGENTS.md
git commit -m "docs(deploy): document left4me deployment contract"
```
## Self-Review
- Spec coverage: the plan covers `left4me` user setup, `/var/lib/left4me` home/state, `/opt/left4me` source/venv, `/etc/left4me` env files, `LEFT4ME_ROOT`, global units under `/usr/local/lib/systemd/system`, product-level server unit/helper names, constrained sudo helpers, relative overlay refs, deployment from local working tree, admin bootstrap, docs, and verification.
- Placeholder scan: no task uses unresolved placeholders, unspecified error handling, or references to undefined helper names.
- Type consistency: host root is consistently `LEFT4ME_ROOT`; server systemd unit is consistently `left4me-server@.service`; helpers are consistently `left4me-systemctl` and `left4me-journalctl`; web overlay refs are stored in existing `Overlay.path` and emitted to host specs from that column.
- Scope: the plan does not execute deployment on a real server; it adds local deploy assets and code changes needed before a separate approved deployment run.
Reference in a new issue View git blame Copy permalink