From 450f9f1591234a5e5c23528e9c50d8e084eb67e5 Mon Sep 17 00:00:00 2001 From: mwiegand Date: Fri, 15 May 2026 19:48:59 +0200 Subject: [PATCH] deploy/docs+cleanup: describe symlink model; drop stale scripts/ tracked paths deploy/README.md: rewrite intro to reflect that deploy/files/ and deploy/scripts/ are the canonical sources of truth (not examples), with hardening drop-ins explicitly listed; reference fixtures in files/usr/local/lib/systemd/system/ noted as such. spec: add ## Status block marking the deployment-responsibility migration shipped 2026-05-15. Cleanup: remove the old scripts/{libexec,sbin,tests}/ paths that were still tracked after the 2834ad4 move to deploy/scripts/. The content is already present at deploy/scripts/; these entries were a tracking artifact from an incomplete git mv. Co-Authored-By: Claude Sonnet 4.6 --- deploy/README.md | 61 ++--- ...-05-15-deployment-responsibility-design.md | 6 + scripts/libexec/left4me-journalctl | 53 ---- scripts/libexec/left4me-overlay | 244 ------------------ scripts/libexec/left4me-script-sandbox | 81 ------ scripts/libexec/left4me-systemctl | 44 ---- scripts/sbin/left4me | 17 -- scripts/tests/conftest.py | 36 --- scripts/tests/test_helpers_use_fixed_paths.py | 15 -- scripts/tests/test_journalctl_helper.py | 31 --- scripts/tests/test_overlay.py | 32 --- scripts/tests/test_script_sandbox.py | 146 ----------- scripts/tests/test_sudoers_grants.py | 38 --- scripts/tests/test_systemctl_helper.py | 39 --- 14 files changed, 38 insertions(+), 805 deletions(-) delete mode 100755 scripts/libexec/left4me-journalctl delete mode 100644 scripts/libexec/left4me-overlay delete mode 100755 scripts/libexec/left4me-script-sandbox delete mode 100755 scripts/libexec/left4me-systemctl delete mode 100755 scripts/sbin/left4me delete mode 100644 scripts/tests/conftest.py delete mode 100644 scripts/tests/test_helpers_use_fixed_paths.py delete mode 100644 scripts/tests/test_journalctl_helper.py delete mode 100644 scripts/tests/test_overlay.py delete mode 100644 scripts/tests/test_script_sandbox.py delete mode 100644 scripts/tests/test_sudoers_grants.py delete mode 100644 scripts/tests/test_systemctl_helper.py diff --git a/deploy/README.md b/deploy/README.md index e7bc5b1..e027ebb 100644 --- a/deploy/README.md +++ b/deploy/README.md @@ -1,48 +1,51 @@ # left4me deploy — reference exemplar -> **This directory is reference material, not the source of truth.** > The canonical deploy of `ovh.left4me` is driven by > [ckn-bw](https://git.sublimity.de/cronekorkn/ckn-bw)'s `bundles/left4me/` > (attached via `groups/applications/left4me.py`); run `bw apply ovh.left4me` > from the ckn-bw repo to deploy. > -> The privileged scripts the application installs live under -> [`deploy/scripts/libexec/`](scripts/libexec/) and -> [`deploy/scripts/sbin/`](scripts/sbin/) — application code that also -> lives under `deploy/` for layout consistency. ckn-bw creates target-side -> symlinks from `/usr/local/{libexec/left4me,sbin}/` into +> **`deploy/files/` is the canonical source of truth** for static deployment +> artifacts — sudoers, sysctl drop-in, and hardening drop-ins for the +> systemd service units. ckn-bw delivers these via **target-side symlinks** +> from their on-host paths into `/opt/left4me/src/deploy/files/...` (safe +> because `/opt/left4me/src` is root-owned at runtime; the application cannot +> rewrite its own deployment artifacts). +> +> **`deploy/scripts/` is the canonical source of truth** for privileged +> helpers. ckn-bw creates target-side symlinks from +> `/usr/local/{libexec/left4me,sbin}/` into > `/opt/left4me/src/deploy/scripts/{libexec,sbin}/` after `git_deploy`. > -> What remains under `deploy/files/` and `deploy/templates/` is a set of -> readable **examples** — sudoers, sysctl, sandbox-resolv.conf, env -> templates, and a curated subset of the systemd units ckn-bw's reactor -> emits at apply time. They exist so a fresh consumer (other than ckn-bw) -> could read this tree and assemble an equivalent deployment. They are -> **not** the bytes ckn-bw installs; ckn-bw carries its own copies of the -> verbatim configs in `bundles/left4me/files/etc/`, and emits the live -> units from `bundles/left4me/metadata.py`'s `systemd_units` reactor. +> What remains under `deploy/files/usr/local/lib/systemd/system/` is a set +> of **reference fixtures** — a curated subset of the systemd units ckn-bw's +> reactor emits at apply time. They exist so a fresh consumer (other than +> ckn-bw) can read this tree and understand the live unit shape, and so that +> `deploy/tests/test_example_units.py` can assert the reference matches the +> live form. The live base units are emitted by ckn-bw's `systemd/units` +> reactor with per-host CPU pinning and worker counts; the reference files +> must not include hardening directives (those live in the drop-ins, not the +> base units). ## What's here | Path | Role | |---|---| -| `files/etc/sudoers.d/left4me` | Example sudoers grants. Lockdown test: `deploy/scripts/tests/test_sudoers_grants.py`. | -| `files/etc/sysctl.d/99-left4me.conf` | Example sysctl perf baseline (UDP buffers, fq_codel + BBR). | -| `files/etc/left4me/sandbox-resolv.conf` | Example `/etc/resolv.conf` bound into the script-overlay sandbox. | -| `files/usr/local/lib/systemd/system/left4me-web.service` | Example of the web-app unit the reactor emits. | -| `files/usr/local/lib/systemd/system/left4me-server@.service` | Example of the per-instance gameserver unit. | -| `files/usr/local/lib/systemd/system/left4me-workshop-refresh.{service,timer}` | Example of the daily workshop-refresh cron-equivalent. | -| `files/usr/local/lib/systemd/system/l4d2-{game,build}.slice` | Example slice definitions (CPU/IO weights). | +| `files/etc/sudoers.d/left4me` | **Canonical** sudoers grants. Symlinked to `/etc/sudoers.d/left4me`. CI syntax test: `tests/test_sudoers.py`. | +| `files/etc/sysctl.d/99-left4me.conf` | **Canonical** sysctl drop-in (UDP buffers, fq_codel + BBR, `kernel.yama.ptrace_scope=2`). Symlinked to `/etc/sysctl.d/99-left4me.conf`. | +| `files/etc/systemd/system/left4me-web.service.d/10-hardening.conf` | **Canonical** hardening drop-in for `left4me-web.service`. Symlinked to the same on-host path. | +| `files/etc/systemd/system/left4me-server@.service.d/10-hardening.conf` | **Canonical** hardening drop-in for `left4me-server@.service`. Symlinked to the same on-host path. | +| `files/etc/left4me/sandbox-resolv.conf` | Example `/etc/resolv.conf` bound into the script-overlay sandbox (delivered as a bw `files{}` item, not a symlink). | +| `files/usr/local/lib/systemd/system/left4me-web.service` | **Reference fixture** — the web-app unit the reactor emits (per-host worker/thread counts omitted). | +| `files/usr/local/lib/systemd/system/left4me-server@.service` | **Reference fixture** — the per-instance gameserver unit template the reactor emits. | +| `files/usr/local/lib/systemd/system/left4me-workshop-refresh.{service,timer}` | **Reference fixture** — the daily workshop-refresh cron-equivalent. | +| `files/usr/local/lib/systemd/system/l4d2-{game,build}.slice` | **Reference fixture** — slice definitions (CPU/IO weights; reactor fills in `AllowedCPUs=` from host metadata). | +| `scripts/libexec/{left4me-overlay,left4me-systemctl,left4me-journalctl,left4me-script-sandbox}` | **Canonical** privileged helper commands. Symlinked under `/usr/local/libexec/left4me/`. | +| `scripts/sbin/left4me` | **Canonical** admin CLI wrapper. Symlinked to `/usr/local/sbin/left4me`. | | `templates/etc/left4me/host.env` | Example host-library env (deployment-fixed paths). | | `templates/etc/left4me/web.env.template` | Example web-app env. ckn-bw renders the real version via the matching Mako template in `bundles/left4me/files/etc/left4me/web.env.mako`. | -| `tests/test_example_units.py` | Locks down the example units & env templates above. | - -The privileged scripts (`left4me-overlay`, `left4me-script-sandbox`, -`left4me-systemctl`, `left4me-journalctl`, `sbin/left4me`) used to live -under this tree at `files/usr/local/{libexec,sbin}/`; they moved first to -`scripts/{libexec,sbin}/` and then to `deploy/scripts/{libexec,sbin}/` for -layout consistency (everything ckn-bw deploys to the host now lives under -`deploy/`). +| `tests/test_example_units.py` | Locks down the reference units and env templates above; also asserts hardening drop-in shape. | +| `tests/test_sudoers.py` | Runs `visudo -cf` against the sudoers file in CI. | ## Target layout diff --git a/docs/superpowers/specs/2026-05-15-deployment-responsibility-design.md b/docs/superpowers/specs/2026-05-15-deployment-responsibility-design.md index e15fe4e..a307375 100644 --- a/docs/superpowers/specs/2026-05-15-deployment-responsibility-design.md +++ b/docs/superpowers/specs/2026-05-15-deployment-responsibility-design.md @@ -1,5 +1,11 @@ # Deployment responsibility — design +## Status + +**Shipped 2026-05-15.** All five migration steps landed and verified on +ovh.left4me. Implementation plan: +`docs/superpowers/plans/2026-05-15-deployment-responsibility.md`. + ## Context Trace: `2026-05-06-left4me-deployment-design.md` established the original diff --git a/scripts/libexec/left4me-journalctl b/scripts/libexec/left4me-journalctl deleted file mode 100755 index 2e5d3df..0000000 --- a/scripts/libexec/left4me-journalctl +++ /dev/null @@ -1,53 +0,0 @@ -#!/bin/sh -set -eu - -usage() { - printf '%s\n' "usage: left4me-journalctl --lines --follow|--no-follow" >&2 - exit 2 -} - -validate_name() { - name=$1 - [ -n "$name" ] || usage - case "$name" in - .*|*..*|*/*|*\\*) usage ;; - esac - case "$name" in - *[!A-Za-z0-9_.-]*) usage ;; - esac -} - -[ "$#" -eq 4 ] || usage -name=$1 -lines_flag=$2 -lines=$3 -follow_flag=$4 - -validate_name "$name" -[ "$lines_flag" = "--lines" ] || usage -case "$lines" in - ''|*[!0-9]*) usage ;; -esac - -follow_arg= -case "$follow_flag" in - --follow) follow_arg=-f ;; - --no-follow) ;; - *) usage ;; -esac - -unit="left4me-server@${name}.service" -if [ -x /bin/journalctl ]; then - journalctl=/bin/journalctl -elif [ -x /usr/bin/journalctl ]; then - journalctl=/usr/bin/journalctl -else - printf '%s\n' 'journalctl not found at /bin/journalctl or /usr/bin/journalctl' >&2 - exit 69 -fi - -if [ -n "$follow_arg" ]; then - exec "$journalctl" -u "$unit" -n "$lines" -o cat "$follow_arg" -fi - -exec "$journalctl" -u "$unit" -n "$lines" -o cat diff --git a/scripts/libexec/left4me-overlay b/scripts/libexec/left4me-overlay deleted file mode 100644 index 369ed3f..0000000 --- a/scripts/libexec/left4me-overlay +++ /dev/null @@ -1,244 +0,0 @@ -#!/usr/bin/python3 -"""Privileged overlay mount helper for left4me. - -Invoked from the systemd unit's ExecStartPre / ExecStopPost via -`+/usr/bin/nsenter --mount=/proc/1/ns/mnt -- …`. The unit-level -nsenter is what makes this work: it runs the helper Python interpreter -inside PID 1's mount namespace. Without it, the `+` Exec prefix -removes the sandbox/credentials but does NOT detach from the unit's -per-service mount namespace, and the helper process itself would pin -that namespace alive — turning every umount into a multi-second EBUSY -race with the kernel's deferred namespace cleanup. With the unit-level -nsenter the helper has no such reference and umount succeeds first try. - -Validates inputs strictly, then performs `mount -t overlay` / -`umount` directly — no internal nsenter, since the helper is already -running where the syscalls need to take effect. - -Verbs: - mount Reads ${LEFT4ME_ROOT}/instances//instance.env - for L4D2_LOWERDIRS, validates every lowerdir is - under one of installation/overlays/workshop_cache/ - global_overlay_cache, then mounts the kernel - overlay at runtime//merged. - umount Unmounts runtime//merged and cleans up the - kernel-overlayfs `work/work` orphan. - -Set LEFT4ME_OVERLAY_PRINT_ONLY=1 to print the would-be argv (one line, -shell-quoted) and exit 0 instead of execv. Used by tests. -""" - -import os -import re -import shlex -import shutil -import subprocess -import sys -from pathlib import Path - -NAME_RE = re.compile(r"^[a-z0-9][a-z0-9_-]{0,63}$") -DEFAULT_ROOT = "/var/lib/left4me" -LOWERDIR_ALLOWLIST = ( - "installation", - "overlays", - "global_overlay_cache", - "workshop_cache", -) -MAX_LOWERDIRS = 500 -MOUNT_BIN = "/bin/mount" -UMOUNT_BIN = "/bin/umount" - - -def die(msg: str) -> None: - sys.stderr.write(f"left4me-overlay: {msg}\n") - sys.exit(1) - - -def root() -> Path: - return Path(os.environ.get("LEFT4ME_ROOT") or DEFAULT_ROOT) - - -def validate_name(name: str) -> str: - if not NAME_RE.fullmatch(name): - die(f"invalid instance name: {name!r}") - return name - - -def parse_lowerdirs(env_path: Path) -> list[str]: - if not env_path.is_file(): - die(f"instance.env not found: {env_path}") - raw = None - for line in env_path.read_text().splitlines(): - if "=" not in line: - continue - key, value = line.split("=", 1) - if key.strip() == "L4D2_LOWERDIRS": - raw = value - break - if raw is None: - die(f"L4D2_LOWERDIRS not set in {env_path}") - if raw == "": - die(f"L4D2_LOWERDIRS is empty in {env_path}") - parts = raw.split(":") - if any(p == "" for p in parts): - die(f"L4D2_LOWERDIRS contains an empty entry: {raw!r}") - if len(parts) > MAX_LOWERDIRS: - die(f"L4D2_LOWERDIRS has {len(parts)} entries (cap {MAX_LOWERDIRS})") - return parts - - -def canonical_under(allowed_roots: list[Path], path: Path) -> Path: - try: - canonical = path.resolve(strict=True) - except (FileNotFoundError, RuntimeError): - die(f"path does not exist or has a symlink loop: {path}") - for r in allowed_roots: - if canonical == r or r in canonical.parents: - return canonical - die(f"path is outside the permitted roots: {path} (resolved: {canonical})") - - -_LISTXATTR = getattr(os, "listxattr", None) - - -def _entry_has_fuse_xattr(path: str) -> str | None: - if _LISTXATTR is None: - return None - try: - attrs = _LISTXATTR(path, follow_symlinks=False) - except OSError: - return None - for a in attrs: - if a.startswith("user.fuseoverlayfs."): - return a - return None - - -def assert_no_fuse_xattrs(upper: Path) -> None: - if not upper.exists() or _LISTXATTR is None: - return - for dirpath, dirnames, filenames in os.walk(upper): - for entry in (dirpath, *(os.path.join(dirpath, n) for n in dirnames), - *(os.path.join(dirpath, n) for n in filenames)): - tainted = _entry_has_fuse_xattr(entry) - if tainted: - die( - f"upperdir contains fuse-overlayfs xattr {tainted!r} on {entry}; " - "wipe upper/ and work/ before mounting" - ) - - -def _print_argv(argv: list[str]) -> None: - """Emit one shell-quoted argv line to stdout (PRINT_ONLY helper, no exit).""" - print(" ".join(shlex.quote(a) for a in argv)) - - -def exec_or_print(argv: list[str]) -> None: - if os.environ.get("LEFT4ME_OVERLAY_PRINT_ONLY") == "1": - _print_argv(argv) - sys.exit(0) - os.execv(argv[0], argv) - - -def cmd_mount(name: str) -> None: - name = validate_name(name) - r = root() - runtime_name_dir = (r / "runtime" / name).resolve(strict=True) - merged_for_check = (runtime_name_dir / "merged").resolve(strict=True) - - # Idempotency for unit restart cycles: if a previous start mounted - # successfully but ExecStart failed afterwards (and Restart=on-failure - # fires another cycle), the second ExecStartPre would otherwise refuse - # to mount-on-top. Short-circuit here so the second cycle just gets - # straight to ExecStart. PRINT_ONLY (test mode) bypasses this so the - # tests can exercise the full nsenter argv regardless of mount state. - if ( - os.environ.get("LEFT4ME_OVERLAY_PRINT_ONLY") != "1" - and os.path.ismount(merged_for_check) - ): - return - - instance_env = r / "instances" / name / "instance.env" - raw_lowerdirs = parse_lowerdirs(instance_env) - - allowed_roots = [(r / sub).resolve() for sub in LOWERDIR_ALLOWLIST] - canonical_lowerdirs = [str(canonical_under(allowed_roots, Path(p))) for p in raw_lowerdirs] - - upper = (runtime_name_dir / "upper").resolve(strict=True) - work = (runtime_name_dir / "work").resolve(strict=True) - merged = merged_for_check - for label, path in (("upper", upper), ("work", work), ("merged", merged)): - if path.parent != runtime_name_dir: - die(f"{label} resolved outside runtime/{name}: {path}") - - assert_no_fuse_xattrs(upper) - - options = f"lowerdir={':'.join(canonical_lowerdirs)},upperdir={upper},workdir={work}" - argv = [ - MOUNT_BIN, - "-t", "overlay", - "overlay", - "-o", options, - str(merged), - ] - exec_or_print(argv) - - -def cmd_umount(name: str) -> None: - name = validate_name(name) - r = root() - runtime_name_dir = (r / "runtime" / name).resolve(strict=True) - merged_path = runtime_name_dir / "merged" - work_inner = runtime_name_dir / "work" / "work" - - overlay_umount_argv = [ - UMOUNT_BIN, - # Resolve only if it exists; PRINT_ONLY tests always pre-create it. - str(merged_path.resolve(strict=True) if merged_path.exists() else merged_path), - ] - - if os.environ.get("LEFT4ME_OVERLAY_PRINT_ONLY") == "1": - _print_argv(overlay_umount_argv) - sys.exit(0) - - if merged_path.exists(): - merged = merged_path.resolve(strict=True) - if merged.parent != runtime_name_dir: - die(f"merged resolved outside runtime/{name}: {merged}") - # Idempotency: only umount if currently a mount point. Mirrors - # cmd_mount's symmetric check; a redundant cleanup pass — or a - # call after a partial _purge_instance — must be a no-op. - # - # No retry loop here: with the helper running in PID 1's mount - # namespace (via the unit-level `nsenter --mount=/proc/1/ns/mnt` - # in ExecStopPost), it holds no reference to the unit's - # per-service mount namespace, so the cgroup-empty → namespace - # reaped → umount-clears sequence happens without any race - # window for us to ride out. EBUSY here is a real error. - if os.path.ismount(merged): - subprocess.run(overlay_umount_argv, check=True) - - # Kernel-overlayfs creates work_inner during mount with root:root mode - # 0/0. After unmount it's an orphan that the unit's User= (left4me) - # cannot traverse via shutil.rmtree, so reset/delete in instances.py - # blows up with EACCES on `runtime//work/work`. The helper is - # the only code path with root that knows about this directory, so - # the cleanup belongs here. Safe to nuke — the kernel re-creates it - # on the next mount. Run unconditionally — covers both "we just - # unmounted" and "previous teardown didn't finish" cases. - if work_inner.exists(): - shutil.rmtree(work_inner) - - -def main(argv: list[str]) -> None: - if len(argv) != 3 or argv[1] not in ("mount", "umount"): - sys.stderr.write("usage: left4me-overlay mount|umount \n") - sys.exit(2) - if argv[1] == "mount": - cmd_mount(argv[2]) - else: - cmd_umount(argv[2]) - - -if __name__ == "__main__": - main(sys.argv) diff --git a/scripts/libexec/left4me-script-sandbox b/scripts/libexec/left4me-script-sandbox deleted file mode 100755 index 748d260..0000000 --- a/scripts/libexec/left4me-script-sandbox +++ /dev/null @@ -1,81 +0,0 @@ -#!/bin/bash -# Privileged sandbox launcher for left4me script overlays. -# -# Invoked via sudo by the web user with two arguments: -# numeric overlay id; bind-mounts /var/lib/left4me/overlays/ -# read-write at /overlay inside the sandbox. -# absolute path to a bash file already written by the web app; -# bind-mounted read-only at /script.sh inside the sandbox. -# -# The script runs as a transient systemd .service with the full hardening -# surface: cgroup limits + walltime kill, NoNewPrivileges, ProtectSystem, -# ProtectHome, kernel-tunable / -module / -log protection, namespace -# restriction, address-family restriction, capability bounding (empty), -# seccomp filter (@system-service @network-io), MemoryDenyWriteExecute, -# LockPersonality, RestrictSUIDSGID. Network namespace is *not* restricted — -# scripts must reach the public internet to download workshop / l4d2center -# / cedapug content. PID namespace is shared with the host (no -# PrivatePID= directive in systemd); host PIDs are visible via /proc. -# Same-uid attack surface (the sandbox runs as left4me, so do the -# gameservers and the web app) is covered by the hardening profile plus -# system-wide kernel.yama.ptrace_scope=2 — see -# docs/superpowers/specs/2026-05-15-hardening-threat-model.md. -set -euo pipefail - -# Self-wrap into PID 1's mount namespace before doing anything mount-related. -# The web app's left4me-web.service has PrivateTmp=true, which gives it a -# private mount namespace. When the worker invokes us via sudo, we inherit -# that namespace; our `mount --bind` would land there. systemd-run below -# spawns transient units in PID 1's namespace (where they don't see the -# private bind), so the sandbox would bind onto an empty staging dir and -# permission-deny on every write. The sentinel env var avoids an exec loop. -if [[ "${L4D2_SANDBOX_IN_PID1_MNT_NS:-}" != "1" ]]; then - exec env L4D2_SANDBOX_IN_PID1_MNT_NS=1 \ - /usr/bin/nsenter --mount=/proc/1/ns/mnt -- "$0" "$@" -fi - -[[ $# -eq 2 ]] || { echo "usage: $0