diff --git a/docs/superpowers/specs/2026-05-15-handoff-uv-workspace.md b/docs/superpowers/specs/2026-05-15-handoff-uv-workspace.md new file mode 100644 index 0000000..a4a4579 --- /dev/null +++ b/docs/superpowers/specs/2026-05-15-handoff-uv-workspace.md @@ -0,0 +1,464 @@ +# Handoff — collapse venv chain into uv workspace + `uv sync` + +## Status + +Queued. Independent of the deployment-responsibility reshape +(`2026-05-15-deployment-responsibility-design.md`) which just shipped; +that work left the venv-chain shape intact. This handoff replaces the +chain end-to-end. + +## Goal + +Replace the current five-action venv chain in `bundles/left4me/items.py` +with a single `uv sync --frozen` action driven by a committed +`uv.lock` at the left4me repo root. Eliminate the tempdir-copy dance +in `pip_install` (8 lines of shell working around setuptools writing +`.egg-info/` into a root-owned source tree). + +Net change: 5 actions → 3 actions; deterministic deploys via locked +dep versions; single command in dev and prod; one new build-time +dependency (`uv`) on the host. + +## Why + +Three motivations, listed in priority order. + +**Deterministic prod deploys.** Today's chain installs whatever pip +resolves at apply time. A transitive dep getting a CVE-relevant bump +between two `bw apply` runs is invisible until it breaks something. +`uv sync --frozen` against a committed `uv.lock` makes the installed +version set reproducible from git history alone. + +**Lower cognitive cost in `items.py`.** The `pip_install` action is +the longest, gnarliest action in the bundle — it does its own +tempdir/cleanup-trap/cp-r dance because the obvious `pip install +/opt/left4me/src/...` would write egg-info to a root-owned source +tree. uv's `sdist-then-wheel-from-tarball` build path makes this +problem go away: the source is read-only throughout. + +**Workspace declares what's actually true.** `l4d2web` already imports +from `l4d2host` (5 files use `from l4d2host.paths import ...`). +Today's setup happens to work because both packages get installed +side-by-side via `pip install -e ./l4d2host -e ./l4d2web`, but the +dependency relationship is implicit. A uv workspace makes it explicit +via `[tool.uv.sources] l4d2host = { workspace = true }`. + +## Current state — the 5-action chain + +(All in `~/Projekte/ckn-bw/bundles/left4me/items.py`, ~lines 285-425.) + +``` +git_deploy:/opt/left4me/src + ├── triggers → left4me_pip_install + │ ├── needs ← left4me_create_venv (always-on, gated unless) + │ │ └── triggers → left4me_pip_upgrade + │ └── triggers → left4me_alembic_upgrade + │ ├── triggers → left4me_seed_overlays + │ └── triggers → svc_systemd:left4me-web.service:restart + ├── triggers → left4me_alembic_upgrade (belt-and-braces direct trigger) + └── triggers → left4me_daemon_reload +``` + +`left4me_pip_install` body (the part that simplifies): + +```sh +sudo -u left4me sh -c ' +set -e +tmpdir=$(mktemp -d -t left4me-build-XXXXXX) +trap "rm -rf \"$tmpdir\"" EXIT +cp -r /opt/left4me/src/l4d2host /opt/left4me/src/l4d2web "$tmpdir/" +/var/lib/left4me/.venv/bin/pip install --force-reinstall "$tmpdir/l4d2host" "$tmpdir/l4d2web" +' +``` + +## Target state — uv workspace + single sync action + +Three actions instead of five: + +``` +git_deploy:/opt/left4me/src + ├── triggers → left4me_uv_sync + │ └── triggers → left4me_alembic_upgrade + │ ├── triggers → left4me_seed_overlays + │ └── triggers → svc_systemd:left4me-web.service:restart + ├── triggers → left4me_alembic_upgrade (belt-and-braces) + └── triggers → left4me_daemon_reload +``` + +`left4me_uv_sync` body: + +```python +actions['left4me_uv_sync'] = { + 'command': ( + 'sudo -u left4me ' + 'env UV_PROJECT_ENVIRONMENT=/var/lib/left4me/.venv ' + 'uv sync --frozen --project /opt/left4me/src' + ), + 'triggered': True, + 'cascade_skip': False, + 'needs': [ + 'git_deploy:/opt/left4me/src', + 'pkg_apt:uv', + 'directory:/var/lib/left4me', + 'user:left4me', + ], + 'triggers': [ + 'action:left4me_alembic_upgrade', + ], +} +``` + +`UV_PROJECT_ENVIRONMENT` redirects uv's default venv path (`/.venv`) +to our writable runtime location at `/var/lib/left4me/.venv` (the source +at `/opt/left4me/src` is root-owned, so the default would be a permission +error). + +`--frozen` requires `uv.lock` to be present and consistent with +`pyproject.toml` — refuses to silently update the lockfile during deploy. + +## Empirical spike — do this FIRST + +Before touching anything, verify the architectural assumption that +`uv` actually keeps a root-owned source directory pristine during +build. ~5 minute test on the live host: + +```bash +ssh ckn@left4.me 'sudo apt-get install -y uv' +ssh ckn@left4.me ' + sudo -u left4me sh -c " + wheels=\$(mktemp -d) + uv build --wheel --sdist /opt/left4me/src/l4d2host --out-dir \$wheels + ls \$wheels + sudo git -C /opt/left4me/src status --porcelain + " +' +``` + +Expected: the wheel + sdist exist in the tempdir, AND `git status` +reports the source tree clean (no new `l4d2host.egg-info/` directory). + +If the source stays clean, proceed with the full migration. + +If the source picks up `l4d2host.egg-info/` (uv's build invoked +setuptools.build_meta directly on the source instead of via the sdist +intermediate), fall back to **Medium scope**: keep the tempdir-copy +dance but use `uv pip install` in place of `pip install` (1:1 swap, +no workspace, smaller change). Update this handoff with the fallback +decision. + +## What changes — left4me side + +### New: `/Users/mwiegand/Projekte/left4me/pyproject.toml` + +Workspace root. Short: + +```toml +[project] +name = "left4me" +version = "0.0.0" +description = "Workspace root; packaging lives in the members." +requires-python = ">=3.13" + +[tool.uv.workspace] +members = ["l4d2host", "l4d2web"] + +# Dev-only dependencies (pytest, etc.) for the workspace. +[dependency-groups] +dev = [ + "pytest", +] +``` + +### Modified: `l4d2host/pyproject.toml` and `l4d2web/pyproject.toml` + +No real change to declared deps. `l4d2web` adds the workspace cross-dep: + +```toml +# l4d2web/pyproject.toml +[project] +dependencies = [ + "Flask>=3.0", + "SQLAlchemy>=2.0", + "alembic>=1.13", + "PyYAML>=6.0", + "gunicorn>=22.0", + "requests>=2.31", + "l4d2host", # NEW: declares the import relationship +] + +[tool.uv.sources] +l4d2host = { workspace = true } # NEW: resolves to the in-workspace member +``` + +This makes explicit what's already true: `l4d2web/routes/overlay_routes.py`, +`l4d2web/services/overlay_creation.py`, and three other files import +from `l4d2host.paths`. + +### New: `/Users/mwiegand/Projekte/left4me/uv.lock` + +Generated by `uv lock` at the repo root. Committed to git. Pins every +transitive dep version. + +### Modified: `/Users/mwiegand/Projekte/left4me/.envrc` + +Today: +``` +layout python python3.13 +``` + +New (direnv hands off to uv for venv management): +``` +# direnv's stdlib uv helper creates .venv via `uv sync` and activates it. +# Equivalent to: uv sync && source .venv/bin/activate +use uv +``` + +If `use uv` isn't available in this direnv version (it's a stdlib +function added in direnv 2.34+), fall back to: +``` +uv sync >/dev/null +source .venv/bin/activate +``` + +### Modified: `README.md`, `AGENTS.md`, `l4d2web/README.md` + +Update install instructions from: +``` +pip install -e ./l4d2host -e ./l4d2web pytest +``` +to: +``` +uv sync # creates .venv, installs members editable, installs dev deps +``` + +One-time prereq for developers: install uv (`brew install uv` on +macOS, `apt install uv` on Debian Trixie+, or curl-pipe-sh from +astral.sh for older distros). + +### Modified: `.gitignore` + +Probably no change needed. uv's caches default to `~/.cache/uv` (not +in-repo). The `.venv` is already ignored. + +## What changes — ckn-bw side + +All edits in `~/Projekte/ckn-bw/bundles/left4me/`. + +### `metadata.py` + +Add `uv` to `apt.packages`: +```python +'apt': { + 'packages': { + ... + 'uv': {}, # Required by left4me_uv_sync for production install. + ... + }, +}, +``` + +Drop `python3-pip` if nothing else needs it (uv replaces pip). Keep +`python3-venv` if anything else on the host uses `python3 -m venv`; if +not, drop it too. `python3` and `python3-dev` stay (uv invokes them). + +### `items.py` + +Delete three actions: +- `left4me_create_venv` +- `left4me_pip_upgrade` +- `left4me_pip_install` + +Add one action: `left4me_uv_sync` (body in the "Target state" section +above). + +Update `git_deploy:/opt/left4me/src` triggers: +- Remove: `action:left4me_pip_install` +- Add: `action:left4me_uv_sync` +- Keep: `action:left4me_alembic_upgrade`, `action:left4me_daemon_reload` + +`alembic_upgrade` and `seed_overlays` are unchanged — they invoke the +venv's `alembic` and `flask` binaries by absolute path, which `uv sync` +ensures exist. Update their `needs:` lists to point at +`action:left4me_uv_sync` instead of `action:left4me_pip_install`. + +### `README.md` + +Update the bundle README's deploy-flow description to mention `uv sync` +instead of `pip install -e`, matching the new shape. + +## Migration order + +1. **Spike test** (above): confirm uv preserves source cleanliness. + If fails, retreat to Medium scope. +2. **left4me-side preparation** (independent PR, can land first): + - Add root `pyproject.toml`, declare workspace + - Add `l4d2host` to `l4d2web`'s deps + workspace source + - Run `uv lock`, commit `uv.lock` + - Update `.envrc` + - Update local-dev docs + - Run `uv sync` locally, run `pytest` — all green + - Commit + push +3. **ckn-bw-side install** (depends on step 2): + - Add `pkg_apt: uv` to bundle defaults + - Delete the three old actions, add `uv_sync` + - Update `git_deploy` triggers and downstream `needs:` + - `bw test` clean +4. **First apply to ovh.left4me**: + - Expect: `pkg_apt: uv` installed, three old actions removed from + the graph, new `uv_sync` action fires (because git_deploy fires + with the new commit), runs `uv sync --frozen` against the new + workspace, alembic_upgrade + seed_overlays + web restart cascade. + - The existing `/var/lib/left4me/.venv` (created by + `python3 -m venv`) is structurally a uv-compatible venv; uv + should adopt it without recreation. If uv refuses to adopt + (incompatible metadata), one-shot fix on the host: + ``` + sudo -u left4me rm -rf /var/lib/left4me/.venv + # bw apply will recreate via `uv sync` + ``` +5. **Idempotency check + verification matrix**: + - `bw apply` idempotent (`0 fixed, 0 failed`) + - `pip show l4d2{host,web}` reports the locked version + - Web service active, gameserver round-trip works +6. **Commit ckn-bw side, do not push** (operator pushes manually). + +## What does NOT change + +- **Source ownership**: `/opt/left4me/src` stays `root:root` (the + runtime-state relocation made it so; uv reads it as world-readable). +- **Venv location**: `/var/lib/left4me/.venv` stays where it is, owned + by `left4me`, accessed via `UV_PROJECT_ENVIRONMENT`. +- **Hardening drop-ins, sudoers, sysctl, helpers**: all stable from + the deployment-responsibility migration. uv migration is independent. +- **systemd unit shapes**: reactor-emitted, per-host parameters + unchanged. +- **`alembic_upgrade` and `seed_overlays`**: same shell, same + triggering, same binaries (just from a uv-managed venv). +- **`pkg_apt: python3` and `python3-dev`**: kept (uv shells out to + the system Python interpreter). +- **CI workflows**: no CI currently exists; nothing to update. + +## Out of scope + +- Merging `l4d2host` and `l4d2web` into a single package. They stay + as separate workspace members. +- Switching to a non-direnv-based dev flow. `direnv` + `use uv` stays. +- Migrating other ckn-bw bundles to uv. This is left4me-specific. +- Pinning the host's `uv` version below the apt-current. If lockfile + format issues surface, address as a follow-up (e.g., apt-pin or + switch to astral.sh-installed uv). + +## Risks + +1. **Spike test failure**: uv build isn't actually source-clean → falls + back to Medium scope. Captured above; this is a graceful degradation. +2. **Lockfile format skew**: dev's brew-installed uv (latest) ahead of + prod's apt-installed uv (Trixie's version) → lockfile produced in + dev rejected in prod. Mitigation: stick to features supported by + the apt-installed version; if needed, switch prod to a pinned + astral.sh install. +3. **`alembic` invocation path**: today the action calls + `/var/lib/left4me/.venv/bin/alembic`. After uv sync, this path + should still exist (uv installs the same console_scripts entrypoint + as pip). Verify in step 4. +4. **direnv `use uv` availability**: `use uv` was added to direnv's + stdlib relatively recently. If the dev's direnv is older, use the + fallback `.envrc` snippet (`uv sync >/dev/null && source .venv/bin/activate`). +5. **`--force-reinstall` semantics gone**: today's chain uses + `pip install --force-reinstall` to work around the static + `0.1.0` version in pyproject.toml — without it pip would skip on + no-op. `uv sync --frozen` is version-aware via the lockfile, not + the package version string, so this concern goes away. + +## Verification (end-to-end) + +After ckn-bw apply: + +1. **Source still clean**: + ``` + ssh ckn@left4.me 'sudo git -C /opt/left4me/src status --porcelain' + ``` + Empty output. + +2. **Venv has the workspace members installed**: + ``` + ssh ckn@left4.me 'sudo -u left4me /var/lib/left4me/.venv/bin/python -c "import l4d2host; import l4d2web; print(l4d2host.__file__, l4d2web.__file__)"' + ``` + Both paths point inside `/var/lib/left4me/.venv/lib/python3.13/site-packages/`. + +3. **Pinned versions match the lockfile**: + ``` + ssh ckn@left4.me 'sudo -u left4me /var/lib/left4me/.venv/bin/pip show flask | grep Version' + ``` + Matches the Flask version in `uv.lock`. + +4. **Web service health**: + ``` + ssh ckn@left4.me 'sudo systemctl is-active left4me-web.service' + ``` + `active`. + +5. **Idempotent apply**: + ``` + (cd ~/Projekte/ckn-bw && .venv/bin/bw apply ovh.left4me) + ``` + `0 fixed, 0 failed`. + +6. **Gameserver round-trip**: start a verify instance via + `left4me-systemctl enable verify`, check journal for clean + srcds_run startup behaviour (modulo any missing instance dir), + disable. + +## Pointers + +- Deployment-responsibility design (just shipped; the venv chain it + did NOT touch is what this handoff replaces): + `docs/superpowers/specs/2026-05-15-deployment-responsibility-design.md` +- Runtime state relocation (made `/opt/left4me/src` root-owned, which + is why the current `pip_install` needs the tempdir dance): + `docs/superpowers/specs/2026-05-15-runtime-state-relocation-design.md` +- ckn-bw left4me bundle: + `~/Projekte/ckn-bw/bundles/left4me/` + - `items.py:285-306` — `git_deploy` triggers + - `items.py:328-340` — `left4me_create_venv` + - `items.py:342-352` — `left4me_pip_upgrade` + - `items.py:354-382` — `left4me_pip_install` (the tempdir dance) + - `items.py:384-407` — `left4me_alembic_upgrade` + - `items.py:409-424` — `left4me_seed_overlays` +- uv docs: https://docs.astral.sh/uv/ — workspace, `uv sync`, + `UV_PROJECT_ENVIRONMENT`. + +## Commit messages (suggested) + +left4me side (root pyproject + lockfile + member deps + .envrc + docs): + +``` +refactor(repo): uv workspace + lockfile + +Declare the repo as a uv workspace with l4d2host and l4d2web as +members. Add uv.lock for deterministic dep resolution. l4d2web now +declares its cross-dep on l4d2host explicitly via tool.uv.sources. + +Local-dev install switches from `pip install -e ./l4d2host -e ./l4d2web` +to `uv sync` (creates venv, installs members editable, installs dev +deps from one source). .envrc uses direnv's `use uv` helper. + +Prereq for the ckn-bw bundle uv-sync action (handoff: +docs/superpowers/specs/2026-05-15-handoff-uv-workspace.md). +``` + +ckn-bw side (drop chain, install uv, single sync action): + +``` +refactor(left4me): collapse venv chain into uv sync + +Replace left4me_create_venv + left4me_pip_upgrade + left4me_pip_install +(the tempdir-copy dance) with a single left4me_uv_sync action driven +by left4me's committed uv.lock. Deterministic dep versions, no source +mutation during build, three actions instead of five. + +pkg_apt: uv added. python3-pip removed (uv replaces it). + +Per docs/superpowers/specs/2026-05-15-handoff-uv-workspace.md (in the +left4me repo). +```