left4me/docs/superpowers/plans/2026-05-15-uv-workspace-execution.md
mwiegand 49992b3a26
refactor(repo): uv workspace + hatchling + layout restructure
Migrate from pip-install-e + setuptools to a uv workspace with a
committed uv.lock for deterministic deps. Switch both members to
hatchling, and move package sources into nested standard layout
(l4d2host/l4d2host/, l4d2web/l4d2web/) so builds work from a
read-only source tree — setuptools wrote egg-info to source under
the old layout, which broke uv sync on the root-owned /opt/left4me/src.

Local dev install: `pip install -e ./l4d2host -e ./l4d2web` -> `uv sync`.
.envrc switches from `layout python python3.13` to `use uv`. Python
pinned to 3.13 via .python-version.

l4d2web now declares its cross-dep on l4d2host explicitly via
[tool.uv.sources] (workspace = true). l4d2web/alembic.ini and
l4d2web/alembic/ stay at the project root (standard alembic layout).

Test fixes:
- tests/__init__.py added to both test dirs so pytest doesn't shadow
  l4d2host as a namespace package via outer-dir walk.
- 3 CWD-relative paths in tests (l4d2web/static/css/{tokens,layout}.css
  and js/sse.js) anchored to Path(__file__) so they survive layout
  changes.
- Two test_install.py tests now monkeypatch HOME to tmp_path so they
  stop silently mutating ~/.steam/sdk32 on every run.

628 tests pass under sandboxed `uv run pytest`.

Per docs/superpowers/plans/2026-05-15-uv-workspace-execution.md;
prereq for the ckn-bw bundle's uv-sync action (queued).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-15 22:04:29 +02:00

22 KiB
Raw Blame History

Plan — collapse left4me venv chain into uv workspace + uv sync

Status: executed (left4me side). ckn-bw side queued — see ~/Projekte/ckn-bw/bundles/left4me/ and the matching section below.

Notable deviations from the original handoff (docs/superpowers/specs/2026-05-15-handoff-uv-workspace.md):

  • Handoff assumed pkg_apt: uv works on Debian Trixie. It does not — uv is in experimental/sid only. Replaced with a left4me_install_uv action that downloads a pinned 0.11.8 tarball from astral-sh/uv releases, SHA256-verifies, installs to /usr/local/bin/.
  • Handoff assumed the existing layout (l4d2host/pyproject.toml with package-dir = ".") was workspace-compatible. It was not — setuptools writes egg-info/ to source during any build, which fails on the root-owned /opt/left4me/src tree. Required layout restructure to l4d2host/l4d2host/ (package source nested) plus a switch from setuptools to hatchling.
  • git is not installed on the prod host (bw drives git from the control machine). Verification check #1 uses find for build artifacts instead of git status.

Context

The production deploy of left4me to ovh.left4me currently uses a 5-action chain in ckn-bw/bundles/left4me/items.py that builds out a Python venv under /var/lib/left4me/.venv by chaining python3 -m venvpip upgradepip install (with an 8-line tempdir-copy dance because the source at /opt/left4me/src is root-owned and setuptools wants to write .egg-info/ into it) → alembic upgradeseed_overlays. The chain has three problems:

  1. Non-deterministic prod deploys. pip install resolves whatever is latest at apply time. A transitive CVE-relevant bump between two bw apply runs is invisible until something breaks.
  2. Cognitive cost. The tempdir-copy in left4me_pip_install is the single longest, gnarliest action in the bundle.
  3. Implicit cross-package dep. l4d2web imports from l4d2host.paths in 5 files but doesn't declare the dependency — today's setup works only because both get pip install -e'd side-by-side.

This plan migrates the repo to a uv workspace with a committed uv.lock, replacing the 5-action chain with left4me_install_uv (download + SHA256 verify, idempotent — only re-runs on version change) plus left4me_uv_sync. On the steady-state path (uv already pinned at 0.11.8), only uv_sync fires per deploy. Both sides of the change (left4me repo and the ckn-bw left4me bundle) ship together. The plan executes the migration sequence already documented in /Users/mwiegand/Projekte/left4me/docs/superpowers/specs/2026-05-15-handoff-uv-workspace.md — treat that handoff as the design document. This plan adds the empirically-verified ground truth, resolves the small open questions, and encodes the executable sequence.

Source of truth

  • Design: docs/superpowers/specs/2026-05-15-handoff-uv-workspace.md (in the left4me repo) — read this first; do not duplicate its content here.
  • Sibling context (don't dive in): docs/superpowers/specs/2026-05-15-deployment-responsibility-design.md (just-shipped; left the venv chain alone), docs/superpowers/specs/2026-05-15-runtime-state-relocation-design.md (made /opt/left4me/src root-owned, which is why the current tempdir-copy dance exists).

Resolved questions (from planning)

  • Branch flow: direct-to-master on both sides. (Matches left4me's recent workflow, e.g. b13d164, 55b0138. ckn-bw side committed but NOT pushed — operator pushes manually.)
  • Python version alignment: align all three pyprojects (root + both members) to requires-python = ">=3.13". Matches .envrc and the production host. Removes the workspace-vs-member skew.
  • Spike test scope: extend beyond the handoff to also dry-run a uv sync --frozen shape against a root-owned source — the production command path is sync, not build, and they're different code paths.
  • Scope handoff at git push: agent's deliverable is two ready-to-deploy commits (left4me pushed; ckn-bw committed but unpushed). The user runs bw apply ovh.left4me, the post-apply restart, and the 6-check verification matrix themselves. (Per session memory: feedback_left4me_deploy_workflow — supersedes the original prompt's ask to drive apply + verify end-to-end.) The spike test remains agent work — it's information gathering, and the one-shot direct install fits the "one-shot via direct command" rule from the same memory.
  • uv install vector: direct GitHub tarball download + SHA256 verify against the official .sha256 sibling, install to /usr/local/bin/. The handoff doc's pkg_apt: uv assumption was wrong — uv is not in Debian Trixie's apt archive (in experimental/sid only). Astral's canonical methods are curl-pipe-sh and direct tarball; we chose tarball for auditability and pattern-match with the existing left4me_install_steamcmd action. Pin to uv 0.11.8 to match the local brew-installed version, eliminating the lockfile-format-skew risk between dev and prod.

Ground-truth from exploration

  • Cross-package imports confirmed: 5 files in l4d2web/ import from l4d2host.paths:
    • l4d2web/routes/overlay_routes.py
    • l4d2web/services/overlay_creation.py
    • l4d2web/services/overlay_builders.py
    • l4d2web/services/overlay_files.py
    • l4d2web/services/workshop_paths.py
  • Layout compatibility: both members use [tool.setuptools.package-dir] {name} = "." (pyproject lives inside the package directory). uv workspace members = ["l4d2host", "l4d2web"] handles this fine — uv uses the pyproject as the project root regardless of the package-dir mapping.
  • .gitignore already covers *.egg-info/, .venv/, __pycache__/, etc. No .gitignore changes needed.
  • No pytest.ini / [tool.pytest.ini_options] exists — pytest defaults work; uv run pytest from repo root will discover tests in l4d2host/tests/ and l4d2web/tests/.
  • Bundle action conventions (from ckn-bw/bundles/left4me/items.py and neighbors): every action sets cascade_skip: False explicitly. Action keys in use: command, triggered, cascade_skip, unless, needs, triggers, comment.
  • Additional git_deploy consumer: left4me_chmod_scripts at items.py:324 also needs: 'git_deploy:/opt/left4me/src'. Untouched by this refactor, but listed here so it's not missed during review.
  • Bundle README §"deploy-flow": lines 8490 of bundles/left4me/README.md document the pip_install tempdir dance. This is the prose to rewrite (not vague — those exact lines).
  • apt.packages declaration: metadata.py:2949. Currently lists python3, python3-venv, python3-pip, python3-dev, plus i386 multiarch entries.
  • uv NOT in Debian Trixie apt archive (verified via apt-cache search "^uv$" and apt-cache policy uv on the live host — both return nothing for the actual uv package). Handoff doc's assumption was wrong on this point.
  • git is NOT installed on the production host (verified via command -v git on prod returning empty; /usr/bin/git doesn't exist). The bw git_deploy item operates from the control machine (dev laptop), pushing files to prod via SSH — prod itself needs no git. Implication: the handoff's verification check #1 (sudo git -C /opt/left4me/src status --porcelain) cannot be used. Replace with find /opt/left4me/src \( -name '*.egg-info' -o -name build -o -name dist \) -print.
  • ckn-bw is currently EVEN with origin/master (verified via git status -sb showing ## master...origin/master with empty log). The original prompt's "7 commits ahead" was stale — the operator has since pushed. After our ckn-bw commit lands locally, the repo will be 1 commit ahead (not 8).
  • Prod arch: x86_64 / amd64. Prod curl: 8.14.1 at /usr/bin/curl. Prod tar: GNU tar 1.35. Prod install: GNU coreutils 9.7. /usr/local/bin exists, root-owned, currently contains only the downtime binary.
  • Current prod venv state: /var/lib/left4me/.venv/ exists, owned by left4me:left4me, contains python3.13, pip, alembic, flask, gunicorn, l4d2ctl. pip show l4d2host / pip show l4d2web both report version 0.1.0. So uv will be adopting a venv that already has working installs of both members + their deps.
  • Local dev environment: uv 0.11.8 (brew), direnv 2.37.1 (supports use uv), python 3.13.13. No .venv exists locally yet — clean slate.

Critical files

left4me repo

  • NEW /Users/mwiegand/Projekte/left4me/pyproject.toml — workspace root
  • NEW /Users/mwiegand/Projekte/left4me/uv.lock — generated via uv lock
  • l4d2host/pyproject.toml:10 — bump requires-python to >=3.13
  • l4d2web/pyproject.toml:1018 — bump requires-python, add "l4d2host" to dependencies, add [tool.uv.sources] l4d2host = { workspace = true }
  • .envrc — replace layout python python3.13 with use uv (with fallback if direnv stdlib is too old)
  • README.md, AGENTS.md, l4d2web/README.md — update install instructions

ckn-bw repo (~/Projekte/ckn-bw/)

  • bundles/left4me/metadata.py:2949ensure 'curl': {} is in apt.packages (required by the new install action; verify it's not already inherited from a base bundle). Drop 'python3-pip' (uv replaces pip; bundle has no other consumer). Drop 'python3-venv' (the chain no longer uses python3 -m venv; uv creates its own venv via UV_PROJECT_ENVIRONMENT). Keep 'python3', 'python3-dev', and the i386 multiarch entries. Do NOT add 'uv': {} — uv is not in Trixie's apt archive.
  • bundles/left4me/items.py:285305 — update git_deploy:/opt/left4me/src triggers: replace action:left4me_pip_install with action:left4me_uv_sync
  • bundles/left4me/items.py:328340DELETE left4me_create_venv
  • bundles/left4me/items.py:342352DELETE left4me_pip_upgrade
  • bundles/left4me/items.py:354382DELETE left4me_pip_install (replaced by left4me_uv_sync below)
  • bundles/left4me/items.py:384407left4me_alembic_upgrade: update needs: (or triggered_by: equivalent) to point at action:left4me_uv_sync instead of action:left4me_pip_install
  • bundles/left4me/items.pyADD two new actions:
    • left4me_install_uv: download pinned 0.11.8 tarball from github.com/astral-sh/uv/releases/, SHA256-verify, install to /usr/local/bin/. Idempotent via unless: '/usr/local/bin/uv --version | grep -qx "uv 0.11.8"'. needs: ['pkg_apt:curl'], triggers: ['action:left4me_uv_sync']. (Body matches the approved preview, with unless: refined to grep -qx for BRE portability.)
    • left4me_uv_sync: sudo -u left4me env UV_PROJECT_ENVIRONMENT=/var/lib/left4me/.venv /usr/local/bin/uv sync --frozen --project /opt/left4me/src. triggered: True, cascade_skip: False, needs: includes 'git_deploy:/opt/left4me/src', 'action:left4me_install_uv', 'directory:/var/lib/left4me', 'user:left4me'. triggers: ['action:left4me_alembic_upgrade'].
  • bundles/left4me/README.md:8490 — rewrite the deploy-flow description to mention the install_uv + uv_sync chain instead of the tempdir-dance

Execution steps

Step 0 — Spike test (extended) — DO FIRST

Verify the architectural assumption empirically on the live host. Uses the SAME install vector the production action will use (direct tarball + SHA256 verify), so the spike doubles as a smoke test for the install action itself.

# A. Install pinned uv on prod (one-shot via direct command; matches
#    what the future bw action will do).
ssh ckn@left4.me '
  set -e
  tmpdir=$(mktemp -d); trap "rm -rf $tmpdir" EXIT
  base=https://github.com/astral-sh/uv/releases/download/0.11.8
  tar=uv-x86_64-unknown-linux-gnu.tar.gz
  curl -fsSL -o $tmpdir/$tar       $base/$tar
  curl -fsSL -o $tmpdir/$tar.sha256 $base/$tar.sha256
  (cd $tmpdir && sha256sum -c $tar.sha256)
  tar -xzf $tmpdir/$tar -C $tmpdir --strip-components=1
  sudo install -m 0755 $tmpdir/uv  /usr/local/bin/uv
  sudo install -m 0755 $tmpdir/uvx /usr/local/bin/uvx
  /usr/local/bin/uv --version
'

# B. uv build against root-owned source: source must stay clean.
ssh ckn@left4.me '
  sudo -u left4me sh -c "
    wheels=\$(mktemp -d)
    /usr/local/bin/uv build --wheel --sdist /opt/left4me/src/l4d2host --out-dir \$wheels
    ls \$wheels
  "
'
# Cleanliness probe — git not on prod, so use find for build artifacts.
# Expected: only-existing egg-info dirs (the ones already on disk from
# the current pip install -e flow); NO NEW artifacts from this run.
# Capture a baseline BEFORE the build, compare AFTER.
ssh ckn@left4.me 'sudo find /opt/left4me/src \( -name "*.egg-info" -o -name build -o -name dist -o -name "__pycache__" \) -printf "%T@ %p\n" | sort'

# C. Extended sync-shape check — dry-run `uv sync --frozen` against a
#    root-owned workspace mock in /tmp. Verify the project root stays
#    clean (no .python-version written, no transient files left over).
#    This validates that `uv sync` (not just `uv build`) is safe against
#    a read-only project tree, which is the actual production code path.

Decision gate:

  • Source stays clean across B and C → proceed with full plan.
  • New *.egg-info / build/ / dist/ directories appear in /opt/left4me/src after uv build → fall back to Medium scope (handoff §"Empirical spike" → fallback). Update the handoff doc to record the fallback decision and re-plan.
  • uv sync writes into the project root during step C → also fall back to Medium scope. Same handoff update.

Step 1 — left4me workspace setup (local)

  1. Write /Users/mwiegand/Projekte/left4me/pyproject.toml (workspace root) — see handoff §"What changes — left4me side / New: pyproject.toml"
  2. Bump l4d2host/pyproject.toml:10 to requires-python = ">=3.13"
  3. Update l4d2web/pyproject.toml: bump requires-python, add "l4d2host" to dependencies, append [tool.uv.sources] block
  4. uv lock at the repo root → produces uv.lock
  5. uv sync → creates .venv/, installs both members editable + pytest
  6. uv run pytest → all green
  7. Update .envrc: replace layout python python3.13 with use uv (fallback to uv sync >/dev/null && source .venv/bin/activate if the dev's direnv version doesn't ship use uv)
  8. Update README.md, AGENTS.md, l4d2web/README.md: replace the pip install -e ... invocation with uv sync and add the one-time prereq line about installing uv. Mention macOS (brew install uv) and Linux (curl-pipe-sh from astral.sh) — do NOT suggest apt install uv, as it's not in Debian's apt archive yet (only experimental/sid).

Step 2 — left4me commit + push

Single commit using the suggested message from the handoff (§"Commit messages — left4me side"). Push to origin (gitlab on sublimity.de — confirmed safe-publish-exempt per memory). The commit makes the workspace and lockfile available to ckn-bw's git_deploy.

Step 3 — ckn-bw bundle refactor

  1. Edit bundles/left4me/metadata.py:2949:
    • Ensure 'curl': {} is in apt.packages (verify it's not already inherited from a base bundle; if not, add it explicitly).
    • Drop 'python3-pip' (uv replaces pip; bundle has no other consumer — grep the bundle to confirm).
    • Drop 'python3-venv' (chain no longer uses python3 -m venv).
    • Keep 'python3', 'python3-dev', and the i386 multiarch entries.
    • Do NOT add 'uv': {} — not in Trixie's apt.
  2. Edit bundles/left4me/items.py:
    • Delete left4me_create_venv, left4me_pip_upgrade, left4me_pip_install blocks (lines 328382 inclusive).
    • Add left4me_install_uv action: downloads pinned uv 0.11.8 tarball from github.com/astral-sh/uv/releases/, SHA256-verifies against the official .sha256 sibling, installs to /usr/local/bin/{uv,uvx}. Idempotent via unless: '/usr/local/bin/uv --version 2>/dev/null | grep -qx "uv 0.11.8"'. needs: ['pkg_apt:curl'], triggers: ['action:left4me_uv_sync'], triggered: False, cascade_skip: False.
    • Add left4me_uv_sync action: sudo -u left4me env UV_PROJECT_ENVIRONMENT=/var/lib/left4me/.venv /usr/local/bin/uv sync --frozen --project /opt/left4me/src. triggered: True, cascade_skip: False. needs: includes 'git_deploy:/opt/left4me/src', 'action:left4me_install_uv', 'directory:/var/lib/left4me', 'user:left4me'. triggers: ['action:left4me_alembic_upgrade'].
    • Update git_deploy:/opt/left4me/src triggers (lines 285305): replace 'action:left4me_pip_install' with 'action:left4me_uv_sync'. Keep left4me_alembic_upgrade and left4me_daemon_reload triggers.
    • Update left4me_alembic_upgrade (lines 384407): its dependency on left4me_pip_install must now point at left4me_uv_sync.
  3. Rewrite bundles/left4me/README.md:8490 to describe the new install_uv → uv_sync → alembic_upgrade → seed_overlays + restart chain (drop the pip + tempdir-dance prose).
  4. (cd ~/Projekte/ckn-bw && .venv/bin/bw test) → must pass clean.

Step 4 — ckn-bw commit (DO NOT PUSH)

Single commit using the suggested message from the handoff (§"Commit messages — ckn-bw side"). Do not git push. Per verified state today, ckn-bw is currently EVEN with origin/master (not 7 ahead as the original prompt claimed — the operator pushed since the prompt was written). After this commit lands locally, the repo will be 1 commit ahead of origin.

Step 5 — Report to operator (handoff to user for deploy)

Agent's work ends here. Brief summary to the user including:

  • Spike outcome (full uv-workspace path confirmed, or Medium-scope fallback taken — including any handoff doc updates if the latter).
  • What's committed and where it sits: left4me pushed to origin/master; ckn-bw committed locally, now 1 commit ahead of origin (unpushed).
  • The bw apply ovh.left4me invocation for the user to run, with the expected output (left4me_install_uv runs the download+verify, three old actions removed from the graph, two new actions present (install_uv + uv_sync), alembic+seed+restart cascade fires).
  • The 6-check verification matrix from handoff §"Verification (end-to-end)" for the user to walk through after apply — with check #1 amended: use sudo find /opt/left4me/src \( -name '*.egg-info' -o -name build -o -name dist \) -newer <baseline> instead of git status, because git isn't installed on prod.
  • Recovery path if uv refuses to adopt the existing venv: one-shot ssh ckn@left4.me 'sudo -u left4me rm -rf /var/lib/left4me/.venv', then re-apply.
  • Open follow-ups (uv version pinning policy — bump cadence, signing, etc; direnv use uv fallback applied or not; whether to add a separate pkg_apt: curl if it wasn't already declared).

Do NOT run bw apply, the verification matrix, or the gameserver round-trip — those are explicitly user-side per session memory.

Plan storage after approval

Per the user's global AGENTS.md (~/.claude/agents/AGENTS.md): specs and plans live in the repo they describe, typically under docs/. After ExitPlanMode and approval, this plan should be copied to /Users/mwiegand/Projekte/left4me/docs/superpowers/plans/2026-05-15-uv-workspace-execution.md as a peer to the design handoff, then committed alongside the left4me changes in Step 2.

What does NOT change (out of scope)

  • Source ownership: /opt/left4me/src stays root-owned.
  • Venv location: /var/lib/left4me/.venv stays where it is, owned by the left4me user, accessed via UV_PROJECT_ENVIRONMENT.
  • Hardening drop-ins, sudoers, sysctl, helpers — all stable from the deployment-responsibility migration.
  • systemd unit shapes — reactor-emitted, unchanged.
  • alembic_upgrade and seed_overlays shell bodies — same commands, just triggered from uv_sync instead of pip_install.
  • pkg_apt: python3 and python3-dev — kept (uv shells out to system Python).
  • Other ckn-bw bundles — this is left4me-specific.
  • The build-overlay-unit refactor — separate queued thread.
  • CI — none currently exists.

Risks (carried from handoff, sized empirically)

  1. Spike test failure → fall back to Medium scope. Graceful.
  2. Lockfile format skew between dev and prodMITIGATED by pinning prod uv to 0.11.8 (same as local brew). Lockfile generated by dev's uv 0.11.8 will be consumed by prod's uv 0.11.8 byte-for-byte compatible. Risk effectively eliminated unless dev's brew bumps uv independently — track this in the pinning-policy follow-up.
  3. direnv use uv availability → local direnv is 2.37.1 (use uv added in 2.34+, so we're fine). Fallback snippet documented in case another dev has an older direnv.
  4. alembic/flask binary paths → uv installs the same console_scripts entrypoints as pip, so paths under /var/lib/left4me/.venv/bin/ are identical. Verify in verification matrix.
  5. --force-reinstall semantics → no longer needed; uv sync is lockfile-aware, not package-version-aware.
  6. uv release artifact availability → if github.com/astral-sh/uv takes down release 0.11.8 (extremely unlikely but theoretically possible), the install action would fail. Mitigation: pin a recent stable release, monitor astral's deprecation cadence; if needed, mirror the artifact to an internal location for future-proofing (out of scope for this migration).
  7. SHA256 of the tarball → we trust the .sha256 sibling fetched from the same github release. A future hardening pass could embed the checksum in the bundle source for offline verification, but the current trust model matches steamcmd's (also github-sourced).