left4me/docs/superpowers/specs/2026-05-15-handoff-uv-workspace.md
mwiegand 49992b3a26
refactor(repo): uv workspace + hatchling + layout restructure
Migrate from pip-install-e + setuptools to a uv workspace with a
committed uv.lock for deterministic deps. Switch both members to
hatchling, and move package sources into nested standard layout
(l4d2host/l4d2host/, l4d2web/l4d2web/) so builds work from a
read-only source tree — setuptools wrote egg-info to source under
the old layout, which broke uv sync on the root-owned /opt/left4me/src.

Local dev install: `pip install -e ./l4d2host -e ./l4d2web` -> `uv sync`.
.envrc switches from `layout python python3.13` to `use uv`. Python
pinned to 3.13 via .python-version.

l4d2web now declares its cross-dep on l4d2host explicitly via
[tool.uv.sources] (workspace = true). l4d2web/alembic.ini and
l4d2web/alembic/ stay at the project root (standard alembic layout).

Test fixes:
- tests/__init__.py added to both test dirs so pytest doesn't shadow
  l4d2host as a namespace package via outer-dir walk.
- 3 CWD-relative paths in tests (l4d2web/static/css/{tokens,layout}.css
  and js/sse.js) anchored to Path(__file__) so they survive layout
  changes.
- Two test_install.py tests now monkeypatch HOME to tmp_path so they
  stop silently mutating ~/.steam/sdk32 on every run.

628 tests pass under sandboxed `uv run pytest`.

Per docs/superpowers/plans/2026-05-15-uv-workspace-execution.md;
prereq for the ckn-bw bundle's uv-sync action (queued).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-15 22:04:29 +02:00

16 KiB

Handoff — collapse venv chain into uv workspace + uv sync

Status

Executed (left4me side) — see docs/superpowers/plans/2026-05-15-uv-workspace-execution.md for what actually shipped and what diverged from the assumptions below. Three load-bearing assumptions in this doc turned out to be wrong (no pkg_apt: uv on Trixie; existing layout incompatible with read-only source builds via setuptools; no git on prod). The executed plan records the corrections.

Goal

Replace the current five-action venv chain in bundles/left4me/items.py with a single uv sync --frozen action driven by a committed uv.lock at the left4me repo root. Eliminate the tempdir-copy dance in pip_install (8 lines of shell working around setuptools writing <pkg>.egg-info/ into a root-owned source tree).

Net change: 5 actions → 3 actions; deterministic deploys via locked dep versions; single command in dev and prod; one new build-time dependency (uv) on the host.

Why

Three motivations, listed in priority order.

Deterministic prod deploys. Today's chain installs whatever pip resolves at apply time. A transitive dep getting a CVE-relevant bump between two bw apply runs is invisible until it breaks something. uv sync --frozen against a committed uv.lock makes the installed version set reproducible from git history alone.

Lower cognitive cost in items.py. The pip_install action is the longest, gnarliest action in the bundle — it does its own tempdir/cleanup-trap/cp-r dance because the obvious pip install /opt/left4me/src/... would write egg-info to a root-owned source tree. uv's sdist-then-wheel-from-tarball build path makes this problem go away: the source is read-only throughout.

Workspace declares what's actually true. l4d2web already imports from l4d2host (5 files use from l4d2host.paths import ...). Today's setup happens to work because both packages get installed side-by-side via pip install -e ./l4d2host -e ./l4d2web, but the dependency relationship is implicit. A uv workspace makes it explicit via [tool.uv.sources] l4d2host = { workspace = true }.

Current state — the 5-action chain

(All in ~/Projekte/ckn-bw/bundles/left4me/items.py, ~lines 285-425.)

git_deploy:/opt/left4me/src
   ├── triggers → left4me_pip_install
   │                ├── needs ← left4me_create_venv  (always-on, gated unless)
   │                │             └── triggers → left4me_pip_upgrade
   │                └── triggers → left4me_alembic_upgrade
   │                                 ├── triggers → left4me_seed_overlays
   │                                 └── triggers → svc_systemd:left4me-web.service:restart
   ├── triggers → left4me_alembic_upgrade  (belt-and-braces direct trigger)
   └── triggers → left4me_daemon_reload

left4me_pip_install body (the part that simplifies):

sudo -u left4me sh -c '
set -e
tmpdir=$(mktemp -d -t left4me-build-XXXXXX)
trap "rm -rf \"$tmpdir\"" EXIT
cp -r /opt/left4me/src/l4d2host /opt/left4me/src/l4d2web "$tmpdir/"
/var/lib/left4me/.venv/bin/pip install --force-reinstall "$tmpdir/l4d2host" "$tmpdir/l4d2web"
'

Target state — uv workspace + single sync action

Three actions instead of five:

git_deploy:/opt/left4me/src
   ├── triggers → left4me_uv_sync
   │                └── triggers → left4me_alembic_upgrade
   │                                 ├── triggers → left4me_seed_overlays
   │                                 └── triggers → svc_systemd:left4me-web.service:restart
   ├── triggers → left4me_alembic_upgrade  (belt-and-braces)
   └── triggers → left4me_daemon_reload

left4me_uv_sync body:

actions['left4me_uv_sync'] = {
    'command': (
        'sudo -u left4me '
        'env UV_PROJECT_ENVIRONMENT=/var/lib/left4me/.venv '
        'uv sync --frozen --project /opt/left4me/src'
    ),
    'triggered': True,
    'cascade_skip': False,
    'needs': [
        'git_deploy:/opt/left4me/src',
        'pkg_apt:uv',
        'directory:/var/lib/left4me',
        'user:left4me',
    ],
    'triggers': [
        'action:left4me_alembic_upgrade',
    ],
}

UV_PROJECT_ENVIRONMENT redirects uv's default venv path (<project>/.venv) to our writable runtime location at /var/lib/left4me/.venv (the source at /opt/left4me/src is root-owned, so the default would be a permission error).

--frozen requires uv.lock to be present and consistent with pyproject.toml — refuses to silently update the lockfile during deploy.

Empirical spike — do this FIRST

Before touching anything, verify the architectural assumption that uv actually keeps a root-owned source directory pristine during build. ~5 minute test on the live host:

ssh ckn@left4.me 'sudo apt-get install -y uv'
ssh ckn@left4.me '
  sudo -u left4me sh -c "
    wheels=\$(mktemp -d)
    uv build --wheel --sdist /opt/left4me/src/l4d2host --out-dir \$wheels
    ls \$wheels
    sudo git -C /opt/left4me/src status --porcelain
  "
'

Expected: the wheel + sdist exist in the tempdir, AND git status reports the source tree clean (no new l4d2host.egg-info/ directory).

If the source stays clean, proceed with the full migration.

If the source picks up l4d2host.egg-info/ (uv's build invoked setuptools.build_meta directly on the source instead of via the sdist intermediate), fall back to Medium scope: keep the tempdir-copy dance but use uv pip install in place of pip install (1:1 swap, no workspace, smaller change). Update this handoff with the fallback decision.

What changes — left4me side

New: /Users/mwiegand/Projekte/left4me/pyproject.toml

Workspace root. Short:

[project]
name = "left4me"
version = "0.0.0"
description = "Workspace root; packaging lives in the members."
requires-python = ">=3.13"

[tool.uv.workspace]
members = ["l4d2host", "l4d2web"]

# Dev-only dependencies (pytest, etc.) for the workspace.
[dependency-groups]
dev = [
    "pytest",
]

Modified: l4d2host/pyproject.toml and l4d2web/pyproject.toml

No real change to declared deps. l4d2web adds the workspace cross-dep:

# l4d2web/pyproject.toml
[project]
dependencies = [
    "Flask>=3.0",
    "SQLAlchemy>=2.0",
    "alembic>=1.13",
    "PyYAML>=6.0",
    "gunicorn>=22.0",
    "requests>=2.31",
    "l4d2host",                          # NEW: declares the import relationship
]

[tool.uv.sources]
l4d2host = { workspace = true }          # NEW: resolves to the in-workspace member

This makes explicit what's already true: l4d2web/routes/overlay_routes.py, l4d2web/services/overlay_creation.py, and three other files import from l4d2host.paths.

New: /Users/mwiegand/Projekte/left4me/uv.lock

Generated by uv lock at the repo root. Committed to git. Pins every transitive dep version.

Modified: /Users/mwiegand/Projekte/left4me/.envrc

Today:

layout python python3.13

New (direnv hands off to uv for venv management):

# direnv's stdlib uv helper creates .venv via `uv sync` and activates it.
# Equivalent to: uv sync && source .venv/bin/activate
use uv

If use uv isn't available in this direnv version (it's a stdlib function added in direnv 2.34+), fall back to:

uv sync >/dev/null
source .venv/bin/activate

Modified: README.md, AGENTS.md, l4d2web/README.md

Update install instructions from:

pip install -e ./l4d2host -e ./l4d2web pytest

to:

uv sync           # creates .venv, installs members editable, installs dev deps

One-time prereq for developers: install uv (brew install uv on macOS, apt install uv on Debian Trixie+, or curl-pipe-sh from astral.sh for older distros).

Modified: .gitignore

Probably no change needed. uv's caches default to ~/.cache/uv (not in-repo). The .venv is already ignored.

What changes — ckn-bw side

All edits in ~/Projekte/ckn-bw/bundles/left4me/.

metadata.py

Add uv to apt.packages:

'apt': {
    'packages': {
        ...
        'uv': {},  # Required by left4me_uv_sync for production install.
        ...
    },
},

Drop python3-pip if nothing else needs it (uv replaces pip). Keep python3-venv if anything else on the host uses python3 -m venv; if not, drop it too. python3 and python3-dev stay (uv invokes them).

items.py

Delete three actions:

  • left4me_create_venv
  • left4me_pip_upgrade
  • left4me_pip_install

Add one action: left4me_uv_sync (body in the "Target state" section above).

Update git_deploy:/opt/left4me/src triggers:

  • Remove: action:left4me_pip_install
  • Add: action:left4me_uv_sync
  • Keep: action:left4me_alembic_upgrade, action:left4me_daemon_reload

alembic_upgrade and seed_overlays are unchanged — they invoke the venv's alembic and flask binaries by absolute path, which uv sync ensures exist. Update their needs: lists to point at action:left4me_uv_sync instead of action:left4me_pip_install.

README.md

Update the bundle README's deploy-flow description to mention uv sync instead of pip install -e, matching the new shape.

Migration order

  1. Spike test (above): confirm uv preserves source cleanliness. If fails, retreat to Medium scope.
  2. left4me-side preparation (independent PR, can land first):
    • Add root pyproject.toml, declare workspace
    • Add l4d2host to l4d2web's deps + workspace source
    • Run uv lock, commit uv.lock
    • Update .envrc
    • Update local-dev docs
    • Run uv sync locally, run pytest — all green
    • Commit + push
  3. ckn-bw-side install (depends on step 2):
    • Add pkg_apt: uv to bundle defaults
    • Delete the three old actions, add uv_sync
    • Update git_deploy triggers and downstream needs:
    • bw test clean
  4. First apply to ovh.left4me:
    • Expect: pkg_apt: uv installed, three old actions removed from the graph, new uv_sync action fires (because git_deploy fires with the new commit), runs uv sync --frozen against the new workspace, alembic_upgrade + seed_overlays + web restart cascade.
    • The existing /var/lib/left4me/.venv (created by python3 -m venv) is structurally a uv-compatible venv; uv should adopt it without recreation. If uv refuses to adopt (incompatible metadata), one-shot fix on the host:
      sudo -u left4me rm -rf /var/lib/left4me/.venv
      # bw apply will recreate via `uv sync`
      
  5. Idempotency check + verification matrix:
    • bw apply idempotent (0 fixed, 0 failed)
    • pip show l4d2{host,web} reports the locked version
    • Web service active, gameserver round-trip works
  6. Commit ckn-bw side, do not push (operator pushes manually).

What does NOT change

  • Source ownership: /opt/left4me/src stays root:root (the runtime-state relocation made it so; uv reads it as world-readable).
  • Venv location: /var/lib/left4me/.venv stays where it is, owned by left4me, accessed via UV_PROJECT_ENVIRONMENT.
  • Hardening drop-ins, sudoers, sysctl, helpers: all stable from the deployment-responsibility migration. uv migration is independent.
  • systemd unit shapes: reactor-emitted, per-host parameters unchanged.
  • alembic_upgrade and seed_overlays: same shell, same triggering, same binaries (just from a uv-managed venv).
  • pkg_apt: python3 and python3-dev: kept (uv shells out to the system Python interpreter).
  • CI workflows: no CI currently exists; nothing to update.

Out of scope

  • Merging l4d2host and l4d2web into a single package. They stay as separate workspace members.
  • Switching to a non-direnv-based dev flow. direnv + use uv stays.
  • Migrating other ckn-bw bundles to uv. This is left4me-specific.
  • Pinning the host's uv version below the apt-current. If lockfile format issues surface, address as a follow-up (e.g., apt-pin or switch to astral.sh-installed uv).

Risks

  1. Spike test failure: uv build isn't actually source-clean → falls back to Medium scope. Captured above; this is a graceful degradation.
  2. Lockfile format skew: dev's brew-installed uv (latest) ahead of prod's apt-installed uv (Trixie's version) → lockfile produced in dev rejected in prod. Mitigation: stick to features supported by the apt-installed version; if needed, switch prod to a pinned astral.sh install.
  3. alembic invocation path: today the action calls /var/lib/left4me/.venv/bin/alembic. After uv sync, this path should still exist (uv installs the same console_scripts entrypoint as pip). Verify in step 4.
  4. direnv use uv availability: use uv was added to direnv's stdlib relatively recently. If the dev's direnv is older, use the fallback .envrc snippet (uv sync >/dev/null && source .venv/bin/activate).
  5. --force-reinstall semantics gone: today's chain uses pip install --force-reinstall to work around the static 0.1.0 version in pyproject.toml — without it pip would skip on no-op. uv sync --frozen is version-aware via the lockfile, not the package version string, so this concern goes away.

Verification (end-to-end)

After ckn-bw apply:

  1. Source still clean:

    ssh ckn@left4.me 'sudo git -C /opt/left4me/src status --porcelain'
    

    Empty output.

  2. Venv has the workspace members installed:

    ssh ckn@left4.me 'sudo -u left4me /var/lib/left4me/.venv/bin/python -c "import l4d2host; import l4d2web; print(l4d2host.__file__, l4d2web.__file__)"'
    

    Both paths point inside /var/lib/left4me/.venv/lib/python3.13/site-packages/.

  3. Pinned versions match the lockfile:

    ssh ckn@left4.me 'sudo -u left4me /var/lib/left4me/.venv/bin/pip show flask | grep Version'
    

    Matches the Flask version in uv.lock.

  4. Web service health:

    ssh ckn@left4.me 'sudo systemctl is-active left4me-web.service'
    

    active.

  5. Idempotent apply:

    (cd ~/Projekte/ckn-bw && .venv/bin/bw apply ovh.left4me)
    

    0 fixed, 0 failed.

  6. Gameserver round-trip: start a verify instance via left4me-systemctl enable verify, check journal for clean srcds_run startup behaviour (modulo any missing instance dir), disable.

Pointers

  • Deployment-responsibility design (just shipped; the venv chain it did NOT touch is what this handoff replaces): docs/superpowers/specs/2026-05-15-deployment-responsibility-design.md
  • Runtime state relocation (made /opt/left4me/src root-owned, which is why the current pip_install needs the tempdir dance): docs/superpowers/specs/2026-05-15-runtime-state-relocation-design.md
  • ckn-bw left4me bundle: ~/Projekte/ckn-bw/bundles/left4me/
    • items.py:285-306git_deploy triggers
    • items.py:328-340left4me_create_venv
    • items.py:342-352left4me_pip_upgrade
    • items.py:354-382left4me_pip_install (the tempdir dance)
    • items.py:384-407left4me_alembic_upgrade
    • items.py:409-424left4me_seed_overlays
  • uv docs: https://docs.astral.sh/uv/ — workspace, uv sync, UV_PROJECT_ENVIRONMENT.

Commit messages (suggested)

left4me side (root pyproject + lockfile + member deps + .envrc + docs):

refactor(repo): uv workspace + lockfile

Declare the repo as a uv workspace with l4d2host and l4d2web as
members. Add uv.lock for deterministic dep resolution. l4d2web now
declares its cross-dep on l4d2host explicitly via tool.uv.sources.

Local-dev install switches from `pip install -e ./l4d2host -e ./l4d2web`
to `uv sync` (creates venv, installs members editable, installs dev
deps from one source). .envrc uses direnv's `use uv` helper.

Prereq for the ckn-bw bundle uv-sync action (handoff:
docs/superpowers/specs/2026-05-15-handoff-uv-workspace.md).

ckn-bw side (drop chain, install uv, single sync action):

refactor(left4me): collapse venv chain into uv sync

Replace left4me_create_venv + left4me_pip_upgrade + left4me_pip_install
(the tempdir-copy dance) with a single left4me_uv_sync action driven
by left4me's committed uv.lock. Deterministic dep versions, no source
mutation during build, three actions instead of five.

pkg_apt: uv added. python3-pip removed (uv replaces it).

Per docs/superpowers/specs/2026-05-15-handoff-uv-workspace.md (in the
left4me repo).