left4me/docs/superpowers/specs/2026-05-15-runtime-state-relocation-design.md

Runtime state relocation + non-editable install — design (as shipped)

Status

Shipped 2026-05-15. Supersedes 2026-05-15-handoff-noneditable-install.md (the narrower prereq spec). The scope expanded during execution after a hidden constraint surfaced; this doc records what actually shipped.

What shipped

Two related changes, landed together:

1. /opt/left4me/ becomes a root-owned deploy-artifact root. Only /opt/left4me/src lives there (eventually /opt/left4me/scripts/ after the deployment-responsibility reshape too). Both /opt/left4me and /opt/left4me/src are root:root.
2. Runtime mutable state moved to /var/lib/left4me/. Specifically:
   • /var/lib/left4me/.venv (was /opt/left4me/.venv)
   • /var/lib/left4me/steam (was /opt/left4me/steam)
3. Production install model is non-editable. ckn-bw's left4me_pip_install action copies the (root-owned) source to a left4me-owned tempdir under $TMPDIR and runs pip install --force-reinstall "$tmpdir/l4d2host" "$tmpdir/l4d2web" from there. The tempdir is cleaned by a trap on EXIT.

Local-development install flows are unchanged: developers still use pip install -e ./l4d2host -e ./l4d2web via direnv.

Why the scope expanded

The original handoff (2026-05-15-handoff-noneditable-install.md) proposed a minimal change: flip /opt/left4me/src to root:root and switch pip install -e → pip install directly. That spec stated "Non-editable install works out of the box; no packaging edits needed."

That premise turned out to be wrong. setuptools.build_meta's PEP 517 get_requires_for_build_wheel hook runs setup.py egg_info in the source directory by design, and egg_info writes <pkg>.egg-info/ back to the cwd. Against a root-owned source tree, this fails with Permission denied. python -m build was tried as an alternative — its build isolation only sandboxes build dependencies, not the source, so it hits the same failure.

The fix that actually works is to copy the source to a writable location and build from the copy. Once that one-shot copy is in the pip_install action, the original narrow scope becomes inconsistent: the source is root-owned, the venv is left4me-owned, both live under /opt/left4me/. Moving runtime mutable state out of /opt/left4me/ and into /var/lib/left4me/ resolves the inconsistency by aligning with FHS conventions: /opt/<app> = read-only deploy artifacts, /var/lib/<app> = mutable runtime state.

The operator opted for the larger reshape ("do the best long-term solution now") rather than landing the narrow change and queuing the relocation separately.

What changed concretely

ckn-bw side (bundles/left4me/)

• items.py
  • directory:/opt/left4me → root:root 0755 (was left4me:left4me)
  • directory:/opt/left4me/src → root:root (was left4me:left4me)
  • directory:/opt/left4me/steam removed
  • directory:/var/lib/left4me/steam added (left4me:left4me)
  • action:left4me_chown_src deleted (was the every-apply chown-to-left4me self-heal; no longer needed)
  • action:left4me_install_steamcmd — paths flipped to /var/lib/left4me/steam
  • action:left4me_create_venv, left4me_pip_upgrade, left4me_alembic_upgrade, left4me_seed_overlays — all venv paths changed from /opt/left4me/.venv to /var/lib/left4me/.venv
  • action:left4me_pip_install — completely rewritten to use the cp-to-tempdir + pip install approach, marked triggered: True so it only fires on actual code changes (the cp + wheel build is too heavy to run on every apply)
  • git_deploy:/opt/left4me/src triggers list — added action:left4me_pip_install (the new wiring); kept alembic_upgrade as belt-and-braces and install_left4me_scripts
• metadata.py
  • left4me-web.service: Environment=PATH= and ExecStart= use /var/lib/left4me/.venv
  • left4me-workshop-refresh.service: same
  • left4me-server@.service BindReadOnlyPaths entry for steamcmd → /var/lib/left4me/steam
• files/etc/left4me/host.env.mako: LEFT4ME_STEAMCMD → /var/lib/left4me/steam/steamcmd.sh
• README.md: updated description

left4me side

• deploy/files/usr/local/lib/systemd/system/left4me-web.service — reference unit: PATH= + ExecStart= use /var/lib/left4me/.venv
• deploy/files/usr/local/lib/systemd/system/left4me-workshop-refresh.service — reference unit: WorkingDirectory=/opt/left4me/src (was /opt/left4me), PATH= + ExecStart= use /var/lib/left4me/.venv
• scripts/sbin/left4me wrapper — flask path
• deploy/tests/test_example_units.py — PATH + ExecStart assertions updated; the assertion previously read "Environment=PATH=..." which was already broken (the unit has Environment=HOME=... PATH=... on one line), now reads just "PATH=..."
• deploy/README.md — paths described
• l4d2host/tests/test_cli.py — LEFT4ME_STEAMCMD fixture path

Host transition (one-shot, performed manually during the deploy)

1. systemctl stop left4me-web.service
2. mv /opt/left4me/steam /var/lib/left4me/steam (atomic same-fs rename; running gameserver BindReadOnlyPaths bindings keep working because they reference inodes, not paths)
3. rm -rf /opt/left4me/.venv /opt/left4me/wheels
4. bw apply -s svc_systemd:left4me-web.service ovh.left4me — creates new venv at /var/lib/left4me/.venv and runs pip_upgrade. -s to skip the web service item so bw doesn't try to start it before pip_install has populated the new venv.
5. Manual pip_install (the same command the bundle's left4me_pip_install action runs) to install l4d2host + l4d2web non-editably into the new venv.
6. Manual alembic upgrade head + systemctl start left4me-web.service.
7. Second bw apply ovh.left4me to confirm idempotent.

Gameservers (left4me-server@1, @2) stayed up throughout — they don't link to the Python venv and their bind mounts survived the steam dir rename via inode-level binding.

Verification (six checks, all green)

1. stat -c '%U:%G %a %n' /opt/left4me /opt/left4me/src /var/lib/left4me/.venv /var/lib/left4me/steam → /opt/left4me and /opt/left4me/src both root:root 755; .venv and steam left4me:left4me.
2. pip show l4d2host l4d2web → Location: is /var/lib/left4me/.venv/lib/python3.13/site-packages, no Editable project location: line.
3. systemctl is-active left4me-web.service → active.
4. alembic current → 0012_command_history (head).
5. Gameserver fresh-restart deferred — running instances are unaffected (inode-level binds survive the rename); a fresh bw apply confirms the new unit content has the new bind paths. Will validate on the next operator-initiated server restart.
6. Second bw apply ovh.left4me → 0 fixed, 0 failed. Idempotent.

What this does NOT change

• The deployment-responsibility brainstorm (2026-05-15-handoff-deployment-responsibility.md) — still queued. This prereq just makes target-side symlinks into /opt/left4me/src/deploy/files/... safe by construction (left4me cannot rewrite its own hardening profile).
• Sudoers content (still in deploy/files/etc/sudoers.d/left4me
  • the verbatim mirror in ckn-bw; consolidation queued).
• scripts/{libexec,sbin}/ location in left4me — still under the repo root; the deployment-responsibility brainstorm decides whether to move them into deploy/scripts/.
• Hardening drop-ins — still inline in ckn-bw's systemd/units reactor; whether to move them to deploy/files/... is also the deployment-responsibility brainstorm's call.

Pointers

• Original (now-superseded) handoff: docs/superpowers/specs/2026-05-15-handoff-noneditable-install.md
• Deployment-responsibility brainstorm handoff: docs/superpowers/specs/2026-05-15-handoff-deployment-responsibility.md
• ckn-bw bundle: ~/Projekte/ckn-bw/bundles/left4me/