left4me/deploy

left4me Deployment

This directory contains the production-like test deployment for a Linux server. It installs the repository into a fixed host layout, configures a dedicated runtime user, installs systemd units, and wires the web app to host operations through privileged helper commands.

Target Layout

The deployment uses these paths:

• /etc/left4me/host.env: host library environment configuration.
• /etc/left4me/web.env: web app environment configuration.
• /opt/left4me/.venv: Python virtual environment for deployed commands.
• /opt/left4me: deployed repository contents.
• /var/lib/left4me/left4me.db: SQLite database used by the web app.
• /var/lib/left4me/installation: shared L4D2 installation.
• /var/lib/left4me/overlays: overlay directories. Each overlay lives at ${overlay_id} under here.
• /var/lib/left4me/workshop_cache: deduplicated cache of .vpk files downloaded for workshop overlays. One file per Steam item, named {steam_id}.vpk. Workshop overlays symlink into this tree.
• /var/lib/left4me/global_overlay_cache: cache of non-Steam map archives and extracted .vpk files used by managed global map overlays.
• /var/lib/left4me/instances: rendered instance specifications and per-instance state.
• /var/lib/left4me/runtime: per-instance runtime mount directories.
• /var/lib/left4me/tmp: temporary files used by deployment/runtime operations.
• /usr/local/lib/systemd/system: global systemd unit files, including left4me-server@.service.
• /usr/local/libexec/left4me: privileged helper commands, including left4me-systemctl, left4me-journalctl, and left4me-overlay (the latter mounts the per-instance kernel overlay in PID 1's mount namespace via nsenter).
• /etc/sudoers.d/left4me: sudoers rules allowing the web/runtime commands to call the helpers non-interactively.

Static units are generated for /var/lib/left4me. If LEFT4ME_ROOT changes, regenerate and reinstall the unit files instead of reusing the existing static units.

Runtime User

The deployment creates and runs host operations as the dedicated runtime user:

• Username: left4me
• Home: /var/lib/left4me
• Shell: /usr/sbin/nologin

Running A Test Deployment

Run the deployment from the repository root:

deploy/deploy-test-server.sh deploy-user@example-host

The SSH user must be able to run sudo on the target host. The deployment configures system packages, directories, environment files, helper scripts, sudoers rules, Python dependencies, and systemd units.

Admin Bootstrap

Set the bootstrap credentials in the environment when creating the first admin user:

LEFT4ME_ADMIN_USERNAME=admin \
LEFT4ME_ADMIN_PASSWORD='change-me' \
flask create-user "$LEFT4ME_ADMIN_USERNAME" --admin

Use a strong one-time password and rotate it after first login if needed.

Overlay References

Overlay references are relative paths below ${LEFT4ME_ROOT}/overlays. With the default deployment root, they resolve under /var/lib/left4me/overlays. New overlays use ${overlay_id} as their path; the digit-only form is the only one created by the web app.

Invalid references are rejected:

• Absolute paths such as /srv/overlay.
• Parent traversal such as ../other or competitive/../../base.
• Empty path components such as competitive//base.
• Symlink escapes that resolve outside ${LEFT4ME_ROOT}/overlays.

The web app currently supports two overlay surfaces:

• workshop overlays (user-owned) — populated by downloading .vpk files from the public Steam Web API into ${LEFT4ME_ROOT}/workshop_cache/{steam_id}.vpk and creating absolute symlinks under ${LEFT4ME_ROOT}/overlays/{overlay_id}/left4dead2/addons/{steam_id}.vpk.
• script overlays — populated by an arbitrary user-authored bash script that runs inside bubblewrap + systemd-run --scope as the unprivileged l4d2-sandbox UID, with the overlay directory bind-mounted RW at /overlay. Resource caps: 1h walltime, 4 GB RAM, 512 tasks, 200% CPU, 20 GB post-build disk cap.

Both the caches and the overlay directories are owned by the left4me runtime user; if the web service ever runs as a different uid, ensure it shares a group with the host process and that both trees are group-readable.

Performance Tuning

The deployment ships a host-side perf baseline (slices, unit directives, sysctls). See docs/superpowers/specs/2026-05-09-l4d2-server-host-perf-baseline-design.md for design rationale.

The following knobs are documented escape hatches — they are not auto-applied. Apply only if you have measured a need and understand the failure modes.

CPU governor

The performance governor squeezes a few percent off jitter under bursty load. schedutil is acceptable for sustained UDP workloads.

sudo cpupower frequency-set -g performance

Install via sudo apt install linux-cpupower if the binary isn't present.

Persist via your distro's CPU-frequency tooling (e.g. /etc/default/cpufrequtils).

CPU isolation (cores)

The deploy script writes four AllowedCPUs= drop-ins so that, by default, only l4d2-game.slice is allowed to run on cores 1..N-1; system.slice, user.slice, and l4d2-build.slice are pinned to core 0. Game servers thus get the host minus core 0 exclusively, the build sandbox and the web app stay on core 0, and a logged-in admin running CPU-heavy work in their shell can't steal cycles from a live match.

Override the split by setting either env var when running the deploy:

LEFT4ME_SYSTEM_CPUS="0,1" LEFT4ME_GAME_CPUS="2-7" deploy/deploy-test-server.sh deploy-user@host

On single-core hosts the deploy skips the cpuset drop-ins entirely and prints a warning to stderr; the rest of the perf baseline (cgroup weights, sysctls, OOM scores) still applies. To force isolation on a single-core host anyway (rarely useful), set either env var explicitly.

Per-instance CPUAffinity= (next subsection) composes on top of this — the per-instance value must be a subset of l4d2-game.slice's AllowedCPUs=, which the kernel enforces.

Per-instance CPU affinity

srcds is single-threaded per instance. On a multi-core host, pinning each instance to its own core can cut jitter under contention. Drop in /etc/systemd/system/left4me-server@<name>.service.d/affinity.conf:

[Service]
CPUAffinity=2

This pins the instance to CPU 2 specifically; per-instance values would typically be 1, 2, 3, ... so each server has its own core.

A reasonable strategy on an N-core host: leave core 0 for the kernel + IRQs + system services, then pin one instance per remaining core.

NIC tuning

Hardware-specific (install via sudo apt install ethtool if not present). On a host with a single primary interface (replace eth0):

sudo ethtool -G eth0 rx 4096 tx 4096
sudo ethtool -K eth0 gro on lro off

If you run a high instance count, also pin the NIC's interrupts off the cores that game servers occupy (see /proc/interrupts and /proc/irq/<n>/smp_affinity).

Real-time scheduling (advanced, opt-in)

Source-engine servers do not need real-time scheduling, and a misbehaving srcds at any RT priority can starve kernel threads — even with the default kernel.sched_rt_runtime_us=950000 throttling 5% of CPU back. Use only if you have a measured jitter problem that the baseline does not solve.

/etc/systemd/system/left4me-server@.service.d/realtime.conf:

[Service]
CPUSchedulingPolicy=fifo
CPUSchedulingPriority=10
LimitRTPRIO=10
AmbientCapabilities=CAP_SYS_NICE

The AmbientCapabilities=CAP_SYS_NICE line is needed because the service runs as User=left4me with NoNewPrivileges=true; without it some kernels/systemd combinations refuse to apply the RT policy.

Applying changes to running servers

Unit-file changes do not apply to already-running services. After any change:

sudo systemctl daemon-reload
# Restart each game server via the web UI's stop + start, or:
sudo systemctl restart 'left4me-server@*.service'