# left4me Deployment

This directory contains the production-like test deployment for a Linux server. It installs the repository into a fixed host layout, configures a dedicated runtime user, installs systemd units, and wires the web app to host operations through privileged helper commands.

## Target Layout

The deployment uses these paths:

- `/etc/left4me/host.env`: host library environment configuration.
- `/etc/left4me/web.env`: web app environment configuration.
- `/opt/left4me/.venv`: Python virtual environment for deployed commands.
- `/opt/left4me`: deployed repository contents.
- `/var/lib/left4me/left4me.db`: SQLite database used by the web app.
- `/var/lib/left4me/installation`: shared L4D2 installation.
- `/var/lib/left4me/overlays`: overlay directories. Each overlay lives at `${overlay_id}` under here.
- `/var/lib/left4me/workshop_cache`: deduplicated cache of `.vpk` files downloaded for workshop overlays. One file per Steam item, named `{steam_id}.vpk`. Workshop overlays symlink into this tree.
- `/var/lib/left4me/global_overlay_cache`: cache of non-Steam map archives and extracted `.vpk` files used by managed global map overlays.
- `/var/lib/left4me/instances`: rendered instance specifications and per-instance state.
- `/var/lib/left4me/runtime`: per-instance runtime mount directories.
- `/var/lib/left4me/tmp`: temporary files used by deployment/runtime operations.
- `/usr/local/lib/systemd/system`: global systemd unit files, including `left4me-server@.service`.
- `/usr/local/libexec/left4me`: privileged helper commands, including `left4me-systemctl`, `left4me-journalctl`, and `left4me-overlay` (the latter mounts the per-instance kernel overlay in PID 1's mount namespace via `nsenter`).
- `/etc/sudoers.d/left4me`: sudoers rules allowing the web/runtime commands to call the helpers non-interactively.

Static units are generated for `/var/lib/left4me`. If `LEFT4ME_ROOT` changes, regenerate and reinstall the unit files instead of reusing the existing static units.

## Runtime User

The deployment creates and runs host operations as the dedicated runtime user:

- Username: `left4me`
- Home: `/var/lib/left4me`
- Shell: `/usr/sbin/nologin`

## Running A Test Deployment

Run the deployment from the repository root:

```bash
deploy/deploy-test-server.sh deploy-user@example-host
```

The SSH user must be able to run `sudo` on the target host. The deployment configures system packages, directories, environment files, helper scripts, sudoers rules, Python dependencies, and systemd units.

## Admin Bootstrap

Set the bootstrap credentials in the environment when creating the first admin user:

```bash
LEFT4ME_ADMIN_USERNAME=admin \
LEFT4ME_ADMIN_PASSWORD='change-me' \
flask create-user "$LEFT4ME_ADMIN_USERNAME" --admin
```

Use a strong one-time password and rotate it after first login if needed.

## Overlay References

Overlay references are relative paths below `${LEFT4ME_ROOT}/overlays`. With the default deployment root, they resolve under `/var/lib/left4me/overlays`. New overlays use `${overlay_id}` as their path; the digit-only form is the only one created by the web app.

Invalid references are rejected:

- Absolute paths such as `/srv/overlay`.
- Parent traversal such as `../other` or `competitive/../../base`.
- Empty path components such as `competitive//base`.
- Symlink escapes that resolve outside `${LEFT4ME_ROOT}/overlays`.

The web app currently supports two overlay surfaces:

- `workshop` overlays (user-owned) — populated by downloading `.vpk` files from the public Steam Web API into `${LEFT4ME_ROOT}/workshop_cache/{steam_id}.vpk` and creating absolute symlinks under `${LEFT4ME_ROOT}/overlays/{overlay_id}/left4dead2/addons/{steam_id}.vpk`.
- `script` overlays — populated by an arbitrary user-authored bash script that runs inside `bubblewrap` + `systemd-run --scope` as the unprivileged `l4d2-sandbox` UID, with the overlay directory bind-mounted RW at `/overlay`. Resource caps: 1h walltime, 4 GB RAM, 512 tasks, 200% CPU, 20 GB post-build disk cap.

Both the caches and the overlay directories are owned by the `left4me` runtime user; if the web service ever runs as a different uid, ensure it shares a group with the host process and that both trees are group-readable.

## Performance Tuning

The deployment ships a host-side perf baseline (slices, unit directives, sysctls). See `docs/superpowers/specs/2026-05-09-l4d2-server-host-perf-baseline-design.md` for design rationale.

The following knobs are documented escape hatches — they are **not** auto-applied. Apply only if you have measured a need and understand the failure modes.

### CPU governor

The performance governor squeezes a few percent off jitter under bursty load. `schedutil` is acceptable for sustained UDP workloads.

```sh
sudo cpupower frequency-set -g performance
```

Persist via your distro's CPU-frequency tooling (e.g. `/etc/default/cpufrequtils`).

### Per-instance CPU affinity

`srcds` is single-threaded per instance. On a multi-core host, pinning each instance to its own core can cut jitter under contention. Drop in `/etc/systemd/system/left4me-server@<name>.service.d/affinity.conf`:

```ini
[Service]
CPUAffinity=2
```

A reasonable strategy on an N-core host: leave core 0 for the kernel + IRQs + system services, then pin one instance per remaining core.

### NIC tuning

Hardware-specific. On a host with a single primary interface (replace `eth0`):

```sh
sudo ethtool -G eth0 rx 4096 tx 4096
sudo ethtool -K eth0 gro on lro off
```

If you run a high instance count, also pin the NIC's interrupts off the cores that game servers occupy (see `/proc/interrupts` and `/proc/irq/<n>/smp_affinity`).

### Real-time scheduling (advanced, opt-in)

Source-engine servers do not need real-time scheduling, and a misbehaving `srcds` at any RT priority can starve kernel threads — even with the default `kernel.sched_rt_runtime_us=950000` throttling 5% of CPU back. Use only if you have a measured jitter problem that the baseline does not solve.

`/etc/systemd/system/left4me-server@.service.d/realtime.conf`:

```ini
[Service]
CPUSchedulingPolicy=fifo
CPUSchedulingPriority=10
LimitRTPRIO=10
```

### Applying changes to running servers

Unit-file changes do not apply to already-running services. After any change:

```sh
sudo systemctl daemon-reload
# Restart each game server via the web UI's stop + start, or:
sudo systemctl restart 'left4me-server@*.service'
```