# L4D2 Server Host Perf Baseline Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Apply a host-side performance and resource-isolation baseline (systemd directives, slice hierarchy, host sysctls) to every L4D2 server instance, leaving game ConVars to the maintainer.

**Architecture:** Add resource-control directives to `left4me-server@.service`; introduce two flat top-level slices (`l4d2-game.slice` weight 1000, `l4d2-build.slice` weight 10) so the build sandbox is starved by the kernel under contention; ship `/etc/sysctl.d/99-left4me.conf` for UDP buffer and netdev tuning; place the script-sandbox transient unit into `l4d2-build.slice` with `OOMScoreAdjust=500`. RT scheduling, CPU governor, CPUAffinity, NIC tuning are documentation-only escape hatches.

**Tech Stack:** systemd unit files (service + slice), `systemd-run` properties, Linux sysctl, bash deploy script, pytest text-assertion tests under `deploy/tests/test_deploy_artifacts.py`.

**Spec:** `docs/superpowers/specs/2026-05-09-l4d2-server-host-perf-baseline-design.md`

---

## File Structure

Files to create:

- `deploy/files/usr/local/lib/systemd/system/l4d2-game.slice` — high-weight slice for game-server instances.
- `deploy/files/usr/local/lib/systemd/system/l4d2-build.slice` — low-weight slice for sandboxed script-overlay builds.
- `deploy/files/etc/sysctl.d/99-left4me.conf` — host UDP/netdev/swap sysctls.

Files to modify:

- `deploy/files/usr/local/lib/systemd/system/left4me-server@.service` — add resource-control directives (`Slice`, `Nice`, `IOSchedulingClass`, `IOSchedulingPriority`, `OOMScoreAdjust`, `MemoryHigh`, `MemoryMax`, `TasksMax`, `LimitNOFILE`, `KillSignal`, `TimeoutStopSec`, `LogRateLimitIntervalSec`).
- `deploy/files/usr/local/libexec/left4me/left4me-script-sandbox` — add `--slice=l4d2-build.slice` and `-p OOMScoreAdjust=500` to the `systemd-run` invocation.
- `deploy/deploy-test-server.sh` — copy the two slice files and the sysctl conf during deploy; run `sysctl --system` so values take effect immediately.
- `deploy/README.md` — append a "Performance tuning" section with the four documented escape hatches.
- `deploy/tests/test_deploy_artifacts.py` — new tests for each artifact above (text assertions following the existing `assert "X" in text` style).

No application code (Python, Flask, host library) is touched.

---

## Pre-flight

- [ ] **Step 0a: Verify clean working tree**

Run: `git status`
Expected: `nothing to commit, working tree clean`

- [ ] **Step 0b: Verify the existing deploy tests pass**

Run: `cd /Users/mwiegand/Projekte/left4me && pytest deploy/tests/test_deploy_artifacts.py -q`
Expected: all green.

If any test is already red, stop and surface — this plan assumes the baseline is green.

---

## Task 1: Per-Instance Unit Resource-Control Directives

Add the per-instance baseline to `left4me-server@.service`. This task is self-contained even though `Slice=l4d2-game.slice` references a slice that doesn't exist yet — systemd does not validate the reference until the unit is actually started, and the deploy artifact tests are pure text checks.

**Files:**
- Modify: `deploy/files/usr/local/lib/systemd/system/left4me-server@.service`
- Test: `deploy/tests/test_deploy_artifacts.py` (new test function)

- [ ] **Step 1.1: Add the failing test**

Open `deploy/tests/test_deploy_artifacts.py` and append (after `test_server_unit_contains_required_runtime_contract`):

```python
def test_server_unit_contains_perf_baseline_directives():
    unit = SERVER_UNIT.read_text()

    # Slice membership.
    assert "Slice=l4d2-game.slice" in unit

    # CFS priority bump (no SCHED_FIFO).
    assert "Nice=-5" in unit
    assert "CPUSchedulingPolicy=" not in unit

    # I/O priority.
    assert "IOSchedulingClass=best-effort" in unit
    assert "IOSchedulingPriority=4" in unit

    # OOM ordering: game servers survive, sandbox dies first.
    assert "OOMScoreAdjust=-200" in unit

    # Memory caps with headroom for map-load spikes.
    assert "MemoryHigh=1.5G" in unit
    assert "MemoryMax=2G" in unit

    # Bounded fork surface.
    assert "TasksMax=256" in unit

    # Plenty of fds for plugin-heavy setups.
    assert "LimitNOFILE=65536" in unit

    # srcds clean shutdown via SIGINT, with time to flush.
    assert "KillSignal=SIGINT" in unit
    assert "TimeoutStopSec=15s" in unit

    # Per-unit override of journald rate limiting (default drops srcds output).
    assert "LogRateLimitIntervalSec=0" in unit
```

- [ ] **Step 1.2: Run the new test, verify it fails**

Run: `cd /Users/mwiegand/Projekte/left4me && pytest deploy/tests/test_deploy_artifacts.py::test_server_unit_contains_perf_baseline_directives -v`
Expected: FAIL — first failing assert is on `Slice=l4d2-game.slice`.

- [ ] **Step 1.3: Edit the unit file**

Open `deploy/files/usr/local/lib/systemd/system/left4me-server@.service` and replace its contents with:

```ini
[Unit]
Description=left4me server instance %i
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=left4me
Group=left4me
EnvironmentFile=/etc/left4me/host.env
EnvironmentFile=/var/lib/left4me/instances/%i/instance.env
WorkingDirectory=/var/lib/left4me/runtime/%i/merged/left4dead2
ExecStart=/var/lib/left4me/installation/srcds_run -game left4dead2 +hostport ${L4D2_PORT} $L4D2_ARGS
Restart=on-failure
RestartSec=5

# Resource control baseline — see docs/superpowers/specs/2026-05-09-l4d2-server-host-perf-baseline-design.md
Slice=l4d2-game.slice
Nice=-5
IOSchedulingClass=best-effort
IOSchedulingPriority=4
OOMScoreAdjust=-200
MemoryHigh=1.5G
MemoryMax=2G
TasksMax=256
LimitNOFILE=65536
KillSignal=SIGINT
TimeoutStopSec=15s
LogRateLimitIntervalSec=0

# Hardening (unchanged from previous baseline).
NoNewPrivileges=true
PrivateTmp=true
PrivateDevices=true
ProtectHome=true
ProtectSystem=strict
ReadOnlyPaths=/var/lib/left4me/installation /var/lib/left4me/overlays
ReadWritePaths=/var/lib/left4me/runtime/%i
RestrictSUIDSGID=true
LockPersonality=true

[Install]
WantedBy=multi-user.target
```

- [ ] **Step 1.4: Run the new test, verify it passes**

Run: `cd /Users/mwiegand/Projekte/left4me && pytest deploy/tests/test_deploy_artifacts.py::test_server_unit_contains_perf_baseline_directives -v`
Expected: PASS.

- [ ] **Step 1.5: Re-run the existing server-unit test, verify still passes**

Run: `cd /Users/mwiegand/Projekte/left4me && pytest deploy/tests/test_deploy_artifacts.py::test_server_unit_contains_required_runtime_contract -v`
Expected: PASS — the existing assertions (`User=left4me`, `Group=left4me`, hardening directives, etc.) still match.

- [ ] **Step 1.6: Commit**

```bash
git add deploy/files/usr/local/lib/systemd/system/left4me-server@.service deploy/tests/test_deploy_artifacts.py
git commit -m "$(cat <<'EOF'
feat(deploy): perf-baseline directives on left4me-server@.service

Slice=l4d2-game.slice, Nice=-5, IOSchedulingClass=best-effort,
OOMScoreAdjust=-200, MemoryHigh=1.5G, MemoryMax=2G, TasksMax=256,
LimitNOFILE=65536, KillSignal=SIGINT, TimeoutStopSec=15s,
LogRateLimitIntervalSec=0. Spec:
docs/superpowers/specs/2026-05-09-l4d2-server-host-perf-baseline-design.md
EOF
)"
```

---

## Task 2: Slice Unit Files

Create the two slice unit files. After this task the perf unit's `Slice=l4d2-game.slice` reference is satisfied.

**Files:**
- Create: `deploy/files/usr/local/lib/systemd/system/l4d2-game.slice`
- Create: `deploy/files/usr/local/lib/systemd/system/l4d2-build.slice`
- Test: `deploy/tests/test_deploy_artifacts.py` (new constants + new test functions)

- [ ] **Step 2.1: Add path constants and failing tests**

Open `deploy/tests/test_deploy_artifacts.py`. After the existing `SERVER_UNIT = ...` line, add:

```python
GAME_SLICE = DEPLOY / "files/usr/local/lib/systemd/system/l4d2-game.slice"
BUILD_SLICE = DEPLOY / "files/usr/local/lib/systemd/system/l4d2-build.slice"
```

After the new `test_server_unit_contains_perf_baseline_directives`, append:

```python
def test_l4d2_game_slice_exists_with_high_weights():
    assert GAME_SLICE.is_file()
    text = GAME_SLICE.read_text()
    assert "[Slice]" in text
    assert "CPUWeight=1000" in text
    assert "IOWeight=1000" in text


def test_l4d2_build_slice_exists_with_low_weights():
    assert BUILD_SLICE.is_file()
    text = BUILD_SLICE.read_text()
    assert "[Slice]" in text
    assert "CPUWeight=10" in text
    assert "IOWeight=10" in text
```

- [ ] **Step 2.2: Run the new tests, verify they fail**

Run: `cd /Users/mwiegand/Projekte/left4me && pytest deploy/tests/test_deploy_artifacts.py::test_l4d2_game_slice_exists_with_high_weights deploy/tests/test_deploy_artifacts.py::test_l4d2_build_slice_exists_with_low_weights -v`
Expected: FAIL on `assert GAME_SLICE.is_file()` (file does not exist).

- [ ] **Step 2.3: Create the game slice file**

Create `deploy/files/usr/local/lib/systemd/system/l4d2-game.slice` with:

```ini
[Unit]
Description=left4me game-server slice
Before=slices.target

[Slice]
CPUWeight=1000
IOWeight=1000
```

- [ ] **Step 2.4: Create the build slice file**

Create `deploy/files/usr/local/lib/systemd/system/l4d2-build.slice` with:

```ini
[Unit]
Description=left4me script-sandbox build slice
Before=slices.target

[Slice]
CPUWeight=10
IOWeight=10
```

- [ ] **Step 2.5: Run the new tests, verify they pass**

Run: `cd /Users/mwiegand/Projekte/left4me && pytest deploy/tests/test_deploy_artifacts.py::test_l4d2_game_slice_exists_with_high_weights deploy/tests/test_deploy_artifacts.py::test_l4d2_build_slice_exists_with_low_weights -v`
Expected: PASS.

- [ ] **Step 2.6: Commit**

```bash
git add deploy/files/usr/local/lib/systemd/system/l4d2-game.slice deploy/files/usr/local/lib/systemd/system/l4d2-build.slice deploy/tests/test_deploy_artifacts.py
git commit -m "$(cat <<'EOF'
feat(deploy): l4d2-game.slice + l4d2-build.slice with 100:1 weight ratio

Flat top-level slices. Game wins under contention; build still gets
the box when uncontended. Referenced by left4me-server@.service and
the script-sandbox systemd-run invocation.
EOF
)"
```

---

## Task 3: Host Sysctls

Ship a `/etc/sysctl.d/` drop-in for UDP buffers, netdev backlog, netdev budget, and `vm.swappiness`.

**Files:**
- Create: `deploy/files/etc/sysctl.d/99-left4me.conf`
- Test: `deploy/tests/test_deploy_artifacts.py` (new constant + new test function)

- [ ] **Step 3.1: Add path constant and failing test**

Open `deploy/tests/test_deploy_artifacts.py`. After the slice constants, add:

```python
SYSCTL_CONF = DEPLOY / "files/etc/sysctl.d/99-left4me.conf"
```

Append a new test:

```python
def test_sysctl_conf_present_with_perf_settings():
    assert SYSCTL_CONF.is_file()
    text = SYSCTL_CONF.read_text()
    for line in (
        "net.core.rmem_max = 8388608",
        "net.core.wmem_max = 8388608",
        "net.core.rmem_default = 524288",
        "net.core.wmem_default = 524288",
        "net.core.netdev_max_backlog = 5000",
        "net.core.netdev_budget = 600",
        "vm.swappiness = 10",
    ):
        assert line in text, f"missing {line!r} in 99-left4me.conf"
```

- [ ] **Step 3.2: Run the new test, verify it fails**

Run: `cd /Users/mwiegand/Projekte/left4me && pytest deploy/tests/test_deploy_artifacts.py::test_sysctl_conf_present_with_perf_settings -v`
Expected: FAIL on `assert SYSCTL_CONF.is_file()`.

- [ ] **Step 3.3: Create the sysctl conf file**

Create `deploy/files/etc/sysctl.d/99-left4me.conf` with:

```
# Host-side perf baseline for left4me — see
# docs/superpowers/specs/2026-05-09-l4d2-server-host-perf-baseline-design.md
#
# UDP socket buffers: distro defaults of ~128 KiB are too small for sustained
# Source-engine UDP across multiple instances. 8 MiB matches the standard
# 1 Gbit recommendation; rmem_default/wmem_default protect sockets that don't
# explicitly enlarge their buffers.
net.core.rmem_max = 8388608
net.core.wmem_max = 8388608
net.core.rmem_default = 524288
net.core.wmem_default = 524288

# Kernel softirq UDP path: the per-CPU backlog queue starts dropping packets
# at the default 1000 under multi-instance burst; 5000 absorbs realistic peaks.
# netdev_budget = 600 gives softirq more drain headroom per pass.
net.core.netdev_max_backlog = 5000
net.core.netdev_budget = 600

# Latency-sensitive default: avoid swap unless the box is really under
# pressure. Harmless on swapless hosts.
vm.swappiness = 10
```

- [ ] **Step 3.4: Run the new test, verify it passes**

Run: `cd /Users/mwiegand/Projekte/left4me && pytest deploy/tests/test_deploy_artifacts.py::test_sysctl_conf_present_with_perf_settings -v`
Expected: PASS.

- [ ] **Step 3.5: Commit**

```bash
git add deploy/files/etc/sysctl.d/99-left4me.conf deploy/tests/test_deploy_artifacts.py
git commit -m "$(cat <<'EOF'
feat(deploy): host sysctls for UDP buffers + netdev backlog/budget

99-left4me.conf: rmem_max/wmem_max=8M (with 512K defaults),
netdev_max_backlog=5000, netdev_budget=600, vm.swappiness=10.
EOF
)"
```

---

## Task 4: Sandbox in Build Slice

Place the script-sandbox transient unit into `l4d2-build.slice` and give it `OOMScoreAdjust=500` so it dies first under memory pressure.

**Files:**
- Modify: `deploy/files/usr/local/libexec/left4me/left4me-script-sandbox`
- Test: `deploy/tests/test_deploy_artifacts.py` (new test function)

- [ ] **Step 4.1: Add the failing test**

Open `deploy/tests/test_deploy_artifacts.py`. Append:

```python
def test_script_sandbox_in_build_slice_with_oom_adjust():
    text = SCRIPT_SANDBOX_HELPER.read_text()

    # Put the transient unit in the low-weight build slice so it yields to
    # game-server instances under CPU/IO contention.
    assert "--slice=l4d2-build.slice" in text

    # Sandbox dies first if the host hits memory pressure; servers
    # (OOMScoreAdjust=-200) survive.
    assert "-p OOMScoreAdjust=500" in text
```

- [ ] **Step 4.2: Run the new test, verify it fails**

Run: `cd /Users/mwiegand/Projekte/left4me && pytest deploy/tests/test_deploy_artifacts.py::test_script_sandbox_in_build_slice_with_oom_adjust -v`
Expected: FAIL — neither string is in the helper yet.

- [ ] **Step 4.3: Edit the sandbox helper**

Open `deploy/files/usr/local/libexec/left4me/left4me-script-sandbox`. Locate the `systemd-run` invocation that begins with:

```
systemd-run --quiet --collect --wait --pipe \
    --unit="left4me-script-${OVERLAY_ID}-$$" \
```

Insert two new lines immediately after the `--unit=` line, before `-p User=l4d2-sandbox`. The block becomes:

```
systemd-run --quiet --collect --wait --pipe \
    --unit="left4me-script-${OVERLAY_ID}-$$" \
    --slice=l4d2-build.slice \
    -p OOMScoreAdjust=500 \
    -p User=l4d2-sandbox -p Group=l4d2-sandbox \
```

Leave every other `-p` line untouched.

- [ ] **Step 4.4: Verify shell syntax still parses**

Run: `bash -n deploy/files/usr/local/libexec/left4me/left4me-script-sandbox`
Expected: exit 0, no output.

- [ ] **Step 4.5: Run the new test and the existing sandbox-helper tests, verify they pass**

Run: `cd /Users/mwiegand/Projekte/left4me && pytest deploy/tests/test_deploy_artifacts.py::test_script_sandbox_in_build_slice_with_oom_adjust deploy/tests/test_deploy_artifacts.py::test_script_sandbox_helper_invokes_systemd_run_with_hardening deploy/tests/test_deploy_artifacts.py::test_script_sandbox_helper_passes_shell_syntax_check -v`
Expected: PASS for all three. The hardening test still matches because it only checks for substring presence; we added strings, didn't remove any.

- [ ] **Step 4.6: Commit**

```bash
git add deploy/files/usr/local/libexec/left4me/left4me-script-sandbox deploy/tests/test_deploy_artifacts.py
git commit -m "$(cat <<'EOF'
feat(deploy): script-sandbox runs in l4d2-build.slice + OOMScoreAdjust=500

Builds yield CPU/IO to game-server instances under contention via the
slice's weight=10, and are killed first under memory pressure
(servers have OOMScoreAdjust=-200).
EOF
)"
```

---

## Task 5: Deploy Script Installs Slice + Sysctl Artifacts

Wire the new artifacts into `deploy-test-server.sh` so a fresh deploy actually puts them on disk and applies the sysctls.

**Files:**
- Modify: `deploy/deploy-test-server.sh`
- Test: `deploy/tests/test_deploy_artifacts.py` (new test function)

- [ ] **Step 5.1: Add the failing test**

Open `deploy/tests/test_deploy_artifacts.py`. Append:

```python
def test_deploy_script_installs_perf_artifacts():
    script = DEPLOY_SCRIPT.read_text()

    # Slice files copied into the system-wide systemd unit dir.
    assert "/usr/local/lib/systemd/system/l4d2-game.slice" in script
    assert "/usr/local/lib/systemd/system/l4d2-build.slice" in script

    # Sysctl drop-in installed under /etc/sysctl.d/.
    assert "/etc/sysctl.d/99-left4me.conf" in script

    # Values applied immediately, not on next boot.
    assert "sysctl --system" in script
```

- [ ] **Step 5.2: Run the new test, verify it fails**

Run: `cd /Users/mwiegand/Projekte/left4me && pytest deploy/tests/test_deploy_artifacts.py::test_deploy_script_installs_perf_artifacts -v`
Expected: FAIL on the first assertion.

- [ ] **Step 5.3: Edit the deploy script — copy the slice + sysctl files**

Open `deploy/deploy-test-server.sh`. Find the block that copies unit files (currently around line 138):

```sh
$sudo_cmd cp /opt/left4me/deploy/files/usr/local/lib/systemd/system/left4me-web.service /usr/local/lib/systemd/system/left4me-web.service
$sudo_cmd cp /opt/left4me/deploy/files/usr/local/lib/systemd/system/left4me-server@.service /usr/local/lib/systemd/system/left4me-server@.service
```

Add two new lines immediately after the `left4me-server@.service` copy line, so the block becomes:

```sh
$sudo_cmd cp /opt/left4me/deploy/files/usr/local/lib/systemd/system/left4me-web.service /usr/local/lib/systemd/system/left4me-web.service
$sudo_cmd cp /opt/left4me/deploy/files/usr/local/lib/systemd/system/left4me-server@.service /usr/local/lib/systemd/system/left4me-server@.service
$sudo_cmd cp /opt/left4me/deploy/files/usr/local/lib/systemd/system/l4d2-game.slice /usr/local/lib/systemd/system/l4d2-game.slice
$sudo_cmd cp /opt/left4me/deploy/files/usr/local/lib/systemd/system/l4d2-build.slice /usr/local/lib/systemd/system/l4d2-build.slice
```

- [ ] **Step 5.4: Edit the deploy script — install the sysctl conf and apply it**

In `deploy/deploy-test-server.sh`, find the block that installs `/etc/left4me/sandbox-resolv.conf` (currently around lines 153–155):

```sh
$sudo_cmd install -m 0644 -o root -g root \
    /opt/left4me/deploy/files/etc/left4me/sandbox-resolv.conf \
    /etc/left4me/sandbox-resolv.conf
```

Immediately after that block, add:

```sh
# Host perf-baseline sysctls. Apply with `sysctl --system` so values
# take effect this deploy, not on next reboot.
$sudo_cmd install -m 0644 -o root -g root \
    /opt/left4me/deploy/files/etc/sysctl.d/99-left4me.conf \
    /etc/sysctl.d/99-left4me.conf
$sudo_cmd sysctl --system >/dev/null
```

- [ ] **Step 5.5: Verify the deploy script's shell syntax still parses**

Run: `sh -n deploy/deploy-test-server.sh`
Expected: exit 0, no output.

- [ ] **Step 5.6: Run the new test and the existing deploy-script tests, verify they pass**

Run: `cd /Users/mwiegand/Projekte/left4me && pytest deploy/tests/test_deploy_artifacts.py::test_deploy_script_installs_perf_artifacts deploy/tests/test_deploy_artifacts.py::test_deploy_script_has_safe_defaults_and_preserves_state deploy/tests/test_deploy_artifacts.py::test_deploy_script_shell_syntax -v`
Expected: PASS for all three.

- [ ] **Step 5.7: Commit**

```bash
git add deploy/deploy-test-server.sh deploy/tests/test_deploy_artifacts.py
git commit -m "$(cat <<'EOF'
feat(deploy): install slice + sysctl artifacts and apply via sysctl --system

Copies l4d2-game.slice and l4d2-build.slice into
/usr/local/lib/systemd/system/, installs 99-left4me.conf into
/etc/sysctl.d/, and runs sysctl --system so the perf baseline is
live this deploy, not on next reboot.
EOF
)"
```

---

## Task 6: Performance-Tuning Section in deploy/README.md

Document the four escape hatches the spec lists as opt-in: CPU governor, per-instance `CPUAffinity`, NIC tuning, and SCHED_FIFO.

**Files:**
- Modify: `deploy/README.md`

No test for this task — README content is documentation, not contract.

- [ ] **Step 6.1: Append the Performance Tuning section**

Open `deploy/README.md`. Append (after the existing final paragraph) a new section:

```markdown
## Performance Tuning

The deployment ships a host-side perf baseline (slices, unit directives, sysctls). See `docs/superpowers/specs/2026-05-09-l4d2-server-host-perf-baseline-design.md` for design rationale.

The following knobs are documented escape hatches — they are **not** auto-applied. Apply only if you have measured a need and understand the failure modes.

### CPU governor

The performance governor squeezes a few percent off jitter under bursty load. `schedutil` is acceptable for sustained UDP workloads.

```sh
sudo cpupower frequency-set -g performance
```

Persist via your distro's CPU-frequency tooling (e.g. `/etc/default/cpufrequtils`).

### Per-instance CPU affinity

`srcds` is single-threaded per instance. On a multi-core host, pinning each instance to its own core can cut jitter under contention. Drop in `/etc/systemd/system/left4me-server@<name>.service.d/affinity.conf`:

```ini
[Service]
CPUAffinity=2
```

A reasonable strategy on an N-core host: leave core 0 for the kernel + IRQs + system services, then pin one instance per remaining core.

### NIC tuning

Hardware-specific. On a host with a single primary interface (replace `eth0`):

```sh
sudo ethtool -G eth0 rx 4096 tx 4096
sudo ethtool -K eth0 gro on lro off
```

If you run a high instance count, also pin the NIC's interrupts off the cores that game servers occupy (see `/proc/interrupts` and `/proc/irq/<n>/smp_affinity`).

### Real-time scheduling (advanced, opt-in)

Source-engine servers do not need real-time scheduling, and a misbehaving `srcds` at any RT priority can starve kernel threads — even with the default `kernel.sched_rt_runtime_us=950000` throttling 5% of CPU back. Use only if you have a measured jitter problem that the baseline does not solve.

`/etc/systemd/system/left4me-server@.service.d/realtime.conf`:

```ini
[Service]
CPUSchedulingPolicy=fifo
CPUSchedulingPriority=10
LimitRTPRIO=10
```

### Applying changes to running servers

Unit-file changes do not apply to already-running services. After any change:

```sh
sudo systemctl daemon-reload
# Restart each game server via the web UI's stop + start, or:
sudo systemctl restart 'left4me-server@*.service'
```
```

- [ ] **Step 6.2: Run the full deploy test suite and verify it stays green**

Run: `cd /Users/mwiegand/Projekte/left4me && pytest deploy/tests/test_deploy_artifacts.py -q`
Expected: all green. README changes have no test, but should not break any existing tests.

- [ ] **Step 6.3: Commit**

```bash
git add deploy/README.md
git commit -m "$(cat <<'EOF'
docs(deploy): performance-tuning escape-hatch section in README

Documents CPU governor, per-instance CPUAffinity, NIC tuning, and
SCHED_FIFO opt-in patterns. None of these are auto-applied; they're
ops-side knobs for measured problems the perf baseline doesn't solve.
EOF
)"
```

---

## Final Verification

- [ ] **Step F.1: Full deploy test suite green**

Run: `cd /Users/mwiegand/Projekte/left4me && pytest deploy/tests/ -q`
Expected: all green.

- [ ] **Step F.2: Host library + web tests still green (regression check)**

Run: `cd /Users/mwiegand/Projekte/left4me && pytest l4d2host/tests -q && pytest l4d2web/tests -q`
Expected: all green. Nothing in this plan touches host or web Python code, but a clean run rules out accidental import-time damage.

- [ ] **Step F.3: Working tree clean and commits in order**

Run: `git status && git log --oneline -8`
Expected:
- `git status`: `nothing to commit, working tree clean`.
- `git log`: six new commits in this order, top-most first:
  1. `docs(deploy): performance-tuning escape-hatch section in README`
  2. `feat(deploy): install slice + sysctl artifacts and apply via sysctl --system`
  3. `feat(deploy): script-sandbox runs in l4d2-build.slice + OOMScoreAdjust=500`
  4. `feat(deploy): host sysctls for UDP buffers + netdev backlog/budget`
  5. `feat(deploy): l4d2-game.slice + l4d2-build.slice with 100:1 weight ratio`
  6. `feat(deploy): perf-baseline directives on left4me-server@.service`

If any step is missing or out of order, do not amend — diagnose, fix, and create new commits.

- [ ] **Step F.4: Manual deploy smoke test (deferred, ops-side)**

This plan ships artifacts. Confirming that systemd actually accepts and applies them on a real host requires running the deploy script against a test target. That validation is operator-side, not part of this implementation:

```sh
deploy/deploy-test-server.sh deploy-user@example-host
ssh deploy-user@example-host 'systemctl cat l4d2-game.slice'
ssh deploy-user@example-host 'sysctl net.core.rmem_max'   # expect 8388608
ssh deploy-user@example-host 'systemd-analyze verify /usr/local/lib/systemd/system/left4me-server@.service'
```

Document any deploy-time problems back into the spec or this plan as v1.x corrections. Do not invent fixes that go beyond the spec.

---

## Out of Scope (do NOT implement here)

Listed in the spec — repeated for clarity:

- ConVars / blueprint arguments / tickrate / sv_minrate.
- SCHED_FIFO auto-apply.
- CPU governor auto-apply.
- Per-instance `CPUAffinity` auto-apply.
- NIC ring-buffer / IRQ-pinning code.
- Job-scheduler awareness ("don't build while server X has players").
- Hardening tightening (`ProtectKernelTunables=yes`, etc.).

If you find yourself touching any of these, stop — they belong in a separate spec.