# l4d2 cpu pinning — decision record (deferred)

Date: 2026-05-09
Status: decision (no implementation)

## Question

After the lifecycle + drift fix landed (commits `8552c55`, `67b5521`), the
question came up: with `AllowedCPUs=1-7` already constraining game servers
to cores 1–7, do CFS scheduler migrations *within* that range still cause
meaningful jitter? Should we hard-pin each instance to a single core?

## Investigation

The classic "lazy CFS" sysctl knob is **gone** on modern kernels. Verified
on Trixie's running kernel 6.12 (`ckn@10.0.4.128`):

```
/sbin/sysctl -a | grep -E "sched_migration_cost|sched_min_granularity|sched_wakeup_granularity|sched_latency"
# (no output)
```

`kernel.sched_migration_cost_ns` and the other classic CFS tunables were
removed in 5.13+ as part of the scheduler internals refactor that culminated
in EEVDF (6.6). Only `kernel.sched_rt_period_us` / `sched_rt_runtime_us`
remain. There is no global "be lazy about migrations" knob anymore.

### Available paths

| Option | Cost | Strictness | Pays off when |
|---|---|---|---|
| Trust CFS + `Nice=-5` + `AllowedCPUs=1-7` (current) | None | Soft | ≤ 3 instances on 7 cores; CFS rarely migrates active CPU-bound nice<0 tasks |
| Per-instance `CPUAffinity=N` drop-in | Web-app machinery to write drop-ins, daemon-reload, modulo or DB-persisted assignment | Strict | ≥ 4 instances (each gets exclusive core), or measured jitter |
| `isolcpus=1-7 nohz_full=1-7 rcu_nocbs=1-7` kernel cmdline | GRUB edit + reboot, host-specific | Strongest (also evicts kernel softirqs/RCU/timer ticks from game cores) | Tickrate-128 with measurable kernel-induced jitter |
| `SCHED_FIFO` per unit | Risky (RT misconfig can stall kernel) | Strict | Already documented as ops-side escape hatch in `deploy/README.md` |

### Why deferring is defensible

- The slice's `AllowedCPUs=1-7` already prevents game servers from running on core 0. The open question is "do they migrate within 1–7?" — yes, CFS can migrate, but for long-running CPU-bound `srcds` with `Nice=-5`, migrations are infrequent. CFS prefers cache locality and only migrates when an idle core "steals" or a periodic load-balance tick detects imbalance.
- With ≤ 3 instances on 7 game cores, the load balancer rarely sees imbalance to fix.
- Per-instance hard pinning adds non-trivial machinery (drop-in writer through `left4me-systemctl`, or extending `instance.env` + a `taskset` wrapper in the unit). Not warranted unless we observe a real problem.
- `deploy/README.md` already documents the `CPUAffinity=N` per-instance drop-in as an opt-in escape hatch. An operator who measures jitter can apply it without code changes.

## Decision

**No code change.** Keep the current setup:

- Slice-level `AllowedCPUs=1-7` ensures game servers never touch core 0.
- `Nice=-5` keeps active srcds tasks weighted heavily so CFS prefers leaving them alone.
- The `CPUAffinity=N` per-instance drop-in remains the documented escape hatch.

## Revisit triggers

Any of these signals appears, then design + implement strict per-instance pinning:

- ≥ 4 game-server instances running simultaneously on one host.
- A specific server reports tickrate dips / rubber-banding correlated with another instance starting or a build sandbox firing.
- `perf stat -e sched:sched_migrate_task -p <srcds-pid>` shows > 1 migration/sec under load.

When revisiting, two implementation paths to choose from:

1. **Modulo assignment in the host library.** Read `LEFT4ME_GAME_CPUS` (or parse the slice's `AllowedCPUs=` drop-in), pick `game_cpus[(int(name) - 1) % len(game_cpus)]`, write `L4D2_CPU=N` into `instance.env`, wrap the unit's `ExecStart` with `taskset -c ${L4D2_CPU}`. Stateless, deterministic, no DB column. **Preferred.**
2. **Persisted assignment.** Add `Server.cpu_pin` column, web app picks at initialize time and stores. Survives `LEFT4ME_GAME_CPUS` changes (each server keeps its assigned core). Bigger ripple.

## Verification (no-op confirmation)

```sh
ssh ckn@10.0.4.128 'systemctl show l4d2-game.slice -p AllowedCPUs'
# expect: AllowedCPUs=1-7

ssh ckn@10.0.4.128 'cat /sys/fs/cgroup/system.slice/cpuset.cpus.effective'
# expect: 0   (everything-not-game still pinned to core 0)

# When ≥ 1 server is running:
ssh ckn@10.0.4.128 'for p in $(pgrep srcds); do grep ^Cpus_allowed_list /proc/$p/status; done'
# expect: 1-7   (CFS picks whichever of those is hottest at any given moment)
```

## References

- `docs/superpowers/specs/2026-05-09-l4d2-cpu-isolation-design.md` — sibling design that introduced the `AllowedCPUs=1-7` slice constraint this record builds on.
- `deploy/README.md` "Performance Tuning" section — the `CPUAffinity=N` per-instance escape hatch.
- Linux kernel changelog 5.13+ — removal of classic CFS tunable sysctls.
