docs(deploy): document network-shaping defaults + opt-in network knobs

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
mwiegand 2026-05-10 01:09:28 +02:00
parent 26f3d270b0
commit 9f0b51b455
No known key found for this signature in database

View file

@ -78,6 +78,51 @@ The deployment ships a host-side perf baseline (slices, unit directives, sysctls
The following knobs are documented escape hatches — they are **not** auto-applied. Apply only if you have measured a need and understand the failure modes.
### Network shaping
The deploy ships three things that affect player-experience network behaviour:
1. **Per-flow marking.** `left4me-nft-mark.service` loads a small nftables
table (`inet left4me_mark`) that marks every UDP packet from uid `left4me`
with DSCP EF and `skb->priority` 6. srcds doesn't set these itself, so
without this rule its UDP is indistinguishable from any other flow.
2. **Sysctl baseline.** `99-left4me.conf` sets `udp_rmem_min=16384`,
`udp_wmem_min=16384`, `default_qdisc=fq_codel`, and
`tcp_congestion_control=bbr`. Reduces head-of-line blocking when bulk
TCP egress (backups, package fetches, web responses) coexists with
game UDP.
3. **CAKE egress shaping.** `left4me-cake.service` runs
`tc qdisc replace dev <iface> root cake bandwidth Xmbit internet
diffserv4 dual-dsthost` from `/etc/left4me/cake.env`. CAKE only shapes
if its declared bandwidth is **below** the real bottleneck, so set
`LEFT4ME_UPLINK_MBIT` to ≈95% of measured uplink:
sudoedit /etc/left4me/cake.env
# set LEFT4ME_UPLINK_MBIT=480 (or whatever ~95% of your uplink is)
sudo systemctl restart left4me-cake.service
`LEFT4ME_UPLINK_IFACE` is auto-detected from the IPv4 default route;
override only on hosts with multi-homed setups.
At idle 500 Mbit with no competing egress, CAKE shapes nothing — that's
expected, not a bug. The win materialises when bulk traffic on the
same uplink would otherwise bufferbloat the link the players share.
**Production hosts running `systemd-networkd`** should NOT use the
`left4me-cake.service` oneshot. Instead, configure the equivalent in the
matching `.network` file, which systemd-networkd reapplies across iface
lifecycle events:
# /etc/systemd/network/<your-uplink>.network
[CAKE]
Bandwidth=480M
OverheadKeyword=internet
PriorityQueueingPreset=diffserv4
EgressHostIsolation=yes
The nftables marking from (1) is qdisc-installer-agnostic and ships
unchanged on production.
### CPU governor
The performance governor squeezes a few percent off jitter under bursty load. `schedutil` is acceptable for sustained UDP workloads.
@ -144,6 +189,31 @@ AmbientCapabilities=CAP_SYS_NICE
The `AmbientCapabilities=CAP_SYS_NICE` line is needed because the service runs as `User=left4me` with `NoNewPrivileges=true`; without it some kernels/systemd combinations refuse to apply the RT policy.
### Additional opt-in network knobs
- **Ingress shaping via IFB.** Egress CAKE alone does not protect srcds
receive against ingress saturation (large workshop downloads, package
fetches arriving at line rate). One-liner:
sudo modprobe ifb && sudo ip link set ifb0 up
sudo tc qdisc add dev <uplink> handle ffff: ingress
sudo tc filter add dev <uplink> parent ffff: protocol ip u32 \
match u32 0 0 action mirred egress redirect dev ifb0
sudo tc qdisc add dev ifb0 root cake bandwidth Xmbit ingress \
diffserv4 dual-srchost
Worth flipping only when measurement shows ingress hurting receive.
- **`net.core.busy_poll = 50` / `net.core.busy_read = 50`.** Reduces UDP
receive median latency by polling for incoming packets briefly at
syscall boundaries. Cost: measurable CPU per syscall under load. Worth
flipping if a host is dedicated to game serving and CPU headroom is
plentiful.
- **`ethtool -K <iface> gro off`.** Some Source-engine ops disable
generic receive offload to avoid receive-side coalescing latency.
Hardware/driver dependent; document only.
### Applying changes to running servers
Unit-file changes do not apply to already-running services. After any change: