Commit graph

1217 commits

Author SHA1 Message Date
0a9f3dae88
agents/commands: read-only command cheat sheet
Adds a flag-combinations table for bw test (selectors + the
HIJKMSp/IJKMp default-flag split), bw metadata -k/-b/-f (with
the -f sensitive-data warning), bw items --blame/-f, bw verify
-o bundle:, bw hash -m/-d. Also documents the shared target-
selector grammar.

Surfaced by the left4me-integration session, where the agent
relied on bare bw test / bw metadata / bw items invocations and
missed leverage from the available flags.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 20:26:27 +02:00
422a275d97
agents: drop bw bundles, add bw verify to read-only allowlist
bw bundles is not a subcommand of the installed fork (the actual
list is apply/debug/diff/groups/hash/ipmi/items/lock/metadata/
nodes/plot/pw/repo/run/stats/test/verify/zen). bw verify is
read-only and was missing from the list.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 20:25:43 +02:00
3ed0264be6
docs/specs: round-1 agents-md refactor design (gaps 1-6)
Captures the brainstorm + per-commit wording for the first six
gaps from the left4me-integration handoff, plus a side-quest
read-only command cheat sheet for docs/agents/commands.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 20:24:03 +02:00
d49259ff07
nginx: move 80.conf to sites-available so it's actually included
The bundle was shipping 80.conf (HTTP-to-HTTPS redirect + acme-challenge
alias) to /etc/nginx/sites/80.conf, but nginx.conf only `include`s
/etc/nginx/sites-enabled/* (which is a symlink to sites-available).
The file was orphaned — no node had a working port-80 listener.

Move the destination to /etc/nginx/sites-available/80.conf so the
existing sites-enabled symlink picks it up. The /etc/nginx purge will
clean up any stale /etc/nginx/sites/80.conf on existing hosts.
2026-05-10 19:59:17 +02:00
ed141a9300
left4me: drop chown_src from git_deploy triggers (self-healing now)
Same constraint pattern: items in a triggers list must be
triggered:True. chown_src dropped triggered:True in the prior commit
to become self-healing every-apply, so it can't stay in git_deploy's
triggers list. Now git_deploy has no triggers at all — chown_src and
pip_install both run every apply, gated by their own `unless` guards.
2026-05-10 18:58:30 +02:00
9d17c69b22
left4me: make chown_src self-healing too
Same problem as pip_install: chown_src was triggered:True and only
fired when git_deploy did. After a partial first-apply where git_deploy
succeeded (extracting root-owned files) but the chown didn't happen
yet, subsequent applies left files root-owned forever — pip_install
fails with "permission denied" trying to write .egg-info/.

Drop triggered:True. Add an unless guard:
  test -z "$(find /opt/left4me/src ! -user left4me -print -quit)"
i.e. skip the chown only when no non-left4me-owned file exists in the
tree.
2026-05-10 18:57:50 +02:00
5bf95cb065
left4me: drop pip_install from pip_upgrade triggers (pip_install now always-runs) 2026-05-10 18:56:30 +02:00
cac04a456b
left4me: make pip_install self-healing on every apply
The previous shape (`triggered: True`, in git_deploy's triggers list)
meant pip_install only ran when something upstream fired. After a
partial first-apply failure (where git_deploy succeeded but pip_install
failed for an unrelated reason), subsequent applies couldn't recover —
git_deploy was already in desired state, nothing fired pip_install.

Drop `triggered: True`. Drop pip_install from git_deploy's triggers
(bw enforces a triggers→triggered:True invariant). Add `unless`:
sudo -u left4me /opt/left4me/.venv/bin/python -c "import l4d2host, l4d2web"
to short-circuit when the venv is already correct. Editable installs
pick up code changes automatically — no need to re-pip on every git
update.

For dep changes (rare), nudge manually:
  bw run ovh.left4me 'sudo -u left4me /opt/left4me/.venv/bin/pip install -e /opt/left4me/src/l4d2host -e /opt/left4me/src/l4d2web'
2026-05-10 18:55:24 +02:00
c2cc3866f3
left4me: chown /opt/left4me/src after git_deploy
bw's git_deploy extracts the git archive as the connecting user (root
after sudo), so files end up root-owned. The subsequent pip install
runs as left4me and needs to write .egg-info/ inside each editable
package, which fails with "permission denied".

Add action:left4me_chown_src triggered by git_deploy and required by
pip_install. Idempotent (chown -R is fine to re-run).
2026-05-10 18:52:37 +02:00
d548235dfe
left4me: declare /opt/left4me/src as a directory: item
bw's git_deploy item assumes the destination directory exists on the
host — its fix path runs `find <dest> -mindepth 1 -delete` to clear
existing contents before unpacking the new archive, which fails on a
fresh box where the directory was never created. Flask follows the
same pattern (bundles/flask/items.py:13).
2026-05-10 18:51:05 +02:00
149ce6c870
left4me: use https git URL so bw clones locally per-apply
bw's git_deploy.py:103 falls into a per-apply temp clone path when the
repo URL contains '://' (HTTPS, ssh://, …). Without that, it requires
a static git_deploy_repos map file pointing at a long-lived local
clone — which is the wrong shape for left4me, where the source of
truth is git.sublimity.de.

Switching the default to the HTTPS URL means anyone with the bundle
gets a working clone-from-source on `bw apply`, no operator-side
mirror map required.

Note: the host will pull whatever is pushed to git.sublimity.de
master. Push local commits before applying.
2026-05-10 18:49:10 +02:00
0479c96ae9
gitignore: add bundlewrap git_deploy_repos map (operator-specific paths) 2026-05-10 18:43:59 +02:00
5d69180466
left4me: terse bundle-membership asserts 2026-05-10 18:34:09 +02:00
7d3554f8a5
left4me: split derived_from_domain into one reactor per consumer
Each reactor now scopes to a single downstream bundle:
  nginx_vhosts    -> nginx/vhosts
  nftables_input  -> nftables/input

Easier to grep "what writes nginx/vhosts" and harder to accidentally
couple unrelated keys together. Same merged metadata.
2026-05-10 18:33:11 +02:00
fc66267656
left4me: reuse nginx bundle's auto-monitoring via check_path
bundles/nginx/metadata.py:91-104 already creates a monitoring/services
entry per nginx/vhost using the vhost's check_protocol/check_path. Set
check_path: '/health' on the left4me vhost so the auto-check hits the
Flask health endpoint, drop the explicit monitoring/services/left4me-web
block from this reactor.

Net effect: same curl command lands in monitoring as before, but the
service name is now 'left4.me' (the hostname, per the nginx reactor's
naming convention) instead of 'left4me-web'.
2026-05-10 18:31:52 +02:00
758660b131
left4me: drop redundant letsencrypt/domains from reactor
bundles/nginx/metadata.py auto-populates letsencrypt/domains from
nginx/vhosts.keys(). Declaring it again in the left4me reactor was a
no-op duplication. Removed; bw metadata still shows the same merged
state (left4.me with reload: [nginx]).
2026-05-10 18:29:15 +02:00
7b291acca1
left4me: refresh README + opt ovh.left4me in via groups
README:
  Updated metadata example to show domain as the only required key.
  Documented the bundle's derived_from_domain reactor as the source of
  nginx/letsencrypt/monitoring/nftables-input wiring, and the
  bundle-defaults source of backup/paths.

nodes/ovh.left4me.py:
  - groups: + backup, + left4me, + webserver
  - bundles: dropped 'left4me' and 'nftables' (come via groups now;
    nftables ships with debian-13).
  - metadata: pinned vm/cores=4, vm/threads=8 (4-core HT box) so the
    nginx bundle's worker_processes resolves; left4me block reduced to
    {'domain': 'left4.me'} — git_url, git_branch, secret_key, and the
    nginx/letsencrypt/monitoring/nftables/backup blocks now come from
    bundle defaults / the derived_from_domain reactor.
2026-05-10 18:24:03 +02:00
90f14b69e4
left4me: pull node-agnostic metadata into the bundle
Nodes should only carry node-specific metadata. Previously each node
running left4me had to declare git_url, git_branch, secret_key, plus
nginx vhost / letsencrypt / monitoring / nftables-input blocks for
every game port. All of those are derivable from one truly node-
specific value: the domain.

Move into the bundle:
  - git_url + git_branch as defaults (override per-node only if needed).
  - secret_key as a per-node vault-derived value
    (random_bytes_as_base64_for f'{node.name} left4me secret_key',
    same convention as postgresql/mosquitto/etc.).
  - backup/paths defaults (set-merged with backup group / node paths).

Add a `derived_from_domain` reactor that reads left4me/domain and
emits:
  - nginx/vhosts/<domain> proxying 127.0.0.1:8000
  - letsencrypt/domains/<domain>
  - monitoring/services/left4me-web (curl /health)
  - nftables/input rules for the configured port range
    (defaults 27015-27115, derived from left4me/port_range_*).

Net effect: a node opting into left4me declares only
  metadata.left4me.domain = 'whatever.tld'
plus the universal node-level stuff (id, vm/cores, network, …).
2026-05-10 18:23:34 +02:00
3bffd7b8f5
bind-acme: guard against letsencrypt clients without internal LAN
The acme_zone reactor's first ACL branch iterates nodes that have
letsencrypt/domains and reads their network/internal/ipv4. Until now
that crashed for any node with letsencrypt but no internal LAN — the
node had to either fake a network/internal/ipv4 or skip TLS.

Add a `metadata.get(..., None)` guard to filter such nodes out of this
branch. The wireguard branch below already covers them (any node with
the wireguard bundle gets its wireguard/my_ip into the ACL), so ACME
DNS-01 reachability still works for cross-Internet nodes that join the
fleet via wireguard.

Surfaced by ovh.left4me: dedicated server with no Hetzner/internal
network, reachable from the bind-acme node only via wireguard.
2026-05-10 18:23:21 +02:00
43f0c57438
groups: add applications/left4me
Single bundle group; pulls in bundles/left4me. Joined by nodes that run
the L4D2 game-server platform. nftables and systemd come in via the
debian-13 group on Debian-13 nodes, so this group needs only the
left4me bundle itself.
2026-05-10 18:08:36 +02:00
d425afad02
left4me: write bundle README 2026-05-10 18:07:58 +02:00
f9bf289ef0
left4me: assert nftables + systemd bundle membership
Catches misconfiguration at bw test time if a node attaches left4me
without those two bundles. Both contribute load-bearing metadata
materializers (nftables/output rules; systemd/units → unit files).
2026-05-10 18:06:35 +02:00
a8fc3f2298
left4me: fix bundle defects surfaced by real-node validation
Three issues caught once `bw test ovh.left4me` ran with the bundle
actually attached (vs. the earlier `bw test` with no node opting in,
which only checks parsing):

1. systemd_services + nftables_output reactors didn't read any metadata.
   bw rejects this with "did not request any metadata, you might want
   to use defaults instead". Both contributions are static, so they
   belong in `defaults` — moved.

2. git_deploy:/opt/left4me/src triggered action:left4me_create_venv,
   but create_venv lacked `triggered: True`. bw enforces that any
   action in a triggers list must be `triggered: True`. Removed
   create_venv from the trigger list — it's gated by `unless` for
   idempotency and doesn't need to refire on git updates anyway
   (the venv persists). pip_install stays in triggers so editable
   installs pick up new code.
2026-05-10 18:05:38 +02:00
c82737b162
left4me: contribute uid-based DSCP/priority marks to nftables/output
Replaces the per-app inet left4me_mark table from
deploy/files/usr/local/lib/left4me/nft/left4me-mark.nft with two rules
in the central bundles/nftables/ inet filter table's output chain.
Same selectors (skuid left4me + l4proto udp), same actions (DSCP EF +
priority 6) for both v4 and v6.
2026-05-10 17:53:17 +02:00
b1edcac3c7
left4me: enable+start left4me-web.service via systemd/services
The server@ template intentionally has no svc_systemd entry — instances
are started on-demand by the web app through the left4me-systemctl
helper. Slices are activated implicitly when units use Slice=.
2026-05-10 17:49:50 +02:00
72da6c0a8d
left4me: pin EnvironmentFile order via tuples (was sets)
Sets in libs/systemd.py:18 are sorted alphabetically. The current
output is correct by accident — host.env < web.env, host.env < /var.
Adding a third path later would silently reorder. Tuples preserve
insertion order; generate_unitfile() iterates them the same way.

Environment (HOME=, PATH=) stays a set: each line is an independent
KEY=VALUE assignment, order is irrelevant.
2026-05-10 17:48:03 +02:00
6965441e9a
left4me: emit server@ template + game/build slice units
Translates the remaining three unit files from left4me/deploy/files/.
Server template carries the full hardening + cgroup/IO/Mem keys
verbatim. Slices need the bundles/systemd .slice support added in
prior commit.
2026-05-10 17:43:25 +02:00
6bf46ce9a4
left4me: emit left4me-web.service via systemd/units reactor
Translates left4me/deploy/files/usr/local/lib/systemd/system/left4me-web.service
into a Python dict consumed by bundles/systemd/. Two changes vs. the
shell-deploy unit:
  - --bind 0.0.0.0:8000 -> 127.0.0.1:8000 (nginx terminates TLS in front)
  - workers/threads are templated from left4me/gunicorn_{workers,threads}
    (defaults: 1 worker + 32 threads — same as the static unit)
2026-05-10 17:38:15 +02:00
def010c976
left4me: git_deploy + venv/pip/alembic/seed action chain
Mirrors deploy-test-server.sh:233-242 + :329-333. Single pip command
installs both editable packages (l4d2host + l4d2web) from the same
checkout. Alembic and seed-overlays run as the left4me user with
JOB_WORKER_ENABLED=false sourced from web.env.
2026-05-10 17:32:19 +02:00
433c403ddc
left4me: validate sudoers file with visudo before install
A malformed /etc/sudoers.d/left4me would lock sudo on the target
(blast radius: every other bundle using sudo at apply time). bw's
file: items support test_with, which runs the supplied command on the
locally-rendered file before transfer. Use it to gate the sudoers
file on visudo -cf — analogous to the visudo -cf check the original
deploy script ran inline (deploy-test-server.sh:186).
2026-05-10 17:29:01 +02:00
80d2a79b97
left4me: declare directories, users, files, sysctl-reload action
Modes/owners match the upstream left4me deploy script:
  helpers          0755 root:root
  sudoers.d/left4me 0440 root:root (validated with visudo -cf)
  sysctl conf      0644 root:root  (triggers sysctl --system)
  sandbox-resolv   0644 root:root
  /etc/left4me/host.env  0644 root:root  (Mako)
  /etc/left4me/web.env   0640 root:left4me (Mako, contains SECRET_KEY)
  /var/lib/left4me 0711 left4me:left4me (l4d2-sandbox traversal)
UIDs/GIDs pinned at 980/981 for deterministic ownership.
2026-05-10 17:23:03 +02:00
e842e7caa6
left4me: wire LEFT4ME_PORT_RANGE_{START,END} into web.env
Bundle metadata declares port_range_start/end in defaults, but the
running app (l4d2web/config.py:34-35) reads them from
LEFT4ME_PORT_RANGE_START/END env vars. Without these in web.env, the
bundle's metadata values were dead code and the app fell back to its
own hardcoded defaults. Wiring them through closes the loop.
2026-05-10 17:19:02 +02:00
3afd4d60cc
left4me: add Mako templates for host.env and web.env
SECRET_KEY pulled from node metadata (set via !32_random_bytes_as_base64_for:
in the node file). SESSION_COOKIE_SECURE flips to true since nginx fronts
gunicorn with TLS.
2026-05-10 17:14:36 +02:00
6db792ce6a
left4me: vendor privileged helpers + sudoers/sysctl/sandbox-resolv
Copied verbatim from left4me/deploy/files/. Helpers are the trust unit
the sudoers rules grant access to; left as static files (not generated)
so the audit trail stays grep-able. Modes/owners are set via items.py
in the next commit.
2026-05-10 17:10:17 +02:00
7547d041a2
left4me: scaffold bundle (items/metadata/README stubs)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 17:05:13 +02:00
cc1c6a5767
systemd: accept .slice extension in unit-file routing
Slices are a standard systemd unit type; the existing routing only
covered timer/service/mount/swap/target and raised on .slice. Same
install path (/usr/local/lib/systemd/system/<name>) and same
systemd-reload trigger as the other unit kinds.
2026-05-10 17:00:45 +02:00
af78e40fda
left4me wireguard 2026-05-10 16:57:52 +02:00
c6bf2e0fc8
spec: banner stale sections so partial readers see the pivot
§0 Revisions notes that §3 and §7 Phase 2 are pre-pivot, but a reader
deep-linking into either section bypasses §0. Add a section-level
banner at the top of each that points back to §0 and to bundles/AGENTS.md
for the current per-bundle convention. Content is preserved as a record
of the original design.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 16:14:12 +02:00
d4dedde0ad
add implementation plan to repo
vendors ~/.claude/plans/btw-are-you-sure-crystalline-balloon.md into
docs/superpowers/plans/2026-05-10-agent-friendliness-plan.md so the
plan lives alongside its spec and handoff. tagged with a top-of-file
note flagging it as a frozen pre-pivot artifact (the per-bundle-doc
section, the AGENTS.template.md reference, and the Phase 2 seed-list
all reflect original intent, not what shipped).

handoff's pointer updated to the in-repo path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 16:07:35 +02:00
7b44a8ad3a
spec/handoff: record per-bundle README pivot
- spec §0 gets a new revision bullet noting that per-bundle docs
  are README.md (not AGENTS.md), the rigid template is gone, and
  Phase 2 was dropped. flags §3 and §7 as pre-pivot intent only;
  doesn't back-fit them.
- handoff replaced with a short status note (~50 lines vs the
  original ~390): what landed, where current truth lives, and the
  fact that nothing is planned for a next session.

implementation plan in ~/.claude/plans/ is left as a frozen
pre-pivot artifact.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 16:06:25 +02:00
9e1bb2ac45
docs: per-bundle docs are README.md, not AGENTS.md
drops the per-bundle AGENTS.md convention and the rigid template
that went with it. each bundle has (or gets) one README.md that
serves humans and agents both.

bundles/AGENTS.md now has a "Per-bundle README" section pointing
at the more substantial existing READMEs (flask, dm-crypt, apt,
nextcloud) for orientation, plus loose guidance on what to cover
and what to skip. no required structure — match the bundle's
actual surface.

removes bundles/AGENTS.template.md; the template was prescriptive
in a way that wouldn't survive contact with this repo's actual
bundles, where READMEs range from one-paragraph balanced docs to
operational scratchpads.

phase-2 seed-bundle work stays deferred and will land as plain
README updates when bundles are materially edited.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 16:02:24 +02:00
04558a9189
docs: scaffold agent-friendly entry points (Phase 1)
introduces a balanced set of agent + human docs:

- root AGENTS.md (with CLAUDE.md symlink) — 5-rule quickstart,
  layout map, mental model, use-case keyed example pointers.
- docs/agents/conventions.md — vault/demagify, eval-loader
  constraints, group inheritance, naming, do-not-touch list,
  suspension idioms, working-style notes.
- docs/agents/commands.md — repo-specific deltas to the fork's
  bw runbook (apt-key offline-verify, *.py_ suspended-node
  visibility, vault-echo rule).
- per-area AGENTS.md for bundles/, nodes/, groups/, libs/,
  hooks/, data/, items/, bin/ — mechanism-focused, no enumeration.
- bundles/AGENTS.template.md — per-bundle doc template with
  optional `## Writes into` section for cross-namespace reactors.

bundlewrap-language reference (item types, dep keywords, reactors,
runbook, three-tier safety envelope) is not duplicated here; we
link out to the fork's AGENTS.md instead.

bw test still green. all internal links resolve. Phase 0 invariants
preserved (libs/hooks docstrings, bin/* # purpose: headers).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 15:44:45 +02:00
730625e36c
libs/hooks/bin: add one-line module docstrings and # purpose: headers
every libs/*.py and hooks/*.py now starts with a one-line module
docstring; every bin/* script starts with a `# purpose:` header.
discovery-by-`ls`-and-read instead of by index.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 15:36:19 +02:00
136313e9c3
add implementation handoff for the next session
Self-contained handover covering: Phase 0 commits already landed,
Phase 0 remainder (docstring/header pass), Phase 1 scaffolding order,
Phase 2 seed bundle list, captured decisions, pitfalls (bw-syntax
corrections, /etc/hosts macOS quirk, sandbox), and verification
criteria. Cross-references the spec, the user-stories validation, the
plan, and the fork's AGENTS.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 15:28:34 +02:00
1da70970e5
README: drop stale 'install bw fork' instruction
The bundlewrap install is now captured in requirements.txt as an editable
github reference, and the file:/// path in the README pointed at a local
clone with no relation to the actual install method.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 15:19:44 +02:00
3daf70dae7
spec: incorporate fork pivot and bw-syntax corrections
- drop the docs/agents/bundlewrap/ folder; bundlewrap-language docs now
  live in the personal fork's AGENTS.md (canonical reference). ckn-bw
  links out instead of duplicating
- slim commands.md scope from ~80-120 to ~30-50 lines (fork carries the
  generic bw runbook; ckn-bw keeps only repo-specific deltas: apt-key
  verification, *.py_ suspended-node behavior, vault-echo guidance)
- sync bw command syntax against 5.0.3 source (no -p flag; use bare or
  --preview; bw hash takes only literal node/group names; replace
  bw groups -n with bw nodes -a groups)
- rebalance phase 2 seed list: php -> routeros-monitoring (highest-churn
  bundle in 18mo per user-story analysis)
- update fork install pointer to editable github reference
- new section 0 documents revisions inline so a reader sees current shape

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 15:19:17 +02:00
b804350f17
add user-stories validation doc
21 recurring user stories derived from 1169 commits of git history (with
detailed analysis of the last 222 commits / 18 months). Grounded in
concrete commit evidence; each story carries an "Implications for agent
docs" section that drives content additions in the agent-friendliness
implementation plan.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 15:14:38 +02:00
7486c78ae1
switch bundlewrap install to editable from CroneKorkN/bundlewrap@main
Replaces the PyPI 5.0.3 pin with an editable github clone of the personal
fork. The fork tracks upstream main and carries an agent-oriented
AGENTS.md the rest of this repo's docs link to.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 15:14:31 +02:00
c03b033ad9
macbook dummy 2026-05-10 11:57:26 +02:00
186d5039af
migrate to bundlewrap 5
- pin bundlewrap ~=5.0
- rewrite non-reading and KeyError-driven metadata reactors per
  https://docs.bundlewrap.org/guide/migrate_45/ (defaults / metadata.get
  paths / MetadataUnavailable)
- rename custom Download item methods (cdict/sdict/get_auto_deps ->
  expected_state/actual_state/get_auto_attrs)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 11:56:49 +02:00