Compare commits

..

10 commits

Author SHA1 Message Date
d4dedde0ad
add implementation plan to repo
vendors ~/.claude/plans/btw-are-you-sure-crystalline-balloon.md into
docs/superpowers/plans/2026-05-10-agent-friendliness-plan.md so the
plan lives alongside its spec and handoff. tagged with a top-of-file
note flagging it as a frozen pre-pivot artifact (the per-bundle-doc
section, the AGENTS.template.md reference, and the Phase 2 seed-list
all reflect original intent, not what shipped).

handoff's pointer updated to the in-repo path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 16:07:35 +02:00
7b44a8ad3a
spec/handoff: record per-bundle README pivot
- spec §0 gets a new revision bullet noting that per-bundle docs
  are README.md (not AGENTS.md), the rigid template is gone, and
  Phase 2 was dropped. flags §3 and §7 as pre-pivot intent only;
  doesn't back-fit them.
- handoff replaced with a short status note (~50 lines vs the
  original ~390): what landed, where current truth lives, and the
  fact that nothing is planned for a next session.

implementation plan in ~/.claude/plans/ is left as a frozen
pre-pivot artifact.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 16:06:25 +02:00
9e1bb2ac45
docs: per-bundle docs are README.md, not AGENTS.md
drops the per-bundle AGENTS.md convention and the rigid template
that went with it. each bundle has (or gets) one README.md that
serves humans and agents both.

bundles/AGENTS.md now has a "Per-bundle README" section pointing
at the more substantial existing READMEs (flask, dm-crypt, apt,
nextcloud) for orientation, plus loose guidance on what to cover
and what to skip. no required structure — match the bundle's
actual surface.

removes bundles/AGENTS.template.md; the template was prescriptive
in a way that wouldn't survive contact with this repo's actual
bundles, where READMEs range from one-paragraph balanced docs to
operational scratchpads.

phase-2 seed-bundle work stays deferred and will land as plain
README updates when bundles are materially edited.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 16:02:24 +02:00
04558a9189
docs: scaffold agent-friendly entry points (Phase 1)
introduces a balanced set of agent + human docs:

- root AGENTS.md (with CLAUDE.md symlink) — 5-rule quickstart,
  layout map, mental model, use-case keyed example pointers.
- docs/agents/conventions.md — vault/demagify, eval-loader
  constraints, group inheritance, naming, do-not-touch list,
  suspension idioms, working-style notes.
- docs/agents/commands.md — repo-specific deltas to the fork's
  bw runbook (apt-key offline-verify, *.py_ suspended-node
  visibility, vault-echo rule).
- per-area AGENTS.md for bundles/, nodes/, groups/, libs/,
  hooks/, data/, items/, bin/ — mechanism-focused, no enumeration.
- bundles/AGENTS.template.md — per-bundle doc template with
  optional `## Writes into` section for cross-namespace reactors.

bundlewrap-language reference (item types, dep keywords, reactors,
runbook, three-tier safety envelope) is not duplicated here; we
link out to the fork's AGENTS.md instead.

bw test still green. all internal links resolve. Phase 0 invariants
preserved (libs/hooks docstrings, bin/* # purpose: headers).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 15:44:45 +02:00
730625e36c
libs/hooks/bin: add one-line module docstrings and # purpose: headers
every libs/*.py and hooks/*.py now starts with a one-line module
docstring; every bin/* script starts with a `# purpose:` header.
discovery-by-`ls`-and-read instead of by index.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 15:36:19 +02:00
136313e9c3
add implementation handoff for the next session
Self-contained handover covering: Phase 0 commits already landed,
Phase 0 remainder (docstring/header pass), Phase 1 scaffolding order,
Phase 2 seed bundle list, captured decisions, pitfalls (bw-syntax
corrections, /etc/hosts macOS quirk, sandbox), and verification
criteria. Cross-references the spec, the user-stories validation, the
plan, and the fork's AGENTS.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 15:28:34 +02:00
1da70970e5
README: drop stale 'install bw fork' instruction
The bundlewrap install is now captured in requirements.txt as an editable
github reference, and the file:/// path in the README pointed at a local
clone with no relation to the actual install method.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 15:19:44 +02:00
3daf70dae7
spec: incorporate fork pivot and bw-syntax corrections
- drop the docs/agents/bundlewrap/ folder; bundlewrap-language docs now
  live in the personal fork's AGENTS.md (canonical reference). ckn-bw
  links out instead of duplicating
- slim commands.md scope from ~80-120 to ~30-50 lines (fork carries the
  generic bw runbook; ckn-bw keeps only repo-specific deltas: apt-key
  verification, *.py_ suspended-node behavior, vault-echo guidance)
- sync bw command syntax against 5.0.3 source (no -p flag; use bare or
  --preview; bw hash takes only literal node/group names; replace
  bw groups -n with bw nodes -a groups)
- rebalance phase 2 seed list: php -> routeros-monitoring (highest-churn
  bundle in 18mo per user-story analysis)
- update fork install pointer to editable github reference
- new section 0 documents revisions inline so a reader sees current shape

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 15:19:17 +02:00
b804350f17
add user-stories validation doc
21 recurring user stories derived from 1169 commits of git history (with
detailed analysis of the last 222 commits / 18 months). Grounded in
concrete commit evidence; each story carries an "Implications for agent
docs" section that drives content additions in the agent-friendliness
implementation plan.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 15:14:38 +02:00
7486c78ae1
switch bundlewrap install to editable from CroneKorkN/bundlewrap@main
Replaces the PyPI 5.0.3 pin with an editable github clone of the personal
fork. The fork tracks upstream main and carries an agent-oriented
AGENTS.md the rest of this repo's docs link to.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 15:14:31 +02:00
51 changed files with 2452 additions and 122 deletions

106
AGENTS.md Normal file
View file

@ -0,0 +1,106 @@
# ckn-bw — agent & contributor guide
## What this repo is
A [BundleWrap](https://bundlewrap.org/) configuration-management repo
for ~22 personal/family-infra nodes. Nodes, groups, and bundles are
defined in plain Python; `bw apply` deploys the resulting state to
real machines.
Note: the root `README.md` is the maintainer's personal scratchpad,
not project documentation. Onboarding lives **here**, in `AGENTS.md`.
## Quickstart for agents
Five rules; follow these and you won't break things:
1. **Read-only by default.** Never run `bw apply`, `bw run`, or
`bw lock` without explicit user request — even with `-i`. Stick
to `bw test`, `bw nodes`, `bw groups`, `bw bundles`,
`bw items`, `bw metadata`, `bw hash`, `bw debug`. See
[`docs/agents/commands.md`](docs/agents/commands.md) and the
fork's [safety envelope](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md).
2. **Never echo decrypted secrets.** Don't print, paste, or log the
value behind a `!password_for:`, `!decrypt:`, or
`!32_random_bytes_as_base64_for:` magic string — not even from
`bw debug` exploration. See
[`conventions.md#secrets`](docs/agents/conventions.md#secrets).
3. **Don't touch the do-not-modify list.** `.secrets.cfg*`, `.venv`,
`.cache`, `.bw_debug_history`, `.envrc`, root `README.md`. Treat
`hooks/` and `items/` (custom item types) with extra care: a
broken hook or item type breaks every `bw` command repo-wide.
4. **Use the fork.** The venv runs editable from
[`github.com/CroneKorkN/bundlewrap`](https://github.com/CroneKorkN/bundlewrap)
(branch `main`). Behavior tracks upstream `main`; the fork's
[`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
is the canonical bundlewrap-language reference. See
[`conventions.md#bundlewrap-version`](docs/agents/conventions.md#bundlewrap-version).
5. **Prefer adding helpers to `libs/`** over duplicating logic across
bundles. Repo-wide helpers go in
[`libs/`](libs/AGENTS.md), reachable as `repo.libs.<x>`.
## Layout
| Dir | What's there |
|---|---|
| [`bundles/`](bundles/AGENTS.md) | 103 bundles. One subdir per bundle (`items.py`, `metadata.py`, `files/`). |
| [`nodes/`](nodes/AGENTS.md) | One file per node (~22). `eval()`-loaded; demagified through `repo.vault`. |
| [`groups/`](groups/AGENTS.md) | Group definitions, organized by axis (`applications/`, `locations/`, `machine/`, `os/`). |
| [`libs/`](libs/AGENTS.md) | Shared Python helpers reachable as `repo.libs.<modulename>`. |
| [`hooks/`](hooks/AGENTS.md) | bw lifecycle hooks (`apply_start`, `test`, `node_apply_start`, …). |
| [`data/`](data/AGENTS.md) | Out-of-bundle data assets (apt keys, grafana dashboards, …). |
| [`items/`](items/AGENTS.md) | Custom item types (currently `download:`). |
| [`bin/`](bin/AGENTS.md) | Operator scripts; not invoked by bundlewrap. |
| [`docs/agents/`](docs/agents/conventions.md) | Repo conventions and command deltas. |
## How nodes, groups, and bundles fit together
- A **node** (`nodes/<location>.<role>.py`) declares the groups it
belongs to and any node-local bundles + metadata overrides.
- A **group** (`groups/<axis>/<x>.py`) attaches bundles and shared
metadata to its members. Groups inherit via `supergroups`.
- A **bundle** (`bundles/<x>/`) is one chunk of configuration:
`items.py` produces the items (files, services, packages),
`metadata.py` declares `defaults` and `@metadata_reactor` functions
that derive metadata from other metadata.
- The repo-root loaders (`nodes.py`, `groups.py`) walk these dirs and
`eval()` each file. `nodes.py` additionally **demagifies** the
result, resolving `!password_for:` etc. through `repo.vault`. See
[`conventions.md#eval-loaded-node-and-group-files`](docs/agents/conventions.md#eval-loaded-node-and-group-files)
for the constraints this places on editors.
- Metadata merges along: `all → location → os → machine →
applications → node`.
## Conventions you must know
| Topic | Where |
|---|---|
| Bundlewrap-language reference (item types, dep keywords, reactors) | Fork's [`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md) — read first if new to bundlewrap |
| Vault / demagify magic strings | [`conventions.md#secrets`](docs/agents/conventions.md#secrets) |
| Bundlewrap install (editable from the fork) | [`conventions.md#bundlewrap-version`](docs/agents/conventions.md#bundlewrap-version) |
| Group inheritance order, naming patterns | [`conventions.md#group-inheritance-order`](docs/agents/conventions.md#group-inheritance-order), [`#naming-conventions`](docs/agents/conventions.md#naming-conventions) |
| Repo-specific bw command deltas (apt keys, suspended nodes, vault echo) | [`commands.md`](docs/agents/commands.md) |
| Lib helpers | top-of-file docstrings in `libs/*.py` (`head -1 libs/*.py`) |
| Suspension idioms (`*.py_`, `_old/`, "for now") | [`conventions.md#suspension-and-soft-delete-idioms`](docs/agents/conventions.md#suspension-and-soft-delete-idioms) |
## Where to look for examples
When writing a new bundle, copy patterns from one that already does
the thing you need:
| Pattern | Look at |
|---|---|
| Vault calls inside metadata reactors | `bundles/dm-crypt/metadata.py` (compact, focused) |
| Mako-templated files | `bundles/bind/items.py` (DNS zonefile rendering) |
| Cross-bundle reactor writing | `bundles/nextcloud/metadata.py` (writes into `apt.packages`, `archive.paths`) |
| Custom `download:` items | `bundles/minecraft/items.py` |
| Node file (single-purpose) | `nodes/home.server.py` |
| Group with `supergroups` chain | `groups/os/debian-13.py` |
## Where this doc lives
- This file: `AGENTS.md` at the repo root.
- `CLAUDE.md` is a symlink to this file — both names point to the same
content so different tools can find it.
- The personal TODO scratchpad (`README.md`) is **separate** and not
project documentation.

1
CLAUDE.md Symbolic link
View file

@ -0,0 +1 @@
AGENTS.md

View file

@ -13,10 +13,6 @@ Raspberry pi as soundcard
- OTG g_audio
- https://audiosciencereview.com/forum/index.php?threads/raspberry-pi-as-usb-to-i2s-adapter.8567/post-215824
# install bw fork
pip3 install --editable git+file:///Users/mwiegand/Projekte/bundlewrap-fork@main#egg=bundlewrap
# monitor timers
```sh

62
bin/AGENTS.md Normal file
View file

@ -0,0 +1,62 @@
# bin/
## What's here
Operator scripts — invoked manually by the maintainer, **not** by
bundlewrap itself. Each is a standalone Python (or shell) script that
opens the repo via `Repository(dirname(dirname(realpath(__file__))))`.
Discovery is by `ls bin/` plus the `# purpose:` header line at the top
of each script:
```sh
head -2 bin/*
```
## Conventions
- **`# purpose:` header.** Every script under `bin/` starts with
`#!/usr/bin/env python3` (or appropriate shebang), then a
`# purpose: <one-line description>` comment. Baseline enforced by
`grep -L '^# purpose' bin/*`.
- **Self-contained.** A script must work when run from anywhere — it
resolves the repo via the script's own path, not `cwd`.
- **Read-only by default.** Most operator scripts query/print state
(`passwords-for`, `wireguard-client-config`). Mutating scripts
(`upgrade_and_restart_all`, `mikrotik-firmware-updater`,
`sync_1password`) are the exception, not the rule, and prompt for
confirmation.
## How to add a script
1. Start from [`bin/script_template`](script_template) — it carries
the canonical shebang + `# purpose:` header + `Repository(...)`
bootstrap.
2. Add the `# purpose:` line; lowercase, terse, include a `usage:`
example if the script takes arguments.
3. `chmod +x bin/<name>`.
4. The script can reach helpers via `bw.libs.<x>` exactly like a
bundle does.
## Pitfalls
- **`bin/` is not on `$PATH` by default.** Invoke as `bin/<name>` from
the repo root, or via `direnv` if `.envrc` exposes it.
- **Mutating scripts can hit Tier-3 territory** (per the fork's
safety envelope). Don't run `upgrade_and_restart_all`,
`mikrotik-firmware-updater`, or anything that does `node.run(...)`
without explicit user instruction. See the fork's
[`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
for the three-tier model.
- **Vault echo.** Scripts like `passwords-for` print decrypted values
by design; that's allowed for the human at the terminal but *not*
for the agent — never paste output into chat, ticket, or PR
description.
## See also
- [`script_template`](script_template) — canonical starter.
- [`docs/agents/conventions.md`](../docs/agents/conventions.md) —
vault rules.
- [`docs/agents/commands.md`](../docs/agents/commands.md) — read-only
bw-command guidance.

View file

@ -1,4 +1,5 @@
#!/usr/bin/env python3
# purpose: upgrade RouterOS and routerboard firmware on `bundle:routeros` (or any selector) — usage: mikrotik-firmware-updater [<selector>...] [--yes].
from argparse import ArgumentParser
from time import sleep

View file

@ -1,4 +1,5 @@
#!/usr/bin/env python3
# purpose: print node.password and selected metadata-key passwords for one node — usage: passwords-for <node>.
from bundlewrap.repo import Repository
from os.path import realpath, dirname

View file

@ -1,4 +1,5 @@
#!/usr/bin/env python3
# purpose: send an RCON command to a left4dead2 server defined in node metadata — usage: rcon (list) | rcon <server> <command>.
from sys import argv
from os.path import realpath, dirname

View file

@ -1,4 +1,5 @@
#!/usr/bin/env python3
# purpose: starter template for new operator scripts under bin/.
from bundlewrap.repo import Repository
from os.path import realpath, dirname

View file

@ -1,4 +1,5 @@
#!/usr/bin/env python3
# purpose: upsert one 1Password login per `bundle:routeros` node, keyed on the bw node id.
from bundlewrap.repo import Repository
from os.path import realpath, dirname

View file

@ -1,4 +1,5 @@
#!/usr/bin/env python3
# purpose: add missing EXIF/QuickTime timestamps to photos in a directory using mdls + exiftool — usage: timestamp_icloud_photos_for_nextcloud -d <dir>.
from subprocess import check_output, CalledProcessError
from datetime import datetime, timedelta

View file

@ -1,4 +1,5 @@
#!/usr/bin/env python3
# purpose: apt-update and full-upgrade every non-dummy debian node, then reboot in WireGuard-aware order.
from bundlewrap.repo import Repository
from os.path import realpath, dirname

View file

@ -1,4 +1,5 @@
#!/usr/bin/env python3
# purpose: wake one node via WoL by name — usage: wake <node>.
from bundlewrap.repo import Repository
from os.path import realpath, dirname

View file

@ -1,4 +1,5 @@
#!/usr/bin/env python3
# purpose: print or QR-render a WireGuard client config from htz.mails metadata — usage: wireguard-client-config <client>.
from bundlewrap.repo import Repository
from os.path import realpath, dirname

156
bundles/AGENTS.md Normal file
View file

@ -0,0 +1,156 @@
# bundles/
## Before you start
Read [`docs/agents/conventions.md`](../docs/agents/conventions.md) first
— it covers vault calls, demagify, the `repo.libs.<x>` helpers, and the
files agents must not modify. Skipping it leads to subtly broken bundles
(vault calls in the wrong place, dict-in-set `TypeError` because of
unhashable nesting, etc.).
For bundlewrap-language reference (item types, dep keywords,
`metadata_reactor`, `defaults`, item-file template syntax) see the fork's
[`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
and its [`docs/content/guide/item_file_templates.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/docs/content/guide/item_file_templates.md).
## What's here
103 bundles. Each is a directory `bundles/<name>/` containing some of:
```
bundles/<name>/
├── items.py # the items this bundle creates (files, services, packages, …)
├── metadata.py # `defaults` + `@metadata_reactor` functions
├── files/ # static or templated file payloads referenced from items.py
└── README.md # one doc per bundle, for humans and agents (see "Per-bundle README" below)
```
## Conventions
- **Bundle names** are lowercase, hyphen-separated: `backup-server`,
`bind-acme`, `dm-crypt`. No underscores in new bundle names — see
[`conventions.md#naming-conventions`](../docs/agents/conventions.md#naming-conventions).
- **`items.py`** is plain Python; it produces `files = {...}`,
`pkg_apt = {...}`, `svc_systemd = {...}`, etc. dicts at module scope.
Cross-item dependencies use `needs` / `triggers` / `triggered_by`
see the fork's `AGENTS.md` for the full keyword cheat sheet.
- **`metadata.py`** uses `defaults = {...}` for static seed values and
`@metadata_reactor.provides(...)` for derived values. Reactors are
pure functions of `(metadata,)` — no side effects, no I/O.
- **Helpers go in [`libs/`](../libs/AGENTS.md)** when they're useful to
more than one bundle. Don't duplicate logic across bundles.
- **Custom item types** (e.g. `download:`) live in
[`items/`](../items/AGENTS.md), not per-bundle.
## How to add a new bundle
1. `mkdir bundles/<name>/` (lowercase, hyphenated).
2. Write `items.py` and (if anything is configurable) `metadata.py`.
Use `repo.libs.hashable.hashable(...)` when you need to nest a dict
or set inside a metadata set; raw dicts/sets aren't hashable.
3. Drop static payloads into `bundles/<name>/files/`. For Mako-templated
files, declare `'content_type': 'mako'` on the `file:` item — see
the fork's
[item-file-templates guide](https://github.com/CroneKorkN/bundlewrap/blob/main/docs/content/guide/item_file_templates.md).
4. **Wire to nodes.** Either add an entry to the relevant
[`groups/<axis>/<x>.py`](../groups/AGENTS.md) (preferred for shared
bundles) or to the node's `bundles` list directly
([`nodes/AGENTS.md`](../nodes/AGENTS.md)).
5. Verify, in this order:
- `bw test` — sanity (loaders + reactors).
- `bw items <node>` — confirm new items appear on a node that opts in.
- `bw hash <node>` — confirm the change is what you expected. See
[`docs/agents/commands.md`](../docs/agents/commands.md) and the
fork's hash-diff workflow.
6. Add a `bundles/<name>/README.md`. See "Per-bundle README" below
for what to cover.
## How to remove a bundle
1. `git grep '<name>'` in `nodes/`, `groups/`, and other `bundles/` to
find references.
2. Remove those references.
3. `rm -rf bundles/<name>/`.
4. `bw test` and `bw nodes` to confirm clean.
## Pitfalls
- **`metadata.py` is evaluated at load time** for *every* node, every
invocation of `bw`. Heavy work or I/O slows the whole repo. Keep
reactors pure and fast; pre-compute in `libs/` if you must.
- **Static files vs templates.** `bundles/<x>/files/<f>` is static
unless the matching `file:` item declares `content_type='mako'`
(or a templating extension triggers it). To check, read the matching
`file:` entry in `items.py`.
- **Reactors writing across namespaces.** Some bundles' reactors write
into other bundles' metadata namespaces (e.g. `nextcloud` writes
into `apt.packages`, `archive.paths`). When you change such a bundle,
every consumer's metadata changes too. The bundle's `README.md`
often calls these out — but the authoritative source is `metadata.py`
itself; grep `'<other-bundle>':` in the reactors when in doubt.
- **`bw hash` doesn't accept selectors.** Use `bw hash <node>` per
literal name; see the fork's runbook.
## Per-bundle README
Each bundle has (or should have) a `README.md`. One doc per bundle,
written for humans and agents both. There's no fixed structure —
match the bundle's actual surface, write what helps a future reader
(or future you) avoid trial-and-error.
The existing READMEs vary in quality and shape. For orientation,
look at the bigger ones, not the two-line ones:
- [`bundles/flask/README.md`](flask/README.md) — title + one-sentence
purpose, a metadata example as a Python dict, then the contract
the consuming git repo has to satisfy + a logging pitfall. The
closest thing to a "balanced doc" in tree.
- [`bundles/dm-crypt/README.md`](dm-crypt/README.md) — same shape,
shorter: purpose + metadata example + one sentence on effect.
- [`bundles/apt/README.md`](apt/README.md) — relevant upstream URLs
at the top, then a Python metadata example with rich inline
comments (type / optionality / where keys come from).
- [`bundles/nextcloud/README.md`](nextcloud/README.md) — operational
scratchpad: iPhone-import recipe, preview-generator commands,
reset queries. Captures muscle-memory the maintainer would
otherwise re-learn each time.
Useful things to include, when relevant:
- A sentence or two on what the bundle does and when you'd attach it.
- A metadata example as a Python dict literal, with `#` comments
on each key (type, required vs default, units, where it comes
from). This is the cleanest way to communicate the schema and
matches how `metadata.py` actually looks.
- Anything non-obvious about wiring it up — required keys without
defaults, group-membership expectations, manual one-time steps.
- Cross-namespace metadata writes, when this bundle's reactors
populate another bundle's namespace. Easy to miss, cheap to flag.
- Gotchas, debug recipes, failure modes you've actually hit.
What to skip:
- An exhaustive item list — `items.py` is shorter and more accurate.
- Anything that would just rot — version numbers, "TODO" lists,
change notes. Use git history.
If a single paragraph is enough to say what's worth saying, write a
single paragraph. Verbosity isn't a goal.
Convention going forward is leave-as-you-go: any time you materially
edit a bundle, top up its README (or write one if it's missing).
Don't burn a session bulk-reformatting the existing ones — uneven
quality is part of what we accept in exchange for not blocking other
work.
## See also
- [`docs/agents/conventions.md`](../docs/agents/conventions.md) — repo
idioms (vault, demagify, naming, do-not-touch list).
- [`docs/agents/commands.md`](../docs/agents/commands.md) — repo-specific
command deltas.
- [`items/AGENTS.md`](../items/AGENTS.md) — custom item types
(`download:`); when to write a new one vs use `file:`.
- [`libs/AGENTS.md`](../libs/AGENTS.md) — shared helpers.
- Fork's [`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
— bundlewrap-language reference + safety envelope.

63
data/AGENTS.md Normal file
View file

@ -0,0 +1,63 @@
# data/
## What's here
Out-of-bundle data assets consumed by one or more bundles. Each subdir
maps to a consumer:
```
data/
├── apt/keys/ # binary GPG keys for apt sources
├── grafana/rows/ # Mako-templated dashboard panels (Python)
├── nginx/ # nginx snippets shared across vhosts
├── homeassistant/, mailman/, nextcloud/, ...
└── network.py # repo-wide network metadata (one-off file)
```
## Two distinct content models
Same directory shape, different content kinds. When you add a new
`data/<x>/` subdir, declare which model it follows:
| Model | Example | Consumer |
|---|---|---|
| **Binary / static** | `data/apt/keys/*.{asc,gpg}` | `bundles/apt` reads files at apply time |
| **Python module / template** | `data/grafana/rows/*.py`, `data/routeros-monitoring/*.py` | bundle `import`s and renders |
If a data asset is read by **exactly one bundle**, prefer
`bundles/<x>/files/` instead of `data/<x>/`. `data/` is for
shared / multi-consumer artifacts. Single-instance evidence: commit
`78a8abc` moved `mikrotik.mib` *out* of `data/` *into* the bundle for
this reason.
## Conventions
- **One subdir per consumer.** Subdir name = consumer bundle name
(`data/apt/`, `data/nextcloud/`).
- **`network.py` exception.** A single file at `data/network.py` holds
repo-wide network metadata; it doesn't belong to one bundle. Treat
it as cross-cutting infrastructure metadata.
## How to add data
1. Decide the content model (binary or Python).
2. `mkdir data/<consumer>/`.
3. Drop assets in.
4. The consumer bundle (`bundles/<consumer>/items.py` or
`metadata.py`) reads them via `repo.path` + `os.path.join` or
similar.
## Pitfalls
- **Apt keys** trigger an offline-verify rule before they're committed.
See [`commands.md#apt-key-changes-need-offline-verification`](../docs/agents/commands.md#apt-key-changes-need-offline-verification).
- **Mako-templated Python data** evaluates at bundle render time. Side
effects in those modules slow the whole repo (same caveat as
[`libs/`](../libs/AGENTS.md)).
## See also
- [`bundles/AGENTS.md`](../bundles/AGENTS.md) — when bundle `files/`
beats `data/`.
- [`docs/agents/commands.md`](../docs/agents/commands.md) — apt-key
verification rule.

50
docs/agents/commands.md Normal file
View file

@ -0,0 +1,50 @@
# Commands
The canonical bw-command runbook — read-only allowlist, three-tier
safety envelope, after-change table, hash-diff workflow, `bw debug`
sketch — lives in the fork's
[`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md).
Read that first.
This file collects only the deltas specific to `ckn-bw`.
## Apt-key changes need offline verification
Editing files under `data/apt/keys/*.{asc,gpg}` rotates a signing key
the whole apt subsystem trusts. Trial-and-error with `bw apply` is the
*failure* path: a wrong key blocks unattended upgrades cluster-wide
until corrected manually.
Before touching `data/apt/keys/`:
1. Fetch the new key from its upstream source (project release page,
`keys.openpgp.org`, etc.).
2. `gpg --show-keys <newkey>` — print the fingerprint.
3. Diff against the fingerprint published by the upstream source.
4. Only after the fingerprint matches, place the file under
`data/apt/keys/` and let `bundles/apt` consume it on the next
apply.
## `*.py_` suspended nodes are invisible to `bw nodes`
The repo loader (`nodes.py`) only matches files ending in `.py`. Files
ending in `.py_` are silently skipped. If `bw nodes` reports a node
missing, check whether its file has been parked:
```sh
ls nodes/ | grep '\.py_$'
```
This is the [suspension idiom](conventions.md#suspension-and-soft-delete-idioms),
not a bug.
## Vault output never leaves the terminal
The fork's runbook calls out that `bw debug` resolves vault magic
strings transparently. In `ckn-bw` specifically: never echo, log, or
paste decrypted values, even from a `bw debug -c` one-liner. If you
need to verify a secret resolved correctly, hash or fingerprint it
instead.
See [`conventions.md#secrets`](conventions.md#secrets) for the
demagify magic-string list and the rule's full rationale.

146
docs/agents/conventions.md Normal file
View file

@ -0,0 +1,146 @@
# Conventions
Repo-specific idioms an agent has to know before editing this BundleWrap
config. For bundlewrap-the-language reference (item types, dep keywords,
metadata-reactor semantics, after-change runbook), see the fork's
[`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md).
## Secrets
Secrets live in `.secrets.cfg` and are referenced from node files by
"demagify" magic strings. The loader (`nodes.py`) walks node dicts and
resolves any leaf string of the form `!<verb>:<arg>`.
| Magic string | Resolves to |
|---|---|
| `!password_for:<id>` | `repo.vault.password_for(id)` |
| `!decrypt:<ciphertext>` | `repo.vault.decrypt(ciphertext)` |
| `!decrypt_file:<path>` | `repo.vault.decrypt_file(path)` |
| `!32_random_bytes_as_base64_for:<id>` | `repo.vault.random_bytes_as_base64_for(id, length=32)` |
Magic strings are only resolved inside **node files** (everything under
`nodes/`). They are *not* resolved in group files, bundle metadata
defaults, or item attributes — call `repo.vault.<verb>(...)` directly
there.
**Never echo decrypted values.** Don't print, log, or paste them, even
in `bw debug` exploration. If you need a sanity check, hash or
fingerprint instead. This applies to AI agents and humans equally.
## Bundlewrap version
The venv runs **editable** from the maintainer's fork:
```
-e git+https://github.com/CroneKorkN/bundlewrap.git@main#egg=bundlewrap
```
Pinned in `requirements.txt`. The fork's `main` tracks upstream
`bundlewrap/bundlewrap` master; the fork's [`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
is the canonical agent-oriented bundlewrap-language reference.
To pull upstream changes into the venv:
```sh
(cd .venv/src/bundlewrap && git pull)
```
## Eval-loaded node and group files
`nodes/*.py` and `groups/*.py` are loaded by `eval()` in `nodes.py` /
`groups.py` (top of the repo). Each file must be a single Python
expression — typically a dict literal.
Consequences:
- **No top-level imports.** No `from foo import bar`, no `import os`.
Helpers go in `libs/` and are reached via `repo.libs.<x>` from
bundle code, not from node files.
- **No statements.** No `if`/`for`/`def` at the top level. Use
expressions (ternaries, comprehensions) instead.
- **Silent drop on parse error.** Vanilla bundlewrap silently omits a
node/group whose file fails to eval. The maintainer patched
`groups.py` (commit `dc40295`) to print the error instead — but it
still skips the file. Symptom: `bw nodes` lists fewer nodes than you
expect. Cure: re-read the file and check for accidental imports or
syntax errors.
## Group inheritance order
Metadata merges along this chain (from the fork's docs):
```
all → location → os → machine → applications → node
```
`groups/all.py` is the universal base; `groups/{locations,os,machine,applications}/`
hold the per-axis groups; the node's own metadata wins last.
## Naming conventions
| Where | Convention | Example |
|---|---|---|
| `nodes/*.py` | `<location>.<role>.py` | `home.server.py`, `htz.mails.py` |
| `groups/*.py` | one file per group; subdir = purpose | `groups/applications/nextcloud.py`, `groups/os/debian-13.py` |
| `bundles/<name>` | lowercase, hyphen-separated | `backup-server`, `bind-acme`, `routeros-monitoring` |
| Custom items | `items/<type>.py` | `items/download.py` |
Underscores in bundle names appear only in `_old`-suffixed leftovers
(see below); don't introduce new ones.
## Suspension and soft-delete idioms
These conventions look like dead code; they aren't. Don't clean them up.
- **`*.py_` parked node.** A node file whose name ends in `.py_` is
silently excluded from `bw nodes` (the loader only matches `*.py`).
Used to keep decommissioned-but-not-deleted node configs in tree.
Example: `nodes/htz.l4d2.py_`.
- **"for now / disable / dummy / offline" markers.** Commits or comments
containing these phrases mark deliberate suspensions, not bugs. Check
`git log -- <file>` before re-enabling — the maintainer reverses
these manually when the underlying condition resolves.
- **`_old` / `_old2` directories.** Recovery buffers during big
refactors. Don't delete them without asking, even if they look
orphaned.
## Files agents must not modify
| Path | Why |
|---|---|
| `.secrets.cfg*` | vault key material |
| `.venv/` | local Python environment |
| `.cache/` | bw runtime cache |
| `.bw_debug_history` | shell history for `bw debug` |
| `.envrc` | direnv local-environment config |
| `/.cocoindex_code/` | local code index, not in git |
| `README.md` (root) | maintainer's personal TODO scratchpad — not project docs |
Treat `hooks/` and `items/` (custom item types) with extra care: they
affect `bw`'s behavior for the whole repo, not a single bundle.
## Working style
- **Iterative commits are normal.** The maintainer commits in small
checkpoints (`+`, `fix`, `whitespace`, terse one-liners). Don't
rebase WIP branches without asking. As an agent, prefer
complete-feeling commits over mimicking the iterative style.
- **Burst-state awareness.** Before writing into a subsystem, run
`git log --since='1 month ago' bundles/<x>` (or `nodes/<x>.py`).
≥10 recent commits means the subsystem is in flux; read the most
recent diffs first — your assumptions about its metadata shape may
already be stale.
- **Branch names.** PRs go through self-hosted Gitea/Forgejo. Branch
names are lowercase snake_case, descriptive
(`debian-13`, `htz.mails_debian_13_squash`, `l4d2_the_next`).
## Bundlewrap-version migration recipe
When the next bw major lands:
1. Read the upstream migration guide.
2. `git grep` for affected reactor / item patterns.
3. Rewrite each (one commit per pattern is fine — see Working style).
4. Bump `requirements.txt` last.
Pattern from commit `186d503` (bw 4 → 5).

View file

@ -0,0 +1,59 @@
# Handoff — agent-friendly docs for ckn-bw (status: complete)
The 2026-05-10 work is finished. This file replaces the original
multi-page handoff (which described work that's now done plus a
Phase 2 that was dropped). Kept as a breadcrumb so future agents
know what landed.
## What landed
- **Phase 0** (`730625e`) — top-of-file docstrings on every
`libs/*.py` and `hooks/*.py`; `# purpose:` headers on every
`bin/*` script. Discovery is `ls + head -1`.
- **Phase 1** (`04558a9`) — root `AGENTS.md` (with `CLAUDE.md`
symlink), `docs/agents/{conventions,commands}.md`, per-area
`AGENTS.md` for `bundles/`, `nodes/`, `groups/`, `libs/`,
`hooks/`, `data/`, `items/`, `bin/`. Mechanism-focused; no
enumeration of contents.
- **Convention pivot** (`9e1bb2a`) — per-bundle docs are plain
`README.md`, not `AGENTS.md`. No template. `bundles/AGENTS.md`
"Per-bundle README" gives loose orientation pointing at the
more substantial existing READMEs.
`bw test` exit 0 throughout. All internal links in the new docs
resolve. Commits sit on local `master`, none pushed.
## Where current truth lives
- Root entry: [`AGENTS.md`](../../../AGENTS.md) (symlinked to
`CLAUDE.md`).
- Repo conventions + bw command deltas:
[`docs/agents/`](../../agents/).
- Per-area docs: `<dir>/AGENTS.md` for each top-level dir.
- Per-bundle docs: existing `README.md` files (~33 of them);
guidance in [`bundles/AGENTS.md`](../../../bundles/AGENTS.md).
- Self-describing files: `libs/*.py`, `hooks/*.py`, `bin/*`.
## Source documents
- Spec: [`../specs/2026-05-10-agent-friendliness-design.md`](../specs/2026-05-10-agent-friendliness-design.md)
— read its `§0. Revisions` first; §3 and §7 are now pre-pivot
intent only.
- User-stories validation: [`../specs/2026-05-10-user-stories-from-history.md`](../specs/2026-05-10-user-stories-from-history.md).
- Implementation plan: [`../plans/2026-05-10-agent-friendliness-plan.md`](../plans/2026-05-10-agent-friendliness-plan.md)
(frozen pre-pivot artifact; see its top-of-file note).
## What's open
Nothing planned. Per-bundle README updates are reactive — when a
bundle gets materially edited, top up its `README.md` then. The
~33 existing READMEs vary in quality and shape; that's accepted.
Two minor items the maintainer may want to clean up at some point:
- The implementation plan and the spec's §3 / §7 still describe
the pre-pivot per-bundle `AGENTS.md` convention. Spec's §0 flags
this; the plan is a frozen artifact and probably stays as-is.
- The original handoff's verification list mentioned `bw bundles`,
which isn't a real bw subcommand. Not user-facing in any current
doc.

View file

@ -0,0 +1,505 @@
# Implementation plan — agent-friendliness docs
> **Note (post-execution):** This plan is a frozen pre-pivot artifact.
> It captures the approach as designed before Phase 1 landed and the
> per-bundle convention pivoted from `AGENTS.md` to `README.md`.
> Sections describing per-bundle `AGENTS.md`,
> `bundles/AGENTS.template.md`, and the Phase-2 seed-bundle migration
> reflect the original intent, not what shipped. For the current
> shape, read the spec's §0 Revisions
> ([`../specs/2026-05-10-agent-friendliness-design.md`](../specs/2026-05-10-agent-friendliness-design.md))
> and the handoff status note
> ([`../handoffs/2026-05-10-implementation-handoff.md`](../handoffs/2026-05-10-implementation-handoff.md)).
> Kept in tree as a record of how the work was scoped and what
> validation findings (workflow + user-story) shaped Phase 1.
## Context
This BundleWrap config repo (`ckn-bw`, ~22 nodes / 103 bundles) currently has
no agent-facing orientation: no `CLAUDE.md` / `AGENTS.md`, only ~10 of 103
bundles have a `README`, and the root `README.md` is a personal TODO. Agents
landing cold spelunk to figure out conventions (demagify magic strings,
`metadata_reactor` patterns, lib helpers, what's safe to run).
Goal: ship one PR that scaffolds the documentation surface described in
`docs/superpowers/specs/2026-05-10-agent-friendliness-design.md`, plus a
second PR seeding 10 high-value bundle docs. After this work, an agent can
land useful work in nodes / groups / bundles / libs without trial-and-error,
and never invokes a state-mutating `bw` command without explicit user request.
**Corrections since the spec was written.**
- Verification on 2026-05-10 found the active venv ran upstream `bundlewrap 4.24.0`,
the README's fork install was stale, and the user has since upgraded the
venv to PyPI `bundlewrap 5.0.3`.
- The user refreshed `/Users/mwiegand/Projekte/bundlewrap-fork` to track upstream
master at commit `a97cdb13` (also version 5.0.3), and decided to make the
fork agent-friendly *first* (separate session, separate plan), then return
here. The ckn-bw plan therefore drops the `docs/agents/bundlewrap/` folder
it originally planned — bundlewrap-language docs live in the fork now.
This plan reflects the reduced ckn-bw scope. It is gated on the fork's
`AGENTS.md` existing, since this repo's root `AGENTS.md` links out to it.
See the handoff document delivered separately for the fork-session work.
## Scope
Two PRs.
- **PR1 — scaffolding.** Root `AGENTS.md`, `CLAUDE.md` symlink, the
`docs/agents/` tree (just `conventions.md` and `commands.md` — no
`bundlewrap/` folder; that lives in the fork), all eight per-area
`AGENTS.md` files, the per-bundle template, the docstring/header pass on
`libs/*.py` / `hooks/*.py` / `bin/*`, and the README cleanup (remove the
stale fork section, keep everything else). Spec correction in the same PR.
- **PR2 — seed bundles.** Per-bundle `AGENTS.md` for the 10 seed bundles;
fold and remove any existing per-bundle `README.md` in those.
Phase 3 from the spec ("leave-as-you-go") is a convention, not a code task —
captured in `bundles/AGENTS.md` as a contribution rule.
## Approach (recommended)
### PR1 — scaffolding
Order is dependency-respecting; each step's output is referenced by later
steps' link targets.
**Precondition:** the fork's root `AGENTS.md` exists at
`/Users/mwiegand/Projekte/bundlewrap-fork/AGENTS.md`. PR1 links out to it,
so PR1 cannot land cleanly until the fork session has produced that file.
1. **Spec correction.** In
`docs/superpowers/specs/2026-05-10-agent-friendliness-design.md`:
- Replace "fork" framing with the actual reality: venv runs `bundlewrap
5.0.3` (PyPI install today), the user maintains a separate fork at
`github.com/CroneKorkN/bundlewrap` whose `master` tracks upstream and
whose `AGENTS.md` is the canonical bundlewrap-language reference.
- Replace the `docs/agents/bundlewrap/` folder content (Section 2 IA,
Section 6) with a single bullet: "bundlewrap-language reference lives
in the fork at `<URL>/AGENTS.md`; ckn-bw links to it from root
`AGENTS.md` and `conventions.md`."
- Update Section 4 quickstart and Section 6 `conventions.md` accordingly.
- Single commit before any other ckn-bw docs are written.
2. **`docs/agents/conventions.md`** (~80120 lines): demagify magic strings;
bundlewrap version note (`5.0.3`, link to the fork's `AGENTS.md` for
language reference); group inheritance order; node/group naming
conventions; the `eval()` idiom in `nodes.py` / `groups.py` and what
that constrains for editors/agents (no top-level imports, etc.);
do-not-touch file list (`.secrets.cfg*`, `.venv`, `.cache`,
`.bw_debug_history`, `.envrc`).
3. **`docs/agents/commands.md`** (~3050 lines, slimmed). The fork's
`AGENTS.md` (at `https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md`)
now carries the canonical bw runbook — tiers, after-change table,
hash-diff workflow, `bw debug` sketch — verified against 5.0.3 source.
This file shrinks to ckn-bw-specific deltas:
- One-line lead pointing at the fork's `AGENTS.md` for the canonical
read-only allowlist + after-change runbook.
- **Apt-key after-change row** (S7 finding): editing
`data/apt/keys/*.{asc,gpg}` → first verify with
`gpg --show-keys <newkey>` locally + fingerprint diff against the
expected source. Trial via `bw apply` is the *failure* path (a wrong
key blocks unattended upgrades cluster-wide). Not in the fork's
runbook because it's repo-specific.
- **`*.py_` suspended-node interaction with `bw nodes`**: a node file
ending in `.py_` is silently excluded from the loader; `bw nodes`
won't list it. Document this so an agent doesn't think a node is
missing when it's actually parked.
- **Vault magic-string handling**: never echo decrypted output, even
in `bw debug` exploration. Cross-link to `conventions.md#secrets`.
4. **Per-area `AGENTS.md`** — eight files, mechanism-focused, no
enumeration. Order: `bundles/`, `nodes/`, `groups/`, `libs/`, `hooks/`,
`data/`, `items/`, `bin/`. Each ~3080 lines, same five-section shape
from spec Section 5.
5. **`bundles/AGENTS.template.md`** — the per-bundle template (spec
Section 3 verbatim, with placeholder text in each section).
6. **Root `AGENTS.md`** — written so all link targets exist. Spec
Section 4, with the "Conventions you must know" first bullet now
pointing at the fork's `AGENTS.md` (canonical bundlewrap language)
instead of an internal `bundlewrap/` folder. Quickstart bullet about
the fork updated to: "bundlewrap reference lives in `<fork-URL>`
read first if new to bundlewrap." Then create `CLAUDE.md` as a symlink:
`ln -s AGENTS.md CLAUDE.md`.
7. **Docstring/header pass** — for every `libs/*.py` and `hooks/*.py`
without a top-of-file module docstring, add a one-liner. For every
`bin/*` script without a header comment, add a `# purpose: …` line. Do
not touch files that already have one. Mechanical: read each file's
first ~10 lines, decide, edit only if missing.
8. **Root `README.md` cleanup** — remove the stale "install bw fork"
section (the `pip3 install --editable git+file:///…/bundlewrap-fork…`
block) since the venv runs PyPI 5.0.3. Leave the rest of the README
untouched.
### PR2 — seed bundles
For each of the 10 seed bundles, write `bundles/<name>/AGENTS.md` from
`bundles/AGENTS.template.md`. Where a bundle already has `README.md`, fold
its content into the new `AGENTS.md` and remove the old `README.md` in the
same commit.
Seed list (from spec Section 7, validated empirically 2026-05-10):
| # | Bundle | Existing README? | Notes |
|---|---|---|---|
| 1 | `monitored` | check | meta-bundle, often misunderstood |
| 2 | `postgresql` | check | foundational |
| 3 | `wireguard` | check | own lib + bin script |
| 4 | `routeros-monitoring` | no | most-churned bundle (15 commits / 18mo); replaces `php` per user-story rebalance |
| 5 | `apt` | check | own lib |
| 6 | `nginx` | check | web foundational |
| 7 | `telegraf` | check | high cross-bundle ripple |
| 8 | `backup` | check | cross-node coordination |
| 9 | `letsencrypt` | check | cross-cutting |
| 10 | `nextcloud` | yes (verified) | complex, recent activity |
For each: derive `Metadata` dict by reading the bundle's `metadata.py` and
running `bw metadata <one-node-with-the-bundle>` to confirm the resolved
shape. Derive `Produces` from `items.py`. Derive `Depends on` by checking
which other bundles' artifacts (apt packages, systemd services) the
bundle's reactors and items reference. Use the ccc index (built 2026-05-10)
for fast cross-bundle lookups when filling `Depends on`.
## Workflow validation findings (2026-05-10) — content additions
Traced the "implement a new bundle" workflow against the planned docs.
Natural agent path is root → `bundles/AGENTS.md` → example bundle → write
files → wire to a node → verify. Seven gaps found; six are one-line
additions to `bundles/AGENTS.md`, one is a rewrite of root §6's example
pointers (~8 lines). All low-cost; fold into the corresponding files when
written in PR1.
1. **`bundles/AGENTS.md` — "Before you start" header.** `conventions.md`
is off the natural path; call it out as required reading at the top,
not just in "see also." This repo's idioms (vault calls in reactors,
`repo.libs.hashable.hashable(...)`, demagify) live there. An agent
who skips it will write subtly wrong code (e.g. dict-in-set
`TypeError`, vault calls in the wrong place).
2. **Root `AGENTS.md` §6 — use-case keyed example pointers.** Replace the
spec's "small bundle / complex bundle / node file" with one-line
pointers per pattern: vault usage, templated files, cross-bundle
reactor writing, `download` custom item. Pick concrete bundles at write
time (grep for the patterns to find good exemplars). ~8 lines.
3. **`bundles/AGENTS.md` "How to add" — explicit wiring step.** Add a
numbered step: "(4) Wire to nodes — see `groups/AGENTS.md` for
application-style group wiring or `nodes/AGENTS.md` for direct
attachment via a node's `bundles` list." Currently the wiring step is
implicit; agent has to discover by cross-reading two area docs.
4. **`bundles/AGENTS.md` Conventions — bundle naming.** One line:
bundle directory names are lowercase with hyphens (e.g. `backup-server`,
`bind-acme`, `dm-crypt`); avoid underscores. Verify the convention by
`ls bundles | grep _` before writing — if the convention is mixed,
document the actual rule.
5. **`bundles/AGENTS.md` "See also" — items and templates.** Cross-link
to `items/AGENTS.md` (for the `download` custom item and how to write
new custom item types) and to the fork's
`docs/content/guide/item_file_templates.md` (template syntax). Both
are common needs an agent writing a new bundle hits early.
6. **`bundles/AGENTS.md` — first-thing-to-run after writing.** One-liner
pointing at `commands.md` with the canonical sequence: `bw test`
(sanity) → `bw items <node>` (do items show up?) → `bw hash <node>`
(changed as expected?). Saves the agent from hunting the runbook.
7. **`bundles/AGENTS.template.md` — empty-section guidance.** Add a note
at the top of the template: "For a brand-new bundle without consumers
yet, leave `Depends on` and `Produces` empty or marked TBD; fill in
after the first verify run."
## User-story validation findings (2026-05-10) — additional content adds
Empirical user-story extraction from 1169 commits (full history, with
detailed analysis of the last 222 commits / 18 months) is at
`docs/superpowers/specs/2026-05-10-user-stories-from-history.md`. It
identified 21 recurring user stories. Coverage assessment vs the planned
docs: 5 ✅ / 13 ⚠ / 3 ❌. Below are the 16 additional content adds, grouped
by target file. Each is one paragraph or less. The ❌ items shape an
agent's *judgment* (highest value); the ⚠ items shape lookups.
### Root `AGENTS.md`
- **F9 — Personal TODO callout.** "Note: `README.md` is the maintainer's
personal scratchpad, not project documentation. Onboarding lives here in
`AGENTS.md`." One sentence in §1 ("What this repo is").
### `docs/agents/conventions.md`
- **S4 ❌ — Iterative-commit workflow style.** "User commits are
iterative checkpoints, not landing-ready snapshots. Terse messages
(`+`, `fix`, `whitespace`, `dowsnt exist`) and successive 'fix' commits
on the same file are normal. Don't rebase WIP without asking. As an
agent, prefer to land complete-feeling commits rather than mimic the
iterative style."
- **S5 ❌ — Burst-state awareness.** "Before writing into a subsystem,
check `git log --since='1 month ago' bundles/<x>` (or `nodes/<x>.py`,
etc.). If it shows ≥10 recent commits, the subsystem is in flux and
your assumptions about its metadata shape may already be stale. Read
the most recent diffs first."
- **S11 ❌ — Suspension idiom ("for now / disable / dummy / offline").**
"Commits with these markers indicate deliberate suspension, not bugs.
If you encounter a stub or commented-out block, check
`git log -- <file>` for the suspension reason before re-enabling. The
user reverses these manually when ready."
- **F6 — `_old` / `_old2` soft-delete pattern.** "Suffixed-with-`_old`
directories are the user's recovery buffer during big refactors. Don't
delete them without asking, even if they look orphaned."
- **F8 — Branch naming for PRs.** "PRs go through self-hosted
Gitea/Forgejo. Branch names are lowercase-snake_case descriptive
(`debian-13`, `htz.mails_debian_13_squash`, `l4d2_the_next`)."
- **S20 — Bundlewrap version-migration recipe (optional).** "When the
next major bw version lands: read upstream migration guide → grep for
affected reactor patterns → rewrite each → bump `requirements.txt`
last." Captures the pattern from `186d503` (bw 4 → 5). Useful given
the maintainer's tool-design pivot.
### `docs/agents/commands.md`
- **S7 — Apt-keys verification.** Add a row to the after-change table:
`data/apt/keys/*.{asc,gpg}` → first check is `gpg --show-keys
<newkey>` locally + visual diff against expected fingerprint. Trial
via `bw apply` is the *failure* path (a wrong key blocks unattended
upgrades cluster-wide).
- ~~**S21 — `bw debug` content sketch.**~~ Resolved upstream: the fork's
`AGENTS.md` now carries the canonical `bw debug` content sketch (probes
for `repo.get_node(...).metadata`, `repo.libs.<x>`, `repo.path`). No
ckn-bw addition needed — Approach step 3 already points at the fork.
### `bundles/AGENTS.md`
- **S3 — Template recognition.** One paragraph: "Files under
`bundles/<x>/files/` are static unless the `file:` item declares
`content_type='mako'` or the file extension triggers templating
(see fork's `docs/content/guide/item_file_templates.md`). To check:
read the matching `file:` entry in `items.py`."
- **S13 — How to remove a bundle.** 5-line section, symmetric to "How
to add": "(1) `git grep '<name>'` to find references in nodes/groups/
other bundles; (2) remove those references; (3) `rm -rf bundles/<x>/`;
(4) `bw test` and `bw nodes` to confirm clean."
- **S18 — README transition state.** "If a bundle has both `README.md`
and `AGENTS.md`, `AGENTS.md` is canonical; the README is being phased
out. ~23 bundle READMEs remain after the seed PR — Phase 3 leave-
as-you-go folds them in over time."
### `bundles/AGENTS.template.md`
- **S12 — Optional `## Writes into` section.** Add to template (after
`## Depends on`): "List other namespaces this bundle's `defaults` or
reactors write into (e.g. nextcloud writes into `apt.packages` and
`archive.paths`). Skip section if none — most bundles don't write
cross-namespace, but the ones that do create the highest-blast-
radius surprises."
### `nodes/AGENTS.md`
- **S2 — Silent eval-load-failure pitfall.** "Node files are `eval()`'d.
A syntax error or top-level `import` causes the loader to silently
drop the node. If `bw nodes` reports fewer nodes than expected, check
`groups.py` (the user added explicit error printing in commit
`dc40295` after being bitten)."
- **S9 — `*.py_` suspend convention.** "Appending `_` to a node
filename (e.g. `htz.l4d2.py_`) parks it without loading. Used to keep
decommissioned-but-not-deleted node configs in tree."
- **S9 — Symmetric "How to add a node" workflow.** Numbered steps
parallel to `bundles/AGENTS.md` "How to add": (1) create
`nodes/<location>.<role>.py` with `eval()`-safe expression syntax,
(2) populate `id`, `hostname`, `groups`, `bundles`, `metadata`, (3)
add to relevant `groups/<area>/<x>.py` if group membership is the
attachment point, (4) verify with `bw nodes`, `bw nodes <node> -a groups`,
`bw metadata <node>`.
- **S19 — Node-rename failure mode.** "Renaming a node file renames
the node. Vault entries (via `!password_for:<node>`), `bw hash`
records, and ssh known_hosts associations all key on node name —
search and replace before renaming, or vault lookups silently
return new (wrong) values."
### `groups/AGENTS.md`
- **S10 — Family-file pattern.** "OS variants commonly share a
`*-common.py` parent (e.g. `debian-13-common.py` shared by
`debian-13.py` and `debian-13-pve.py`). Use this when introducing
related-but-distinct OS group families."
- **S10/S15 — New-OS-variant recipe.** "To introduce a new OS major:
(1) add `groups/os/debian-N.py` + `debian-N-common.py` parallel to
the existing files (don't edit in place); (2) add
`data/apt/keys/debian-N-*.{asc,gpg}`; (3) bump dependent bundles
that branch on OS string (e.g. `bundles/bind/items.py`); (4) bump
affected nodes' `groups` lists one at a time; (5) delete the old
OS file when no node references it."
### `data/AGENTS.md`
- **S7 — Two distinct content models.** "`data/apt/keys/` holds binary
GPG keys consumed by `bundles/apt`; `data/grafana/rows/` holds Python
modules for Mako-templated dashboards. Same directory shape, different
content models — when adding a new data subdir, declare which model
it follows."
- **F4 — `data/` vs `bundles/<x>/files/` heuristic.** "If a data asset
is read by exactly one bundle, prefer `bundles/<x>/files/`. Use
`data/` for shared/multi-consumer artifacts. Single-instance evidence:
`78a8abc` moved `mikrotik.mib` from data/ into the bundle for this
reason."
### `hooks/AGENTS.md`
- **S17 — Broken-hook failure mode.** "A hook that errors at load time
breaks every `bw` command that fires that lifecycle (including
`bw test`, defeating the obvious diagnostic). Test new hooks in
isolation first: `bw debug` then `import sys; sys.path.insert(0,
'hooks'); import <hookmodule>`. Iterate there until the import is
clean."
### `libs/AGENTS.md`
- **S16 — Find-consumers snippet.** Add as a one-liner under
Pitfalls: "Before changing a lib's API, find consumers:
`git grep -l 'repo.libs.<x>'`. Lib changes have repo-wide blast
radius — every bundle that imports the lib re-evaluates."
### `bin/AGENTS.md`
- **S14 — `bin/script_template`.** "When introducing a new operator
script, start from `bin/script_template` (the user maintains it as
the canonical starter)."
## Phase 2 seed list rebalance
User-story analysis (story 5, subsystem-burst evidence) found that 3 of
5 recent burst targets are unseeded. Strongest single swap:
- **Drop `php`** (8 node refs but ~zero recent commits — usage hub that
doesn't change). Add **`routeros-monitoring`** (15 recent commits, the
most-touched bundle in 18 months; not a dependency hub but high churn).
Optional second swap (judgment call):
- **Drop `apt`** (6 refs, has own lib, foundational but stable) and add
**`bootshorn`** (recent burst target, has its own subsystem). Or keep
`apt` for usage-hub coverage and accept that the seed misses bootshorn
— Phase 3 covers it lazily.
Default this plan to the **single swap** (php → routeros-monitoring);
note `apt → bootshorn` as a second-swap option if the user wants it.
## Critical files
**New:**
- `AGENTS.md` (root)
- `CLAUDE.md` (symlink → `AGENTS.md`)
- `docs/agents/conventions.md`
- `docs/agents/commands.md`
- `bundles/AGENTS.md`, `bundles/AGENTS.template.md`
- `nodes/AGENTS.md`, `groups/AGENTS.md`, `libs/AGENTS.md`,
`hooks/AGENTS.md`, `data/AGENTS.md`, `items/AGENTS.md`, `bin/AGENTS.md`
- `bundles/{monitored,postgresql,wireguard,routeros-monitoring,apt,nginx,telegraf,backup,letsencrypt,nextcloud}/AGENTS.md`
**Modified:**
- `docs/superpowers/specs/2026-05-10-agent-friendliness-design.md` (fork → upstream correction).
- `README.md` (remove stale "install bw fork" section).
- `libs/*.py`, `hooks/*.py` (add module docstrings where missing).
- `bin/*` (add `# purpose:` header where missing).
**Deleted:**
- `bundles/<name>/README.md` for each of the ~10 seed bundles that have one
(content folded into `AGENTS.md`).
## Existing utilities & assets to reuse
- **Spec** at `docs/superpowers/specs/2026-05-10-agent-friendliness-design.md`
the source of truth for what each file contains. Sections 3, 4, 5, 6 of
the spec are nearly copyable into the actual files.
- **User-story validation doc** at
`docs/superpowers/specs/2026-05-10-user-stories-from-history.md`
21 stories grounded in git history with concrete commit evidence.
When writing per-bundle docs, look up the relevant story for evidence
of typical changes (e.g. for `nextcloud`, see Story 1 + Story 5
l4d2-style burst comparison). The "Implications for agent docs"
paragraphs in each story map directly to the additions in §"User-
story validation findings" above.
- **ccc index** built at `.cocoindex_code/` on 2026-05-10 (768 chunks, 340
files) — useful in PR2 for finding bundles that consume a metadata key
or import a particular lib.
- **Bundlewrap CLI** for verification: `bw metadata <node>`, `bw items <node>`,
`bw test`, `bw hash`. Read-only; safe to run during writing.
- **Existing bundle READMEs** at `bundles/{freescout,influxdb2,dm-crypt,gcloud,flask,nextcloud,build-server,raspberrymatic-cert,letsencrypt,nodejs}/README.md`
(and any others — verify with `find bundles -name README.md`) — content
to fold into the matching `AGENTS.md`. These are the only existing
human-prose source for those bundles; do not lose information when migrating.
## Verification
Run after PR1:
1. `bw test` — repo-level sanity (passes today; should still pass).
2. `bw nodes`, `bw groups`, `bw bundles` — sanity that loaders work after
the docstring/header additions.
3. Check every internal link in `AGENTS.md` and `docs/agents/**.md`
resolves to a real file. A tiny shell loop with `grep -oE '\]\([^)]+\.md[^)]*\)'`
then `test -f` each path.
4. `readlink CLAUDE.md` resolves to `AGENTS.md`.
5. `grep -L '"""' libs/*.py hooks/*.py` reports zero files (every lib/hook
has a module docstring).
6. `grep -L '^# purpose' bin/*` reports zero non-binary scripts.
7. `git grep -i "bw fork\|bundlewrap-fork"` reports only the corrected
docs locations (no leftover fork references in `README.md`).
8. **Workflow walk-through.** Trace the "implement a new bundle" path
end-to-end against the written docs: root → `bundles/AGENTS.md`
example pointer → can the writer locate vault/hashable/template
guidance from there without spelunking? Confirms the §"Workflow
validation findings" fixes actually closed the gaps.
Run after PR2:
1. `bw test` — still green.
2. For each seed bundle `<x>`: pick one node that has it, run
`bw metadata <node>` and confirm the keys listed in
`bundles/<x>/AGENTS.md` `Metadata` section actually appear in the
resolved metadata. Catches drift between docs and reality.
3. `find bundles -name README.md` — confirm none exist for the 10 seed
bundles (folded into `AGENTS.md`).
4. Each new `bundles/<x>/AGENTS.md` follows the template structure
(`grep -L '^## Metadata' bundles/*/AGENTS.md` reports zero).
## Non-goals (re-asserted from spec)
- No tooling changes (no `bw` wrapper, no Makefile, no lint, no CI).
- No code refactoring, renaming, or splitting bundles.
- No mass-fill of the remaining ~93 bundles up front.
- No upstream contribution to bundlewrap (acknowledged future work).
- The root `README.md` keeps everything except the fork section.
## Risks & mitigations
- **Drift between docs and code.** Mitigation: per-bundle docs are short
(low maintenance), the area docs are mechanism-focused (changes less
often than enumerations), and PR2 verification step 2 catches metadata
drift at write time.
- **PR1 is gated on the fork's `AGENTS.md` existing.** If you want to
start ckn-bw work before the fork session lands, you can stub the link
to the fork's `AGENTS.md` (e.g. point at the repo URL even if the file
isn't there yet) and merge PR1, but that risks broken links if the fork
URL or filename changes. Cleaner: do the fork session first, then PR1.
- **PR1 is now a moderate writing chunk** (~8001000 lines instead of
~15002000), since the bundlewrap folder moved out. Single PR is
comfortable. The user-story validation findings (16 small adds) push
it to ~10001200; still manageable as one PR.
- **README undercount.** The plan/spec estimated ~10 existing bundle
READMEs to fold; actual is **33** (verified 2026-05-10 from git
history). The seed list intersects with only 45 of those (nextcloud
has one verified; others — `find bundles -name README.md` will
enumerate). PR2 only folds READMEs in seed bundles; ~28 remain
untouched, addressed lazily by Phase 3. `bundles/AGENTS.md` notes
this transition state explicitly (per user-story finding §S18) so
agents reading both don't get confused.

View file

@ -2,6 +2,54 @@
Date: 2026-05-10
## 0. Revisions
Material revisions since this spec was first written, kept here so anyone
reading the spec sees the current shape rather than the original intent.
- **Fork pivot.** Originally the spec planned a `docs/agents/bundlewrap/`
folder (`README.md` + `items.md` + `metadata.md`) explaining
bundlewrap-the-language inside ckn-bw. That folder is gone; the
maintainer maintains a personal bundlewrap fork at
`github.com/CroneKorkN/bundlewrap` whose root `AGENTS.md` carries the
canonical agent-oriented bundlewrap-language reference. ckn-bw's docs
link out to the fork instead. The venv installs editable from the fork
(`-e git+https://github.com/CroneKorkN/bundlewrap.git@main`).
- **`commands.md` slimmed** from ~80120 lines to ~3050 lines: the fork's
`AGENTS.md` carries the canonical bw runbook (read-only allowlist,
after-change table, hash-diff workflow, `bw debug` sketch); ckn-bw's
`commands.md` shrinks to repo-specific deltas (apt-key verification,
`*.py_` suspended-node behavior, vault-echo guidance).
- **Phase 2 seed-list rebalance.** `php` swapped out for
`routeros-monitoring` based on user-story analysis: php is a low-churn
usage hub (8 refs but ~zero recent commits); routeros-monitoring is
high-churn (15 commits in 18 months), exactly where seeded docs pay off
most. See plan for empirical justification.
- **bw-syntax corrections** found by per-task code-review during the fork's
AGENTS.md implementation, synced in: `bw items <node> <id> -p` does not
exist (use bare or `--preview`); `bw hash` accepts only literal node /
group names (selectors like `bundle:<x>` work for `bw nodes` etc., but
not for `bw hash`); `bw groups -n <node>` does not exist (use
`bw nodes <node> -a groups`).
- **Workflow + user-story validation findings** (16 small content adds
across area docs, the per-bundle template, `commands.md`, and
`conventions.md`) are recorded in the implementation plan rather than
back-fitted into this spec — they're additions to file content, not
scope changes.
- **Per-bundle docs are `README.md`, not `AGENTS.md`** (revised
2026-05-10, after Phase 1 scaffolding landed). The spec originally
specified one balanced `AGENTS.md` per bundle (§3 template) plus a
Phase 2 seed migration that folded existing READMEs into new
AGENTS.md files (§7). After Phase 1 landed, the maintainer flagged
that the rigid template wouldn't survive contact with the existing
READMEs (which range from one-paragraph balanced docs to operational
scratchpads — see `bundles/{flask,dm-crypt,apt,nextcloud}/README.md`).
Resolution: one `README.md` per bundle, no fixed shape, no template;
Phase 2 dropped; existing READMEs stay in place under leave-as-you-go.
Current convention lives in `bundles/AGENTS.md` "Per-bundle README".
**Sections §3 and §7 are no longer authoritative — read them as
pre-pivot intent only.**
## 1. Goals & non-goals
**Goal.** Make this BundleWrap config repo legible to agents (and humans) so an
@ -17,11 +65,10 @@ spelunking and without unsafe side effects.
- Per-bundle `AGENTS.md`: one balanced doc per bundle, replacing existing
bundle `README.md` files. Template provided.
- `docs/agents/conventions.md`: repo-specific idioms (vault magic strings,
custom bundlewrap fork, files-not-to-touch).
- `docs/agents/commands.md`: read-only `bw` command allowlist and an
after-change runbook keyed by what was edited.
- `docs/agents/bundlewrap/`: a focused folder explaining bundlewrap-as-used-here.
Three files at first: `README.md`, `items.md`, `metadata.md`.
bundlewrap-fork install pointer, files-not-to-touch).
- `docs/agents/commands.md`: ckn-bw-specific deltas to the fork's bw
runbook (apt-key verification, suspended-node behavior, vault-echo
guidance). Canonical bw command reference lives in the fork's `AGENTS.md`.
- A docstring/header pass on `libs/*.py`, `hooks/*.py`, `bin/*` so each
individual file self-describes.
- Phase 2 seed: per-bundle `AGENTS.md` for 10 bundles selected empirically.
@ -45,11 +92,8 @@ ckn-bw/
├── docs/
│ └── agents/
│ ├── conventions.md
│ ├── commands.md
│ └── bundlewrap/
│ ├── README.md
│ ├── items.md
│ └── metadata.md
│ └── commands.md # ckn-bw deltas; canonical bw runbook is in
│ # the fork's AGENTS.md (linked from here)
├── bundles/
│ ├── AGENTS.md # what bundles are, how they compose
│ ├── AGENTS.template.md # template for per-bundle docs
@ -68,7 +112,10 @@ ckn-bw/
**Reading order an agent should follow.** Root `AGENTS.md` → relevant area
`AGENTS.md` → specific `bundles/<x>/AGENTS.md``docs/agents/conventions.md`
or `docs/agents/bundlewrap/<file>` only when something non-obvious comes up.
when a repo-specific idiom is in play → fork's `AGENTS.md` (at
`https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md`) for any
bundlewrap-language question (item types, dep keywords, metadata reactor
semantics).
**Per-area files (not just root).** An agent editing `bundles/nextcloud/items.py`
already has `bundles/AGENTS.md` and `bundles/nextcloud/AGENTS.md` adjacent in
@ -149,8 +196,11 @@ Target ~150 lines. Sections in order:
`.envrc`. Everything else is editable, but treat `hooks/` and `items/`
(custom item types) with extra care — they affect bw's behavior or item
resolution across the whole repo.
- Uses a custom **bundlewrap fork**, not upstream — check
`docs/agents/conventions.md` before assuming upstream behavior.
- Repo runs editable from the maintainer's bundlewrap fork
(`github.com/CroneKorkN/bundlewrap`, branch `main`); behavior tracks
upstream main but the fork's `AGENTS.md` is the canonical
bundlewrap-language reference. See `docs/agents/conventions.md` for
install detail.
- Prefer adding helpers to `libs/` over duplicating logic across bundles.
3. **Layout map.** Terse, link-rich. One line per top-level dir, each linking
to that area's `AGENTS.md`.
@ -159,12 +209,17 @@ Target ~150 lines. Sections in order:
`nodes.py` and `groups.py` (root) are the loaders that walk the dirs and
run `demagify`.
5. **Conventions you must know.** One-line summary + link for each:
- `docs/agents/bundlewrap/README.md` — read first if new to bundlewrap.
`items.md` and `metadata.md` are deep dives for the hard parts.
- Fork's `AGENTS.md`
(`https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md`) —
read first if new to bundlewrap. Carries the safety envelope, the
after-change runbook, and cheat-sheets for item dep keywords +
`metadata.py` pitfalls.
- `docs/agents/conventions.md#secrets` — secrets / demagify magic strings.
- `docs/agents/conventions.md#fork` — custom bundlewrap fork.
- `docs/agents/conventions.md#bundlewrap-version` — install pointer
(editable from the fork's `main`).
- `docs/agents/conventions.md#groups` — group inheritance order.
- `docs/agents/commands.md` — safe `bw` commands and after-change checks.
- `docs/agents/commands.md` — ckn-bw-specific deltas to the bw runbook
(apt keys, suspended nodes, vault-echo guidance).
- Lib helpers — see top-of-file docstrings in `libs/*.py`.
6. **Where to look for examples.** Pointers to a small bundle, a complex
bundle, and a node file.
@ -194,7 +249,8 @@ Per-area specifics:
- **`bundles/AGENTS.md`** — bundle anatomy (`items.py`, `metadata.py`, `files/`,
`templates/`), where helpers go, when to extract to `libs/`. Links out to
`docs/agents/bundlewrap/items.md` and `metadata.md` for language-level detail.
the fork's `AGENTS.md` (item types, dep keywords, metadata reactor
semantics) for language-level detail.
- **`nodes/AGENTS.md`** — `eval()` loading mechanism via `nodes.py`, demagify
magic-string syntax, naming convention pattern (`<location>.<role>.py`).
**Pitfall:** because node files are `eval()`'d, no top-level imports — only
@ -222,9 +278,10 @@ Per-area specifics:
`!32_random_bytes_as_base64_for:`. What each does, where they're allowed
(node files, evaluated through `nodes.py`), why agents must never echo
decrypted values.
- **Custom bundlewrap fork.** How it's installed
(`pip install --editable git+file:///…/bundlewrap-fork@main`), implications
(don't assume upstream-only behavior), pointer to the fork source.
- **Bundlewrap version / install.** Repo runs editable from the maintainer's
personal fork: `pip install -e git+https://github.com/CroneKorkN/bundlewrap.git@main#egg=bundlewrap`.
Captured in `requirements.txt`. The fork's `main` tracks upstream main; the
fork's `AGENTS.md` is the canonical bundlewrap-language reference.
- **Group inheritance order** & how metadata merges
(`all.py` → location → os → machine → applications → node).
- **Naming conventions** for nodes (`<location>.<role>.py`) and groups
@ -232,99 +289,53 @@ Per-area specifics:
- **Files agents must not modify.** `.secrets.cfg*`, `.venv`, `.cache`,
`.bw_debug_history`, `.envrc`.
### `commands.md` (~80120 lines)
### `commands.md` (~3050 lines)
**Side-effect model** (paragraph up front):
The fork's `AGENTS.md` is the canonical bw runbook — read-only command
allowlist, after-change table, hash-diff workflow, `bw debug` sketch,
verified against 5.0.3 source. ckn-bw's `commands.md` carries only
repo-specific deltas:
- `bundles/<x>/metadata.py` `defaults` and `@metadata_reactor` can write into
*any* namespace (e.g. nextcloud's metadata writes into `apt.packages` and
`archive.paths`). Changing it can ripple into other bundles' inputs.
- `libs/<x>.py` is imported by both `items.py` and `metadata.py` across many
bundles — biggest blast radius.
- `groups/*.py` changes membership (which bundles a node gets) and merged
metadata.
- `bw hash` is the primary integrated check because it captures bundle
membership + metadata + items + file content.
**Read-only command reference** (one line each):
`bw hash`, `bw metadata`, `bw items`, `bw items <node> <id> -p`, `bw nodes`,
`bw groups`, `bw verify`, `bw debug`, `bw test`, `bw plot`. Each tagged
read-only.
**After-change checks, keyed by what you changed:**
| You changed | First check | Drill-in |
|---|---|---|
| `bundles/<x>/items.py` | `bw hash <node>` for a node with bundle `<x>` | `bw items <node> <id> -p`; `bw verify <node>` |
| `bundles/<x>/metadata.py` | `bw hash` for *all nodes including bundle `<x>`* (reactors can ripple beyond `<x>`'s namespace) | `bw metadata <node>`; `bw metadata <node> -k <key>` |
| `bundles/<x>/files/*` (template) | `bw hash <node>` | `bw items <node> <path> -p` |
| `groups/*.py` | `bw hash` every node in/near the group | `bw groups -n <node>`; `bw metadata <node>` |
| `libs/<x>.py` | `bw hash` **all** nodes (cheap, no network) — biggest blast radius | `bw debug` to inspect helper outputs |
| `nodes/<x>.py` | `bw hash <node>` | `bw metadata <node>` |
| `hooks/*.py` | re-run the `bw` command whose lifecycle the hook hooks | — |
| Anything | `bw test` — cheapest repo-level sanity | — |
**Mutating commands (forbidden without explicit user request).** `bw apply`,
`bw run`, `bw lock` — what each does and why agents must not invoke them
autonomously.
**Hash diff workflow.** Capture `bw hash > before.txt`, make change,
`bw hash > after.txt`, diff. Canonical pre/post comparison.
**Targeting.** How to scope to one node / one group (`-t`, group selectors).
### `bundlewrap/README.md` (~80 lines)
Mental model paragraph: nodes ← groups → bundles → items; metadata flow
(groups → node → metadata processors → bundle items); hooks vs items vs libs.
Glossary, one paragraph each: node, group, bundle, item, items.py, metadata.py,
metadata reactor, hook, lib, `repo.libs`. Folder index. Fork callout
(explicit "we use a fork — here's what differs (or 'nothing day-to-day')";
link to fork source). Links out to <https://docs.bundlewrap.org> for depth.
### `bundlewrap/items.md` (~200300 lines)
The item types this repo actually uses — file, pkg_apt, svc_systemd, action,
symlink, group, user, … — with common attributes and examples drawn from
this repo. The dependency keyword glossary: `needs`, `needed_by`, `triggers`,
`triggered`, `triggered_skip_for`, `tags`, `cascade_skip`, `unless` — each
with one sentence and a real example. Custom item types from `items/`
(currently `download`). "When in doubt, see upstream items reference" with
a link.
### `bundlewrap/metadata.md` (~200300 lines)
`defaults` dict semantics (deep-merge into resolved metadata, can write into
any namespace). `@metadata_reactor.provides(...)` contract: signature, return
shape, fixpoint resolution, when to declare `provides` narrowly vs broadly.
Vault integration: `repo.vault.password_for`, `repo.vault.random_bytes_as_base64_for`,
`repo.vault.decrypt`, `repo.vault.decrypt_file` — and how they tie back to
demagify magic strings in node files. `repo.libs.hashable.hashable(...)` for
putting dicts in sets — a pattern used heavily in this repo, worth explicit
example. Common pitfalls: reactor declared narrower than it writes; reactor
that doesn't reach fixpoint; reactor that reads a key it didn't declare reading.
- **One-line lead** pointing at
`https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md` for the
full runbook.
- **Apt-key after-change row.** Editing `data/apt/keys/*.{asc,gpg}`
first verify with `gpg --show-keys <newkey>` locally + fingerprint diff
against the expected source. Trial via `bw apply` is the *failure* path
(a wrong key blocks unattended upgrades cluster-wide). Not in the
fork's runbook because it's repo-specific.
- **`*.py_` suspended-node interaction.** A node file ending in `.py_` is
silently excluded from the loader; `bw nodes` won't list it. Document
this so an agent doesn't think a node is missing when it's actually
parked.
- **Vault magic-string handling.** Never echo decrypted output, even in
`bw debug` exploration. Cross-link to `conventions.md#secrets`.
## 7. Seed work & rollout
### Phase 1 — scaffolding (one PR-sized chunk)
### Phase 1 — scaffolding
1. Root `AGENTS.md` (Section 4) + `CLAUDE.md` symlink → `AGENTS.md`.
2. `docs/agents/bundlewrap/README.md`, `items.md`, `metadata.md` (Section 6).
3. `docs/agents/conventions.md` and `commands.md` (Section 6).
4. Per-area `AGENTS.md` for `bundles/`, `nodes/`, `groups/`, `libs/`, `hooks/`,
Gated on: the fork's `AGENTS.md` exists and is reachable at the URL above
(verified 2026-05-10).
1. `docs/agents/conventions.md` (Section 6).
2. `docs/agents/commands.md` (Section 6).
3. Per-area `AGENTS.md` for `bundles/`, `nodes/`, `groups/`, `libs/`, `hooks/`,
`data/`, `items/`, `bin/` (Section 5).
5. `bundles/AGENTS.template.md` so future bundle docs have something to copy.
4. `bundles/AGENTS.template.md` so future bundle docs have something to copy.
5. Root `AGENTS.md` (Section 4) + `CLAUDE.md` symlink → `AGENTS.md` (written
last so all internal link targets exist).
6. Docstring/header pass: add a one-line module docstring to any `libs/*.py`
and `hooks/*.py` lacking one; `# purpose:` header to any `bin/*` script
lacking one.
Order within Phase 1: root → bundlewrap folder → conventions → commands →
area docs → docstring pass → template. Each piece can be reviewed in
isolation; the work bisects cleanly.
Order rationale: build link targets bottom-up (conventions → commands →
area docs → template), then root last, then the docstring pass last. Each
piece can be reviewed in isolation; the work bisects cleanly.
Honest scope: the bundlewrap folder is ~600 lines of focused writing total
(80 + 250 + 250). The rest is shorter — area docs and conventions land in
30120 lines each.
Honest scope: ~8001000 lines of focused writing total now that
bundlewrap-language docs live in the fork. Area docs + conventions land
in 30120 lines each; root `AGENTS.md` is ~150 lines.
### Phase 2 — seed bundles (10)
@ -336,7 +347,7 @@ activity, validated 2026-05-10):
1. `monitored` (12 node refs) — meta-bundle, often misunderstood.
2. `postgresql` (9 refs, 3 cross-bundle).
3. `wireguard` (8 refs, has own lib + bin script).
4. `php` (8 refs).
4. `routeros-monitoring` (15 commits in 18 months — most-churned bundle).
5. `apt` (6 refs, has own lib).
6. `nginx` (4 refs, web foundational).
@ -348,6 +359,10 @@ activity, validated 2026-05-10):
9. `letsencrypt` (6 refs, cross-cutting).
10. `nextcloud` (5 recent commits, complex, actively edited).
(Original §0 noted: `php` was originally seeded but swapped for
`routeros-monitoring` after user-story analysis showed it's a low-churn
hub, while routeros-monitoring is the highest-churn target in the repo.)
For each: write one `AGENTS.md` from the template — purpose, usage, metadata
dict, produces, depends-on, gotchas. Migrate any existing
`bundles/<x>/README.md` content into it and remove the old `README.md`.
@ -362,14 +377,12 @@ dict, produces, depends-on, gotchas. Migrate any existing
## 8. Future work (not this spec)
- Contributing an `AGENTS.md` to bundlewrap upstream (or to your fork)
describing items/metadata semantics for agents — would shrink the
bundlewrap folder over time and shift authority back upstream.
- Tooling: a read-only `bw` wrapper or lint that nudges new bundles toward
having an `AGENTS.md`. Worth considering only after Phase 1+2 reveal which
conventions actually drift.
- More bundlewrap docs files (`groups-nodes.md`, `hooks.md`) if real gaps
surface during Phase 2 or Phase 3 work.
having an `AGENTS.md`. Worth considering only after Phase 1+2 reveal
which conventions actually drift.
- Pushing the fork's `AGENTS.md` upstream to `bundlewrap/bundlewrap`
it's written in a style that allows it; a follow-up the maintainer
may pursue.
## 9. Open questions / risks
@ -377,10 +390,13 @@ dict, produces, depends-on, gotchas. Migrate any existing
code. Mitigations: per-bundle docs are short (low maintenance); Phase 3
rule attaches doc updates to material code edits; the area docs are
mechanism-focused, which changes less often than enumerations.
- **Risk: bundlewrap-folder duplicates upstream.** Acknowledged trade-off.
Mitigation: the folder is scoped to *as we use it here*, with explicit
upstream links; not trying to be a full bundlewrap manual.
- **Open: which seed bundles to swap.** Phase 2 list is empirically grounded
but not rigid — `zfs` (8 refs), `bind` (4 refs, own lib), and
`routeros-monitoring` (15 recent commits, specialized) are honourable
mentions if a swap is wanted later.
- **Risk: fork drifts from upstream.** ckn-bw's docs link to the fork's
`AGENTS.md`; if the fork falls far behind upstream main, the linked
semantics might not match what real bundlewrap users see. Mitigation:
the fork tracks upstream main via periodic merges; ckn-bw's
`requirements.txt` pins `@main` so the venv stays aligned with the
fork's documented behavior.
- **Open: seed bundles.** Phase 2 list is empirically grounded but not
rigid — `zfs` (8 refs), `bind` (4 refs, own lib), and `bootshorn`
(recent burst target) are honourable mentions if a swap is wanted
later.

View file

@ -0,0 +1,714 @@
# User stories — derived from ckn-bw git history
Date: 2026-05-10
History range examined: first commit `572e29e` (2021-06-11) … HEAD `c03b033` (2026-05-10), total 1169 commits. Detailed per-file analysis covers the last 18 months (222 commits, 2024-11 → 2026-05); subject-line scans extended back 36 months for context. The pre-2024 history is dominated by the same patterns at lower velocity, so deeper sampling there did not surface new stories.
## Methodology
Three passes:
1. **Bulk extraction.** `git log --since="18 months ago" --name-status -M -C` written to `/tmp/ckn-bw-history.txt`. 1023 lines covering all 222 recent commits with file changes and rename/copy detection. This is the primary corpus.
2. **Frequency analysis.** Per-bundle / per-node / per-area churn counts; first-word commit-message frequency; A/M/D/R action breakdown (461 modifies, 95 adds, 22 deletes, 9 renames over the window); rename pairs extracted for naming-convention inference.
3. **Targeted spot reads.** `git show --stat` and full diffs on 10 commits representative of the high-stakes / cross-cutting / burst patterns (bw5 migration `186d503`, telegraf reactor refactor `4959461`, wol-waker scope-down `985a15e`, l4d burst seed `3469d98`, mailman introduction `d3b8e2e`, debian-13 `c41e6f8`, etc.) to verify intent matches message.
Clustering rule: a story is a recurring **user intent**, not a recurring **file pattern**. Two commits both touching `metadata.py` may be different stories. Aimed for stories with ≥3 grounding commits each. Final count: 19 stories.
The prior pass produced 12 stories. This pass adds 7 (investigation, trial-and-error microbursts, disable-for-now, cleanup-of-dead-code, subsystem deep-dive bursts, README touch-ups, naming-convention renames) and subdivides several of the original twelve.
## Story index
Ordered by frequency, recent-window unless noted.
1. **S1 — Tune one bundle's metadata** (~50+ commits). Small targeted `metadata.py` edits.
2. **S2 — Tweak one node's config** (~80+ commits, top-heavy). Per-node tuning, often serial.
3. **S3 — Edit a file template** (~40+ commits). `bundles/<x>/files/*` changes.
4. **S4 — Trial-and-error microburst** (frequent within bursts; ~15 burst-clusters). Successive "fix"/"+"/"wip" commits.
5. **S5 — Subsystem deep-dive burst** (5 major bursts in 18 months: l4d2, routeros, routeros-monitoring, bootshorn, debian-13).
6. **S6 — Modify items.py for a bundle** (~30 commits). Item logic / dep tweaks.
7. **S7 — Templated data update** (~15 commits). `data/apt/keys/*`, `data/grafana/rows/*`.
8. **S8 — Add a new bundle** (10 in 18 months: yourls, mailman, routeros, routeros-monitoring, pppoe, bootshorn, kea-dhcpd, proxmox-ve, ifupdown, network).
9. **S9 — Onboard a new node** (10 nodes added in 18 months).
10. **S10 — Group adjustment / new OS variant** (~12 commits). `groups/os/*`, `groups/locations/*`.
11. **S11 — Disable temporarily / "for now"** (~10 commits). Commenting out, dummy-ifying.
12. **S12 — Cross-bundle metadata-reactor refactor** (4 high-impact commits in 18 months).
13. **S13 — Cleanup / delete obsolete code** (~8 commits). `_old` bundles, removed nodes.
14. **S14 — Add or revise an operator script** (`bin/`) (~8 commits, 4 new scripts in 18 months).
15. **S15 — OS-version upgrade campaign** (3 campaigns: debian-12, debian-13, trixie/server-only).
16. **S16 — Add/modify a lib** (`libs/`) (5 commits in 18 months; 1 new lib).
17. **S17 — Add/modify a hook** (`hooks/`) (4 commits in 18 months; 2 new hooks).
18. **S18 — Per-bundle README touch-up** (~8 commits, sporadic).
19. **S19 — Naming-convention rename** (4 renames in 18 months).
20. **S20 — Bundlewrap version migration** (1 commit, but uniquely high-stakes — counted because it shapes the maintainer's roadmap).
21. **S21 — Interactive `bw debug` investigation** (.bw_debug_history evidence).
(Numbering shows 21 because three are kept short — S20 is a single high-stakes commit, S17 / S21 are low-frequency but materially distinct workflows. Anything that did not reach ≥3 grounding events was rolled into cross-cutting findings.)
## Stories
### Story 1: Tune one bundle's metadata
**Pattern.** Open `bundles/<x>/metadata.py`, change a value or add a key, commit. The most common single-file edit. Often very small (`fix typo`, `relax telegraf collection`, `mailserver zfs params`). The change is *intra-bundle* — no reactor surprise; just adjusting a default the bundle itself reads.
**Frequency.** ~50 commits over 18 months across many bundles. Recent: `7f20c94` (telegraf deprecations, 2026-03-09), `a7c7aaf` (nc preview options, 2026-03-09), `5fab21b` (apt install ca-certificates, 2026-02-10), `0d35bc2` (linux relax icmp ratelimit, 2026-02-10), `5fb1ee5` (less annoying root passwords / users, 2025-08-09).
**Files typically touched.** Single file: `bundles/<x>/metadata.py`. Sometimes one paired sibling file when the new metadata has an items consequence (see Story 6 for pure items work).
**Variations.**
- (a) **Tune an existing value.** Change a number, add a member to a set, swap a string. ~70% of cases.
- (b) **Add a new key.** `'unattended-upgrades': {}` added to the apt packages dict in `186d503`; `apt install ca-certificates` in `5fab21b`.
- (c) **Restructure a sub-tree for new behavior.** Less common; tends to bleed into Story 12 if it changes a reactor's contract.
**Evidence.**
- `7f20c94` (2026-03-09) — telegraf deprecations (2 files: bundles/routeros-monitoring/metadata.py + bundles/telegraf/items.py).
- `0d35bc2` (2026-02-10) — `linux relax icmp ratelimit` (1 file).
- `5fab21b` (2026-02-10) — `apt install ca-certificates` (1 file).
- `a0f5f80` (2026-01-11) — typo fix in routeros-monitoring metadata.
- `bdb9fa0` (2025-06-22) — `gitea disable registration` (changes `bundles/gitea/files/app.ini` template — boundary case with Story 3).
**Co-modification.** Usually none — single file. Occasionally pulls in `items.py` if the new key changes which file/service item is rendered. Almost never touches nodes (the value sits in defaults, nodes don't need to opt in unless a top-level key didn't exist before).
**Stakes.** Routine when the bundle's metadata is read only by itself. Moderate when the bundle uses cross-bundle reactor writes (e.g. telegraf, monitored). The maintainer's mental model assumes "metadata.py change → only the bundles using this bundle are affected"; that's only true if the bundle keeps to its own namespace. See Story 12 for the cross-bundle case.
**Implications for agent docs.** Well-served by `bundles/AGENTS.md` plus per-bundle `AGENTS.md` (PR2). Agents need to know:
- The bundle's own resolved metadata shape — covered by per-bundle template's `Metadata` section.
- Whether *this* bundle writes into other namespaces (cross-bundle ripple) — must be in the per-bundle `Gotchas` for any bundle that does (telegraf, monitored, wol-waker, archive). `commands.md`'s after-change row already covers it generally, but a per-bundle callout shortens the agent's loop.
- After-change verification: `bw hash <node>` — the table in `commands.md` covers this. **No gap.**
### Story 2: Tweak one node's config
**Pattern.** Edit a single `nodes/<name>.py` to change something node-specific: an interface, a hostname, a group membership, a metadata override. The most common workflow, but heavily skewed to a few "hot" nodes — `home.router.py` (25 touches/18mo), `ovh.secondary.py` (29), `home.server.py` (16). Most edits are tiny.
**Frequency.** ~80 node touches in 18 months across 21 nodes. 70% concentrated in 5 nodes.
**Files typically touched.** Single node file. Occasionally two when the change is "move responsibility from node A to node B" (e.g. `5a8dc7e` 2025-12-03 moves wakeonlan to its own vlan, edits home.backups.py + home.router.py + home.switch-rack-poe.py + groups/os/routeros.py).
**Variations.**
- (a) **Per-application config tweak** (e.g. ovh.secondary l4d server config — see Story 5).
- (b) **Hardware/network shape change** (home.server "more swap"; "fan/motherboard sensors"; home.router "fix dhcp"; "fix ip").
- (c) **Group membership change** — adding/removing strings in the node's `groups` list. Cross-cuts with Story 10.
- (d) **Bundle attachment** — adding a bundle to the node's `bundles` list (rare; usually goes via group instead).
**Evidence.**
- `838da64` (2026-03-09) — home server fan/motherboard sensors (1 file).
- `fcd92db` (2026-03-09) — more swap (1 file, home.server).
- `4652a42` (2026-01-10) — disable zfs mirror for now (home.backups.py — also a Story 11 example).
- `60f29aa` (2025-08-24) — fix hue dhcp (home.hue).
- `0e78afe` (2024-11-23) — fix ip (home.router).
**Co-modification.** Often none. When the change is a group-membership shift, the receiving group's file (`groups/os/routeros.py`, `groups/locations/home.py`) gets a paired edit. Network-shape edits frequently pull in 2-3 nodes plus `bundles/network/metadata.py`.
**Stakes.** Routine for value tweaks; moderate when the node provides services others depend on (home.router, htz.mails). High if node-file syntax is broken — recall that `nodes/<x>.py` is `eval()`'d, so a syntax error or top-level `import` breaks the loader for all nodes (commit `dc40295` in 2025-06-22 added an explicit `print message on parsing group error` to `groups.py`, suggesting the user has been bitten by silent loader failures).
**Implications for agent docs.** `nodes/AGENTS.md` should call out:
- The `eval()` constraint (no top-level imports, only expression-level constructs) — already in spec §5.
- Where to find resolved metadata after the edit (`bw metadata <node>`) — covered by `commands.md` row "nodes/<x>.py".
- Naming convention `<location>.<role>.py` and how it's tied to the loader — covered.
- **Gap:** the `eval()` failure mode is silent; the user himself patched the loader to print a message. Worth an explicit *pitfall* in `nodes/AGENTS.md`: "if `bw nodes` reports nothing or fewer nodes than expected, you have an `eval()` error in a node file." Otherwise an agent will assume the file was loaded.
### Story 3: Edit a file template
**Pattern.** Open `bundles/<x>/files/<template>` (a Mako/plain-text/script file rendered by `file` items), change content, commit. Often more frequent than `metadata.py` for service-config-heavy bundles.
**Frequency.** ~40+ commits over 18 months. Top hits: `bundles/left4dead2/files/setup` (11 touches), `bundles/left4dead2/files/start` (6), `bundles/left4dead2/files/server.cfg` (4), `bundles/left4dead2/files/scripts/overlays/tickrate` (4), `bundles/bootshorn/files/temperature` (4).
**Files typically touched.** Single template file most of the time. Sometimes paired with `items.py` if the template's render context (`content_from`, conditionals) needs an `items.py` edit too.
**Variations.**
- (a) **Trivial edit** (one-line tweak in `nginx.conf`, a setting in `roundcube/files/config.inc.php`).
- (b) **Mako template logic change** (e.g. `data/grafana/flux.mako` — Mako-based dashboards).
- (c) **Whole-file rewrite** (less common — usually a new feature).
**Evidence.**
- `a629024` (2026-01-11) — `bundles/roundcube/files/config.inc.php`: smtp use domain name from cert.
- `bf38520` (2026-03-07) — comment out slow download workshop maps in l4d overlay file.
- `64f8691` (2025-08-09) — bind named.conf.local: zones.rfc1918 only affect recursive views.
- `e117aca` (2025-01-09) — `bundles/backup/files/backup_all`: don't stop on first error.
**Co-modification.** Often none. Pairs with items.py when the file's render context changes.
**Stakes.** Routine. Side effects don't ripple — the file is rendered once on the destination node. Templates with embedded scripts (e.g. `bundles/backup/files/backup_all` is a shell script) are a hidden risk: a bug there runs on production.
**Implications for agent docs.** `bundles/AGENTS.md` should mention `files/` at all (currently the spec template lists "files/, templates/" in the bundle anatomy section but doesn't say *which* files are rendered as templates vs static, or how to recognize Mako vs plain). Need:
- One paragraph: how the `file` item picks template vs static (default extension behavior, `content_from`, `content_type`).
- A pointer to the **fork's** `docs/content/guide/item_file_templates.md` — already noted in the plan's "Workflow validation findings" §5. **Confirmed needed.**
- For service-config-heavy bundles (left4dead2, bind, postgresql), per-bundle `Gotchas` should call out non-trivial template logic. Currently only nextcloud, mailman, freescout have ad-hoc READMEs — most are bare.
### Story 4: Trial-and-error microburst
**Pattern.** A series of 3-15 short commits in quick succession (often the same hour) that walk a single change toward working. Subjects like `fix`, `+`, `whitespace`, `dowsnt exist`, `fix indent`, `fix set not dict`. Usually one bundle or one node, sometimes a single file modified 5-10 times in a day. This is a *workflow style*, not a single change.
**Frequency.** ~15 burst-clusters in 18 months. Recurring, not anomalous. Examples: nftables fixes (2025-07-11: `e7c5fe9`, `5a1ce55`, `8829902`), bootshorn temperature tweaks (2025-07-13: `c1917d5`, `6f86abd`, `951fa63`), l4d2 setup script (10+ commits within hours on 2025-10-29), routeros-monitoring (consecutive small edits across 2025-12-13 / 12-16 / 12-30).
**Files typically touched.** One file, repeatedly. Sometimes two paired files (items.py + metadata.py).
**Variations.**
- (a) **Compile-fix burst.** "fix indent" / "fix set not dict" — Python typo / structural mistake immediately corrected.
- (b) **Behavior-tuning burst.** Successive value changes ("more swap", "more workshop maps", "tickrate", "nc preview").
- (c) **Apply-then-correct.** A change that broke something on the next `bw apply`, fixed in the next commit.
**Evidence.**
- `e7c5fe9``5a1ce55``8829902` (2025-07-11, all within the same day, all `bundles/nftables/`).
- `c1917d5``75017a9``6f86abd``951fa63` (2025-07-13, all `bundles/bootshorn/`).
- `19c1945``a9e4013``8391afd``d91b205``3311bfb``351ce24``9572ac8` (2025-10-29, all l4d2).
**Co-modification.** None — repeats on the same files.
**Stakes.** Low individually, but reveals important workflow context: **commits are not landing-ready snapshots — they are the user's iterative checkpoints.** This shapes everything: an agent that reads recent history to learn "how does the user write code" will see noise unless it understands this.
**Implications for agent docs.** Not a per-feature docs gap — but **highly relevant context for every agent**. Belongs in root `AGENTS.md` quickstart or `conventions.md`:
- "Commit messages are terse and frequently iterative; expect successive commits on the same file. Don't rebase the user's WIP commits without asking." (Style guidance for an agent commit-creating workflow.)
- Verification chain: agents should run `bw test` / `bw hash` *before* committing rather than relying on later "fix" commits — an agent's commits are more durable than the user's, since they tend to land in batches.
- **Currently neither covered.** ⚠ Partial gap — adjacent to spec §4 quickstart but not explicit.
### Story 5: Subsystem deep-dive burst
**Pattern.** Periods of 1-6 weeks where 80%+ of commits target one bundle/subsystem. This is Story 4 scaled up — instead of one day of microcommits, it's a multi-week project. The user is *learning the subsystem in production*: introducing it, finding issues, refining, often with cross-cutting touches (groups, nodes, libs) that wouldn't appear in a single-bundle story.
**Frequency.** 5 major bursts in 18 months, plus 2-3 minor ones.
| Burst | Window | Commits | Subsystem |
|---|---|---|---|
| **left4dead2 server rebuild** | 2025-08 → 2026-02 | ~50 | `bundles/left4dead2/`, `nodes/ovh.secondary.py` |
| **routeros / mikrotik switches** | 2025-06-29 → 2026-01-11 | ~30 | `bundles/routeros/`, `bundles/routeros-monitoring/`, `groups/os/routeros.py`, 4 switch nodes |
| **debian-13 upgrade** | 2025-08-09 → 2026-03-07 | ~15 | `bundles/dovecot/`, `bundles/redis/`, `bundles/bind/`, `groups/os/debian-13*`, multiple nodes |
| **bootshorn (laptop temperature monitoring)** | 2025-07-11 → 2025-08-24 | ~15 | new bundle from scratch + `home.bootshorn-laptop` node |
| **pppoe (telekom uplink)** | 2025-07-11 → 2025-08-03 | ~12 | `bundles/pppoe/`, `bundles/network/`, `bundles/nftables/`, home.router |
**Files typically touched.** Mixed: cluster around one bundle, with pulls into `groups/os/<x>.py`, the relevant node, sometimes a new lib (e.g. `libs/version.py` introduced in the routeros burst at `86d9b8b`).
**Variations.**
- (a) **Bring-up of a new capability** (routeros switches, pppoe, mailman, bootshorn) — touches multiple areas as scaffolding lands.
- (b) **Refactor of an existing capability** (l4d2 rebuild from August onward — note the `_old` bundles created and later deleted; the `3469d98` "next l4d2 server iteration" commit is the burst seed).
- (c) **Migration burst** (debian-13) — see Story 15.
**Evidence (one per burst seed and one mid-burst):**
- l4d2: seed `3469d98` (2025-08-24, "the next l4d2 server iteration"), mid-burst `9572ac8` (2025-10-29, "l4d2 dynamic overlays"), wind-down `ac8e7e2` (2026-02-10, "delete old l4d bundles").
- routeros: seed `85daf26` (2025-06-29, initial bundle + switches), mid `0e6a705` (2025-07-01, "routeros switches ok"), monitoring expansion `8539f59` (2025-12-13, "mikrotik snmp monitoring").
- bootshorn: seed `5274639` (2025-07-11, "bootshorn recording" — full bundle skeleton in one commit), tuning `75017a9``951fa63` (Story 4 microburst).
**Co-modification.** Heavy. Burst commits routinely touch a bundle + a group + 1-2 nodes + (sometimes) a lib + (sometimes) a hook.
**Stakes.** Cumulatively high. Each burst introduces or substantially reshapes a subsystem. Mid-burst, the subsystem is in flux — items.py and metadata.py contracts shift, file templates churn, the docs (if any) are stale. **An agent landing in a hot burst should be the most cautious, not the least.**
**Implications for agent docs.**
- The planned per-area docs and per-bundle docs cover the **steady state** but not the **burst state**. A burst-aware agent would need:
- A way to know "this subsystem is in active flux" — derivable from `git log --since="1 month ago" --pretty=format:'' --name-only | grep <area> | wc -l`. Could be a one-liner in `bundles/AGENTS.md`'s "Pitfalls": "before writing into a subsystem, check `git log --since='1 month ago' bundles/<x>` — if it's hot, your assumptions about the metadata shape may already be stale."
- The seed bundle list (Phase 2 in spec §7) **misses three of the five recent burst targets**: routeros (15 commits), routeros-monitoring (15), bootshorn (15), pppoe (12). Of those, only `wireguard` (8 refs) is on the seed list. The spec acknowledges routeros-monitoring as "honourable mention." **⚠ Recommend rebalancing seed list toward currently-hot subsystems**, since those are where doc absence costs the most agent time.
- **Gap.** The subsystem-burst pattern itself isn't documented anywhere planned. Could be one paragraph in `conventions.md` about "how this repo evolves" — it would let an agent recognize and respond to flux.
### Story 6: Modify items.py for a bundle
**Pattern.** Open `bundles/<x>/items.py`, add/remove/rewire a `file:`, `pkg_apt:`, `svc_systemd:`, `action:`, etc. Often paired with metadata.py if the new item reads from a key that didn't exist. The change is *what gets installed/managed*, not *with what value*.
**Frequency.** ~30 commits over 18 months that primarily target items.py.
**Files typically touched.** Single `items.py`. Frequently paired (a) `bundles/<x>/files/<new>` if a new file item is added, (b) `bundles/<x>/metadata.py` when the items.py reads a new metadata key.
**Variations.**
- (a) **Wire deps right**`needs`/`needed_by`/`triggers` adjustments. Lots of these (`9733a55` 2025-06-22 "svc_systemd:systemd-networkd add .service to name"; `c244645` "kea deps"; `663116c` "/var/lib/mysql needs mysql user to exist"; `befdf5a` "fixmo mariadb dependency"; `fb22a01` "systemd fix dependency overwrite").
- (b) **Add a new managed file/service** — usually appears alongside an `items.py` skeleton in a bundle creation (Story 8).
- (c) **Fix permissions** (`6616ae7`, `43e7c1f` redis permissions; `e17b023` grafana permission).
**Evidence.**
- `9733a55` (2025-06-22) — `svc_systemd:systemd-networkd add .service to name`.
- `47b69f0` (2025-11-04) — `l4d items stop script` (adds a file item).
- `c244645` (2024-11-23) — `kea deps`.
- `b838935` (2025-08-09) — `dont purge sudoers`.
- `8066efb` (2025-12-03) — `routeros: also add comment to interface` (logical edit in items.py).
**Co-modification.** Frequently with the bundle's own `metadata.py`, occasionally with a node when an items.py change broke one specific node's resolved state.
**Stakes.** Routine for value/tag/dep tweaks. Moderate when restructuring an item's identity (renaming a file path, changing `name=` of an `svc_systemd:`) — a rename without redirect is **two state changes** (delete old, create new) which can cause a service flap.
**Implications for agent docs.** Covered by `bundles/AGENTS.md` (anatomy) and the fork's `AGENTS.md`/items reference (item-type semantics). The repo-specific gap:
- The custom `download:` item type from `items/download.py` is mentioned in the plan but its semantics aren't documented in any planned area doc beyond `items/AGENTS.md`. The bw5 migration commit `186d503` touched `items/download.py` — agents adding a `download:` item need to know the actual interface (`expected_state`/`actual_state`/`get_auto_attrs`) post-bw5. **⚠ Need explicit per-item-type docstring on `items/download.py`** (the docstring/header pass in PR1 step 7 should cover this; verify it does).
- Dependency-keyword vocabulary (`needs`, `triggers`, `triggered`, `cascade_skip`, etc.) lives in the fork's `AGENTS.md` per the plan. **No additional ckn-bw doc gap.**
### Story 7: Templated data update (apt keys, dashboards)
**Pattern.** Two distinct sub-flavors that use the same `data/` directory mechanically:
- (a) **Apt key refresh** — replace a `.gpg` / `.asc` file with a freshly downloaded key. ~5 commits in 18 months. Mechanical but high stakes (a wrong key breaks `apt update` on every node using that source).
- (b) **Grafana dashboard edits**`data/grafana/rows/*.py` and `data/grafana/flux.mako`. ~10 commits. Active design work, often tied to a bursting subsystem (e.g. routeros monitoring).
**Frequency.** 15+ data/ touches in 18 months.
**Files typically touched.**
- (a) Apt: `data/apt/keys/<src>.{gpg,asc}` — single file, sometimes 2-3.
- (b) Grafana: `data/grafana/rows/<row>.py` + sometimes `data/grafana/flux.mako`.
**Variations.**
- (a) Apt key replace (one-shot).
- (b) Apt key migration (e.g. `f0d1cf9` "new icinga apt key" added a `.gpg` and renamed the old `.asc``.asc_`, evidence the user keeps stale-but-not-removed keys around).
- (c) Grafana dashboard tweak (cosmetic).
- (d) Grafana new row (new monitoring source).
**Evidence.**
- `1907c38` (2026-01-07) — `data/apt/keys/influxdata.asc: update`.
- `487fdff` (2025-12-01) — update some apt keys (crystal, grafana).
- `f0d1cf9` (2024-11-23) — new icinga apt key.
- `da29405` (2026-01-11) — `data/grafana/rows/routeros_*: update names`.
- `4a4167e` (2025-12-13) — routeros grafana discards and errors (new rows).
**Co-modification.** Apt: usually only `data/apt/keys/`. Sometimes paired with `bundles/apt/metadata.py` if the source URL/format also changed. Grafana: usually paired with `bundles/grafana/items.py` or `bundles/grafana/metadata.py`.
**Stakes.** Apt: **high** (broken key → blocked unattended upgrades cluster-wide). Grafana: low (dashboard reads only).
**Implications for agent docs.** `data/AGENTS.md` (planned) needs to:
- Distinguish the apt-keys-as-data-source pattern from the grafana-rows-as-Python-dashboards pattern — they look the same at the directory level but have totally different content models.
- Call out the verification step for apt keys: trial via `bw apply` is *not* safe (it's the failure path). Better: gpg-verify the key locally before committing. **⚠ Likely gap** — `commands.md`'s after-change table doesn't have a row for `data/apt/keys/*` (it falls into "Anything"). One-line addition warranted.
- Grafana rows: covered by general-template guidance.
### Story 8: Add a new bundle
**Pattern.** Create `bundles/<name>/` from scratch — at minimum `items.py` + `metadata.py`, often `files/<file>`. Can be a single commit landing the skeleton or the seed of a deep-dive burst (Story 5).
**Frequency.** 10 new bundles in 18 months: yourls (2025-06-22), mailman (2025-06-29), routeros (2025-06-29), routeros-monitoring (2025-12-13), pppoe (2025-07-11), bootshorn (2025-07-11), kea-dhcpd (2024-09-18 inside `TOTAL FACKUP`), proxmox-ve (2025-06-22), ifupdown (2025-06-22), network/items.py (2025-07-10).
**Files typically touched.**
- Always: `bundles/<name>/items.py`, `bundles/<name>/metadata.py`.
- Often: 1-3 `bundles/<name>/files/<file>`.
- Often: a node file (`nodes/<x>.py`) — to attach the bundle.
- Sometimes: a group file (`groups/applications/<x>.py` or `groups/os/<x>.py`) when wiring as application/OS-level.
- Occasionally: `data/<name>/` for bundle-templated data.
- Sometimes: a README.md (mailman, pppoe, l4d2 — not consistent).
**Variations.**
- (a) **One-commit skeleton** (most common). E.g. `5274639` bootshorn brought the whole bundle in 5 files at once.
- (b) **Seed + burst.** Skeleton lands, then several follow-up commits refine (Story 4 + Story 5).
- (c) **Inside a megacommit.** `67d5a4b` "TOTAL FACKUP" added kea-dhcpd, linux items.py + metadata.py among 20+ other changes. Anti-pattern, but it happens.
**Evidence.**
- `5274639` (2025-07-11) — bootshorn introduction.
- `aeb0a4f` (2025-06-22) — `nodes/mseibert.yourls.py: introduce` (5 files: yourls bundle + node + data/yourls/).
- `d3b8e2e` (2025-06-29) — mailman (10 files added).
- `e4e3c57` (2025-07-11) — pppoe telekom (full bundle).
- `8539f59` (2025-12-13) — mikrotik snmp monitoring (routeros-monitoring bundle).
**Co-modification.** Always cross-area: bundle + node, often + group. This is the canonical "wire it in" workflow.
**Stakes.** Moderate to high. A new bundle wired wrong (group memberships off, missing dep, reactor that overlaps a sibling's namespace) can break unrelated nodes' apply.
**Implications for agent docs.**
- Spec §3 per-bundle template is good shape — covers Metadata, Produces, Depends on, Gotchas.
- Plan workflow validation finding §3 already calls out the explicit-wiring step. ✅
- **Open question.** The seed bundle list for Phase 2 (`monitored, postgresql, wireguard, php, apt, nginx, telegraf, backup, letsencrypt, nextcloud`) was chosen for usage hubs + recent activity. Of the 10 *recently introduced* bundles in this story, only mailman and bootshorn would be obvious targets, and neither is on the seed list. **Recommendation (don't act, just surface):** when an agent uses `bundles/AGENTS.template.md` to write a new bundle's docs from scratch in Phase 3, the template's example shapes should be derived from a "freshly-introduced bundle that has no consumers yet" — the plan workflow finding §7 already raises this.
- Pattern-grounded: the template should be writable end-to-end given just the skeleton commit's contents (items.py + metadata.py + files/*); verify by tracing one (`5274639` bootshorn) through the template fields. If any field is unfillable from the artifacts, that's an alignment gap. **Suggest verification step** during PR2.
### Story 9: Onboard a new node
**Pattern.** Create `nodes/<location>.<role>.py`, populate metadata + groups, often touch group files to wire membership. Sometimes accompanies adding a new bundle (Story 8) — but more often the bundles already exist and this is "deploy bundles X, Y, Z to a new machine."
**Frequency.** 10 new nodes in 18 months: home.bootshorn-laptop, home.rack-switch-10g (later renamed), home.switch-rack-poe, home.switch-vorratsraum-poe, home.switch-wohnzimmer-10g, home.wohnzimmer-switch-10g (later renamed/deleted), htz.l4d2.py_ (intentionally non-loaded — see naming), mseibert.mailman, mseibert.yourls, ovh.left4me.
**Files typically touched.**
- Always: `nodes/<x>.py` (new file).
- Often: 1-3 `groups/*.py` (membership lists; or `groups/os/<os>.py` to add the node).
- Sometimes: a new bundle (Story 8 overlap) or a new ssh known-hosts entry (`metadata['known_hosts']`).
**Variations.**
- (a) **New machine, existing bundles.** Fastest. Just node + group entries. E.g. `home.switch-rack-10g`-class introductions paired with `routeros` group additions.
- (b) **New machine, new bundle.** Story 8+9 combined. mseibert.yourls (`aeb0a4f`).
- (c) **Suspended-state node** (e.g. `nodes/htz.l4d2.py_` — appended underscore so the loader doesn't pick it up). Notable convention: loader file-pattern based.
**Evidence.**
- `e99fd4b` (2026-05-10) — add ovh.left4me (also updates 2 existing nodes; 39 lines in the new node file).
- `aeb0a4f` (2025-06-22) — mseibert.yourls (yourls bundle + vhost + node introduced together).
- `5274639` (2025-07-11) — bootshorn-laptop introduction.
- `9a51943` (2025-07-10) — switch-rack-poe.
- `58007f5` (2026-03-07) — `dowsnt exist` adds `nodes/htz.l4d2.py_` (suspend convention).
**Co-modification.** Always cross-area: node + (group or bundle).
**Stakes.** Routine for adding to existing capacity. Moderate when the new node introduces a new role (mailman host, l4d server). High when the node has a typo in its `groups` list — typo silently fails to inherit, which the user explicitly added an error printer for (`dc40295`).
**Implications for agent docs.**
- `nodes/AGENTS.md` covers the eval() mechanism and naming convention.
- **Gap A:** the *suspend* convention (`*.py_` doesn't load) is a real, recurring practice — see `nodes/htz.l4d2.py_`. The spec/plan don't mention it. One line in `nodes/AGENTS.md` "Conventions" section warranted.
- **Gap B:** "first-time node onboarding" sequence isn't a workflow that has docs anywhere. Possibly belongs in `bundles/AGENTS.md` "How to add" cross-linked to `nodes/AGENTS.md`. The plan's workflow validation already added a wiring step for *adding a bundle*; a symmetric "How to add a node" in `nodes/AGENTS.md` would be parallel.
- Verification: `bw nodes`, `bw groups -n <node>`, `bw metadata <node>` — already in `commands.md`. ✅
### Story 10: Group adjustment / new OS variant
**Pattern.** Edit `groups/<area>/<x>.py` (or add a new file there). The "membership" function — which nodes get which bundles, and what default metadata flows. A small but high-leverage area.
**Frequency.** ~12 commits in 18 months. Concentrated in `groups/os/routeros.py` (10 touches — driven by the routeros burst) and the debian-13 family (`groups/os/debian-13-common.py`, `debian-13.py`, `debian-13-pve.py` all introduced 2025-08-09 onwards).
**Files typically touched.**
- Most often a single `groups/os/<os>.py`.
- For OS-version upgrades: 1-2 new files in `groups/os/`, parallel to existing version files.
- Membership edits: occasionally also pull in a node file (paired Story 9).
**Variations.**
- (a) **New OS variant.** debian-13 introductions (`c41e6f8` added `debian-13-common.py`, `debian-13.py` and deleted `debian-11.py`). proxmox-ve added `debian-12-pve.py`.
- (b) **Membership tweak.** `b81c9e9` "allow snmp at home" toggles `groups/locations/home.py`.
- (c) **Per-OS metadata adjustment.** `groups/os/routeros.py` got 10 touches as the routeros bundle matured.
**Evidence.**
- `c41e6f8` (2025-08-09) — debian 13 (introduces `groups/os/debian-13.py`, `debian-13-common.py`; deletes `debian-11.py`).
- `b81c9e9` (2025-12-16) — allow snmp at home (groups/locations/home.py).
- `8d941eb` (2025-06-28) — open fw for iperf (groups/os/debian.py).
- `463cf87` (2025-12-03) — mikrotik more port config (groups/os/routeros.py + 4 switch nodes).
**Co-modification.** Group edits often pull in 2-5 nodes (when adding/removing a group, the affected nodes' files are also adjusted). For new OS variants, add a node group file + bump nodes' `groups` lists + sometimes add a new apt key.
**Stakes.** **High.** A group's membership controls which bundles a node ships with, and metadata flows down via merge order (`all` → location → os → machine → applications → node). Wrong group membership = wrong bundles installed = real-machine consequences.
**Implications for agent docs.**
- `groups/AGENTS.md` covers eval() loading, the subdir convention (`applications/`, `locations/`, `machine/`, `os/`), and `all.py`. ✅
- Inheritance/merge order is in `conventions.md` (planned). ✅
- **Gap A:** the "create a new OS variant" recipe isn't documented. Concrete steps the user followed in `c41e6f8`: (1) add `groups/os/debian-13.py` + `debian-13-common.py`; (2) add `data/apt/keys/debian-13-*.asc`; (3) ensure dependent bundles handle the new os string (e.g. `bundles/bind/items.py` was bumped). Worth a paragraph in `groups/AGENTS.md`.
- **Gap B:** the *family-file pattern* (`debian-13-common.py` shared by `debian-13.py` and `debian-13-pve.py`) — a non-trivial idiom. Not in the spec/plan. One line in `groups/AGENTS.md` Conventions.
### Story 11: Disable temporarily / "for now"
**Pattern.** Comment out or stub out a block of working config to take it offline temporarily — service is broken upstream, host is offline, user is debugging, etc. Marker phrases in the commit message: "for now", "disable", "comment out", "dummy". The changes are not deletions; they're meant to be reversed.
**Frequency.** ~10 commits in 18 months. Recurring enough to be a recognized workflow.
**Files typically touched.** Single file. Usually a node file or a metadata.py.
**Variations.**
- (a) **Disable a node entirely** (e.g. `35243fd` 2025-06-22 "offsitebackup offline"; `4f990f8` 2024-01-08 "stromzaehler is offline for now").
- (b) **Comment out a metadata block** (`bf38520` 2026-03-07 "comment out slow download workshop maps"; `16313b9` 2025-01-09 "disable tasnomta charge"; `4652a42` 2026-01-10 "disable zfs mirror for now").
- (c) **Dummy the role out** (`19a8d28` 2025-07-08 "homeassistant os is dummy"; `c03b033` 2026-05-10 "macbook dummy" — yes, the most recent commit).
**Evidence.**
- `4652a42` (2026-01-10) — disable zfs mirror for now.
- `bf38520` (2026-03-07) — comment out slow download workshop maps.
- `c03b033` (2026-05-10) — macbook dummy (most recent commit, `nodes/macbook.py`).
- `35243fd` (2025-06-22) — offsitebackup offline.
- `4f990f8` (2024-01-08) — stromzaehler is offline for now.
**Co-modification.** None.
**Stakes.** Routine, but **easy for an agent to misread**. Disabled-but-committed code looks broken at first glance. An agent that "fixes" it will undo a deliberate suspension.
**Implications for agent docs.**
- **⚠ Real gap.** Nothing in the planned docs warns an agent about the "for now" idiom. An agent doing a "tidy-up" pass could re-enable a deliberately-disabled node.
- Belongs in `conventions.md` (planned) under a "Patterns to recognize and not break" subsection. One line: "Commits with 'for now' / 'disable' / 'dummy' / 'offline' in the message indicate a deliberate suspension. If you encounter a stub or commented-out block, check `git log -- <file>` for that pattern before re-enabling. The user reverses these manually when ready."
- Verification: `bw nodes` is silent on whether a node *should* be picking up a bundle — there's no signal in tooling. Pure-doc fix.
### Story 12: Cross-bundle metadata-reactor refactor
**Pattern.** A change in *one* bundle's `metadata.py` (its `defaults` dict or a `@metadata_reactor`) that writes into other bundles' namespaces, materially altering many bundles' resolved state at once. Rare but expensive.
**Frequency.** 4 high-impact instances in 18 months.
**Files typically touched.** One bundle's `metadata.py` (the source of the change), plus 5-15 other bundles' `metadata.py` (the consumers being updated to match the new contract).
**Evidence.**
- `4959461` (2026-01-11) — `bundles/telegraf/items.py: use new bundle from isac`. Restructures the entire telegraf metadata namespace; touches **12 bundles** (apcupsd, bind, hardware, postfix, postgresql, raspberry-pi, routeros-monitoring, smartctl, tasmota-charge, telegraf, zfs). 660 lines changed in routeros-monitoring/metadata.py alone.
- `985a15e` (2026-01-11) — `wol waker only allow wakeonlan command`. Touches 8 bundles' metadata + 5 nodes via the wol-waker reactor's contract.
- `186d503` (2026-05-10) — bundlewrap 5 migration. **Cross-cutting** rewrite of "non-reading and KeyError-driven metadata reactors per the bw 4→5 migration guide." 11 bundles + items/download.py.
- `7eac09e` (2025-08-09) — `ovh.secondary cake`. Cross-bundle: linux + network + pppoe metadata + node.
**Co-modification.** Always wide. The change pattern fans out from one bundle to many.
**Stakes.** **Highest blast radius in the repo.** Reactors that write across namespaces are the spec's documented warning case (commands.md side-effect-model paragraph). When such a reactor's contract changes, every consumer must update in lockstep.
**Implications for agent docs.**
- `commands.md` already calls out that `bundles/<x>/metadata.py` changes can ripple — first row of the after-change table says "for all nodes including bundle x" and notes "reactors can ripple beyond x's namespace." ✅
- **⚠ Per-bundle gap.** The agent's signal that *this* bundle is one of the cross-writers should live in the per-bundle `Gotchas` or a `Writes into` section. Currently the per-bundle template (spec §3) has `Depends on` (which other bundles this bundle needs) but **not** `Writes into` (which other bundles' namespaces this one writes via reactor). For telegraf, monitored, archive, wol-waker, apt — that's the most important fact.
- **Recommend (don't act):** consider adding an optional `## Writes into` section to the per-bundle template, or fold this into `Gotchas`. Without it, an agent editing `bundles/telegraf/metadata.py` won't know it's about to ripple unless it reads the diff carefully.
- Bundlewrap 5 migration kind of work is one-shot — not worth a separate doc, but worth a sentence in `conventions.md` noting "this repo is on bundlewrap 5.0.3" so an agent doesn't apply 4.x patterns. Plan already covers this. ✅
### Story 13: Cleanup / delete obsolete code
**Pattern.** Pure-deletion or near-pure-deletion commits that remove dead bundles / nodes / templates. Often follows a Story 5 burst (the burst ended; old artifacts now obsolete).
**Frequency.** ~8 commits in 18 months.
**Files typically touched.**
- Bundle deletions: 3-10 files in `bundles/<x>/`.
- Node deletions: 1 node file.
- Single-file template removals: 1 file.
**Variations.**
- (a) **Remove an obsolete bundle entirely.** `849c305` "remove obsolete homeassistant supervised" deletes 3 files. `ac8e7e2` "delete old l4d bundles" deletes 12 files across left4dead2_old, left4dead2_old2, left4dead2_steam_old.
- (b) **Remove an obsolete node.** `e65aa8f` deletes `nodes/home.openhab.py`. `c41e6f8` deletes `nodes/htz.games.py`.
- (c) **Remove a single dead artifact.** `2899cd5` deletes `bundles/nextcloud/files/rescan`. `32ea52c` deletes `bundles/mariadb/files/override.conf`.
**Evidence.**
- `ac8e7e2` (2026-02-10) — delete old l4d bundles (12 files deleted, 0 added).
- `849c305` (2025-07-13) — remove obsolete homeassistant supervised.
- `800bd90` (2025-06-28) — remove apcupsd.
- `e65aa8f` (2025-08-09) — openhab no longer exists.
**Co-modification.** Sometimes nudges nodes that referenced the deleted bundle (e.g. `e65aa8f` also edits home.server.py to remove the openhab include). Pure-deletion if the bundle was already orphaned.
**Stakes.** Low when the asset is truly orphaned. Moderate if a node still references the deleted bundle (silent failure mode: bw will complain at apply time). **The user's workflow is: delete → next bw command surfaces dangling refs → fix.**
**Implications for agent docs.**
- `bundles/AGENTS.md` "How to add" exists; **no symmetric "How to remove"** — and the recurring pattern is real. Worth a 5-line section: "To remove a bundle: (1) `git grep '<name>'` to find references in nodes/groups/other bundles; (2) remove those references; (3) `rm -rf bundles/<x>/`; (4) `bw test` and `bw nodes` to confirm clean."
- Verification command sequence is in `commands.md`. ✅
### Story 14: Add or revise an operator script (`bin/`)
**Pattern.** Either introduce a new `bin/<script>` (one-off operator tooling, not invoked by bundlewrap) or revise an existing one. Standalone — usually does not pair with bundle changes.
**Frequency.** ~8 commits in 18 months. 4 new scripts: `bin/passwords-for`, `bin/sync_1password`, `bin/timestamp_icloud_photos_for_nextcloud`, `bin/mikrotik-firmware-updater`.
**Files typically touched.** Single `bin/<script>` file.
**Variations.**
- (a) **Introduce new script** (commit subject often includes ": introduce").
- (b) **Refactor existing script**`bin/wireguard-client-config` was renamed from `bin/wireguard_client_config` in `1f4aaad` and the same commit improved it.
- (c) **Bug fix.** `dcd2ebc` "dist-upgrade -> full-upgrade" in `bin/upgrade_and_restart_all`.
**Evidence.**
- `60c2c42` (2026-03-09) — `bin/timestamp_icloud_photos_for_nextcloud: introduce`.
- `979c7e1` (2025-12-03) — `bin/passwords-for: introduce`.
- `2b873e4` (2025-12-01) — `bin/sync_1password` (script for syncing routeros logins to 1password).
- `86d9b8b` (2025-12-13) — `bin/mikrotik-firmware-updater` introduced (alongside `libs/version.py`).
- `fe5e340` (2025-12-03) — `bin/script_template: repo -> bw` (the user keeps a template script for new bin scripts).
**Co-modification.** `86d9b8b` paired with `libs/version.py` (script-needs-helper pattern). Otherwise standalone.
**Stakes.** Low. Operator scripts run on demand by the user, not automatically.
**Implications for agent docs.**
- `bin/AGENTS.md` (planned) covers what bin/ is for. ✅
- The existence of `bin/script_template` is itself a useful convention — an agent should start a new bin script from this template. **⚠ Mention it explicitly** in `bin/AGENTS.md` "How to add" — currently not in spec.
- Plan PR1 step 7 adds `# purpose:` headers to bin scripts. The existing scripts use varied conventions — verify the header pass actually achieves uniformity. ✅
### Story 15: OS-version upgrade campaign
**Pattern.** Move nodes from one Debian major to the next. A long-tail campaign (per-node, weeks apart) rather than a single commit. Touches: new `groups/os/debian-N*.py`, new apt keys in `data/apt/keys/`, per-bundle adjustments where the new OS shipped with different packages or service names.
**Frequency.** 3 campaigns in the visible history: debian-12 (Jul 2023, ~10 commits), debian-13 (Aug 2025, ~15 commits), trixie-on-server-only (Mar 2026, 1 commit so far — early).
**Files typically touched (per campaign).**
- New: `groups/os/debian-N*.py`, `data/apt/keys/debian-N-*.{asc,gpg}`, potentially `data/apt/keys/proxmox-ve-N.gpg`.
- Modified: per-bundle metadata to handle new OS string; nodes' `groups` lists bumped.
- Deleted: old OS group file (`groups/os/debian-11.py` deleted in debian-13 campaign; old keys retained as `_.asc` for ref).
**Evidence.**
- `c41e6f8` (2025-08-09) — debian 13 (campaign seed: introduces debian-13 group files, adds apt keys, adjusts bind, removes debian-11 group, removes htz.games node).
- `9621184` (2025-08-10) — htz.mails debian 13 (per-node bump + dovecot/redis/roundcube/systemd-swap adjustments).
- `bc656cd` (2025-08-09) — backups debian 13 (one-line node bump).
- `cb19c38` (2026-03-07) — update home.server to trixie (campaign seed for the next round; touches proxmox-ve metadata, ssh items, debian-13 keys/groups, multiple nodes).
**Co-modification.** Wide: data/ + groups/ + bundles/ + nodes/ in one commit each campaign-step.
**Stakes.** **High.** Each step touches a real machine; failures are recovery work. The user has been bitten — `c41e6f8` deleted `htz.games.py` outright, suggesting that node was decommissioned alongside its OS.
**Implications for agent docs.**
- `conventions.md` (planned) mentions group inheritance order. ✅
- **Gap.** The "campaign" workflow itself isn't documented anywhere. An agent helping with an OS bump won't know whether to (a) start a new `groups/os/debian-N*.py` family or (b) edit the existing one. The user's pattern is consistent: parallel new files, swap node memberships gradually, delete the old when no consumers remain.
- Worth a few lines in `groups/AGENTS.md` "How to add / modify": "When introducing a new OS major, mirror the existing `debian-N-common.py` + `debian-N.py` + `debian-N-pve.py` triad and bump nodes one at a time. Don't edit the old OS file in place — let it die when no nodes reference it."
- Verification: pre-bump `bw hash <node>` vs post-bump diff (planned). ✅
### Story 16: Add or modify a lib
**Pattern.** Edit `libs/<x>.py` (a Python module importable from bundles via `repo.libs.<x>`) or add a new lib. The widest blast radius in the repo — every bundle that imports the lib re-evaluates on next apply.
**Frequency.** 5 commits over 18 months. 1 new lib (`libs/version.py` in `86d9b8b`).
**Files typically touched.**
- Single `libs/<x>.py` for tweaks.
- Sometimes paired: `requirements.txt` if a new dependency is needed (`1d8361c` 2025-06-22 changed `libs/rsa.py` and `requirements.txt` together: `cache_to_disk broken`).
**Evidence.**
- `86d9b8b` (2025-12-13) — `libs/version.py` introduced (mikrotik-firmware-updater needs version compare).
- `1d8361c` (2025-06-22) — `libs/rsa.py` cache_to_disk broken.
- `32ea52c` (2025-06-27) — `libs/ini.py` modified (mariadb refactor uses ini parser).
- `e486aad` (2025-12-13) — `libs/bind.py` whitespace.
**Co-modification.** Rare. Lib changes don't pull in their consumers unless the lib's API changed.
**Stakes.** **Highest blast radius among non-reactor changes.** All bundles that import the lib re-evaluate; all nodes that include those bundles see the effect.
**Implications for agent docs.**
- `libs/AGENTS.md` (planned) covers the convention. ✅
- `commands.md` already says `libs/<x>.py` change → `bw hash` all nodes. ✅
- Plan PR1 step 7 adds module docstrings to libs. ✅ (Critical here — without a one-line docstring, the agent has to read the body to know what the lib does, defeating discovery-by-`ls`.)
- **Gap (minor).** The plan doesn't mention checking which bundles import a given lib — an agent making a breaking change to `libs/hashable.py` should know who imports it. `git grep -l 'repo.libs.hashable'` is a one-line snippet that could live in `libs/AGENTS.md` "Pitfalls."
### Story 17: Add or modify a hook
**Pattern.** Edit `hooks/<x>.py` (lifecycle hook fired by bw on specific events, e.g. before-apply, after-test). Like libs, repo-wide blast radius — a hook fires for *every* bw command of the right kind.
**Frequency.** 4 commits in 18 months. 2 new hooks.
**Files typically touched.** Single hook file. Sometimes paired with the bundle whose data the hook reads.
**Evidence.**
- `0603a8c` (2025-12-01) — `hooks/unique_node_ids.py: introduce` (a `bw test`-time uniqueness check).
- `7ea760d` (2026-01-11) — `hooks/test_ptr_records.py: introduce` (paired with mailserver metadata edit).
- `7f43efc` (2025-12-03) — `hooks/wake_on_lan.py: dedup` (refactor existing hook).
**Co-modification.** Sparse. The new ptr-records test hook also touched `bundles/mailserver/metadata.py` because the hook reads from there.
**Stakes.** **High.** A broken hook breaks every bw command that fires it — an agent can't even run `bw test` to see what's wrong if the hook errors at hook-load time. Recovery requires either fixing the hook or `git stash`.
**Implications for agent docs.**
- `hooks/AGENTS.md` (planned) covers the bw hook lifecycle and how to write one. ✅
- Plan PR1 step 7 adds module docstrings. ✅
- **⚠ Gap.** The "broken hook breaks all bw commands" failure mode isn't documented. An agent introducing a hook should test it in isolation first (e.g. import the module in `bw debug`) before letting it run in `bw test`. Worth one paragraph in `hooks/AGENTS.md` "Pitfalls."
### Story 18: Per-bundle README touch-up
**Pattern.** Edit `bundles/<x>/README.md` — sporadic, prose-only, usually adding a note to a bundle the user wants to remember something about. ~33 bundles currently have READMEs (more than the spec's "~10 of 103" estimate).
**Frequency.** ~8 commits in 18 months. mailman (`9bbaeb6`, `7df2187`, `980fdc8`), freescout (`64029d2`, `8081f12`, `4ec2d51`), l4d2 (`a397399`, `278f6de`), apt (`3dffc05`).
**Files typically touched.** Single README.
**Evidence.**
- `9bbaeb6` (2025-07-12) — mailman poc email sent (creates README and tests).
- `64029d2` (2024-11-23) — freescout readme.
- `a397399` (2026-02-10) — l4d readme.
**Co-modification.** Sometimes paired with code changes in the same bundle.
**Stakes.** Routine.
**Implications for agent docs.**
- **Big alignment issue.** PR2 plan replaces existing per-bundle READMEs with `AGENTS.md` for the 10 seed bundles. The plan correctly notes "verify with `find bundles -name README.md`" — actual count is **33 READMEs**, not the ~10 in the plan's table. The seed list intersects only minimally with the README list:
- On both lists (seed + has README): nextcloud, telegraf, apt, mariadb (note: mariadb isn't on the seed list, but has README), letsencrypt.
- Seed-list bundles **without** existing READMEs: monitored, postgresql, wireguard, php, nginx, backup. (Verified by `find bundles -name README.md`.)
- READMEs on bundles **not** in seed list: ~25, including freescout, mailman, mailserver, mariadb, dm-crypt, dovecot, nodejs, nginx-rtmps, nftables, smartctl, wol-sleeper, wordpress, archive, gcloud, grafana, icingaweb2, influxdb2, raspberrymatic-cert, routeros, systemd, systemd-timers, tasmota-charge, telegraf, zfs, build-server, apcupsd, flask.
- **Recommendation (don't act, just surface):** Phase 3 ("leave-as-you-go") will pick these up over time. **But:** the Phase 1/2 plan should mention that 23 existing READMEs will *remain* (untouched, parallel to AGENTS.md as PR2 only addresses seeds). Eventually those will need folding too. Worth being explicit in the plan that this is a known asymmetry, since `find bundles -name README.md` will report 23 left after PR2 — verifiers will trip on this.
- For agents: `bundles/AGENTS.md` should mention "if a bundle has both a `README.md` and an `AGENTS.md`, the `AGENTS.md` is canonical; the `README.md` is being phased out." Currently planned doc is silent on the transition state.
### Story 19: Naming-convention rename
**Pattern.** Rename a node/bundle/file from underscore-style (or otherwise-irregular) to the canonical lowercase-hyphenated style. Done sporadically as the user notices inconsistency.
**Frequency.** 4 renames in 18 months (excluding internal moves like the mikrotik.mib relocation).
**Evidence.**
- `1f4aaad` (2025-12-16) — `bin/wireguard_client_config``bin/wireguard-client-config` (and improved the script in the same commit).
- `d54eff3` (2025-06-30) — `nodes/home.rack-switch-10g.py``nodes/home.switch-rack-10g.py` (puts "switch" before location-detail per `<location>.<role>.py`).
- `d54eff3``nodes/home.wohnzimmer-switch-10g.py` → renamed in same commit (deleted in subsequent rename to switch-wohnzimmer-10g format).
- `f0d1cf9` (2024-11-23) — `data/apt/keys/icinga.asc``data/apt/keys/icinga_.asc` (here the rename adds an underscore, but it's actually a "park the old file" pattern so the user can replace with a new gpg).
- `78a8abc` (2025-12-16) — `data/routeros-monitoring/files/mikrotik.mib``bundles/routeros-monitoring/files/mikrotik.mib` (move from `data/` into the bundle — *relocation* convention, see cross-cutting findings).
**Files typically touched.** 1-2 paths (the rename), plus often 1-3 references in other files (groups, nodes).
**Co-modification.** The wireguard rename in `1f4aaad` was paired with script changes ("improve wireguard config gen"); the switch renames in `d54eff3` paired with the routeros bundle work.
**Stakes.** Low individually but **silent-trap-prone**: a rename of a node file changes the node's identity (since the loader file-name → node-name). Anything keyed by node name (vault entries via `!password_for:<node>`, hash records, ssh known_hosts) needs a parallel rename or the node loses its associations.
**Implications for agent docs.**
- The plan's workflow validation finding §4 already calls for documenting the bundle naming convention (lowercase-hyphenated, no underscores). ✅
- **Gap.** The node rename failure-mode isn't documented. Agents may rename a node file thinking it's purely cosmetic. One sentence in `nodes/AGENTS.md` Pitfalls: "Renaming a node file renames the node. `!password_for:<old-name>` magic strings, vault entries, and known_hosts associations all key on node name — search and replace before renaming."
### Story 20: Bundlewrap version migration (one-shot)
**Pattern.** Migrating the repo from one major version of bundlewrap to the next. Has happened once in the visible history (4.x → 5.0.3 in `186d503` on 2026-05-10) and is uniquely high-stakes.
**Frequency.** 1 commit in 18 months — but the maintainer is, per the handoff context, also planning to design their own config-management tool, so **this story is effectively an N=1 sample of "language migrations" the maintainer cares about.**
**Files touched.** 13 files: 11 `bundles/<x>/metadata.py`, `items/download.py`, `requirements.txt`. Per the commit body: rewrites "non-reading and KeyError-driven metadata reactors" per the bw 4→5 migration guide; renames custom Download item methods (`cdict/sdict/get_auto_deps` → `expected_state/actual_state/get_auto_attrs`).
**Evidence.** `186d503` (2026-05-10).
**Co-modification.** Wide cross-bundle. Mixes Story 12 (cross-bundle reactor) with Story 6 (items.py interface) into one operation.
**Stakes.** Highest. Rewriting reactors and item-type interfaces simultaneously means the repo is broken until the last file is corrected.
**Implications for agent docs.**
- `conventions.md` (planned) notes the bundlewrap version. ✅
- `commands.md` notes `requirements.txt` is the version pin. ✅
- **Gap.** The migration was done with Claude assistance (per the commit's `Co-Authored-By`). An agent helping with the *next* major migration would benefit from a one-paragraph "how to do a bw version migration" recipe in `conventions.md`: read upstream migration guide → list affected reactor patterns → grep for them → rewrite each → bump `requirements.txt` last. **Recommendation (don't act):** worth capturing as a section in conventions, given the maintainer's own tool-design plans suggest such migrations will recur.
### Story 21: Interactive `bw debug` investigation
**Pattern.** Open `bw debug` (the bundlewrap interactive Python REPL with `repo` and `bw` preloaded), explore the repo state, prototype a code snippet. The 15-line `.bw_debug_history` is short but representative of how the user investigates before changing.
**Frequency.** Hard to count from git (history file is gitignored as of `39d5fb8` 2025-10-28). Evidence in-tree is one persistent file with 15 unique lines.
**What's in `.bw_debug_history`:**
```
bw.get_node('home.switch-wohnzimmer-10g')
repo.get_node('home.switch-wohnzimmer-10g')
repo.get_node('home.switch-wohnzimmer-10g').password
from os import path, listdir
path.join(repo.path, 'bundles/left4dead2/files/scripts/overlays')
listdir(path.join(repo.path, 'bundles/left4dead2/files/scripts/overlays'))
listdir(path.join(repo.path, 'bundles/left4dead2/files/scripts/overlays'))[0]
listdir(path.join(repo.path, 'bundles/left4dead2/files/scripts/overlays'))
from os import path, listdir
a = listdir(path.join(repo.path, 'bundles/left4dead2/files/scripts/overlays'))[1]
a
type(a)
repo.libs.ovh
type(a)
repo.libs.ovh
```
**What this reveals.** The user (a) treats `bw debug` as a Python REPL with `repo`/`bw` pre-bound; (b) inspects nodes by name; (c) walks the filesystem inside `bundles/<x>/files/<dir>/` (the l4d overlay scripts work from Story 5); (d) probes `repo.libs.<name>` to see what's available — the trailing `repo.libs.ovh` looked up an ovh helper. This is **dynamic discovery**, complementary to file reads.
**Stakes.** Read-only — `bw debug` doesn't apply state. Routine.
**Implications for agent docs.**
- `commands.md` lists `bw debug` as read-only. ✅
- **⚠ Gap.** The *content* of `bw debug` (what's in scope: `repo`, `bw`, `node = repo.get_node(...)`, `node.metadata`, `repo.libs.<x>`) isn't sketched in `commands.md`. An agent that wants to investigate dynamically won't know what to type. One paragraph + 5 example lines (drawn directly from `.bw_debug_history`) would suffice.
- Mention the gitignore: `.bw_debug_history` is gitignored (as of `39d5fb8`), so an agent's debug session won't accidentally land in a commit. Already covered by spec §4 quickstart's "do not modify" list. ✅
## Cross-cutting findings
### F1. Commit-message style is terse, iterative, and unreliable
First-word frequency: `l4d` (22), `fix` (18), `bootshorn` (8), `routeros` (6), `remove` (6), `update` (4), `more` (4), `add` (4). Many subjects are typos (`fixmo`, `besteffort`, `dowsnt exist`, `mroe`, `unfault`, `englisch sprache schwere sprache`). The terseness is feature, not bug — see Story 4 (microbursts as workflow). Conclusion: **agents must read diffs, not subjects.** The detailed-history extraction approach (`git log --name-status -M -C`) was load-bearing; subject-only clustering would have over-bucketed.
### F2. The "introduce" verb is a reliable seed marker
When the user names a commit `<path>: introduce` (or `<path>: introduce + …`), it's almost always the seed of a new bundle/script/hook/node skeleton. ~7 such commits in 18 months. Not a story per se, but a useful corpus marker for "where did X come from."
### F3. Naming convention: lowercase-hyphenated
Confirmed across the bundle list — every current bundle uses lowercase-hyphenated naming (e.g. `backup-server`, `bind-acme`, `dm-crypt`, `homeassistant-supervised`, `kea-dhcpd`, `mailserver-autoconfig`, `nginx-rtmps`, `nextcloud-picsort`, `proxmox-ve`, `raspberry-pi`, `raspberrymatic-cert`, `routeros-monitoring`, `systemd-networkd`, `systemd-swap`, `systemd-timers`, `tasmota-charge`, `wol-sleeper`, `wol-waker`, `zfs-mirror`). The few underscore-named bundles (`left4dead2_old`, `left4dead2_old2`, `left4dead2_steam_old`) were all deleted via `ac8e7e2`. Bin scripts mostly hyphenated (after `1f4aaad` rename) but **a handful still underscored** (`bin/sync_1password`, `bin/timestamp_icloud_photos_for_nextcloud`, `bin/upgrade_and_restart_all`, `bin/script_template`, `bin/passwords-for`). Verdict: **convention is hyphenated, but underscored exceptions exist in `bin/`.** An agent should normalize to hyphenated for new bundles/nodes; for `bin/`, match the user's existing style (mixed).
### F4. Subdirectory-relocation pattern in active bundles
`78a8abc` moved `data/routeros-monitoring/files/mikrotik.mib` into `bundles/routeros-monitoring/files/mikrotik.mib`, with the commit message "move to bundle bc why not." This was a *consolidation* — pulling bundle-specific data from `data/` into the bundle itself. Single instance, not yet a strong story, but suggests the user's mental model is `data/` = "shared/templated artifacts read by multiple bundles", `bundles/<x>/files/` = "this bundle's static files." Worth a sentence in `data/AGENTS.md`: "if a data asset is read by exactly one bundle, prefer placing it in `bundles/<x>/files/`."
### F5. Bursts have a characteristic shape
Pattern observed in all 5 bursts: (i) seed commit introduces the skeleton in 1-3 hours of intense work; (ii) 1-2 day cooling period; (iii) 2-6 weeks of follow-up commits with shrinking commit-size; (iv) often a "delete obsolete" cleanup commit (Story 13) at the end. l4d2 followed this shape exactly (seed `3469d98` Aug 24, refinement Aug-Feb, cleanup `ac8e7e2` Feb 10). Recognizable signature for an agent reading recent history: many commits to one area in N days = active subsystem.
### F6. `_old` and `_old2` as a "soft delete" pattern
The l4d2 rebuild renamed working code into `bundles/left4dead2_old/` and `bundles/left4dead2_old2/` (commit `3469d98`) before later deletion in `ac8e7e2`. The user uses suffixed-with-`_old` directories as a recovery buffer during big refactors. **Agents shouldn't delete `_old`/`_old2`-suffixed bundles without checking with the user**; they may be deliberate parking spots. Currently there are none in-tree (the `delete old l4d bundles` cleanup landed), but the pattern will recur.
### F7. The repo has no test suite, no CI, no formal type annotations
Confirmed by tree inspection and history. The verification model is `bw test` (loader sanity), `bw hash` (state diff), `bw verify` (drift check) — a runtime check, not a static one. This shapes agent expectations: **don't propose test additions or CI without explicit user request** (the spec's "no tooling changes" non-goal aligns). Agents should use `bw debug` for ad-hoc validation instead of writing fixtures.
### F8. Merge commits are real, but bursts are usually merged via squash
PRs visible in the history: `#18` homeassistant-supervised, `#19` mseibert_yourls, `#22` proxmox_mergable, `#23` routeros, `#24` ipv6_picking, `#25` debian-13, `#26` htz.mails_debian_13_squash, `#27` l4d2_the_next. The user uses Gitea/Forgejo for PRs (self-hosted). Naming pattern: descriptive branch name → PR. For agents creating PRs in this repo, branch-name style: lowercase-snake_case description (mirrors PR titles).
### F9. The README.md is a personal TODO, not a project README
First lines of `README.md`: `# TODO\n\n- dont spamfilter forwarded mails\n- gollum wiki\n- blog?`. Already noted in spec §1 ("untouched personal TODO list") and plan PR1 step 8. **Agents must not regard the root README as documentation for the repo.** Unique enough to merit explicit callout in root `AGENTS.md`: "Note: `README.md` is the maintainer's personal scratchpad. Real onboarding lives here." Spec §4 and plan don't currently include that exact callout.
### F10. Commit cadence is bursty across years, not just months
Total commits since first commit (2021-06-11): 1169. Last 36 months: 343. Last 24 months: 251. Last 18 months: 222. Last 12 months: 158. So ~70% of commits are in the last 24 months, but only ~10% in the last 12 months — meaning **commit activity has been declining**. Agents reading "recent history" should weight 6-12 months heavier than 12-24 months for "current state." (One nuance: the very last burst — May 2026 bundlewrap-5 migration — is recent, suggesting renewed activity ahead of the planned tool-design pivot.)
## Story coverage assessment vs the planned docs
For each story, traffic-light against PR1+PR2 as currently planned (root `AGENTS.md`, 8 area docs, conventions.md, commands.md, per-bundle template, 10 seed per-bundle docs, docstring/header pass on libs/hooks/bin, fork's `AGENTS.md` for bundlewrap language).
| # | Story | Status | Justification |
|---|---|---|---|
| 1 | Tune one bundle's metadata | ✅ | Per-bundle `Metadata` section; `commands.md` after-change row; mostly works as planned. |
| 2 | Tweak one node's config | ⚠ | `nodes/AGENTS.md` covers eval() but should explicitly add the silent-load-failure pitfall (the user himself patched the loader for this — `dc40295`). |
| 3 | Edit a file template | ⚠ | `bundles/AGENTS.md` lists `files/` in anatomy, but doesn't document Mako-vs-static recognition. Plan workflow finding §5 already adds the link to the fork's template guide — likely sufficient when written. |
| 4 | Trial-and-error microburst | ❌ | Workflow-style context not captured anywhere. Belongs in root `AGENTS.md` quickstart or `conventions.md`: "user's commits are iterative; don't rebase WIP without asking." |
| 5 | Subsystem deep-dive burst | ❌ | "Hot subsystem" awareness not in any planned doc. Seed bundle list (Phase 2) misses 3 of 5 recent burst targets (routeros, routeros-monitoring, bootshorn). Recommend rebalancing or documenting how to detect flux. |
| 6 | Modify items.py for a bundle | ✅ | `bundles/AGENTS.md` + fork's items reference covers item types & deps. Item `download.py` docstring (PR1 step 7) closes the custom-item gap when written. |
| 7 | Templated data update | ⚠ | `data/AGENTS.md` will exist; needs to distinguish apt-keys-as-data vs grafana-rows-as-Python. After-change check for apt keys not explicit in `commands.md`'s table — a one-line addition. |
| 8 | Add a new bundle | ✅ | Per-bundle template + spec §3 is well-shaped. Plan workflow finding §3 added explicit wiring step. ✅ |
| 9 | Onboard a new node | ⚠ | `nodes/AGENTS.md` covers eval() & naming, but missing: (a) the `*.py_` suspend convention; (b) symmetric "How to add a node" workflow paragraph. Both are one-line fixes. |
| 10 | Group adjustment / new OS variant | ⚠ | `groups/AGENTS.md` covers basics; missing the family-file pattern (`debian-N-common.py` shared by `debian-N.py` + `debian-N-pve.py`) and the new-OS recipe. Both are paragraph-sized adds. |
| 11 | Disable temporarily / "for now" | ❌ | Not in any planned doc. Real risk: agents "fixing" deliberately-stubbed code. Belongs in `conventions.md` as a single paragraph. |
| 12 | Cross-bundle metadata-reactor refactor | ⚠ | `commands.md` warns about cross-namespace ripple in general. Per-bundle template lacks a `Writes into` field — recommend adding it (or folding into `Gotchas`) for cross-writing bundles (telegraf, monitored, archive, wol-waker, apt). |
| 13 | Cleanup / delete obsolete code | ⚠ | "How to remove a bundle" not in `bundles/AGENTS.md`. Symmetric to "How to add" — 5-line addition. |
| 14 | Add or revise an operator script (`bin/`) | ⚠ | `bin/AGENTS.md` covers what bin/ is; missing mention of `bin/script_template` as the canonical starter. One-line. |
| 15 | OS-version upgrade campaign | ⚠ | Inheritance order in `conventions.md`; no campaign recipe. Belongs in `groups/AGENTS.md` "How to add" — paragraph-sized. |
| 16 | Add or modify a lib | ✅ | `libs/AGENTS.md` + docstring pass + `commands.md` blast-radius row. Could optionally add a "find consumers" snippet (`git grep`). Minor. |
| 17 | Add or modify a hook | ⚠ | `hooks/AGENTS.md` covers lifecycle; missing the "broken hook breaks all bw commands" failure-mode and isolation-test recipe. One paragraph. |
| 18 | Per-bundle README touch-up | ⚠ | Plan undercounts existing READMEs (10 → actual 33). PR2 only addresses seed bundles; ~23 READMEs survive. Plan should acknowledge the transition state, and `bundles/AGENTS.md` should note "if both exist, AGENTS.md is canonical." |
| 19 | Naming-convention rename | ⚠ | Plan workflow finding §4 covers bundle naming. Missing: node-rename failure mode (renaming changes node identity, breaks vault keys). Sentence in `nodes/AGENTS.md`. |
| 20 | Bundlewrap version migration | ✅ | One-shot; `conventions.md` will note version. Optional: paragraph in `conventions.md` capturing the migration recipe — useful given maintainer's tool-design pivot. |
| 21 | Interactive `bw debug` investigation | ⚠ | `commands.md` lists `bw debug` as read-only. Missing: what's in scope inside `bw debug` (paragraph + 5 example lines pulled from `.bw_debug_history`). |
**Tally.** ✅ 5 / ⚠ 13 / ❌ 3.
The ❌ gaps (S4 trial-and-error workflow context, S5 burst-state awareness, S11 disable-for-now idiom) are the highest-value adds because they shape an agent's *judgment*, not just its lookup steps. The ⚠ gaps are mostly one-line / one-paragraph additions to area docs that are already in scope for PR1.
The Phase 2 seed list is the largest single open question raised by this analysis: it leans toward usage-frequency hubs (postgresql, wireguard, php, nginx) but underweights currently-hot subsystems (routeros, routeros-monitoring, bootshorn). A swap of 2-3 entries — e.g. `php``routeros` and `apt``bootshorn` — would better match the work the maintainer (and any helper agent) will actually be doing in the next quarter. This is for human decision; not acted on.

110
groups/AGENTS.md Normal file
View file

@ -0,0 +1,110 @@
# groups/
## What's here
Groups attach bundles and shared metadata to nodes. One file per group,
organized by axis:
```
groups/
├── all.py # universal base (every node belongs)
├── applications/<x>.py # role-shaped groups (mailserver, monitored, …)
├── locations/<x>.py # physical/network location (home, htz, …)
├── machine/<x>.py # hardware kind (hardware, hetzner-cloud, raspberry-pi)
└── os/<x>.py # OS major/variant (debian-13, debian-13-pve, routeros, …)
```
## Loader mechanism
`groups.py` (top of the repo) walks `groups/` and runs `eval()` on each
`*.py`. Same eval-as-expression rule as
[`nodes/`](../nodes/AGENTS.md#loader-mechanism): one dict literal, no
top-level imports, no statements. Errors print and the group is skipped
— a real foot-gun, since a missing group silently changes node
membership.
**Group files are *not* demagified.** Magic strings like
`!password_for:<x>` only resolve in `nodes/*.py`. Inside a group file,
call `repo.vault.<verb>(...)` directly.
## Inheritance and merge order
Metadata merges along this chain:
```
all → location → os → machine → applications → node
```
Per-axis subdirs are conventional, not enforced — `bw` doesn't read the
subdir. Each group lists its `supergroups`, and `bw` resolves the DAG.
Membership is set-union; metadata merge follows the order above, with
the node's own `metadata` block winning last.
## Conventions
- **One group per file.** Filename without `.py` = group name. Subdir
groups them by axis for humans, not for bw.
- **Family files for OS variants.** Common parent + per-variant child.
Example: `debian-13-common.py` is shared by `debian-13.py` and
`debian-13-pve.py`. Use this pattern when introducing
related-but-distinct OS group families.
- **`all.py` is the universal default.** Currently empty (`{}`); kept
for the rare repo-wide opt-in.
## How to add a group
1. Pick the right axis subdir (or root `all.py` for universal default).
2. Create `groups/<axis>/<name>.py` as a single dict expression:
```python
{
'supergroups': [
# parent groups whose bundles/metadata this one extends
],
'bundles': [
# bundles every member of this group should have
],
'metadata': {
# shared metadata for members
},
}
```
3. Wire the group into the relevant `nodes/*.py` (`'groups': {...}`)
or `groups/*.py` `supergroups` list.
4. Verify with `bw nodes <node> -a groups` and `bw metadata <node>`.
## How to add a new OS major (recipe)
Pattern from prior debian-12 → debian-13 work:
1. Add `groups/os/debian-N.py` and `groups/os/debian-N-common.py`
parallel to the existing files. Don't edit in place.
2. Add `data/apt/keys/debian-N-*.{asc,gpg}` for the new release's
signing keys. See
[`commands.md#apt-key-changes-need-offline-verification`](../docs/agents/commands.md#apt-key-changes-need-offline-verification)
before pushing keys live.
3. Bump dependent bundles that branch on `os_version` /
`os_codename` (`bundles/bind/items.py`, etc.).
4. Bump affected nodes' `groups` lists one at a time. Apply, watch.
5. Delete the old OS group file once no node references it.
## Pitfalls
- **`bw groups -n <node>` doesn't exist.** Use
`bw nodes <node> -a groups`.
- **Cycles.** A group can't be its own supergroup transitively;
`bw test` catches this but the error message is terse.
- **Silent eval failure.** A group file with a syntax error is skipped
and prints a one-line error. If a node loses bundles unexpectedly,
scan `groups.py` output for the error.
## See also
- [`nodes/AGENTS.md`](../nodes/AGENTS.md) — node files; how
`groups: {...}` attaches groups.
- [`docs/agents/conventions.md`](../docs/agents/conventions.md) —
inheritance order, naming conventions, eval-loader constraints.
- Fork's [`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
— group attribute reference, metadata-merge semantics.

62
hooks/AGENTS.md Normal file
View file

@ -0,0 +1,62 @@
# hooks/
## What's here
Repo-level lifecycle hooks. Each `*.py` exports one or more functions
named after the lifecycle event they listen to (`apply_start`,
`node_apply_start`, `node_run_start`, `test`, `test_node`, …). Bw
discovers them by importing each module from `hooks/`.
Discovery is by `ls hooks/` + the one-line docstring at the top of
each file:
```sh
head -1 hooks/*.py
```
## Conventions
- **One-line module docstring.** Every `hooks/*.py` starts with
`"""<name>: <one-line purpose>."""`. Add one when introducing a new
hook; baseline is enforced by `grep -L '"""' hooks/*.py`.
- **Function name = event name.** Bw calls
`apply_start(repo, target, nodes, interactive=False, **kwargs)`,
`node_apply_start(repo, node, interactive, **kwargs)`,
`test(repo, **kwargs)`, etc. Always accept `**kwargs` so future bw
arguments don't break the hook.
- **Test gates use `test` / `test_node`.** Anything that should fail
`bw test` (and therefore CI / pre-apply sanity) goes here; avoid
doing test-style assertions in `apply_start`.
## How to add a hook
1. Pick the lifecycle event (see fork's `AGENTS.md` for the full list).
2. Create `hooks/<name>.py` with the matching function and a
docstring.
3. Run `bw test` once to confirm the hook loads cleanly.
## Pitfalls
- **A hook that errors at import breaks every `bw` invocation** that
fires the hook's lifecycle — including `bw test` itself, which
defeats the obvious diagnostic. Test new hooks in isolation first:
```sh
bw debug -c "import sys; sys.path.insert(0, 'hooks'); import <hookmodule>"
```
Iterate there until the import is clean, then commit.
- **Hooks have access to the full repo (`repo`, `node`, `nodes`).**
Don't make them block on network unless that's the explicit purpose
(e.g. `test_ptr_records.py` does `dig` against `9.9.9.9`).
- **Order is not guaranteed across hook files.** Two hooks that both
define `apply_start` will both fire; don't assume which runs first.
## See also
- [`docs/agents/conventions.md`](../docs/agents/conventions.md) —
files-not-to-touch, vault rules.
- [`docs/agents/commands.md`](../docs/agents/commands.md) — test
workflow.
- Fork's [`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
— full hook lifecycle and signatures.

View file

@ -1,3 +1,5 @@
"""known_hosts: pre-apply hook that writes all nodes' ssh/is_known_as entries to ~/.ssh/known_hosts_ckn."""
from os.path import expanduser

View file

@ -1,3 +1,5 @@
"""skip_local_nodes: skip apply on `localhost` nodes whose metadata `id` doesn't match this host's local_id."""
from bundlewrap.exceptions import SkipNode
def node_apply_start(repo, node, interactive, **kwargs):

View file

@ -1,3 +1,5 @@
"""test_ptr_records: bw-test gate verifying live A/PTR DNS records for the `mailserver` group via `dig @9.9.9.9`."""
from subprocess import check_output

View file

@ -1,3 +1,6 @@
"""unique_node_ids: bw-test + pre-apply gate ensuring metadata `id` is unique across all nodes."""
def test_unique_node_ids(repo):
ids = {}
for node in repo.nodes:

View file

@ -1,3 +1,6 @@
"""wake_on_lan: pre-apply / pre-run hook that wakes a node via libs.wol before bw connects to it."""
def wake_on_lan(node):
node.repo.libs.wol.wake(node)

64
items/AGENTS.md Normal file
View file

@ -0,0 +1,64 @@
# items/
## What's here
Custom item types — each `*.py` is a `bundlewrap.items.Item` subclass
that adds a new item kind to the repo (e.g. `download:` from
`items/download.py`).
Currently:
- `items/download.py` — verifying file downloads with sha256 / GPG.
(Originally from `bundlewrap/plugins/item_download`; vendored in
because bw 4 dropped plugin support and it never came back.)
## Conventions
- **Filename = item-type slug.** `items/download.py` defines item
type `download:`. The class's `BUNDLE_ATTRIBUTE_NAME` is the dict
name in `items.py` (`downloads = {...}`).
- **Subclass `bundlewrap.items.Item`.** See the upstream
[bundlewrap docs on writing item types](https://docs.bundlewrap.org/dev/items/)
for the contract: `ITEM_ATTRIBUTES`, `fix`, `sdict`, `cdict`,
`display_dicts`, etc.
- **No metadata-reactor integration unless deliberate.** Item types
get to declare `NEEDS_STATIC` (cross-type ordering hints) — use it
sparingly; broad `NEEDS_STATIC` slows the whole DAG.
## When to write a new custom item type vs. use `file:` (or
`action:`, etc.)
| Situation | Use |
|---|---|
| One-off shell command at apply time | `action:` in the bundle |
| File whose source is in `bundles/<x>/files/` | `file:` |
| File you have to **fetch + verify** at apply | `download:` (already custom) |
| Behavior bw doesn't model and that recurs across bundles | new custom item |
If you only need it once, an `action:` is almost always enough.
Custom item types are repo-wide and load on every `bw` invocation —
the cost is paid forever.
## How to add a custom item type
1. Pick a slug (lowercase, no underscore).
2. Create `items/<slug>.py` with an `Item` subclass.
3. `bw test` — broken item types break the loader for every bundle
that uses them, so test in isolation first.
4. Document the contract in the file's module docstring.
## Pitfalls
- **Items affect the whole repo.** A change to `items/<x>.py` runs in
every node's apply. Treat custom items the way you treat `libs/`
pure, fast, side-effect-free at import.
- **`NEEDS_STATIC` is a coarse hammer.** It enforces ordering across
*all* nodes for *all* items of those types; don't add unless
you've actually hit ordering issues.
## See also
- [`bundles/AGENTS.md`](../bundles/AGENTS.md) — items are consumed by
`items.py` in each bundle.
- Fork's [`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
— built-in item-type catalogue.

68
libs/AGENTS.md Normal file
View file

@ -0,0 +1,68 @@
# libs/
## What's here
Shared Python helpers reachable from any bundle as
`repo.libs.<modulename>.<symbol>`. One module per file (no packages).
Discovery is by `ls libs/` plus the one-line module docstring at the
top of each file.
```sh
head -1 libs/*.py
```
## Conventions
- **One file per module.** Filename (without `.py`) is the
importable name. `libs/wireguard.py` exposes `repo.libs.wireguard`.
- **One-line module docstring.** Every `libs/*.py` starts with
`"""<name>: <one-line purpose>."""` so `ls + head` is a working index.
Add one when introducing a new lib; the rule is enforced by the
Phase-0 baseline (`grep -L '"""' libs/*.py` should report zero
files).
- **Pure helpers, no I/O at import.** Libs are imported on every `bw`
invocation. Heavy work or filesystem reads at module scope slow the
whole repo. Use functions, not module-level side effects.
- **No magic-string demagification here.** Libs receive already-
resolved values from bundles or nodes. Don't pass `!password_for:`
strings into libs — they won't be resolved.
- **`repo` and `vault` are globals available at runtime.** A few libs
(`wireguard.py`, etc.) reach `repo.vault.<verb>(...)` directly.
That's intentional inside libs, since libs run in the same loader
scope as bundles.
## How to add a helper
1. Either extract from a bundle that's growing complex, or add a new
`libs/<name>.py` file.
2. Top line: `"""<name>: <one-line purpose>."""`.
3. Pure functions only; document side effects in the docstring if any
slip through.
4. Use it from `bundles/<x>/items.py` or `metadata.py` as
`repo.libs.<name>.<symbol>(...)`.
## Pitfalls
- **Lib changes have repo-wide blast radius.** Every bundle that
imports the lib re-evaluates on the next `bw test` / `bw apply`.
Before changing a lib's API, find consumers:
```sh
git grep -l 'repo.libs.<x>'
```
- **`@cache` is your friend** for deterministic key derivations
(`wireguard`, `rsa`, `ssh`); without it, `bw` recomputes per call,
which is slow.
- **Hashable wrappers.** `libs/hashable.py` exists because raw dicts
and sets aren't hashable, so you can't put them inside metadata
sets. Wrap with `repo.libs.hashable.hashable(...)` before nesting.
## See also
- [`bundles/AGENTS.md`](../bundles/AGENTS.md) — when to extract a
helper into `libs/` instead of duplicating across bundles.
- [`docs/agents/conventions.md`](../docs/agents/conventions.md) —
vault, demagify, naming.
- The fork's [`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
`repo.libs` mechanism.

View file

@ -1,3 +1,5 @@
"""apt: render sources.list.d entries and resolve apt-key file extensions, with codename/version templating."""
from re import match
from glob import glob
from os.path import join, basename, exists

View file

@ -1,3 +1,5 @@
"""bind: DNS view-routing helpers — match A/AAAA records against internal vs external views via repo-wide collation."""
from ipaddress import ip_address
def _values_from_all_nodes(type, name, zone):

View file

@ -1,4 +1,5 @@
#!/usr/bin/env python
"""derive_string: deterministic byte-string derivation from a seed via ChaCha20 keystream."""
from hashlib import sha3_256
from itertools import count, islice

View file

@ -1,3 +1,5 @@
"""grafana: panel and Flux-query generators for Mako-templated dashboards under data/grafana/rows/."""
from mako.template import Template
from copy import deepcopy

View file

@ -1,3 +1,5 @@
"""hashable: dict/set subclasses with stable __hash__ via canonical JSON — lets you nest dicts/sets inside sets in metadata."""
import json
from functools import total_ordering

View file

@ -1,3 +1,5 @@
"""hmac: base64-encoded HMAC-SHA512 helper."""
import hmac, hashlib, base64
def hmac_sha512(secret, iv):

View file

@ -1,3 +1,5 @@
"""ini: case-preserving ConfigParser parse/dumps for INI-style files where exact key casing must round-trip."""
from configparser import ConfigParser
import json
from bundlewrap.metadata import MetadataJSONEncoder

View file

@ -1,3 +1,5 @@
"""ip: A/AAAA-record derivation from a node's `network` metadata + cross-node IP collection."""
from ipaddress import ip_address, ip_interface

View file

@ -1,3 +1,5 @@
"""local: identifier of the bw host itself (read from ~/.config/bundlewrap/local_id) — paired with hooks/skip_local_nodes.py."""
from os.path import expanduser, exists
from functools import cache

View file

@ -1,2 +1,5 @@
"""nextcloud: build `sudo -u www-data php occ ...` shell strings for use in actions."""
def occ(command, *args, **kwargs):
return f"""sudo -u www-data php /opt/nextcloud/occ {command} {' '.join(args)} {' '.join(f'--{name.replace("_", "-")}' + (f'={value}' if value else '') for name, value in kwargs.items())}"""

View file

@ -1,3 +1,6 @@
"""nginx: recursive nginx config-block rendering from a nested dict."""
def render_config(config):
return '\n'.join(render_lines(config))

View file

@ -1,3 +1,5 @@
"""postgres: SCRAM-SHA-256 password-hash generation for postgres role provisioning."""
from base64 import standard_b64encode
from hashlib import pbkdf2_hmac, sha256
import hmac

View file

@ -1,3 +1,5 @@
"""rsa: deterministic RSA private-key generation backed by a seeded PRNG."""
# https://stackoverflow.com/a/18266970
from Crypto.PublicKey import RSA

View file

@ -1,3 +1,5 @@
"""ssh: deterministic ed25519 keypair generation and salted known_hosts hashing."""
from base64 import b64decode, b64encode
from hashlib import sha3_224, sha1
from functools import cache

View file

@ -1,3 +1,5 @@
"""systemd: unit-file rendering (Mako) and a reusable hardening-options dict for sandboxed services."""
from mako.template import Template
template = '''

View file

@ -1,3 +1,5 @@
"""tools: identifier→IPs resolver (node|group|cidr), subnet-set deduplication, and bundle-requirement assertion."""
from ipaddress import ip_address, ip_network, IPv4Address, IPv4Network
from bundlewrap.exceptions import NoSuchGroup, NoSuchNode, BundleError

View file

@ -1,3 +1,5 @@
"""version: comparable Version class for dotted-int version strings."""
from functools import total_ordering

View file

@ -1,3 +1,5 @@
"""wireguard: deterministic WireGuard private/public key + PSK derivation, backed by repo.vault.random_bytes_as_base64_for."""
import base64
from nacl.public import PrivateKey
from nacl.encoding import Base64Encoder

View file

@ -1,3 +1,5 @@
"""wol: trigger a `wol-sleeper` waker node to run its configured wake command."""
from bundlewrap.utils.ui import io
from bundlewrap.utils.text import yellow, bold

92
nodes/AGENTS.md Normal file
View file

@ -0,0 +1,92 @@
# nodes/
## What's here
One file per node, ~22 in total. Each file is a single Python
expression (a dict literal) describing one machine: hostname, groups,
bundles, and metadata.
## Loader mechanism
`nodes.py` (top of the repo) walks `nodes/`, reads each `*.py` file,
runs `eval()` on its content, and then *demagifies* the result —
resolving any `!password_for:`, `!decrypt:`, `!decrypt_file:`, and
`!32_random_bytes_as_base64_for:` magic strings via `repo.vault`. See
[`docs/agents/conventions.md#secrets`](../docs/agents/conventions.md#secrets)
for the magic-string list.
This loader shape has consequences:
- **No top-level imports.** The file must be a single expression. No
`import os`, no `def`, no `if`. Use `repo.libs.<x>` from bundle code
if you need a helper.
- **Silent drop on parse failure.** Vanilla bundlewrap omits a node
whose file fails to eval. The maintainer's `groups.py` was patched
in commit `dc40295` to print the error; the node-loader prints
on `nodes.py` errors via the same shape. Symptom either way:
`bw nodes` lists fewer nodes than expected.
## Conventions
- **Filename = node name.** `home.server.py` defines the node `home.server`.
- **Naming pattern: `<location>.<role>.py`.** Examples:
`home.server.py`, `htz.mails.py`, `ovh.left4me.py`,
`mseibert.freescout.py`.
- **`*.py_` parks a node** without loading it. Used to keep
decommissioned-but-not-deleted configs in tree (e.g.
`htz.l4d2.py_`). The loader only matches `*.py`.
- **Magic strings only resolve here.** `!password_for:` etc. work in
node files; in groups, bundles, or items they don't — call
`repo.vault.<verb>(...)` directly there.
## How to add a new node
1. Create `nodes/<location>.<role>.py` with a single dict expression:
```python
{
'id': 'a-uuid-or-stable-name',
'hostname': '<dns-name-or-ip>',
'groups': {
'debian-13',
'monitored',
# …
},
'bundles': {
# only bundles not provided by the groups above
},
'metadata': {
# node-local overrides and required keys
},
}
```
2. Add to relevant `groups/<axis>/<x>.py` if group membership is the
attachment point (preferred over per-node `bundles` lists).
3. Verify:
- `bw nodes` — your node should appear.
- `bw nodes <node> -a groups` — confirm group membership resolved
as expected (`bw groups -n <node>` does **not** exist).
- `bw metadata <node>` — confirm merged metadata.
## Pitfalls
- **Renaming a node renames the node.** Vault entries (anything keyed
on `!password_for:<node>`), `bw hash` records, and ssh known_hosts
associations all key on node name. Search-and-replace before
renaming, or vault lookups silently return new (wrong) values.
- **Don't restore `_old` or `*.py_` files** without checking
[`conventions.md#suspension-and-soft-delete-idioms`](../docs/agents/conventions.md#suspension-and-soft-delete-idioms).
These are intentional parks/buffers, not bugs.
- **`id` must be unique.** A pre-apply hook (`hooks/unique_node_ids.py`)
enforces this; duplicate IDs fail `bw test` and `bw apply`.
## See also
- [`groups/AGENTS.md`](../groups/AGENTS.md) — group-membership patterns,
how metadata merges along the chain.
- [`docs/agents/conventions.md`](../docs/agents/conventions.md) —
demagify magic-strings, naming, eval-loader constraints.
- Fork's [`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
— node attribute reference (`hostname`, `username`, `dummy`, etc.).

View file

@ -1,4 +1,4 @@
bundlewrap ~=5.0, >=5.0.3
-e git+https://github.com/CroneKorkN/bundlewrap.git@main#egg=bundlewrap
pycryptodome
PyNaCl
PyYAML