Compare commits
10 commits
c03b033ad9
...
d4dedde0ad
| Author | SHA1 | Date | |
|---|---|---|---|
| d4dedde0ad | |||
| 7b44a8ad3a | |||
| 9e1bb2ac45 | |||
| 04558a9189 | |||
| 730625e36c | |||
| 136313e9c3 | |||
| 1da70970e5 | |||
| 3daf70dae7 | |||
| b804350f17 | |||
| 7486c78ae1 |
51 changed files with 2452 additions and 122 deletions
106
AGENTS.md
Normal file
106
AGENTS.md
Normal file
|
|
@ -0,0 +1,106 @@
|
|||
# ckn-bw — agent & contributor guide
|
||||
|
||||
## What this repo is
|
||||
|
||||
A [BundleWrap](https://bundlewrap.org/) configuration-management repo
|
||||
for ~22 personal/family-infra nodes. Nodes, groups, and bundles are
|
||||
defined in plain Python; `bw apply` deploys the resulting state to
|
||||
real machines.
|
||||
|
||||
Note: the root `README.md` is the maintainer's personal scratchpad,
|
||||
not project documentation. Onboarding lives **here**, in `AGENTS.md`.
|
||||
|
||||
## Quickstart for agents
|
||||
|
||||
Five rules; follow these and you won't break things:
|
||||
|
||||
1. **Read-only by default.** Never run `bw apply`, `bw run`, or
|
||||
`bw lock` without explicit user request — even with `-i`. Stick
|
||||
to `bw test`, `bw nodes`, `bw groups`, `bw bundles`,
|
||||
`bw items`, `bw metadata`, `bw hash`, `bw debug`. See
|
||||
[`docs/agents/commands.md`](docs/agents/commands.md) and the
|
||||
fork's [safety envelope](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md).
|
||||
2. **Never echo decrypted secrets.** Don't print, paste, or log the
|
||||
value behind a `!password_for:`, `!decrypt:`, or
|
||||
`!32_random_bytes_as_base64_for:` magic string — not even from
|
||||
`bw debug` exploration. See
|
||||
[`conventions.md#secrets`](docs/agents/conventions.md#secrets).
|
||||
3. **Don't touch the do-not-modify list.** `.secrets.cfg*`, `.venv`,
|
||||
`.cache`, `.bw_debug_history`, `.envrc`, root `README.md`. Treat
|
||||
`hooks/` and `items/` (custom item types) with extra care: a
|
||||
broken hook or item type breaks every `bw` command repo-wide.
|
||||
4. **Use the fork.** The venv runs editable from
|
||||
[`github.com/CroneKorkN/bundlewrap`](https://github.com/CroneKorkN/bundlewrap)
|
||||
(branch `main`). Behavior tracks upstream `main`; the fork's
|
||||
[`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
|
||||
is the canonical bundlewrap-language reference. See
|
||||
[`conventions.md#bundlewrap-version`](docs/agents/conventions.md#bundlewrap-version).
|
||||
5. **Prefer adding helpers to `libs/`** over duplicating logic across
|
||||
bundles. Repo-wide helpers go in
|
||||
[`libs/`](libs/AGENTS.md), reachable as `repo.libs.<x>`.
|
||||
|
||||
## Layout
|
||||
|
||||
| Dir | What's there |
|
||||
|---|---|
|
||||
| [`bundles/`](bundles/AGENTS.md) | 103 bundles. One subdir per bundle (`items.py`, `metadata.py`, `files/`). |
|
||||
| [`nodes/`](nodes/AGENTS.md) | One file per node (~22). `eval()`-loaded; demagified through `repo.vault`. |
|
||||
| [`groups/`](groups/AGENTS.md) | Group definitions, organized by axis (`applications/`, `locations/`, `machine/`, `os/`). |
|
||||
| [`libs/`](libs/AGENTS.md) | Shared Python helpers reachable as `repo.libs.<modulename>`. |
|
||||
| [`hooks/`](hooks/AGENTS.md) | bw lifecycle hooks (`apply_start`, `test`, `node_apply_start`, …). |
|
||||
| [`data/`](data/AGENTS.md) | Out-of-bundle data assets (apt keys, grafana dashboards, …). |
|
||||
| [`items/`](items/AGENTS.md) | Custom item types (currently `download:`). |
|
||||
| [`bin/`](bin/AGENTS.md) | Operator scripts; not invoked by bundlewrap. |
|
||||
| [`docs/agents/`](docs/agents/conventions.md) | Repo conventions and command deltas. |
|
||||
|
||||
## How nodes, groups, and bundles fit together
|
||||
|
||||
- A **node** (`nodes/<location>.<role>.py`) declares the groups it
|
||||
belongs to and any node-local bundles + metadata overrides.
|
||||
- A **group** (`groups/<axis>/<x>.py`) attaches bundles and shared
|
||||
metadata to its members. Groups inherit via `supergroups`.
|
||||
- A **bundle** (`bundles/<x>/`) is one chunk of configuration:
|
||||
`items.py` produces the items (files, services, packages),
|
||||
`metadata.py` declares `defaults` and `@metadata_reactor` functions
|
||||
that derive metadata from other metadata.
|
||||
- The repo-root loaders (`nodes.py`, `groups.py`) walk these dirs and
|
||||
`eval()` each file. `nodes.py` additionally **demagifies** the
|
||||
result, resolving `!password_for:` etc. through `repo.vault`. See
|
||||
[`conventions.md#eval-loaded-node-and-group-files`](docs/agents/conventions.md#eval-loaded-node-and-group-files)
|
||||
for the constraints this places on editors.
|
||||
- Metadata merges along: `all → location → os → machine →
|
||||
applications → node`.
|
||||
|
||||
## Conventions you must know
|
||||
|
||||
| Topic | Where |
|
||||
|---|---|
|
||||
| Bundlewrap-language reference (item types, dep keywords, reactors) | Fork's [`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md) — read first if new to bundlewrap |
|
||||
| Vault / demagify magic strings | [`conventions.md#secrets`](docs/agents/conventions.md#secrets) |
|
||||
| Bundlewrap install (editable from the fork) | [`conventions.md#bundlewrap-version`](docs/agents/conventions.md#bundlewrap-version) |
|
||||
| Group inheritance order, naming patterns | [`conventions.md#group-inheritance-order`](docs/agents/conventions.md#group-inheritance-order), [`#naming-conventions`](docs/agents/conventions.md#naming-conventions) |
|
||||
| Repo-specific bw command deltas (apt keys, suspended nodes, vault echo) | [`commands.md`](docs/agents/commands.md) |
|
||||
| Lib helpers | top-of-file docstrings in `libs/*.py` (`head -1 libs/*.py`) |
|
||||
| Suspension idioms (`*.py_`, `_old/`, "for now") | [`conventions.md#suspension-and-soft-delete-idioms`](docs/agents/conventions.md#suspension-and-soft-delete-idioms) |
|
||||
|
||||
## Where to look for examples
|
||||
|
||||
When writing a new bundle, copy patterns from one that already does
|
||||
the thing you need:
|
||||
|
||||
| Pattern | Look at |
|
||||
|---|---|
|
||||
| Vault calls inside metadata reactors | `bundles/dm-crypt/metadata.py` (compact, focused) |
|
||||
| Mako-templated files | `bundles/bind/items.py` (DNS zonefile rendering) |
|
||||
| Cross-bundle reactor writing | `bundles/nextcloud/metadata.py` (writes into `apt.packages`, `archive.paths`) |
|
||||
| Custom `download:` items | `bundles/minecraft/items.py` |
|
||||
| Node file (single-purpose) | `nodes/home.server.py` |
|
||||
| Group with `supergroups` chain | `groups/os/debian-13.py` |
|
||||
|
||||
## Where this doc lives
|
||||
|
||||
- This file: `AGENTS.md` at the repo root.
|
||||
- `CLAUDE.md` is a symlink to this file — both names point to the same
|
||||
content so different tools can find it.
|
||||
- The personal TODO scratchpad (`README.md`) is **separate** and not
|
||||
project documentation.
|
||||
1
CLAUDE.md
Symbolic link
1
CLAUDE.md
Symbolic link
|
|
@ -0,0 +1 @@
|
|||
AGENTS.md
|
||||
|
|
@ -13,10 +13,6 @@ Raspberry pi as soundcard
|
|||
- OTG g_audio
|
||||
- https://audiosciencereview.com/forum/index.php?threads/raspberry-pi-as-usb-to-i2s-adapter.8567/post-215824
|
||||
|
||||
# install bw fork
|
||||
|
||||
pip3 install --editable git+file:///Users/mwiegand/Projekte/bundlewrap-fork@main#egg=bundlewrap
|
||||
|
||||
# monitor timers
|
||||
|
||||
```sh
|
||||
|
|
|
|||
62
bin/AGENTS.md
Normal file
62
bin/AGENTS.md
Normal file
|
|
@ -0,0 +1,62 @@
|
|||
# bin/
|
||||
|
||||
## What's here
|
||||
|
||||
Operator scripts — invoked manually by the maintainer, **not** by
|
||||
bundlewrap itself. Each is a standalone Python (or shell) script that
|
||||
opens the repo via `Repository(dirname(dirname(realpath(__file__))))`.
|
||||
|
||||
Discovery is by `ls bin/` plus the `# purpose:` header line at the top
|
||||
of each script:
|
||||
|
||||
```sh
|
||||
head -2 bin/*
|
||||
```
|
||||
|
||||
## Conventions
|
||||
|
||||
- **`# purpose:` header.** Every script under `bin/` starts with
|
||||
`#!/usr/bin/env python3` (or appropriate shebang), then a
|
||||
`# purpose: <one-line description>` comment. Baseline enforced by
|
||||
`grep -L '^# purpose' bin/*`.
|
||||
- **Self-contained.** A script must work when run from anywhere — it
|
||||
resolves the repo via the script's own path, not `cwd`.
|
||||
- **Read-only by default.** Most operator scripts query/print state
|
||||
(`passwords-for`, `wireguard-client-config`). Mutating scripts
|
||||
(`upgrade_and_restart_all`, `mikrotik-firmware-updater`,
|
||||
`sync_1password`) are the exception, not the rule, and prompt for
|
||||
confirmation.
|
||||
|
||||
## How to add a script
|
||||
|
||||
1. Start from [`bin/script_template`](script_template) — it carries
|
||||
the canonical shebang + `# purpose:` header + `Repository(...)`
|
||||
bootstrap.
|
||||
2. Add the `# purpose:` line; lowercase, terse, include a `usage:`
|
||||
example if the script takes arguments.
|
||||
3. `chmod +x bin/<name>`.
|
||||
4. The script can reach helpers via `bw.libs.<x>` exactly like a
|
||||
bundle does.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **`bin/` is not on `$PATH` by default.** Invoke as `bin/<name>` from
|
||||
the repo root, or via `direnv` if `.envrc` exposes it.
|
||||
- **Mutating scripts can hit Tier-3 territory** (per the fork's
|
||||
safety envelope). Don't run `upgrade_and_restart_all`,
|
||||
`mikrotik-firmware-updater`, or anything that does `node.run(...)`
|
||||
without explicit user instruction. See the fork's
|
||||
[`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
|
||||
for the three-tier model.
|
||||
- **Vault echo.** Scripts like `passwords-for` print decrypted values
|
||||
by design; that's allowed for the human at the terminal but *not*
|
||||
for the agent — never paste output into chat, ticket, or PR
|
||||
description.
|
||||
|
||||
## See also
|
||||
|
||||
- [`script_template`](script_template) — canonical starter.
|
||||
- [`docs/agents/conventions.md`](../docs/agents/conventions.md) —
|
||||
vault rules.
|
||||
- [`docs/agents/commands.md`](../docs/agents/commands.md) — read-only
|
||||
bw-command guidance.
|
||||
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python3
|
||||
# purpose: upgrade RouterOS and routerboard firmware on `bundle:routeros` (or any selector) — usage: mikrotik-firmware-updater [<selector>...] [--yes].
|
||||
|
||||
from argparse import ArgumentParser
|
||||
from time import sleep
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python3
|
||||
# purpose: print node.password and selected metadata-key passwords for one node — usage: passwords-for <node>.
|
||||
|
||||
from bundlewrap.repo import Repository
|
||||
from os.path import realpath, dirname
|
||||
|
|
|
|||
1
bin/rcon
1
bin/rcon
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python3
|
||||
# purpose: send an RCON command to a left4dead2 server defined in node metadata — usage: rcon (list) | rcon <server> <command>.
|
||||
|
||||
from sys import argv
|
||||
from os.path import realpath, dirname
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python3
|
||||
# purpose: starter template for new operator scripts under bin/.
|
||||
|
||||
from bundlewrap.repo import Repository
|
||||
from os.path import realpath, dirname
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python3
|
||||
# purpose: upsert one 1Password login per `bundle:routeros` node, keyed on the bw node id.
|
||||
|
||||
from bundlewrap.repo import Repository
|
||||
from os.path import realpath, dirname
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python3
|
||||
# purpose: add missing EXIF/QuickTime timestamps to photos in a directory using mdls + exiftool — usage: timestamp_icloud_photos_for_nextcloud -d <dir>.
|
||||
|
||||
from subprocess import check_output, CalledProcessError
|
||||
from datetime import datetime, timedelta
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python3
|
||||
# purpose: apt-update and full-upgrade every non-dummy debian node, then reboot in WireGuard-aware order.
|
||||
|
||||
from bundlewrap.repo import Repository
|
||||
from os.path import realpath, dirname
|
||||
|
|
|
|||
1
bin/wake
1
bin/wake
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python3
|
||||
# purpose: wake one node via WoL by name — usage: wake <node>.
|
||||
|
||||
from bundlewrap.repo import Repository
|
||||
from os.path import realpath, dirname
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python3
|
||||
# purpose: print or QR-render a WireGuard client config from htz.mails metadata — usage: wireguard-client-config <client>.
|
||||
|
||||
from bundlewrap.repo import Repository
|
||||
from os.path import realpath, dirname
|
||||
|
|
|
|||
156
bundles/AGENTS.md
Normal file
156
bundles/AGENTS.md
Normal file
|
|
@ -0,0 +1,156 @@
|
|||
# bundles/
|
||||
|
||||
## Before you start
|
||||
|
||||
Read [`docs/agents/conventions.md`](../docs/agents/conventions.md) first
|
||||
— it covers vault calls, demagify, the `repo.libs.<x>` helpers, and the
|
||||
files agents must not modify. Skipping it leads to subtly broken bundles
|
||||
(vault calls in the wrong place, dict-in-set `TypeError` because of
|
||||
unhashable nesting, etc.).
|
||||
|
||||
For bundlewrap-language reference (item types, dep keywords,
|
||||
`metadata_reactor`, `defaults`, item-file template syntax) see the fork's
|
||||
[`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
|
||||
and its [`docs/content/guide/item_file_templates.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/docs/content/guide/item_file_templates.md).
|
||||
|
||||
## What's here
|
||||
|
||||
103 bundles. Each is a directory `bundles/<name>/` containing some of:
|
||||
|
||||
```
|
||||
bundles/<name>/
|
||||
├── items.py # the items this bundle creates (files, services, packages, …)
|
||||
├── metadata.py # `defaults` + `@metadata_reactor` functions
|
||||
├── files/ # static or templated file payloads referenced from items.py
|
||||
└── README.md # one doc per bundle, for humans and agents (see "Per-bundle README" below)
|
||||
```
|
||||
|
||||
## Conventions
|
||||
|
||||
- **Bundle names** are lowercase, hyphen-separated: `backup-server`,
|
||||
`bind-acme`, `dm-crypt`. No underscores in new bundle names — see
|
||||
[`conventions.md#naming-conventions`](../docs/agents/conventions.md#naming-conventions).
|
||||
- **`items.py`** is plain Python; it produces `files = {...}`,
|
||||
`pkg_apt = {...}`, `svc_systemd = {...}`, etc. dicts at module scope.
|
||||
Cross-item dependencies use `needs` / `triggers` / `triggered_by` —
|
||||
see the fork's `AGENTS.md` for the full keyword cheat sheet.
|
||||
- **`metadata.py`** uses `defaults = {...}` for static seed values and
|
||||
`@metadata_reactor.provides(...)` for derived values. Reactors are
|
||||
pure functions of `(metadata,)` — no side effects, no I/O.
|
||||
- **Helpers go in [`libs/`](../libs/AGENTS.md)** when they're useful to
|
||||
more than one bundle. Don't duplicate logic across bundles.
|
||||
- **Custom item types** (e.g. `download:`) live in
|
||||
[`items/`](../items/AGENTS.md), not per-bundle.
|
||||
|
||||
## How to add a new bundle
|
||||
|
||||
1. `mkdir bundles/<name>/` (lowercase, hyphenated).
|
||||
2. Write `items.py` and (if anything is configurable) `metadata.py`.
|
||||
Use `repo.libs.hashable.hashable(...)` when you need to nest a dict
|
||||
or set inside a metadata set; raw dicts/sets aren't hashable.
|
||||
3. Drop static payloads into `bundles/<name>/files/`. For Mako-templated
|
||||
files, declare `'content_type': 'mako'` on the `file:` item — see
|
||||
the fork's
|
||||
[item-file-templates guide](https://github.com/CroneKorkN/bundlewrap/blob/main/docs/content/guide/item_file_templates.md).
|
||||
4. **Wire to nodes.** Either add an entry to the relevant
|
||||
[`groups/<axis>/<x>.py`](../groups/AGENTS.md) (preferred for shared
|
||||
bundles) or to the node's `bundles` list directly
|
||||
([`nodes/AGENTS.md`](../nodes/AGENTS.md)).
|
||||
5. Verify, in this order:
|
||||
- `bw test` — sanity (loaders + reactors).
|
||||
- `bw items <node>` — confirm new items appear on a node that opts in.
|
||||
- `bw hash <node>` — confirm the change is what you expected. See
|
||||
[`docs/agents/commands.md`](../docs/agents/commands.md) and the
|
||||
fork's hash-diff workflow.
|
||||
6. Add a `bundles/<name>/README.md`. See "Per-bundle README" below
|
||||
for what to cover.
|
||||
|
||||
## How to remove a bundle
|
||||
|
||||
1. `git grep '<name>'` in `nodes/`, `groups/`, and other `bundles/` to
|
||||
find references.
|
||||
2. Remove those references.
|
||||
3. `rm -rf bundles/<name>/`.
|
||||
4. `bw test` and `bw nodes` to confirm clean.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **`metadata.py` is evaluated at load time** for *every* node, every
|
||||
invocation of `bw`. Heavy work or I/O slows the whole repo. Keep
|
||||
reactors pure and fast; pre-compute in `libs/` if you must.
|
||||
- **Static files vs templates.** `bundles/<x>/files/<f>` is static
|
||||
unless the matching `file:` item declares `content_type='mako'`
|
||||
(or a templating extension triggers it). To check, read the matching
|
||||
`file:` entry in `items.py`.
|
||||
- **Reactors writing across namespaces.** Some bundles' reactors write
|
||||
into other bundles' metadata namespaces (e.g. `nextcloud` writes
|
||||
into `apt.packages`, `archive.paths`). When you change such a bundle,
|
||||
every consumer's metadata changes too. The bundle's `README.md`
|
||||
often calls these out — but the authoritative source is `metadata.py`
|
||||
itself; grep `'<other-bundle>':` in the reactors when in doubt.
|
||||
- **`bw hash` doesn't accept selectors.** Use `bw hash <node>` per
|
||||
literal name; see the fork's runbook.
|
||||
|
||||
## Per-bundle README
|
||||
|
||||
Each bundle has (or should have) a `README.md`. One doc per bundle,
|
||||
written for humans and agents both. There's no fixed structure —
|
||||
match the bundle's actual surface, write what helps a future reader
|
||||
(or future you) avoid trial-and-error.
|
||||
|
||||
The existing READMEs vary in quality and shape. For orientation,
|
||||
look at the bigger ones, not the two-line ones:
|
||||
|
||||
- [`bundles/flask/README.md`](flask/README.md) — title + one-sentence
|
||||
purpose, a metadata example as a Python dict, then the contract
|
||||
the consuming git repo has to satisfy + a logging pitfall. The
|
||||
closest thing to a "balanced doc" in tree.
|
||||
- [`bundles/dm-crypt/README.md`](dm-crypt/README.md) — same shape,
|
||||
shorter: purpose + metadata example + one sentence on effect.
|
||||
- [`bundles/apt/README.md`](apt/README.md) — relevant upstream URLs
|
||||
at the top, then a Python metadata example with rich inline
|
||||
comments (type / optionality / where keys come from).
|
||||
- [`bundles/nextcloud/README.md`](nextcloud/README.md) — operational
|
||||
scratchpad: iPhone-import recipe, preview-generator commands,
|
||||
reset queries. Captures muscle-memory the maintainer would
|
||||
otherwise re-learn each time.
|
||||
|
||||
Useful things to include, when relevant:
|
||||
|
||||
- A sentence or two on what the bundle does and when you'd attach it.
|
||||
- A metadata example as a Python dict literal, with `#` comments
|
||||
on each key (type, required vs default, units, where it comes
|
||||
from). This is the cleanest way to communicate the schema and
|
||||
matches how `metadata.py` actually looks.
|
||||
- Anything non-obvious about wiring it up — required keys without
|
||||
defaults, group-membership expectations, manual one-time steps.
|
||||
- Cross-namespace metadata writes, when this bundle's reactors
|
||||
populate another bundle's namespace. Easy to miss, cheap to flag.
|
||||
- Gotchas, debug recipes, failure modes you've actually hit.
|
||||
|
||||
What to skip:
|
||||
|
||||
- An exhaustive item list — `items.py` is shorter and more accurate.
|
||||
- Anything that would just rot — version numbers, "TODO" lists,
|
||||
change notes. Use git history.
|
||||
|
||||
If a single paragraph is enough to say what's worth saying, write a
|
||||
single paragraph. Verbosity isn't a goal.
|
||||
|
||||
Convention going forward is leave-as-you-go: any time you materially
|
||||
edit a bundle, top up its README (or write one if it's missing).
|
||||
Don't burn a session bulk-reformatting the existing ones — uneven
|
||||
quality is part of what we accept in exchange for not blocking other
|
||||
work.
|
||||
|
||||
## See also
|
||||
|
||||
- [`docs/agents/conventions.md`](../docs/agents/conventions.md) — repo
|
||||
idioms (vault, demagify, naming, do-not-touch list).
|
||||
- [`docs/agents/commands.md`](../docs/agents/commands.md) — repo-specific
|
||||
command deltas.
|
||||
- [`items/AGENTS.md`](../items/AGENTS.md) — custom item types
|
||||
(`download:`); when to write a new one vs use `file:`.
|
||||
- [`libs/AGENTS.md`](../libs/AGENTS.md) — shared helpers.
|
||||
- Fork's [`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
|
||||
— bundlewrap-language reference + safety envelope.
|
||||
63
data/AGENTS.md
Normal file
63
data/AGENTS.md
Normal file
|
|
@ -0,0 +1,63 @@
|
|||
# data/
|
||||
|
||||
## What's here
|
||||
|
||||
Out-of-bundle data assets consumed by one or more bundles. Each subdir
|
||||
maps to a consumer:
|
||||
|
||||
```
|
||||
data/
|
||||
├── apt/keys/ # binary GPG keys for apt sources
|
||||
├── grafana/rows/ # Mako-templated dashboard panels (Python)
|
||||
├── nginx/ # nginx snippets shared across vhosts
|
||||
├── homeassistant/, mailman/, nextcloud/, ...
|
||||
└── network.py # repo-wide network metadata (one-off file)
|
||||
```
|
||||
|
||||
## Two distinct content models
|
||||
|
||||
Same directory shape, different content kinds. When you add a new
|
||||
`data/<x>/` subdir, declare which model it follows:
|
||||
|
||||
| Model | Example | Consumer |
|
||||
|---|---|---|
|
||||
| **Binary / static** | `data/apt/keys/*.{asc,gpg}` | `bundles/apt` reads files at apply time |
|
||||
| **Python module / template** | `data/grafana/rows/*.py`, `data/routeros-monitoring/*.py` | bundle `import`s and renders |
|
||||
|
||||
If a data asset is read by **exactly one bundle**, prefer
|
||||
`bundles/<x>/files/` instead of `data/<x>/`. `data/` is for
|
||||
shared / multi-consumer artifacts. Single-instance evidence: commit
|
||||
`78a8abc` moved `mikrotik.mib` *out* of `data/` *into* the bundle for
|
||||
this reason.
|
||||
|
||||
## Conventions
|
||||
|
||||
- **One subdir per consumer.** Subdir name = consumer bundle name
|
||||
(`data/apt/`, `data/nextcloud/`).
|
||||
- **`network.py` exception.** A single file at `data/network.py` holds
|
||||
repo-wide network metadata; it doesn't belong to one bundle. Treat
|
||||
it as cross-cutting infrastructure metadata.
|
||||
|
||||
## How to add data
|
||||
|
||||
1. Decide the content model (binary or Python).
|
||||
2. `mkdir data/<consumer>/`.
|
||||
3. Drop assets in.
|
||||
4. The consumer bundle (`bundles/<consumer>/items.py` or
|
||||
`metadata.py`) reads them via `repo.path` + `os.path.join` or
|
||||
similar.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **Apt keys** trigger an offline-verify rule before they're committed.
|
||||
See [`commands.md#apt-key-changes-need-offline-verification`](../docs/agents/commands.md#apt-key-changes-need-offline-verification).
|
||||
- **Mako-templated Python data** evaluates at bundle render time. Side
|
||||
effects in those modules slow the whole repo (same caveat as
|
||||
[`libs/`](../libs/AGENTS.md)).
|
||||
|
||||
## See also
|
||||
|
||||
- [`bundles/AGENTS.md`](../bundles/AGENTS.md) — when bundle `files/`
|
||||
beats `data/`.
|
||||
- [`docs/agents/commands.md`](../docs/agents/commands.md) — apt-key
|
||||
verification rule.
|
||||
50
docs/agents/commands.md
Normal file
50
docs/agents/commands.md
Normal file
|
|
@ -0,0 +1,50 @@
|
|||
# Commands
|
||||
|
||||
The canonical bw-command runbook — read-only allowlist, three-tier
|
||||
safety envelope, after-change table, hash-diff workflow, `bw debug`
|
||||
sketch — lives in the fork's
|
||||
[`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md).
|
||||
Read that first.
|
||||
|
||||
This file collects only the deltas specific to `ckn-bw`.
|
||||
|
||||
## Apt-key changes need offline verification
|
||||
|
||||
Editing files under `data/apt/keys/*.{asc,gpg}` rotates a signing key
|
||||
the whole apt subsystem trusts. Trial-and-error with `bw apply` is the
|
||||
*failure* path: a wrong key blocks unattended upgrades cluster-wide
|
||||
until corrected manually.
|
||||
|
||||
Before touching `data/apt/keys/`:
|
||||
|
||||
1. Fetch the new key from its upstream source (project release page,
|
||||
`keys.openpgp.org`, etc.).
|
||||
2. `gpg --show-keys <newkey>` — print the fingerprint.
|
||||
3. Diff against the fingerprint published by the upstream source.
|
||||
4. Only after the fingerprint matches, place the file under
|
||||
`data/apt/keys/` and let `bundles/apt` consume it on the next
|
||||
apply.
|
||||
|
||||
## `*.py_` suspended nodes are invisible to `bw nodes`
|
||||
|
||||
The repo loader (`nodes.py`) only matches files ending in `.py`. Files
|
||||
ending in `.py_` are silently skipped. If `bw nodes` reports a node
|
||||
missing, check whether its file has been parked:
|
||||
|
||||
```sh
|
||||
ls nodes/ | grep '\.py_$'
|
||||
```
|
||||
|
||||
This is the [suspension idiom](conventions.md#suspension-and-soft-delete-idioms),
|
||||
not a bug.
|
||||
|
||||
## Vault output never leaves the terminal
|
||||
|
||||
The fork's runbook calls out that `bw debug` resolves vault magic
|
||||
strings transparently. In `ckn-bw` specifically: never echo, log, or
|
||||
paste decrypted values, even from a `bw debug -c` one-liner. If you
|
||||
need to verify a secret resolved correctly, hash or fingerprint it
|
||||
instead.
|
||||
|
||||
See [`conventions.md#secrets`](conventions.md#secrets) for the
|
||||
demagify magic-string list and the rule's full rationale.
|
||||
146
docs/agents/conventions.md
Normal file
146
docs/agents/conventions.md
Normal file
|
|
@ -0,0 +1,146 @@
|
|||
# Conventions
|
||||
|
||||
Repo-specific idioms an agent has to know before editing this BundleWrap
|
||||
config. For bundlewrap-the-language reference (item types, dep keywords,
|
||||
metadata-reactor semantics, after-change runbook), see the fork's
|
||||
[`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md).
|
||||
|
||||
## Secrets
|
||||
|
||||
Secrets live in `.secrets.cfg` and are referenced from node files by
|
||||
"demagify" magic strings. The loader (`nodes.py`) walks node dicts and
|
||||
resolves any leaf string of the form `!<verb>:<arg>`.
|
||||
|
||||
| Magic string | Resolves to |
|
||||
|---|---|
|
||||
| `!password_for:<id>` | `repo.vault.password_for(id)` |
|
||||
| `!decrypt:<ciphertext>` | `repo.vault.decrypt(ciphertext)` |
|
||||
| `!decrypt_file:<path>` | `repo.vault.decrypt_file(path)` |
|
||||
| `!32_random_bytes_as_base64_for:<id>` | `repo.vault.random_bytes_as_base64_for(id, length=32)` |
|
||||
|
||||
Magic strings are only resolved inside **node files** (everything under
|
||||
`nodes/`). They are *not* resolved in group files, bundle metadata
|
||||
defaults, or item attributes — call `repo.vault.<verb>(...)` directly
|
||||
there.
|
||||
|
||||
**Never echo decrypted values.** Don't print, log, or paste them, even
|
||||
in `bw debug` exploration. If you need a sanity check, hash or
|
||||
fingerprint instead. This applies to AI agents and humans equally.
|
||||
|
||||
## Bundlewrap version
|
||||
|
||||
The venv runs **editable** from the maintainer's fork:
|
||||
|
||||
```
|
||||
-e git+https://github.com/CroneKorkN/bundlewrap.git@main#egg=bundlewrap
|
||||
```
|
||||
|
||||
Pinned in `requirements.txt`. The fork's `main` tracks upstream
|
||||
`bundlewrap/bundlewrap` master; the fork's [`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
|
||||
is the canonical agent-oriented bundlewrap-language reference.
|
||||
|
||||
To pull upstream changes into the venv:
|
||||
|
||||
```sh
|
||||
(cd .venv/src/bundlewrap && git pull)
|
||||
```
|
||||
|
||||
## Eval-loaded node and group files
|
||||
|
||||
`nodes/*.py` and `groups/*.py` are loaded by `eval()` in `nodes.py` /
|
||||
`groups.py` (top of the repo). Each file must be a single Python
|
||||
expression — typically a dict literal.
|
||||
|
||||
Consequences:
|
||||
|
||||
- **No top-level imports.** No `from foo import bar`, no `import os`.
|
||||
Helpers go in `libs/` and are reached via `repo.libs.<x>` from
|
||||
bundle code, not from node files.
|
||||
- **No statements.** No `if`/`for`/`def` at the top level. Use
|
||||
expressions (ternaries, comprehensions) instead.
|
||||
- **Silent drop on parse error.** Vanilla bundlewrap silently omits a
|
||||
node/group whose file fails to eval. The maintainer patched
|
||||
`groups.py` (commit `dc40295`) to print the error instead — but it
|
||||
still skips the file. Symptom: `bw nodes` lists fewer nodes than you
|
||||
expect. Cure: re-read the file and check for accidental imports or
|
||||
syntax errors.
|
||||
|
||||
## Group inheritance order
|
||||
|
||||
Metadata merges along this chain (from the fork's docs):
|
||||
|
||||
```
|
||||
all → location → os → machine → applications → node
|
||||
```
|
||||
|
||||
`groups/all.py` is the universal base; `groups/{locations,os,machine,applications}/`
|
||||
hold the per-axis groups; the node's own metadata wins last.
|
||||
|
||||
## Naming conventions
|
||||
|
||||
| Where | Convention | Example |
|
||||
|---|---|---|
|
||||
| `nodes/*.py` | `<location>.<role>.py` | `home.server.py`, `htz.mails.py` |
|
||||
| `groups/*.py` | one file per group; subdir = purpose | `groups/applications/nextcloud.py`, `groups/os/debian-13.py` |
|
||||
| `bundles/<name>` | lowercase, hyphen-separated | `backup-server`, `bind-acme`, `routeros-monitoring` |
|
||||
| Custom items | `items/<type>.py` | `items/download.py` |
|
||||
|
||||
Underscores in bundle names appear only in `_old`-suffixed leftovers
|
||||
(see below); don't introduce new ones.
|
||||
|
||||
## Suspension and soft-delete idioms
|
||||
|
||||
These conventions look like dead code; they aren't. Don't clean them up.
|
||||
|
||||
- **`*.py_` parked node.** A node file whose name ends in `.py_` is
|
||||
silently excluded from `bw nodes` (the loader only matches `*.py`).
|
||||
Used to keep decommissioned-but-not-deleted node configs in tree.
|
||||
Example: `nodes/htz.l4d2.py_`.
|
||||
- **"for now / disable / dummy / offline" markers.** Commits or comments
|
||||
containing these phrases mark deliberate suspensions, not bugs. Check
|
||||
`git log -- <file>` before re-enabling — the maintainer reverses
|
||||
these manually when the underlying condition resolves.
|
||||
- **`_old` / `_old2` directories.** Recovery buffers during big
|
||||
refactors. Don't delete them without asking, even if they look
|
||||
orphaned.
|
||||
|
||||
## Files agents must not modify
|
||||
|
||||
| Path | Why |
|
||||
|---|---|
|
||||
| `.secrets.cfg*` | vault key material |
|
||||
| `.venv/` | local Python environment |
|
||||
| `.cache/` | bw runtime cache |
|
||||
| `.bw_debug_history` | shell history for `bw debug` |
|
||||
| `.envrc` | direnv local-environment config |
|
||||
| `/.cocoindex_code/` | local code index, not in git |
|
||||
| `README.md` (root) | maintainer's personal TODO scratchpad — not project docs |
|
||||
|
||||
Treat `hooks/` and `items/` (custom item types) with extra care: they
|
||||
affect `bw`'s behavior for the whole repo, not a single bundle.
|
||||
|
||||
## Working style
|
||||
|
||||
- **Iterative commits are normal.** The maintainer commits in small
|
||||
checkpoints (`+`, `fix`, `whitespace`, terse one-liners). Don't
|
||||
rebase WIP branches without asking. As an agent, prefer
|
||||
complete-feeling commits over mimicking the iterative style.
|
||||
- **Burst-state awareness.** Before writing into a subsystem, run
|
||||
`git log --since='1 month ago' bundles/<x>` (or `nodes/<x>.py`).
|
||||
≥10 recent commits means the subsystem is in flux; read the most
|
||||
recent diffs first — your assumptions about its metadata shape may
|
||||
already be stale.
|
||||
- **Branch names.** PRs go through self-hosted Gitea/Forgejo. Branch
|
||||
names are lowercase snake_case, descriptive
|
||||
(`debian-13`, `htz.mails_debian_13_squash`, `l4d2_the_next`).
|
||||
|
||||
## Bundlewrap-version migration recipe
|
||||
|
||||
When the next bw major lands:
|
||||
|
||||
1. Read the upstream migration guide.
|
||||
2. `git grep` for affected reactor / item patterns.
|
||||
3. Rewrite each (one commit per pattern is fine — see Working style).
|
||||
4. Bump `requirements.txt` last.
|
||||
|
||||
Pattern from commit `186d503` (bw 4 → 5).
|
||||
|
|
@ -0,0 +1,59 @@
|
|||
# Handoff — agent-friendly docs for ckn-bw (status: complete)
|
||||
|
||||
The 2026-05-10 work is finished. This file replaces the original
|
||||
multi-page handoff (which described work that's now done plus a
|
||||
Phase 2 that was dropped). Kept as a breadcrumb so future agents
|
||||
know what landed.
|
||||
|
||||
## What landed
|
||||
|
||||
- **Phase 0** (`730625e`) — top-of-file docstrings on every
|
||||
`libs/*.py` and `hooks/*.py`; `# purpose:` headers on every
|
||||
`bin/*` script. Discovery is `ls + head -1`.
|
||||
- **Phase 1** (`04558a9`) — root `AGENTS.md` (with `CLAUDE.md`
|
||||
symlink), `docs/agents/{conventions,commands}.md`, per-area
|
||||
`AGENTS.md` for `bundles/`, `nodes/`, `groups/`, `libs/`,
|
||||
`hooks/`, `data/`, `items/`, `bin/`. Mechanism-focused; no
|
||||
enumeration of contents.
|
||||
- **Convention pivot** (`9e1bb2a`) — per-bundle docs are plain
|
||||
`README.md`, not `AGENTS.md`. No template. `bundles/AGENTS.md`
|
||||
"Per-bundle README" gives loose orientation pointing at the
|
||||
more substantial existing READMEs.
|
||||
|
||||
`bw test` exit 0 throughout. All internal links in the new docs
|
||||
resolve. Commits sit on local `master`, none pushed.
|
||||
|
||||
## Where current truth lives
|
||||
|
||||
- Root entry: [`AGENTS.md`](../../../AGENTS.md) (symlinked to
|
||||
`CLAUDE.md`).
|
||||
- Repo conventions + bw command deltas:
|
||||
[`docs/agents/`](../../agents/).
|
||||
- Per-area docs: `<dir>/AGENTS.md` for each top-level dir.
|
||||
- Per-bundle docs: existing `README.md` files (~33 of them);
|
||||
guidance in [`bundles/AGENTS.md`](../../../bundles/AGENTS.md).
|
||||
- Self-describing files: `libs/*.py`, `hooks/*.py`, `bin/*`.
|
||||
|
||||
## Source documents
|
||||
|
||||
- Spec: [`../specs/2026-05-10-agent-friendliness-design.md`](../specs/2026-05-10-agent-friendliness-design.md)
|
||||
— read its `§0. Revisions` first; §3 and §7 are now pre-pivot
|
||||
intent only.
|
||||
- User-stories validation: [`../specs/2026-05-10-user-stories-from-history.md`](../specs/2026-05-10-user-stories-from-history.md).
|
||||
- Implementation plan: [`../plans/2026-05-10-agent-friendliness-plan.md`](../plans/2026-05-10-agent-friendliness-plan.md)
|
||||
(frozen pre-pivot artifact; see its top-of-file note).
|
||||
|
||||
## What's open
|
||||
|
||||
Nothing planned. Per-bundle README updates are reactive — when a
|
||||
bundle gets materially edited, top up its `README.md` then. The
|
||||
~33 existing READMEs vary in quality and shape; that's accepted.
|
||||
|
||||
Two minor items the maintainer may want to clean up at some point:
|
||||
|
||||
- The implementation plan and the spec's §3 / §7 still describe
|
||||
the pre-pivot per-bundle `AGENTS.md` convention. Spec's §0 flags
|
||||
this; the plan is a frozen artifact and probably stays as-is.
|
||||
- The original handoff's verification list mentioned `bw bundles`,
|
||||
which isn't a real bw subcommand. Not user-facing in any current
|
||||
doc.
|
||||
505
docs/superpowers/plans/2026-05-10-agent-friendliness-plan.md
Normal file
505
docs/superpowers/plans/2026-05-10-agent-friendliness-plan.md
Normal file
|
|
@ -0,0 +1,505 @@
|
|||
# Implementation plan — agent-friendliness docs
|
||||
|
||||
> **Note (post-execution):** This plan is a frozen pre-pivot artifact.
|
||||
> It captures the approach as designed before Phase 1 landed and the
|
||||
> per-bundle convention pivoted from `AGENTS.md` to `README.md`.
|
||||
> Sections describing per-bundle `AGENTS.md`,
|
||||
> `bundles/AGENTS.template.md`, and the Phase-2 seed-bundle migration
|
||||
> reflect the original intent, not what shipped. For the current
|
||||
> shape, read the spec's §0 Revisions
|
||||
> ([`../specs/2026-05-10-agent-friendliness-design.md`](../specs/2026-05-10-agent-friendliness-design.md))
|
||||
> and the handoff status note
|
||||
> ([`../handoffs/2026-05-10-implementation-handoff.md`](../handoffs/2026-05-10-implementation-handoff.md)).
|
||||
> Kept in tree as a record of how the work was scoped and what
|
||||
> validation findings (workflow + user-story) shaped Phase 1.
|
||||
|
||||
## Context
|
||||
|
||||
This BundleWrap config repo (`ckn-bw`, ~22 nodes / 103 bundles) currently has
|
||||
no agent-facing orientation: no `CLAUDE.md` / `AGENTS.md`, only ~10 of 103
|
||||
bundles have a `README`, and the root `README.md` is a personal TODO. Agents
|
||||
landing cold spelunk to figure out conventions (demagify magic strings,
|
||||
`metadata_reactor` patterns, lib helpers, what's safe to run).
|
||||
|
||||
Goal: ship one PR that scaffolds the documentation surface described in
|
||||
`docs/superpowers/specs/2026-05-10-agent-friendliness-design.md`, plus a
|
||||
second PR seeding 10 high-value bundle docs. After this work, an agent can
|
||||
land useful work in nodes / groups / bundles / libs without trial-and-error,
|
||||
and never invokes a state-mutating `bw` command without explicit user request.
|
||||
|
||||
**Corrections since the spec was written.**
|
||||
|
||||
- Verification on 2026-05-10 found the active venv ran upstream `bundlewrap 4.24.0`,
|
||||
the README's fork install was stale, and the user has since upgraded the
|
||||
venv to PyPI `bundlewrap 5.0.3`.
|
||||
- The user refreshed `/Users/mwiegand/Projekte/bundlewrap-fork` to track upstream
|
||||
master at commit `a97cdb13` (also version 5.0.3), and decided to make the
|
||||
fork agent-friendly *first* (separate session, separate plan), then return
|
||||
here. The ckn-bw plan therefore drops the `docs/agents/bundlewrap/` folder
|
||||
it originally planned — bundlewrap-language docs live in the fork now.
|
||||
|
||||
This plan reflects the reduced ckn-bw scope. It is gated on the fork's
|
||||
`AGENTS.md` existing, since this repo's root `AGENTS.md` links out to it.
|
||||
See the handoff document delivered separately for the fork-session work.
|
||||
|
||||
## Scope
|
||||
|
||||
Two PRs.
|
||||
|
||||
- **PR1 — scaffolding.** Root `AGENTS.md`, `CLAUDE.md` symlink, the
|
||||
`docs/agents/` tree (just `conventions.md` and `commands.md` — no
|
||||
`bundlewrap/` folder; that lives in the fork), all eight per-area
|
||||
`AGENTS.md` files, the per-bundle template, the docstring/header pass on
|
||||
`libs/*.py` / `hooks/*.py` / `bin/*`, and the README cleanup (remove the
|
||||
stale fork section, keep everything else). Spec correction in the same PR.
|
||||
- **PR2 — seed bundles.** Per-bundle `AGENTS.md` for the 10 seed bundles;
|
||||
fold and remove any existing per-bundle `README.md` in those.
|
||||
|
||||
Phase 3 from the spec ("leave-as-you-go") is a convention, not a code task —
|
||||
captured in `bundles/AGENTS.md` as a contribution rule.
|
||||
|
||||
## Approach (recommended)
|
||||
|
||||
### PR1 — scaffolding
|
||||
|
||||
Order is dependency-respecting; each step's output is referenced by later
|
||||
steps' link targets.
|
||||
|
||||
**Precondition:** the fork's root `AGENTS.md` exists at
|
||||
`/Users/mwiegand/Projekte/bundlewrap-fork/AGENTS.md`. PR1 links out to it,
|
||||
so PR1 cannot land cleanly until the fork session has produced that file.
|
||||
|
||||
1. **Spec correction.** In
|
||||
`docs/superpowers/specs/2026-05-10-agent-friendliness-design.md`:
|
||||
- Replace "fork" framing with the actual reality: venv runs `bundlewrap
|
||||
5.0.3` (PyPI install today), the user maintains a separate fork at
|
||||
`github.com/CroneKorkN/bundlewrap` whose `master` tracks upstream and
|
||||
whose `AGENTS.md` is the canonical bundlewrap-language reference.
|
||||
- Replace the `docs/agents/bundlewrap/` folder content (Section 2 IA,
|
||||
Section 6) with a single bullet: "bundlewrap-language reference lives
|
||||
in the fork at `<URL>/AGENTS.md`; ckn-bw links to it from root
|
||||
`AGENTS.md` and `conventions.md`."
|
||||
- Update Section 4 quickstart and Section 6 `conventions.md` accordingly.
|
||||
- Single commit before any other ckn-bw docs are written.
|
||||
|
||||
2. **`docs/agents/conventions.md`** (~80–120 lines): demagify magic strings;
|
||||
bundlewrap version note (`5.0.3`, link to the fork's `AGENTS.md` for
|
||||
language reference); group inheritance order; node/group naming
|
||||
conventions; the `eval()` idiom in `nodes.py` / `groups.py` and what
|
||||
that constrains for editors/agents (no top-level imports, etc.);
|
||||
do-not-touch file list (`.secrets.cfg*`, `.venv`, `.cache`,
|
||||
`.bw_debug_history`, `.envrc`).
|
||||
|
||||
3. **`docs/agents/commands.md`** (~30–50 lines, slimmed). The fork's
|
||||
`AGENTS.md` (at `https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md`)
|
||||
now carries the canonical bw runbook — tiers, after-change table,
|
||||
hash-diff workflow, `bw debug` sketch — verified against 5.0.3 source.
|
||||
This file shrinks to ckn-bw-specific deltas:
|
||||
- One-line lead pointing at the fork's `AGENTS.md` for the canonical
|
||||
read-only allowlist + after-change runbook.
|
||||
- **Apt-key after-change row** (S7 finding): editing
|
||||
`data/apt/keys/*.{asc,gpg}` → first verify with
|
||||
`gpg --show-keys <newkey>` locally + fingerprint diff against the
|
||||
expected source. Trial via `bw apply` is the *failure* path (a wrong
|
||||
key blocks unattended upgrades cluster-wide). Not in the fork's
|
||||
runbook because it's repo-specific.
|
||||
- **`*.py_` suspended-node interaction with `bw nodes`**: a node file
|
||||
ending in `.py_` is silently excluded from the loader; `bw nodes`
|
||||
won't list it. Document this so an agent doesn't think a node is
|
||||
missing when it's actually parked.
|
||||
- **Vault magic-string handling**: never echo decrypted output, even
|
||||
in `bw debug` exploration. Cross-link to `conventions.md#secrets`.
|
||||
|
||||
4. **Per-area `AGENTS.md`** — eight files, mechanism-focused, no
|
||||
enumeration. Order: `bundles/`, `nodes/`, `groups/`, `libs/`, `hooks/`,
|
||||
`data/`, `items/`, `bin/`. Each ~30–80 lines, same five-section shape
|
||||
from spec Section 5.
|
||||
|
||||
5. **`bundles/AGENTS.template.md`** — the per-bundle template (spec
|
||||
Section 3 verbatim, with placeholder text in each section).
|
||||
|
||||
6. **Root `AGENTS.md`** — written so all link targets exist. Spec
|
||||
Section 4, with the "Conventions you must know" first bullet now
|
||||
pointing at the fork's `AGENTS.md` (canonical bundlewrap language)
|
||||
instead of an internal `bundlewrap/` folder. Quickstart bullet about
|
||||
the fork updated to: "bundlewrap reference lives in `<fork-URL>` —
|
||||
read first if new to bundlewrap." Then create `CLAUDE.md` as a symlink:
|
||||
`ln -s AGENTS.md CLAUDE.md`.
|
||||
|
||||
7. **Docstring/header pass** — for every `libs/*.py` and `hooks/*.py`
|
||||
without a top-of-file module docstring, add a one-liner. For every
|
||||
`bin/*` script without a header comment, add a `# purpose: …` line. Do
|
||||
not touch files that already have one. Mechanical: read each file's
|
||||
first ~10 lines, decide, edit only if missing.
|
||||
|
||||
8. **Root `README.md` cleanup** — remove the stale "install bw fork"
|
||||
section (the `pip3 install --editable git+file:///…/bundlewrap-fork…`
|
||||
block) since the venv runs PyPI 5.0.3. Leave the rest of the README
|
||||
untouched.
|
||||
|
||||
### PR2 — seed bundles
|
||||
|
||||
For each of the 10 seed bundles, write `bundles/<name>/AGENTS.md` from
|
||||
`bundles/AGENTS.template.md`. Where a bundle already has `README.md`, fold
|
||||
its content into the new `AGENTS.md` and remove the old `README.md` in the
|
||||
same commit.
|
||||
|
||||
Seed list (from spec Section 7, validated empirically 2026-05-10):
|
||||
|
||||
| # | Bundle | Existing README? | Notes |
|
||||
|---|---|---|---|
|
||||
| 1 | `monitored` | check | meta-bundle, often misunderstood |
|
||||
| 2 | `postgresql` | check | foundational |
|
||||
| 3 | `wireguard` | check | own lib + bin script |
|
||||
| 4 | `routeros-monitoring` | no | most-churned bundle (15 commits / 18mo); replaces `php` per user-story rebalance |
|
||||
| 5 | `apt` | check | own lib |
|
||||
| 6 | `nginx` | check | web foundational |
|
||||
| 7 | `telegraf` | check | high cross-bundle ripple |
|
||||
| 8 | `backup` | check | cross-node coordination |
|
||||
| 9 | `letsencrypt` | check | cross-cutting |
|
||||
| 10 | `nextcloud` | yes (verified) | complex, recent activity |
|
||||
|
||||
For each: derive `Metadata` dict by reading the bundle's `metadata.py` and
|
||||
running `bw metadata <one-node-with-the-bundle>` to confirm the resolved
|
||||
shape. Derive `Produces` from `items.py`. Derive `Depends on` by checking
|
||||
which other bundles' artifacts (apt packages, systemd services) the
|
||||
bundle's reactors and items reference. Use the ccc index (built 2026-05-10)
|
||||
for fast cross-bundle lookups when filling `Depends on`.
|
||||
|
||||
## Workflow validation findings (2026-05-10) — content additions
|
||||
|
||||
Traced the "implement a new bundle" workflow against the planned docs.
|
||||
Natural agent path is root → `bundles/AGENTS.md` → example bundle → write
|
||||
files → wire to a node → verify. Seven gaps found; six are one-line
|
||||
additions to `bundles/AGENTS.md`, one is a rewrite of root §6's example
|
||||
pointers (~8 lines). All low-cost; fold into the corresponding files when
|
||||
written in PR1.
|
||||
|
||||
1. **`bundles/AGENTS.md` — "Before you start" header.** `conventions.md`
|
||||
is off the natural path; call it out as required reading at the top,
|
||||
not just in "see also." This repo's idioms (vault calls in reactors,
|
||||
`repo.libs.hashable.hashable(...)`, demagify) live there. An agent
|
||||
who skips it will write subtly wrong code (e.g. dict-in-set
|
||||
`TypeError`, vault calls in the wrong place).
|
||||
|
||||
2. **Root `AGENTS.md` §6 — use-case keyed example pointers.** Replace the
|
||||
spec's "small bundle / complex bundle / node file" with one-line
|
||||
pointers per pattern: vault usage, templated files, cross-bundle
|
||||
reactor writing, `download` custom item. Pick concrete bundles at write
|
||||
time (grep for the patterns to find good exemplars). ~8 lines.
|
||||
|
||||
3. **`bundles/AGENTS.md` "How to add" — explicit wiring step.** Add a
|
||||
numbered step: "(4) Wire to nodes — see `groups/AGENTS.md` for
|
||||
application-style group wiring or `nodes/AGENTS.md` for direct
|
||||
attachment via a node's `bundles` list." Currently the wiring step is
|
||||
implicit; agent has to discover by cross-reading two area docs.
|
||||
|
||||
4. **`bundles/AGENTS.md` Conventions — bundle naming.** One line:
|
||||
bundle directory names are lowercase with hyphens (e.g. `backup-server`,
|
||||
`bind-acme`, `dm-crypt`); avoid underscores. Verify the convention by
|
||||
`ls bundles | grep _` before writing — if the convention is mixed,
|
||||
document the actual rule.
|
||||
|
||||
5. **`bundles/AGENTS.md` "See also" — items and templates.** Cross-link
|
||||
to `items/AGENTS.md` (for the `download` custom item and how to write
|
||||
new custom item types) and to the fork's
|
||||
`docs/content/guide/item_file_templates.md` (template syntax). Both
|
||||
are common needs an agent writing a new bundle hits early.
|
||||
|
||||
6. **`bundles/AGENTS.md` — first-thing-to-run after writing.** One-liner
|
||||
pointing at `commands.md` with the canonical sequence: `bw test`
|
||||
(sanity) → `bw items <node>` (do items show up?) → `bw hash <node>`
|
||||
(changed as expected?). Saves the agent from hunting the runbook.
|
||||
|
||||
7. **`bundles/AGENTS.template.md` — empty-section guidance.** Add a note
|
||||
at the top of the template: "For a brand-new bundle without consumers
|
||||
yet, leave `Depends on` and `Produces` empty or marked TBD; fill in
|
||||
after the first verify run."
|
||||
|
||||
## User-story validation findings (2026-05-10) — additional content adds
|
||||
|
||||
Empirical user-story extraction from 1169 commits (full history, with
|
||||
detailed analysis of the last 222 commits / 18 months) is at
|
||||
`docs/superpowers/specs/2026-05-10-user-stories-from-history.md`. It
|
||||
identified 21 recurring user stories. Coverage assessment vs the planned
|
||||
docs: 5 ✅ / 13 ⚠ / 3 ❌. Below are the 16 additional content adds, grouped
|
||||
by target file. Each is one paragraph or less. The ❌ items shape an
|
||||
agent's *judgment* (highest value); the ⚠ items shape lookups.
|
||||
|
||||
### Root `AGENTS.md`
|
||||
|
||||
- **F9 — Personal TODO callout.** "Note: `README.md` is the maintainer's
|
||||
personal scratchpad, not project documentation. Onboarding lives here in
|
||||
`AGENTS.md`." One sentence in §1 ("What this repo is").
|
||||
|
||||
### `docs/agents/conventions.md`
|
||||
|
||||
- **S4 ❌ — Iterative-commit workflow style.** "User commits are
|
||||
iterative checkpoints, not landing-ready snapshots. Terse messages
|
||||
(`+`, `fix`, `whitespace`, `dowsnt exist`) and successive 'fix' commits
|
||||
on the same file are normal. Don't rebase WIP without asking. As an
|
||||
agent, prefer to land complete-feeling commits rather than mimic the
|
||||
iterative style."
|
||||
- **S5 ❌ — Burst-state awareness.** "Before writing into a subsystem,
|
||||
check `git log --since='1 month ago' bundles/<x>` (or `nodes/<x>.py`,
|
||||
etc.). If it shows ≥10 recent commits, the subsystem is in flux and
|
||||
your assumptions about its metadata shape may already be stale. Read
|
||||
the most recent diffs first."
|
||||
- **S11 ❌ — Suspension idiom ("for now / disable / dummy / offline").**
|
||||
"Commits with these markers indicate deliberate suspension, not bugs.
|
||||
If you encounter a stub or commented-out block, check
|
||||
`git log -- <file>` for the suspension reason before re-enabling. The
|
||||
user reverses these manually when ready."
|
||||
- **F6 — `_old` / `_old2` soft-delete pattern.** "Suffixed-with-`_old`
|
||||
directories are the user's recovery buffer during big refactors. Don't
|
||||
delete them without asking, even if they look orphaned."
|
||||
- **F8 — Branch naming for PRs.** "PRs go through self-hosted
|
||||
Gitea/Forgejo. Branch names are lowercase-snake_case descriptive
|
||||
(`debian-13`, `htz.mails_debian_13_squash`, `l4d2_the_next`)."
|
||||
- **S20 — Bundlewrap version-migration recipe (optional).** "When the
|
||||
next major bw version lands: read upstream migration guide → grep for
|
||||
affected reactor patterns → rewrite each → bump `requirements.txt`
|
||||
last." Captures the pattern from `186d503` (bw 4 → 5). Useful given
|
||||
the maintainer's tool-design pivot.
|
||||
|
||||
### `docs/agents/commands.md`
|
||||
|
||||
- **S7 — Apt-keys verification.** Add a row to the after-change table:
|
||||
`data/apt/keys/*.{asc,gpg}` → first check is `gpg --show-keys
|
||||
<newkey>` locally + visual diff against expected fingerprint. Trial
|
||||
via `bw apply` is the *failure* path (a wrong key blocks unattended
|
||||
upgrades cluster-wide).
|
||||
- ~~**S21 — `bw debug` content sketch.**~~ Resolved upstream: the fork's
|
||||
`AGENTS.md` now carries the canonical `bw debug` content sketch (probes
|
||||
for `repo.get_node(...).metadata`, `repo.libs.<x>`, `repo.path`). No
|
||||
ckn-bw addition needed — Approach step 3 already points at the fork.
|
||||
|
||||
### `bundles/AGENTS.md`
|
||||
|
||||
- **S3 — Template recognition.** One paragraph: "Files under
|
||||
`bundles/<x>/files/` are static unless the `file:` item declares
|
||||
`content_type='mako'` or the file extension triggers templating
|
||||
(see fork's `docs/content/guide/item_file_templates.md`). To check:
|
||||
read the matching `file:` entry in `items.py`."
|
||||
- **S13 — How to remove a bundle.** 5-line section, symmetric to "How
|
||||
to add": "(1) `git grep '<name>'` to find references in nodes/groups/
|
||||
other bundles; (2) remove those references; (3) `rm -rf bundles/<x>/`;
|
||||
(4) `bw test` and `bw nodes` to confirm clean."
|
||||
- **S18 — README transition state.** "If a bundle has both `README.md`
|
||||
and `AGENTS.md`, `AGENTS.md` is canonical; the README is being phased
|
||||
out. ~23 bundle READMEs remain after the seed PR — Phase 3 leave-
|
||||
as-you-go folds them in over time."
|
||||
|
||||
### `bundles/AGENTS.template.md`
|
||||
|
||||
- **S12 — Optional `## Writes into` section.** Add to template (after
|
||||
`## Depends on`): "List other namespaces this bundle's `defaults` or
|
||||
reactors write into (e.g. nextcloud writes into `apt.packages` and
|
||||
`archive.paths`). Skip section if none — most bundles don't write
|
||||
cross-namespace, but the ones that do create the highest-blast-
|
||||
radius surprises."
|
||||
|
||||
### `nodes/AGENTS.md`
|
||||
|
||||
- **S2 — Silent eval-load-failure pitfall.** "Node files are `eval()`'d.
|
||||
A syntax error or top-level `import` causes the loader to silently
|
||||
drop the node. If `bw nodes` reports fewer nodes than expected, check
|
||||
`groups.py` (the user added explicit error printing in commit
|
||||
`dc40295` after being bitten)."
|
||||
- **S9 — `*.py_` suspend convention.** "Appending `_` to a node
|
||||
filename (e.g. `htz.l4d2.py_`) parks it without loading. Used to keep
|
||||
decommissioned-but-not-deleted node configs in tree."
|
||||
- **S9 — Symmetric "How to add a node" workflow.** Numbered steps
|
||||
parallel to `bundles/AGENTS.md` "How to add": (1) create
|
||||
`nodes/<location>.<role>.py` with `eval()`-safe expression syntax,
|
||||
(2) populate `id`, `hostname`, `groups`, `bundles`, `metadata`, (3)
|
||||
add to relevant `groups/<area>/<x>.py` if group membership is the
|
||||
attachment point, (4) verify with `bw nodes`, `bw nodes <node> -a groups`,
|
||||
`bw metadata <node>`.
|
||||
- **S19 — Node-rename failure mode.** "Renaming a node file renames
|
||||
the node. Vault entries (via `!password_for:<node>`), `bw hash`
|
||||
records, and ssh known_hosts associations all key on node name —
|
||||
search and replace before renaming, or vault lookups silently
|
||||
return new (wrong) values."
|
||||
|
||||
### `groups/AGENTS.md`
|
||||
|
||||
- **S10 — Family-file pattern.** "OS variants commonly share a
|
||||
`*-common.py` parent (e.g. `debian-13-common.py` shared by
|
||||
`debian-13.py` and `debian-13-pve.py`). Use this when introducing
|
||||
related-but-distinct OS group families."
|
||||
- **S10/S15 — New-OS-variant recipe.** "To introduce a new OS major:
|
||||
(1) add `groups/os/debian-N.py` + `debian-N-common.py` parallel to
|
||||
the existing files (don't edit in place); (2) add
|
||||
`data/apt/keys/debian-N-*.{asc,gpg}`; (3) bump dependent bundles
|
||||
that branch on OS string (e.g. `bundles/bind/items.py`); (4) bump
|
||||
affected nodes' `groups` lists one at a time; (5) delete the old
|
||||
OS file when no node references it."
|
||||
|
||||
### `data/AGENTS.md`
|
||||
|
||||
- **S7 — Two distinct content models.** "`data/apt/keys/` holds binary
|
||||
GPG keys consumed by `bundles/apt`; `data/grafana/rows/` holds Python
|
||||
modules for Mako-templated dashboards. Same directory shape, different
|
||||
content models — when adding a new data subdir, declare which model
|
||||
it follows."
|
||||
- **F4 — `data/` vs `bundles/<x>/files/` heuristic.** "If a data asset
|
||||
is read by exactly one bundle, prefer `bundles/<x>/files/`. Use
|
||||
`data/` for shared/multi-consumer artifacts. Single-instance evidence:
|
||||
`78a8abc` moved `mikrotik.mib` from data/ into the bundle for this
|
||||
reason."
|
||||
|
||||
### `hooks/AGENTS.md`
|
||||
|
||||
- **S17 — Broken-hook failure mode.** "A hook that errors at load time
|
||||
breaks every `bw` command that fires that lifecycle (including
|
||||
`bw test`, defeating the obvious diagnostic). Test new hooks in
|
||||
isolation first: `bw debug` then `import sys; sys.path.insert(0,
|
||||
'hooks'); import <hookmodule>`. Iterate there until the import is
|
||||
clean."
|
||||
|
||||
### `libs/AGENTS.md`
|
||||
|
||||
- **S16 — Find-consumers snippet.** Add as a one-liner under
|
||||
Pitfalls: "Before changing a lib's API, find consumers:
|
||||
`git grep -l 'repo.libs.<x>'`. Lib changes have repo-wide blast
|
||||
radius — every bundle that imports the lib re-evaluates."
|
||||
|
||||
### `bin/AGENTS.md`
|
||||
|
||||
- **S14 — `bin/script_template`.** "When introducing a new operator
|
||||
script, start from `bin/script_template` (the user maintains it as
|
||||
the canonical starter)."
|
||||
|
||||
## Phase 2 seed list rebalance
|
||||
|
||||
User-story analysis (story 5, subsystem-burst evidence) found that 3 of
|
||||
5 recent burst targets are unseeded. Strongest single swap:
|
||||
|
||||
- **Drop `php`** (8 node refs but ~zero recent commits — usage hub that
|
||||
doesn't change). Add **`routeros-monitoring`** (15 recent commits, the
|
||||
most-touched bundle in 18 months; not a dependency hub but high churn).
|
||||
|
||||
Optional second swap (judgment call):
|
||||
|
||||
- **Drop `apt`** (6 refs, has own lib, foundational but stable) and add
|
||||
**`bootshorn`** (recent burst target, has its own subsystem). Or keep
|
||||
`apt` for usage-hub coverage and accept that the seed misses bootshorn
|
||||
— Phase 3 covers it lazily.
|
||||
|
||||
Default this plan to the **single swap** (php → routeros-monitoring);
|
||||
note `apt → bootshorn` as a second-swap option if the user wants it.
|
||||
|
||||
## Critical files
|
||||
|
||||
**New:**
|
||||
|
||||
- `AGENTS.md` (root)
|
||||
- `CLAUDE.md` (symlink → `AGENTS.md`)
|
||||
- `docs/agents/conventions.md`
|
||||
- `docs/agents/commands.md`
|
||||
- `bundles/AGENTS.md`, `bundles/AGENTS.template.md`
|
||||
- `nodes/AGENTS.md`, `groups/AGENTS.md`, `libs/AGENTS.md`,
|
||||
`hooks/AGENTS.md`, `data/AGENTS.md`, `items/AGENTS.md`, `bin/AGENTS.md`
|
||||
- `bundles/{monitored,postgresql,wireguard,routeros-monitoring,apt,nginx,telegraf,backup,letsencrypt,nextcloud}/AGENTS.md`
|
||||
|
||||
**Modified:**
|
||||
|
||||
- `docs/superpowers/specs/2026-05-10-agent-friendliness-design.md` (fork → upstream correction).
|
||||
- `README.md` (remove stale "install bw fork" section).
|
||||
- `libs/*.py`, `hooks/*.py` (add module docstrings where missing).
|
||||
- `bin/*` (add `# purpose:` header where missing).
|
||||
|
||||
**Deleted:**
|
||||
|
||||
- `bundles/<name>/README.md` for each of the ~10 seed bundles that have one
|
||||
(content folded into `AGENTS.md`).
|
||||
|
||||
## Existing utilities & assets to reuse
|
||||
|
||||
- **Spec** at `docs/superpowers/specs/2026-05-10-agent-friendliness-design.md` —
|
||||
the source of truth for what each file contains. Sections 3, 4, 5, 6 of
|
||||
the spec are nearly copyable into the actual files.
|
||||
- **User-story validation doc** at
|
||||
`docs/superpowers/specs/2026-05-10-user-stories-from-history.md` —
|
||||
21 stories grounded in git history with concrete commit evidence.
|
||||
When writing per-bundle docs, look up the relevant story for evidence
|
||||
of typical changes (e.g. for `nextcloud`, see Story 1 + Story 5
|
||||
l4d2-style burst comparison). The "Implications for agent docs"
|
||||
paragraphs in each story map directly to the additions in §"User-
|
||||
story validation findings" above.
|
||||
- **ccc index** built at `.cocoindex_code/` on 2026-05-10 (768 chunks, 340
|
||||
files) — useful in PR2 for finding bundles that consume a metadata key
|
||||
or import a particular lib.
|
||||
- **Bundlewrap CLI** for verification: `bw metadata <node>`, `bw items <node>`,
|
||||
`bw test`, `bw hash`. Read-only; safe to run during writing.
|
||||
- **Existing bundle READMEs** at `bundles/{freescout,influxdb2,dm-crypt,gcloud,flask,nextcloud,build-server,raspberrymatic-cert,letsencrypt,nodejs}/README.md`
|
||||
(and any others — verify with `find bundles -name README.md`) — content
|
||||
to fold into the matching `AGENTS.md`. These are the only existing
|
||||
human-prose source for those bundles; do not lose information when migrating.
|
||||
|
||||
## Verification
|
||||
|
||||
Run after PR1:
|
||||
|
||||
1. `bw test` — repo-level sanity (passes today; should still pass).
|
||||
2. `bw nodes`, `bw groups`, `bw bundles` — sanity that loaders work after
|
||||
the docstring/header additions.
|
||||
3. Check every internal link in `AGENTS.md` and `docs/agents/**.md`
|
||||
resolves to a real file. A tiny shell loop with `grep -oE '\]\([^)]+\.md[^)]*\)'`
|
||||
then `test -f` each path.
|
||||
4. `readlink CLAUDE.md` resolves to `AGENTS.md`.
|
||||
5. `grep -L '"""' libs/*.py hooks/*.py` reports zero files (every lib/hook
|
||||
has a module docstring).
|
||||
6. `grep -L '^# purpose' bin/*` reports zero non-binary scripts.
|
||||
7. `git grep -i "bw fork\|bundlewrap-fork"` reports only the corrected
|
||||
docs locations (no leftover fork references in `README.md`).
|
||||
8. **Workflow walk-through.** Trace the "implement a new bundle" path
|
||||
end-to-end against the written docs: root → `bundles/AGENTS.md` →
|
||||
example pointer → can the writer locate vault/hashable/template
|
||||
guidance from there without spelunking? Confirms the §"Workflow
|
||||
validation findings" fixes actually closed the gaps.
|
||||
|
||||
Run after PR2:
|
||||
|
||||
1. `bw test` — still green.
|
||||
2. For each seed bundle `<x>`: pick one node that has it, run
|
||||
`bw metadata <node>` and confirm the keys listed in
|
||||
`bundles/<x>/AGENTS.md` `Metadata` section actually appear in the
|
||||
resolved metadata. Catches drift between docs and reality.
|
||||
3. `find bundles -name README.md` — confirm none exist for the 10 seed
|
||||
bundles (folded into `AGENTS.md`).
|
||||
4. Each new `bundles/<x>/AGENTS.md` follows the template structure
|
||||
(`grep -L '^## Metadata' bundles/*/AGENTS.md` reports zero).
|
||||
|
||||
## Non-goals (re-asserted from spec)
|
||||
|
||||
- No tooling changes (no `bw` wrapper, no Makefile, no lint, no CI).
|
||||
- No code refactoring, renaming, or splitting bundles.
|
||||
- No mass-fill of the remaining ~93 bundles up front.
|
||||
- No upstream contribution to bundlewrap (acknowledged future work).
|
||||
- The root `README.md` keeps everything except the fork section.
|
||||
|
||||
## Risks & mitigations
|
||||
|
||||
- **Drift between docs and code.** Mitigation: per-bundle docs are short
|
||||
(low maintenance), the area docs are mechanism-focused (changes less
|
||||
often than enumerations), and PR2 verification step 2 catches metadata
|
||||
drift at write time.
|
||||
- **PR1 is gated on the fork's `AGENTS.md` existing.** If you want to
|
||||
start ckn-bw work before the fork session lands, you can stub the link
|
||||
to the fork's `AGENTS.md` (e.g. point at the repo URL even if the file
|
||||
isn't there yet) and merge PR1, but that risks broken links if the fork
|
||||
URL or filename changes. Cleaner: do the fork session first, then PR1.
|
||||
- **PR1 is now a moderate writing chunk** (~800–1000 lines instead of
|
||||
~1500–2000), since the bundlewrap folder moved out. Single PR is
|
||||
comfortable. The user-story validation findings (16 small adds) push
|
||||
it to ~1000–1200; still manageable as one PR.
|
||||
- **README undercount.** The plan/spec estimated ~10 existing bundle
|
||||
READMEs to fold; actual is **33** (verified 2026-05-10 from git
|
||||
history). The seed list intersects with only 4–5 of those (nextcloud
|
||||
has one verified; others — `find bundles -name README.md` will
|
||||
enumerate). PR2 only folds READMEs in seed bundles; ~28 remain
|
||||
untouched, addressed lazily by Phase 3. `bundles/AGENTS.md` notes
|
||||
this transition state explicitly (per user-story finding §S18) so
|
||||
agents reading both don't get confused.
|
||||
|
|
@ -2,6 +2,54 @@
|
|||
|
||||
Date: 2026-05-10
|
||||
|
||||
## 0. Revisions
|
||||
|
||||
Material revisions since this spec was first written, kept here so anyone
|
||||
reading the spec sees the current shape rather than the original intent.
|
||||
|
||||
- **Fork pivot.** Originally the spec planned a `docs/agents/bundlewrap/`
|
||||
folder (`README.md` + `items.md` + `metadata.md`) explaining
|
||||
bundlewrap-the-language inside ckn-bw. That folder is gone; the
|
||||
maintainer maintains a personal bundlewrap fork at
|
||||
`github.com/CroneKorkN/bundlewrap` whose root `AGENTS.md` carries the
|
||||
canonical agent-oriented bundlewrap-language reference. ckn-bw's docs
|
||||
link out to the fork instead. The venv installs editable from the fork
|
||||
(`-e git+https://github.com/CroneKorkN/bundlewrap.git@main`).
|
||||
- **`commands.md` slimmed** from ~80–120 lines to ~30–50 lines: the fork's
|
||||
`AGENTS.md` carries the canonical bw runbook (read-only allowlist,
|
||||
after-change table, hash-diff workflow, `bw debug` sketch); ckn-bw's
|
||||
`commands.md` shrinks to repo-specific deltas (apt-key verification,
|
||||
`*.py_` suspended-node behavior, vault-echo guidance).
|
||||
- **Phase 2 seed-list rebalance.** `php` swapped out for
|
||||
`routeros-monitoring` based on user-story analysis: php is a low-churn
|
||||
usage hub (8 refs but ~zero recent commits); routeros-monitoring is
|
||||
high-churn (15 commits in 18 months), exactly where seeded docs pay off
|
||||
most. See plan for empirical justification.
|
||||
- **bw-syntax corrections** found by per-task code-review during the fork's
|
||||
AGENTS.md implementation, synced in: `bw items <node> <id> -p` does not
|
||||
exist (use bare or `--preview`); `bw hash` accepts only literal node /
|
||||
group names (selectors like `bundle:<x>` work for `bw nodes` etc., but
|
||||
not for `bw hash`); `bw groups -n <node>` does not exist (use
|
||||
`bw nodes <node> -a groups`).
|
||||
- **Workflow + user-story validation findings** (16 small content adds
|
||||
across area docs, the per-bundle template, `commands.md`, and
|
||||
`conventions.md`) are recorded in the implementation plan rather than
|
||||
back-fitted into this spec — they're additions to file content, not
|
||||
scope changes.
|
||||
- **Per-bundle docs are `README.md`, not `AGENTS.md`** (revised
|
||||
2026-05-10, after Phase 1 scaffolding landed). The spec originally
|
||||
specified one balanced `AGENTS.md` per bundle (§3 template) plus a
|
||||
Phase 2 seed migration that folded existing READMEs into new
|
||||
AGENTS.md files (§7). After Phase 1 landed, the maintainer flagged
|
||||
that the rigid template wouldn't survive contact with the existing
|
||||
READMEs (which range from one-paragraph balanced docs to operational
|
||||
scratchpads — see `bundles/{flask,dm-crypt,apt,nextcloud}/README.md`).
|
||||
Resolution: one `README.md` per bundle, no fixed shape, no template;
|
||||
Phase 2 dropped; existing READMEs stay in place under leave-as-you-go.
|
||||
Current convention lives in `bundles/AGENTS.md` "Per-bundle README".
|
||||
**Sections §3 and §7 are no longer authoritative — read them as
|
||||
pre-pivot intent only.**
|
||||
|
||||
## 1. Goals & non-goals
|
||||
|
||||
**Goal.** Make this BundleWrap config repo legible to agents (and humans) so an
|
||||
|
|
@ -17,11 +65,10 @@ spelunking and without unsafe side effects.
|
|||
- Per-bundle `AGENTS.md`: one balanced doc per bundle, replacing existing
|
||||
bundle `README.md` files. Template provided.
|
||||
- `docs/agents/conventions.md`: repo-specific idioms (vault magic strings,
|
||||
custom bundlewrap fork, files-not-to-touch).
|
||||
- `docs/agents/commands.md`: read-only `bw` command allowlist and an
|
||||
after-change runbook keyed by what was edited.
|
||||
- `docs/agents/bundlewrap/`: a focused folder explaining bundlewrap-as-used-here.
|
||||
Three files at first: `README.md`, `items.md`, `metadata.md`.
|
||||
bundlewrap-fork install pointer, files-not-to-touch).
|
||||
- `docs/agents/commands.md`: ckn-bw-specific deltas to the fork's bw
|
||||
runbook (apt-key verification, suspended-node behavior, vault-echo
|
||||
guidance). Canonical bw command reference lives in the fork's `AGENTS.md`.
|
||||
- A docstring/header pass on `libs/*.py`, `hooks/*.py`, `bin/*` so each
|
||||
individual file self-describes.
|
||||
- Phase 2 seed: per-bundle `AGENTS.md` for 10 bundles selected empirically.
|
||||
|
|
@ -45,11 +92,8 @@ ckn-bw/
|
|||
├── docs/
|
||||
│ └── agents/
|
||||
│ ├── conventions.md
|
||||
│ ├── commands.md
|
||||
│ └── bundlewrap/
|
||||
│ ├── README.md
|
||||
│ ├── items.md
|
||||
│ └── metadata.md
|
||||
│ └── commands.md # ckn-bw deltas; canonical bw runbook is in
|
||||
│ # the fork's AGENTS.md (linked from here)
|
||||
├── bundles/
|
||||
│ ├── AGENTS.md # what bundles are, how they compose
|
||||
│ ├── AGENTS.template.md # template for per-bundle docs
|
||||
|
|
@ -68,7 +112,10 @@ ckn-bw/
|
|||
|
||||
**Reading order an agent should follow.** Root `AGENTS.md` → relevant area
|
||||
`AGENTS.md` → specific `bundles/<x>/AGENTS.md` → `docs/agents/conventions.md`
|
||||
or `docs/agents/bundlewrap/<file>` only when something non-obvious comes up.
|
||||
when a repo-specific idiom is in play → fork's `AGENTS.md` (at
|
||||
`https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md`) for any
|
||||
bundlewrap-language question (item types, dep keywords, metadata reactor
|
||||
semantics).
|
||||
|
||||
**Per-area files (not just root).** An agent editing `bundles/nextcloud/items.py`
|
||||
already has `bundles/AGENTS.md` and `bundles/nextcloud/AGENTS.md` adjacent in
|
||||
|
|
@ -149,8 +196,11 @@ Target ~150 lines. Sections in order:
|
|||
`.envrc`. Everything else is editable, but treat `hooks/` and `items/`
|
||||
(custom item types) with extra care — they affect bw's behavior or item
|
||||
resolution across the whole repo.
|
||||
- Uses a custom **bundlewrap fork**, not upstream — check
|
||||
`docs/agents/conventions.md` before assuming upstream behavior.
|
||||
- Repo runs editable from the maintainer's bundlewrap fork
|
||||
(`github.com/CroneKorkN/bundlewrap`, branch `main`); behavior tracks
|
||||
upstream main but the fork's `AGENTS.md` is the canonical
|
||||
bundlewrap-language reference. See `docs/agents/conventions.md` for
|
||||
install detail.
|
||||
- Prefer adding helpers to `libs/` over duplicating logic across bundles.
|
||||
3. **Layout map.** Terse, link-rich. One line per top-level dir, each linking
|
||||
to that area's `AGENTS.md`.
|
||||
|
|
@ -159,12 +209,17 @@ Target ~150 lines. Sections in order:
|
|||
`nodes.py` and `groups.py` (root) are the loaders that walk the dirs and
|
||||
run `demagify`.
|
||||
5. **Conventions you must know.** One-line summary + link for each:
|
||||
- `docs/agents/bundlewrap/README.md` — read first if new to bundlewrap.
|
||||
`items.md` and `metadata.md` are deep dives for the hard parts.
|
||||
- Fork's `AGENTS.md`
|
||||
(`https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md`) —
|
||||
read first if new to bundlewrap. Carries the safety envelope, the
|
||||
after-change runbook, and cheat-sheets for item dep keywords +
|
||||
`metadata.py` pitfalls.
|
||||
- `docs/agents/conventions.md#secrets` — secrets / demagify magic strings.
|
||||
- `docs/agents/conventions.md#fork` — custom bundlewrap fork.
|
||||
- `docs/agents/conventions.md#bundlewrap-version` — install pointer
|
||||
(editable from the fork's `main`).
|
||||
- `docs/agents/conventions.md#groups` — group inheritance order.
|
||||
- `docs/agents/commands.md` — safe `bw` commands and after-change checks.
|
||||
- `docs/agents/commands.md` — ckn-bw-specific deltas to the bw runbook
|
||||
(apt keys, suspended nodes, vault-echo guidance).
|
||||
- Lib helpers — see top-of-file docstrings in `libs/*.py`.
|
||||
6. **Where to look for examples.** Pointers to a small bundle, a complex
|
||||
bundle, and a node file.
|
||||
|
|
@ -194,7 +249,8 @@ Per-area specifics:
|
|||
|
||||
- **`bundles/AGENTS.md`** — bundle anatomy (`items.py`, `metadata.py`, `files/`,
|
||||
`templates/`), where helpers go, when to extract to `libs/`. Links out to
|
||||
`docs/agents/bundlewrap/items.md` and `metadata.md` for language-level detail.
|
||||
the fork's `AGENTS.md` (item types, dep keywords, metadata reactor
|
||||
semantics) for language-level detail.
|
||||
- **`nodes/AGENTS.md`** — `eval()` loading mechanism via `nodes.py`, demagify
|
||||
magic-string syntax, naming convention pattern (`<location>.<role>.py`).
|
||||
**Pitfall:** because node files are `eval()`'d, no top-level imports — only
|
||||
|
|
@ -222,9 +278,10 @@ Per-area specifics:
|
|||
`!32_random_bytes_as_base64_for:`. What each does, where they're allowed
|
||||
(node files, evaluated through `nodes.py`), why agents must never echo
|
||||
decrypted values.
|
||||
- **Custom bundlewrap fork.** How it's installed
|
||||
(`pip install --editable git+file:///…/bundlewrap-fork@main`), implications
|
||||
(don't assume upstream-only behavior), pointer to the fork source.
|
||||
- **Bundlewrap version / install.** Repo runs editable from the maintainer's
|
||||
personal fork: `pip install -e git+https://github.com/CroneKorkN/bundlewrap.git@main#egg=bundlewrap`.
|
||||
Captured in `requirements.txt`. The fork's `main` tracks upstream main; the
|
||||
fork's `AGENTS.md` is the canonical bundlewrap-language reference.
|
||||
- **Group inheritance order** & how metadata merges
|
||||
(`all.py` → location → os → machine → applications → node).
|
||||
- **Naming conventions** for nodes (`<location>.<role>.py`) and groups
|
||||
|
|
@ -232,99 +289,53 @@ Per-area specifics:
|
|||
- **Files agents must not modify.** `.secrets.cfg*`, `.venv`, `.cache`,
|
||||
`.bw_debug_history`, `.envrc`.
|
||||
|
||||
### `commands.md` (~80–120 lines)
|
||||
### `commands.md` (~30–50 lines)
|
||||
|
||||
**Side-effect model** (paragraph up front):
|
||||
The fork's `AGENTS.md` is the canonical bw runbook — read-only command
|
||||
allowlist, after-change table, hash-diff workflow, `bw debug` sketch,
|
||||
verified against 5.0.3 source. ckn-bw's `commands.md` carries only
|
||||
repo-specific deltas:
|
||||
|
||||
- `bundles/<x>/metadata.py` `defaults` and `@metadata_reactor` can write into
|
||||
*any* namespace (e.g. nextcloud's metadata writes into `apt.packages` and
|
||||
`archive.paths`). Changing it can ripple into other bundles' inputs.
|
||||
- `libs/<x>.py` is imported by both `items.py` and `metadata.py` across many
|
||||
bundles — biggest blast radius.
|
||||
- `groups/*.py` changes membership (which bundles a node gets) and merged
|
||||
metadata.
|
||||
- `bw hash` is the primary integrated check because it captures bundle
|
||||
membership + metadata + items + file content.
|
||||
|
||||
**Read-only command reference** (one line each):
|
||||
`bw hash`, `bw metadata`, `bw items`, `bw items <node> <id> -p`, `bw nodes`,
|
||||
`bw groups`, `bw verify`, `bw debug`, `bw test`, `bw plot`. Each tagged
|
||||
read-only.
|
||||
|
||||
**After-change checks, keyed by what you changed:**
|
||||
|
||||
| You changed | First check | Drill-in |
|
||||
|---|---|---|
|
||||
| `bundles/<x>/items.py` | `bw hash <node>` for a node with bundle `<x>` | `bw items <node> <id> -p`; `bw verify <node>` |
|
||||
| `bundles/<x>/metadata.py` | `bw hash` for *all nodes including bundle `<x>`* (reactors can ripple beyond `<x>`'s namespace) | `bw metadata <node>`; `bw metadata <node> -k <key>` |
|
||||
| `bundles/<x>/files/*` (template) | `bw hash <node>` | `bw items <node> <path> -p` |
|
||||
| `groups/*.py` | `bw hash` every node in/near the group | `bw groups -n <node>`; `bw metadata <node>` |
|
||||
| `libs/<x>.py` | `bw hash` **all** nodes (cheap, no network) — biggest blast radius | `bw debug` to inspect helper outputs |
|
||||
| `nodes/<x>.py` | `bw hash <node>` | `bw metadata <node>` |
|
||||
| `hooks/*.py` | re-run the `bw` command whose lifecycle the hook hooks | — |
|
||||
| Anything | `bw test` — cheapest repo-level sanity | — |
|
||||
|
||||
**Mutating commands (forbidden without explicit user request).** `bw apply`,
|
||||
`bw run`, `bw lock` — what each does and why agents must not invoke them
|
||||
autonomously.
|
||||
|
||||
**Hash diff workflow.** Capture `bw hash > before.txt`, make change,
|
||||
`bw hash > after.txt`, diff. Canonical pre/post comparison.
|
||||
|
||||
**Targeting.** How to scope to one node / one group (`-t`, group selectors).
|
||||
|
||||
### `bundlewrap/README.md` (~80 lines)
|
||||
|
||||
Mental model paragraph: nodes ← groups → bundles → items; metadata flow
|
||||
(groups → node → metadata processors → bundle items); hooks vs items vs libs.
|
||||
Glossary, one paragraph each: node, group, bundle, item, items.py, metadata.py,
|
||||
metadata reactor, hook, lib, `repo.libs`. Folder index. Fork callout
|
||||
(explicit "we use a fork — here's what differs (or 'nothing day-to-day')";
|
||||
link to fork source). Links out to <https://docs.bundlewrap.org> for depth.
|
||||
|
||||
### `bundlewrap/items.md` (~200–300 lines)
|
||||
|
||||
The item types this repo actually uses — file, pkg_apt, svc_systemd, action,
|
||||
symlink, group, user, … — with common attributes and examples drawn from
|
||||
this repo. The dependency keyword glossary: `needs`, `needed_by`, `triggers`,
|
||||
`triggered`, `triggered_skip_for`, `tags`, `cascade_skip`, `unless` — each
|
||||
with one sentence and a real example. Custom item types from `items/`
|
||||
(currently `download`). "When in doubt, see upstream items reference" with
|
||||
a link.
|
||||
|
||||
### `bundlewrap/metadata.md` (~200–300 lines)
|
||||
|
||||
`defaults` dict semantics (deep-merge into resolved metadata, can write into
|
||||
any namespace). `@metadata_reactor.provides(...)` contract: signature, return
|
||||
shape, fixpoint resolution, when to declare `provides` narrowly vs broadly.
|
||||
Vault integration: `repo.vault.password_for`, `repo.vault.random_bytes_as_base64_for`,
|
||||
`repo.vault.decrypt`, `repo.vault.decrypt_file` — and how they tie back to
|
||||
demagify magic strings in node files. `repo.libs.hashable.hashable(...)` for
|
||||
putting dicts in sets — a pattern used heavily in this repo, worth explicit
|
||||
example. Common pitfalls: reactor declared narrower than it writes; reactor
|
||||
that doesn't reach fixpoint; reactor that reads a key it didn't declare reading.
|
||||
- **One-line lead** pointing at
|
||||
`https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md` for the
|
||||
full runbook.
|
||||
- **Apt-key after-change row.** Editing `data/apt/keys/*.{asc,gpg}` →
|
||||
first verify with `gpg --show-keys <newkey>` locally + fingerprint diff
|
||||
against the expected source. Trial via `bw apply` is the *failure* path
|
||||
(a wrong key blocks unattended upgrades cluster-wide). Not in the
|
||||
fork's runbook because it's repo-specific.
|
||||
- **`*.py_` suspended-node interaction.** A node file ending in `.py_` is
|
||||
silently excluded from the loader; `bw nodes` won't list it. Document
|
||||
this so an agent doesn't think a node is missing when it's actually
|
||||
parked.
|
||||
- **Vault magic-string handling.** Never echo decrypted output, even in
|
||||
`bw debug` exploration. Cross-link to `conventions.md#secrets`.
|
||||
|
||||
## 7. Seed work & rollout
|
||||
|
||||
### Phase 1 — scaffolding (one PR-sized chunk)
|
||||
### Phase 1 — scaffolding
|
||||
|
||||
1. Root `AGENTS.md` (Section 4) + `CLAUDE.md` symlink → `AGENTS.md`.
|
||||
2. `docs/agents/bundlewrap/README.md`, `items.md`, `metadata.md` (Section 6).
|
||||
3. `docs/agents/conventions.md` and `commands.md` (Section 6).
|
||||
4. Per-area `AGENTS.md` for `bundles/`, `nodes/`, `groups/`, `libs/`, `hooks/`,
|
||||
Gated on: the fork's `AGENTS.md` exists and is reachable at the URL above
|
||||
(verified 2026-05-10).
|
||||
|
||||
1. `docs/agents/conventions.md` (Section 6).
|
||||
2. `docs/agents/commands.md` (Section 6).
|
||||
3. Per-area `AGENTS.md` for `bundles/`, `nodes/`, `groups/`, `libs/`, `hooks/`,
|
||||
`data/`, `items/`, `bin/` (Section 5).
|
||||
5. `bundles/AGENTS.template.md` so future bundle docs have something to copy.
|
||||
4. `bundles/AGENTS.template.md` so future bundle docs have something to copy.
|
||||
5. Root `AGENTS.md` (Section 4) + `CLAUDE.md` symlink → `AGENTS.md` (written
|
||||
last so all internal link targets exist).
|
||||
6. Docstring/header pass: add a one-line module docstring to any `libs/*.py`
|
||||
and `hooks/*.py` lacking one; `# purpose:` header to any `bin/*` script
|
||||
lacking one.
|
||||
|
||||
Order within Phase 1: root → bundlewrap folder → conventions → commands →
|
||||
area docs → docstring pass → template. Each piece can be reviewed in
|
||||
isolation; the work bisects cleanly.
|
||||
Order rationale: build link targets bottom-up (conventions → commands →
|
||||
area docs → template), then root last, then the docstring pass last. Each
|
||||
piece can be reviewed in isolation; the work bisects cleanly.
|
||||
|
||||
Honest scope: the bundlewrap folder is ~600 lines of focused writing total
|
||||
(80 + 250 + 250). The rest is shorter — area docs and conventions land in
|
||||
30–120 lines each.
|
||||
Honest scope: ~800–1000 lines of focused writing total now that
|
||||
bundlewrap-language docs live in the fork. Area docs + conventions land
|
||||
in 30–120 lines each; root `AGENTS.md` is ~150 lines.
|
||||
|
||||
### Phase 2 — seed bundles (10)
|
||||
|
||||
|
|
@ -336,7 +347,7 @@ activity, validated 2026-05-10):
|
|||
1. `monitored` (12 node refs) — meta-bundle, often misunderstood.
|
||||
2. `postgresql` (9 refs, 3 cross-bundle).
|
||||
3. `wireguard` (8 refs, has own lib + bin script).
|
||||
4. `php` (8 refs).
|
||||
4. `routeros-monitoring` (15 commits in 18 months — most-churned bundle).
|
||||
5. `apt` (6 refs, has own lib).
|
||||
6. `nginx` (4 refs, web foundational).
|
||||
|
||||
|
|
@ -348,6 +359,10 @@ activity, validated 2026-05-10):
|
|||
9. `letsencrypt` (6 refs, cross-cutting).
|
||||
10. `nextcloud` (5 recent commits, complex, actively edited).
|
||||
|
||||
(Original §0 noted: `php` was originally seeded but swapped for
|
||||
`routeros-monitoring` after user-story analysis showed it's a low-churn
|
||||
hub, while routeros-monitoring is the highest-churn target in the repo.)
|
||||
|
||||
For each: write one `AGENTS.md` from the template — purpose, usage, metadata
|
||||
dict, produces, depends-on, gotchas. Migrate any existing
|
||||
`bundles/<x>/README.md` content into it and remove the old `README.md`.
|
||||
|
|
@ -362,14 +377,12 @@ dict, produces, depends-on, gotchas. Migrate any existing
|
|||
|
||||
## 8. Future work (not this spec)
|
||||
|
||||
- Contributing an `AGENTS.md` to bundlewrap upstream (or to your fork)
|
||||
describing items/metadata semantics for agents — would shrink the
|
||||
bundlewrap folder over time and shift authority back upstream.
|
||||
- Tooling: a read-only `bw` wrapper or lint that nudges new bundles toward
|
||||
having an `AGENTS.md`. Worth considering only after Phase 1+2 reveal which
|
||||
conventions actually drift.
|
||||
- More bundlewrap docs files (`groups-nodes.md`, `hooks.md`) if real gaps
|
||||
surface during Phase 2 or Phase 3 work.
|
||||
having an `AGENTS.md`. Worth considering only after Phase 1+2 reveal
|
||||
which conventions actually drift.
|
||||
- Pushing the fork's `AGENTS.md` upstream to `bundlewrap/bundlewrap` —
|
||||
it's written in a style that allows it; a follow-up the maintainer
|
||||
may pursue.
|
||||
|
||||
## 9. Open questions / risks
|
||||
|
||||
|
|
@ -377,10 +390,13 @@ dict, produces, depends-on, gotchas. Migrate any existing
|
|||
code. Mitigations: per-bundle docs are short (low maintenance); Phase 3
|
||||
rule attaches doc updates to material code edits; the area docs are
|
||||
mechanism-focused, which changes less often than enumerations.
|
||||
- **Risk: bundlewrap-folder duplicates upstream.** Acknowledged trade-off.
|
||||
Mitigation: the folder is scoped to *as we use it here*, with explicit
|
||||
upstream links; not trying to be a full bundlewrap manual.
|
||||
- **Open: which seed bundles to swap.** Phase 2 list is empirically grounded
|
||||
but not rigid — `zfs` (8 refs), `bind` (4 refs, own lib), and
|
||||
`routeros-monitoring` (15 recent commits, specialized) are honourable
|
||||
mentions if a swap is wanted later.
|
||||
- **Risk: fork drifts from upstream.** ckn-bw's docs link to the fork's
|
||||
`AGENTS.md`; if the fork falls far behind upstream main, the linked
|
||||
semantics might not match what real bundlewrap users see. Mitigation:
|
||||
the fork tracks upstream main via periodic merges; ckn-bw's
|
||||
`requirements.txt` pins `@main` so the venv stays aligned with the
|
||||
fork's documented behavior.
|
||||
- **Open: seed bundles.** Phase 2 list is empirically grounded but not
|
||||
rigid — `zfs` (8 refs), `bind` (4 refs, own lib), and `bootshorn`
|
||||
(recent burst target) are honourable mentions if a swap is wanted
|
||||
later.
|
||||
|
|
|
|||
714
docs/superpowers/specs/2026-05-10-user-stories-from-history.md
Normal file
714
docs/superpowers/specs/2026-05-10-user-stories-from-history.md
Normal file
|
|
@ -0,0 +1,714 @@
|
|||
# User stories — derived from ckn-bw git history
|
||||
|
||||
Date: 2026-05-10
|
||||
History range examined: first commit `572e29e` (2021-06-11) … HEAD `c03b033` (2026-05-10), total 1169 commits. Detailed per-file analysis covers the last 18 months (222 commits, 2024-11 → 2026-05); subject-line scans extended back 36 months for context. The pre-2024 history is dominated by the same patterns at lower velocity, so deeper sampling there did not surface new stories.
|
||||
|
||||
## Methodology
|
||||
|
||||
Three passes:
|
||||
|
||||
1. **Bulk extraction.** `git log --since="18 months ago" --name-status -M -C` written to `/tmp/ckn-bw-history.txt`. 1023 lines covering all 222 recent commits with file changes and rename/copy detection. This is the primary corpus.
|
||||
2. **Frequency analysis.** Per-bundle / per-node / per-area churn counts; first-word commit-message frequency; A/M/D/R action breakdown (461 modifies, 95 adds, 22 deletes, 9 renames over the window); rename pairs extracted for naming-convention inference.
|
||||
3. **Targeted spot reads.** `git show --stat` and full diffs on 10 commits representative of the high-stakes / cross-cutting / burst patterns (bw5 migration `186d503`, telegraf reactor refactor `4959461`, wol-waker scope-down `985a15e`, l4d burst seed `3469d98`, mailman introduction `d3b8e2e`, debian-13 `c41e6f8`, etc.) to verify intent matches message.
|
||||
|
||||
Clustering rule: a story is a recurring **user intent**, not a recurring **file pattern**. Two commits both touching `metadata.py` may be different stories. Aimed for stories with ≥3 grounding commits each. Final count: 19 stories.
|
||||
|
||||
The prior pass produced 12 stories. This pass adds 7 (investigation, trial-and-error microbursts, disable-for-now, cleanup-of-dead-code, subsystem deep-dive bursts, README touch-ups, naming-convention renames) and subdivides several of the original twelve.
|
||||
|
||||
## Story index
|
||||
|
||||
Ordered by frequency, recent-window unless noted.
|
||||
|
||||
1. **S1 — Tune one bundle's metadata** (~50+ commits). Small targeted `metadata.py` edits.
|
||||
2. **S2 — Tweak one node's config** (~80+ commits, top-heavy). Per-node tuning, often serial.
|
||||
3. **S3 — Edit a file template** (~40+ commits). `bundles/<x>/files/*` changes.
|
||||
4. **S4 — Trial-and-error microburst** (frequent within bursts; ~15 burst-clusters). Successive "fix"/"+"/"wip" commits.
|
||||
5. **S5 — Subsystem deep-dive burst** (5 major bursts in 18 months: l4d2, routeros, routeros-monitoring, bootshorn, debian-13).
|
||||
6. **S6 — Modify items.py for a bundle** (~30 commits). Item logic / dep tweaks.
|
||||
7. **S7 — Templated data update** (~15 commits). `data/apt/keys/*`, `data/grafana/rows/*`.
|
||||
8. **S8 — Add a new bundle** (10 in 18 months: yourls, mailman, routeros, routeros-monitoring, pppoe, bootshorn, kea-dhcpd, proxmox-ve, ifupdown, network).
|
||||
9. **S9 — Onboard a new node** (10 nodes added in 18 months).
|
||||
10. **S10 — Group adjustment / new OS variant** (~12 commits). `groups/os/*`, `groups/locations/*`.
|
||||
11. **S11 — Disable temporarily / "for now"** (~10 commits). Commenting out, dummy-ifying.
|
||||
12. **S12 — Cross-bundle metadata-reactor refactor** (4 high-impact commits in 18 months).
|
||||
13. **S13 — Cleanup / delete obsolete code** (~8 commits). `_old` bundles, removed nodes.
|
||||
14. **S14 — Add or revise an operator script** (`bin/`) (~8 commits, 4 new scripts in 18 months).
|
||||
15. **S15 — OS-version upgrade campaign** (3 campaigns: debian-12, debian-13, trixie/server-only).
|
||||
16. **S16 — Add/modify a lib** (`libs/`) (5 commits in 18 months; 1 new lib).
|
||||
17. **S17 — Add/modify a hook** (`hooks/`) (4 commits in 18 months; 2 new hooks).
|
||||
18. **S18 — Per-bundle README touch-up** (~8 commits, sporadic).
|
||||
19. **S19 — Naming-convention rename** (4 renames in 18 months).
|
||||
20. **S20 — Bundlewrap version migration** (1 commit, but uniquely high-stakes — counted because it shapes the maintainer's roadmap).
|
||||
21. **S21 — Interactive `bw debug` investigation** (.bw_debug_history evidence).
|
||||
|
||||
(Numbering shows 21 because three are kept short — S20 is a single high-stakes commit, S17 / S21 are low-frequency but materially distinct workflows. Anything that did not reach ≥3 grounding events was rolled into cross-cutting findings.)
|
||||
|
||||
## Stories
|
||||
|
||||
### Story 1: Tune one bundle's metadata
|
||||
|
||||
**Pattern.** Open `bundles/<x>/metadata.py`, change a value or add a key, commit. The most common single-file edit. Often very small (`fix typo`, `relax telegraf collection`, `mailserver zfs params`). The change is *intra-bundle* — no reactor surprise; just adjusting a default the bundle itself reads.
|
||||
|
||||
**Frequency.** ~50 commits over 18 months across many bundles. Recent: `7f20c94` (telegraf deprecations, 2026-03-09), `a7c7aaf` (nc preview options, 2026-03-09), `5fab21b` (apt install ca-certificates, 2026-02-10), `0d35bc2` (linux relax icmp ratelimit, 2026-02-10), `5fb1ee5` (less annoying root passwords / users, 2025-08-09).
|
||||
|
||||
**Files typically touched.** Single file: `bundles/<x>/metadata.py`. Sometimes one paired sibling file when the new metadata has an items consequence (see Story 6 for pure items work).
|
||||
|
||||
**Variations.**
|
||||
- (a) **Tune an existing value.** Change a number, add a member to a set, swap a string. ~70% of cases.
|
||||
- (b) **Add a new key.** `'unattended-upgrades': {}` added to the apt packages dict in `186d503`; `apt install ca-certificates` in `5fab21b`.
|
||||
- (c) **Restructure a sub-tree for new behavior.** Less common; tends to bleed into Story 12 if it changes a reactor's contract.
|
||||
|
||||
**Evidence.**
|
||||
- `7f20c94` (2026-03-09) — telegraf deprecations (2 files: bundles/routeros-monitoring/metadata.py + bundles/telegraf/items.py).
|
||||
- `0d35bc2` (2026-02-10) — `linux relax icmp ratelimit` (1 file).
|
||||
- `5fab21b` (2026-02-10) — `apt install ca-certificates` (1 file).
|
||||
- `a0f5f80` (2026-01-11) — typo fix in routeros-monitoring metadata.
|
||||
- `bdb9fa0` (2025-06-22) — `gitea disable registration` (changes `bundles/gitea/files/app.ini` template — boundary case with Story 3).
|
||||
|
||||
**Co-modification.** Usually none — single file. Occasionally pulls in `items.py` if the new key changes which file/service item is rendered. Almost never touches nodes (the value sits in defaults, nodes don't need to opt in unless a top-level key didn't exist before).
|
||||
|
||||
**Stakes.** Routine when the bundle's metadata is read only by itself. Moderate when the bundle uses cross-bundle reactor writes (e.g. telegraf, monitored). The maintainer's mental model assumes "metadata.py change → only the bundles using this bundle are affected"; that's only true if the bundle keeps to its own namespace. See Story 12 for the cross-bundle case.
|
||||
|
||||
**Implications for agent docs.** Well-served by `bundles/AGENTS.md` plus per-bundle `AGENTS.md` (PR2). Agents need to know:
|
||||
- The bundle's own resolved metadata shape — covered by per-bundle template's `Metadata` section.
|
||||
- Whether *this* bundle writes into other namespaces (cross-bundle ripple) — must be in the per-bundle `Gotchas` for any bundle that does (telegraf, monitored, wol-waker, archive). `commands.md`'s after-change row already covers it generally, but a per-bundle callout shortens the agent's loop.
|
||||
- After-change verification: `bw hash <node>` — the table in `commands.md` covers this. **No gap.**
|
||||
|
||||
### Story 2: Tweak one node's config
|
||||
|
||||
**Pattern.** Edit a single `nodes/<name>.py` to change something node-specific: an interface, a hostname, a group membership, a metadata override. The most common workflow, but heavily skewed to a few "hot" nodes — `home.router.py` (25 touches/18mo), `ovh.secondary.py` (29), `home.server.py` (16). Most edits are tiny.
|
||||
|
||||
**Frequency.** ~80 node touches in 18 months across 21 nodes. 70% concentrated in 5 nodes.
|
||||
|
||||
**Files typically touched.** Single node file. Occasionally two when the change is "move responsibility from node A to node B" (e.g. `5a8dc7e` 2025-12-03 moves wakeonlan to its own vlan, edits home.backups.py + home.router.py + home.switch-rack-poe.py + groups/os/routeros.py).
|
||||
|
||||
**Variations.**
|
||||
- (a) **Per-application config tweak** (e.g. ovh.secondary l4d server config — see Story 5).
|
||||
- (b) **Hardware/network shape change** (home.server "more swap"; "fan/motherboard sensors"; home.router "fix dhcp"; "fix ip").
|
||||
- (c) **Group membership change** — adding/removing strings in the node's `groups` list. Cross-cuts with Story 10.
|
||||
- (d) **Bundle attachment** — adding a bundle to the node's `bundles` list (rare; usually goes via group instead).
|
||||
|
||||
**Evidence.**
|
||||
- `838da64` (2026-03-09) — home server fan/motherboard sensors (1 file).
|
||||
- `fcd92db` (2026-03-09) — more swap (1 file, home.server).
|
||||
- `4652a42` (2026-01-10) — disable zfs mirror for now (home.backups.py — also a Story 11 example).
|
||||
- `60f29aa` (2025-08-24) — fix hue dhcp (home.hue).
|
||||
- `0e78afe` (2024-11-23) — fix ip (home.router).
|
||||
|
||||
**Co-modification.** Often none. When the change is a group-membership shift, the receiving group's file (`groups/os/routeros.py`, `groups/locations/home.py`) gets a paired edit. Network-shape edits frequently pull in 2-3 nodes plus `bundles/network/metadata.py`.
|
||||
|
||||
**Stakes.** Routine for value tweaks; moderate when the node provides services others depend on (home.router, htz.mails). High if node-file syntax is broken — recall that `nodes/<x>.py` is `eval()`'d, so a syntax error or top-level `import` breaks the loader for all nodes (commit `dc40295` in 2025-06-22 added an explicit `print message on parsing group error` to `groups.py`, suggesting the user has been bitten by silent loader failures).
|
||||
|
||||
**Implications for agent docs.** `nodes/AGENTS.md` should call out:
|
||||
- The `eval()` constraint (no top-level imports, only expression-level constructs) — already in spec §5.
|
||||
- Where to find resolved metadata after the edit (`bw metadata <node>`) — covered by `commands.md` row "nodes/<x>.py".
|
||||
- Naming convention `<location>.<role>.py` and how it's tied to the loader — covered.
|
||||
- **Gap:** the `eval()` failure mode is silent; the user himself patched the loader to print a message. Worth an explicit *pitfall* in `nodes/AGENTS.md`: "if `bw nodes` reports nothing or fewer nodes than expected, you have an `eval()` error in a node file." Otherwise an agent will assume the file was loaded.
|
||||
|
||||
### Story 3: Edit a file template
|
||||
|
||||
**Pattern.** Open `bundles/<x>/files/<template>` (a Mako/plain-text/script file rendered by `file` items), change content, commit. Often more frequent than `metadata.py` for service-config-heavy bundles.
|
||||
|
||||
**Frequency.** ~40+ commits over 18 months. Top hits: `bundles/left4dead2/files/setup` (11 touches), `bundles/left4dead2/files/start` (6), `bundles/left4dead2/files/server.cfg` (4), `bundles/left4dead2/files/scripts/overlays/tickrate` (4), `bundles/bootshorn/files/temperature` (4).
|
||||
|
||||
**Files typically touched.** Single template file most of the time. Sometimes paired with `items.py` if the template's render context (`content_from`, conditionals) needs an `items.py` edit too.
|
||||
|
||||
**Variations.**
|
||||
- (a) **Trivial edit** (one-line tweak in `nginx.conf`, a setting in `roundcube/files/config.inc.php`).
|
||||
- (b) **Mako template logic change** (e.g. `data/grafana/flux.mako` — Mako-based dashboards).
|
||||
- (c) **Whole-file rewrite** (less common — usually a new feature).
|
||||
|
||||
**Evidence.**
|
||||
- `a629024` (2026-01-11) — `bundles/roundcube/files/config.inc.php`: smtp use domain name from cert.
|
||||
- `bf38520` (2026-03-07) — comment out slow download workshop maps in l4d overlay file.
|
||||
- `64f8691` (2025-08-09) — bind named.conf.local: zones.rfc1918 only affect recursive views.
|
||||
- `e117aca` (2025-01-09) — `bundles/backup/files/backup_all`: don't stop on first error.
|
||||
|
||||
**Co-modification.** Often none. Pairs with items.py when the file's render context changes.
|
||||
|
||||
**Stakes.** Routine. Side effects don't ripple — the file is rendered once on the destination node. Templates with embedded scripts (e.g. `bundles/backup/files/backup_all` is a shell script) are a hidden risk: a bug there runs on production.
|
||||
|
||||
**Implications for agent docs.** `bundles/AGENTS.md` should mention `files/` at all (currently the spec template lists "files/, templates/" in the bundle anatomy section but doesn't say *which* files are rendered as templates vs static, or how to recognize Mako vs plain). Need:
|
||||
- One paragraph: how the `file` item picks template vs static (default extension behavior, `content_from`, `content_type`).
|
||||
- A pointer to the **fork's** `docs/content/guide/item_file_templates.md` — already noted in the plan's "Workflow validation findings" §5. **Confirmed needed.**
|
||||
- For service-config-heavy bundles (left4dead2, bind, postgresql), per-bundle `Gotchas` should call out non-trivial template logic. Currently only nextcloud, mailman, freescout have ad-hoc READMEs — most are bare.
|
||||
|
||||
### Story 4: Trial-and-error microburst
|
||||
|
||||
**Pattern.** A series of 3-15 short commits in quick succession (often the same hour) that walk a single change toward working. Subjects like `fix`, `+`, `whitespace`, `dowsnt exist`, `fix indent`, `fix set not dict`. Usually one bundle or one node, sometimes a single file modified 5-10 times in a day. This is a *workflow style*, not a single change.
|
||||
|
||||
**Frequency.** ~15 burst-clusters in 18 months. Recurring, not anomalous. Examples: nftables fixes (2025-07-11: `e7c5fe9`, `5a1ce55`, `8829902`), bootshorn temperature tweaks (2025-07-13: `c1917d5`, `6f86abd`, `951fa63`), l4d2 setup script (10+ commits within hours on 2025-10-29), routeros-monitoring (consecutive small edits across 2025-12-13 / 12-16 / 12-30).
|
||||
|
||||
**Files typically touched.** One file, repeatedly. Sometimes two paired files (items.py + metadata.py).
|
||||
|
||||
**Variations.**
|
||||
- (a) **Compile-fix burst.** "fix indent" / "fix set not dict" — Python typo / structural mistake immediately corrected.
|
||||
- (b) **Behavior-tuning burst.** Successive value changes ("more swap", "more workshop maps", "tickrate", "nc preview").
|
||||
- (c) **Apply-then-correct.** A change that broke something on the next `bw apply`, fixed in the next commit.
|
||||
|
||||
**Evidence.**
|
||||
- `e7c5fe9` → `5a1ce55` → `8829902` (2025-07-11, all within the same day, all `bundles/nftables/`).
|
||||
- `c1917d5` → `75017a9` → `6f86abd` → `951fa63` (2025-07-13, all `bundles/bootshorn/`).
|
||||
- `19c1945` → `a9e4013` → `8391afd` → `d91b205` → `3311bfb` → `351ce24` → `9572ac8` (2025-10-29, all l4d2).
|
||||
|
||||
**Co-modification.** None — repeats on the same files.
|
||||
|
||||
**Stakes.** Low individually, but reveals important workflow context: **commits are not landing-ready snapshots — they are the user's iterative checkpoints.** This shapes everything: an agent that reads recent history to learn "how does the user write code" will see noise unless it understands this.
|
||||
|
||||
**Implications for agent docs.** Not a per-feature docs gap — but **highly relevant context for every agent**. Belongs in root `AGENTS.md` quickstart or `conventions.md`:
|
||||
- "Commit messages are terse and frequently iterative; expect successive commits on the same file. Don't rebase the user's WIP commits without asking." (Style guidance for an agent commit-creating workflow.)
|
||||
- Verification chain: agents should run `bw test` / `bw hash` *before* committing rather than relying on later "fix" commits — an agent's commits are more durable than the user's, since they tend to land in batches.
|
||||
- **Currently neither covered.** ⚠ Partial gap — adjacent to spec §4 quickstart but not explicit.
|
||||
|
||||
### Story 5: Subsystem deep-dive burst
|
||||
|
||||
**Pattern.** Periods of 1-6 weeks where 80%+ of commits target one bundle/subsystem. This is Story 4 scaled up — instead of one day of microcommits, it's a multi-week project. The user is *learning the subsystem in production*: introducing it, finding issues, refining, often with cross-cutting touches (groups, nodes, libs) that wouldn't appear in a single-bundle story.
|
||||
|
||||
**Frequency.** 5 major bursts in 18 months, plus 2-3 minor ones.
|
||||
|
||||
| Burst | Window | Commits | Subsystem |
|
||||
|---|---|---|---|
|
||||
| **left4dead2 server rebuild** | 2025-08 → 2026-02 | ~50 | `bundles/left4dead2/`, `nodes/ovh.secondary.py` |
|
||||
| **routeros / mikrotik switches** | 2025-06-29 → 2026-01-11 | ~30 | `bundles/routeros/`, `bundles/routeros-monitoring/`, `groups/os/routeros.py`, 4 switch nodes |
|
||||
| **debian-13 upgrade** | 2025-08-09 → 2026-03-07 | ~15 | `bundles/dovecot/`, `bundles/redis/`, `bundles/bind/`, `groups/os/debian-13*`, multiple nodes |
|
||||
| **bootshorn (laptop temperature monitoring)** | 2025-07-11 → 2025-08-24 | ~15 | new bundle from scratch + `home.bootshorn-laptop` node |
|
||||
| **pppoe (telekom uplink)** | 2025-07-11 → 2025-08-03 | ~12 | `bundles/pppoe/`, `bundles/network/`, `bundles/nftables/`, home.router |
|
||||
|
||||
**Files typically touched.** Mixed: cluster around one bundle, with pulls into `groups/os/<x>.py`, the relevant node, sometimes a new lib (e.g. `libs/version.py` introduced in the routeros burst at `86d9b8b`).
|
||||
|
||||
**Variations.**
|
||||
- (a) **Bring-up of a new capability** (routeros switches, pppoe, mailman, bootshorn) — touches multiple areas as scaffolding lands.
|
||||
- (b) **Refactor of an existing capability** (l4d2 rebuild from August onward — note the `_old` bundles created and later deleted; the `3469d98` "next l4d2 server iteration" commit is the burst seed).
|
||||
- (c) **Migration burst** (debian-13) — see Story 15.
|
||||
|
||||
**Evidence (one per burst seed and one mid-burst):**
|
||||
- l4d2: seed `3469d98` (2025-08-24, "the next l4d2 server iteration"), mid-burst `9572ac8` (2025-10-29, "l4d2 dynamic overlays"), wind-down `ac8e7e2` (2026-02-10, "delete old l4d bundles").
|
||||
- routeros: seed `85daf26` (2025-06-29, initial bundle + switches), mid `0e6a705` (2025-07-01, "routeros switches ok"), monitoring expansion `8539f59` (2025-12-13, "mikrotik snmp monitoring").
|
||||
- bootshorn: seed `5274639` (2025-07-11, "bootshorn recording" — full bundle skeleton in one commit), tuning `75017a9` → `951fa63` (Story 4 microburst).
|
||||
|
||||
**Co-modification.** Heavy. Burst commits routinely touch a bundle + a group + 1-2 nodes + (sometimes) a lib + (sometimes) a hook.
|
||||
|
||||
**Stakes.** Cumulatively high. Each burst introduces or substantially reshapes a subsystem. Mid-burst, the subsystem is in flux — items.py and metadata.py contracts shift, file templates churn, the docs (if any) are stale. **An agent landing in a hot burst should be the most cautious, not the least.**
|
||||
|
||||
**Implications for agent docs.**
|
||||
- The planned per-area docs and per-bundle docs cover the **steady state** but not the **burst state**. A burst-aware agent would need:
|
||||
- A way to know "this subsystem is in active flux" — derivable from `git log --since="1 month ago" --pretty=format:'' --name-only | grep <area> | wc -l`. Could be a one-liner in `bundles/AGENTS.md`'s "Pitfalls": "before writing into a subsystem, check `git log --since='1 month ago' bundles/<x>` — if it's hot, your assumptions about the metadata shape may already be stale."
|
||||
- The seed bundle list (Phase 2 in spec §7) **misses three of the five recent burst targets**: routeros (15 commits), routeros-monitoring (15), bootshorn (15), pppoe (12). Of those, only `wireguard` (8 refs) is on the seed list. The spec acknowledges routeros-monitoring as "honourable mention." **⚠ Recommend rebalancing seed list toward currently-hot subsystems**, since those are where doc absence costs the most agent time.
|
||||
- **Gap.** The subsystem-burst pattern itself isn't documented anywhere planned. Could be one paragraph in `conventions.md` about "how this repo evolves" — it would let an agent recognize and respond to flux.
|
||||
|
||||
### Story 6: Modify items.py for a bundle
|
||||
|
||||
**Pattern.** Open `bundles/<x>/items.py`, add/remove/rewire a `file:`, `pkg_apt:`, `svc_systemd:`, `action:`, etc. Often paired with metadata.py if the new item reads from a key that didn't exist. The change is *what gets installed/managed*, not *with what value*.
|
||||
|
||||
**Frequency.** ~30 commits over 18 months that primarily target items.py.
|
||||
|
||||
**Files typically touched.** Single `items.py`. Frequently paired (a) `bundles/<x>/files/<new>` if a new file item is added, (b) `bundles/<x>/metadata.py` when the items.py reads a new metadata key.
|
||||
|
||||
**Variations.**
|
||||
- (a) **Wire deps right** — `needs`/`needed_by`/`triggers` adjustments. Lots of these (`9733a55` 2025-06-22 "svc_systemd:systemd-networkd add .service to name"; `c244645` "kea deps"; `663116c` "/var/lib/mysql needs mysql user to exist"; `befdf5a` "fixmo mariadb dependency"; `fb22a01` "systemd fix dependency overwrite").
|
||||
- (b) **Add a new managed file/service** — usually appears alongside an `items.py` skeleton in a bundle creation (Story 8).
|
||||
- (c) **Fix permissions** (`6616ae7`, `43e7c1f` redis permissions; `e17b023` grafana permission).
|
||||
|
||||
**Evidence.**
|
||||
- `9733a55` (2025-06-22) — `svc_systemd:systemd-networkd add .service to name`.
|
||||
- `47b69f0` (2025-11-04) — `l4d items stop script` (adds a file item).
|
||||
- `c244645` (2024-11-23) — `kea deps`.
|
||||
- `b838935` (2025-08-09) — `dont purge sudoers`.
|
||||
- `8066efb` (2025-12-03) — `routeros: also add comment to interface` (logical edit in items.py).
|
||||
|
||||
**Co-modification.** Frequently with the bundle's own `metadata.py`, occasionally with a node when an items.py change broke one specific node's resolved state.
|
||||
|
||||
**Stakes.** Routine for value/tag/dep tweaks. Moderate when restructuring an item's identity (renaming a file path, changing `name=` of an `svc_systemd:`) — a rename without redirect is **two state changes** (delete old, create new) which can cause a service flap.
|
||||
|
||||
**Implications for agent docs.** Covered by `bundles/AGENTS.md` (anatomy) and the fork's `AGENTS.md`/items reference (item-type semantics). The repo-specific gap:
|
||||
- The custom `download:` item type from `items/download.py` is mentioned in the plan but its semantics aren't documented in any planned area doc beyond `items/AGENTS.md`. The bw5 migration commit `186d503` touched `items/download.py` — agents adding a `download:` item need to know the actual interface (`expected_state`/`actual_state`/`get_auto_attrs`) post-bw5. **⚠ Need explicit per-item-type docstring on `items/download.py`** (the docstring/header pass in PR1 step 7 should cover this; verify it does).
|
||||
- Dependency-keyword vocabulary (`needs`, `triggers`, `triggered`, `cascade_skip`, etc.) lives in the fork's `AGENTS.md` per the plan. **No additional ckn-bw doc gap.** ✅
|
||||
|
||||
### Story 7: Templated data update (apt keys, dashboards)
|
||||
|
||||
**Pattern.** Two distinct sub-flavors that use the same `data/` directory mechanically:
|
||||
- (a) **Apt key refresh** — replace a `.gpg` / `.asc` file with a freshly downloaded key. ~5 commits in 18 months. Mechanical but high stakes (a wrong key breaks `apt update` on every node using that source).
|
||||
- (b) **Grafana dashboard edits** — `data/grafana/rows/*.py` and `data/grafana/flux.mako`. ~10 commits. Active design work, often tied to a bursting subsystem (e.g. routeros monitoring).
|
||||
|
||||
**Frequency.** 15+ data/ touches in 18 months.
|
||||
|
||||
**Files typically touched.**
|
||||
- (a) Apt: `data/apt/keys/<src>.{gpg,asc}` — single file, sometimes 2-3.
|
||||
- (b) Grafana: `data/grafana/rows/<row>.py` + sometimes `data/grafana/flux.mako`.
|
||||
|
||||
**Variations.**
|
||||
- (a) Apt key replace (one-shot).
|
||||
- (b) Apt key migration (e.g. `f0d1cf9` "new icinga apt key" added a `.gpg` and renamed the old `.asc` → `.asc_`, evidence the user keeps stale-but-not-removed keys around).
|
||||
- (c) Grafana dashboard tweak (cosmetic).
|
||||
- (d) Grafana new row (new monitoring source).
|
||||
|
||||
**Evidence.**
|
||||
- `1907c38` (2026-01-07) — `data/apt/keys/influxdata.asc: update`.
|
||||
- `487fdff` (2025-12-01) — update some apt keys (crystal, grafana).
|
||||
- `f0d1cf9` (2024-11-23) — new icinga apt key.
|
||||
- `da29405` (2026-01-11) — `data/grafana/rows/routeros_*: update names`.
|
||||
- `4a4167e` (2025-12-13) — routeros grafana discards and errors (new rows).
|
||||
|
||||
**Co-modification.** Apt: usually only `data/apt/keys/`. Sometimes paired with `bundles/apt/metadata.py` if the source URL/format also changed. Grafana: usually paired with `bundles/grafana/items.py` or `bundles/grafana/metadata.py`.
|
||||
|
||||
**Stakes.** Apt: **high** (broken key → blocked unattended upgrades cluster-wide). Grafana: low (dashboard reads only).
|
||||
|
||||
**Implications for agent docs.** `data/AGENTS.md` (planned) needs to:
|
||||
- Distinguish the apt-keys-as-data-source pattern from the grafana-rows-as-Python-dashboards pattern — they look the same at the directory level but have totally different content models.
|
||||
- Call out the verification step for apt keys: trial via `bw apply` is *not* safe (it's the failure path). Better: gpg-verify the key locally before committing. **⚠ Likely gap** — `commands.md`'s after-change table doesn't have a row for `data/apt/keys/*` (it falls into "Anything"). One-line addition warranted.
|
||||
- Grafana rows: covered by general-template guidance.
|
||||
|
||||
### Story 8: Add a new bundle
|
||||
|
||||
**Pattern.** Create `bundles/<name>/` from scratch — at minimum `items.py` + `metadata.py`, often `files/<file>`. Can be a single commit landing the skeleton or the seed of a deep-dive burst (Story 5).
|
||||
|
||||
**Frequency.** 10 new bundles in 18 months: yourls (2025-06-22), mailman (2025-06-29), routeros (2025-06-29), routeros-monitoring (2025-12-13), pppoe (2025-07-11), bootshorn (2025-07-11), kea-dhcpd (2024-09-18 inside `TOTAL FACKUP`), proxmox-ve (2025-06-22), ifupdown (2025-06-22), network/items.py (2025-07-10).
|
||||
|
||||
**Files typically touched.**
|
||||
- Always: `bundles/<name>/items.py`, `bundles/<name>/metadata.py`.
|
||||
- Often: 1-3 `bundles/<name>/files/<file>`.
|
||||
- Often: a node file (`nodes/<x>.py`) — to attach the bundle.
|
||||
- Sometimes: a group file (`groups/applications/<x>.py` or `groups/os/<x>.py`) when wiring as application/OS-level.
|
||||
- Occasionally: `data/<name>/` for bundle-templated data.
|
||||
- Sometimes: a README.md (mailman, pppoe, l4d2 — not consistent).
|
||||
|
||||
**Variations.**
|
||||
- (a) **One-commit skeleton** (most common). E.g. `5274639` bootshorn brought the whole bundle in 5 files at once.
|
||||
- (b) **Seed + burst.** Skeleton lands, then several follow-up commits refine (Story 4 + Story 5).
|
||||
- (c) **Inside a megacommit.** `67d5a4b` "TOTAL FACKUP" added kea-dhcpd, linux items.py + metadata.py among 20+ other changes. Anti-pattern, but it happens.
|
||||
|
||||
**Evidence.**
|
||||
- `5274639` (2025-07-11) — bootshorn introduction.
|
||||
- `aeb0a4f` (2025-06-22) — `nodes/mseibert.yourls.py: introduce` (5 files: yourls bundle + node + data/yourls/).
|
||||
- `d3b8e2e` (2025-06-29) — mailman (10 files added).
|
||||
- `e4e3c57` (2025-07-11) — pppoe telekom (full bundle).
|
||||
- `8539f59` (2025-12-13) — mikrotik snmp monitoring (routeros-monitoring bundle).
|
||||
|
||||
**Co-modification.** Always cross-area: bundle + node, often + group. This is the canonical "wire it in" workflow.
|
||||
|
||||
**Stakes.** Moderate to high. A new bundle wired wrong (group memberships off, missing dep, reactor that overlaps a sibling's namespace) can break unrelated nodes' apply.
|
||||
|
||||
**Implications for agent docs.**
|
||||
- Spec §3 per-bundle template is good shape — covers Metadata, Produces, Depends on, Gotchas.
|
||||
- Plan workflow validation finding §3 already calls out the explicit-wiring step. ✅
|
||||
- **Open question.** The seed bundle list for Phase 2 (`monitored, postgresql, wireguard, php, apt, nginx, telegraf, backup, letsencrypt, nextcloud`) was chosen for usage hubs + recent activity. Of the 10 *recently introduced* bundles in this story, only mailman and bootshorn would be obvious targets, and neither is on the seed list. **Recommendation (don't act, just surface):** when an agent uses `bundles/AGENTS.template.md` to write a new bundle's docs from scratch in Phase 3, the template's example shapes should be derived from a "freshly-introduced bundle that has no consumers yet" — the plan workflow finding §7 already raises this.
|
||||
- Pattern-grounded: the template should be writable end-to-end given just the skeleton commit's contents (items.py + metadata.py + files/*); verify by tracing one (`5274639` bootshorn) through the template fields. If any field is unfillable from the artifacts, that's an alignment gap. **Suggest verification step** during PR2.
|
||||
|
||||
### Story 9: Onboard a new node
|
||||
|
||||
**Pattern.** Create `nodes/<location>.<role>.py`, populate metadata + groups, often touch group files to wire membership. Sometimes accompanies adding a new bundle (Story 8) — but more often the bundles already exist and this is "deploy bundles X, Y, Z to a new machine."
|
||||
|
||||
**Frequency.** 10 new nodes in 18 months: home.bootshorn-laptop, home.rack-switch-10g (later renamed), home.switch-rack-poe, home.switch-vorratsraum-poe, home.switch-wohnzimmer-10g, home.wohnzimmer-switch-10g (later renamed/deleted), htz.l4d2.py_ (intentionally non-loaded — see naming), mseibert.mailman, mseibert.yourls, ovh.left4me.
|
||||
|
||||
**Files typically touched.**
|
||||
- Always: `nodes/<x>.py` (new file).
|
||||
- Often: 1-3 `groups/*.py` (membership lists; or `groups/os/<os>.py` to add the node).
|
||||
- Sometimes: a new bundle (Story 8 overlap) or a new ssh known-hosts entry (`metadata['known_hosts']`).
|
||||
|
||||
**Variations.**
|
||||
- (a) **New machine, existing bundles.** Fastest. Just node + group entries. E.g. `home.switch-rack-10g`-class introductions paired with `routeros` group additions.
|
||||
- (b) **New machine, new bundle.** Story 8+9 combined. mseibert.yourls (`aeb0a4f`).
|
||||
- (c) **Suspended-state node** (e.g. `nodes/htz.l4d2.py_` — appended underscore so the loader doesn't pick it up). Notable convention: loader file-pattern based.
|
||||
|
||||
**Evidence.**
|
||||
- `e99fd4b` (2026-05-10) — add ovh.left4me (also updates 2 existing nodes; 39 lines in the new node file).
|
||||
- `aeb0a4f` (2025-06-22) — mseibert.yourls (yourls bundle + vhost + node introduced together).
|
||||
- `5274639` (2025-07-11) — bootshorn-laptop introduction.
|
||||
- `9a51943` (2025-07-10) — switch-rack-poe.
|
||||
- `58007f5` (2026-03-07) — `dowsnt exist` adds `nodes/htz.l4d2.py_` (suspend convention).
|
||||
|
||||
**Co-modification.** Always cross-area: node + (group or bundle).
|
||||
|
||||
**Stakes.** Routine for adding to existing capacity. Moderate when the new node introduces a new role (mailman host, l4d server). High when the node has a typo in its `groups` list — typo silently fails to inherit, which the user explicitly added an error printer for (`dc40295`).
|
||||
|
||||
**Implications for agent docs.**
|
||||
- `nodes/AGENTS.md` covers the eval() mechanism and naming convention.
|
||||
- **Gap A:** the *suspend* convention (`*.py_` doesn't load) is a real, recurring practice — see `nodes/htz.l4d2.py_`. The spec/plan don't mention it. One line in `nodes/AGENTS.md` "Conventions" section warranted.
|
||||
- **Gap B:** "first-time node onboarding" sequence isn't a workflow that has docs anywhere. Possibly belongs in `bundles/AGENTS.md` "How to add" cross-linked to `nodes/AGENTS.md`. The plan's workflow validation already added a wiring step for *adding a bundle*; a symmetric "How to add a node" in `nodes/AGENTS.md` would be parallel.
|
||||
- Verification: `bw nodes`, `bw groups -n <node>`, `bw metadata <node>` — already in `commands.md`. ✅
|
||||
|
||||
### Story 10: Group adjustment / new OS variant
|
||||
|
||||
**Pattern.** Edit `groups/<area>/<x>.py` (or add a new file there). The "membership" function — which nodes get which bundles, and what default metadata flows. A small but high-leverage area.
|
||||
|
||||
**Frequency.** ~12 commits in 18 months. Concentrated in `groups/os/routeros.py` (10 touches — driven by the routeros burst) and the debian-13 family (`groups/os/debian-13-common.py`, `debian-13.py`, `debian-13-pve.py` all introduced 2025-08-09 onwards).
|
||||
|
||||
**Files typically touched.**
|
||||
- Most often a single `groups/os/<os>.py`.
|
||||
- For OS-version upgrades: 1-2 new files in `groups/os/`, parallel to existing version files.
|
||||
- Membership edits: occasionally also pull in a node file (paired Story 9).
|
||||
|
||||
**Variations.**
|
||||
- (a) **New OS variant.** debian-13 introductions (`c41e6f8` added `debian-13-common.py`, `debian-13.py` and deleted `debian-11.py`). proxmox-ve added `debian-12-pve.py`.
|
||||
- (b) **Membership tweak.** `b81c9e9` "allow snmp at home" toggles `groups/locations/home.py`.
|
||||
- (c) **Per-OS metadata adjustment.** `groups/os/routeros.py` got 10 touches as the routeros bundle matured.
|
||||
|
||||
**Evidence.**
|
||||
- `c41e6f8` (2025-08-09) — debian 13 (introduces `groups/os/debian-13.py`, `debian-13-common.py`; deletes `debian-11.py`).
|
||||
- `b81c9e9` (2025-12-16) — allow snmp at home (groups/locations/home.py).
|
||||
- `8d941eb` (2025-06-28) — open fw for iperf (groups/os/debian.py).
|
||||
- `463cf87` (2025-12-03) — mikrotik more port config (groups/os/routeros.py + 4 switch nodes).
|
||||
|
||||
**Co-modification.** Group edits often pull in 2-5 nodes (when adding/removing a group, the affected nodes' files are also adjusted). For new OS variants, add a node group file + bump nodes' `groups` lists + sometimes add a new apt key.
|
||||
|
||||
**Stakes.** **High.** A group's membership controls which bundles a node ships with, and metadata flows down via merge order (`all` → location → os → machine → applications → node). Wrong group membership = wrong bundles installed = real-machine consequences.
|
||||
|
||||
**Implications for agent docs.**
|
||||
- `groups/AGENTS.md` covers eval() loading, the subdir convention (`applications/`, `locations/`, `machine/`, `os/`), and `all.py`. ✅
|
||||
- Inheritance/merge order is in `conventions.md` (planned). ✅
|
||||
- **Gap A:** the "create a new OS variant" recipe isn't documented. Concrete steps the user followed in `c41e6f8`: (1) add `groups/os/debian-13.py` + `debian-13-common.py`; (2) add `data/apt/keys/debian-13-*.asc`; (3) ensure dependent bundles handle the new os string (e.g. `bundles/bind/items.py` was bumped). Worth a paragraph in `groups/AGENTS.md`.
|
||||
- **Gap B:** the *family-file pattern* (`debian-13-common.py` shared by `debian-13.py` and `debian-13-pve.py`) — a non-trivial idiom. Not in the spec/plan. One line in `groups/AGENTS.md` Conventions.
|
||||
|
||||
### Story 11: Disable temporarily / "for now"
|
||||
|
||||
**Pattern.** Comment out or stub out a block of working config to take it offline temporarily — service is broken upstream, host is offline, user is debugging, etc. Marker phrases in the commit message: "for now", "disable", "comment out", "dummy". The changes are not deletions; they're meant to be reversed.
|
||||
|
||||
**Frequency.** ~10 commits in 18 months. Recurring enough to be a recognized workflow.
|
||||
|
||||
**Files typically touched.** Single file. Usually a node file or a metadata.py.
|
||||
|
||||
**Variations.**
|
||||
- (a) **Disable a node entirely** (e.g. `35243fd` 2025-06-22 "offsitebackup offline"; `4f990f8` 2024-01-08 "stromzaehler is offline for now").
|
||||
- (b) **Comment out a metadata block** (`bf38520` 2026-03-07 "comment out slow download workshop maps"; `16313b9` 2025-01-09 "disable tasnomta charge"; `4652a42` 2026-01-10 "disable zfs mirror for now").
|
||||
- (c) **Dummy the role out** (`19a8d28` 2025-07-08 "homeassistant os is dummy"; `c03b033` 2026-05-10 "macbook dummy" — yes, the most recent commit).
|
||||
|
||||
**Evidence.**
|
||||
- `4652a42` (2026-01-10) — disable zfs mirror for now.
|
||||
- `bf38520` (2026-03-07) — comment out slow download workshop maps.
|
||||
- `c03b033` (2026-05-10) — macbook dummy (most recent commit, `nodes/macbook.py`).
|
||||
- `35243fd` (2025-06-22) — offsitebackup offline.
|
||||
- `4f990f8` (2024-01-08) — stromzaehler is offline for now.
|
||||
|
||||
**Co-modification.** None.
|
||||
|
||||
**Stakes.** Routine, but **easy for an agent to misread**. Disabled-but-committed code looks broken at first glance. An agent that "fixes" it will undo a deliberate suspension.
|
||||
|
||||
**Implications for agent docs.**
|
||||
- **⚠ Real gap.** Nothing in the planned docs warns an agent about the "for now" idiom. An agent doing a "tidy-up" pass could re-enable a deliberately-disabled node.
|
||||
- Belongs in `conventions.md` (planned) under a "Patterns to recognize and not break" subsection. One line: "Commits with 'for now' / 'disable' / 'dummy' / 'offline' in the message indicate a deliberate suspension. If you encounter a stub or commented-out block, check `git log -- <file>` for that pattern before re-enabling. The user reverses these manually when ready."
|
||||
- Verification: `bw nodes` is silent on whether a node *should* be picking up a bundle — there's no signal in tooling. Pure-doc fix.
|
||||
|
||||
### Story 12: Cross-bundle metadata-reactor refactor
|
||||
|
||||
**Pattern.** A change in *one* bundle's `metadata.py` (its `defaults` dict or a `@metadata_reactor`) that writes into other bundles' namespaces, materially altering many bundles' resolved state at once. Rare but expensive.
|
||||
|
||||
**Frequency.** 4 high-impact instances in 18 months.
|
||||
|
||||
**Files typically touched.** One bundle's `metadata.py` (the source of the change), plus 5-15 other bundles' `metadata.py` (the consumers being updated to match the new contract).
|
||||
|
||||
**Evidence.**
|
||||
- `4959461` (2026-01-11) — `bundles/telegraf/items.py: use new bundle from isac`. Restructures the entire telegraf metadata namespace; touches **12 bundles** (apcupsd, bind, hardware, postfix, postgresql, raspberry-pi, routeros-monitoring, smartctl, tasmota-charge, telegraf, zfs). 660 lines changed in routeros-monitoring/metadata.py alone.
|
||||
- `985a15e` (2026-01-11) — `wol waker only allow wakeonlan command`. Touches 8 bundles' metadata + 5 nodes via the wol-waker reactor's contract.
|
||||
- `186d503` (2026-05-10) — bundlewrap 5 migration. **Cross-cutting** rewrite of "non-reading and KeyError-driven metadata reactors per the bw 4→5 migration guide." 11 bundles + items/download.py.
|
||||
- `7eac09e` (2025-08-09) — `ovh.secondary cake`. Cross-bundle: linux + network + pppoe metadata + node.
|
||||
|
||||
**Co-modification.** Always wide. The change pattern fans out from one bundle to many.
|
||||
|
||||
**Stakes.** **Highest blast radius in the repo.** Reactors that write across namespaces are the spec's documented warning case (commands.md side-effect-model paragraph). When such a reactor's contract changes, every consumer must update in lockstep.
|
||||
|
||||
**Implications for agent docs.**
|
||||
- `commands.md` already calls out that `bundles/<x>/metadata.py` changes can ripple — first row of the after-change table says "for all nodes including bundle x" and notes "reactors can ripple beyond x's namespace." ✅
|
||||
- **⚠ Per-bundle gap.** The agent's signal that *this* bundle is one of the cross-writers should live in the per-bundle `Gotchas` or a `Writes into` section. Currently the per-bundle template (spec §3) has `Depends on` (which other bundles this bundle needs) but **not** `Writes into` (which other bundles' namespaces this one writes via reactor). For telegraf, monitored, archive, wol-waker, apt — that's the most important fact.
|
||||
- **Recommend (don't act):** consider adding an optional `## Writes into` section to the per-bundle template, or fold this into `Gotchas`. Without it, an agent editing `bundles/telegraf/metadata.py` won't know it's about to ripple unless it reads the diff carefully.
|
||||
- Bundlewrap 5 migration kind of work is one-shot — not worth a separate doc, but worth a sentence in `conventions.md` noting "this repo is on bundlewrap 5.0.3" so an agent doesn't apply 4.x patterns. Plan already covers this. ✅
|
||||
|
||||
### Story 13: Cleanup / delete obsolete code
|
||||
|
||||
**Pattern.** Pure-deletion or near-pure-deletion commits that remove dead bundles / nodes / templates. Often follows a Story 5 burst (the burst ended; old artifacts now obsolete).
|
||||
|
||||
**Frequency.** ~8 commits in 18 months.
|
||||
|
||||
**Files typically touched.**
|
||||
- Bundle deletions: 3-10 files in `bundles/<x>/`.
|
||||
- Node deletions: 1 node file.
|
||||
- Single-file template removals: 1 file.
|
||||
|
||||
**Variations.**
|
||||
- (a) **Remove an obsolete bundle entirely.** `849c305` "remove obsolete homeassistant supervised" deletes 3 files. `ac8e7e2` "delete old l4d bundles" deletes 12 files across left4dead2_old, left4dead2_old2, left4dead2_steam_old.
|
||||
- (b) **Remove an obsolete node.** `e65aa8f` deletes `nodes/home.openhab.py`. `c41e6f8` deletes `nodes/htz.games.py`.
|
||||
- (c) **Remove a single dead artifact.** `2899cd5` deletes `bundles/nextcloud/files/rescan`. `32ea52c` deletes `bundles/mariadb/files/override.conf`.
|
||||
|
||||
**Evidence.**
|
||||
- `ac8e7e2` (2026-02-10) — delete old l4d bundles (12 files deleted, 0 added).
|
||||
- `849c305` (2025-07-13) — remove obsolete homeassistant supervised.
|
||||
- `800bd90` (2025-06-28) — remove apcupsd.
|
||||
- `e65aa8f` (2025-08-09) — openhab no longer exists.
|
||||
|
||||
**Co-modification.** Sometimes nudges nodes that referenced the deleted bundle (e.g. `e65aa8f` also edits home.server.py to remove the openhab include). Pure-deletion if the bundle was already orphaned.
|
||||
|
||||
**Stakes.** Low when the asset is truly orphaned. Moderate if a node still references the deleted bundle (silent failure mode: bw will complain at apply time). **The user's workflow is: delete → next bw command surfaces dangling refs → fix.**
|
||||
|
||||
**Implications for agent docs.**
|
||||
- `bundles/AGENTS.md` "How to add" exists; **no symmetric "How to remove"** — and the recurring pattern is real. Worth a 5-line section: "To remove a bundle: (1) `git grep '<name>'` to find references in nodes/groups/other bundles; (2) remove those references; (3) `rm -rf bundles/<x>/`; (4) `bw test` and `bw nodes` to confirm clean."
|
||||
- Verification command sequence is in `commands.md`. ✅
|
||||
|
||||
### Story 14: Add or revise an operator script (`bin/`)
|
||||
|
||||
**Pattern.** Either introduce a new `bin/<script>` (one-off operator tooling, not invoked by bundlewrap) or revise an existing one. Standalone — usually does not pair with bundle changes.
|
||||
|
||||
**Frequency.** ~8 commits in 18 months. 4 new scripts: `bin/passwords-for`, `bin/sync_1password`, `bin/timestamp_icloud_photos_for_nextcloud`, `bin/mikrotik-firmware-updater`.
|
||||
|
||||
**Files typically touched.** Single `bin/<script>` file.
|
||||
|
||||
**Variations.**
|
||||
- (a) **Introduce new script** (commit subject often includes ": introduce").
|
||||
- (b) **Refactor existing script** — `bin/wireguard-client-config` was renamed from `bin/wireguard_client_config` in `1f4aaad` and the same commit improved it.
|
||||
- (c) **Bug fix.** `dcd2ebc` "dist-upgrade -> full-upgrade" in `bin/upgrade_and_restart_all`.
|
||||
|
||||
**Evidence.**
|
||||
- `60c2c42` (2026-03-09) — `bin/timestamp_icloud_photos_for_nextcloud: introduce`.
|
||||
- `979c7e1` (2025-12-03) — `bin/passwords-for: introduce`.
|
||||
- `2b873e4` (2025-12-01) — `bin/sync_1password` (script for syncing routeros logins to 1password).
|
||||
- `86d9b8b` (2025-12-13) — `bin/mikrotik-firmware-updater` introduced (alongside `libs/version.py`).
|
||||
- `fe5e340` (2025-12-03) — `bin/script_template: repo -> bw` (the user keeps a template script for new bin scripts).
|
||||
|
||||
**Co-modification.** `86d9b8b` paired with `libs/version.py` (script-needs-helper pattern). Otherwise standalone.
|
||||
|
||||
**Stakes.** Low. Operator scripts run on demand by the user, not automatically.
|
||||
|
||||
**Implications for agent docs.**
|
||||
- `bin/AGENTS.md` (planned) covers what bin/ is for. ✅
|
||||
- The existence of `bin/script_template` is itself a useful convention — an agent should start a new bin script from this template. **⚠ Mention it explicitly** in `bin/AGENTS.md` "How to add" — currently not in spec.
|
||||
- Plan PR1 step 7 adds `# purpose:` headers to bin scripts. The existing scripts use varied conventions — verify the header pass actually achieves uniformity. ✅
|
||||
|
||||
### Story 15: OS-version upgrade campaign
|
||||
|
||||
**Pattern.** Move nodes from one Debian major to the next. A long-tail campaign (per-node, weeks apart) rather than a single commit. Touches: new `groups/os/debian-N*.py`, new apt keys in `data/apt/keys/`, per-bundle adjustments where the new OS shipped with different packages or service names.
|
||||
|
||||
**Frequency.** 3 campaigns in the visible history: debian-12 (Jul 2023, ~10 commits), debian-13 (Aug 2025, ~15 commits), trixie-on-server-only (Mar 2026, 1 commit so far — early).
|
||||
|
||||
**Files typically touched (per campaign).**
|
||||
- New: `groups/os/debian-N*.py`, `data/apt/keys/debian-N-*.{asc,gpg}`, potentially `data/apt/keys/proxmox-ve-N.gpg`.
|
||||
- Modified: per-bundle metadata to handle new OS string; nodes' `groups` lists bumped.
|
||||
- Deleted: old OS group file (`groups/os/debian-11.py` deleted in debian-13 campaign; old keys retained as `_.asc` for ref).
|
||||
|
||||
**Evidence.**
|
||||
- `c41e6f8` (2025-08-09) — debian 13 (campaign seed: introduces debian-13 group files, adds apt keys, adjusts bind, removes debian-11 group, removes htz.games node).
|
||||
- `9621184` (2025-08-10) — htz.mails debian 13 (per-node bump + dovecot/redis/roundcube/systemd-swap adjustments).
|
||||
- `bc656cd` (2025-08-09) — backups debian 13 (one-line node bump).
|
||||
- `cb19c38` (2026-03-07) — update home.server to trixie (campaign seed for the next round; touches proxmox-ve metadata, ssh items, debian-13 keys/groups, multiple nodes).
|
||||
|
||||
**Co-modification.** Wide: data/ + groups/ + bundles/ + nodes/ in one commit each campaign-step.
|
||||
|
||||
**Stakes.** **High.** Each step touches a real machine; failures are recovery work. The user has been bitten — `c41e6f8` deleted `htz.games.py` outright, suggesting that node was decommissioned alongside its OS.
|
||||
|
||||
**Implications for agent docs.**
|
||||
- `conventions.md` (planned) mentions group inheritance order. ✅
|
||||
- **Gap.** The "campaign" workflow itself isn't documented anywhere. An agent helping with an OS bump won't know whether to (a) start a new `groups/os/debian-N*.py` family or (b) edit the existing one. The user's pattern is consistent: parallel new files, swap node memberships gradually, delete the old when no consumers remain.
|
||||
- Worth a few lines in `groups/AGENTS.md` "How to add / modify": "When introducing a new OS major, mirror the existing `debian-N-common.py` + `debian-N.py` + `debian-N-pve.py` triad and bump nodes one at a time. Don't edit the old OS file in place — let it die when no nodes reference it."
|
||||
- Verification: pre-bump `bw hash <node>` vs post-bump diff (planned). ✅
|
||||
|
||||
### Story 16: Add or modify a lib
|
||||
|
||||
**Pattern.** Edit `libs/<x>.py` (a Python module importable from bundles via `repo.libs.<x>`) or add a new lib. The widest blast radius in the repo — every bundle that imports the lib re-evaluates on next apply.
|
||||
|
||||
**Frequency.** 5 commits over 18 months. 1 new lib (`libs/version.py` in `86d9b8b`).
|
||||
|
||||
**Files typically touched.**
|
||||
- Single `libs/<x>.py` for tweaks.
|
||||
- Sometimes paired: `requirements.txt` if a new dependency is needed (`1d8361c` 2025-06-22 changed `libs/rsa.py` and `requirements.txt` together: `cache_to_disk broken`).
|
||||
|
||||
**Evidence.**
|
||||
- `86d9b8b` (2025-12-13) — `libs/version.py` introduced (mikrotik-firmware-updater needs version compare).
|
||||
- `1d8361c` (2025-06-22) — `libs/rsa.py` cache_to_disk broken.
|
||||
- `32ea52c` (2025-06-27) — `libs/ini.py` modified (mariadb refactor uses ini parser).
|
||||
- `e486aad` (2025-12-13) — `libs/bind.py` whitespace.
|
||||
|
||||
**Co-modification.** Rare. Lib changes don't pull in their consumers unless the lib's API changed.
|
||||
|
||||
**Stakes.** **Highest blast radius among non-reactor changes.** All bundles that import the lib re-evaluate; all nodes that include those bundles see the effect.
|
||||
|
||||
**Implications for agent docs.**
|
||||
- `libs/AGENTS.md` (planned) covers the convention. ✅
|
||||
- `commands.md` already says `libs/<x>.py` change → `bw hash` all nodes. ✅
|
||||
- Plan PR1 step 7 adds module docstrings to libs. ✅ (Critical here — without a one-line docstring, the agent has to read the body to know what the lib does, defeating discovery-by-`ls`.)
|
||||
- **Gap (minor).** The plan doesn't mention checking which bundles import a given lib — an agent making a breaking change to `libs/hashable.py` should know who imports it. `git grep -l 'repo.libs.hashable'` is a one-line snippet that could live in `libs/AGENTS.md` "Pitfalls."
|
||||
|
||||
### Story 17: Add or modify a hook
|
||||
|
||||
**Pattern.** Edit `hooks/<x>.py` (lifecycle hook fired by bw on specific events, e.g. before-apply, after-test). Like libs, repo-wide blast radius — a hook fires for *every* bw command of the right kind.
|
||||
|
||||
**Frequency.** 4 commits in 18 months. 2 new hooks.
|
||||
|
||||
**Files typically touched.** Single hook file. Sometimes paired with the bundle whose data the hook reads.
|
||||
|
||||
**Evidence.**
|
||||
- `0603a8c` (2025-12-01) — `hooks/unique_node_ids.py: introduce` (a `bw test`-time uniqueness check).
|
||||
- `7ea760d` (2026-01-11) — `hooks/test_ptr_records.py: introduce` (paired with mailserver metadata edit).
|
||||
- `7f43efc` (2025-12-03) — `hooks/wake_on_lan.py: dedup` (refactor existing hook).
|
||||
|
||||
**Co-modification.** Sparse. The new ptr-records test hook also touched `bundles/mailserver/metadata.py` because the hook reads from there.
|
||||
|
||||
**Stakes.** **High.** A broken hook breaks every bw command that fires it — an agent can't even run `bw test` to see what's wrong if the hook errors at hook-load time. Recovery requires either fixing the hook or `git stash`.
|
||||
|
||||
**Implications for agent docs.**
|
||||
- `hooks/AGENTS.md` (planned) covers the bw hook lifecycle and how to write one. ✅
|
||||
- Plan PR1 step 7 adds module docstrings. ✅
|
||||
- **⚠ Gap.** The "broken hook breaks all bw commands" failure mode isn't documented. An agent introducing a hook should test it in isolation first (e.g. import the module in `bw debug`) before letting it run in `bw test`. Worth one paragraph in `hooks/AGENTS.md` "Pitfalls."
|
||||
|
||||
### Story 18: Per-bundle README touch-up
|
||||
|
||||
**Pattern.** Edit `bundles/<x>/README.md` — sporadic, prose-only, usually adding a note to a bundle the user wants to remember something about. ~33 bundles currently have READMEs (more than the spec's "~10 of 103" estimate).
|
||||
|
||||
**Frequency.** ~8 commits in 18 months. mailman (`9bbaeb6`, `7df2187`, `980fdc8`), freescout (`64029d2`, `8081f12`, `4ec2d51`), l4d2 (`a397399`, `278f6de`), apt (`3dffc05`).
|
||||
|
||||
**Files typically touched.** Single README.
|
||||
|
||||
**Evidence.**
|
||||
- `9bbaeb6` (2025-07-12) — mailman poc email sent (creates README and tests).
|
||||
- `64029d2` (2024-11-23) — freescout readme.
|
||||
- `a397399` (2026-02-10) — l4d readme.
|
||||
|
||||
**Co-modification.** Sometimes paired with code changes in the same bundle.
|
||||
|
||||
**Stakes.** Routine.
|
||||
|
||||
**Implications for agent docs.**
|
||||
- **Big alignment issue.** PR2 plan replaces existing per-bundle READMEs with `AGENTS.md` for the 10 seed bundles. The plan correctly notes "verify with `find bundles -name README.md`" — actual count is **33 READMEs**, not the ~10 in the plan's table. The seed list intersects only minimally with the README list:
|
||||
- On both lists (seed + has README): nextcloud, telegraf, apt, mariadb (note: mariadb isn't on the seed list, but has README), letsencrypt.
|
||||
- Seed-list bundles **without** existing READMEs: monitored, postgresql, wireguard, php, nginx, backup. (Verified by `find bundles -name README.md`.)
|
||||
- READMEs on bundles **not** in seed list: ~25, including freescout, mailman, mailserver, mariadb, dm-crypt, dovecot, nodejs, nginx-rtmps, nftables, smartctl, wol-sleeper, wordpress, archive, gcloud, grafana, icingaweb2, influxdb2, raspberrymatic-cert, routeros, systemd, systemd-timers, tasmota-charge, telegraf, zfs, build-server, apcupsd, flask.
|
||||
- **Recommendation (don't act, just surface):** Phase 3 ("leave-as-you-go") will pick these up over time. **But:** the Phase 1/2 plan should mention that 23 existing READMEs will *remain* (untouched, parallel to AGENTS.md as PR2 only addresses seeds). Eventually those will need folding too. Worth being explicit in the plan that this is a known asymmetry, since `find bundles -name README.md` will report 23 left after PR2 — verifiers will trip on this.
|
||||
- For agents: `bundles/AGENTS.md` should mention "if a bundle has both a `README.md` and an `AGENTS.md`, the `AGENTS.md` is canonical; the `README.md` is being phased out." Currently planned doc is silent on the transition state.
|
||||
|
||||
### Story 19: Naming-convention rename
|
||||
|
||||
**Pattern.** Rename a node/bundle/file from underscore-style (or otherwise-irregular) to the canonical lowercase-hyphenated style. Done sporadically as the user notices inconsistency.
|
||||
|
||||
**Frequency.** 4 renames in 18 months (excluding internal moves like the mikrotik.mib relocation).
|
||||
|
||||
**Evidence.**
|
||||
- `1f4aaad` (2025-12-16) — `bin/wireguard_client_config` → `bin/wireguard-client-config` (and improved the script in the same commit).
|
||||
- `d54eff3` (2025-06-30) — `nodes/home.rack-switch-10g.py` → `nodes/home.switch-rack-10g.py` (puts "switch" before location-detail per `<location>.<role>.py`).
|
||||
- `d54eff3` — `nodes/home.wohnzimmer-switch-10g.py` → renamed in same commit (deleted in subsequent rename to switch-wohnzimmer-10g format).
|
||||
- `f0d1cf9` (2024-11-23) — `data/apt/keys/icinga.asc` → `data/apt/keys/icinga_.asc` (here the rename adds an underscore, but it's actually a "park the old file" pattern so the user can replace with a new gpg).
|
||||
- `78a8abc` (2025-12-16) — `data/routeros-monitoring/files/mikrotik.mib` → `bundles/routeros-monitoring/files/mikrotik.mib` (move from `data/` into the bundle — *relocation* convention, see cross-cutting findings).
|
||||
|
||||
**Files typically touched.** 1-2 paths (the rename), plus often 1-3 references in other files (groups, nodes).
|
||||
|
||||
**Co-modification.** The wireguard rename in `1f4aaad` was paired with script changes ("improve wireguard config gen"); the switch renames in `d54eff3` paired with the routeros bundle work.
|
||||
|
||||
**Stakes.** Low individually but **silent-trap-prone**: a rename of a node file changes the node's identity (since the loader file-name → node-name). Anything keyed by node name (vault entries via `!password_for:<node>`, hash records, ssh known_hosts) needs a parallel rename or the node loses its associations.
|
||||
|
||||
**Implications for agent docs.**
|
||||
- The plan's workflow validation finding §4 already calls for documenting the bundle naming convention (lowercase-hyphenated, no underscores). ✅
|
||||
- **Gap.** The node rename failure-mode isn't documented. Agents may rename a node file thinking it's purely cosmetic. One sentence in `nodes/AGENTS.md` Pitfalls: "Renaming a node file renames the node. `!password_for:<old-name>` magic strings, vault entries, and known_hosts associations all key on node name — search and replace before renaming."
|
||||
|
||||
### Story 20: Bundlewrap version migration (one-shot)
|
||||
|
||||
**Pattern.** Migrating the repo from one major version of bundlewrap to the next. Has happened once in the visible history (4.x → 5.0.3 in `186d503` on 2026-05-10) and is uniquely high-stakes.
|
||||
|
||||
**Frequency.** 1 commit in 18 months — but the maintainer is, per the handoff context, also planning to design their own config-management tool, so **this story is effectively an N=1 sample of "language migrations" the maintainer cares about.**
|
||||
|
||||
**Files touched.** 13 files: 11 `bundles/<x>/metadata.py`, `items/download.py`, `requirements.txt`. Per the commit body: rewrites "non-reading and KeyError-driven metadata reactors" per the bw 4→5 migration guide; renames custom Download item methods (`cdict/sdict/get_auto_deps` → `expected_state/actual_state/get_auto_attrs`).
|
||||
|
||||
**Evidence.** `186d503` (2026-05-10).
|
||||
|
||||
**Co-modification.** Wide cross-bundle. Mixes Story 12 (cross-bundle reactor) with Story 6 (items.py interface) into one operation.
|
||||
|
||||
**Stakes.** Highest. Rewriting reactors and item-type interfaces simultaneously means the repo is broken until the last file is corrected.
|
||||
|
||||
**Implications for agent docs.**
|
||||
- `conventions.md` (planned) notes the bundlewrap version. ✅
|
||||
- `commands.md` notes `requirements.txt` is the version pin. ✅
|
||||
- **Gap.** The migration was done with Claude assistance (per the commit's `Co-Authored-By`). An agent helping with the *next* major migration would benefit from a one-paragraph "how to do a bw version migration" recipe in `conventions.md`: read upstream migration guide → list affected reactor patterns → grep for them → rewrite each → bump `requirements.txt` last. **Recommendation (don't act):** worth capturing as a section in conventions, given the maintainer's own tool-design plans suggest such migrations will recur.
|
||||
|
||||
### Story 21: Interactive `bw debug` investigation
|
||||
|
||||
**Pattern.** Open `bw debug` (the bundlewrap interactive Python REPL with `repo` and `bw` preloaded), explore the repo state, prototype a code snippet. The 15-line `.bw_debug_history` is short but representative of how the user investigates before changing.
|
||||
|
||||
**Frequency.** Hard to count from git (history file is gitignored as of `39d5fb8` 2025-10-28). Evidence in-tree is one persistent file with 15 unique lines.
|
||||
|
||||
**What's in `.bw_debug_history`:**
|
||||
```
|
||||
bw.get_node('home.switch-wohnzimmer-10g')
|
||||
repo.get_node('home.switch-wohnzimmer-10g')
|
||||
repo.get_node('home.switch-wohnzimmer-10g').password
|
||||
from os import path, listdir
|
||||
path.join(repo.path, 'bundles/left4dead2/files/scripts/overlays')
|
||||
listdir(path.join(repo.path, 'bundles/left4dead2/files/scripts/overlays'))
|
||||
listdir(path.join(repo.path, 'bundles/left4dead2/files/scripts/overlays'))[0]
|
||||
listdir(path.join(repo.path, 'bundles/left4dead2/files/scripts/overlays'))
|
||||
from os import path, listdir
|
||||
a = listdir(path.join(repo.path, 'bundles/left4dead2/files/scripts/overlays'))[1]
|
||||
a
|
||||
type(a)
|
||||
repo.libs.ovh
|
||||
type(a)
|
||||
repo.libs.ovh
|
||||
```
|
||||
|
||||
**What this reveals.** The user (a) treats `bw debug` as a Python REPL with `repo`/`bw` pre-bound; (b) inspects nodes by name; (c) walks the filesystem inside `bundles/<x>/files/<dir>/` (the l4d overlay scripts work from Story 5); (d) probes `repo.libs.<name>` to see what's available — the trailing `repo.libs.ovh` looked up an ovh helper. This is **dynamic discovery**, complementary to file reads.
|
||||
|
||||
**Stakes.** Read-only — `bw debug` doesn't apply state. Routine.
|
||||
|
||||
**Implications for agent docs.**
|
||||
- `commands.md` lists `bw debug` as read-only. ✅
|
||||
- **⚠ Gap.** The *content* of `bw debug` (what's in scope: `repo`, `bw`, `node = repo.get_node(...)`, `node.metadata`, `repo.libs.<x>`) isn't sketched in `commands.md`. An agent that wants to investigate dynamically won't know what to type. One paragraph + 5 example lines (drawn directly from `.bw_debug_history`) would suffice.
|
||||
- Mention the gitignore: `.bw_debug_history` is gitignored (as of `39d5fb8`), so an agent's debug session won't accidentally land in a commit. Already covered by spec §4 quickstart's "do not modify" list. ✅
|
||||
|
||||
## Cross-cutting findings
|
||||
|
||||
### F1. Commit-message style is terse, iterative, and unreliable
|
||||
First-word frequency: `l4d` (22), `fix` (18), `bootshorn` (8), `routeros` (6), `remove` (6), `update` (4), `more` (4), `add` (4). Many subjects are typos (`fixmo`, `besteffort`, `dowsnt exist`, `mroe`, `unfault`, `englisch sprache schwere sprache`). The terseness is feature, not bug — see Story 4 (microbursts as workflow). Conclusion: **agents must read diffs, not subjects.** The detailed-history extraction approach (`git log --name-status -M -C`) was load-bearing; subject-only clustering would have over-bucketed.
|
||||
|
||||
### F2. The "introduce" verb is a reliable seed marker
|
||||
When the user names a commit `<path>: introduce` (or `<path>: introduce + …`), it's almost always the seed of a new bundle/script/hook/node skeleton. ~7 such commits in 18 months. Not a story per se, but a useful corpus marker for "where did X come from."
|
||||
|
||||
### F3. Naming convention: lowercase-hyphenated
|
||||
Confirmed across the bundle list — every current bundle uses lowercase-hyphenated naming (e.g. `backup-server`, `bind-acme`, `dm-crypt`, `homeassistant-supervised`, `kea-dhcpd`, `mailserver-autoconfig`, `nginx-rtmps`, `nextcloud-picsort`, `proxmox-ve`, `raspberry-pi`, `raspberrymatic-cert`, `routeros-monitoring`, `systemd-networkd`, `systemd-swap`, `systemd-timers`, `tasmota-charge`, `wol-sleeper`, `wol-waker`, `zfs-mirror`). The few underscore-named bundles (`left4dead2_old`, `left4dead2_old2`, `left4dead2_steam_old`) were all deleted via `ac8e7e2`. Bin scripts mostly hyphenated (after `1f4aaad` rename) but **a handful still underscored** (`bin/sync_1password`, `bin/timestamp_icloud_photos_for_nextcloud`, `bin/upgrade_and_restart_all`, `bin/script_template`, `bin/passwords-for`). Verdict: **convention is hyphenated, but underscored exceptions exist in `bin/`.** An agent should normalize to hyphenated for new bundles/nodes; for `bin/`, match the user's existing style (mixed).
|
||||
|
||||
### F4. Subdirectory-relocation pattern in active bundles
|
||||
`78a8abc` moved `data/routeros-monitoring/files/mikrotik.mib` into `bundles/routeros-monitoring/files/mikrotik.mib`, with the commit message "move to bundle bc why not." This was a *consolidation* — pulling bundle-specific data from `data/` into the bundle itself. Single instance, not yet a strong story, but suggests the user's mental model is `data/` = "shared/templated artifacts read by multiple bundles", `bundles/<x>/files/` = "this bundle's static files." Worth a sentence in `data/AGENTS.md`: "if a data asset is read by exactly one bundle, prefer placing it in `bundles/<x>/files/`."
|
||||
|
||||
### F5. Bursts have a characteristic shape
|
||||
Pattern observed in all 5 bursts: (i) seed commit introduces the skeleton in 1-3 hours of intense work; (ii) 1-2 day cooling period; (iii) 2-6 weeks of follow-up commits with shrinking commit-size; (iv) often a "delete obsolete" cleanup commit (Story 13) at the end. l4d2 followed this shape exactly (seed `3469d98` Aug 24, refinement Aug-Feb, cleanup `ac8e7e2` Feb 10). Recognizable signature for an agent reading recent history: many commits to one area in N days = active subsystem.
|
||||
|
||||
### F6. `_old` and `_old2` as a "soft delete" pattern
|
||||
The l4d2 rebuild renamed working code into `bundles/left4dead2_old/` and `bundles/left4dead2_old2/` (commit `3469d98`) before later deletion in `ac8e7e2`. The user uses suffixed-with-`_old` directories as a recovery buffer during big refactors. **Agents shouldn't delete `_old`/`_old2`-suffixed bundles without checking with the user**; they may be deliberate parking spots. Currently there are none in-tree (the `delete old l4d bundles` cleanup landed), but the pattern will recur.
|
||||
|
||||
### F7. The repo has no test suite, no CI, no formal type annotations
|
||||
Confirmed by tree inspection and history. The verification model is `bw test` (loader sanity), `bw hash` (state diff), `bw verify` (drift check) — a runtime check, not a static one. This shapes agent expectations: **don't propose test additions or CI without explicit user request** (the spec's "no tooling changes" non-goal aligns). Agents should use `bw debug` for ad-hoc validation instead of writing fixtures.
|
||||
|
||||
### F8. Merge commits are real, but bursts are usually merged via squash
|
||||
PRs visible in the history: `#18` homeassistant-supervised, `#19` mseibert_yourls, `#22` proxmox_mergable, `#23` routeros, `#24` ipv6_picking, `#25` debian-13, `#26` htz.mails_debian_13_squash, `#27` l4d2_the_next. The user uses Gitea/Forgejo for PRs (self-hosted). Naming pattern: descriptive branch name → PR. For agents creating PRs in this repo, branch-name style: lowercase-snake_case description (mirrors PR titles).
|
||||
|
||||
### F9. The README.md is a personal TODO, not a project README
|
||||
First lines of `README.md`: `# TODO\n\n- dont spamfilter forwarded mails\n- gollum wiki\n- blog?`. Already noted in spec §1 ("untouched personal TODO list") and plan PR1 step 8. **Agents must not regard the root README as documentation for the repo.** Unique enough to merit explicit callout in root `AGENTS.md`: "Note: `README.md` is the maintainer's personal scratchpad. Real onboarding lives here." Spec §4 and plan don't currently include that exact callout.
|
||||
|
||||
### F10. Commit cadence is bursty across years, not just months
|
||||
Total commits since first commit (2021-06-11): 1169. Last 36 months: 343. Last 24 months: 251. Last 18 months: 222. Last 12 months: 158. So ~70% of commits are in the last 24 months, but only ~10% in the last 12 months — meaning **commit activity has been declining**. Agents reading "recent history" should weight 6-12 months heavier than 12-24 months for "current state." (One nuance: the very last burst — May 2026 bundlewrap-5 migration — is recent, suggesting renewed activity ahead of the planned tool-design pivot.)
|
||||
|
||||
## Story coverage assessment vs the planned docs
|
||||
|
||||
For each story, traffic-light against PR1+PR2 as currently planned (root `AGENTS.md`, 8 area docs, conventions.md, commands.md, per-bundle template, 10 seed per-bundle docs, docstring/header pass on libs/hooks/bin, fork's `AGENTS.md` for bundlewrap language).
|
||||
|
||||
| # | Story | Status | Justification |
|
||||
|---|---|---|---|
|
||||
| 1 | Tune one bundle's metadata | ✅ | Per-bundle `Metadata` section; `commands.md` after-change row; mostly works as planned. |
|
||||
| 2 | Tweak one node's config | ⚠ | `nodes/AGENTS.md` covers eval() but should explicitly add the silent-load-failure pitfall (the user himself patched the loader for this — `dc40295`). |
|
||||
| 3 | Edit a file template | ⚠ | `bundles/AGENTS.md` lists `files/` in anatomy, but doesn't document Mako-vs-static recognition. Plan workflow finding §5 already adds the link to the fork's template guide — likely sufficient when written. |
|
||||
| 4 | Trial-and-error microburst | ❌ | Workflow-style context not captured anywhere. Belongs in root `AGENTS.md` quickstart or `conventions.md`: "user's commits are iterative; don't rebase WIP without asking." |
|
||||
| 5 | Subsystem deep-dive burst | ❌ | "Hot subsystem" awareness not in any planned doc. Seed bundle list (Phase 2) misses 3 of 5 recent burst targets (routeros, routeros-monitoring, bootshorn). Recommend rebalancing or documenting how to detect flux. |
|
||||
| 6 | Modify items.py for a bundle | ✅ | `bundles/AGENTS.md` + fork's items reference covers item types & deps. Item `download.py` docstring (PR1 step 7) closes the custom-item gap when written. |
|
||||
| 7 | Templated data update | ⚠ | `data/AGENTS.md` will exist; needs to distinguish apt-keys-as-data vs grafana-rows-as-Python. After-change check for apt keys not explicit in `commands.md`'s table — a one-line addition. |
|
||||
| 8 | Add a new bundle | ✅ | Per-bundle template + spec §3 is well-shaped. Plan workflow finding §3 added explicit wiring step. ✅ |
|
||||
| 9 | Onboard a new node | ⚠ | `nodes/AGENTS.md` covers eval() & naming, but missing: (a) the `*.py_` suspend convention; (b) symmetric "How to add a node" workflow paragraph. Both are one-line fixes. |
|
||||
| 10 | Group adjustment / new OS variant | ⚠ | `groups/AGENTS.md` covers basics; missing the family-file pattern (`debian-N-common.py` shared by `debian-N.py` + `debian-N-pve.py`) and the new-OS recipe. Both are paragraph-sized adds. |
|
||||
| 11 | Disable temporarily / "for now" | ❌ | Not in any planned doc. Real risk: agents "fixing" deliberately-stubbed code. Belongs in `conventions.md` as a single paragraph. |
|
||||
| 12 | Cross-bundle metadata-reactor refactor | ⚠ | `commands.md` warns about cross-namespace ripple in general. Per-bundle template lacks a `Writes into` field — recommend adding it (or folding into `Gotchas`) for cross-writing bundles (telegraf, monitored, archive, wol-waker, apt). |
|
||||
| 13 | Cleanup / delete obsolete code | ⚠ | "How to remove a bundle" not in `bundles/AGENTS.md`. Symmetric to "How to add" — 5-line addition. |
|
||||
| 14 | Add or revise an operator script (`bin/`) | ⚠ | `bin/AGENTS.md` covers what bin/ is; missing mention of `bin/script_template` as the canonical starter. One-line. |
|
||||
| 15 | OS-version upgrade campaign | ⚠ | Inheritance order in `conventions.md`; no campaign recipe. Belongs in `groups/AGENTS.md` "How to add" — paragraph-sized. |
|
||||
| 16 | Add or modify a lib | ✅ | `libs/AGENTS.md` + docstring pass + `commands.md` blast-radius row. Could optionally add a "find consumers" snippet (`git grep`). Minor. |
|
||||
| 17 | Add or modify a hook | ⚠ | `hooks/AGENTS.md` covers lifecycle; missing the "broken hook breaks all bw commands" failure-mode and isolation-test recipe. One paragraph. |
|
||||
| 18 | Per-bundle README touch-up | ⚠ | Plan undercounts existing READMEs (10 → actual 33). PR2 only addresses seed bundles; ~23 READMEs survive. Plan should acknowledge the transition state, and `bundles/AGENTS.md` should note "if both exist, AGENTS.md is canonical." |
|
||||
| 19 | Naming-convention rename | ⚠ | Plan workflow finding §4 covers bundle naming. Missing: node-rename failure mode (renaming changes node identity, breaks vault keys). Sentence in `nodes/AGENTS.md`. |
|
||||
| 20 | Bundlewrap version migration | ✅ | One-shot; `conventions.md` will note version. Optional: paragraph in `conventions.md` capturing the migration recipe — useful given maintainer's tool-design pivot. |
|
||||
| 21 | Interactive `bw debug` investigation | ⚠ | `commands.md` lists `bw debug` as read-only. Missing: what's in scope inside `bw debug` (paragraph + 5 example lines pulled from `.bw_debug_history`). |
|
||||
|
||||
**Tally.** ✅ 5 / ⚠ 13 / ❌ 3.
|
||||
|
||||
The ❌ gaps (S4 trial-and-error workflow context, S5 burst-state awareness, S11 disable-for-now idiom) are the highest-value adds because they shape an agent's *judgment*, not just its lookup steps. The ⚠ gaps are mostly one-line / one-paragraph additions to area docs that are already in scope for PR1.
|
||||
|
||||
The Phase 2 seed list is the largest single open question raised by this analysis: it leans toward usage-frequency hubs (postgresql, wireguard, php, nginx) but underweights currently-hot subsystems (routeros, routeros-monitoring, bootshorn). A swap of 2-3 entries — e.g. `php` → `routeros` and `apt` → `bootshorn` — would better match the work the maintainer (and any helper agent) will actually be doing in the next quarter. This is for human decision; not acted on.
|
||||
110
groups/AGENTS.md
Normal file
110
groups/AGENTS.md
Normal file
|
|
@ -0,0 +1,110 @@
|
|||
# groups/
|
||||
|
||||
## What's here
|
||||
|
||||
Groups attach bundles and shared metadata to nodes. One file per group,
|
||||
organized by axis:
|
||||
|
||||
```
|
||||
groups/
|
||||
├── all.py # universal base (every node belongs)
|
||||
├── applications/<x>.py # role-shaped groups (mailserver, monitored, …)
|
||||
├── locations/<x>.py # physical/network location (home, htz, …)
|
||||
├── machine/<x>.py # hardware kind (hardware, hetzner-cloud, raspberry-pi)
|
||||
└── os/<x>.py # OS major/variant (debian-13, debian-13-pve, routeros, …)
|
||||
```
|
||||
|
||||
## Loader mechanism
|
||||
|
||||
`groups.py` (top of the repo) walks `groups/` and runs `eval()` on each
|
||||
`*.py`. Same eval-as-expression rule as
|
||||
[`nodes/`](../nodes/AGENTS.md#loader-mechanism): one dict literal, no
|
||||
top-level imports, no statements. Errors print and the group is skipped
|
||||
— a real foot-gun, since a missing group silently changes node
|
||||
membership.
|
||||
|
||||
**Group files are *not* demagified.** Magic strings like
|
||||
`!password_for:<x>` only resolve in `nodes/*.py`. Inside a group file,
|
||||
call `repo.vault.<verb>(...)` directly.
|
||||
|
||||
## Inheritance and merge order
|
||||
|
||||
Metadata merges along this chain:
|
||||
|
||||
```
|
||||
all → location → os → machine → applications → node
|
||||
```
|
||||
|
||||
Per-axis subdirs are conventional, not enforced — `bw` doesn't read the
|
||||
subdir. Each group lists its `supergroups`, and `bw` resolves the DAG.
|
||||
Membership is set-union; metadata merge follows the order above, with
|
||||
the node's own `metadata` block winning last.
|
||||
|
||||
## Conventions
|
||||
|
||||
- **One group per file.** Filename without `.py` = group name. Subdir
|
||||
groups them by axis for humans, not for bw.
|
||||
- **Family files for OS variants.** Common parent + per-variant child.
|
||||
Example: `debian-13-common.py` is shared by `debian-13.py` and
|
||||
`debian-13-pve.py`. Use this pattern when introducing
|
||||
related-but-distinct OS group families.
|
||||
- **`all.py` is the universal default.** Currently empty (`{}`); kept
|
||||
for the rare repo-wide opt-in.
|
||||
|
||||
## How to add a group
|
||||
|
||||
1. Pick the right axis subdir (or root `all.py` for universal default).
|
||||
2. Create `groups/<axis>/<name>.py` as a single dict expression:
|
||||
|
||||
```python
|
||||
{
|
||||
'supergroups': [
|
||||
# parent groups whose bundles/metadata this one extends
|
||||
],
|
||||
'bundles': [
|
||||
# bundles every member of this group should have
|
||||
],
|
||||
'metadata': {
|
||||
# shared metadata for members
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
3. Wire the group into the relevant `nodes/*.py` (`'groups': {...}`)
|
||||
or `groups/*.py` `supergroups` list.
|
||||
|
||||
4. Verify with `bw nodes <node> -a groups` and `bw metadata <node>`.
|
||||
|
||||
## How to add a new OS major (recipe)
|
||||
|
||||
Pattern from prior debian-12 → debian-13 work:
|
||||
|
||||
1. Add `groups/os/debian-N.py` and `groups/os/debian-N-common.py`
|
||||
parallel to the existing files. Don't edit in place.
|
||||
2. Add `data/apt/keys/debian-N-*.{asc,gpg}` for the new release's
|
||||
signing keys. See
|
||||
[`commands.md#apt-key-changes-need-offline-verification`](../docs/agents/commands.md#apt-key-changes-need-offline-verification)
|
||||
before pushing keys live.
|
||||
3. Bump dependent bundles that branch on `os_version` /
|
||||
`os_codename` (`bundles/bind/items.py`, etc.).
|
||||
4. Bump affected nodes' `groups` lists one at a time. Apply, watch.
|
||||
5. Delete the old OS group file once no node references it.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **`bw groups -n <node>` doesn't exist.** Use
|
||||
`bw nodes <node> -a groups`.
|
||||
- **Cycles.** A group can't be its own supergroup transitively;
|
||||
`bw test` catches this but the error message is terse.
|
||||
- **Silent eval failure.** A group file with a syntax error is skipped
|
||||
and prints a one-line error. If a node loses bundles unexpectedly,
|
||||
scan `groups.py` output for the error.
|
||||
|
||||
## See also
|
||||
|
||||
- [`nodes/AGENTS.md`](../nodes/AGENTS.md) — node files; how
|
||||
`groups: {...}` attaches groups.
|
||||
- [`docs/agents/conventions.md`](../docs/agents/conventions.md) —
|
||||
inheritance order, naming conventions, eval-loader constraints.
|
||||
- Fork's [`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
|
||||
— group attribute reference, metadata-merge semantics.
|
||||
62
hooks/AGENTS.md
Normal file
62
hooks/AGENTS.md
Normal file
|
|
@ -0,0 +1,62 @@
|
|||
# hooks/
|
||||
|
||||
## What's here
|
||||
|
||||
Repo-level lifecycle hooks. Each `*.py` exports one or more functions
|
||||
named after the lifecycle event they listen to (`apply_start`,
|
||||
`node_apply_start`, `node_run_start`, `test`, `test_node`, …). Bw
|
||||
discovers them by importing each module from `hooks/`.
|
||||
|
||||
Discovery is by `ls hooks/` + the one-line docstring at the top of
|
||||
each file:
|
||||
|
||||
```sh
|
||||
head -1 hooks/*.py
|
||||
```
|
||||
|
||||
## Conventions
|
||||
|
||||
- **One-line module docstring.** Every `hooks/*.py` starts with
|
||||
`"""<name>: <one-line purpose>."""`. Add one when introducing a new
|
||||
hook; baseline is enforced by `grep -L '"""' hooks/*.py`.
|
||||
- **Function name = event name.** Bw calls
|
||||
`apply_start(repo, target, nodes, interactive=False, **kwargs)`,
|
||||
`node_apply_start(repo, node, interactive, **kwargs)`,
|
||||
`test(repo, **kwargs)`, etc. Always accept `**kwargs` so future bw
|
||||
arguments don't break the hook.
|
||||
- **Test gates use `test` / `test_node`.** Anything that should fail
|
||||
`bw test` (and therefore CI / pre-apply sanity) goes here; avoid
|
||||
doing test-style assertions in `apply_start`.
|
||||
|
||||
## How to add a hook
|
||||
|
||||
1. Pick the lifecycle event (see fork's `AGENTS.md` for the full list).
|
||||
2. Create `hooks/<name>.py` with the matching function and a
|
||||
docstring.
|
||||
3. Run `bw test` once to confirm the hook loads cleanly.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **A hook that errors at import breaks every `bw` invocation** that
|
||||
fires the hook's lifecycle — including `bw test` itself, which
|
||||
defeats the obvious diagnostic. Test new hooks in isolation first:
|
||||
|
||||
```sh
|
||||
bw debug -c "import sys; sys.path.insert(0, 'hooks'); import <hookmodule>"
|
||||
```
|
||||
|
||||
Iterate there until the import is clean, then commit.
|
||||
- **Hooks have access to the full repo (`repo`, `node`, `nodes`).**
|
||||
Don't make them block on network unless that's the explicit purpose
|
||||
(e.g. `test_ptr_records.py` does `dig` against `9.9.9.9`).
|
||||
- **Order is not guaranteed across hook files.** Two hooks that both
|
||||
define `apply_start` will both fire; don't assume which runs first.
|
||||
|
||||
## See also
|
||||
|
||||
- [`docs/agents/conventions.md`](../docs/agents/conventions.md) —
|
||||
files-not-to-touch, vault rules.
|
||||
- [`docs/agents/commands.md`](../docs/agents/commands.md) — test
|
||||
workflow.
|
||||
- Fork's [`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
|
||||
— full hook lifecycle and signatures.
|
||||
|
|
@ -1,3 +1,5 @@
|
|||
"""known_hosts: pre-apply hook that writes all nodes' ssh/is_known_as entries to ~/.ssh/known_hosts_ckn."""
|
||||
|
||||
from os.path import expanduser
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
"""skip_local_nodes: skip apply on `localhost` nodes whose metadata `id` doesn't match this host's local_id."""
|
||||
|
||||
from bundlewrap.exceptions import SkipNode
|
||||
|
||||
def node_apply_start(repo, node, interactive, **kwargs):
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
"""test_ptr_records: bw-test gate verifying live A/PTR DNS records for the `mailserver` group via `dig @9.9.9.9`."""
|
||||
|
||||
from subprocess import check_output
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,3 +1,6 @@
|
|||
"""unique_node_ids: bw-test + pre-apply gate ensuring metadata `id` is unique across all nodes."""
|
||||
|
||||
|
||||
def test_unique_node_ids(repo):
|
||||
ids = {}
|
||||
for node in repo.nodes:
|
||||
|
|
|
|||
|
|
@ -1,3 +1,6 @@
|
|||
"""wake_on_lan: pre-apply / pre-run hook that wakes a node via libs.wol before bw connects to it."""
|
||||
|
||||
|
||||
def wake_on_lan(node):
|
||||
node.repo.libs.wol.wake(node)
|
||||
|
||||
|
|
|
|||
64
items/AGENTS.md
Normal file
64
items/AGENTS.md
Normal file
|
|
@ -0,0 +1,64 @@
|
|||
# items/
|
||||
|
||||
## What's here
|
||||
|
||||
Custom item types — each `*.py` is a `bundlewrap.items.Item` subclass
|
||||
that adds a new item kind to the repo (e.g. `download:` from
|
||||
`items/download.py`).
|
||||
|
||||
Currently:
|
||||
|
||||
- `items/download.py` — verifying file downloads with sha256 / GPG.
|
||||
(Originally from `bundlewrap/plugins/item_download`; vendored in
|
||||
because bw 4 dropped plugin support and it never came back.)
|
||||
|
||||
## Conventions
|
||||
|
||||
- **Filename = item-type slug.** `items/download.py` defines item
|
||||
type `download:`. The class's `BUNDLE_ATTRIBUTE_NAME` is the dict
|
||||
name in `items.py` (`downloads = {...}`).
|
||||
- **Subclass `bundlewrap.items.Item`.** See the upstream
|
||||
[bundlewrap docs on writing item types](https://docs.bundlewrap.org/dev/items/)
|
||||
for the contract: `ITEM_ATTRIBUTES`, `fix`, `sdict`, `cdict`,
|
||||
`display_dicts`, etc.
|
||||
- **No metadata-reactor integration unless deliberate.** Item types
|
||||
get to declare `NEEDS_STATIC` (cross-type ordering hints) — use it
|
||||
sparingly; broad `NEEDS_STATIC` slows the whole DAG.
|
||||
|
||||
## When to write a new custom item type vs. use `file:` (or
|
||||
`action:`, etc.)
|
||||
|
||||
| Situation | Use |
|
||||
|---|---|
|
||||
| One-off shell command at apply time | `action:` in the bundle |
|
||||
| File whose source is in `bundles/<x>/files/` | `file:` |
|
||||
| File you have to **fetch + verify** at apply | `download:` (already custom) |
|
||||
| Behavior bw doesn't model and that recurs across bundles | new custom item |
|
||||
|
||||
If you only need it once, an `action:` is almost always enough.
|
||||
Custom item types are repo-wide and load on every `bw` invocation —
|
||||
the cost is paid forever.
|
||||
|
||||
## How to add a custom item type
|
||||
|
||||
1. Pick a slug (lowercase, no underscore).
|
||||
2. Create `items/<slug>.py` with an `Item` subclass.
|
||||
3. `bw test` — broken item types break the loader for every bundle
|
||||
that uses them, so test in isolation first.
|
||||
4. Document the contract in the file's module docstring.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **Items affect the whole repo.** A change to `items/<x>.py` runs in
|
||||
every node's apply. Treat custom items the way you treat `libs/` —
|
||||
pure, fast, side-effect-free at import.
|
||||
- **`NEEDS_STATIC` is a coarse hammer.** It enforces ordering across
|
||||
*all* nodes for *all* items of those types; don't add unless
|
||||
you've actually hit ordering issues.
|
||||
|
||||
## See also
|
||||
|
||||
- [`bundles/AGENTS.md`](../bundles/AGENTS.md) — items are consumed by
|
||||
`items.py` in each bundle.
|
||||
- Fork's [`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
|
||||
— built-in item-type catalogue.
|
||||
68
libs/AGENTS.md
Normal file
68
libs/AGENTS.md
Normal file
|
|
@ -0,0 +1,68 @@
|
|||
# libs/
|
||||
|
||||
## What's here
|
||||
|
||||
Shared Python helpers reachable from any bundle as
|
||||
`repo.libs.<modulename>.<symbol>`. One module per file (no packages).
|
||||
Discovery is by `ls libs/` plus the one-line module docstring at the
|
||||
top of each file.
|
||||
|
||||
```sh
|
||||
head -1 libs/*.py
|
||||
```
|
||||
|
||||
## Conventions
|
||||
|
||||
- **One file per module.** Filename (without `.py`) is the
|
||||
importable name. `libs/wireguard.py` exposes `repo.libs.wireguard`.
|
||||
- **One-line module docstring.** Every `libs/*.py` starts with
|
||||
`"""<name>: <one-line purpose>."""` so `ls + head` is a working index.
|
||||
Add one when introducing a new lib; the rule is enforced by the
|
||||
Phase-0 baseline (`grep -L '"""' libs/*.py` should report zero
|
||||
files).
|
||||
- **Pure helpers, no I/O at import.** Libs are imported on every `bw`
|
||||
invocation. Heavy work or filesystem reads at module scope slow the
|
||||
whole repo. Use functions, not module-level side effects.
|
||||
- **No magic-string demagification here.** Libs receive already-
|
||||
resolved values from bundles or nodes. Don't pass `!password_for:`
|
||||
strings into libs — they won't be resolved.
|
||||
- **`repo` and `vault` are globals available at runtime.** A few libs
|
||||
(`wireguard.py`, etc.) reach `repo.vault.<verb>(...)` directly.
|
||||
That's intentional inside libs, since libs run in the same loader
|
||||
scope as bundles.
|
||||
|
||||
## How to add a helper
|
||||
|
||||
1. Either extract from a bundle that's growing complex, or add a new
|
||||
`libs/<name>.py` file.
|
||||
2. Top line: `"""<name>: <one-line purpose>."""`.
|
||||
3. Pure functions only; document side effects in the docstring if any
|
||||
slip through.
|
||||
4. Use it from `bundles/<x>/items.py` or `metadata.py` as
|
||||
`repo.libs.<name>.<symbol>(...)`.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **Lib changes have repo-wide blast radius.** Every bundle that
|
||||
imports the lib re-evaluates on the next `bw test` / `bw apply`.
|
||||
Before changing a lib's API, find consumers:
|
||||
|
||||
```sh
|
||||
git grep -l 'repo.libs.<x>'
|
||||
```
|
||||
|
||||
- **`@cache` is your friend** for deterministic key derivations
|
||||
(`wireguard`, `rsa`, `ssh`); without it, `bw` recomputes per call,
|
||||
which is slow.
|
||||
- **Hashable wrappers.** `libs/hashable.py` exists because raw dicts
|
||||
and sets aren't hashable, so you can't put them inside metadata
|
||||
sets. Wrap with `repo.libs.hashable.hashable(...)` before nesting.
|
||||
|
||||
## See also
|
||||
|
||||
- [`bundles/AGENTS.md`](../bundles/AGENTS.md) — when to extract a
|
||||
helper into `libs/` instead of duplicating across bundles.
|
||||
- [`docs/agents/conventions.md`](../docs/agents/conventions.md) —
|
||||
vault, demagify, naming.
|
||||
- The fork's [`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
|
||||
— `repo.libs` mechanism.
|
||||
|
|
@ -1,3 +1,5 @@
|
|||
"""apt: render sources.list.d entries and resolve apt-key file extensions, with codename/version templating."""
|
||||
|
||||
from re import match
|
||||
from glob import glob
|
||||
from os.path import join, basename, exists
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
"""bind: DNS view-routing helpers — match A/AAAA records against internal vs external views via repo-wide collation."""
|
||||
|
||||
from ipaddress import ip_address
|
||||
|
||||
def _values_from_all_nodes(type, name, zone):
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python
|
||||
"""derive_string: deterministic byte-string derivation from a seed via ChaCha20 keystream."""
|
||||
|
||||
from hashlib import sha3_256
|
||||
from itertools import count, islice
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
"""grafana: panel and Flux-query generators for Mako-templated dashboards under data/grafana/rows/."""
|
||||
|
||||
from mako.template import Template
|
||||
from copy import deepcopy
|
||||
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
"""hashable: dict/set subclasses with stable __hash__ via canonical JSON — lets you nest dicts/sets inside sets in metadata."""
|
||||
|
||||
import json
|
||||
from functools import total_ordering
|
||||
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
"""hmac: base64-encoded HMAC-SHA512 helper."""
|
||||
|
||||
import hmac, hashlib, base64
|
||||
|
||||
def hmac_sha512(secret, iv):
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
"""ini: case-preserving ConfigParser parse/dumps for INI-style files where exact key casing must round-trip."""
|
||||
|
||||
from configparser import ConfigParser
|
||||
import json
|
||||
from bundlewrap.metadata import MetadataJSONEncoder
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
"""ip: A/AAAA-record derivation from a node's `network` metadata + cross-node IP collection."""
|
||||
|
||||
from ipaddress import ip_address, ip_interface
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
"""local: identifier of the bw host itself (read from ~/.config/bundlewrap/local_id) — paired with hooks/skip_local_nodes.py."""
|
||||
|
||||
from os.path import expanduser, exists
|
||||
from functools import cache
|
||||
|
||||
|
|
|
|||
|
|
@ -1,2 +1,5 @@
|
|||
"""nextcloud: build `sudo -u www-data php occ ...` shell strings for use in actions."""
|
||||
|
||||
|
||||
def occ(command, *args, **kwargs):
|
||||
return f"""sudo -u www-data php /opt/nextcloud/occ {command} {' '.join(args)} {' '.join(f'--{name.replace("_", "-")}' + (f'={value}' if value else '') for name, value in kwargs.items())}"""
|
||||
|
|
|
|||
|
|
@ -1,3 +1,6 @@
|
|||
"""nginx: recursive nginx config-block rendering from a nested dict."""
|
||||
|
||||
|
||||
def render_config(config):
|
||||
return '\n'.join(render_lines(config))
|
||||
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
"""postgres: SCRAM-SHA-256 password-hash generation for postgres role provisioning."""
|
||||
|
||||
from base64 import standard_b64encode
|
||||
from hashlib import pbkdf2_hmac, sha256
|
||||
import hmac
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
"""rsa: deterministic RSA private-key generation backed by a seeded PRNG."""
|
||||
|
||||
# https://stackoverflow.com/a/18266970
|
||||
|
||||
from Crypto.PublicKey import RSA
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
"""ssh: deterministic ed25519 keypair generation and salted known_hosts hashing."""
|
||||
|
||||
from base64 import b64decode, b64encode
|
||||
from hashlib import sha3_224, sha1
|
||||
from functools import cache
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
"""systemd: unit-file rendering (Mako) and a reusable hardening-options dict for sandboxed services."""
|
||||
|
||||
from mako.template import Template
|
||||
|
||||
template = '''
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
"""tools: identifier→IPs resolver (node|group|cidr), subnet-set deduplication, and bundle-requirement assertion."""
|
||||
|
||||
from ipaddress import ip_address, ip_network, IPv4Address, IPv4Network
|
||||
|
||||
from bundlewrap.exceptions import NoSuchGroup, NoSuchNode, BundleError
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
"""version: comparable Version class for dotted-int version strings."""
|
||||
|
||||
from functools import total_ordering
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
"""wireguard: deterministic WireGuard private/public key + PSK derivation, backed by repo.vault.random_bytes_as_base64_for."""
|
||||
|
||||
import base64
|
||||
from nacl.public import PrivateKey
|
||||
from nacl.encoding import Base64Encoder
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
"""wol: trigger a `wol-sleeper` waker node to run its configured wake command."""
|
||||
|
||||
from bundlewrap.utils.ui import io
|
||||
from bundlewrap.utils.text import yellow, bold
|
||||
|
||||
|
|
|
|||
92
nodes/AGENTS.md
Normal file
92
nodes/AGENTS.md
Normal file
|
|
@ -0,0 +1,92 @@
|
|||
# nodes/
|
||||
|
||||
## What's here
|
||||
|
||||
One file per node, ~22 in total. Each file is a single Python
|
||||
expression (a dict literal) describing one machine: hostname, groups,
|
||||
bundles, and metadata.
|
||||
|
||||
## Loader mechanism
|
||||
|
||||
`nodes.py` (top of the repo) walks `nodes/`, reads each `*.py` file,
|
||||
runs `eval()` on its content, and then *demagifies* the result —
|
||||
resolving any `!password_for:`, `!decrypt:`, `!decrypt_file:`, and
|
||||
`!32_random_bytes_as_base64_for:` magic strings via `repo.vault`. See
|
||||
[`docs/agents/conventions.md#secrets`](../docs/agents/conventions.md#secrets)
|
||||
for the magic-string list.
|
||||
|
||||
This loader shape has consequences:
|
||||
|
||||
- **No top-level imports.** The file must be a single expression. No
|
||||
`import os`, no `def`, no `if`. Use `repo.libs.<x>` from bundle code
|
||||
if you need a helper.
|
||||
- **Silent drop on parse failure.** Vanilla bundlewrap omits a node
|
||||
whose file fails to eval. The maintainer's `groups.py` was patched
|
||||
in commit `dc40295` to print the error; the node-loader prints
|
||||
on `nodes.py` errors via the same shape. Symptom either way:
|
||||
`bw nodes` lists fewer nodes than expected.
|
||||
|
||||
## Conventions
|
||||
|
||||
- **Filename = node name.** `home.server.py` defines the node `home.server`.
|
||||
- **Naming pattern: `<location>.<role>.py`.** Examples:
|
||||
`home.server.py`, `htz.mails.py`, `ovh.left4me.py`,
|
||||
`mseibert.freescout.py`.
|
||||
- **`*.py_` parks a node** without loading it. Used to keep
|
||||
decommissioned-but-not-deleted configs in tree (e.g.
|
||||
`htz.l4d2.py_`). The loader only matches `*.py`.
|
||||
- **Magic strings only resolve here.** `!password_for:` etc. work in
|
||||
node files; in groups, bundles, or items they don't — call
|
||||
`repo.vault.<verb>(...)` directly there.
|
||||
|
||||
## How to add a new node
|
||||
|
||||
1. Create `nodes/<location>.<role>.py` with a single dict expression:
|
||||
|
||||
```python
|
||||
{
|
||||
'id': 'a-uuid-or-stable-name',
|
||||
'hostname': '<dns-name-or-ip>',
|
||||
'groups': {
|
||||
'debian-13',
|
||||
'monitored',
|
||||
# …
|
||||
},
|
||||
'bundles': {
|
||||
# only bundles not provided by the groups above
|
||||
},
|
||||
'metadata': {
|
||||
# node-local overrides and required keys
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
2. Add to relevant `groups/<axis>/<x>.py` if group membership is the
|
||||
attachment point (preferred over per-node `bundles` lists).
|
||||
|
||||
3. Verify:
|
||||
- `bw nodes` — your node should appear.
|
||||
- `bw nodes <node> -a groups` — confirm group membership resolved
|
||||
as expected (`bw groups -n <node>` does **not** exist).
|
||||
- `bw metadata <node>` — confirm merged metadata.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **Renaming a node renames the node.** Vault entries (anything keyed
|
||||
on `!password_for:<node>`), `bw hash` records, and ssh known_hosts
|
||||
associations all key on node name. Search-and-replace before
|
||||
renaming, or vault lookups silently return new (wrong) values.
|
||||
- **Don't restore `_old` or `*.py_` files** without checking
|
||||
[`conventions.md#suspension-and-soft-delete-idioms`](../docs/agents/conventions.md#suspension-and-soft-delete-idioms).
|
||||
These are intentional parks/buffers, not bugs.
|
||||
- **`id` must be unique.** A pre-apply hook (`hooks/unique_node_ids.py`)
|
||||
enforces this; duplicate IDs fail `bw test` and `bw apply`.
|
||||
|
||||
## See also
|
||||
|
||||
- [`groups/AGENTS.md`](../groups/AGENTS.md) — group-membership patterns,
|
||||
how metadata merges along the chain.
|
||||
- [`docs/agents/conventions.md`](../docs/agents/conventions.md) —
|
||||
demagify magic-strings, naming, eval-loader constraints.
|
||||
- Fork's [`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
|
||||
— node attribute reference (`hostname`, `username`, `dummy`, etc.).
|
||||
|
|
@ -1,4 +1,4 @@
|
|||
bundlewrap ~=5.0, >=5.0.3
|
||||
-e git+https://github.com/CroneKorkN/bundlewrap.git@main#egg=bundlewrap
|
||||
pycryptodome
|
||||
PyNaCl
|
||||
PyYAML
|
||||
|
|
|
|||
Loading…
Reference in a new issue