docs: scaffold agent-friendly entry points (Phase 1)
introduces a balanced set of agent + human docs: - root AGENTS.md (with CLAUDE.md symlink) — 5-rule quickstart, layout map, mental model, use-case keyed example pointers. - docs/agents/conventions.md — vault/demagify, eval-loader constraints, group inheritance, naming, do-not-touch list, suspension idioms, working-style notes. - docs/agents/commands.md — repo-specific deltas to the fork's bw runbook (apt-key offline-verify, *.py_ suspended-node visibility, vault-echo rule). - per-area AGENTS.md for bundles/, nodes/, groups/, libs/, hooks/, data/, items/, bin/ — mechanism-focused, no enumeration. - bundles/AGENTS.template.md — per-bundle doc template with optional `## Writes into` section for cross-namespace reactors. bundlewrap-language reference (item types, dep keywords, reactors, runbook, three-tier safety envelope) is not duplicated here; we link out to the fork's AGENTS.md instead. bw test still green. all internal links resolve. Phase 0 invariants preserved (libs/hooks docstrings, bin/* # purpose: headers). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
730625e36c
commit
04558a9189
SSH key fingerprint:
SHA256:v0410ZKfuO1QHdgKBsdQNF64xmTxOF8osF1LIqwTcVw
13 changed files with 1009 additions and 0 deletions
106
AGENTS.md
Normal file
106
AGENTS.md
Normal file
|
|
@ -0,0 +1,106 @@
|
||||||
|
# ckn-bw — agent & contributor guide
|
||||||
|
|
||||||
|
## What this repo is
|
||||||
|
|
||||||
|
A [BundleWrap](https://bundlewrap.org/) configuration-management repo
|
||||||
|
for ~22 personal/family-infra nodes. Nodes, groups, and bundles are
|
||||||
|
defined in plain Python; `bw apply` deploys the resulting state to
|
||||||
|
real machines.
|
||||||
|
|
||||||
|
Note: the root `README.md` is the maintainer's personal scratchpad,
|
||||||
|
not project documentation. Onboarding lives **here**, in `AGENTS.md`.
|
||||||
|
|
||||||
|
## Quickstart for agents
|
||||||
|
|
||||||
|
Five rules; follow these and you won't break things:
|
||||||
|
|
||||||
|
1. **Read-only by default.** Never run `bw apply`, `bw run`, or
|
||||||
|
`bw lock` without explicit user request — even with `-i`. Stick
|
||||||
|
to `bw test`, `bw nodes`, `bw groups`, `bw bundles`,
|
||||||
|
`bw items`, `bw metadata`, `bw hash`, `bw debug`. See
|
||||||
|
[`docs/agents/commands.md`](docs/agents/commands.md) and the
|
||||||
|
fork's [safety envelope](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md).
|
||||||
|
2. **Never echo decrypted secrets.** Don't print, paste, or log the
|
||||||
|
value behind a `!password_for:`, `!decrypt:`, or
|
||||||
|
`!32_random_bytes_as_base64_for:` magic string — not even from
|
||||||
|
`bw debug` exploration. See
|
||||||
|
[`conventions.md#secrets`](docs/agents/conventions.md#secrets).
|
||||||
|
3. **Don't touch the do-not-modify list.** `.secrets.cfg*`, `.venv`,
|
||||||
|
`.cache`, `.bw_debug_history`, `.envrc`, root `README.md`. Treat
|
||||||
|
`hooks/` and `items/` (custom item types) with extra care: a
|
||||||
|
broken hook or item type breaks every `bw` command repo-wide.
|
||||||
|
4. **Use the fork.** The venv runs editable from
|
||||||
|
[`github.com/CroneKorkN/bundlewrap`](https://github.com/CroneKorkN/bundlewrap)
|
||||||
|
(branch `main`). Behavior tracks upstream `main`; the fork's
|
||||||
|
[`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
|
||||||
|
is the canonical bundlewrap-language reference. See
|
||||||
|
[`conventions.md#bundlewrap-version`](docs/agents/conventions.md#bundlewrap-version).
|
||||||
|
5. **Prefer adding helpers to `libs/`** over duplicating logic across
|
||||||
|
bundles. Repo-wide helpers go in
|
||||||
|
[`libs/`](libs/AGENTS.md), reachable as `repo.libs.<x>`.
|
||||||
|
|
||||||
|
## Layout
|
||||||
|
|
||||||
|
| Dir | What's there |
|
||||||
|
|---|---|
|
||||||
|
| [`bundles/`](bundles/AGENTS.md) | 103 bundles. One subdir per bundle (`items.py`, `metadata.py`, `files/`). |
|
||||||
|
| [`nodes/`](nodes/AGENTS.md) | One file per node (~22). `eval()`-loaded; demagified through `repo.vault`. |
|
||||||
|
| [`groups/`](groups/AGENTS.md) | Group definitions, organized by axis (`applications/`, `locations/`, `machine/`, `os/`). |
|
||||||
|
| [`libs/`](libs/AGENTS.md) | Shared Python helpers reachable as `repo.libs.<modulename>`. |
|
||||||
|
| [`hooks/`](hooks/AGENTS.md) | bw lifecycle hooks (`apply_start`, `test`, `node_apply_start`, …). |
|
||||||
|
| [`data/`](data/AGENTS.md) | Out-of-bundle data assets (apt keys, grafana dashboards, …). |
|
||||||
|
| [`items/`](items/AGENTS.md) | Custom item types (currently `download:`). |
|
||||||
|
| [`bin/`](bin/AGENTS.md) | Operator scripts; not invoked by bundlewrap. |
|
||||||
|
| [`docs/agents/`](docs/agents/conventions.md) | Repo conventions and command deltas. |
|
||||||
|
|
||||||
|
## How nodes, groups, and bundles fit together
|
||||||
|
|
||||||
|
- A **node** (`nodes/<location>.<role>.py`) declares the groups it
|
||||||
|
belongs to and any node-local bundles + metadata overrides.
|
||||||
|
- A **group** (`groups/<axis>/<x>.py`) attaches bundles and shared
|
||||||
|
metadata to its members. Groups inherit via `supergroups`.
|
||||||
|
- A **bundle** (`bundles/<x>/`) is one chunk of configuration:
|
||||||
|
`items.py` produces the items (files, services, packages),
|
||||||
|
`metadata.py` declares `defaults` and `@metadata_reactor` functions
|
||||||
|
that derive metadata from other metadata.
|
||||||
|
- The repo-root loaders (`nodes.py`, `groups.py`) walk these dirs and
|
||||||
|
`eval()` each file. `nodes.py` additionally **demagifies** the
|
||||||
|
result, resolving `!password_for:` etc. through `repo.vault`. See
|
||||||
|
[`conventions.md#eval-loaded-node-and-group-files`](docs/agents/conventions.md#eval-loaded-node-and-group-files)
|
||||||
|
for the constraints this places on editors.
|
||||||
|
- Metadata merges along: `all → location → os → machine →
|
||||||
|
applications → node`.
|
||||||
|
|
||||||
|
## Conventions you must know
|
||||||
|
|
||||||
|
| Topic | Where |
|
||||||
|
|---|---|
|
||||||
|
| Bundlewrap-language reference (item types, dep keywords, reactors) | Fork's [`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md) — read first if new to bundlewrap |
|
||||||
|
| Vault / demagify magic strings | [`conventions.md#secrets`](docs/agents/conventions.md#secrets) |
|
||||||
|
| Bundlewrap install (editable from the fork) | [`conventions.md#bundlewrap-version`](docs/agents/conventions.md#bundlewrap-version) |
|
||||||
|
| Group inheritance order, naming patterns | [`conventions.md#group-inheritance-order`](docs/agents/conventions.md#group-inheritance-order), [`#naming-conventions`](docs/agents/conventions.md#naming-conventions) |
|
||||||
|
| Repo-specific bw command deltas (apt keys, suspended nodes, vault echo) | [`commands.md`](docs/agents/commands.md) |
|
||||||
|
| Lib helpers | top-of-file docstrings in `libs/*.py` (`head -1 libs/*.py`) |
|
||||||
|
| Suspension idioms (`*.py_`, `_old/`, "for now") | [`conventions.md#suspension-and-soft-delete-idioms`](docs/agents/conventions.md#suspension-and-soft-delete-idioms) |
|
||||||
|
|
||||||
|
## Where to look for examples
|
||||||
|
|
||||||
|
When writing a new bundle, copy patterns from one that already does
|
||||||
|
the thing you need:
|
||||||
|
|
||||||
|
| Pattern | Look at |
|
||||||
|
|---|---|
|
||||||
|
| Vault calls inside metadata reactors | `bundles/dm-crypt/metadata.py` (compact, focused) |
|
||||||
|
| Mako-templated files | `bundles/bind/items.py` (DNS zonefile rendering) |
|
||||||
|
| Cross-bundle reactor writing | `bundles/nextcloud/metadata.py` (writes into `apt.packages`, `archive.paths`) |
|
||||||
|
| Custom `download:` items | `bundles/minecraft/items.py` |
|
||||||
|
| Node file (single-purpose) | `nodes/home.server.py` |
|
||||||
|
| Group with `supergroups` chain | `groups/os/debian-13.py` |
|
||||||
|
|
||||||
|
## Where this doc lives
|
||||||
|
|
||||||
|
- This file: `AGENTS.md` at the repo root.
|
||||||
|
- `CLAUDE.md` is a symlink to this file — both names point to the same
|
||||||
|
content so different tools can find it.
|
||||||
|
- The personal TODO scratchpad (`README.md`) is **separate** and not
|
||||||
|
project documentation.
|
||||||
1
CLAUDE.md
Symbolic link
1
CLAUDE.md
Symbolic link
|
|
@ -0,0 +1 @@
|
||||||
|
AGENTS.md
|
||||||
62
bin/AGENTS.md
Normal file
62
bin/AGENTS.md
Normal file
|
|
@ -0,0 +1,62 @@
|
||||||
|
# bin/
|
||||||
|
|
||||||
|
## What's here
|
||||||
|
|
||||||
|
Operator scripts — invoked manually by the maintainer, **not** by
|
||||||
|
bundlewrap itself. Each is a standalone Python (or shell) script that
|
||||||
|
opens the repo via `Repository(dirname(dirname(realpath(__file__))))`.
|
||||||
|
|
||||||
|
Discovery is by `ls bin/` plus the `# purpose:` header line at the top
|
||||||
|
of each script:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
head -2 bin/*
|
||||||
|
```
|
||||||
|
|
||||||
|
## Conventions
|
||||||
|
|
||||||
|
- **`# purpose:` header.** Every script under `bin/` starts with
|
||||||
|
`#!/usr/bin/env python3` (or appropriate shebang), then a
|
||||||
|
`# purpose: <one-line description>` comment. Baseline enforced by
|
||||||
|
`grep -L '^# purpose' bin/*`.
|
||||||
|
- **Self-contained.** A script must work when run from anywhere — it
|
||||||
|
resolves the repo via the script's own path, not `cwd`.
|
||||||
|
- **Read-only by default.** Most operator scripts query/print state
|
||||||
|
(`passwords-for`, `wireguard-client-config`). Mutating scripts
|
||||||
|
(`upgrade_and_restart_all`, `mikrotik-firmware-updater`,
|
||||||
|
`sync_1password`) are the exception, not the rule, and prompt for
|
||||||
|
confirmation.
|
||||||
|
|
||||||
|
## How to add a script
|
||||||
|
|
||||||
|
1. Start from [`bin/script_template`](script_template) — it carries
|
||||||
|
the canonical shebang + `# purpose:` header + `Repository(...)`
|
||||||
|
bootstrap.
|
||||||
|
2. Add the `# purpose:` line; lowercase, terse, include a `usage:`
|
||||||
|
example if the script takes arguments.
|
||||||
|
3. `chmod +x bin/<name>`.
|
||||||
|
4. The script can reach helpers via `bw.libs.<x>` exactly like a
|
||||||
|
bundle does.
|
||||||
|
|
||||||
|
## Pitfalls
|
||||||
|
|
||||||
|
- **`bin/` is not on `$PATH` by default.** Invoke as `bin/<name>` from
|
||||||
|
the repo root, or via `direnv` if `.envrc` exposes it.
|
||||||
|
- **Mutating scripts can hit Tier-3 territory** (per the fork's
|
||||||
|
safety envelope). Don't run `upgrade_and_restart_all`,
|
||||||
|
`mikrotik-firmware-updater`, or anything that does `node.run(...)`
|
||||||
|
without explicit user instruction. See the fork's
|
||||||
|
[`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
|
||||||
|
for the three-tier model.
|
||||||
|
- **Vault echo.** Scripts like `passwords-for` print decrypted values
|
||||||
|
by design; that's allowed for the human at the terminal but *not*
|
||||||
|
for the agent — never paste output into chat, ticket, or PR
|
||||||
|
description.
|
||||||
|
|
||||||
|
## See also
|
||||||
|
|
||||||
|
- [`script_template`](script_template) — canonical starter.
|
||||||
|
- [`docs/agents/conventions.md`](../docs/agents/conventions.md) —
|
||||||
|
vault rules.
|
||||||
|
- [`docs/agents/commands.md`](../docs/agents/commands.md) — read-only
|
||||||
|
bw-command guidance.
|
||||||
124
bundles/AGENTS.md
Normal file
124
bundles/AGENTS.md
Normal file
|
|
@ -0,0 +1,124 @@
|
||||||
|
# bundles/
|
||||||
|
|
||||||
|
## Before you start
|
||||||
|
|
||||||
|
Read [`docs/agents/conventions.md`](../docs/agents/conventions.md) first
|
||||||
|
— it covers vault calls, demagify, the `repo.libs.<x>` helpers, and the
|
||||||
|
files agents must not modify. Skipping it leads to subtly broken bundles
|
||||||
|
(vault calls in the wrong place, dict-in-set `TypeError` because of
|
||||||
|
unhashable nesting, etc.).
|
||||||
|
|
||||||
|
For bundlewrap-language reference (item types, dep keywords,
|
||||||
|
`metadata_reactor`, `defaults`, item-file template syntax) see the fork's
|
||||||
|
[`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
|
||||||
|
and its [`docs/content/guide/item_file_templates.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/docs/content/guide/item_file_templates.md).
|
||||||
|
|
||||||
|
## What's here
|
||||||
|
|
||||||
|
103 bundles. Each is a directory `bundles/<name>/` containing some of:
|
||||||
|
|
||||||
|
```
|
||||||
|
bundles/<name>/
|
||||||
|
├── items.py # the items this bundle creates (files, services, packages, …)
|
||||||
|
├── metadata.py # `defaults` + `@metadata_reactor` functions
|
||||||
|
├── files/ # static or templated file payloads referenced from items.py
|
||||||
|
├── AGENTS.md # this bundle's doc (template at AGENTS.template.md)
|
||||||
|
└── README.md # legacy; being phased out (see "Documentation transition" below)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Conventions
|
||||||
|
|
||||||
|
- **Bundle names** are lowercase, hyphen-separated: `backup-server`,
|
||||||
|
`bind-acme`, `dm-crypt`. No underscores in new bundle names — see
|
||||||
|
[`conventions.md#naming-conventions`](../docs/agents/conventions.md#naming-conventions).
|
||||||
|
- **`items.py`** is plain Python; it produces `files = {...}`,
|
||||||
|
`pkg_apt = {...}`, `svc_systemd = {...}`, etc. dicts at module scope.
|
||||||
|
Cross-item dependencies use `needs` / `triggers` / `triggered_by` —
|
||||||
|
see the fork's `AGENTS.md` for the full keyword cheat sheet.
|
||||||
|
- **`metadata.py`** uses `defaults = {...}` for static seed values and
|
||||||
|
`@metadata_reactor.provides(...)` for derived values. Reactors are
|
||||||
|
pure functions of `(metadata,)` — no side effects, no I/O.
|
||||||
|
- **Helpers go in [`libs/`](../libs/AGENTS.md)** when they're useful to
|
||||||
|
more than one bundle. Don't duplicate logic across bundles.
|
||||||
|
- **Custom item types** (e.g. `download:`) live in
|
||||||
|
[`items/`](../items/AGENTS.md), not per-bundle.
|
||||||
|
|
||||||
|
## How to add a new bundle
|
||||||
|
|
||||||
|
1. `mkdir bundles/<name>/` (lowercase, hyphenated).
|
||||||
|
2. Write `items.py` and (if anything is configurable) `metadata.py`.
|
||||||
|
Use `repo.libs.hashable.hashable(...)` when you need to nest a dict
|
||||||
|
or set inside a metadata set; raw dicts/sets aren't hashable.
|
||||||
|
3. Drop static payloads into `bundles/<name>/files/`. For Mako-templated
|
||||||
|
files, declare `'content_type': 'mako'` on the `file:` item — see
|
||||||
|
the fork's
|
||||||
|
[item-file-templates guide](https://github.com/CroneKorkN/bundlewrap/blob/main/docs/content/guide/item_file_templates.md).
|
||||||
|
4. **Wire to nodes.** Either add an entry to the relevant
|
||||||
|
[`groups/<axis>/<x>.py`](../groups/AGENTS.md) (preferred for shared
|
||||||
|
bundles) or to the node's `bundles` list directly
|
||||||
|
([`nodes/AGENTS.md`](../nodes/AGENTS.md)).
|
||||||
|
5. Verify, in this order:
|
||||||
|
- `bw test` — sanity (loaders + reactors).
|
||||||
|
- `bw items <node>` — confirm new items appear on a node that opts in.
|
||||||
|
- `bw hash <node>` — confirm the change is what you expected. See
|
||||||
|
[`docs/agents/commands.md`](../docs/agents/commands.md) and the
|
||||||
|
fork's hash-diff workflow.
|
||||||
|
6. Create `bundles/<name>/AGENTS.md` from
|
||||||
|
[`AGENTS.template.md`](AGENTS.template.md). For a brand-new bundle
|
||||||
|
without consumers yet, leave `Depends on` and `Produces` empty or
|
||||||
|
marked TBD; fill them in after the first verify run.
|
||||||
|
|
||||||
|
## How to remove a bundle
|
||||||
|
|
||||||
|
1. `git grep '<name>'` in `nodes/`, `groups/`, and other `bundles/` to
|
||||||
|
find references.
|
||||||
|
2. Remove those references.
|
||||||
|
3. `rm -rf bundles/<name>/`.
|
||||||
|
4. `bw test` and `bw nodes` to confirm clean.
|
||||||
|
|
||||||
|
## Pitfalls
|
||||||
|
|
||||||
|
- **`metadata.py` is evaluated at load time** for *every* node, every
|
||||||
|
invocation of `bw`. Heavy work or I/O slows the whole repo. Keep
|
||||||
|
reactors pure and fast; pre-compute in `libs/` if you must.
|
||||||
|
- **Static files vs templates.** `bundles/<x>/files/<f>` is static
|
||||||
|
unless the matching `file:` item declares `content_type='mako'`
|
||||||
|
(or a templating extension triggers it). To check, read the matching
|
||||||
|
`file:` entry in `items.py`.
|
||||||
|
- **Reactors writing across namespaces.** Some bundles' reactors write
|
||||||
|
into other bundles' metadata namespaces (e.g. `nextcloud` writes
|
||||||
|
into `apt.packages`, `archive.paths`). When you change such a bundle,
|
||||||
|
every consumer's metadata changes too. Per-bundle docs declare these
|
||||||
|
in an optional `## Writes into` section — read it before assuming the
|
||||||
|
blast radius is local.
|
||||||
|
- **`bw hash` doesn't accept selectors.** Use `bw hash <node>` per
|
||||||
|
literal name; see the fork's runbook.
|
||||||
|
|
||||||
|
## Documentation transition
|
||||||
|
|
||||||
|
This repo is migrating from bundle `README.md` files to per-bundle
|
||||||
|
`AGENTS.md` files (one balanced doc per bundle, agents + humans).
|
||||||
|
|
||||||
|
- Where both exist, **`AGENTS.md` is canonical**; the `README.md` is
|
||||||
|
being phased out.
|
||||||
|
- ~28 bundle READMEs survive after the seed migration (the seed PR
|
||||||
|
folds in 5–10 of them; the rest are addressed lazily on the next
|
||||||
|
material edit — Phase 3 leave-as-you-go).
|
||||||
|
- Phase-3 rule: any time you (or any agent) materially edits a bundle,
|
||||||
|
top-up its `AGENTS.md` or create one from
|
||||||
|
[`AGENTS.template.md`](AGENTS.template.md). If a stale `README.md`
|
||||||
|
is still around in the bundle, fold it in and remove it in the same
|
||||||
|
commit.
|
||||||
|
|
||||||
|
## See also
|
||||||
|
|
||||||
|
- [`AGENTS.template.md`](AGENTS.template.md) — per-bundle doc template.
|
||||||
|
- [`docs/agents/conventions.md`](../docs/agents/conventions.md) — repo
|
||||||
|
idioms (vault, demagify, naming, do-not-touch list).
|
||||||
|
- [`docs/agents/commands.md`](../docs/agents/commands.md) — repo-specific
|
||||||
|
command deltas.
|
||||||
|
- [`items/AGENTS.md`](../items/AGENTS.md) — custom item types
|
||||||
|
(`download:`); when to write a new one vs use `file:`.
|
||||||
|
- [`libs/AGENTS.md`](../libs/AGENTS.md) — shared helpers.
|
||||||
|
- Fork's [`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
|
||||||
|
— bundlewrap-language reference + safety envelope.
|
||||||
61
bundles/AGENTS.template.md
Normal file
61
bundles/AGENTS.template.md
Normal file
|
|
@ -0,0 +1,61 @@
|
||||||
|
# <bundle-name>
|
||||||
|
|
||||||
|
<!--
|
||||||
|
Per-bundle doc template. Copy to `bundles/<name>/AGENTS.md` and fill in.
|
||||||
|
|
||||||
|
For a brand-new bundle without consumers yet, leave `Depends on` and
|
||||||
|
`Produces` empty or marked TBD; fill them in after the first verify run.
|
||||||
|
|
||||||
|
Skip the `## Gotchas` section if there are no real gotchas — empty
|
||||||
|
"Gotchas: none" sections are noise.
|
||||||
|
|
||||||
|
Skip the `## Writes into` section unless this bundle's `defaults` or
|
||||||
|
reactors write into other bundles' metadata namespaces. Most don't.
|
||||||
|
-->
|
||||||
|
|
||||||
|
<1–3 sentences: what this bundle does and when you'd use it.>
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
<How to apply: which group(s) typically include it, or how a node opts
|
||||||
|
in. Minimal example of node metadata if any keys are required.>
|
||||||
|
|
||||||
|
## Metadata
|
||||||
|
|
||||||
|
Keys read from `node.metadata`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
{
|
||||||
|
'<bundle>': {
|
||||||
|
'key': 'value', # str, required — short note
|
||||||
|
'flag': True, # bool, default True
|
||||||
|
'list': [], # list[str], default [] — short note
|
||||||
|
'nested': {
|
||||||
|
'subkey': 0, # int, default 0
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Produces
|
||||||
|
|
||||||
|
<Items created: files, services, packages, users, etc. One line each.
|
||||||
|
Skip if trivially obvious from items.py.>
|
||||||
|
|
||||||
|
## Depends on
|
||||||
|
|
||||||
|
<Other bundles required, or "none". Note ordering quirks if any.>
|
||||||
|
|
||||||
|
## Writes into
|
||||||
|
|
||||||
|
<!-- Optional. Most bundles don't write cross-namespace; skip the
|
||||||
|
section if this bundle doesn't either. List the foreign metadata
|
||||||
|
namespaces this bundle's `defaults` or reactors populate (e.g.
|
||||||
|
`apt.packages`, `archive.paths`). Cross-namespace writes are the
|
||||||
|
single most surprising blast-radius source in this repo — call them
|
||||||
|
out explicitly. -->
|
||||||
|
|
||||||
|
## Gotchas
|
||||||
|
|
||||||
|
<Non-obvious behavior, manual one-time steps, known pitfalls. Omit
|
||||||
|
the section if there are none.>
|
||||||
63
data/AGENTS.md
Normal file
63
data/AGENTS.md
Normal file
|
|
@ -0,0 +1,63 @@
|
||||||
|
# data/
|
||||||
|
|
||||||
|
## What's here
|
||||||
|
|
||||||
|
Out-of-bundle data assets consumed by one or more bundles. Each subdir
|
||||||
|
maps to a consumer:
|
||||||
|
|
||||||
|
```
|
||||||
|
data/
|
||||||
|
├── apt/keys/ # binary GPG keys for apt sources
|
||||||
|
├── grafana/rows/ # Mako-templated dashboard panels (Python)
|
||||||
|
├── nginx/ # nginx snippets shared across vhosts
|
||||||
|
├── homeassistant/, mailman/, nextcloud/, ...
|
||||||
|
└── network.py # repo-wide network metadata (one-off file)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Two distinct content models
|
||||||
|
|
||||||
|
Same directory shape, different content kinds. When you add a new
|
||||||
|
`data/<x>/` subdir, declare which model it follows:
|
||||||
|
|
||||||
|
| Model | Example | Consumer |
|
||||||
|
|---|---|---|
|
||||||
|
| **Binary / static** | `data/apt/keys/*.{asc,gpg}` | `bundles/apt` reads files at apply time |
|
||||||
|
| **Python module / template** | `data/grafana/rows/*.py`, `data/routeros-monitoring/*.py` | bundle `import`s and renders |
|
||||||
|
|
||||||
|
If a data asset is read by **exactly one bundle**, prefer
|
||||||
|
`bundles/<x>/files/` instead of `data/<x>/`. `data/` is for
|
||||||
|
shared / multi-consumer artifacts. Single-instance evidence: commit
|
||||||
|
`78a8abc` moved `mikrotik.mib` *out* of `data/` *into* the bundle for
|
||||||
|
this reason.
|
||||||
|
|
||||||
|
## Conventions
|
||||||
|
|
||||||
|
- **One subdir per consumer.** Subdir name = consumer bundle name
|
||||||
|
(`data/apt/`, `data/nextcloud/`).
|
||||||
|
- **`network.py` exception.** A single file at `data/network.py` holds
|
||||||
|
repo-wide network metadata; it doesn't belong to one bundle. Treat
|
||||||
|
it as cross-cutting infrastructure metadata.
|
||||||
|
|
||||||
|
## How to add data
|
||||||
|
|
||||||
|
1. Decide the content model (binary or Python).
|
||||||
|
2. `mkdir data/<consumer>/`.
|
||||||
|
3. Drop assets in.
|
||||||
|
4. The consumer bundle (`bundles/<consumer>/items.py` or
|
||||||
|
`metadata.py`) reads them via `repo.path` + `os.path.join` or
|
||||||
|
similar.
|
||||||
|
|
||||||
|
## Pitfalls
|
||||||
|
|
||||||
|
- **Apt keys** trigger an offline-verify rule before they're committed.
|
||||||
|
See [`commands.md#apt-key-changes-need-offline-verification`](../docs/agents/commands.md#apt-key-changes-need-offline-verification).
|
||||||
|
- **Mako-templated Python data** evaluates at bundle render time. Side
|
||||||
|
effects in those modules slow the whole repo (same caveat as
|
||||||
|
[`libs/`](../libs/AGENTS.md)).
|
||||||
|
|
||||||
|
## See also
|
||||||
|
|
||||||
|
- [`bundles/AGENTS.md`](../bundles/AGENTS.md) — when bundle `files/`
|
||||||
|
beats `data/`.
|
||||||
|
- [`docs/agents/commands.md`](../docs/agents/commands.md) — apt-key
|
||||||
|
verification rule.
|
||||||
50
docs/agents/commands.md
Normal file
50
docs/agents/commands.md
Normal file
|
|
@ -0,0 +1,50 @@
|
||||||
|
# Commands
|
||||||
|
|
||||||
|
The canonical bw-command runbook — read-only allowlist, three-tier
|
||||||
|
safety envelope, after-change table, hash-diff workflow, `bw debug`
|
||||||
|
sketch — lives in the fork's
|
||||||
|
[`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md).
|
||||||
|
Read that first.
|
||||||
|
|
||||||
|
This file collects only the deltas specific to `ckn-bw`.
|
||||||
|
|
||||||
|
## Apt-key changes need offline verification
|
||||||
|
|
||||||
|
Editing files under `data/apt/keys/*.{asc,gpg}` rotates a signing key
|
||||||
|
the whole apt subsystem trusts. Trial-and-error with `bw apply` is the
|
||||||
|
*failure* path: a wrong key blocks unattended upgrades cluster-wide
|
||||||
|
until corrected manually.
|
||||||
|
|
||||||
|
Before touching `data/apt/keys/`:
|
||||||
|
|
||||||
|
1. Fetch the new key from its upstream source (project release page,
|
||||||
|
`keys.openpgp.org`, etc.).
|
||||||
|
2. `gpg --show-keys <newkey>` — print the fingerprint.
|
||||||
|
3. Diff against the fingerprint published by the upstream source.
|
||||||
|
4. Only after the fingerprint matches, place the file under
|
||||||
|
`data/apt/keys/` and let `bundles/apt` consume it on the next
|
||||||
|
apply.
|
||||||
|
|
||||||
|
## `*.py_` suspended nodes are invisible to `bw nodes`
|
||||||
|
|
||||||
|
The repo loader (`nodes.py`) only matches files ending in `.py`. Files
|
||||||
|
ending in `.py_` are silently skipped. If `bw nodes` reports a node
|
||||||
|
missing, check whether its file has been parked:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
ls nodes/ | grep '\.py_$'
|
||||||
|
```
|
||||||
|
|
||||||
|
This is the [suspension idiom](conventions.md#suspension-and-soft-delete-idioms),
|
||||||
|
not a bug.
|
||||||
|
|
||||||
|
## Vault output never leaves the terminal
|
||||||
|
|
||||||
|
The fork's runbook calls out that `bw debug` resolves vault magic
|
||||||
|
strings transparently. In `ckn-bw` specifically: never echo, log, or
|
||||||
|
paste decrypted values, even from a `bw debug -c` one-liner. If you
|
||||||
|
need to verify a secret resolved correctly, hash or fingerprint it
|
||||||
|
instead.
|
||||||
|
|
||||||
|
See [`conventions.md#secrets`](conventions.md#secrets) for the
|
||||||
|
demagify magic-string list and the rule's full rationale.
|
||||||
146
docs/agents/conventions.md
Normal file
146
docs/agents/conventions.md
Normal file
|
|
@ -0,0 +1,146 @@
|
||||||
|
# Conventions
|
||||||
|
|
||||||
|
Repo-specific idioms an agent has to know before editing this BundleWrap
|
||||||
|
config. For bundlewrap-the-language reference (item types, dep keywords,
|
||||||
|
metadata-reactor semantics, after-change runbook), see the fork's
|
||||||
|
[`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md).
|
||||||
|
|
||||||
|
## Secrets
|
||||||
|
|
||||||
|
Secrets live in `.secrets.cfg` and are referenced from node files by
|
||||||
|
"demagify" magic strings. The loader (`nodes.py`) walks node dicts and
|
||||||
|
resolves any leaf string of the form `!<verb>:<arg>`.
|
||||||
|
|
||||||
|
| Magic string | Resolves to |
|
||||||
|
|---|---|
|
||||||
|
| `!password_for:<id>` | `repo.vault.password_for(id)` |
|
||||||
|
| `!decrypt:<ciphertext>` | `repo.vault.decrypt(ciphertext)` |
|
||||||
|
| `!decrypt_file:<path>` | `repo.vault.decrypt_file(path)` |
|
||||||
|
| `!32_random_bytes_as_base64_for:<id>` | `repo.vault.random_bytes_as_base64_for(id, length=32)` |
|
||||||
|
|
||||||
|
Magic strings are only resolved inside **node files** (everything under
|
||||||
|
`nodes/`). They are *not* resolved in group files, bundle metadata
|
||||||
|
defaults, or item attributes — call `repo.vault.<verb>(...)` directly
|
||||||
|
there.
|
||||||
|
|
||||||
|
**Never echo decrypted values.** Don't print, log, or paste them, even
|
||||||
|
in `bw debug` exploration. If you need a sanity check, hash or
|
||||||
|
fingerprint instead. This applies to AI agents and humans equally.
|
||||||
|
|
||||||
|
## Bundlewrap version
|
||||||
|
|
||||||
|
The venv runs **editable** from the maintainer's fork:
|
||||||
|
|
||||||
|
```
|
||||||
|
-e git+https://github.com/CroneKorkN/bundlewrap.git@main#egg=bundlewrap
|
||||||
|
```
|
||||||
|
|
||||||
|
Pinned in `requirements.txt`. The fork's `main` tracks upstream
|
||||||
|
`bundlewrap/bundlewrap` master; the fork's [`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
|
||||||
|
is the canonical agent-oriented bundlewrap-language reference.
|
||||||
|
|
||||||
|
To pull upstream changes into the venv:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
(cd .venv/src/bundlewrap && git pull)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Eval-loaded node and group files
|
||||||
|
|
||||||
|
`nodes/*.py` and `groups/*.py` are loaded by `eval()` in `nodes.py` /
|
||||||
|
`groups.py` (top of the repo). Each file must be a single Python
|
||||||
|
expression — typically a dict literal.
|
||||||
|
|
||||||
|
Consequences:
|
||||||
|
|
||||||
|
- **No top-level imports.** No `from foo import bar`, no `import os`.
|
||||||
|
Helpers go in `libs/` and are reached via `repo.libs.<x>` from
|
||||||
|
bundle code, not from node files.
|
||||||
|
- **No statements.** No `if`/`for`/`def` at the top level. Use
|
||||||
|
expressions (ternaries, comprehensions) instead.
|
||||||
|
- **Silent drop on parse error.** Vanilla bundlewrap silently omits a
|
||||||
|
node/group whose file fails to eval. The maintainer patched
|
||||||
|
`groups.py` (commit `dc40295`) to print the error instead — but it
|
||||||
|
still skips the file. Symptom: `bw nodes` lists fewer nodes than you
|
||||||
|
expect. Cure: re-read the file and check for accidental imports or
|
||||||
|
syntax errors.
|
||||||
|
|
||||||
|
## Group inheritance order
|
||||||
|
|
||||||
|
Metadata merges along this chain (from the fork's docs):
|
||||||
|
|
||||||
|
```
|
||||||
|
all → location → os → machine → applications → node
|
||||||
|
```
|
||||||
|
|
||||||
|
`groups/all.py` is the universal base; `groups/{locations,os,machine,applications}/`
|
||||||
|
hold the per-axis groups; the node's own metadata wins last.
|
||||||
|
|
||||||
|
## Naming conventions
|
||||||
|
|
||||||
|
| Where | Convention | Example |
|
||||||
|
|---|---|---|
|
||||||
|
| `nodes/*.py` | `<location>.<role>.py` | `home.server.py`, `htz.mails.py` |
|
||||||
|
| `groups/*.py` | one file per group; subdir = purpose | `groups/applications/nextcloud.py`, `groups/os/debian-13.py` |
|
||||||
|
| `bundles/<name>` | lowercase, hyphen-separated | `backup-server`, `bind-acme`, `routeros-monitoring` |
|
||||||
|
| Custom items | `items/<type>.py` | `items/download.py` |
|
||||||
|
|
||||||
|
Underscores in bundle names appear only in `_old`-suffixed leftovers
|
||||||
|
(see below); don't introduce new ones.
|
||||||
|
|
||||||
|
## Suspension and soft-delete idioms
|
||||||
|
|
||||||
|
These conventions look like dead code; they aren't. Don't clean them up.
|
||||||
|
|
||||||
|
- **`*.py_` parked node.** A node file whose name ends in `.py_` is
|
||||||
|
silently excluded from `bw nodes` (the loader only matches `*.py`).
|
||||||
|
Used to keep decommissioned-but-not-deleted node configs in tree.
|
||||||
|
Example: `nodes/htz.l4d2.py_`.
|
||||||
|
- **"for now / disable / dummy / offline" markers.** Commits or comments
|
||||||
|
containing these phrases mark deliberate suspensions, not bugs. Check
|
||||||
|
`git log -- <file>` before re-enabling — the maintainer reverses
|
||||||
|
these manually when the underlying condition resolves.
|
||||||
|
- **`_old` / `_old2` directories.** Recovery buffers during big
|
||||||
|
refactors. Don't delete them without asking, even if they look
|
||||||
|
orphaned.
|
||||||
|
|
||||||
|
## Files agents must not modify
|
||||||
|
|
||||||
|
| Path | Why |
|
||||||
|
|---|---|
|
||||||
|
| `.secrets.cfg*` | vault key material |
|
||||||
|
| `.venv/` | local Python environment |
|
||||||
|
| `.cache/` | bw runtime cache |
|
||||||
|
| `.bw_debug_history` | shell history for `bw debug` |
|
||||||
|
| `.envrc` | direnv local-environment config |
|
||||||
|
| `/.cocoindex_code/` | local code index, not in git |
|
||||||
|
| `README.md` (root) | maintainer's personal TODO scratchpad — not project docs |
|
||||||
|
|
||||||
|
Treat `hooks/` and `items/` (custom item types) with extra care: they
|
||||||
|
affect `bw`'s behavior for the whole repo, not a single bundle.
|
||||||
|
|
||||||
|
## Working style
|
||||||
|
|
||||||
|
- **Iterative commits are normal.** The maintainer commits in small
|
||||||
|
checkpoints (`+`, `fix`, `whitespace`, terse one-liners). Don't
|
||||||
|
rebase WIP branches without asking. As an agent, prefer
|
||||||
|
complete-feeling commits over mimicking the iterative style.
|
||||||
|
- **Burst-state awareness.** Before writing into a subsystem, run
|
||||||
|
`git log --since='1 month ago' bundles/<x>` (or `nodes/<x>.py`).
|
||||||
|
≥10 recent commits means the subsystem is in flux; read the most
|
||||||
|
recent diffs first — your assumptions about its metadata shape may
|
||||||
|
already be stale.
|
||||||
|
- **Branch names.** PRs go through self-hosted Gitea/Forgejo. Branch
|
||||||
|
names are lowercase snake_case, descriptive
|
||||||
|
(`debian-13`, `htz.mails_debian_13_squash`, `l4d2_the_next`).
|
||||||
|
|
||||||
|
## Bundlewrap-version migration recipe
|
||||||
|
|
||||||
|
When the next bw major lands:
|
||||||
|
|
||||||
|
1. Read the upstream migration guide.
|
||||||
|
2. `git grep` for affected reactor / item patterns.
|
||||||
|
3. Rewrite each (one commit per pattern is fine — see Working style).
|
||||||
|
4. Bump `requirements.txt` last.
|
||||||
|
|
||||||
|
Pattern from commit `186d503` (bw 4 → 5).
|
||||||
110
groups/AGENTS.md
Normal file
110
groups/AGENTS.md
Normal file
|
|
@ -0,0 +1,110 @@
|
||||||
|
# groups/
|
||||||
|
|
||||||
|
## What's here
|
||||||
|
|
||||||
|
Groups attach bundles and shared metadata to nodes. One file per group,
|
||||||
|
organized by axis:
|
||||||
|
|
||||||
|
```
|
||||||
|
groups/
|
||||||
|
├── all.py # universal base (every node belongs)
|
||||||
|
├── applications/<x>.py # role-shaped groups (mailserver, monitored, …)
|
||||||
|
├── locations/<x>.py # physical/network location (home, htz, …)
|
||||||
|
├── machine/<x>.py # hardware kind (hardware, hetzner-cloud, raspberry-pi)
|
||||||
|
└── os/<x>.py # OS major/variant (debian-13, debian-13-pve, routeros, …)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Loader mechanism
|
||||||
|
|
||||||
|
`groups.py` (top of the repo) walks `groups/` and runs `eval()` on each
|
||||||
|
`*.py`. Same eval-as-expression rule as
|
||||||
|
[`nodes/`](../nodes/AGENTS.md#loader-mechanism): one dict literal, no
|
||||||
|
top-level imports, no statements. Errors print and the group is skipped
|
||||||
|
— a real foot-gun, since a missing group silently changes node
|
||||||
|
membership.
|
||||||
|
|
||||||
|
**Group files are *not* demagified.** Magic strings like
|
||||||
|
`!password_for:<x>` only resolve in `nodes/*.py`. Inside a group file,
|
||||||
|
call `repo.vault.<verb>(...)` directly.
|
||||||
|
|
||||||
|
## Inheritance and merge order
|
||||||
|
|
||||||
|
Metadata merges along this chain:
|
||||||
|
|
||||||
|
```
|
||||||
|
all → location → os → machine → applications → node
|
||||||
|
```
|
||||||
|
|
||||||
|
Per-axis subdirs are conventional, not enforced — `bw` doesn't read the
|
||||||
|
subdir. Each group lists its `supergroups`, and `bw` resolves the DAG.
|
||||||
|
Membership is set-union; metadata merge follows the order above, with
|
||||||
|
the node's own `metadata` block winning last.
|
||||||
|
|
||||||
|
## Conventions
|
||||||
|
|
||||||
|
- **One group per file.** Filename without `.py` = group name. Subdir
|
||||||
|
groups them by axis for humans, not for bw.
|
||||||
|
- **Family files for OS variants.** Common parent + per-variant child.
|
||||||
|
Example: `debian-13-common.py` is shared by `debian-13.py` and
|
||||||
|
`debian-13-pve.py`. Use this pattern when introducing
|
||||||
|
related-but-distinct OS group families.
|
||||||
|
- **`all.py` is the universal default.** Currently empty (`{}`); kept
|
||||||
|
for the rare repo-wide opt-in.
|
||||||
|
|
||||||
|
## How to add a group
|
||||||
|
|
||||||
|
1. Pick the right axis subdir (or root `all.py` for universal default).
|
||||||
|
2. Create `groups/<axis>/<name>.py` as a single dict expression:
|
||||||
|
|
||||||
|
```python
|
||||||
|
{
|
||||||
|
'supergroups': [
|
||||||
|
# parent groups whose bundles/metadata this one extends
|
||||||
|
],
|
||||||
|
'bundles': [
|
||||||
|
# bundles every member of this group should have
|
||||||
|
],
|
||||||
|
'metadata': {
|
||||||
|
# shared metadata for members
|
||||||
|
},
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
3. Wire the group into the relevant `nodes/*.py` (`'groups': {...}`)
|
||||||
|
or `groups/*.py` `supergroups` list.
|
||||||
|
|
||||||
|
4. Verify with `bw nodes <node> -a groups` and `bw metadata <node>`.
|
||||||
|
|
||||||
|
## How to add a new OS major (recipe)
|
||||||
|
|
||||||
|
Pattern from prior debian-12 → debian-13 work:
|
||||||
|
|
||||||
|
1. Add `groups/os/debian-N.py` and `groups/os/debian-N-common.py`
|
||||||
|
parallel to the existing files. Don't edit in place.
|
||||||
|
2. Add `data/apt/keys/debian-N-*.{asc,gpg}` for the new release's
|
||||||
|
signing keys. See
|
||||||
|
[`commands.md#apt-key-changes-need-offline-verification`](../docs/agents/commands.md#apt-key-changes-need-offline-verification)
|
||||||
|
before pushing keys live.
|
||||||
|
3. Bump dependent bundles that branch on `os_version` /
|
||||||
|
`os_codename` (`bundles/bind/items.py`, etc.).
|
||||||
|
4. Bump affected nodes' `groups` lists one at a time. Apply, watch.
|
||||||
|
5. Delete the old OS group file once no node references it.
|
||||||
|
|
||||||
|
## Pitfalls
|
||||||
|
|
||||||
|
- **`bw groups -n <node>` doesn't exist.** Use
|
||||||
|
`bw nodes <node> -a groups`.
|
||||||
|
- **Cycles.** A group can't be its own supergroup transitively;
|
||||||
|
`bw test` catches this but the error message is terse.
|
||||||
|
- **Silent eval failure.** A group file with a syntax error is skipped
|
||||||
|
and prints a one-line error. If a node loses bundles unexpectedly,
|
||||||
|
scan `groups.py` output for the error.
|
||||||
|
|
||||||
|
## See also
|
||||||
|
|
||||||
|
- [`nodes/AGENTS.md`](../nodes/AGENTS.md) — node files; how
|
||||||
|
`groups: {...}` attaches groups.
|
||||||
|
- [`docs/agents/conventions.md`](../docs/agents/conventions.md) —
|
||||||
|
inheritance order, naming conventions, eval-loader constraints.
|
||||||
|
- Fork's [`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
|
||||||
|
— group attribute reference, metadata-merge semantics.
|
||||||
62
hooks/AGENTS.md
Normal file
62
hooks/AGENTS.md
Normal file
|
|
@ -0,0 +1,62 @@
|
||||||
|
# hooks/
|
||||||
|
|
||||||
|
## What's here
|
||||||
|
|
||||||
|
Repo-level lifecycle hooks. Each `*.py` exports one or more functions
|
||||||
|
named after the lifecycle event they listen to (`apply_start`,
|
||||||
|
`node_apply_start`, `node_run_start`, `test`, `test_node`, …). Bw
|
||||||
|
discovers them by importing each module from `hooks/`.
|
||||||
|
|
||||||
|
Discovery is by `ls hooks/` + the one-line docstring at the top of
|
||||||
|
each file:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
head -1 hooks/*.py
|
||||||
|
```
|
||||||
|
|
||||||
|
## Conventions
|
||||||
|
|
||||||
|
- **One-line module docstring.** Every `hooks/*.py` starts with
|
||||||
|
`"""<name>: <one-line purpose>."""`. Add one when introducing a new
|
||||||
|
hook; baseline is enforced by `grep -L '"""' hooks/*.py`.
|
||||||
|
- **Function name = event name.** Bw calls
|
||||||
|
`apply_start(repo, target, nodes, interactive=False, **kwargs)`,
|
||||||
|
`node_apply_start(repo, node, interactive, **kwargs)`,
|
||||||
|
`test(repo, **kwargs)`, etc. Always accept `**kwargs` so future bw
|
||||||
|
arguments don't break the hook.
|
||||||
|
- **Test gates use `test` / `test_node`.** Anything that should fail
|
||||||
|
`bw test` (and therefore CI / pre-apply sanity) goes here; avoid
|
||||||
|
doing test-style assertions in `apply_start`.
|
||||||
|
|
||||||
|
## How to add a hook
|
||||||
|
|
||||||
|
1. Pick the lifecycle event (see fork's `AGENTS.md` for the full list).
|
||||||
|
2. Create `hooks/<name>.py` with the matching function and a
|
||||||
|
docstring.
|
||||||
|
3. Run `bw test` once to confirm the hook loads cleanly.
|
||||||
|
|
||||||
|
## Pitfalls
|
||||||
|
|
||||||
|
- **A hook that errors at import breaks every `bw` invocation** that
|
||||||
|
fires the hook's lifecycle — including `bw test` itself, which
|
||||||
|
defeats the obvious diagnostic. Test new hooks in isolation first:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
bw debug -c "import sys; sys.path.insert(0, 'hooks'); import <hookmodule>"
|
||||||
|
```
|
||||||
|
|
||||||
|
Iterate there until the import is clean, then commit.
|
||||||
|
- **Hooks have access to the full repo (`repo`, `node`, `nodes`).**
|
||||||
|
Don't make them block on network unless that's the explicit purpose
|
||||||
|
(e.g. `test_ptr_records.py` does `dig` against `9.9.9.9`).
|
||||||
|
- **Order is not guaranteed across hook files.** Two hooks that both
|
||||||
|
define `apply_start` will both fire; don't assume which runs first.
|
||||||
|
|
||||||
|
## See also
|
||||||
|
|
||||||
|
- [`docs/agents/conventions.md`](../docs/agents/conventions.md) —
|
||||||
|
files-not-to-touch, vault rules.
|
||||||
|
- [`docs/agents/commands.md`](../docs/agents/commands.md) — test
|
||||||
|
workflow.
|
||||||
|
- Fork's [`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
|
||||||
|
— full hook lifecycle and signatures.
|
||||||
64
items/AGENTS.md
Normal file
64
items/AGENTS.md
Normal file
|
|
@ -0,0 +1,64 @@
|
||||||
|
# items/
|
||||||
|
|
||||||
|
## What's here
|
||||||
|
|
||||||
|
Custom item types — each `*.py` is a `bundlewrap.items.Item` subclass
|
||||||
|
that adds a new item kind to the repo (e.g. `download:` from
|
||||||
|
`items/download.py`).
|
||||||
|
|
||||||
|
Currently:
|
||||||
|
|
||||||
|
- `items/download.py` — verifying file downloads with sha256 / GPG.
|
||||||
|
(Originally from `bundlewrap/plugins/item_download`; vendored in
|
||||||
|
because bw 4 dropped plugin support and it never came back.)
|
||||||
|
|
||||||
|
## Conventions
|
||||||
|
|
||||||
|
- **Filename = item-type slug.** `items/download.py` defines item
|
||||||
|
type `download:`. The class's `BUNDLE_ATTRIBUTE_NAME` is the dict
|
||||||
|
name in `items.py` (`downloads = {...}`).
|
||||||
|
- **Subclass `bundlewrap.items.Item`.** See the upstream
|
||||||
|
[bundlewrap docs on writing item types](https://docs.bundlewrap.org/dev/items/)
|
||||||
|
for the contract: `ITEM_ATTRIBUTES`, `fix`, `sdict`, `cdict`,
|
||||||
|
`display_dicts`, etc.
|
||||||
|
- **No metadata-reactor integration unless deliberate.** Item types
|
||||||
|
get to declare `NEEDS_STATIC` (cross-type ordering hints) — use it
|
||||||
|
sparingly; broad `NEEDS_STATIC` slows the whole DAG.
|
||||||
|
|
||||||
|
## When to write a new custom item type vs. use `file:` (or
|
||||||
|
`action:`, etc.)
|
||||||
|
|
||||||
|
| Situation | Use |
|
||||||
|
|---|---|
|
||||||
|
| One-off shell command at apply time | `action:` in the bundle |
|
||||||
|
| File whose source is in `bundles/<x>/files/` | `file:` |
|
||||||
|
| File you have to **fetch + verify** at apply | `download:` (already custom) |
|
||||||
|
| Behavior bw doesn't model and that recurs across bundles | new custom item |
|
||||||
|
|
||||||
|
If you only need it once, an `action:` is almost always enough.
|
||||||
|
Custom item types are repo-wide and load on every `bw` invocation —
|
||||||
|
the cost is paid forever.
|
||||||
|
|
||||||
|
## How to add a custom item type
|
||||||
|
|
||||||
|
1. Pick a slug (lowercase, no underscore).
|
||||||
|
2. Create `items/<slug>.py` with an `Item` subclass.
|
||||||
|
3. `bw test` — broken item types break the loader for every bundle
|
||||||
|
that uses them, so test in isolation first.
|
||||||
|
4. Document the contract in the file's module docstring.
|
||||||
|
|
||||||
|
## Pitfalls
|
||||||
|
|
||||||
|
- **Items affect the whole repo.** A change to `items/<x>.py` runs in
|
||||||
|
every node's apply. Treat custom items the way you treat `libs/` —
|
||||||
|
pure, fast, side-effect-free at import.
|
||||||
|
- **`NEEDS_STATIC` is a coarse hammer.** It enforces ordering across
|
||||||
|
*all* nodes for *all* items of those types; don't add unless
|
||||||
|
you've actually hit ordering issues.
|
||||||
|
|
||||||
|
## See also
|
||||||
|
|
||||||
|
- [`bundles/AGENTS.md`](../bundles/AGENTS.md) — items are consumed by
|
||||||
|
`items.py` in each bundle.
|
||||||
|
- Fork's [`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
|
||||||
|
— built-in item-type catalogue.
|
||||||
68
libs/AGENTS.md
Normal file
68
libs/AGENTS.md
Normal file
|
|
@ -0,0 +1,68 @@
|
||||||
|
# libs/
|
||||||
|
|
||||||
|
## What's here
|
||||||
|
|
||||||
|
Shared Python helpers reachable from any bundle as
|
||||||
|
`repo.libs.<modulename>.<symbol>`. One module per file (no packages).
|
||||||
|
Discovery is by `ls libs/` plus the one-line module docstring at the
|
||||||
|
top of each file.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
head -1 libs/*.py
|
||||||
|
```
|
||||||
|
|
||||||
|
## Conventions
|
||||||
|
|
||||||
|
- **One file per module.** Filename (without `.py`) is the
|
||||||
|
importable name. `libs/wireguard.py` exposes `repo.libs.wireguard`.
|
||||||
|
- **One-line module docstring.** Every `libs/*.py` starts with
|
||||||
|
`"""<name>: <one-line purpose>."""` so `ls + head` is a working index.
|
||||||
|
Add one when introducing a new lib; the rule is enforced by the
|
||||||
|
Phase-0 baseline (`grep -L '"""' libs/*.py` should report zero
|
||||||
|
files).
|
||||||
|
- **Pure helpers, no I/O at import.** Libs are imported on every `bw`
|
||||||
|
invocation. Heavy work or filesystem reads at module scope slow the
|
||||||
|
whole repo. Use functions, not module-level side effects.
|
||||||
|
- **No magic-string demagification here.** Libs receive already-
|
||||||
|
resolved values from bundles or nodes. Don't pass `!password_for:`
|
||||||
|
strings into libs — they won't be resolved.
|
||||||
|
- **`repo` and `vault` are globals available at runtime.** A few libs
|
||||||
|
(`wireguard.py`, etc.) reach `repo.vault.<verb>(...)` directly.
|
||||||
|
That's intentional inside libs, since libs run in the same loader
|
||||||
|
scope as bundles.
|
||||||
|
|
||||||
|
## How to add a helper
|
||||||
|
|
||||||
|
1. Either extract from a bundle that's growing complex, or add a new
|
||||||
|
`libs/<name>.py` file.
|
||||||
|
2. Top line: `"""<name>: <one-line purpose>."""`.
|
||||||
|
3. Pure functions only; document side effects in the docstring if any
|
||||||
|
slip through.
|
||||||
|
4. Use it from `bundles/<x>/items.py` or `metadata.py` as
|
||||||
|
`repo.libs.<name>.<symbol>(...)`.
|
||||||
|
|
||||||
|
## Pitfalls
|
||||||
|
|
||||||
|
- **Lib changes have repo-wide blast radius.** Every bundle that
|
||||||
|
imports the lib re-evaluates on the next `bw test` / `bw apply`.
|
||||||
|
Before changing a lib's API, find consumers:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
git grep -l 'repo.libs.<x>'
|
||||||
|
```
|
||||||
|
|
||||||
|
- **`@cache` is your friend** for deterministic key derivations
|
||||||
|
(`wireguard`, `rsa`, `ssh`); without it, `bw` recomputes per call,
|
||||||
|
which is slow.
|
||||||
|
- **Hashable wrappers.** `libs/hashable.py` exists because raw dicts
|
||||||
|
and sets aren't hashable, so you can't put them inside metadata
|
||||||
|
sets. Wrap with `repo.libs.hashable.hashable(...)` before nesting.
|
||||||
|
|
||||||
|
## See also
|
||||||
|
|
||||||
|
- [`bundles/AGENTS.md`](../bundles/AGENTS.md) — when to extract a
|
||||||
|
helper into `libs/` instead of duplicating across bundles.
|
||||||
|
- [`docs/agents/conventions.md`](../docs/agents/conventions.md) —
|
||||||
|
vault, demagify, naming.
|
||||||
|
- The fork's [`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
|
||||||
|
— `repo.libs` mechanism.
|
||||||
92
nodes/AGENTS.md
Normal file
92
nodes/AGENTS.md
Normal file
|
|
@ -0,0 +1,92 @@
|
||||||
|
# nodes/
|
||||||
|
|
||||||
|
## What's here
|
||||||
|
|
||||||
|
One file per node, ~22 in total. Each file is a single Python
|
||||||
|
expression (a dict literal) describing one machine: hostname, groups,
|
||||||
|
bundles, and metadata.
|
||||||
|
|
||||||
|
## Loader mechanism
|
||||||
|
|
||||||
|
`nodes.py` (top of the repo) walks `nodes/`, reads each `*.py` file,
|
||||||
|
runs `eval()` on its content, and then *demagifies* the result —
|
||||||
|
resolving any `!password_for:`, `!decrypt:`, `!decrypt_file:`, and
|
||||||
|
`!32_random_bytes_as_base64_for:` magic strings via `repo.vault`. See
|
||||||
|
[`docs/agents/conventions.md#secrets`](../docs/agents/conventions.md#secrets)
|
||||||
|
for the magic-string list.
|
||||||
|
|
||||||
|
This loader shape has consequences:
|
||||||
|
|
||||||
|
- **No top-level imports.** The file must be a single expression. No
|
||||||
|
`import os`, no `def`, no `if`. Use `repo.libs.<x>` from bundle code
|
||||||
|
if you need a helper.
|
||||||
|
- **Silent drop on parse failure.** Vanilla bundlewrap omits a node
|
||||||
|
whose file fails to eval. The maintainer's `groups.py` was patched
|
||||||
|
in commit `dc40295` to print the error; the node-loader prints
|
||||||
|
on `nodes.py` errors via the same shape. Symptom either way:
|
||||||
|
`bw nodes` lists fewer nodes than expected.
|
||||||
|
|
||||||
|
## Conventions
|
||||||
|
|
||||||
|
- **Filename = node name.** `home.server.py` defines the node `home.server`.
|
||||||
|
- **Naming pattern: `<location>.<role>.py`.** Examples:
|
||||||
|
`home.server.py`, `htz.mails.py`, `ovh.left4me.py`,
|
||||||
|
`mseibert.freescout.py`.
|
||||||
|
- **`*.py_` parks a node** without loading it. Used to keep
|
||||||
|
decommissioned-but-not-deleted configs in tree (e.g.
|
||||||
|
`htz.l4d2.py_`). The loader only matches `*.py`.
|
||||||
|
- **Magic strings only resolve here.** `!password_for:` etc. work in
|
||||||
|
node files; in groups, bundles, or items they don't — call
|
||||||
|
`repo.vault.<verb>(...)` directly there.
|
||||||
|
|
||||||
|
## How to add a new node
|
||||||
|
|
||||||
|
1. Create `nodes/<location>.<role>.py` with a single dict expression:
|
||||||
|
|
||||||
|
```python
|
||||||
|
{
|
||||||
|
'id': 'a-uuid-or-stable-name',
|
||||||
|
'hostname': '<dns-name-or-ip>',
|
||||||
|
'groups': {
|
||||||
|
'debian-13',
|
||||||
|
'monitored',
|
||||||
|
# …
|
||||||
|
},
|
||||||
|
'bundles': {
|
||||||
|
# only bundles not provided by the groups above
|
||||||
|
},
|
||||||
|
'metadata': {
|
||||||
|
# node-local overrides and required keys
|
||||||
|
},
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
2. Add to relevant `groups/<axis>/<x>.py` if group membership is the
|
||||||
|
attachment point (preferred over per-node `bundles` lists).
|
||||||
|
|
||||||
|
3. Verify:
|
||||||
|
- `bw nodes` — your node should appear.
|
||||||
|
- `bw nodes <node> -a groups` — confirm group membership resolved
|
||||||
|
as expected (`bw groups -n <node>` does **not** exist).
|
||||||
|
- `bw metadata <node>` — confirm merged metadata.
|
||||||
|
|
||||||
|
## Pitfalls
|
||||||
|
|
||||||
|
- **Renaming a node renames the node.** Vault entries (anything keyed
|
||||||
|
on `!password_for:<node>`), `bw hash` records, and ssh known_hosts
|
||||||
|
associations all key on node name. Search-and-replace before
|
||||||
|
renaming, or vault lookups silently return new (wrong) values.
|
||||||
|
- **Don't restore `_old` or `*.py_` files** without checking
|
||||||
|
[`conventions.md#suspension-and-soft-delete-idioms`](../docs/agents/conventions.md#suspension-and-soft-delete-idioms).
|
||||||
|
These are intentional parks/buffers, not bugs.
|
||||||
|
- **`id` must be unique.** A pre-apply hook (`hooks/unique_node_ids.py`)
|
||||||
|
enforces this; duplicate IDs fail `bw test` and `bw apply`.
|
||||||
|
|
||||||
|
## See also
|
||||||
|
|
||||||
|
- [`groups/AGENTS.md`](../groups/AGENTS.md) — group-membership patterns,
|
||||||
|
how metadata merges along the chain.
|
||||||
|
- [`docs/agents/conventions.md`](../docs/agents/conventions.md) —
|
||||||
|
demagify magic-strings, naming, eval-loader constraints.
|
||||||
|
- Fork's [`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
|
||||||
|
— node attribute reference (`hostname`, `username`, `dummy`, etc.).
|
||||||
Loading…
Reference in a new issue