.envrc

@@ -2,6 +2,6 @@
 
 PATH_add bin
 
-layout uv
-
+source_env ~/.local/share/direnv/pyenv
+source_env ~/.local/share/direnv/venv
 source_env ~/.local/share/direnv/bundlewrap

.gitignore

@@ -2,8 +2,3 @@
 .venv
 .cache
 *.pyc
-.bw_debug_history
-# CocoIndex Code (ccc)
-/.cocoindex_code/
-# bundlewrap git_deploy local-mirror map (operator-specific paths)
-git_deploy_repos

AGENTS.md

@@ -1,108 +0,0 @@
-# ckn-bw — agent & contributor guide
-
-## What this repo is
-
-A [BundleWrap](https://bundlewrap.org/) configuration-management repo
-for ~22 personal/family-infra nodes. Nodes, groups, and bundles are
-defined in plain Python; `bw apply` deploys the resulting state to
-real machines.
-
-Note: the root `README.md` is the maintainer's personal scratchpad,
-not project documentation. Onboarding lives **here**, in `AGENTS.md`.
-
-## Quickstart for agents
-
-Five rules; follow these and you won't break things:
-
-1. **Read-only by default.** Never run `bw apply`, `bw run`, or
-   `bw lock` without explicit user request — even with `-i`. Stick
-   to `bw test`, `bw nodes`, `bw groups`, `bw items`,
-   `bw metadata`, `bw hash`, `bw verify`, `bw debug`. See
-   [`docs/agents/commands.md`](docs/agents/commands.md) and the
-   fork's [safety envelope](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md).
-2. **Never echo decrypted secrets.** Don't print, paste, or log the
-   value behind a `!password_for:`, `!decrypt:`, or
-   `!32_random_bytes_as_base64_for:` magic string — not even from
-   `bw debug` exploration. See
-   [`conventions.md#secrets`](docs/agents/conventions.md#secrets).
-3. **Don't touch the do-not-modify list.** `.secrets.cfg*`, `.venv`,
-   `.cache`, `.bw_debug_history`, root `README.md`. Treat
-   `hooks/` and `items/` (custom item types) with extra care: a
-   broken hook or item type breaks every `bw` command repo-wide.
-4. **Use the fork.** Bundlewrap is pinned to the `main` branch of
-   [`github.com/CroneKorkN/bundlewrap`](https://github.com/CroneKorkN/bundlewrap)
-   via `[tool.uv.sources]` in `pyproject.toml`; `uv sync` (run by
-   direnv on entry) installs it. Behavior tracks the fork's `main`;
-   the fork's
-   [`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
-   is the canonical bundlewrap-language reference. See
-   [`conventions.md#bundlewrap-version`](docs/agents/conventions.md#bundlewrap-version).
-5. **Prefer adding helpers to `libs/`** over duplicating logic across
-   bundles. Repo-wide helpers go in
-   [`libs/`](libs/AGENTS.md), reachable as `repo.libs.<x>`.
-
-## Layout
-
-| Dir | What's there |
-|---|---|
-| [`bundles/`](bundles/AGENTS.md)              | 103 bundles. One subdir per bundle (`items.py`, `metadata.py`, `files/`). |
-| [`nodes/`](nodes/AGENTS.md)                  | One file per node (~22). `eval()`-loaded; demagified through `repo.vault`. |
-| [`groups/`](groups/AGENTS.md)                | Group definitions, organized by axis (`applications/`, `locations/`, `machine/`, `os/`). |
-| [`libs/`](libs/AGENTS.md)                    | Shared Python helpers reachable as `repo.libs.<modulename>`. |
-| [`hooks/`](hooks/AGENTS.md)                  | bw lifecycle hooks (`apply_start`, `test`, `node_apply_start`, …). |
-| [`data/`](data/AGENTS.md)                    | Out-of-bundle data assets (apt keys, grafana dashboards, …). |
-| [`items/`](items/AGENTS.md)                  | Custom item types (currently `download:`). |
-| [`bin/`](bin/AGENTS.md)                      | Operator scripts; not invoked by bundlewrap. |
-| [`docs/agents/`](docs/agents/conventions.md) | Repo conventions and command deltas. |
-
-## How nodes, groups, and bundles fit together
-
-- A **node** (`nodes/<location>.<role>.py`) declares the groups it
-  belongs to and any node-local bundles + metadata overrides.
-- A **group** (`groups/<axis>/<x>.py`) attaches bundles and shared
-  metadata to its members. Groups inherit via `supergroups`.
-- A **bundle** (`bundles/<x>/`) is one chunk of configuration:
-  `items.py` produces the items (files, services, packages),
-  `metadata.py` declares `defaults` and `@metadata_reactor` functions
-  that derive metadata from other metadata.
-- The repo-root loaders (`nodes.py`, `groups.py`) walk these dirs and
-  `eval()` each file. `nodes.py` additionally **demagifies** the
-  result, resolving `!password_for:` etc. through `repo.vault`. See
-  [`conventions.md#eval-loaded-node-and-group-files`](docs/agents/conventions.md#eval-loaded-node-and-group-files)
-  for the constraints this places on editors.
-- Metadata merges along: `all → location → os → machine →
-  applications → node`.
-
-## Conventions you must know
-
-| Topic | Where |
-|---|---|
-| Bundlewrap-language reference (item types, dep keywords, reactors) | Fork's [`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md) — read first if new to bundlewrap |
-| Vault / demagify magic strings                                     | [`conventions.md#secrets`](docs/agents/conventions.md#secrets) |
-| Bundlewrap install (uv-pinned to the fork's `main`)                | [`conventions.md#bundlewrap-version`](docs/agents/conventions.md#bundlewrap-version) |
-| Group inheritance order, naming patterns                           | [`conventions.md#group-inheritance-order`](docs/agents/conventions.md#group-inheritance-order), [`#naming-conventions`](docs/agents/conventions.md#naming-conventions) |
-| Repo-specific bw command deltas (apt keys, suspended nodes, vault echo) | [`commands.md`](docs/agents/commands.md) |
-| Lib helpers                                                        | top-of-file docstrings in `libs/*.py` (`head -1 libs/*.py`) |
-| Suspension idioms (`*.py_`, `_old/`, "for now")                    | [`conventions.md#suspension-and-soft-delete-idioms`](docs/agents/conventions.md#suspension-and-soft-delete-idioms) |
-
-## Where to look for examples
-
-When writing a new bundle, copy patterns from one that already does
-the thing you need:
-
-| Pattern                                  | Look at |
-|---|---|
-| Vault calls inside metadata reactors     | `bundles/dm-crypt/metadata.py` (compact, focused) |
-| Mako-templated files                     | `bundles/bind/items.py` (DNS zonefile rendering) |
-| Cross-bundle reactor writing             | `bundles/nextcloud/metadata.py` (writes into `apt.packages`, `archive.paths`) |
-| Custom `download:` items                 | `bundles/minecraft/items.py` |
-| Node file (single-purpose)               | `nodes/home.server.py` |
-| Group with `supergroups` chain           | `groups/os/debian-13.py` |
-
-## Where this doc lives
-
-- This file: `AGENTS.md` at the repo root.
-- `CLAUDE.md` is a symlink to this file — both names point to the same
-  content so different tools can find it.
-- The personal TODO scratchpad (`README.md`) is **separate** and not
-  project documentation.

CLAUDE.md

@@ -1 +0,0 @@
-AGENTS.md

README.md

@@ -13,6 +13,10 @@ Raspberry pi as soundcard
 - OTG g_audio
 - https://audiosciencereview.com/forum/index.php?threads/raspberry-pi-as-usb-to-i2s-adapter.8567/post-215824
 
+# install bw fork
+
+pip3 install --editable git+file:///Users/mwiegand/Projekte/bundlewrap-fork@main#egg=bundlewrap
+
 # monitor timers
 
 ```sh

bin/AGENTS.md

@@ -1,62 +0,0 @@
-# bin/
-
-## What's here
-
-Operator scripts — invoked manually by the maintainer, **not** by
-bundlewrap itself. Each is a standalone Python (or shell) script that
-opens the repo via `Repository(dirname(dirname(realpath(__file__))))`.
-
-Discovery is by `ls bin/` plus the `# purpose:` header line at the top
-of each script:
-
-```sh
-head -2 bin/*
-```
-
-## Conventions
-
-- **`# purpose:` header.** Every script under `bin/` starts with
-  `#!/usr/bin/env python3` (or appropriate shebang), then a
-  `# purpose: <one-line description>` comment. Baseline enforced by
-  `grep -L '^# purpose' bin/*`.
-- **Self-contained.** A script must work when run from anywhere — it
-  resolves the repo via the script's own path, not `cwd`.
-- **Read-only by default.** Most operator scripts query/print state
-  (`passwords-for`, `wireguard-client-config`). Mutating scripts
-  (`upgrade_and_restart_all`, `mikrotik-firmware-updater`,
-  `sync_1password`) are the exception, not the rule, and prompt for
-  confirmation.
-
-## How to add a script
-
-1. Start from [`bin/script_template`](script_template) — it carries
-   the canonical shebang + `# purpose:` header + `Repository(...)`
-   bootstrap.
-2. Add the `# purpose:` line; lowercase, terse, include a `usage:`
-   example if the script takes arguments.
-3. `chmod +x bin/<name>`.
-4. The script can reach helpers via `bw.libs.<x>` exactly like a
-   bundle does.
-
-## Pitfalls
-
-- **`bin/` is not on `$PATH` by default.** Invoke as `bin/<name>` from
-  the repo root, or via `direnv` if `.envrc` exposes it.
-- **Mutating scripts can hit Tier-3 territory** (per the fork's
-  safety envelope). Don't run `upgrade_and_restart_all`,
-  `mikrotik-firmware-updater`, or anything that does `node.run(...)`
-  without explicit user instruction. See the fork's
-  [`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
-  for the three-tier model.
-- **Vault echo.** Scripts like `passwords-for` print decrypted values
-  by design; that's allowed for the human at the terminal but *not*
-  for the agent — never paste output into chat, ticket, or PR
-  description.
-
-## See also
-
-- [`script_template`](script_template) — canonical starter.
-- [`docs/agents/conventions.md`](../docs/agents/conventions.md) —
-  vault rules.
-- [`docs/agents/commands.md`](../docs/agents/commands.md) — read-only
-  bw-command guidance.

bin/mikrotik-firmware-updater

@@ -1,149 +0,0 @@
-#!/usr/bin/env python3
-# purpose: upgrade RouterOS and routerboard firmware on `bundle:routeros` (or any selector) — usage: mikrotik-firmware-updater [<selector>...] [--yes].
-
-from argparse import ArgumentParser
-from time import sleep
-
-from bundlewrap.exceptions import RemoteException
-from bundlewrap.utils.cmdline import get_target_nodes
-from bundlewrap.utils.ui import io
-from bundlewrap.repo import Repository
-from os.path import realpath, dirname
-
-
-# parse args
-parser = ArgumentParser()
-parser.add_argument("targets", nargs="*", default=['bundle:routeros'], help="bw nodes selector")
-parser.add_argument("--yes", action="store_true", default=False, help="skip confirmation prompts")
-args = parser.parse_args()
-
-
-def wait_up(node):
-    sleep(5)
-    while True:
-        try:
-            node.run_routeros('/system/resource/print')
-        except RemoteException:
-            sleep(2)
-            continue
-        else:
-            io.debug(f"{node.name}: is up")
-            sleep(10)
-            return
-
-
-def upgrade_switch_os(node):
-    # get versions for comparison
-    with io.job(f"{node.name}: checking OS version"):
-        response = node.run_routeros('/system/package/update/check-for-updates').raw[-1]
-        installed_os = bw.libs.version.Version(response['installed-version'])
-        latest_os = bw.libs.version.Version(response['latest-version'])
-        io.debug(f"{node.name}: installed: {installed_os} >= latest: {latest_os}")
-
-    # compare versions
-    if installed_os >= latest_os:
-        # os is up to date
-        io.stdout(f"{node.name}: os up to date ({installed_os})")
-    else:
-        # confirm os upgrade
-        if not args.yes and not io.ask(
-            f"{node.name}: upgrade os from {installed_os} to {latest_os}?", default=True
-        ):
-            io.stdout(f"{node.name}: skipped by user")
-            return
-
-        # download os
-        with io.job(f"{node.name}: downloading OS"):
-            response = node.run_routeros('/system/package/update/download').raw[-1]
-            io.debug(f"{node.name}: OS upgrade download response: {response['status']}")
-
-        # install and wait for reboot
-        with io.job(f"{node.name}: upgrading OS"):
-            try:
-                response = node.run_routeros('/system/package/update/install').raw[-1]
-            except RemoteException:
-                pass
-            wait_up(node)
-
-        # verify new os version
-        with io.job(f"{node.name}: checking new OS version"):
-            new_os = bw.libs.version.Version(node.run_routeros('/system/package/update/check-for-updates').raw[-1]['installed-version'])
-            if new_os == latest_os:
-                io.stdout(f"{node.name}: OS successfully upgraded from {installed_os} to {new_os}")
-            else:
-                raise Exception(f"{node.name}: OS upgrade failed, expected {latest_os}, got {new_os}")
-
-
-def upgrade_switch_firmware(node):
-    # get versions for comparison
-    with io.job(f"{node.name}: checking Firmware version"):
-        response = node.run_routeros('/system/routerboard/print').raw[-1]
-        current_firmware = bw.libs.version.Version(response['current-firmware'])
-        upgrade_firmware = bw.libs.version.Version(response['upgrade-firmware'])
-        io.debug(f"{node.name}: firmware installed: {current_firmware}, upgrade: {upgrade_firmware}")
-
-    # compare versions
-    if current_firmware >= upgrade_firmware:
-        # firmware is up to date
-        io.stdout(f"{node.name}: firmware is up to date ({current_firmware})")
-    else:
-        # confirm firmware upgrade
-        if not args.yes and not io.ask(
-            f"{node.name}: upgrade firmware from {current_firmware} to {upgrade_firmware}?", default=True
-        ):
-            io.stdout(f"{node.name}: skipped by user")
-            return
-
-        # upgrade firmware
-        with io.job(f"{node.name}: upgrading Firmware"):
-            node.run_routeros('/system/routerboard/upgrade')
-
-        # reboot and wait
-        with io.job(f"{node.name}: rebooting"):
-            try:
-                node.run_routeros('/system/reboot')
-            except RemoteException:
-                pass
-            wait_up(node)
-
-        # verify firmware version
-        new_firmware = bw.libs.version.Version(node.run_routeros('/system/routerboard/print').raw[-1]['current-firmware'])
-        if new_firmware == upgrade_firmware:
-            io.stdout(f"{node.name}: firmware successfully upgraded from {current_firmware} to {new_firmware}")
-        else:
-            raise Exception(f"firmware upgrade failed, expected {upgrade_firmware}, got {new_firmware}")
-
-
-def upgrade_switch(node):
-    with io.job(f"{node.name}: checking"):
-        # check if routeros
-        if node.os != 'routeros':
-            io.progress_advance(2)
-            io.stdout(f"{node.name}: skipped, unsupported os {node.os}")
-            return
-
-        # check switch reachability
-        try:
-            node.run_routeros('/system/resource/print')
-        except RemoteException as error:
-            io.progress_advance(2)
-            io.stdout(f"{node.name}: skipped, error {error}")
-            return
-
-    upgrade_switch_os(node)
-    io.progress_advance(1)
-
-    upgrade_switch_firmware(node)
-    io.progress_advance(1)
-
-
-with io:
-    bw = Repository(dirname(dirname(realpath(__file__))))
-
-    nodes = get_target_nodes(bw, args.targets)
-
-    io.progress_set_total(len(nodes) * 2)
-    io.stdout(f"upgrading {len(nodes)} switches: {', '.join([node.name for node in sorted(nodes)])}")
-
-    for node in sorted(nodes):
-        upgrade_switch(node)

bin/passwords-for

@@ -1,23 +0,0 @@
-#!/usr/bin/env python3
-# purpose: print node.password and selected metadata-key passwords for one node — usage: passwords-for <node>.
-
-from bundlewrap.repo import Repository
-from os.path import realpath, dirname
-
-import argparse
-
-parser = argparse.ArgumentParser()
-parser.add_argument('node', help='Node to generate passwords for')
-args = parser.parse_args()
-
-bw = Repository(dirname(dirname(realpath(__file__))))
-node = bw.get_node(args.node)
-
-if node.password:
-    print(f"password: {node.password}")
-
-for metadata_key in sorted([
-    'users/root/password',
-]):
-    if value := node.metadata.get(metadata_key, None):
-        print(f"{metadata_key}: {value}")

bin/rcon

@@ -1,5 +1,4 @@
 #!/usr/bin/env python3
-# purpose: send an RCON command to a left4dead2 server defined in node metadata — usage: rcon (list) | rcon <server> <command>.
 
 from sys import argv
 from os.path import realpath, dirname

bin/script_template

@@ -1,7 +1,6 @@
 #!/usr/bin/env python3
-# purpose: starter template for new operator scripts under bin/.
 
 from bundlewrap.repo import Repository
 from os.path import realpath, dirname
 
-bw = Repository(dirname(dirname(realpath(__file__))))
+repo = Repository(dirname(dirname(realpath(__file__))))

bin/sync_1password

@@ -1,133 +0,0 @@
-#!/usr/bin/env python3
-# purpose: upsert one 1Password login per `bundle:routeros` node, keyed on the bw node id.
-
-from bundlewrap.repo import Repository
-from os.path import realpath, dirname
-
-import json
-import os
-import subprocess
-from dataclasses import dataclass
-from typing import Optional, List
-
-bw = Repository(dirname(dirname(realpath(__file__))))
-
-VAULT=bw.vault.decrypt('encrypt$gAAAAABpLgX_xxb5NmNCl3cgHM0JL65GT6PHVXO5gwly7IkmWoEgkCDSuAcSAkNFB8Tb4RdnTdpzVQEUL1XppTKVto_O7_b11GjATiyQYiSfiQ8KZkTKLvk=').value
-BW_TAG = "bw"
-BUNDLEWRAP_FIELD_LABEL = "bundlewrap node id"
-
-
-@dataclass
-class OpResult:
-    stdout: str
-    stderr: str
-    returncode: int
-
-
-def main():
-    for node in bw.nodes_in_group('routeros'):
-        upsert_node_item(
-            node_name=node.name,
-            node_uuid=node.metadata.get('id'),
-            username=node.username,
-            password=node.password,
-            url=f'http://{node.hostname}',
-        )
-
-
-def run_op(args):
-    proc = subprocess.run(
-        ["op", "--vault", VAULT] + args,
-        env=os.environ.copy(),
-        capture_output=True,
-        text=True,
-    )
-
-    if proc.returncode != 0:
-        raise RuntimeError(
-            f"op {' '.join(args)} failed with code {proc.returncode}:\n"
-            f"STDOUT:\n{proc.stdout}\n\nSTDERR:\n{proc.stderr}"
-        )
-
-    return OpResult(stdout=proc.stdout, stderr=proc.stderr, returncode=proc.returncode)
-
-
-def op_item_list_bw():
-    out = run_op([
-        "item", "list",
-        "--tags", BW_TAG,
-        "--format", "json",
-    ])
-    stdout = out.stdout.strip()
-    return json.loads(stdout) if stdout else []
-
-
-def op_item_get(item_id):
-    args = ["item", "get", item_id, "--format", "json"]
-    return json.loads(run_op(args).stdout)
-
-
-def op_item_create(title, node_uuid, username, password, url):
-    print(f"creating {title}")
-    return json.loads(run_op([
-        "item", "create",
-        "--category", "LOGIN",
-        "--title", title,
-        "--tags", BW_TAG,
-        "--url", url,
-        "--format", "json",
-        f"username={username}",
-        f"password={password}",
-        f"{BUNDLEWRAP_FIELD_LABEL}[text]={node_uuid}",
-    ]).stdout)
-
-
-def op_item_edit(item_id, title, username, password, url):
-    print(f"updating {title}")
-    return json.loads(run_op([
-        "item", "edit",
-        item_id,
-        "--title", title,
-        "--url", url,
-        "--format", "json",
-        f"username={username}",
-        f"password={password}",
-    ]).stdout)
-
-
-def find_node_item_id(node_uuid):
-    for summary in op_item_list_bw():
-        item_id = summary.get("id")
-        if not item_id:
-            continue
-
-        item = op_item_get(item_id)
-        for field in item.get("fields") or []:
-            label = field.get("label")
-            value = field.get("value")
-            if label == BUNDLEWRAP_FIELD_LABEL and value == node_uuid:
-                return item_id
-    return None
-
-
-def upsert_node_item(node_name, node_uuid, username, password, url):
-    if item_id := find_node_item_id(node_uuid):
-        return op_item_edit(
-            item_id=item_id,
-            title=node_name,
-            username=username,
-            password=password,
-            url=url,
-        )
-    else:
-        return op_item_create(
-            title=node_name,
-            node_uuid=node_uuid,
-            username=username,
-            password=password,
-            url=url,
-        )
-
-
-if __name__ == "__main__":
-    main()

bin/timestamp_icloud_photos_for_nextcloud

@@ -1,217 +0,0 @@
-#!/usr/bin/env python3
-# purpose: add missing EXIF/QuickTime timestamps to photos in a directory using mdls + exiftool — usage: timestamp_icloud_photos_for_nextcloud -d <dir>.
-
-from subprocess import check_output, CalledProcessError
-from datetime import datetime, timedelta
-from pathlib import Path
-import json
-from argparse import ArgumentParser
-from concurrent.futures import ThreadPoolExecutor, as_completed
-from os import cpu_count
-from time import sleep
-
-EXT_GROUPS = {
-    "quicktime": {".mp4", ".mov", ".heic", ".cr3"},
-    "exif": {".jpg", ".jpeg", ".cr2"},
-}
-DATETIME_KEYS = [
-    ("Composite", "SubSecDateTimeOriginal"),
-    ("Composite", "SubSecCreateDate"),
-    ('ExifIFD', 'DateTimeOriginal'),
-    ('ExifIFD', 'CreateDate'),
-    ('XMP-xmp', 'CreateDate'),
-    ('Keys', 'CreationDate'),
-    ('QuickTime', 'CreateDate'),
-    ('XMP-photoshop', 'DateCreated'),
-]
-
-def run(command):
-    return check_output(command, text=True).strip()
-
-
-def mdls_timestamp(file):
-    for i in range(5): # retry a few times in case of transient mdls failures
-        try:
-            output = run(('mdls', '-raw', '-name', 'kMDItemContentCreationDate', file))
-        except CalledProcessError as e:
-            print(f"{file}: Error running mdls (attempt {i+1}/5): {e}")
-            continue
-
-        try:
-            return datetime.strptime(output, "%Y-%m-%d %H:%M:%S %z")
-        except ValueError as e:
-            print(f"{file}: Error parsing mdls output (attempt {i+1}/5): {e}")
-            continue
-
-        sleep(1)
-
-    raise RuntimeError(f"Failed to get mdls timestamp for {file} after 5 attempts")
-
-
-def exiftool_data(file):
-    try:
-        output = run((
-            'exiftool',
-            '-j', # json
-            '-a', # unknown tags
-            '-u', # unknown values
-            '-g1', # group by category
-            '-time:all', # all time tags
-            '-api', 'QuickTimeUTC=1', # use UTC for QuickTime timestamps
-            '-d', '%Y-%m-%dT%H:%M:%S%z',
-            file,
-        ))
-    except CalledProcessError as e:
-        print(f"Error running exiftool: {e}")
-        return None
-    else:
-        return json.loads(output)[0]
-
-def exiftool_timestamp(file):
-    data = exiftool_data(file)
-    for category, key in DATETIME_KEYS:
-        try:
-            value = data[category][key]
-            return category, key, datetime.strptime(value, '%Y-%m-%dT%H:%M:%S%z')
-        except (TypeError, KeyError, ValueError) as e:
-            continue
-    print(f"⚠️ {file}: No timestamp found in exiftool: " + json.dumps(data, indent=2))
-    return None, None, None
-
-
-def photo_has_embedded_timestamp(file):
-    mdls_ts = mdls_timestamp(file)
-    category, key, exiftool_ts = exiftool_timestamp(file)
-
-    if not exiftool_ts:
-        print(f"⚠️ {file}: No timestamp found in exiftool")
-        return False
-
-    # normalize timezone for comparison
-    exiftool_ts = exiftool_ts.astimezone(mdls_ts.tzinfo)
-    delta = abs(mdls_ts - exiftool_ts)
-
-    if delta < timedelta(hours=1): # allow for small differences
-        print(f"✅ {file}: {mdls_ts.isoformat()} (#{category}:{key})")
-        return True
-    else:
-        print(f"⚠️ {file}: {mdls_ts.isoformat()} != {exiftool_ts} (Δ={delta})")
-        return False
-
-
-def photos_without_embedded_timestamps(directory):
-    executor = ThreadPoolExecutor(max_workers=cpu_count()//2)
-    try:
-        futures = {
-            executor.submit(photo_has_embedded_timestamp, file): file
-                for file in directory.iterdir()
-                if file.is_file()
-                if file.suffix.lower() not in {".aae"}
-                if not file.name.startswith('.')
-        }
-
-        for future in as_completed(futures):
-            file = futures[future]
-            has_ts = future.result()   # raises immediately on first failed future
-
-            if has_ts:
-                file.rename(file.parent / 'ok' / file.name)
-            else:
-                yield file
-
-    except Exception:
-        executor.shutdown(wait=False, cancel_futures=True)
-        raise
-    else:
-        executor.shutdown(wait=True)
-
-
-def exiftool_write(file, assignments):
-    print(f"🔵 {file}: Writing -- {assignments}")
-    return run((
-        "exiftool", "-overwrite_original",
-        "-api", "QuickTimeUTC=1",
-        *[
-            f"-{group}:{tag}={value}"
-            for group, tag, value in assignments
-        ],
-        str(file),
-    ))
-
-
-def add_missing_timestamp(file):
-    data = exiftool_data(file)
-    mdls_ts = mdls_timestamp(file)
-
-    offset = mdls_ts.strftime("%z")
-    offset = f"{offset[:3]}:{offset[3:]}" if len(offset) == 5 else offset
-
-    exif_ts = mdls_ts.strftime("%Y:%m:%d %H:%M:%S")
-    qt_ts = mdls_ts.strftime("%Y:%m:%d %H:%M:%S")
-    qt_ts_tz = f"{qt_ts}{offset}"
-    ext = file.suffix.lower()
-
-    try:
-        if ext in {".heic"}:
-            exiftool_write(file, [
-                ("ExifIFD", "DateTimeOriginal", qt_ts),
-                ("ExifIFD", "CreateDate", qt_ts),
-                ("ExifIFD", "OffsetTime", offset),
-                ("ExifIFD", "OffsetTimeOriginal", offset),
-                ("ExifIFD", "OffsetTimeDigitized", offset),
-                ("QuickTime", "CreateDate", qt_ts_tz),
-                ("Keys", "CreationDate", qt_ts_tz),
-                ("XMP-xmp", "CreateDate", qt_ts_tz),
-            ])
-        elif "QuickTime" in data or ext in {".mp4", ".mov", ".heic", ".cr3"}:
-            exiftool_write(file, [
-                ("QuickTime", "CreateDate", qt_ts_tz),
-                ("Keys", "CreationDate", qt_ts_tz),
-            ])
-        elif "ExifIFD" in data or ext in {".jpg", ".jpeg", ".cr2", ".webp"}:
-            exiftool_write(file, [
-                ("ExifIFD", "DateTimeOriginal", exif_ts),
-                ("ExifIFD", "CreateDate", exif_ts),
-                ("IFD0", "ModifyDate", exif_ts),
-                ("ExifIFD", "OffsetTime", offset),
-                ("ExifIFD", "OffsetTimeOriginal", offset),
-                ("ExifIFD", "OffsetTimeDigitized", offset),
-            ])
-        elif ext in {".png", ".gif", ".avif"}:
-            exiftool_write(file, [
-                ("XMP-xmp", "CreateDate", qt_ts_tz),
-                ("XMP-photoshop", "DateCreated", exif_ts),
-            ])
-        else:
-            print(f"❌ {file}: unsupported type, skipped")
-            return
-
-        if photo_has_embedded_timestamp(file):
-            print(f"✅ {file}: Timestamp successfully added: {mdls_ts.isoformat()}")
-            file.rename(file.parent / 'processed' / file.name)
-            return
-        else:
-            category, key, exiftool_ts = exiftool_timestamp(file)
-            print(f"❌ {file}: Timestamp still wrong/missing after write '{category}:{key}:{exiftool_ts}': #{json.dumps(data, indent=4)}")
-            return
-    except CalledProcessError as e:
-        print(f"❌ {file}: Failed to write timestamp: {e}")
-        return
-
-
-if __name__ == "__main__":
-    parser = ArgumentParser(description="Print timestamps of photos in the current directory.")
-    parser.add_argument("-d", "--directory", help="Directory to scan for photos")
-    args = parser.parse_args()
-
-    directory = Path(args.directory)
-    (directory/'ok').mkdir(exist_ok=True)
-    (directory/'processed').mkdir(exist_ok=True)
-
-    _photos_without_embedded_timestamps = list(photos_without_embedded_timestamps(directory))
-    print(f"{len(_photos_without_embedded_timestamps)} photos without embedded timestamps found.")
-    print("Press Enter to add missing timestamps...")
-    input()
-
-    for file in _photos_without_embedded_timestamps:
-        add_missing_timestamp(file)

bin/upgrade_and_restart_all

@@ -1,5 +1,4 @@
 #!/usr/bin/env python3
-# purpose: apt-update and full-upgrade every non-dummy debian node, then reboot in WireGuard-aware order.
 
 from bundlewrap.repo import Repository
 from os.path import realpath, dirname

bin/wake

@@ -1,5 +1,4 @@
 #!/usr/bin/env python3
-# purpose: wake one node via WoL by name — usage: wake <node>.
 
 from bundlewrap.repo import Repository
 from os.path import realpath, dirname

bin/wireguard_client_config

@@ -1,25 +1,23 @@
 #!/usr/bin/env python3
-# purpose: print or QR-render a WireGuard client config from htz.mails metadata — usage: wireguard-client-config <client>.
 
 from bundlewrap.repo import Repository
 from os.path import realpath, dirname
 from sys import argv
 from ipaddress import ip_network, ip_interface
-import argparse
 
+if len(argv) != 3:
+    print(f'usage: {argv[0]} <node> <client>')
+    exit(1)
 
-# get info from repo
 repo = Repository(dirname(dirname(realpath(__file__))))
-server_node = repo.get_node('htz.mails')
-available_clients = server_node.metadata.get('wireguard/clients').keys()
+server_node = repo.get_node(argv[1])
 
-# parse args
-parser = argparse.ArgumentParser(description='Generate WireGuard client configuration.')
-parser.add_argument('client', choices=available_clients, help='The client name to generate the configuration for.')
-args = parser.parse_args()
+if argv[2] not in server_node.metadata.get('wireguard/clients'):
+    print(f'client {argv[2]} not found in: {server_node.metadata.get("wireguard/clients").keys()}')
+    exit(1)
+
+data = server_node.metadata.get(f'wireguard/clients/{argv[2]}')
 
-# get cert
-data = server_node.metadata.get(f'wireguard/clients/{args.client}')
 vpn_network = ip_interface(server_node.metadata.get('wireguard/my_ip')).network
 allowed_ips = [
     vpn_network,
@@ -45,15 +43,10 @@ Endpoint = {ip_interface(server_node.metadata.get('network/external/ipv4')).ip}:
 PersistentKeepalive = 10
 '''
 
-answer = input("print config or qrcode? [Cq]: ").strip().upper()
-match answer:
-    case '' | 'C':
-        print('>>>>>>>>>>>>>>>')
-        print(conf)
-        print('<<<<<<<<<<<<<<<')
-    case 'Q':
-        import pyqrcode
-        print(pyqrcode.create(conf).terminal(quiet_zone=1))
-    case _:
-        print(f'Invalid option "{answer}".')
-        exit(1)
+print('>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>')
+print(conf)
+print('<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<')
+
+if input("print qrcode? [Yn]: ").upper() in ['', 'Y']:
+    import pyqrcode
+    print(pyqrcode.create(conf).terminal(quiet_zone=1))

bundles/AGENTS.md

@@ -1,204 +0,0 @@
-# bundles/
-
-## Before you start
-
-Read [`docs/agents/conventions.md`](../docs/agents/conventions.md) first
-— it covers vault calls, demagify, the `repo.libs.<x>` helpers, and the
-files agents must not modify. Skipping it leads to subtly broken bundles
-(vault calls in the wrong place, dict-in-set `TypeError` because of
-unhashable nesting, etc.).
-
-For bundlewrap-language reference (item types, dep keywords,
-`metadata_reactor`, `defaults`, item-file template syntax) see the fork's
-[`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
-and its [`docs/content/guide/item_file_templates.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/docs/content/guide/item_file_templates.md).
-
-## What's here
-
-103 bundles. Each is a directory `bundles/<name>/` containing some of:
-
-```
-bundles/<name>/
-├── items.py        # the items this bundle creates (files, services, packages, …)
-├── metadata.py     # `defaults` + `@metadata_reactor` functions
-├── files/          # static or templated file payloads referenced from items.py
-└── README.md       # one doc per bundle, for humans and agents (see "Per-bundle README" below)
-```
-
-## Conventions
-
-- **Bundle names** are lowercase, hyphen-separated: `backup-server`,
-  `bind-acme`, `dm-crypt`. No underscores in new bundle names — see
-  [`conventions.md#naming-conventions`](../docs/agents/conventions.md#naming-conventions).
-- **`items.py`** is plain Python; it produces `files = {...}`,
-  `pkg_apt = {...}`, `svc_systemd = {...}`, etc. dicts at module scope.
-  Cross-item dependencies use `needs` / `triggers` / `triggered_by` —
-  see the fork's `AGENTS.md` for the full keyword cheat sheet.
-- **`metadata.py`** uses `defaults = {...}` for static seed values and
-  `@metadata_reactor.provides(...)` for derived values. Reactors are
-  pure functions of `(metadata,)` — no side effects, no I/O.
-- **Helpers go in [`libs/`](../libs/AGENTS.md)** when they're useful to
-  more than one bundle. Don't duplicate logic across bundles.
-- **Custom item types** (e.g. `download:`) live in
-  [`items/`](../items/AGENTS.md), not per-bundle.
-- **Bundles own application-wide knowledge; nodes carry only the few
-  per-host knobs the bundle actually needs.** When designing a bundle,
-  identify the per-node knobs (e.g. domain, uplink interface, a
-  vault-id suffix) and put everything else in `defaults`, or in a
-  reactor that derives from those knobs. Per-node random secrets
-  belong in `defaults` via `repo.vault.random_bytes_as_base64_for(...)`
-  keyed on the node — not in the node file. See
-  `bundles/left4me/metadata.py:10` (`secret_key` derived in defaults)
-  and `bundles/postgresql/metadata.py:4` (vault-derived `password_for`
-  at module scope).
-
-## How to add a new bundle
-
-1. `mkdir bundles/<name>/` (lowercase, hyphenated).
-2. Write `items.py` and (if anything is configurable) `metadata.py`.
-   Use `repo.libs.hashable.hashable(...)` when you need to nest a dict
-   or set inside a metadata set; raw dicts/sets aren't hashable.
-3. Drop static payloads into `bundles/<name>/files/`. For Mako-templated
-   files, declare `'content_type': 'mako'` on the `file:` item — see
-   the fork's
-   [item-file-templates guide](https://github.com/CroneKorkN/bundlewrap/blob/main/docs/content/guide/item_file_templates.md).
-4. **Wire to nodes.** Either add an entry to the relevant
-   [`groups/<axis>/<x>.py`](../groups/AGENTS.md) (preferred for shared
-   bundles) or to the node's `bundles` list directly
-   ([`nodes/AGENTS.md`](../nodes/AGENTS.md)).
-5. **Verify, in this order:**
-   - `bw test` — repo-wide parse + cross-cutting hooks. Loads every
-     bundle, but reactors don't fire for nodes that haven't opted into
-     the bundle yet — bugs in new reactors stay hidden here.
-   - **Attach the bundle to a node** (via the node's `bundles` list, or
-     a group it belongs to). Until you do, the next steps don't actually
-     exercise the bundle.
-   - `bw test <node>` — exercises every reactor and item-graph edge for
-     that node. This is where most new-bundle bugs surface.
-   - `bw items <node> --blame` — confirm items materialise with the
-     right paths, authored by the expected bundle.
-   - `bw metadata <node> -k <a/b>` — spot-check derived metadata.
-   - `bw hash <node>` — preview vs current host state.
-
-   See [`docs/agents/commands.md#bundle-validation-workflow`](../docs/agents/commands.md#bundle-validation-workflow)
-   for the rationale.
-6. Add a `bundles/<name>/README.md`. See "Per-bundle README" below
-   for what to cover.
-
-## How to remove a bundle
-
-1. `git grep '<name>'` in `nodes/`, `groups/`, and other `bundles/` to
-   find references.
-2. Remove those references.
-3. `rm -rf bundles/<name>/`.
-4. `bw test` and `bw nodes` to confirm clean.
-
-## Pitfalls
-
-- **`metadata.py` is evaluated at load time** for *every* node, every
-  invocation of `bw`. Heavy work or I/O slows the whole repo. Keep
-  reactors pure and fast; pre-compute in `libs/` if you must.
-- **Static files vs templates.** `bundles/<x>/files/<f>` is static
-  unless the matching `file:` item declares `content_type='mako'`
-  (or a templating extension triggers it). To check, read the matching
-  `file:` entry in `items.py`.
-- **`file:` `source` defaults to the destination basename.** For a
-  destination of `/etc/foo/bar.conf` with no `source` key, bw looks
-  for `bundles/<bundle>/files/bar.conf`. Only declare `source`
-  explicitly when the basename you want differs (e.g. shipping a Mako
-  template named `bar.conf.mako` to a destination of
-  `/etc/foo/bar.conf`).
-- **Reactors writing across namespaces.** Some bundles' reactors write
-  into other bundles' metadata namespaces (e.g. `nextcloud` writes
-  into `apt.packages`, `archive.paths`). When you change such a bundle,
-  every consumer's metadata changes too. The bundle's `README.md`
-  often calls these out — but the authoritative source is `metadata.py`
-  itself; grep `'<other-bundle>':` in the reactors when in doubt.
-- **`bw hash` doesn't accept selectors.** Use `bw hash <node>` per
-  literal name; see the fork's runbook.
-- **Reactors must read metadata.** If a reactor body returns a static
-  dict without calling `metadata.get(...)`, bw raises
-  `ValueError: <reactor> on <node> did not request any metadata, you
-  might want to use defaults instead` once a node consumes the bundle.
-  Fix: fold the contribution into `defaults`. The rule applies even
-  when the reactor writes into another bundle's namespace — a static
-  contribution to e.g. `nftables/output` belongs in `defaults`, where
-  bw merges it with other bundles' contributions.
-- **`triggers` ↔ `triggered: True` invariant.** Any item listed in
-  another's `triggers` list must declare `triggered: True`. bw
-  enforces this at `bw test` time: *"…triggered by …, but missing
-  'triggered' attribute"*. Corollary: an action can't be both in an
-  upstream `triggers` list AND self-healing every apply — pick one.
-- **Triggered actions don't recover from partial failure.** When an
-  upstream item's apply succeeds but its triggered downstream action
-  fails, subsequent applies can't recover via the trigger chain —
-  upstream is "already in desired state" and never re-triggers. For
-  actions that must self-heal (pip installs, chowns, migrations),
-  drop `triggered: True` and gate the command with `unless: <fast-check>`.
-  `unless` is a shell command on the target host whose exit status
-  decides whether the main command runs (exit 0 = skip); it's checked
-  at fire time, after `triggered:` filtering.
-
-## Per-bundle README
-
-Each bundle has (or should have) a `README.md`. One doc per bundle,
-written for humans and agents both. There's no fixed structure —
-match the bundle's actual surface, write what helps a future reader
-(or future you) avoid trial-and-error.
-
-The existing READMEs vary in quality and shape. For orientation,
-look at the bigger ones, not the two-line ones:
-
-- [`bundles/flask/README.md`](flask/README.md) — title + one-sentence
-  purpose, a metadata example as a Python dict, then the contract
-  the consuming git repo has to satisfy + a logging pitfall. The
-  closest thing to a "balanced doc" in tree.
-- [`bundles/dm-crypt/README.md`](dm-crypt/README.md) — same shape,
-  shorter: purpose + metadata example + one sentence on effect.
-- [`bundles/apt/README.md`](apt/README.md) — relevant upstream URLs
-  at the top, then a Python metadata example with rich inline
-  comments (type / optionality / where keys come from).
-- [`bundles/nextcloud/README.md`](nextcloud/README.md) — operational
-  scratchpad: iPhone-import recipe, preview-generator commands,
-  reset queries. Captures muscle-memory the maintainer would
-  otherwise re-learn each time.
-
-Useful things to include, when relevant:
-
-- A sentence or two on what the bundle does and when you'd attach it.
-- A metadata example as a Python dict literal, with `#` comments
-  on each key (type, required vs default, units, where it comes
-  from). This is the cleanest way to communicate the schema and
-  matches how `metadata.py` actually looks.
-- Anything non-obvious about wiring it up — required keys without
-  defaults, group-membership expectations, manual one-time steps.
-- Cross-namespace metadata writes, when this bundle's reactors
-  populate another bundle's namespace. Easy to miss, cheap to flag.
-- Gotchas, debug recipes, failure modes you've actually hit.
-
-What to skip:
-
-- An exhaustive item list — `items.py` is shorter and more accurate.
-- Anything that would just rot — version numbers, "TODO" lists,
-  change notes. Use git history.
-
-If a single paragraph is enough to say what's worth saying, write a
-single paragraph. Verbosity isn't a goal.
-
-Convention going forward is leave-as-you-go: any time you materially
-edit a bundle, top up its README (or write one if it's missing).
-Don't burn a session bulk-reformatting the existing ones — uneven
-quality is part of what we accept in exchange for not blocking other
-work.
-
-## See also
-
-- [`docs/agents/conventions.md`](../docs/agents/conventions.md) — repo
-  idioms (vault, demagify, naming, do-not-touch list).
-- [`docs/agents/commands.md`](../docs/agents/commands.md) — repo-specific
-  command deltas.
-- [`items/AGENTS.md`](../items/AGENTS.md) — custom item types
-  (`download:`); when to write a new one vs use `file:`.
-- [`libs/AGENTS.md`](../libs/AGENTS.md) — shared helpers.
-- Fork's [`AGENTS.md`](https://github.com/CroneKorkN/bundlewrap/blob/main/AGENTS.md)
-  — bundlewrap-language reference + safety envelope.

bundles/apcupsd/metadata.py

@@ -13,14 +13,16 @@ defaults = {
         },
     },
     'telegraf': {
-        'inputs': {
-            'exec': {
-                'apcupsd': {
-                    'commands': ["sudo /usr/local/share/telegraf/apcupsd"],
-                    'name_override': "apcupsd",
-                    'data_format': "influx",
-                    'interval': '30s',
-                    'flush_interval': '30s',
+        'config': {
+            'inputs': {
+                'exec': {
+                    repo.libs.hashable.hashable({
+                        'commands': ["sudo /usr/local/share/telegraf/apcupsd"],
+                        'name_override': "apcupsd",
+                        'data_format': "influx",
+                        'interval': '30s',
+                        'flush_interval': '30s',
+                    }),
                 },
             },
         },

bundles/apt/metadata.py

@@ -4,8 +4,6 @@ defaults = {
             'apt-listchanges': {
                 'installed': False,
             },
-            'ca-certificates': {},
-            'unattended-upgrades': {},
         },
         'config': {
             'DPkg': {
@@ -23,10 +21,6 @@ defaults = {
                 },
             },
             'APT': {
-                'Periodic': {
-                    'Update-Package-Lists': '1',
-                    'Unattended-Upgrade': '1',
-                },
                 'NeverAutoRemove': {
                     '^firmware-linux.*',
                     '^linux-firmware$',
@@ -54,11 +48,6 @@ defaults = {
                     'Error-Mode': 'any',
                 },
             },
-            'Unattended-Upgrade': {
-                'Origins-Pattern': {
-                    "origin=*",
-                },
-            },
         },
         'sources': {},
     },
@@ -117,6 +106,33 @@ def signed_by(metadata):
     }
 
 
+@metadata_reactor.provides(
+    'apt/config',
+    'apt/packages',
+)
+def unattended_upgrades(metadata):
+    return {
+        'apt': {
+            'config': {
+                'APT': {
+                    'Periodic': {
+                        'Update-Package-Lists': '1',
+                        'Unattended-Upgrade': '1',
+                    },
+                },
+                'Unattended-Upgrade': {
+                    'Origins-Pattern': {
+                        "origin=*",
+                    },
+                },
+            },
+            'packages': {
+                'unattended-upgrades': {},
+            },
+        },
+    }
+
+
 # @metadata_reactor.provides(
 #     'apt/config',
 #     'apt/list_changes',

bundles/bind-acme/metadata.py

@@ -33,7 +33,6 @@ def acme_zone(metadata):
             str(ip_interface(other_node.metadata.get('network/internal/ipv4')).ip)
                 for other_node in repo.nodes
                 if other_node.metadata.get('letsencrypt/domains', {})
-                and other_node.metadata.get('network/internal/ipv4', None)
         },
         *{
             str(ip_interface(other_node.metadata.get('wireguard/my_ip')).ip)

bundles/bind/README.md

@@ -1,30 +0,0 @@
-# bind
-
-Authoritative DNS — primary plus optional `bind/master_node` slaves.
-
-## Applying changes needs both nodes
-
-The slave's bw-managed zone files are rendered from the master's
-metadata at slave-apply time (see `bundles/bind/items.py:100`). When
-you change a record on the master (adding a `letsencrypt/domains`
-entry, a new vhost, etc.), the change is only published once you
-apply BOTH:
-
-```sh
-bw apply htz.mails        # primary (where the source records live)
-bw apply ovh.secondary    # secondary (renders its own zone files)
-```
-
-Until both have been applied, `bw verify ovh.secondary` will show
-stale zones and consumers that hit the secondary (Let's Encrypt's
-secondary-region validators in particular) will see NXDOMAIN. Even
-though the slave's named.conf.local declares `type slave;`, don't
-rely on bind's own AXFR catching up — the bw-rendered file on disk
-is what `bw verify` measures.
-
-## See also
-
-- `bundles/bind-acme/` — the in-house ACME-update receiver.
-- `bundles/letsencrypt/README.md` — DNS-01 prerequisites and the
-  negative-cache penalty (the most common operational consequence
-  of forgetting to apply the secondary).

bundles/bind/metadata.py

@@ -49,13 +49,13 @@ defaults = {
         },
     },
     'telegraf': {
-        'inputs': {
-            'bind': {
-                'default': {
+        'config': {
+            'inputs': {
+                'bind': [{
                     'urls': ['http://localhost:8053/xml/v3'],
                     'gather_memory_contexts': False,
                     'gather_views': True,
-                },
+                }],
             },
         },
     },

bundles/bootshorn/files/process

@@ -112,11 +112,6 @@ def process_recording(filename):
 
         sample_num += samples_per_block - overlapping_samples
 
-    # move to PROCESSED_RECORDINGS_DIR
-
-    os.makedirs(PROCESSED_RECORDINGS_DIR, exist_ok=True)
-    shutil.move(os.path.join(RECORDINGS_DIR, filename), os.path.join(PROCESSED_RECORDINGS_DIR, filename))
-
 
 # write a spectrogram using the sound from start to end of the event
 def write_event(current_event, soundfile, samplerate):

bundles/bootshorn/files/record

@@ -19,7 +19,5 @@ do
         -t "3600" \
         -c:a flac \
         -compression_level 12 \
-        "recordings/current/$DATE.flac"
-
-    mv "recordings/current/$DATE.flac" "recordings/$DATE.flac"
+        "recordings/$DATE.flac"
 done

bundles/bootshorn/files/temperature

@@ -8,7 +8,7 @@ urllib3.disable_warnings()
 import os
 
 
-HUE_IP = "${hue_ip}"  # replace with your bridge IP
+HUE_IP = "10.0.0.134"  # replace with your bridge IP
 HUE_APP_KEY = "${hue_app_key}" # local only
 HUE_DEVICE_ID = "31f58786-3242-4e88-b9ce-23f44ba27bbe"
 TEMPERATURE_LOG_DIR = "/opt/bootshorn/temperatures"

bundles/bootshorn/items.py

@@ -7,15 +7,11 @@ directories = {
         'owner': 'ckn',
         'group': 'ckn',
     },
-    '/opt/bootshorn/temperatures': {
-        'owner': 'ckn',
-        'group': 'ckn',
-    },
     '/opt/bootshorn/recordings': {
         'owner': 'ckn',
         'group': 'ckn',
     },
-    '/opt/bootshorn/recordings/current': {
+    '/opt/bootshorn/temperatures': {
         'owner': 'ckn',
         'group': 'ckn',
     },
@@ -38,7 +34,6 @@ files = {
     '/opt/bootshorn/temperature': {
         'content_type': 'mako',
         'context': {
-            'hue_ip': repo.get_node('home.hue').hostname,
             'hue_app_key': repo.vault.decrypt('encrypt$gAAAAABoc2WxZCLbxl-Z4IrSC97CdOeFgBplr9Fp5ujpd0WCCCPNBUY_WquHN86z8hKLq5Y04dwq8TdJW0PMSOSgTFbGgdp_P1q0jOBLEKaW9IIT1YM88h-JYwLf9QGDV_5oEfvnBCtO'),
         },
         'owner': 'ckn',

bundles/build-agent/metadata.py

@@ -27,7 +27,7 @@ def ssh_keys(metadata):
         'users': {
             'build-agent': {
                 'authorized_users': {
-                    f'build-server@{other_node.name}': {}
+                    f'build-server@{other_node.name}'
                         for other_node in repo.nodes
                         if other_node.has_bundle('build-server')
                         for architecture in other_node.metadata.get('build-server/architectures').values()

bundles/build-ci/metadata.py

@@ -14,7 +14,7 @@ def ssh_keys(metadata):
         'users': {
             'build-ci': {
                 'authorized_users': {
-                    f'build-server@{other_node.name}': {}
+                    f'build-server@{other_node.name}'
                         for other_node in repo.nodes
                         if other_node.has_bundle('build-server')
                 },

bundles/crystal/metadata.py

@@ -8,7 +8,6 @@ defaults = {
         'sources': {
             'crystal': {
                 # https://software.opensuse.org/download.html?project=devel%3Alanguages%3Acrystal&package=crystal
-                # curl -fsSL https://download.opensuse.org/repositories/devel:/languages:/crystal/Debian_Testing/Release.key
                 'urls': {
                     'http://download.opensuse.org/repositories/devel:/languages:/crystal/Debian_Testing/',
                 },

bundles/download-server/metadata.py

@@ -5,11 +5,6 @@ defaults = {
             'needs': {
                 'zfs_dataset:tank/downloads'
             },
-            'authorized_users': {
-                f'build-server@{other_node.name}': {}
-                    for other_node in repo.nodes
-                    if other_node.has_bundle('build-server')
-            },
         },
     },
     'zfs': {
@@ -19,15 +14,23 @@ defaults = {
             },
         },
     },
-    'systemd-mount': {
-        '/var/lib/downloads_nginx': {
-            'source': '/var/lib/downloads',
-            'user': 'www-data',
-        },
-    },
 }
 
 
+@metadata_reactor.provides(
+    'systemd-mount'
+)
+def mount_certs(metadata):
+    return  {
+        'systemd-mount': {
+            '/var/lib/downloads_nginx': {
+                'source': '/var/lib/downloads',
+                'user': 'www-data',
+            },
+        },
+    }
+
+
 @metadata_reactor.provides(
     'nginx/vhosts',
 )
@@ -44,3 +47,20 @@ def nginx(metadata):
             },
         },
     }
+
+
+@metadata_reactor.provides(
+    'users/downloads/authorized_users',
+)
+def ssh_keys(metadata):
+    return {
+        'users': {
+            'downloads': {
+                'authorized_users': {
+                    f'build-server@{other_node.name}'
+                        for other_node in repo.nodes
+                        if other_node.has_bundle('build-server')
+                },
+            },
+        },
+    }

bundles/flask/metadata.py

@@ -43,11 +43,11 @@ def units(metadata):
                     'Service': {
                         'Environment': {
                             f'{k}={v}'
-                                for k, v in metadata.get(f'flask/{name}/env', {}).items()
+                                for k, v in conf.get('env', {}).items()
                         },
-                        'User': metadata.get(f'flask/{name}/user'),
-                        'Group': metadata.get(f'flask/{name}/group'),
-                        'ExecStart': f"/opt/{name}/venv/bin/gunicorn -w {metadata.get(f'flask/{name}/workers')} -b 127.0.0.1:{metadata.get(f'flask/{name}/port')} --timeout {metadata.get(f'flask/{name}/timeout')} {metadata.get(f'flask/{name}/app_module')}:app"
+                        'User': conf['user'],
+                        'Group': conf['group'],
+                        'ExecStart': f"/opt/{name}/venv/bin/gunicorn -w {conf['workers']} -b 127.0.0.1:{conf['port']} --timeout {conf['timeout']} {conf['app_module']}:app"
                     },
                     'Install': {
                         'WantedBy': {
@@ -55,7 +55,7 @@ def units(metadata):
                         }
                     },
                 }
-                    for name in metadata.get('flask')
+                    for name, conf in metadata.get('flask').items()
             }
         }
     }

bundles/grafana/items.py

@@ -127,7 +127,7 @@ for dashboard_id, monitored_node in enumerate(monitored_nodes, start=1):
             panel['gridPos']['y'] = (row_id - 1) * panel['gridPos']['h']
 
             if 'display_name' in panel_config:
-                panel['fieldConfig']['defaults']['displayName'] = panel_config['display_name']
+                panel['fieldConfig']['defaults']['displayName'] = '${'+panel_config['display_name']+'}'
 
             if panel_config.get('stacked'):
                 panel['fieldConfig']['defaults']['custom']['stacking']['mode'] = 'normal'
@@ -158,14 +158,13 @@ for dashboard_id, monitored_node in enumerate(monitored_nodes, start=1):
                         host=monitored_node.name,
                         negative=query_config.get('negative', False),
                         boolean_to_int=query_config.get('boolean_to_int', False),
-                        over=query_config.get('over', None),
+                        minimum=query_config.get('minimum', None),
                         filters={
                             'host': monitored_node.name,
                             **query_config['filters'],
                         },
                         exists=query_config.get('exists', []),
                         function=query_config.get('function', None),
-                        multiply=query_config.get('multiply', None),
                     ).strip()
                 })
 
@@ -179,3 +178,4 @@ for dashboard_id, monitored_node in enumerate(monitored_nodes, start=1):
             'svc_systemd:grafana-server:restart',
         ]
     }
+

bundles/hardware/items.py

@@ -2,7 +2,7 @@ files = {
     '/usr/local/share/telegraf/cpu_frequency': {
         'mode': '0755',
         'triggers': {
-            'svc_systemd:telegraf.service:restart',
+            'svc_systemd:telegraf:restart',
         },
     },
 }

bundles/hardware/metadata.py

@@ -14,25 +14,25 @@ defaults = {
         },
     },
     'telegraf': {
-        'inputs': {
-            'sensors': {
-                'default': {
+        'config': {
+            'inputs': {
+                'sensors': {repo.libs.hashable.hashable({
                     'timeout': '2s',
+                })},
+                'exec': {
+                    repo.libs.hashable.hashable({
+                        'commands': ["sudo /usr/local/share/telegraf/cpu_frequency"],
+                        'name_override': "cpu_frequency",
+                        'data_format': "influx",
+                    }),
+                    # repo.libs.hashable.hashable({
+                    #     'commands': ["/bin/bash -c 'expr $(cat /sys/class/thermal/thermal_zone0/temp) / 1000'"],
+                    #     'name_override': "cpu_temperature",
+                    #     'data_format': "value",
+                    #     'data_type': "integer",
+                    # }),
                 },
             },
-            'exec': {
-                'cpu_frequency': {
-                    'commands': ["sudo /usr/local/share/telegraf/cpu_frequency"],
-                    'name_override': "cpu_frequency",
-                    'data_format': "influx",
-                },
-                # repo.libs.hashable.hashable({
-                #     'commands': ["/bin/bash -c 'expr $(cat /sys/class/thermal/thermal_zone0/temp) / 1000'"],
-                #     'name_override': "cpu_temperature",
-                #     'data_format': "value",
-                #     'data_type': "integer",
-                # }),
-            },
         },
     },
 }

bundles/influxdb2/metadata.py

@@ -39,17 +39,6 @@ defaults = {
     },
 }
 
-if node.has_bundle('zfs'):
-    defaults['zfs'] = {
-        'datasets': {
-            'tank/influxdb': {
-                'mountpoint': '/var/lib/influxdb',
-                'recordsize': '8192',
-                'atime': 'off',
-            },
-        },
-    }
-
 @metadata_reactor.provides(
     'influxdb/password',
     'influxdb/admin_token',
@@ -63,6 +52,26 @@ def admin_password(metadata):
     }
 
 
+@metadata_reactor.provides(
+    'zfs/datasets',
+)
+def zfs(metadata):
+    if not node.has_bundle('zfs'):
+        return {}
+
+    return {
+        'zfs': {
+            'datasets': {
+                'tank/influxdb': {
+                    'mountpoint': '/var/lib/influxdb',
+                    'recordsize': '8192',
+                    'atime': 'off',
+                },
+            },
+        },
+    }
+
+
 @metadata_reactor.provides(
     'dns',
 )

bundles/left4dead2/README.md

@@ -1,22 +1 @@
 https://github.com/SirPlease/L4D2-Competitive-Rework/blob/master/Dedicated%20Server%20Install%20Guide/README.md
-
-```python
-    'tick60_maps': {
-        'port': 27030,
-        # add command line arguments
-        'arguments': ['-tickrate 60'],
-        # stack overlays, first is uppermost
-        'overlays': ['tickrate', 'standard'],
-        # server.cfg contents
-        'config': [
-            # configs from overlays are accessible via server_${overlay}.cfg
-            'exec server_tickrate.cfg',
-            # add more options
-            'sv_minupdaterate 101',
-            'sv_maxupdaterate 101',
-            'sv_mincmdrate 101',
-            'sv_maxcmdrate 101',
-            'sv_consistency 0',
-        ],
-    },
-```

bundles/left4dead2/files/scripts/helpers

@@ -1,13 +0,0 @@
-#!/bin/bash
-set -xeuo pipefail
-
-function steam() {
-    # for systemd, so it can terminate the process (for other things sudo would have been enough)
-    setpriv --reuid=steam --regid=steam --init-groups "$@" <&0
-    export HOME=/opt/l4d2/steam
-}
-
-function workshop() {
-    steam mkdir -p "/opt/l4d2/overlays/${overlay}/left4dead2/addons"
-    steam /opt/l4d2/scripts/steam-workshop-download --out "/opt/l4d2/overlays/${overlay}/left4dead2/addons" "$@"
-}

bundles/left4dead2/files/scripts/overlays/competitive_rework

@@ -1,10 +0,0 @@
-#!/bin/bash
-set -xeuo pipefail
-source /opt/l4d2/scripts/helpers
-overlay=$(basename "$0")
-
-# https://github.com/SirPlease/L4D2-Competitive-Rework
-
-steam mkdir -p /opt/l4d2/overlays/$overlay/left4dead2
-test -d /opt/l4d2/overlays/$overlay/left4dead2/cfg/cfgogl || \
-    curl -L https://github.com/SirPlease/L4D2-Competitive-Rework/archive/refs/heads/master.tar.gz | steam tar -xz --strip-components=1 -C /opt/l4d2/overlays/$overlay/left4dead2

bundles/left4dead2/files/scripts/overlays/l4d2center_maps

@@ -1,128 +0,0 @@
-#!/bin/bash
-set -xeuo pipefail
-source /opt/l4d2/scripts/helpers
-overlay=$(basename "$0")
-
-steam mkdir -p /opt/l4d2/overlays/$overlay/left4dead2/addons
-cd /opt/l4d2/overlays/$overlay/left4dead2/addons
-
-# https://l4d2center.com/maps/servers/l4d2center_maps_sync.sh.txt ->
-
-# Exit immediately if a command exits with a non-zero status.
-set -e
-
-# Function to print error messages
-error_exit() {
-    echo "Error: $1" >&2
-    exit 1
-}
-
-# Check if the current directory ends with /left4dead2/addons
-current_dir=$(pwd)
-expected_dir="/left4dead2/addons"
-
-if [[ ! "$current_dir" == *"$expected_dir" ]]; then
-    error_exit "Script must be run from your L4D2 \"addons\" folder. Current directory: $current_dir"
-fi
-
-# Check for required commands
-for cmd in curl md5sum 7z; do
-    if ! command -v "$cmd" >/dev/null 2>&1; then
-        error_exit "Required command '$cmd' is not installed. Please install it and retry."
-    fi
-done
-
-# URL of the CSV file
-CSV_URL="https://l4d2center.com/maps/servers/index.csv"
-
-# Temporary file to store CSV
-TEMP_CSV=$(mktemp)
-
-# Ensure temporary file is removed on exit
-trap 'rm -f "$TEMP_CSV"' EXIT
-
-echo "Downloading CSV from $CSV_URL..."
-curl -sSL -o "$TEMP_CSV" "$CSV_URL" || error_exit "Failed to download CSV."
-
-declare -A map_md5
-declare -A map_links
-
-# Read CSV and populate associative arrays
-{
-    # Skip the first line (header)
-    IFS= read -r header
-
-    while IFS=';' read -r Name Size MD5 DownloadLink || [[ $Name ]]; do
-        # Trim whitespace
-        Name=$(echo "$Name" | xargs)
-        MD5=$(echo "$MD5" | xargs)
-        DownloadLink=$(echo "$DownloadLink" | xargs)
-
-        # Populate associative arrays
-        map_md5["$Name"]="$MD5"
-        map_links["$Name"]="$DownloadLink"
-    done
-} < "$TEMP_CSV"
-
-# Get list of expected VPK files
-expected_vpk=("${!map_md5[@]}")
-
-# Remove VPK files not in expected list or with mismatched MD5
-echo "Cleaning up existing VPK files..."
-for file in *.vpk; do
-    # Check if it's a regular file
-    if [[ -f "$file" ]]; then
-        if [[ -z "${map_md5["$file"]}" ]]; then
-            echo "Removing unexpected file: $file"
-            rm -f "$file"
-        else
-            # Calculate MD5
-            echo "Calculating MD5 for existing file: $file..."
-            current_md5=$(md5sum "$file" | awk '{print $1}')
-            expected_md5="${map_md5["$file"]}"
-
-            if [[ "$current_md5" != "$expected_md5" ]]; then
-                echo "MD5 mismatch for $file. Removing."
-                rm -f "$file"
-            fi
-        fi
-    fi
-done
-
-# Download and extract missing or updated VPK files
-echo "Processing required VPK files..."
-for vpk in "${expected_vpk[@]}"; do
-    if [[ ! -f "$vpk" ]]; then
-        echo "Downloading and extracting $vpk..."
-        download_url="${map_links["$vpk"]}"
-
-        if [[ -z "$download_url" ]]; then
-            echo "No download link found for $vpk. Skipping."
-            continue
-        fi
-
-        encoded_url=$(echo "$download_url" | sed 's/ /%20/g')
-
-        # Download the .7z file to a temporary location
-        TEMP_7Z=$(mktemp --suffix=.7z)
-        curl -# -L -o "$TEMP_7Z" "$encoded_url"
-
-        # Check if the download was successful
-        if [[ $? -ne 0 ]]; then
-            echo "Failed to download $download_url. Skipping."
-            rm -f "$TEMP_7Z"
-            continue
-        fi
-
-        # Extract the .7z file
-        7z x -y "$TEMP_7Z" || { echo "Failed to extract $TEMP_7Z. Skipping."; rm -f "$TEMP_7Z"; continue; }
-
-        # Remove the temporary .7z file
-        rm -f "$TEMP_7Z"
-
-    else
-        echo "$vpk is already up to date."
-    fi
-done
-
-echo "Synchronization complete."

bundles/left4dead2/files/scripts/overlays/standard

@@ -1,12 +0,0 @@
-#!/bin/bash
-set -xeuo pipefail
-source /opt/l4d2/scripts/helpers
-overlay=$(basename "$0")
-
-# Ions Vocalizer
-workshop -i 698857882
-
-# admin system
-workshop --item 2524204971
-steam mkdir -p "/opt/l4d2/overlays/${overlay}/left4dead2/ems/admin system"
-steam echo "STEAM_1:0:12376499" > "/opt/l4d2/overlays/${overlay}/left4dead2/ems/admin system/admins.txt"

bundles/left4dead2/files/scripts/overlays/tickrate

@@ -1,25 +0,0 @@
-#!/bin/bash
-set -xeuo pipefail
-source /opt/l4d2/scripts/helpers
-overlay=$(basename "$0")
-
-# server config
-# https://github.com/SirPlease/L4D2-Competitive-Rework/blob/7ecc3a32a5e2180d6607a40119ff2f3c072502a9/cfg/server.cfg#L58-L69
-# https://www.programmersought.com/article/513810199514/
-steam mkdir -p /opt/l4d2/overlays/$overlay/left4dead2/cfg
-steam cat <<'EOF' > /opt/l4d2/overlays/$overlay/left4dead2/cfg/server.cfg
-# https://github.com/SirPlease/L4D2-Competitive-Rework/blob/7ecc3a32a5e2180d6607a40119ff2f3c072502a9/cfg/server.cfg#L58-L69
-sv_minrate 100000
-sv_maxrate 100000
-nb_update_frequency 0.014
-net_splitpacket_maxrate 50000
-net_maxcleartime 0.0001
-fps_max 0
-EOF
-
-# install tickrate enabler
-steam mkdir -p "/opt/l4d2/overlays/${overlay}/left4dead2/addons"
-for file in tickrate_enabler.dll tickrate_enabler.so tickrate_enabler.vdf
-do
-    curl -L "https://github.com/SirPlease/L4D2-Competitive-Rework/raw/refs/heads/master/addons/${file}" -o "/opt/l4d2/overlays/${overlay}/left4dead2/addons/${file}"
-done

bundles/left4dead2/files/scripts/overlays/workshop_maps

@@ -1,13 +0,0 @@
-#!/bin/bash
-set -xeuo pipefail
-source /opt/l4d2/scripts/helpers
-overlay=$(basename "$0")
-
-# workshop --collection 121115793 # Back To School
-
-# workshop --item 2957035482 # hehe30-part1
-# workshop --item 2973628334 # hehe30-part2
-# workshop --item 3013844371 # hehe30-part3
-
-# workshop --item 3478461158 # 虚伪黎明(Dawn's Deception)
-# workshop --item 3478934394 # 虚伪黎明(Dawn's Deception)PART2

bundles/left4dead2/files/server.cfg

@@ -1,13 +0,0 @@
-// defaults
-hostname ${server_name}
-motd_enabled 0
-rcon_password ${rcon_password}
-sv_steamgroup "38347879"
-
-mp_autoteambalance 0
-sv_forcepreload 1
-
-// server specific
-% for line in config:
-${line}
-% endfor

bundles/left4dead2/files/setup

@@ -2,16 +2,6 @@
 
 set -xeuo pipefail
 
-# -- DEFINE FUNCTIONS AND VARIABLES -- #
-
-function steam() {
-    # for systemd, so it can terminate the process (for other things sudo would have been enough)
-    setpriv --reuid=steam --regid=steam --init-groups "$@" <&0
-    export HOME=/opt/l4d2/steam
-}
-
-# -- PREPARE SYSTEM -- #
-
 getent passwd steam >/dev/null || useradd -M -d /opt/l4d2 -s /bin/bash steam
 mkdir -p /opt/l4d2 /tmp/dumps
 chown steam:steam /opt/l4d2 /tmp/dumps
@@ -19,25 +9,26 @@ dpkg --add-architecture i386
 apt update
 DEBIAN_FRONTEND=noninteractive apt install -y libc6:i386 lib32z1
 
-# workshop downloader
-test -f /opt/l4d2/scripts/steam-workshop-download || \
-    steam wget -4 https://git.sublimity.de/cronekorkn/steam-workshop-downloader/raw/branch/master/steam-workshop-download -P /opt/l4d2/scripts
-steam chmod +x /opt/l4d2/scripts/steam-workshop-download
+function steam() {
+    # für systemd, damit es den prozess beenden kann
+    setpriv --reuid=steam --regid=steam --init-groups "$@"
+    export HOME=/opt/l4d2/steam
+}
 
 # -- STEAM -- #
 
 steam mkdir -p /opt/l4d2/steam
 test -f /opt/l4d2/steam/steamcmd_linux.tar.gz || \
-    steam wget http://media.steampowered.com/installer/steamcmd_linux.tar.gz -P /opt/l4d2/steam
+  steam wget http://media.steampowered.com/installer/steamcmd_linux.tar.gz -P /opt/l4d2/steam
 test -f /opt/l4d2/steam/steamcmd.sh || \
-    steam tar -xvzf /opt/l4d2/steam/steamcmd_linux.tar.gz -C /opt/l4d2/steam
+  steam tar -xvzf /opt/l4d2/steam/steamcmd_linux.tar.gz -C /opt/l4d2/steam
 
 # fix for: /opt/l4d2/.steam/sdk32/steamclient.so: cannot open shared object file: No such file or directory
 steam mkdir -p /opt/l4d2/steam/.steam # needs to be in steam users home dir
 readlink /opt/l4d2/steam/.steam/sdk32 | grep -q ^/opt/l4d2/steam/linux32$ || \
-    steam ln -sf /opt/l4d2/steam/linux32 /opt/l4d2/steam/.steam/sdk32
+  steam ln -sf /opt/l4d2/steam/linux32 /opt/l4d2/steam/.steam/sdk32
 readlink /opt/l4d2/steam/.steam/sdk64 | grep -q ^/opt/l4d2/steam/linux64$ || \
-    steam ln -sf /opt/l4d2/steam/linux64 /opt/l4d2/steam/.steam/sdk64
+  steam ln -sf /opt/l4d2/steam/linux64 /opt/l4d2/steam/.steam/sdk64
 
 # -- INSTALL -- #
 
@@ -58,13 +49,46 @@ steam /opt/l4d2/steam/steamcmd.sh \
 
 # -- OVERLAYS -- #
 
-for overlay_path in /opt/l4d2/scripts/overlays/*; do
-    overlay=$(basename "$overlay_path")
-    steam mkdir -p /opt/l4d2/overlays/$overlay
-    bash -xeuo pipefail "$overlay_path"
-    test -f /opt/l4d2/overlays/$overlay/left4dead2/cfg/server.cfg && \
-        steam cp /opt/l4d2/overlays/$overlay/left4dead2/cfg/server.cfg /opt/l4d2/overlays/$overlay/left4dead2/cfg/server_$overlay.cfg
-done
+steam mkdir -p /opt/l4d2/overlays
+
+# workshop downloader
+test -f /opt/l4d2/steam-workshop-download || \
+    steam wget -4 https://git.sublimity.de/cronekorkn/steam-workshop-downloader/raw/branch/master/steam-workshop-download -P /opt/l4d2
+steam chmod +x /opt/l4d2/steam-workshop-download
+
+# -- OVERLAY PVE -- #
+
+steam mkdir -p /opt/l4d2/overlays/pve
+
+# server config
+steam mkdir -p /opt/l4d2/overlays/pve/left4dead2/cfg
+steam cat <<'EOF' > /opt/l4d2/overlays/pve/left4dead2/cfg/server.cfg
+motd_enabled 0
+
+sv_steamgroup "38347879"
+#sv_steamgroup_exclusive 0
+
+sv_minrate 60000
+sv_maxrate 0
+net_splitpacket_maxrate 60000
+
+#sv_cheats 1
+#sb_all_bot_game 1
+EOF
+
+# admin system
+steam mkdir -p /opt/l4d2/overlays/pve/left4dead2/addons
+test -f /opt/l4d2/overlays/pve/left4dead2/addons/2524204971.vpk || \
+    steam /opt/l4d2/steam-workshop-download 2524204971 --out /opt/l4d2/overlays/pve/left4dead2/addons
+steam mkdir -p "/opt/l4d2/overlays/pve/left4dead2/ems/admin system"
+steam echo "STEAM_1:0:12376499" > "/opt/l4d2/overlays/pve/left4dead2/ems/admin system/admins.txt"
+
+# ions vocalizer
+test -f /opt/l4d2/overlays/pve/left4dead2/addons/698857882.vpk || \
+    steam /opt/l4d2/steam-workshop-download 698857882 --out /opt/l4d2/overlays/pve/left4dead2/addons
+
+test -f /opt/l4d2/overlays/pve/left4dead2/addons/1575673903.vpk || \
+    steam /opt/l4d2/steam-workshop-download 1575673903 --out /opt/l4d2/overlays/pve/left4dead2/addons
 
 # -- SERVERS -- #
 

bundles/left4dead2/files/start

@@ -2,41 +2,9 @@
 
 set -xeuo pipefail
 
-name=""
-port=""
-configfile=""
-overlays=""
-arguments=""
-
-while [[ $# -gt 0 ]]; do
-  case "$1" in
-    -n|--name)
-      name="$2"; shift 2
-      ;;
-    -p|--port)
-      port="$2"; shift 2
-      ;;
-    -c|--config)
-      configfile="$2"; shift 2
-      ;;
-    -o|--overlay)
-      overlays="/opt/l4d2/overlays/$2:$overlays"; shift 2
-      ;;
-    --)
-      shift
-      arguments+="$@"
-      break
-      ;;
-    *)
-      echo "ERROR: unknown argument $1"; exit 1
-      ;;
-  esac
-done
-
-[[ -n "${name}" ]] || { echo "ERROR: -n/--name missing"; exit 1; }
-[[ -n "${port}" ]] || { echo "ERROR: -p/--port missing"; exit 1; }
-
-# -- HELPER FUNCTIONS -- #
+name=$1
+overlay=$2
+port=$3
 
 function steam() {
     # für systemd, damit es den prozess beenden kann
@@ -44,32 +12,17 @@ function steam() {
     export HOME=/opt/l4d2/steam
 }
 
-# -- TIDY UP -- #
-
 mountpoint -q "/opt/l4d2/servers/$name/merged" && umount "/opt/l4d2/servers/$name/merged"
 steam rm -rf "/opt/l4d2/servers/$name"
 
-# -- CREATE DIRECTORIES -- #
-
 steam mkdir -p \
   "/opt/l4d2/servers/$name" \
   "/opt/l4d2/servers/$name/work" \
   "/opt/l4d2/servers/$name/upper" \
   "/opt/l4d2/servers/$name/merged"
 
-# -- MOUNT OVERLAYFS -- #
-
 mount -t overlay overlay \
-  -o "lowerdir=$overlays/opt/l4d2/installation,upperdir=/opt/l4d2/servers/$name/upper,workdir=/opt/l4d2/servers/$name/work" \
+  -o "lowerdir=/opt/l4d2/overlays/$overlay:/opt/l4d2/installation,upperdir=/opt/l4d2/servers/$name/upper,workdir=/opt/l4d2/servers/$name/work" \
   "/opt/l4d2/servers/$name/merged"
 
-# -- REPLACE SERVER.CFG -- #
-
-if [[ -n "$configfile" ]]; then
-  cp "$configfile" "/opt/l4d2/servers/$name/merged/left4dead2/cfg/server.cfg"
-  chown steam:steam "/opt/l4d2/servers/$name/merged/left4dead2/cfg/server.cfg"
-fi
-
-# -- RUN L4D2 -- #
-
-steam "/opt/l4d2/servers/$name/merged/srcds_run" -norestart -pidfile "/opt/l4d2/servers/$name/pid" -game left4dead2 -ip 0.0.0.0 -port "$port" +hostname "Crone_$name" +map c1m1_hotel $arguments
+steam "/opt/l4d2/servers/$name/merged/srcds_run" -norestart -pidfile "/opt/l4d2/servers/$name/pid" -game left4dead2 -ip 0.0.0.0 -port "$port" +hostname "Crone_$name" +map c1m1_hotel

bundles/left4dead2/files/stop

@@ -1,19 +0,0 @@
-#!/bin/bash
-
-set -xeuo pipefail
-
-name=""
-
-while [[ $# -gt 0 ]]; do
-  case "$1" in
-    -n|--name)
-      name="$2"; shift 2
-      ;;
-    *)
-      echo "ERROR: unknown argument $1"; exit 1
-      ;;
-  esac
-done
-
-mountpoint -q "/opt/l4d2/servers/$name/merged" && umount "/opt/l4d2/servers/$name/merged"
-steam rm -rf "/opt/l4d2/servers/$name"

bundles/left4dead2/items.py

@@ -1,97 +1,23 @@
-users = {
-    'steam': {
-        'home': '/opt/l4d2/steam',
-        'shell': '/bin/bash',
-    },
-}
-
-directories = {
-    '/opt/l4d2': {
-        'owner': 'steam', 'group': 'steam',
-    },
-    '/opt/l4d2/steam': {
-        'owner': 'steam', 'group': 'steam',
-    },
-    '/opt/l4d2/configs': {
-        'owner': 'steam', 'group': 'steam',
-        'purge': True,
-    },
-    '/opt/l4d2/scripts': {
-        'owner': 'steam', 'group': 'steam',
-    },
-    '/opt/l4d2/scripts/overlays': {
-        'owner': 'steam', 'group': 'steam',
-        'purge': True,
-    },
-}
-
 files = {
     '/opt/l4d2/setup': {
         'mode': '755',
-        'triggers': {
-            'svc_systemd:left4dead2-initialize.service:restart',
-        },
     },
     '/opt/l4d2/start': {
         'mode': '755',
-        'triggers': {
-            f'svc_systemd:left4dead2-{server_name}.service:restart'
-                for server_name in node.metadata.get('left4dead2/servers').keys()
-        },
-    },
-    '/opt/l4d2/stop': {
-        'mode': '755',
-        'triggers': {
-            f'svc_systemd:left4dead2-{server_name}.service:restart'
-                for server_name in node.metadata.get('left4dead2/servers').keys()
-        },
-    },
-    '/opt/l4d2/scripts/helpers': {
-        'source': 'scripts/helpers',
-        'mode': '755',
-        'triggers': {
-            'svc_systemd:left4dead2-initialize.service:restart',
-        },
     },
 }
 
-for overlay in node.metadata.get('left4dead2/overlays'):
-    files[f'/opt/l4d2/scripts/overlays/{overlay}'] = {
-        'source': f'scripts/overlays/{overlay}',
-        'mode': '755',
-        'triggers': {
-            'svc_systemd:left4dead2-initialize.service:restart',
-        },
-    }
-
 svc_systemd = {
     'left4dead2-initialize.service': {
         'enabled': True,
         'running': None,
         'needs': {
-            'tag:left4dead2-packages',
-            'file:/opt/l4d2/setup',
             'file:/usr/local/lib/systemd/system/left4dead2-initialize.service',
         },
     },
 }
 
-for server_name, config in node.metadata.get('left4dead2/servers').items():
-    files[f'/opt/l4d2/configs/{server_name}.cfg'] = {
-        'source': 'server.cfg',
-        'content_type': 'mako',
-        'context': {
-            'server_name': server_name,
-            'rcon_password': repo.vault.decrypt('encrypt$gAAAAABpAdZhxwJ47I1AXotuZmBvyZP1ecVTt9IXFkLI28JiVS74LKs9QdgIBz-FC-iXtIHHh_GVGxxKQZprn4UrXZcvZ57kCKxfHBs3cE2JiGnbWE8_mfs=').value,
-            'config': config.get('config', []),
-        },
-        'owner': 'steam',
-        'mode': '644',
-        'triggers': {
-            f'svc_systemd:left4dead2-{server_name}.service:restart',
-        },
-    }
-
+for server_name in node.metadata.get('left4dead2').keys():
     svc_systemd[f'left4dead2-{server_name}.service'] = {
         'enabled': True,
         'running': True,
@@ -101,5 +27,5 @@ for server_name, config in node.metadata.get('left4dead2/servers').items():
         'needs': {
             'svc_systemd:left4dead2-initialize.service',
             f'file:/usr/local/lib/systemd/system/left4dead2-{server_name}.service',
-        },
+        }
     }

bundles/left4dead2/metadata.py

@@ -1,36 +1,15 @@
 from re import match
-from os import path, listdir
 
 
 defaults = {
     'apt': {
         'packages': {
-            'libc6_i386': { # installs libc6:i386
-                'tags': {'left4dead2-packages'},
-            },
-            'lib32z1': {
-                'tags': {'left4dead2-packages'},
-            },
-            'unzip': {
-                'tags': {'left4dead2-packages'},
-            },
-            'p7zip-full': { # l4d2center_maps_sync.sh
-                'tags': {'left4dead2-packages'},
-            },
-        },
-    },
-    'left4dead2': {
-        'overlays': set(listdir(path.join(repo.path, 'bundles/left4dead2/files/scripts/overlays'))),
-        'servers': {
-            # 'port': 27017,
-            # 'overlays': ['competitive_rework'],
-            # 'arguments': ['-tickrate 60'],
-            # 'config': [
-            #     'exec server_original.cfg',
-            #     'sm_forcematch zonemod',
-            # ],
+            'libc6_i386': {}, # installs libc6:i386
+            'lib32z1': {},
+            'unzip': {},
         },
     },
+    'left4dead2': {},
     'nftables': {
         'input': {
             'udp dport { 27005, 27020 } accept',
@@ -65,22 +44,10 @@ defaults = {
 def server_units(metadata):
     units = {}
 
-    for name, config in metadata.get('left4dead2/servers').items():
+    for name, config in metadata.get('left4dead2').items():
         assert match(r'^[A-z0-9-_-]+$', name)
+        assert config["overlay"] in {'pve'}
         assert 27000 <= config["port"] <= 27100
-        for overlay in config.get('overlays', []):
-            assert overlay in metadata.get('left4dead2/overlays'), f"unknown overlay {overlay}, known: {metadata.get('left4dead2/overlays')}"
-
-        cmd = f'/opt/l4d2/start -n {name} -p {config["port"]}'
-
-        if 'config' in config:
-            cmd += f' -c /opt/l4d2/configs/{name}.cfg'
-
-        for overlay in config.get('overlays', []):
-            cmd += f' -o {overlay}'
-
-        if 'arguments' in config:
-            cmd += ' -- ' + ' '.join(config['arguments'])
 
         units[f'left4dead2-{name}.service'] = {
             'Unit': {
@@ -90,8 +57,7 @@ def server_units(metadata):
             },
             'Service': {
                 'Type': 'simple',
-                'ExecStart': cmd,
-                'ExecStopPost': f'/opt/l4d2/stop -n {name}',
+                'ExecStart': f'/opt/l4d2/start {name} {config["overlay"]} {config["port"]}',
                 'Restart': 'on-failure',
                 'Nice': -10,
                 'CPUWeight': 200,
@@ -101,9 +67,6 @@ def server_units(metadata):
             'Install': {
                 'WantedBy': {'multi-user.target'},
             },
-            'triggers': {
-                f'svc_systemd:left4dead2-{name}.service:restart',
-            },
         }
 
     return {
@@ -117,7 +80,7 @@ def server_units(metadata):
     'nftables/input',
 )
 def nftables(metadata):
-    ports = sorted(str(config["port"]) for config in metadata.get('left4dead2/servers').values())
+    ports = sorted(str(config["port"]) for config in metadata.get('left4dead2', {}).values())
 
     return {
         'nftables': {

bundles/left4dead2_old/README.md

@@ -0,0 +1,58 @@
+https://developer.valvesoftware.com/wiki/List_of_L4D2_Cvars
+
+Dead Center 	c1m1_hotel
+Dead Center 	c1m2_streets
+Dead Center 	c1m3_mall
+Dead Center 	c1m4_atrium
+Dark Carnival 	c2m1_highway
+Dark Carnival 	c2m2_fairgrounds
+Dark Carnival 	c2m3_coaster
+Dark Carnival 	c2m4_barns
+Dark Carnival 	c2m5_concert
+Swamp Fever 	c3m1_plankcountry
+Swamp Fever 	c3m2_swamp
+Swamp Fever 	c3m3_shantytown
+Swamp Fever 	c3m4_plantation
+Hard Rain 	c4m1_milltown_a
+Hard Rain 	c4m2_sugarmill_a
+Hard Rain 	c4m3_sugarmill_b
+Hard Rain 	c4m4_milltown_b
+Hard Rain 	c4m5_milltown_escape
+The Parish 	c5m1_waterfront_sndscape
+The Parish 	c5m1_waterfront
+The Parish 	c5m2_park
+The Parish 	c5m3_cemetery
+The Parish 	c5m4_quarter
+The Parish 	c5m5_bridge
+The Passing 	c6m1_riverbank
+The Passing 	c6m2_bedlam
+The Passing 	c6m3_port
+The Sacrifice 	c7m1_docks
+The Sacrifice 	c7m2_barge
+The Sacrifice 	c7m3_port
+No Mercy 	c8m1_apartment
+No Mercy 	c8m2_subway
+No Mercy 	c8m3_sewers
+No Mercy 	c8m4_interior
+No Mercy 	c8m5_rooftop
+Crash Course 	c9m1_alleys
+Crash Course 	c9m2_lots
+Death Toll 	c10m1_caves
+Death Toll 	c10m2_drainage
+Death Toll 	c10m3_ranchhouse
+Death Toll 	c10m4_mainstreet
+Death Toll 	c10m5_houseboat
+Dead Air 	c11m1_greenhouse
+Dead Air 	c11m2_offices
+Dead Air 	c11m3_garage
+Dead Air 	c11m4_terminal
+Dead Air 	c11m5_runway
+Blood Harvest 	c12m1_hilltop
+Blood Harvest 	c12m2_traintunnel
+Blood Harvest 	c12m3_bridge
+Blood Harvest 	c12m4_barn
+Blood Harvest 	c12m5_cornfield
+Cold Stream 	c13m1_alpinecreek
+Cold Stream 	c13m2_southpinestream
+Cold Stream 	c13m3_memorialbridge
+Cold Stream 	c13m4_cutthroatcreek

bundles/left4dead2_old/files/server.cfg

@@ -0,0 +1,40 @@
+hostname "CroneKorkN : ${name}"
+sv_contact "admin@sublimity.de"
+
+
+sv_steamgroup "${','.join(steamgroups)}"
+
+rcon_password "${rcon_password}"
+
+
+motd_enabled 0
+
+
+sv_cheats 1
+
+
+sv_consistency 0
+
+
+sv_lan 0
+
+
+sv_allow_lobby_connect_only 0
+
+
+sv_gametypes "coop,realism,survival,versus,teamversus,scavenge,teamscavenge"
+
+
+sv_minrate 30000
+sv_maxrate 60000
+sv_mincmdrate 66
+sv_maxcmdrate 101
+
+
+sv_logsdir "logs-${name}"        //Folder in the game directory where server logs will be stored.
+log on            //Creates a logfile (on | off)
+sv_logecho 0            //default 0; Echo log information to the console.
+sv_logfile 1        //default 1; Log server information in the log file.
+sv_log_onefile 0    //default 0; Log server information to only one file.
+sv_logbans 1        //default 0;Log server bans in the server logs.
+sv_logflush 0        //default 0; Flush the log files to disk on each write (slow).

bundles/left4dead2_old/items.py

@@ -0,0 +1,122 @@
+assert node.has_bundle('steam') and node.has_bundle('steam-workshop-download')
+
+directories = {
+    '/opt/steam/left4dead2-servers': {
+        'owner': 'steam',
+        'group': 'steam',
+        'mode': '0755',
+        'purge': True,
+    },
+    # Current zfs doesnt support zfs upperdir. The support was added in October 2022. Move upperdir - unused anyway -
+    # to another dir. Also move workdir alongside it, as it has to be on same fs.
+    '/opt/steam-zfs-overlay-workarounds': {
+        'owner': 'steam',
+        'group': 'steam',
+        'mode': '0755',
+        'purge': True,
+    },
+}
+
+# /opt/steam/steam/.steam/sdk32/steamclient.so: cannot open shared object file: No such file or directory
+symlinks = {
+    '/opt/steam/steam/.steam/sdk32': {
+        'target': '/opt/steam/steam/linux32',
+        'owner': 'steam',
+        'group': 'steam',
+    }
+}
+
+#
+# SERVERS
+#
+
+for name, config in node.metadata.get('left4dead2/servers').items():
+
+    #overlay
+    directories[f'/opt/steam/left4dead2-servers/{name}'] = {
+        'owner': 'steam',
+        'group': 'steam',
+    }
+    directories[f'/opt/steam-zfs-overlay-workarounds/{name}/upper'] = {
+        'owner': 'steam',
+        'group': 'steam',
+    }
+    directories[f'/opt/steam-zfs-overlay-workarounds/{name}/workdir'] = {
+        'owner': 'steam',
+        'group': 'steam',
+    }
+
+    # conf
+    files[f'/opt/steam/left4dead2-servers/{name}/left4dead2/cfg/server.cfg'] = {
+        'content_type': 'mako',
+        'source': 'server.cfg',
+        'context': {
+            'name': name,
+            'steamgroups': node.metadata.get('left4dead2/steamgroups'),
+            'rcon_password': config['rcon_password'],
+        },
+        'owner': 'steam',
+        'group': 'steam',
+        'triggers': [
+            f'svc_systemd:left4dead2-{name}.service:restart',
+        ],
+    }
+
+    # service
+    svc_systemd[f'left4dead2-{name}.service'] = {
+        'needs': [
+            f'file:/opt/steam/left4dead2-servers/{name}/left4dead2/cfg/server.cfg',
+            f'file:/usr/local/lib/systemd/system/left4dead2-{name}.service',
+        ],
+    }
+
+    #
+    # ADDONS
+    #
+
+    # base
+    files[f'/opt/steam/left4dead2-servers/{name}/left4dead2/addons/readme.txt'] = {
+        'content_type': 'any',
+        'owner': 'steam',
+        'group': 'steam',
+    }
+    directories[f'/opt/steam/left4dead2-servers/{name}/left4dead2/addons'] = {
+        'owner': 'steam',
+        'group': 'steam',
+        'purge': True,
+        'triggers': [
+            f'svc_systemd:left4dead2-{name}.service:restart',
+        ],
+    }
+    for id in [
+        *config.get('workshop', []),
+        *node.metadata.get('left4dead2/workshop'),
+    ]:
+        files[f'/opt/steam/left4dead2-servers/{name}/left4dead2/addons/{id}.vpk'] = {
+            'content_type': 'any',
+            'owner': 'steam',
+            'group': 'steam',
+            'triggers': [
+                f'svc_systemd:left4dead2-{name}.service:restart',
+            ],
+        }
+
+    # admin system
+
+    directories[f'/opt/steam/left4dead2-servers/{name}/left4dead2/ems/admin system'] = {
+        'owner': 'steam',
+        'group': 'steam',
+        'mode': '0755',
+        'triggers': [
+            f'svc_systemd:left4dead2-{name}.service:restart',
+        ],
+    }
+    files[f'/opt/steam/left4dead2-servers/{name}/left4dead2/ems/admin system/admins.txt'] = {
+        'owner': 'steam',
+        'group': 'steam',
+        'mode': '0755',
+        'content': '\n'.join(sorted(node.metadata.get('left4dead2/admins'))),
+        'triggers': [
+            f'svc_systemd:left4dead2-{name}.service:restart',
+        ],
+    }

bundles/left4dead2_old/metadata.py

@@ -0,0 +1,127 @@
+assert node.has_bundle('steam')
+
+from shlex import quote
+
+defaults = {
+    'steam': {
+        'games': {
+            'left4dead2': 222860,
+        },
+    },
+    'left4dead2': {
+        'servers': {},
+        'admins': set(),
+        'workshop': set(),
+    },
+}
+
+
+@metadata_reactor.provides(
+    'left4dead2/servers',
+)
+def rconn_password(metadata):
+    # only works from localhost!
+    return {
+        'left4dead2': {
+            'servers': {
+                server: {
+                    'rcon_password': repo.vault.password_for(f'{node.name} left4dead2 {server} rcon', length=24),
+                }
+                    for server in metadata.get('left4dead2/servers')
+            },
+        },
+    }
+
+
+@metadata_reactor.provides(
+    'steam-workshop-download',
+    'systemd/units',
+)
+def server_units(metadata):
+    units = {}
+    workshop = {}
+
+    for name, config in metadata.get('left4dead2/servers').items():
+        # mount overlay
+        mountpoint = f'/opt/steam/left4dead2-servers/{name}'
+        mount_unit_name = mountpoint[1:].replace('-', '\\x2d').replace('/', '-') + '.mount'
+        units[mount_unit_name] = {
+            'Unit': {
+                'Description': f"Mount left4dead2 server {name} overlay",
+                'Conflicts': {'umount.target'},
+                'Before': {'umount.target'},
+            },
+            'Mount': {
+                'What': 'overlay',
+                'Where': mountpoint,
+                'Type': 'overlay',
+                'Options': ','.join([
+                    'auto',
+                    'lowerdir=/opt/steam/left4dead2',
+                    f'upperdir=/opt/steam-zfs-overlay-workarounds/{name}/upper',
+                    f'workdir=/opt/steam-zfs-overlay-workarounds/{name}/workdir',
+                ]),
+            },
+            'Install': {
+                'RequiredBy': {
+                    f'left4dead2-{name}.service',
+                },
+            },
+        }
+
+        # individual workshop
+        workshop_ids = config.get('workshop', set()) | metadata.get('left4dead2/workshop', set())
+        if  workshop_ids:
+            workshop[f'left4dead2-{name}'] = {
+                'ids': workshop_ids,
+                'path': f'/opt/steam/left4dead2-servers/{name}/left4dead2/addons',
+                'user': 'steam',
+                'requires': {
+                    mount_unit_name,
+                },
+                'required_by': {
+                    f'left4dead2-{name}.service',
+                },
+            }
+
+        # left4dead2 server unit
+        units[f'left4dead2-{name}.service'] = {
+            'Unit': {
+                'Description': f'left4dead2 server {name}',
+                'After': {'steam-update.service'},
+                'Requires': {'steam-update.service'},
+            },
+            'Service': {
+                'User': 'steam',
+                'Group': 'steam',
+                'WorkingDirectory': f'/opt/steam/left4dead2-servers/{name}',
+                'ExecStart': f'/opt/steam/left4dead2-servers/{name}/srcds_run -port {config["port"]} +exec server.cfg',
+                'Restart': 'on-failure',
+            },
+            'Install': {
+                'WantedBy': {'multi-user.target'},
+            },
+        }
+
+    return {
+        'steam-workshop-download': workshop,
+        'systemd': {
+            'units': units,
+        },
+    }
+
+
+@metadata_reactor.provides(
+    'nftables/input',
+)
+def firewall(metadata):
+    ports = set(str(server['port']) for server in metadata.get('left4dead2/servers').values())
+
+    return {
+        'nftables': {
+            'input': {
+                f"tcp dport {{ {', '.join(sorted(ports))} }} accept",
+                f"udp dport {{ {', '.join(sorted(ports))} }} accept",
+            },
+        },
+    }

bundles/left4dead2_old2/README.md

@@ -0,0 +1,97 @@
+# https://github.com/SirPlease/L4D2-Competitive-Rework/blob/master/Dedicated%20Server%20Install%20Guide/README.md
+
+getent passwd steam >/dev/null || useradd -M -d /opt/l4d2 -s /bin/bash steam
+mkdir -p /opt/l4d2 /tmp/dumps
+chown steam:steam /opt/l4d2 /tmp/dumps
+dpkg --add-architecture i386
+apt update
+DEBIAN_FRONTEND=noninteractive apt install -y libc6:i386 lib32z1
+
+function steam() { sudo -Hiu steam $* }
+
+# -- STEAM -- #
+
+steam mkdir -p /opt/l4d2/steam
+test -f /opt/l4d2/steam/steamcmd_linux.tar.gz || \
+  steam wget http://media.steampowered.com/installer/steamcmd_linux.tar.gz -P /opt/l4d2/steam
+test -f /opt/l4d2/steam/steamcmd.sh || \
+  steam tar -xvzf /opt/l4d2/steam/steamcmd_linux.tar.gz -C /opt/l4d2/steam
+
+# fix: /opt/l4d2/.steam/sdk32/steamclient.so: cannot open shared object file: No such file or directory
+steam mkdir -p /opt/l4d2/steam/.steam
+test -f /opt/l4d2/steam/.steam/sdk32/steamclient.so || \
+  steam ln -s /opt/l4d2/steam/linux32 /opt/l4d2/steam/.steam/sdk32
+
+# -- INSTALL -- #
+
+# erst die windows deps zu installieren scheint ein workaround für x64 zu sein?
+steam mkdir -p /opt/l4d2/installation
+steam /opt/l4d2/steam/steamcmd.sh \
+    +force_install_dir /opt/l4d2/installation \
+    +login anonymous \
+    +@sSteamCmdForcePlatformType windows \
+    +app_update 222860 validate \
+    +quit
+steam /opt/l4d2/steam/steamcmd.sh \
+    +force_install_dir /opt/l4d2/installation \
+    +login anonymous \
+    +@sSteamCmdForcePlatformType linux \
+    +app_update 222860 validate \
+    +quit
+
+# -- OVERLAYS -- #
+
+steam mkdir -p /opt/l4d2/overlays
+
+# workshop downloader
+steam wget -4 https://git.sublimity.de/cronekorkn/steam-workshop-downloader/raw/branch/master/steam-workshop-download -P /opt/l4d2
+steam chmod +x /opt/l4d2/steam-workshop-download
+
+# -- OVERLAY PVE -- #
+
+steam mkdir -p /opt/l4d2/overlays/pve
+
+# admin system
+steam mkdir -p /opt/l4d2/overlays/pve/left4dead2/addons
+steam /opt/l4d2/steam-workshop-download 2524204971 --out /opt/l4d2/overlays/pve/left4dead2/addons
+steam mkdir -p "/opt/l4d2/overlays/pve/left4dead2/ems/admin system"
+echo "STEAM_1:0:12376499" | steam tee "/opt/l4d2/overlays/pve/left4dead2/ems/admin system/admins.txt"
+
+# ions vocalizer
+steam /opt/l4d2/steam-workshop-download 698857882 --out /opt/l4d2/overlays/pve/left4dead2/addons
+
+# -- OVERLAY ZONEMOD -- #
+
+true
+
+# -- SERVERS -- #
+
+steam mkdir -p /opt/l4d2/servers
+
+# -- SERVER PVE1 -- #
+
+steam mkdir -p \
+  /opt/l4d2/servers/pve1 \
+  /opt/l4d2/servers/pve1/work \
+  /opt/l4d2/servers/pve1/upper \
+  /opt/l4d2/servers/pve1/merged
+
+mount -t overlay overlay \
+  -o lowerdir=/opt/l4d2/overlays/pve:/opt/l4d2/installation,upperdir=/opt/l4d2/servers/pve1/upper,workdir=/opt/l4d2/servers/pve1/work \
+  /opt/l4d2/servers/pve1/merged
+
+# run server
+steam cat <<'EOF' > /opt/l4d2/servers/pve1/merged/left4dead2/cfg/server.cfg
+hostname "CKNs Server"
+motd_enabled 0
+
+sv_steamgroup "38347879"
+#sv_steamgroup_exclusive 0
+
+sv_minrate 60000
+sv_maxrate 0
+net_splitpacket_maxrate 60000
+
+sv_hibernate_when_empty 0
+EOF
+steam /opt/l4d2/servers/pve1/merged/srcds_run -game left4dead2 -ip 0.0.0.0 -port 27015 +map c1m1_hotel

bundles/left4dead2_old2/items.py

@@ -0,0 +1,183 @@
+from shlex import quote
+
+
+def steam_run(cmd):
+    return f'su - steam -c {quote(cmd)}'
+
+
+users = {
+    'steam': {
+        'home': '/opt/steam',
+    },
+}
+
+directories = {
+    '/opt/steam': {
+        'owner': 'steam',
+        'group': 'steam',
+    },
+    '/opt/steam/.steam': {
+        'owner': 'steam',
+        'group': 'steam',
+    },
+    '/opt/left4dead2': {
+        'owner': 'steam',
+        'group': 'steam',
+    },
+    '/opt/left4dead2/left4dead2/ems/admin system': {
+        'owner': 'steam',
+        'group': 'steam',
+    },
+    '/opt/left4dead2/left4dead2/addons': {
+        'owner': 'steam',
+        'group': 'steam',
+    },
+    '/tmp/dumps': {
+        'owner': 'steam',
+        'group': 'steam',
+        'mode': '1770',
+    },
+}
+
+symlinks = {
+    '/opt/steam/.steam/sdk32': {
+        'target': '/opt/steam/linux32',
+        'owner': 'steam',
+        'group': 'steam',
+    },
+}
+
+files = {
+    '/opt/steam-workshop-download': {
+        'content_type': 'download',
+        'source': 'https://git.sublimity.de/cronekorkn/steam-workshop-downloader/raw/branch/master/steam-workshop-download',
+        'mode': '755',
+    },
+    '/opt/left4dead2/left4dead2/ems/admin system/admins.txt': {
+        'unless': 'test -f /opt/left4dead2/left4dead2/ems/admin system/admins.txt',
+        'content': 'STEAM_1:0:12376499',
+        'owner': 'steam',
+        'group': 'steam',
+    },
+}
+
+actions = {
+    'dpkg_add_architecture': {
+        'command': 'dpkg --add-architecture i386',
+        'unless': 'dpkg --print-foreign-architectures | grep -q i386',
+        'triggers': [
+            'action:apt_update',
+        ],
+        'needed_by': [
+            'pkg_apt:libc6_i386',
+        ],
+    },
+    'download_steam': {
+        'command': steam_run('wget http://media.steampowered.com/installer/steamcmd_linux.tar.gz -P /opt/steam'),
+        'unless':  steam_run('test -f /opt/steam/steamcmd_linux.tar.gz'),
+        'needs': {
+            'pkg_apt:libc6_i386',
+            'directory:/opt/steam',
+        }
+    },
+    'extract_steamcmd': {
+        'command': steam_run('tar -xvzf /opt/steam/steamcmd_linux.tar.gz -C /opt/steam'),
+        'unless': steam_run('test -f /opt/steam/steamcmd.sh'),
+        'needs': {
+            'action:download_steam',
+        }
+    },
+}
+
+for addon_id in [2524204971]:
+    actions[f'download-left4dead2-addon-{addon_id}'] = {
+        'command': steam_run(f'/opt/steam-workshop-download {addon_id} --out /opt/left4dead2/left4dead2/addons'),
+        'unless': steam_run(f'test -f /opt/left4dead2/left4dead2/addons/{addon_id}.vpk'),
+        'needs': {
+            'directory:/opt/left4dead2/left4dead2/addons',
+        },
+        'needed_by': {
+            'tag:left4dead2-servers',
+        },
+    }
+
+svc_systemd = {
+    'left4dead2-install.service': {
+        'enabled': True,
+        'running': False,
+        'needs': {
+            'file:/usr/local/lib/systemd/system/left4dead2-install.service',
+        },
+    },
+}
+
+for server_name, server_config in node.metadata.get('left4dead2/servers', {}).items():
+    svc_systemd[f'left4dead2-{server_name}.service'] = {
+        'enabled': True,
+        'running': True,
+        'tags': {
+            'left4dead2-servers',
+        },
+        'needs': {
+            'svc_systemd:left4dead2-install.service',
+            f'file:/usr/local/lib/systemd/system/left4dead2-{server_name}.service',
+        }
+    }
+
+
+
+# # https://github.com/SirPlease/L4D2-Competitive-Rework/blob/master/Dedicated%20Server%20Install%20Guide/README.md
+
+# mkdir /opt/steam /tmp/dumps
+# useradd -M -d /opt/steam -s /bin/bash steam
+# chown steam:steam /opt/steam /tmp/dumps
+# dpkg --add-architecture i386
+# apt update
+# apt install libc6:i386 lib32z1
+# sudo su - steam -s /bin/bash
+
+# #--------
+
+# wget http://media.steampowered.com/installer/steamcmd_linux.tar.gz
+# tar -xvzf steamcmd_linux.tar.gz
+
+# # fix: /opt/steam/.steam/sdk32/steamclient.so: cannot open shared object file: No such file or directory
+# mkdir /opt/steam/.steam && ln -s /opt/steam/linux32 /opt/steam/.steam/sdk32
+
+# # erst die windows deps zu installieren scheint ein workaround für x64 zu sein?
+# ./steamcmd.sh \
+#     +force_install_dir /opt/steam/left4dead2 \
+#     +login anonymous \
+#     +@sSteamCmdForcePlatformType windows \
+#     +app_update 222860 validate \
+#     +quit
+# ./steamcmd.sh \
+#     +force_install_dir /opt/steam/left4dead2 \
+#     +login anonymous \
+#     +@sSteamCmdForcePlatformType linux \
+#     +app_update 222860 validate \
+#     +quit
+
+# # download admin system
+# wget -4 https://git.sublimity.de/cronekorkn/steam-workshop-downloader/raw/branch/master/steam-workshop-download
+# chmod +x steam-workshop-download
+# ./steam-workshop-download 2524204971 --out /opt/steam/left4dead2/left4dead2/addons
+# mkdir -p "/opt/steam/left4dead2/left4dead2/ems/admin system"
+# echo "STEAM_1:0:12376499" > "/opt/steam/left4dead2/left4dead2/ems/admin system/admins.txt"
+
+# /opt/steam/left4dead2/srcds_run -game left4dead2 -ip 0.0.0.0 -port 27015 +map c1m1_hotel
+
+
+# cat <<'EOF' > /opt/steam/left4dead2/left4dead2/cfg/server.cfg
+# hostname "CKNs Server"
+# motd_enabled 0
+
+# sv_steamgroup "38347879"
+# #sv_steamgroup_exclusive 0
+
+# sv_minrate 60000
+# sv_maxrate 0
+# net_splitpacket_maxrate 60000
+
+# sv_hibernate_when_empty 0
+# EOF

bundles/left4dead2_old2/metadata.py

@@ -0,0 +1,107 @@
+from re import match
+
+defaults = {
+    'apt': {
+        'packages': {
+            'libc6_i386': {}, # installs libc6:i386
+            'lib32z1': {},
+            'unzip': {},
+        },
+    },
+    'left4dead2': {
+        'servers': {},
+    },
+    'nftables': {
+        'input': {
+            'udp dport { 27005, 27020 } accept',
+        },
+    },
+}
+
+
+@metadata_reactor.provides(
+    'nftables/input',
+)
+def nftables(metadata):
+    ports = sorted(str(config["port"]) for config in metadata.get('left4dead2/servers', {}).values())
+
+    return {
+        'nftables': {
+            'input': {
+                f'ip protocol {{ tcp, udp }} th dport {{ {", ".join(ports)} }} accept'
+            },
+        },
+    }
+
+
+@metadata_reactor.provides(
+    'systemd/units',
+)
+def initial_unit(metadata):
+    install_command = (
+        '/opt/steam/steamcmd.sh '
+        '+force_install_dir /opt/left4dead2 '
+        '+login anonymous '
+        '+@sSteamCmdForcePlatformType {platform} '
+        '+app_update 222860 validate '
+        '+quit '
+    )
+
+    return {
+        'systemd': {
+            'units': {
+                'left4dead2-install.service': {
+                    'Unit': {
+                        'Description': 'install or update left4dead2',
+                        'After': 'network-online.target',
+                    },
+                    'Service': {
+                        'Type': 'oneshot',
+                        'RemainAfterExit': 'yes',
+                        'User': 'steam',
+                        'Group': 'steam',
+                        'WorkingDirectory': '/opt/steam',
+                        'ExecStartPre': install_command.format(platform='windows'),
+                        'ExecStart': install_command.format(platform='linux'),
+                    },
+                    'Install': {
+                        'WantedBy': {'multi-user.target'},
+                    },
+                },
+            },
+        },
+    }
+
+
+@metadata_reactor.provides(
+    'systemd/units',
+)
+def server_units(metadata):
+    units = {}
+
+    for name, config in metadata.get('left4dead2/servers').items():
+        assert match(r'^[A-z0-9-_-]+$', name)
+
+        units[f'left4dead2-{name}.service'] = {
+            'Unit': {
+                'Description': f'left4dead2 server {name}',
+                'After': {'left4dead2-install.service'},
+                'Requires': {'left4dead2-install.service'},
+            },
+            'Service': {
+                'User': 'steam',
+                'Group': 'steam',
+                'WorkingDirectory': '/opt/left4dead2',
+                'ExecStart': f'/opt/left4dead2/srcds_run -port {config["port"]} +exec server_{name}.cfg',
+                'Restart': 'on-failure',
+            },
+            'Install': {
+                'WantedBy': {'multi-user.target'},
+            },
+        }
+
+    return {
+        'systemd': {
+            'units': units,
+        },
+    }

bundles/left4dead2_steam_old/items.py

@@ -0,0 +1,54 @@
+users = {
+    'steam': {
+        'home': '/opt/steam/steam',
+    },
+}
+
+directories = {
+    '/opt/steam': {
+        'owner': 'steam',
+        'group': 'steam',
+        'needs': [
+            'zfs_dataset:tank/steam',
+        ],
+    },
+    '/opt/steam/steam': {
+        'owner': 'steam',
+        'group': 'steam',
+    },
+}
+for game in node.metadata.get('steam/games'):
+    directories[f'/opt/steam/{game}'] = {
+        'owner': 'steam',
+        'group': 'steam',
+        'needed_by': [
+            'svc_systemd:steam-update.service',
+        ],
+    }
+
+files = {
+    '/opt/steam/steam/steamcmd_linux.tar.gz': {
+        'content_type': 'download',
+        'source': 'https://steamcdn-a.akamaihd.net/client/installer/steamcmd_linux.tar.gz',
+        'owner': 'steam',
+        'group': 'steam',
+    },
+}
+
+actions = {
+    'extract_steamcmd': {
+        'command': """su - steam -c 'tar xfvz /opt/steam/steam/steamcmd_linux.tar.gz --directory /opt/steam/steam'""",
+        'unless': 'test -f /opt/steam/steam/steamcmd.sh',
+        'needs': [
+            'file:/opt/steam/steam/steamcmd_linux.tar.gz',
+        ],
+    },
+}
+
+svc_systemd['steam-update.service'] = {
+    'running': None,
+    'enabled': True,
+    'needs': {
+        'file:/usr/local/lib/systemd/system/steam-update.service',
+    }
+}

bundles/left4dead2_steam_old/metadata.py

@@ -0,0 +1,52 @@
+defaults = {
+    'apt': {
+        'packages': {
+            'lib32gcc-s1': {},
+            'unzip': {},
+        },
+    },
+    'steam': {
+        'games': {
+            'left4dead2': 222860,
+        },
+    },
+    'zfs': {
+        'datasets': {
+            'tank/steam': {
+                'mountpoint': '/opt/steam',
+                'backup': False,
+            },
+        },
+    },
+}
+
+
+@metadata_reactor.provides(
+    'systemd/units',
+)
+def initial_unit(metadata):
+    return {
+        'systemd': {
+            'units': {
+                'steam-update.service': {
+                    'Unit': {
+                        'Description': 'steam: install and update games',
+                        'After': 'network-online.target',
+                    },
+                    'Service': {
+                        'Type': 'oneshot',
+                        'User': 'steam',
+                        'Group': 'steam',
+                        'WorkingDirectory': '/opt/steam',
+                        'ExecStart': {
+                            f'/opt/steam/steam/steamcmd.sh +force_install_dir /opt/steam/{game} +login anonymous +app_update {id} validate +quit'
+                                for game, id in metadata.get('steam/games').items()
+                        }
+                    },
+                    'Install': {
+                        'WantedBy': {'multi-user.target'},
+                    },
+                },
+            },
+        },
+    }

bundles/left4me/README.md

@@ -1,167 +0,0 @@
-# left4me
-
-L4D2 game-server management platform: a Flask web UI on gunicorn that
-provisions per-instance srcds servers via templated systemd units, with
-kernel-overlayfs layering for shared installations + per-overlay maps,
-and uid-based DSCP/priority marking on the egress path so CAKE on the
-external interface prioritizes srcds UDP over bulk traffic.
-
-## Metadata
-
-```python
-'metadata': {
-    'left4me': {
-        'domain': 'whatever.tld',  # required — the only per-node knob
-        # Everything below is optional and has a sensible default in the
-        # bundle. Override per-node only if the default is wrong:
-        # 'git_url': 'git@git.sublimity.de:cronekorkn/left4me',
-        # 'git_branch': 'master',
-        # 'gunicorn_workers': 1,
-        # 'gunicorn_threads': 32,
-        # 'job_worker_threads': 4,
-        # 'port_range_start': 27015,
-        # 'port_range_end': 27115,
-        # secret_key is auto-derived per node
-        # (repo.vault.random_bytes_as_base64_for f'{node.name} left4me secret_key').
-    },
-},
-```
-
-The bundle's `derived_from_domain` reactor reads `left4me/domain` and
-emits the corresponding `nginx/vhosts`, `letsencrypt/domains`,
-`monitoring/services/left4me-web` (HTTPS health check), and the game-
-port `nftables/input` accept rules. Backup paths
-(`/var/lib/left4me`, `/etc/left4me`) are set-merged into `backup/paths`
-from defaults. None of these need to be declared per-node.
-
-## What this bundle does
-
-The bundle delivers to `ovh.left4me` a mix of:
-
-### Target-side symlinks into the left4me checkout
-
-After `git_deploy:/opt/left4me/src` (root-owned — left4me cannot rewrite
-its own deployment artifacts at runtime), ckn-bw creates symlinks from
-canonical on-host paths into the checkout:
-
-| On-host path | Source in checkout |
-|---|---|
-| `/etc/sudoers.d/left4me` | `deploy/files/etc/sudoers.d/left4me` |
-| `/etc/sysctl.d/99-left4me.conf` | `deploy/files/etc/sysctl.d/99-left4me.conf` |
-| `/etc/systemd/system/left4me-web.service.d/10-hardening.conf` | `deploy/files/etc/systemd/system/left4me-web.service.d/10-hardening.conf` |
-| `/etc/systemd/system/left4me-server@.service.d/10-hardening.conf` | `deploy/files/etc/systemd/system/left4me-server@.service.d/10-hardening.conf` |
-| `/usr/local/libexec/left4me/{left4me-overlay,left4me-systemctl,left4me-journalctl,left4me-script-sandbox}` | `deploy/scripts/libexec/*` |
-| `/usr/local/sbin/left4me` | `deploy/scripts/sbin/left4me` |
-
-The hardening drop-ins and sudoers are the application's own security
-knowledge — they live in the left4me repo and are version-controlled there.
-The privileged helpers are also application code. The symlink pattern
-lets bw manage placement without duplicating content.
-
-Design rationale:
-`left4me/docs/superpowers/specs/2026-05-15-deployment-responsibility-design.md`.
-
-### Reactor-emitted units (per-host shape)
-
-Via `systemd/units` metadata in `metadata.py` (consumed by `bundles/systemd/`):
-
-- `left4me-web.service` — gunicorn on `127.0.0.1:8000`; worker/thread
-  counts from `web.env.mako`. TLS terminates upstream.
-- `left4me-server@.service` — per-instance srcds template; `SocketBindAllow=`
-  ranges from metadata.
-- `l4d2-game.slice` / `l4d2-build.slice` — cgroup slices with per-host
-  `AllowedCPUs=` from `left4me/system_cpus`.
-- `system.slice.d/99-left4me-cpuset.conf` + `user.slice.d/99-left4me-cpuset.conf`
-  — host CPU-set drop-ins, same source.
-
-### bw `files{}` — templated env files
-
-- `host.env.mako` → `/etc/left4me/host.env`
-- `web.env.mako` → `/etc/left4me/web.env`
-- `sandbox-resolv.conf` → `/etc/left4me/sandbox-resolv.conf`
-
-### Action chains — deploy lifecycle
-
-- `git_deploy` → `uv_sync` (`uv sync --frozen` against the workspace's
-  committed `uv.lock`; hatchling PEP 660 editable, doesn't touch source)
-  → `alembic_upgrade` → `seed_overlays` + web restart.
-- One-shot bootstrap: `install_uv` downloads a pinned `uv` binary
-  (SHA256-verified) into `/usr/local/bin` because `uv` isn't in Trixie's
-  apt archive. `unless`-gated, so it's a no-op once the version pin is
-  installed; re-runs only when the constant is bumped.
-- Idempotent gates: `chmod-sudoers` (0440 root:root), `chmod-scripts` (0755 root:root).
-- Post-git-deploy reloads: `systemctl daemon-reload`, `sysctl --system`.
-- Post-apply self-test: `verify-hardening-dropins` (asserts the drop-ins are
-  loaded by the live units before declaring apply done).
-
-### System user
-
-`left4me` (uid/gid 980, home `/var/lib/left4me`, mode 0755) — the same uid
-hosts the web app, gameservers, and the script-overlay sandbox unit (which
-drops privileges via systemd-run with a fully hardened transient service).
-Runtime mutable state lives under `/var/lib/left4me/`; `/opt/left4me/`
-stays as a root-owned deploy-artifact root.
-
-### nftables / nginx / monitoring
-
-- Contributes uid-based DSCP/priority marks for srcds UDP egress to
-  `nftables/output` (via `defaults`).
-- `derived_from_domain` reactor emits the corresponding `nginx/vhosts`,
-  `letsencrypt/domains`, and `monitoring/services/left4me-web` (HTTPS
-  health check).
-
-## Gotchas
-
-- **Requires `bundles/nftables` and `bundles/systemd` on the node.** The
-  bundle asserts membership at `bw test` time. On Debian-13 these ride
-  in via the `debian-13` group, so attaching the bundle to a Debian-13
-  node is enough.
-- **`left4me-web.service` does not have `NoNewPrivileges=true`.** This is
-  intentional — workers `sudo` the privileged helpers; `NoNewPrivileges`
-  would block setuid escalation. Per-instance `server@.service` units
-  *do* have it.
-- **CAKE shaping is configured separately**, via
-  `network/<iface>/cake` on the node (consumed by `bundles/network/`),
-  not by this bundle.
-- **First-run admin user is manual.** After `bw apply`, ssh to the host and
-  bootstrap the admin via the `left4me` wrapper (it sources the env files,
-  drops to the `left4me` user, and runs the flask CLI):
-  `sudo left4me create-user <username> --admin` (prompts for password via
-  the flask CLI, or set `LEFT4ME_ADMIN_PASSWORD` first). The bundle
-  deliberately doesn't seed an admin to keep credentials out of the
-  metadata pipeline. The same `left4me` wrapper accepts any other flask
-  subcommand: `sudo left4me seed-script-overlays <dir>`,
-  `sudo left4me routes`, `sudo left4me shell`, etc.
-- **CPU isolation is managed by this bundle**, driven by one required
-  per-node knob: `left4me/system_cpus` — a set of int CPU ids that
-  pins `system.slice` / `user.slice` / `l4d2-build.slice`. The
-  complement (`set(range(vm/threads)) - system_cpus`) pins
-  `l4d2-game.slice`. On HT hosts, list both SMT siblings of every
-  physical core you want to reserve for system, otherwise games end
-  up sharing L1/L2 with system. Find pairings via
-  `/sys/devices/system/cpu/cpu<n>/topology/thread_siblings_list`. On
-  the prod node (`ovh.left4me`, 4 physical / 8 threads, pairings
-  (0,4) (1,5) (2,6) (3,7)) the node sets `'system_cpus': {0, 4}` to
-  reserve physical core 0 entirely. `l4d2-game.slice` and
-  `l4d2-build.slice` carry `AllowedCPUs=` inline on their unit
-  definitions; `system.slice` and `user.slice` get drop-ins registered
-  under `systemd/units` with the `'<parent>.d/<basename>.conf'` key
-  convention (same shape nginx and autologin use), landing at
-  `/usr/local/lib/systemd/system/<slice>.d/99-left4me-cpuset.conf`.
-  The reactor raises if `system_cpus` includes CPUs outside
-  `[0, vm/threads)` or leaves no cores for games.
-- **Kernel feature requirement:** kernel-overlayfs (`CONFIG_OVERLAY_FS`).
-  Standard on debian-13.
-- **Game ports** open by the web app on demand in the range 27015-27115
-  (UDP+TCP). Add corresponding accept rules to `nftables/input` per
-  node if the host's policy is default-drop on input.
-- **Pinned UIDs/GIDs (980/981).** Chosen for deterministic ownership
-  across rebuilds and backup restores. If you add another bundle that
-  pins UIDs in this repo, make sure it doesn't collide.
-
-## Slice support requires `bundles/systemd` ≥ commit cc1c6a5
-
-This bundle's `l4d2-game.slice` and `l4d2-build.slice` units rely on
-`bundles/systemd/items.py` accepting the `.slice` extension. Older
-revisions raised `Exception(f'unknown type slice')` at apply time.
-The repo-wide `bw test` will catch this if it regresses.

bundles/left4me/files/etc/left4me/host.env.mako

@@ -1,6 +0,0 @@
-# Managed by ckn-bw bundles/left4me. Local edits will be reverted.
-# Deployment units use fixed /var/lib/left4me paths; regenerate units if this changes.
-LEFT4ME_ROOT=/var/lib/left4me
-# l4d2host invokes steamcmd by absolute path — bypasses PATH lookup so the
-# script's `cd "$(dirname "$0")"` resolves next to the real install dir.
-LEFT4ME_STEAMCMD=/var/lib/left4me/steam/steamcmd.sh

bundles/left4me/files/etc/left4me/sandbox-resolv.conf

@@ -1,6 +0,0 @@
-# Sandbox-only resolver config — bind-mounted into script-overlay sandboxes
-# at /etc/resolv.conf. The host's resolver (often a private/LAN DNS server)
-# is unreachable from inside the sandbox because IPAddressDeny= blocks
-# egress to RFC1918 / loopback. Public resolvers keep DNS working.
-nameserver 1.1.1.1
-nameserver 8.8.8.8

bundles/left4me/files/etc/left4me/web.env.mako

@@ -1,14 +0,0 @@
-# Managed by ckn-bw bundles/left4me. Local edits will be reverted.
-DATABASE_URL=sqlite:////var/lib/left4me/left4me.db
-SECRET_KEY=${node.metadata.get('left4me/secret_key')}
-JOB_WORKER_THREADS=${node.metadata.get('left4me/job_worker_threads')}
-SESSION_COOKIE_SECURE=true
-LEFT4ME_PORT_RANGE_START=${node.metadata.get('left4me/port_range_start')}
-LEFT4ME_PORT_RANGE_END=${node.metadata.get('left4me/port_range_end')}
-STEAM_WEB_API_KEY=${node.metadata.get('left4me/steam_web_api_key')}
-# Log listener destination — MUST be non-loopback because Source silently
-# drops logaddress destinations in 127.0.0.0/8. Derived from the node's
-# external IPv4; kernel routes same-host traffic via lo internally but the
-# destination IP in the packet header must not literally be 127.x.
-LOG_LISTENER_ADDR=${node.metadata.get('network/external/ipv4').split('/')[0]}:28000
-LOG_LISTENER_BIND=0.0.0.0:28000

bundles/left4me/items.py

@@ -1,439 +0,0 @@
-# Items for the left4me bundle.
-# Systemd units come from metadata via bundles/systemd/ — there are no
-# .service or .slice files in this bundle's files/ tree. Cpuset drop-ins
-# for system.slice / user.slice are likewise emitted via systemd/units
-# in metadata.py (key: '<parent>.d/<basename>.conf').
-
-directories = {
-    '/opt/left4me': {
-        # Deploy-artifact root. Only /opt/left4me/src lives here; runtime
-        # state (.venv, steamcmd) lives under /var/lib/left4me/. Root-owned
-        # so left4me cannot drop new files alongside src/ (e.g. an attacker
-        # with web-compromise can't plant a 'scripts.d/' loaded by future
-        # deploy logic).
-        'owner': 'root',
-        'group': 'root',
-        'mode': '0755',
-    },
-    '/opt/left4me/src': {
-        # Source checkout. Root-owned because the production install model
-        # is non-editable: pip_install copies the source to a left4me-owned
-        # tempdir before building, so the source tree on disk is never
-        # mutated at runtime and left4me only needs read access (which
-        # world-readable bits provide). Keeps left4me from being able to
-        # rewrite its own future hardening drop-ins / unit files under
-        # /opt/left4me/src/deploy/ (target-side symlink model in the
-        # deployment-responsibility reshape).
-        'owner': 'root',
-        'group': 'root',
-    },
-    '/etc/left4me': {
-        'owner': 'root',
-        'group': 'root',
-        'mode': '0755',
-    },
-    '/var/lib/left4me': {
-        # left4me's home dir — useradd creates with 0700; loosen to 0755 so
-        # the systemd-imposed FS view for transient script-sandbox units
-        # (running as left4me with TemporaryFileSystem=/var/lib + selective
-        # binds) can traverse on its way to the overlay bind targets.
-        'owner': 'left4me',
-        'group': 'left4me',
-        'mode': '0755',
-    },
-    '/var/lib/left4me/installation':   {'owner': 'left4me', 'group': 'left4me'},
-    '/var/lib/left4me/overlays':       {'owner': 'left4me', 'group': 'left4me'},
-    '/var/lib/left4me/instances':      {'owner': 'left4me', 'group': 'left4me'},
-    '/var/lib/left4me/runtime':        {'owner': 'left4me', 'group': 'left4me'},
-    '/var/lib/left4me/workshop_cache': {'owner': 'left4me', 'group': 'left4me'},
-    '/var/lib/left4me/tmp':            {'owner': 'left4me', 'group': 'left4me'},
-    '/var/lib/left4me/steam':          {'owner': 'left4me', 'group': 'left4me'},
-    # Note: the venv (/var/lib/left4me/.venv) is created by the
-    # left4me_create_venv action; declaring it here too would race with
-    # `python -m venv` which expects to create the directory itself.
-    '/usr/local/libexec/left4me': {
-        'owner': 'root',
-        'group': 'root',
-        'mode': '0755',
-    },
-    '/etc/systemd/system/left4me-web.service.d': {
-        'owner': 'root', 'group': 'root', 'mode': '0755',
-    },
-    '/etc/systemd/system/left4me-server@.service.d': {
-        'owner': 'root', 'group': 'root', 'mode': '0755',
-    },
-}
-
-groups = {
-    'left4me': {'gid': 980},
-}
-
-users = {
-    'left4me': {
-        'uid': 980,
-        'gid': 980,
-        'home': '/var/lib/left4me',
-        'shell': '/usr/sbin/nologin',
-    },
-}
-# UID/GID pinned in the system-package range (100-999, per Debian
-# policy) so file ownership is deterministic across rebuilds and
-# backup restores. 980 is unused elsewhere in this repo.
-# (981 — formerly l4d2-sandbox — was collapsed into 980 on 2026-05-15;
-# see left4me/docs/superpowers/plans/2026-05-15-uid-collapse.md.)
-
-# Privileged helpers are delivered via target-side symlinks (see the
-# `symlinks` dict below) pointing into the left4me checkout at
-# `/opt/left4me/src/deploy/scripts/{libexec,sbin}/`. No verbatim copy
-# in this bundle's files/ tree. Sudoers (further below) lists the
-# specific paths that left4me may invoke as root NOPASSWD.
-
-files = {
-    '/etc/left4me/sandbox-resolv.conf': {
-        'source': 'etc/left4me/sandbox-resolv.conf',
-        'mode': '0644',
-        'owner': 'root',
-        'group': 'root',
-    },
-    '/etc/left4me/host.env': {
-        'source': 'etc/left4me/host.env.mako',
-        'content_type': 'mako',
-        'mode': '0640',
-        'owner': 'root',
-        # group=left4me so the alembic + seed-overlays actions (which run as
-        # `sudo -u left4me sh -c '. /etc/left4me/host.env'`) can source it.
-        # Same pattern as web.env below.
-        'group': 'left4me',
-        'needs': [
-            'group:left4me',
-        ],
-    },
-    '/etc/left4me/web.env': {
-        'source': 'etc/left4me/web.env.mako',
-        'content_type': 'mako',
-        'mode': '0640',
-        'owner': 'root',
-        'group': 'left4me',
-        'needs': [
-            'group:left4me',
-        ],
-    },
-}
-
-symlinks = {
-    '/etc/sysctl.d/99-left4me.conf': {
-        'target': '/opt/left4me/src/deploy/files/etc/sysctl.d/99-left4me.conf',
-        'owner': 'root',
-        'group': 'root',
-        'needs': [
-            'git_deploy:/opt/left4me/src',
-        ],
-        'triggers': [
-            'action:left4me_sysctl_reload',
-        ],
-    },
-    '/etc/systemd/system/left4me-web.service.d/10-hardening.conf': {
-        'target': '/opt/left4me/src/deploy/files/etc/systemd/system/left4me-web.service.d/10-hardening.conf',
-        'owner': 'root', 'group': 'root',
-        'needs': [
-            'directory:/etc/systemd/system/left4me-web.service.d',
-            'git_deploy:/opt/left4me/src',
-        ],
-        'triggers': [
-            'action:left4me_daemon_reload',
-        ],
-    },
-    '/etc/systemd/system/left4me-server@.service.d/10-hardening.conf': {
-        'target': '/opt/left4me/src/deploy/files/etc/systemd/system/left4me-server@.service.d/10-hardening.conf',
-        'owner': 'root', 'group': 'root',
-        'needs': [
-            'directory:/etc/systemd/system/left4me-server@.service.d',
-            'git_deploy:/opt/left4me/src',
-        ],
-        'triggers': [
-            'action:left4me_daemon_reload',
-        ],
-    },
-    '/etc/sudoers.d/left4me': {
-        'target': '/opt/left4me/src/deploy/files/etc/sudoers.d/left4me',
-        'owner': 'root', 'group': 'root',
-        'needs': [
-            'action:left4me_chmod_sudoers',
-            'git_deploy:/opt/left4me/src',
-        ],
-        # sudo follows symlinks; with the target file at root:root 0440
-        # in a root-owned source tree, sudo accepts it. No daemon-reload
-        # equivalent — sudo re-reads /etc/sudoers.d/ on each invocation.
-    },
-}
-
-# Helper script source paths (in left4me's checkout) → deployed-form paths.
-# Each gets a symlink item merged into the symlinks dict above.
-_LEFT4ME_LIBEXEC_SCRIPTS = (
-    'left4me-overlay',
-    'left4me-systemctl',
-    'left4me-journalctl',
-    'left4me-script-sandbox',
-)
-_LEFT4ME_SBIN_SCRIPTS = (
-    'left4me',
-)
-
-for _script in _LEFT4ME_LIBEXEC_SCRIPTS:
-    symlinks[f'/usr/local/libexec/left4me/{_script}'] = {
-        'target': f'/opt/left4me/src/deploy/scripts/libexec/{_script}',
-        'owner': 'root', 'group': 'root',
-        'needs': [
-            'directory:/usr/local/libexec/left4me',
-            'action:left4me_chmod_scripts',
-            'git_deploy:/opt/left4me/src',
-        ],
-    }
-
-for _script in _LEFT4ME_SBIN_SCRIPTS:
-    symlinks[f'/usr/local/sbin/{_script}'] = {
-        'target': f'/opt/left4me/src/deploy/scripts/sbin/{_script}',
-        'owner': 'root', 'group': 'root',
-        'needs': [
-            'action:left4me_chmod_scripts',
-            'git_deploy:/opt/left4me/src',
-        ],
-    }
-
-actions = {
-    'left4me_sysctl_reload': {
-        'command': 'sysctl --system >/dev/null',
-        'triggered': True,
-    },
-    'left4me_daemon_reload': {
-        'command': 'systemctl daemon-reload',
-        'triggered': True,
-        'cascade_skip': False,
-    },
-    'left4me_verify_hardening_dropins_loaded': {
-        # Post-apply self-test: confirm systemd actually picked up the
-        # hardening drop-ins we shipped via symlink. Catches the failure
-        # mode where the symlink lands but daemon-reload didn't take or
-        # someone manually unlinked the drop-in. For the gameserver template
-        # we query an imaginary instance — systemd resolves drop-in paths
-        # for `name@instance.service` against the template (`name@.service.d/`),
-        # so the instance need not exist or ever have run.
-        'command': (
-            'systemctl show left4me-server@verify.service -p DropInPaths --value '
-            '| tr " " "\\n" '
-            '| grep -qx /etc/systemd/system/left4me-server@.service.d/10-hardening.conf '
-            '&& '
-            'systemctl show left4me-web.service -p DropInPaths --value '
-            '| tr " " "\\n" '
-            '| grep -qx /etc/systemd/system/left4me-web.service.d/10-hardening.conf'
-        ),
-        'cascade_skip': False,
-        'needs': [
-            'action:left4me_daemon_reload',
-            'symlink:/etc/systemd/system/left4me-web.service.d/10-hardening.conf',
-            'symlink:/etc/systemd/system/left4me-server@.service.d/10-hardening.conf',
-        ],
-    },
-    'left4me_chmod_sudoers': {
-        # sudo refuses sudoers.d entries that aren't 0440 (or 0400) root:root.
-        # git_deploy extracts as root with the in-repo file mode; this action
-        # is belt-and-braces in case the repo mode drifts. Idempotent via
-        # the `unless` gate.
-        'command': 'chmod 0440 /opt/left4me/src/deploy/files/etc/sudoers.d/left4me',
-        'unless': 'test "$(stat -c %a /opt/left4me/src/deploy/files/etc/sudoers.d/left4me)" = "440"',
-        'cascade_skip': False,
-        'needs': [
-            'git_deploy:/opt/left4me/src',
-        ],
-    },
-    'left4me_dpkg_add_i386_arch': {
-        # steamcmd is 32-bit and pulls libc6:i386 + lib32z1 from the i386 arch.
-        # apt-get update is part of this action because newly-added foreign
-        # archs need a fresh package list before any :i386 package resolves.
-        'command': 'dpkg --add-architecture i386 && apt-get update',
-        'unless': 'dpkg --print-foreign-architectures | grep -qx i386',
-        'cascade_skip': False,
-    },
-    'left4me_install_steamcmd': {
-        # Steam's tarball is rolling with no published checksum, so we can't
-        # use download: (which requires a hash). Guard with a presence check
-        # on steamcmd.sh — steamcmd self-updates at runtime, so chasing the
-        # tarball version from bw isn't useful.
-        'command': (
-            'sudo -u left4me sh -c "'
-            'cd /var/lib/left4me/steam && '
-            'curl -fsSL https://media.steampowered.com/installer/steamcmd_linux.tar.gz | '
-            'tar -xz'
-            '"'
-        ),
-        'unless': 'test -x /var/lib/left4me/steam/steamcmd.sh',
-        'cascade_skip': False,
-        'needs': [
-            'directory:/var/lib/left4me/steam',
-            'pkg_apt:curl',
-            'pkg_apt:libc6_i386',  # bw pkg_apt convention: _ → :
-            'pkg_apt:lib32z1',
-            'user:left4me',
-        ],
-    },
-}
-
-# steamcmd is invoked by absolute path (LEFT4ME_STEAMCMD in host.env),
-# not via PATH lookup — see l4d2host/cli.py:install. We don't need to put
-# anything in /usr/local/bin for it.
-
-git_deploy = {
-    '/opt/left4me/src': {
-        'repo': node.metadata.get('left4me/git_url'),
-        'rev': node.metadata.get('left4me/git_branch'),
-        'triggers': [
-            # Re-sync the workspace whenever the checkout changes. uv reads
-            # the committed uv.lock at /opt/left4me/src and installs both
-            # workspace members (l4d2host, l4d2web) editable into
-            # /var/lib/left4me/.venv. Hatchling's PEP 660 editable install
-            # doesn't write to the source tree, so /opt/left4me/src stays
-            # root-owned and untouched. uv_sync cascades into
-            # alembic_upgrade → seed_overlays → web restart.
-            'action:left4me_uv_sync',
-            # alembic upgrade head is idempotent — keeping it as a direct
-            # trigger off git_deploy is belt-and-braces in case the
-            # uv_sync cascade is ever short-circuited.
-            'action:left4me_alembic_upgrade',
-            # Reload systemd unit definitions whenever the checkout changes;
-            # handles updates to hardening drop-in content without requiring
-            # a symlink change.
-            'action:left4me_daemon_reload',
-        ],
-    },
-}
-
-actions['left4me_chmod_scripts'] = {
-    # sudo invokes the helpers by absolute path under /usr/local/...;
-    # those resolve to the checkout via the symlinks above. The target
-    # files must be executable (mode 0755). git_deploy extracts with
-    # the in-repo file modes; this action is belt-and-braces in case
-    # any helper's repo mode regresses to 0644.
-    'command': (
-        'chmod 0755 '
-        '/opt/left4me/src/deploy/scripts/libexec/* '
-        '/opt/left4me/src/deploy/scripts/sbin/*'
-    ),
-    'unless': (
-        '! find /opt/left4me/src/deploy/scripts -type f \\! -perm 755 -print -quit 2>/dev/null | grep -q .'
-    ),
-    'cascade_skip': False,
-    'needs': [
-        'git_deploy:/opt/left4me/src',
-    ],
-}
-
-actions['left4me_install_uv'] = {
-    # uv is not in Debian Trixie's apt archive (only experimental/sid).
-    # Pin to a specific release; download the tarball + its SHA256
-    # sibling from astral-sh/uv releases, verify, install to
-    # /usr/local/bin. Idempotent via `unless` — only re-runs when the
-    # pinned version changes (bump the constant in two places below).
-    # Pattern matches left4me_install_steamcmd (curl+tar) elsewhere in
-    # this bundle. Bump cadence: as needed; both dev (brew uv) and
-    # prod should track the same minor.
-    'command': """set -e
-tmpdir=$(mktemp -d); trap "rm -rf $tmpdir" EXIT
-base=https://github.com/astral-sh/uv/releases/download/0.11.8
-tar=uv-x86_64-unknown-linux-gnu.tar.gz
-curl -fsSL -o $tmpdir/$tar        $base/$tar
-curl -fsSL -o $tmpdir/$tar.sha256 $base/$tar.sha256
-(cd $tmpdir && sha256sum -c $tar.sha256)
-tar -xzf $tmpdir/$tar -C $tmpdir --strip-components=1
-install -m 0755 $tmpdir/uv  /usr/local/bin/uv
-install -m 0755 $tmpdir/uvx /usr/local/bin/uvx
-""",
-    'unless': '/usr/local/bin/uv --version 2>/dev/null | grep -qx "uv 0.11.8"',
-    'cascade_skip': False,
-    'needs': [
-        'pkg_apt:curl',
-    ],
-    # No triggers — install_uv is a one-shot bootstrap. uv_sync needs
-    # it (via `needs`), so the dependency runs install_uv first on a
-    # clean host. After that, this action is a no-op on every apply
-    # unless the version pin changes.
-}
-
-actions['left4me_uv_sync'] = {
-    # The whole "install/refresh the workspace" deploy step, in one
-    # action. uv reads /opt/left4me/src/uv.lock + the workspace's
-    # pyproject.toml and installs both members (l4d2host, l4d2web)
-    # editable into /var/lib/left4me/.venv. Hatchling's PEP 660
-    # editable install drops a .pth pointing at the source tree — no
-    # writes to source, so the root-owned /opt/left4me/src stays clean.
-    #
-    # UV_PROJECT_ENVIRONMENT redirects uv's default venv path
-    # (<project>/.venv) to our writable runtime location. HOME is set
-    # explicitly so uv's cache lands in /var/lib/left4me/.cache/uv
-    # instead of the inherited sudo HOME (which can be unwritable for
-    # the left4me user). cd /var/lib/left4me ensures uv's project-config
-    # walk-up doesn't trip over an unreadable parent (e.g., /root or
-    # /home/ckn). --frozen requires uv.lock to be present and
-    # consistent with pyproject.toml — refuses to silently update the
-    # lockfile during deploy.
-    'command': (
-        'sudo -u left4me sh -c "'
-        'cd /var/lib/left4me && '
-        'env HOME=/var/lib/left4me '
-        'UV_PROJECT_ENVIRONMENT=/var/lib/left4me/.venv '
-        '/usr/local/bin/uv sync --frozen --project /opt/left4me/src'
-        '"'
-    ),
-    'triggered': True,
-    'cascade_skip': False,
-    'needs': [
-        'git_deploy:/opt/left4me/src',
-        'action:left4me_install_uv',
-        'directory:/var/lib/left4me',
-        'user:left4me',
-    ],
-    'triggers': [
-        'action:left4me_alembic_upgrade',
-    ],
-}
-
-actions['left4me_alembic_upgrade'] = {
-    # Mirrors deploy-test-server.sh:239-242. Runs as left4me with both env
-    # files sourced; JOB_WORKER_ENABLED=false so a stray worker doesn't race
-    # with the migration.
-    'command': (
-        'sudo -u left4me sh -c "'
-        'cd /opt/left4me/src/l4d2web && '
-        'set -a && . /etc/left4me/host.env && . /etc/left4me/web.env && set +a && '
-        'env JOB_WORKER_ENABLED=false '
-        '/var/lib/left4me/.venv/bin/alembic -c /opt/left4me/src/l4d2web/alembic.ini upgrade head'
-        '"'
-    ),
-    'triggered': True,
-    'cascade_skip': False,
-    'needs': [
-        'action:left4me_uv_sync',
-        'file:/etc/left4me/host.env',
-        'file:/etc/left4me/web.env',
-    ],
-    'triggers': [
-        'action:left4me_seed_overlays',
-        'svc_systemd:left4me-web.service:restart',
-    ],
-}
-
-actions['left4me_seed_overlays'] = {
-    # Idempotent: refreshes script bodies in place; existing overlay rows keep their ids.
-    'command': (
-        'sudo -u left4me sh -c "'
-        'set -a && . /etc/left4me/host.env && . /etc/left4me/web.env && set +a && '
-        'env JOB_WORKER_ENABLED=false '
-        '/var/lib/left4me/.venv/bin/flask --app l4d2web.app:create_app '
-        'seed-script-overlays /opt/left4me/src/examples/script-overlays'
-        '"'
-    ),
-    'triggered': True,
-    'cascade_skip': False,
-    'needs': [
-        'action:left4me_alembic_upgrade',
-    ],
-}

bundles/left4me/metadata.py

@@ -1,305 +0,0 @@
-assert node.has_bundle('nftables')
-assert node.has_bundle('systemd')
-assert node.has_bundle('systemd-timers')
-
-
-defaults = {
-    'left4me': {
-        # Application-wide defaults; node only overrides if it really needs to.
-        'git_url': 'https://git.sublimity.de/cronekorkn/left4me.git',
-        'git_branch': 'master',
-        'secret_key': repo.vault.random_bytes_as_base64_for(f'{node.name} left4me secret_key', length=32).value,
-        'gunicorn_workers': 1,
-        'gunicorn_threads': 32,
-        'job_worker_threads': 4,
-        # Steam Web API key for the live-state panel's GetPlayerSummaries
-        # lookups (persona names + avatars). Empty default — nodes override
-        # in their own metadata with the actual key. If left empty in prod,
-        # the live-state panel still works but falls back to RCON in-game
-        # names and placeholder avatars.
-        'steam_web_api_key': '',
-        # Whole 27000-block: covers Steam's defaults (27015 game, 27005
-        # client/RCON) plus headroom for ad-hoc ports without further
-        # nftables changes. Mirrored into LEFT4ME_PORT_RANGE_{START,END}
-        # by web.env.mako and into the nftables input rule by the
-        # nftables_input reactor below.
-        'port_range_start': 27000,
-        'port_range_end': 27999,
-    },
-    'apt': {
-        'packages': {
-            'p7zip-full': {},
-            'nftables': {},
-            'iproute2': {},
-            'curl': {},
-            'ca-certificates': {},
-            'python3': {},
-            'python3-dev': {},
-            # steamcmd is a 32-bit ELF; needs i386 multiarch + these libs.
-            # `_` → `:` is bundlewrap's pkg_apt convention for multiarch
-            # names (see pkg_apt.py:48).
-            'libc6_i386': {  # installs libc6:i386
-                'needs': ['action:left4me_dpkg_add_i386_arch'],
-            },
-            'lib32z1': {
-                'needs': ['action:left4me_dpkg_add_i386_arch'],
-            },
-        },
-    },
-    'nftables': {
-        # Match deploy/files/usr/local/lib/left4me/nft/left4me-mark.nft.
-        # Mark srcds UDP egress (uid left4me) with DSCP EF + skb priority 6
-        # so CAKE classifies it into the priority tin.
-        'output': {
-            'meta skuid "left4me" meta l4proto udp ip dscp set ef meta priority set 0006:0000',
-            'meta skuid "left4me" meta l4proto udp ip6 dscp set ef meta priority set 0006:0000',
-        },
-    },
-    'systemd': {
-        'services': {
-            'left4me-web.service': {
-                'enabled': True,
-                'running': True,
-                'needs': [
-                    'action:left4me_alembic_upgrade',
-                    'file:/etc/left4me/host.env',
-                    'file:/etc/left4me/web.env',
-                ],
-            },
-            # Note: left4me-server@.service is a TEMPLATE — instances are
-            # started on-demand by the web app via the left4me-systemctl
-            # helper. Don't enable/start it from here.
-            # The slices are installed (file present) but don't need
-            # enable/start — they're activated implicitly when a unit
-            # uses Slice=.
-        },
-    },
-    'backup': {
-        # Application-owned paths. Set-merged with backup group / node-level paths.
-        'paths': {
-            '/var/lib/left4me',
-            '/etc/left4me',
-        },
-    },
-    'systemd-timers': {
-        # Daily re-fetch of Steam Workshop metadata + .vpk downloads for any
-        # item whose author published an update. The CLI just inserts a
-        # `refresh_workshop_items` job; the web worker picks it up next.
-        # Idempotent — a re-fire while a refresh is already queued/running
-        # is a no-op (see l4d2web/cli.py:workshop_refresh).
-        'left4me-workshop-refresh': {
-            'command': '/var/lib/left4me/.venv/bin/flask --app l4d2web.app:create_app workshop-refresh',
-            'when': '*-*-* 04:00:00',
-            'persistent': True,
-            'user': 'left4me',
-            'working_dir': '/opt/left4me/src',
-            'environment_files': (
-                '/etc/left4me/host.env',
-                '/etc/left4me/web.env',
-            ),
-            'after': {
-                'network-online.target',
-                'left4me-web.service',
-            },
-        },
-    },
-}
-
-
-
-@metadata_reactor.provides(
-    'nginx/vhosts',
-)
-def nginx_vhosts(metadata):
-    # letsencrypt/domains and monitoring/services for the vhost are auto-
-    # populated by bundles/nginx/metadata.py. We just declare check_path:
-    # '/health' so the auto-check hits the Flask health endpoint, not '/'.
-    domain = metadata.get('left4me/domain')
-    return {
-        'nginx': {
-            'vhosts': {
-                domain: {
-                    'content': 'nginx/proxy_pass.conf',
-                    'context': {
-                        'target': 'http://127.0.0.1:8000',
-                    },
-                    'check_path': '/health',
-                },
-            },
-        },
-    }
-
-
-@metadata_reactor.provides(
-    'nftables/input',
-)
-def nftables_input(metadata):
-    port_start = metadata.get('left4me/port_range_start')
-    port_end = metadata.get('left4me/port_range_end')
-    return {
-        'nftables': {
-            'input': {
-                # Players connect via UDP. TCP on the same port range is RCON
-                # — only the local web app should reach it. Loopback bypasses
-                # the input chain (iifname lo accept in bundles/nftables/...),
-                # so 127.0.0.1 RCON works without an explicit TCP accept here.
-                # External TCP on these ports stays blocked by the default
-                # input-chain drop.
-                f'udp dport {port_start}-{port_end} accept',
-            },
-        },
-    }
-
-
-@metadata_reactor.provides(
-    'systemd/units',
-)
-def systemd_units(metadata):
-    workers = metadata.get('left4me/gunicorn_workers')
-    threads = metadata.get('left4me/gunicorn_threads')
-
-    # cgroup-v2 cpuset. `system_cpus` (set of int CPU ids, declared per
-    # node) pins system/user/build; the complement pins l4d2-game. On HT
-    # hosts, list both siblings of a physical core so games don't share
-    # L1/L2 with system work — pairings via
-    # /sys/devices/system/cpu/cpu<n>/topology/thread_siblings_list.
-    vm_threads = metadata.get('vm/threads', metadata.get('vm/cores'))
-    all_cpus = set(range(vm_threads))
-    system_cpus = metadata.get('left4me/system_cpus')
-    if not system_cpus <= all_cpus:
-        raise Exception(
-            f'left4me/system_cpus={sorted(system_cpus)} on {vm_threads}-thread host '
-            f'includes CPUs outside [0, {vm_threads})'
-        )
-    game_cpus = all_cpus - system_cpus
-    if not game_cpus:
-        raise Exception(
-            f'left4me/system_cpus={sorted(system_cpus)} on {vm_threads}-thread host '
-            f'leaves no cores for games'
-        )
-    system_cpus_string = ','.join(str(t) for t in sorted(system_cpus))
-    game_cpus_string = ','.join(str(t) for t in sorted(game_cpus))
-
-    # Drop-in for upstream system.slice / user.slice (units we don't own).
-    # Same '<parent>.d/<basename>.conf' convention as nginx and autologin.
-    cpuset_dropin = {'Slice': {'AllowedCPUs': system_cpus_string}}
-
-    return {
-        'systemd': {
-            'units': {
-                'left4me-web.service': {
-                    'Unit': {
-                        'Description': 'left4me web application',
-                        'After': 'network-online.target',
-                        'Wants': 'network-online.target',
-                    },
-                    'Service': {
-                        'Type': 'simple',
-                        'User': 'left4me',
-                        'Group': 'left4me',
-                        'WorkingDirectory': '/opt/left4me/src',
-                        'Environment': {
-                            'HOME=/var/lib/left4me',
-                            'PATH=/var/lib/left4me/.venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin',
-                        },
-                        'EnvironmentFile': (
-                            '/etc/left4me/host.env',
-                            '/etc/left4me/web.env',
-                        ),
-                        'ExecStart': (
-                            '/var/lib/left4me/.venv/bin/gunicorn '
-                            f'--workers {workers} --threads {threads} '
-                            "--bind 127.0.0.1:8000 'l4d2web.app:create_app()'"
-                        ),
-                        'Restart': 'on-failure',
-                        'RestartSec': '3',
-
-                        # Web app writes broadly under /var/lib/left4me. Kept inline
-                        # because it's web-specific (server@ uses BindPaths to bind
-                        # only its instance dir).
-                        'ReadWritePaths': '/var/lib/left4me',
-
-                        # Hardening profile delivered via
-                        # /etc/systemd/system/left4me-web.service.d/10-hardening.conf
-                        # (target-side symlink into left4me/deploy/files/, owned by left4me).
-                    },
-                    'Install': {
-                        'WantedBy': {'multi-user.target'},
-                    },
-                },
-                'left4me-server@.service': {
-                    'Unit': {
-                        'Description': 'left4me server instance %i',
-                        'After': 'network-online.target',
-                        'Wants': 'network-online.target',
-                        'StartLimitBurst': '5',
-                        'StartLimitIntervalSec': '60s',
-                    },
-                    'Service': {
-                        'Type': 'simple',
-                        'User': 'left4me',
-                        'Group': 'left4me',
-                        'EnvironmentFile': (
-                            '/etc/left4me/host.env',
-                            '/var/lib/left4me/instances/%i/instance.env',
-                        ),
-                        'WorkingDirectory': '-/var/lib/left4me/runtime/%i/merged/left4dead2',
-                        'ExecStartPre': '+/usr/bin/nsenter --mount=/proc/1/ns/mnt -- /usr/local/libexec/left4me/left4me-overlay mount %i',
-                        # +ip 0.0.0.0 binds RCON (TCP) to all interfaces incl. loopback;
-                        # without this, Source auto-selects the primary IP and the web
-                        # app's 127.0.0.1 RCON connect gets ECONNREFUSED. External TCP
-                        # on the game port range is firewall-blocked in nftables_input.
-                        'ExecStart': '/var/lib/left4me/runtime/%i/merged/srcds_run -game left4dead2 +ip 0.0.0.0 +hostport ${L4D2_PORT} $L4D2_ARGS',
-                        'ExecStopPost': '+/usr/bin/nsenter --mount=/proc/1/ns/mnt -- /usr/local/libexec/left4me/left4me-overlay umount %i',
-                        'Restart': 'on-failure',
-                        'RestartSec': '5',
-
-                        # Resource control (baseline from prior performance work).
-                        'Slice': 'l4d2-game.slice',
-                        'Nice': '-5',
-                        'IOSchedulingClass': 'best-effort',
-                        'IOSchedulingPriority': '4',
-                        'OOMScoreAdjust': '-200',
-                        'MemoryHigh': '1.5G',
-                        'MemoryMax': '2G',
-                        'TasksMax': '256',
-                        'LimitNOFILE': '65536',
-                        'KillSignal': 'SIGINT',
-                        'TimeoutStopSec': '15s',
-                        'LogRateLimitIntervalSec': '0',
-
-                        # Hardening profile delivered via
-                        # /etc/systemd/system/left4me-server@.service.d/10-hardening.conf
-                        # (target-side symlink into left4me/deploy/files/, owned by left4me).
-                    },
-                    'Install': {
-                        'WantedBy': {'multi-user.target'},
-                    },
-                },
-                'l4d2-game.slice': {
-                    'Unit': {
-                        'Description': 'left4me game-server slice',
-                        'Before': 'slices.target',
-                    },
-                    'Slice': {
-                        'CPUWeight': '1000',
-                        'IOWeight': '1000',
-                        'AllowedCPUs': game_cpus_string,
-                    },
-                },
-                'l4d2-build.slice': {
-                    'Unit': {
-                        'Description': 'left4me script-sandbox build slice',
-                        'Before': 'slices.target',
-                    },
-                    'Slice': {
-                        'CPUWeight': '10',
-                        'IOWeight': '10',
-                        'AllowedCPUs': system_cpus_string,
-                    },
-                },
-                'system.slice.d/99-left4me-cpuset.conf': cpuset_dropin,
-                'user.slice.d/99-left4me-cpuset.conf':   cpuset_dropin,
-            },
-        },
-    }

bundles/letsencrypt/README.md

@@ -1,60 +1,9 @@
-# letsencrypt
-
-Issues and renews Let's Encrypt certs via [dehydrated][upstream] with
-DNS-01 against the in-house bind-acme server.
-
-[upstream]: https://github.com/dehydrated-io/dehydrated/wiki/example-dns-01-nsupdate-script
-
-## First-apply behaviour
-
-Immediately after `bw apply <node>`, nginx serves a **self-signed
-cert** for each declared domain — generated by
-`/etc/dehydrated/letsencrypt-ensure-some-certificate` so nginx has
-something to start with. The real Let's Encrypt cert arrives at most
-24h later when the systemd timer fires
-(`/usr/bin/dehydrated --cron --accept-terms --challenge dns-01`). To
-shortcut the wait:
-
-```sh
-ssh <node> 'sudo /usr/bin/dehydrated --cron --accept-terms --challenge dns-01'
-ssh <node> 'sudo systemctl reload nginx'
-```
-
-## DNS-01 prerequisites
-
-`hook.sh` does `nsupdate` against the bind-acme server (referenced
-by `letsencrypt/acme_node`). For the challenge to succeed:
-
-1. The acme node must be in the same metadata graph (so
-   `bw metadata <node> -k letsencrypt/acme_node` resolves).
-2. **All NS servers** for the validated domain must serve the
-   `_acme-challenge.<domain>` CNAME — Let's Encrypt validates from
-   primary AND secondary geographic regions; both authoritative
-   servers must agree. If a secondary NS is also a bw-managed node,
-   `bw apply` it after adding the domain (see e.g. `ovh.secondary`).
-3. The bind-acme node's TSIG key must be reachable. `hook.sh` is
-   rendered with the bind-acme server's `network/internal/ipv4` —
-   for clients outside that LAN, the route must exist (typically via
-   wireguard `s2s` peer membership).
-
-## Negative-cache penalty
-
-If the first DNS-01 attempt fails (e.g. zone not yet applied to the
-secondary NS), Let's Encrypt's resolvers cache NXDOMAIN for the SOA's
-negative TTL (often 900s = 15 min). Subsequent attempts during that
-window also fail and refresh the cache. Combined with LE's rate limit
-of **5 failed authorisations per domain per hour**, recovery requires
-you to **stop retrying** for ~15 minutes after fixing the DNS, then
-make at most one attempt.
-
-## nsupdate sample
-
-For interactive testing of the bind-acme TSIG path:
+https://github.com/dehydrated-io/dehydrated/wiki/example-dns-01-nsupdate-script
 
 ```sh
 printf "server 127.0.0.1
 zone acme.resolver.name.
-update add _acme-challenge.ckn.li.acme.resolver.name. 600 IN TXT \"hello\"
+update add _acme-challenge.ckn.li.acme.resolver.name. 600 IN TXT "hello"
 send
 " | nsupdate -y hmac-sha512:acme:XXXXXX
 ```

bundles/letsencrypt/metadata.py

@@ -2,7 +2,7 @@ defaults = {
     'apt': {
         'packages': {
             'dehydrated': {},
-            'bind9-dnsutils': {},
+            'dnsutils': {},
         },
     },
     'letsencrypt': {

bundles/linux/metadata.py

@@ -1,6 +1,4 @@
 defaults = {
-    'sysctl': {
-        'net.ipv4.icmp_ratelimit': '100',
-    },
+    'sysctl': {},
     'modules-load': set(),
 }

bundles/mailserver/metadata.py

@@ -82,7 +82,6 @@ def dns(metadata):
         'dns': dns,
     }
 
-
 @metadata_reactor.provides(
     'letsencrypt/domains',
 )

bundles/monitored/metadata.py

@@ -42,7 +42,7 @@ def user(metadata):
         'users': {
             'sshmon': {
                 'authorized_users': {
-                    'nagios@' + metadata.get('monitoring/icinga2_node'): {},
+                    'nagios@' + metadata.get('monitoring/icinga2_node'),
                 }
             },
         },
@@ -50,7 +50,7 @@ def user(metadata):
             'sshmon': {
                 conf['vars.command']
                     for conf in metadata.get('monitoring/services').values()
-                    if conf.get('check_command') == 'sshmon'
+                    if conf['check_command'] == 'sshmon'
                     and conf.get('vars.sudo', None)
             },
         },

bundles/network/metadata.py

@@ -37,12 +37,11 @@ def dhcp(metadata):
     'modules-load',
 )
 def units(metadata):
-    networks = metadata.get('network', {})
     if node.has_bundle('systemd-networkd'):
         units = {}
         modules_load = set()
 
-        for network_name, network_conf in networks.items():
+        for network_name, network_conf in metadata.get('network').items():
             interface_type = network_conf.get('type', None)
 
             # network
@@ -117,7 +116,6 @@ def units(metadata):
     'systemd/units',
 )
 def queuing_disciplines(metadata):
-    networks = metadata.get('network', {})
     if node.has_bundle('systemd-networkd'):
         return {
             'systemd': {
@@ -138,7 +136,7 @@ def queuing_disciplines(metadata):
                             'WantedBy': 'network-online.target',
                         },
                     }
-                        for network_name, network_conf in networks.items()
+                        for network_name, network_conf in metadata.get('network').items()
                         if 'qdisc' in network_conf
                 },
             },

bundles/nextcloud-picsort/files/nextcloud-picsort

@@ -1,209 +1,78 @@
-#!/usr/bin/env python3
-import argparse
-import base64
-import hashlib
-import os
-import shutil
-import subprocess
-from concurrent.futures import ThreadPoolExecutor, as_completed
-from datetime import datetime
-from pathlib import Path
+#!/bin/bash
 
+USER="$1"
 
-ALLOWED_EXTS = {
-    ".png", ".jpg", ".jpeg", ".heic", ".cr2", ".cr3", ".mp4", ".mov",
-    ".webp", ".avif", ".gif",
-}
+REL_SOURCE_PATH="/$1/files/$2"
+ABS_SOURCE_PATH="/var/lib/nextcloud/$1/files/$2"
 
-DATETIME_KEYS = [
-    ("Composite", "SubSecDateTimeOriginal"),
-    ("Composite", "SubSecCreateDate"),
-    ("ExifIFD", "DateTimeOriginal"),
-    ("ExifIFD", "CreateDate"),
-    ("XMP-xmp", "CreateDate"),
-    ("Keys", "CreationDate"),
-    ("QuickTime", "CreateDate"),
-    ("XMP-photoshop", "DateCreated"),
-]
+REL_DEST_PATH="/$1/files/$3"
+ABS_DEST_PATH="/var/lib/nextcloud/$1/files/$3"
 
+REL_UNSORTABLE_PATH="/$1/files/$4"
+ABS_UNSORTABLE_PATH="/var/lib/nextcloud/$1/files/$4"
 
-def run(command: list[str], check: bool = True) -> subprocess.CompletedProcess:
-    return subprocess.run(command, text=True, capture_output=True, check=check)
+echo "STARTING..."
 
+chown -R www-data:www-data "$ABS_SOURCE_PATH"
+chmod -R 770 "$ABS_SOURCE_PATH"
 
-def exiftool_data(file: Path) -> dict | None:
-    result = run([
-        "exiftool",
-        "-j",
-        "-a",
-        "-u",
-        "-g1",
-        "-time:all",
-        "-api", "QuickTimeUTC=1",
-        "-d", "%Y-%m-%dT%H:%M:%S%z",
-        str(file),
-    ], check=False)
-    if result.returncode != 0:
-        return None
-    try:
-        data = __import__("json").loads(result.stdout)
-        return data[0] if data else None
-    except Exception:
-        return None
+SCAN="FALSE"
+IFS=$'\n'
+for f in `find "$ABS_SOURCE_PATH" -iname *.PNG -o -iname *.JPG -o -iname *.JPEG -o -iname *.HEIC -o -iname *.CR2 -o -iname *.CR3 -o -iname *.MP4 -o -iname *.MOV`; do
+  SCAN="TRUE"
+  echo "PROCESSING: $f"
 
+  EXIF=`exiftool "$f"`
+  if grep -q '^Create Date' <<< $EXIF
+  then
+    DATETIME=`grep -m 1 "^Create Date" <<< $EXIF | cut -d: -f2- | xargs`
+  elif grep -q '^File Modification Date' <<< $EXIF
+  then
+    DATETIME=`grep -m 1 '^File Modification Date' <<< $EXIF | cut -d: -f2- | xargs`
+  else
+    RELPATH=$(realpath --relative-to="$ABS_SOURCE_PATH" "$f")
+    DIRNAME=$(dirname "$ABS_UNSORTABLE_PATH/$RELPATH")
+    echo "UNSORTABLE: $f"
+    mkdir -p "$DIRNAME"
+    mv "$f" "$DIRNAME"
+    continue
+  fi
 
-def exiftool_timestamp(file: Path) -> datetime | None:
-    data = exiftool_data(file)
-    if not data:
-        return None
+  DATE=`cut -d' ' -f1 <<< $DATETIME`
+  TIME=`cut -d' ' -f2 <<< $DATETIME | cut -d'+' -f1`
 
-    for category, key in DATETIME_KEYS:
-        try:
-            value = data[category][key]
-        except (KeyError, TypeError):
-            continue
-        try:
-            return datetime.strptime(value, "%Y-%m-%dT%H:%M:%S%z")
-        except ValueError:
-            continue
+  YEAR=`cut -d':' -f1 <<< $DATE`
+  MONTH=`cut -d':' -f2 <<< $DATE`
+  DAY=`cut -d':' -f3 <<< $DATE`
+  HOUR=`cut -d':' -f1 <<< $TIME`
+  MINUTE=`cut -d':' -f2 <<< $TIME`
+  SECOND=`cut -d':' -f3 <<< $TIME`
 
-    return None
+  HASH=`sha256sum "$f" | xxd -r -p | base64 | head -c 3 | tr '/+' '_-'`
+  EXT=`echo "${f##*.}" | tr '[:upper:]' '[:lower:]'`
+  if [[ "$EXT" = "cr2" ]] ||  [[ "$EXT" = "cr3" ]]
+  then
+    RAW="raw/"
+  else
+    RAW=""
+  fi
+  FILE="$ABS_DEST_PATH/$YEAR-$MONTH/$RAW$YEAR$MONTH$DAY"-"$HOUR$MINUTE$SECOND"_"$HASH"."$EXT"
+  echo "DESTINATION: $FILE"
+  mkdir -p "$(dirname "$FILE")"
+  mv -v "$f" "$FILE"
+done
 
+if [ "$SCAN" == "TRUE" ]; then
+  echo "SCANNING..."
+  # find "$ABS_SOURCE_PATH/"* -type d -empty -delete >> /var/echo/nc-picsort.echo # nextcloud app bug when deleting folders
+  chown -R www-data:www-data "$ABS_DEST_PATH"
+  chown -R www-data:www-data "$ABS_UNSORTABLE_PATH"
+  chmod -R 770 "$ABS_DEST_PATH"
+  chmod -R 770 "$ABS_UNSORTABLE_PATH"
+  sudo -u www-data php /opt/nextcloud/occ files:scan --path "$REL_SOURCE_PATH"
+  sudo -u www-data php /opt/nextcloud/occ files:scan --path "$REL_UNSORTABLE_PATH"
+  sudo -u www-data php /opt/nextcloud/occ files:scan --path "$REL_DEST_PATH"
+  #sudo -u www-data php /opt/nextcloud/occ preview:pre-generate
+fi
 
-def short_hash(file: Path) -> str:
-    h = hashlib.sha256()
-    with file.open("rb") as fh:
-        for chunk in iter(lambda: fh.read(1024 * 1024), b""):
-            h.update(chunk)
-    digest = h.digest()
-    b64 = base64.b64encode(digest).decode("ascii")
-    return b64[:3].replace("/", "_").replace("+", "-")
-
-
-def build_destination(dest_root: Path, file: Path, ts: datetime) -> Path:
-    ext = file.suffix.lower().lstrip(".")
-    year = ts.strftime("%Y")
-    month = ts.strftime("%m")
-    day = ts.strftime("%d")
-    hour = ts.strftime("%H")
-    minute = ts.strftime("%M")
-    second = ts.strftime("%S")
-    hash_part = short_hash(file)
-
-    raw_subdir = "raw" if ext in {"cr2", "cr3"} else None
-    month_dir = dest_root / f"{year}-{month}"
-    if raw_subdir:
-        month_dir = month_dir / raw_subdir
-
-    filename = f"{year}{month}{day}-{hour}{minute}{second}_{hash_part}.{ext}"
-    return month_dir / filename
-
-
-def move_unsortable(file: Path, source_root: Path, unsortable_root: Path) -> None:
-    relpath = file.relative_to(source_root)
-    target_dir = (unsortable_root / relpath).parent
-    target_dir.mkdir(parents=True, exist_ok=True)
-    shutil.chown(str(target_dir), user="www-data", group="www-data")
-    target = target_dir / file.name
-    if target.exists():
-        return
-    shutil.move(str(file), str(target))
-    shutil.chown(str(target), user="www-data", group="www-data")
-
-
-def move_sorted(file: Path, target: Path) -> None:
-    target.parent.mkdir(parents=True, exist_ok=True)
-    shutil.chown(str(target.parent), user="www-data", group="www-data")
-    shutil.move(str(file), str(target))
-    shutil.chown(str(target), user="www-data", group="www-data")
-
-def process_file(file: Path, source_root: Path, dest_root: Path, unsortable_root: Path) -> tuple[Path, str]:
-    print(f"PROCESSING: {file}")
-    ts = exiftool_timestamp(file)
-
-    if ts is None:
-        print(f"UNSORTABLE: {file}")
-        move_unsortable(file, source_root, unsortable_root)
-        return file, "unsortable"
-
-    target = build_destination(dest_root, file, ts)
-    print(f"DESTINATION: {target}")
-    move_sorted(file, target)
-    return file, "sorted"
-
-
-def scan_nextcloud(rel_source: str, rel_unsortable: str, rel_dest: str) -> None:
-    print("SCANNING...")
-    # run(["chown", "-R", "www-data:www-data", abs_source_path], check=True)
-    # run(["chmod", "-R", "770", abs_source_path], check=True)
-
-    # run(["chown", "-R", "www-data:www-data", abs_dest_path], check=True)
-    # run(["chown", "-R", "www-data:www-data", abs_unsortable_path], check=True)
-    # run(["chmod", "-R", "770", abs_dest_path], check=True)
-    # run(["chmod", "-R", "770", abs_unsortable_path], check=True)
-
-    run(["sudo", "-u", "www-data", "php", "/opt/nextcloud/occ", "files:scan", "--path", rel_source], check=True)
-    run(["sudo", "-u", "www-data", "php", "/opt/nextcloud/occ", "files:scan", "--path", rel_unsortable], check=True)
-    run(["sudo", "-u", "www-data", "php", "/opt/nextcloud/occ", "files:scan", "--path", rel_dest], check=True)
-
-    run(["systemctl", "start", "nextcloud-generate-new-previews.service"], check=True)
-
-
-def iter_files(source_root: Path):
-    for path in source_root.rglob("*"):
-        if path.is_file() and path.suffix.lower() in ALLOWED_EXTS:
-            yield path
-
-
-if __name__ == "__main__":
-    parser = argparse.ArgumentParser(
-        description="Sort Nextcloud media files by embedded timestamp."
-    )
-    parser.add_argument("nc_user")
-    parser.add_argument("source_subdir")
-    parser.add_argument("dest_subdir")
-    parser.add_argument("unsortable_subdir")
-    parser.add_argument("--workers", type=int, default=os.cpu_count() or 1)
-    args = parser.parse_args()
-
-    nc_user = args.nc_user
-    source_subdir = args.source_subdir
-    dest_subdir = args.dest_subdir
-    unsortable_subdir = args.unsortable_subdir
-
-    rel_source_path = f"/{nc_user}/files/{source_subdir}"
-    abs_source_path = f"/var/lib/nextcloud/{nc_user}/files/{source_subdir}"
-
-    rel_dest_path = f"/{nc_user}/files/{dest_subdir}"
-    abs_dest_path = f"/var/lib/nextcloud/{nc_user}/files/{dest_subdir}"
-
-    rel_unsortable_path = f"/{nc_user}/files/{unsortable_subdir}"
-    abs_unsortable_path = f"/var/lib/nextcloud/{nc_user}/files/{unsortable_subdir}"
-
-    source_root = Path(abs_source_path)
-    dest_root = Path(abs_dest_path)
-    unsortable_root = Path(abs_unsortable_path)
-
-    print("STARTING...")
-
-    run(["chown", "-R", "www-data:www-data", str(source_root)], check=True)
-    run(["chmod", "-R", "770", str(source_root)], check=True)
-
-    files = list(iter_files(source_root))
-
-    if not files:
-        print("NO MATCHING FILES FOUND.")
-        print("FINISH.")
-        raise SystemExit(0)
-
-    with ThreadPoolExecutor(max_workers=max(1, args.workers)) as executor:
-        futures = {
-            executor.submit(process_file, file, source_root, dest_root, unsortable_root): file
-            for file in files
-        }
-        for future in as_completed(futures):
-            future.result()
-
-    scan_nextcloud(rel_source_path, rel_unsortable_path, rel_dest_path)
-
-    print("FINISH.")
+echo "FINISH."

bundles/nextcloud/README.md

@@ -1,71 +0,0 @@
-Nextcloud
-=========
-
-import iphone pictures
-----------------------
-
-Use Photos app on macOS
-- select library in the left sidebar
-- select the pictures
-- in menu bar open File > Export Unmodified Original for X Photos
-
-The only reliable way to get some files creation time is being lost with rsync, so
-we need to embed those timestamps on macos first:
-
-```sh
-PHOTOS_PATH="/Users/mwiegand/Desktop/photos"
-bin/timestamp_icloud_photos_for_nextcloud -d "$PHOTOS_PATH"
-rsync -avh --progress --rsync-path="sudo rsync" "$PHOTOS_PATH/" ckn@10.0.0.2:/var/lib/nextcloud/ckn/files/SofortUpload/AutoSort/
-```
-
-preview generator
------------------
-
-```
-sudo -u www-data php /opt/nextcloud/occ preview:generate-all -w "$(nproc)" -n -vvv
-```
-
-This index speeds up preview generator dramatically:
-```sh
-CREATE INDEX CONCURRENTLY oc_filecache_path_hash_idx
-ON oc_filecache (path_hash);
-```
-
-delete previews:
-```sh
-psql nextcloud -x -c "DELETE FROM oc_previews;"
-rm -rf /var/lib/nextcloud/appdata_oci6dw1woodz/preview/*
-```
-
-https://docs.nextcloud.com/server/stable/admin_manual/configuration_files/previews_configuration.html#maximum-preview-size
-```php
-    'preview_max_x' => 1920,
-    'preview_max_y' => 1920,
-    'preview_max_scale_factor' => 4,
-```
-
-https://github.com/nextcloud/previewgenerator?tab=readme-ov-file#i-dont-want-to-generate-all-the-preview-sizes
-```sh
-sudo -u www-data php /opt/nextcloud/occ config:app:set --value="64 256" previewgenerator squareSizes
-sudo -u www-data php /opt/nextcloud/occ config:app:set --value="" previewgenerator fillWidthHeightSizes # changed
-sudo -u www-data php /opt/nextcloud/occ config:app:set --value="" previewgenerator widthSizes
-sudo -u www-data php /opt/nextcloud/occ config:app:set --value="" previewgenerator heightSizes
-sudo -u www-data php /opt/nextcloud/occ config:app:set preview jpeg_quality --value="75"
-sudo -u www-data php /opt/nextcloud/occ config:app:set --value=0 --type=integer previewgenerator job_max_previews # in favour of systemd timer
-```
-
-gen previews
-```sh
-php /opt/nextcloud/occ preview:generate-all --workers="$(nproc)" --no-interaction -vvv
-```
-
-check preview geenration
-```sh
-find /var/lib/nextcloud/appdata_oci6dw1woodz/preview
-# /var/lib/nextcloud/appdata_oci6dw1woodz/preview/6/9/1/f/7/b/4/2822419/64-64-crop.jpg
-# /var/lib/nextcloud/appdata_oci6dw1woodz/preview/6/9/1/f/7/b/4/2822419/256-256-crop.jpg
-# /var/lib/nextcloud/appdata_oci6dw1woodz/preview/6/9/1/f/7/b/4/2822419/1280-1920-max.jpg
-
-du -sh /var/lib/nextcloud/appdata_oci6dw1woodz/preview
-# 28G	/var/lib/nextcloud/appdata_oci6dw1woodz/preview
-```

bundles/nextcloud/files/rescan

@@ -0,0 +1,5 @@
+#!/bin/bash
+
+php /opt/nextcloud/occ files:scan --all
+php /opt/nextcloud/occ files:scan-app-data
+#php /opt/nextcloud/occ preview:generate-all

bundles/nextcloud/items.py

@@ -146,3 +146,15 @@ actions['nextcloud_add_missing_inidces'] = {
         f'action:extract_nextcloud',
     ],
 }
+
+# RESCAN
+
+files['/opt/nextcloud_rescan'] = {
+    'source': 'rescan',
+    'owner': 'www-data',
+    'group': 'www-data',
+    'mode': '550',
+    'needs': [
+        'action:extract_nextcloud',
+    ],
+}

bundles/nextcloud/metadata.py

@@ -1,5 +1,5 @@
-from shlex import quote
-
+import string
+from uuid import UUID
 
 defaults = {
     'apt': {
@@ -85,35 +85,11 @@ defaults = {
             'user': 'www-data',
             'kill_mode': 'process',
         },
-        'nextcloud-scan-app-data': {
-            'command': '/usr/bin/php /opt/nextcloud/occ files:scan-app-data',
-            'when': 'yearly',
+        'nextcloud-rescan': {
+            'command': '/opt/nextcloud_rescan',
+            'when': 'Sun 00:00:00',
             'user': 'www-data',
         },
-        'nextcloud-scan-files': {
-            'command': '/usr/bin/php /opt/nextcloud/occ files:scan --all',
-            'when': 'weekly',
-            'user': 'www-data',
-            'after': {
-                'nextcloud-scan-app-data.service',
-            },
-        },
-        'nextcloud-generate-all-previews': {
-            'command': '/bin/bash -c ' + quote('php /opt/nextcloud/occ preview:generate-all --workers="$(nproc)" --no-interaction -vvv'),
-            'when': 'monthly',
-            'user': 'www-data',
-            'after': {
-                'nextcloud-scan-files.service',
-            },
-        },
-        'nextcloud-generate-new-previews': {
-            'command': '/usr/bin/php /opt/nextcloud/occ preview:pre-generate --no-interaction -vvv',
-            'when': '*:0/5', # every 5 minutes
-            'user': 'www-data',
-            'after': {
-                'nextcloud-generate-all-previews.service',
-            },
-        },
     },
 }
 
@@ -158,18 +134,10 @@ def config(metadata):
                     '127.0.0.1',
                     metadata.get('nextcloud/hostname'),
                 ],
-                'enabledPreviewProviders': [
-                    'OC\\Preview\\Image',
-                    'OC\\Preview\\Movie',
-                    'OC\\Preview\\HEIC',
-                ],
-                'preview_max_x': 1920,
-                'preview_max_y': 1920,
-                'preview_max_scale_factor': 4,
                 'log_type': 'syslog',
                 'syslog_tag': 'nextcloud',
                 'logfile': '',
-                'loglevel': 2,
+                'loglevel': 3,
                 'default_phone_region': 'DE',
                 'versions_retention_obligation': 'auto, 90',
                 'simpleSignUpLink.shown': False,

bundles/nginx/README.md

@@ -1,36 +0,0 @@
-# nginx
-
-Webserver. Per-node vhosts in `nginx/vhosts`; per-vhost templates in
-`data/nginx/*.conf`.
-
-## How port 80 is served
-
-The bundle ships a fixed `80.conf` to
-`/etc/nginx/sites-available/80.conf` (picked up by the
-`sites-enabled/` symlink) that handles **all** port-80 traffic
-across vhosts:
-
-1. ACME HTTP-01 challenges (`/.well-known/acme-challenge/`) are
-   served from `/var/lib/dehydrated/acme-challenges/`.
-2. All other port-80 requests are 301-redirected to
-   `https://$host$request_uri`.
-
-Per-vhost templates only declare `listen 443 ssl http2;`, so they
-don't need their own port-80 server blocks. If you need vhost-
-specific port-80 behaviour (e.g. plain-HTTP without redirect),
-override 80.conf or add a per-vhost block.
-
-## Required metadata
-
-- `vm/cores` — read directly by `items.py` for `worker_processes`.
-  No default; `bw items <node>` raises at item-build time if missing.
-  Typically supplied by the `vm` bundle / hetzner-vm group; double-
-  check on bare-metal hosts.
-- `nginx/vhosts` — dict of vhost-name → vhost-config.
-- `nginx/modules` — list of dynamic modules to load.
-
-## Cross-namespace
-
-`items.py` reads `letsencrypt/domains` to skip emitting a per-vhost
-HTTPS block when LE hasn't declared the domain yet — keeps the
-bundle loadable on a node where letsencrypt isn't fully wired up.

bundles/nginx/files/nginx.conf

@@ -32,13 +32,12 @@ http {
 
     % endif
 
-    # Always defined: serves both WS-enabled vhosts (Connection: upgrade for
-    # ws clients) and SSE/keep-alive vhosts (Connection: "" lets nginx manage
-    # the upstream connection for keep-alive, instead of forcing "close").
+    % if has_websockets:
     map $http_upgrade $connection_upgrade {
         default upgrade;
-        '' '';
+        '' close;
     }
+    % endif
 
     include /etc/nginx/sites-enabled/*;
 }

bundles/nginx/items.py

@@ -64,7 +64,7 @@ files = {
             'svc_systemd:nginx:restart',
         },
     },
-    '/etc/nginx/sites-available/80.conf': {
+    '/etc/nginx/sites/80.conf': {
         'triggers': {
             'svc_systemd:nginx:restart',
         },

bundles/postfix/metadata.py

@@ -25,9 +25,9 @@ defaults = {
         },
     },
     'telegraf': {
-        'inputs': {
-            'postfix': {
-                'default': {},
+        'config': {
+            'inputs': {
+                'postfix': [{}],
             },
         },
     },

bundles/postgresql/metadata.py

@@ -31,17 +31,6 @@ defaults = {
     'grafana_rows': set(),
 }
 
-if node.has_bundle('zfs'):
-    defaults['zfs'] = {
-        'datasets': {
-            'tank/postgresql': {
-                'mountpoint': '/var/lib/postgresql',
-                'recordsize': '8192',
-                'atime': 'off',
-            },
-        },
-    }
-
 
 @metadata_reactor.provides(
     'postgresql/conf',
@@ -89,17 +78,37 @@ def apt(metadata):
 
 
 @metadata_reactor.provides(
-    'telegraf/inputs/postgresql/default',
+    'zfs/datasets',
 )
-def telegraf(metadata):
+def zfs(metadata):
+    if not node.has_bundle('zfs'):
+        return {}
+
     return {
-        'telegraf': {
-            'inputs': {
-                'postgresql': {
-                    'default': {
-                        'address': f'postgres://root:{root_password}@localhost:5432/postgres',
-                        'databases': sorted(list(node.metadata.get('postgresql/databases').keys())),
-                    },
+        'zfs': {
+            'datasets': {
+                'tank/postgresql': {
+                    'mountpoint': '/var/lib/postgresql',
+                    'recordsize': '8192',
+                    'atime': 'off',
+                },
+            },
+        },
+    }
+
+
+@metadata_reactor.provides(
+    'telegraf/config/inputs/postgresql',
+)
+def telegraf(metadata):
+    return {
+        'telegraf': {
+            'config': {
+                'inputs': {
+                    'postgresql': [{
+                        'address': f'postgres://root:{root_password}@localhost:5432/postgres',
+                        'databases': sorted(list(node.metadata.get('postgresql/databases').keys())),
+                    }],
                 },
             },
         },

bundles/proxmox-ve/metadata.py

@@ -18,7 +18,7 @@ defaults = {
         'sources': {
             'proxmox-ve': {
                 'options': {
-                    'Architectures': 'amd64',
+                    'aarch': 'amd64',
                 },
                 'urls': {
                     'http://download.proxmox.com/debian/pve',

bundles/raspberry-pi/metadata.py

@@ -6,24 +6,35 @@ defaults = {
     },
 }
 
-if node.has_bundle('zfs'):
-    defaults['zfs'] = {
-        'kernel_params': {
-            'zfs_txg_timeout': 300,
+
+@metadata_reactor.provides(
+    'telegraf/config/agent',
+)
+def telegraf(metadata):
+    return {
+        'telegraf': {
+            'config': {
+                'agent': {
+                    'flush_interval': '30s',
+                    'interval': '30s',
+                },
+            },
         },
     }
 
 
 @metadata_reactor.provides(
-    'telegraf/agent',
+    'zfs/kernel_params',
+    'zfs/datasets',
 )
-def telegraf(metadata):
-    metadata.get('telegraf/agent')  # only override if telegraf bundle is present
+def zfs(metadata):
+    if not node.has_bundle('zfs'):
+        return {}
+
     return {
-        'telegraf': {
-            'agent': {
-                'flush_interval': '30s',
-                'interval': '1m',
+        'zfs': {
+            'kernel_params': {
+                'zfs_txg_timeout': 300,
             },
         },
     }

bundles/raspberrymatic-cert/metadata.py

@@ -1,13 +1,3 @@
-defaults = {
-    'systemd-timers': {
-        'raspberrymatic-cert': {
-            'command': '/opt/raspberrymatic-cert',
-            'when': 'daily',
-        },
-    },
-}
-
-
 @metadata_reactor.provides(
     'letsencrypt/domains',
 )
@@ -21,3 +11,17 @@ def letsencrypt(metadata):
             },
         },
     }
+
+
+@metadata_reactor.provides(
+    'systemd-timers/raspberrymatic-cert',
+)
+def systemd_timers(metadata):
+    return {
+        'systemd-timers': {
+            'raspberrymatic-cert': {
+                'command': '/opt/raspberrymatic-cert',
+                'when': 'daily',
+            }
+        },
+    }

bundles/roundcube/files/config.inc.php

@@ -9,11 +9,11 @@ $config['enable_installer'] = true;
 $config['db_dsnw'] = '${database['provider']}://${database['user']}:${database['password']}@${database['host']}/${database['name']}';
 $config['imap_host'] = 'ssl://${imap_host}';
 $config['imap_port'] = 993;
-#$config['imap_debug'] = true;
-$config['smtp_host'] = 'tls://${imap_host}';
+$config['smtp_host'] = 'tls://localhost';
 $config['smtp_port'] = 587;
 $config['smtp_user'] = '%u';
 $config['smtp_pass'] = '%p';
+#$config['imap_debug'] = true;
 #$config['smtp_debug'] = true;
 $config['support_url'] = '';
 $config['des_key'] = '${des_key}';

bundles/routeros-monitoring/items.py

@@ -1,11 +0,0 @@
-files = {
-    # https://mikrotik.com/download/tools
-    '/usr/share/snmp/mibs/MIKROTIK-MIB.txt': {
-        'source': 'mikrotik.mib',
-        'content_type': 'binary',
-        'mode': '0644',
-        'needed_by': {
-            'svc_systemd:telegraf.service',
-        },
-    },
-}

bundles/routeros-monitoring/metadata.py

@@ -1,444 +0,0 @@
-input_defaults = {
-    "agents": [
-        f"udp://{routeros_node.hostname}:161"
-            for routeros_node in repo.nodes_in_group("routeros")
-    ],
-    "agent_host_tag": "source",
-    "version": 2,
-    "community": "public",
-    "max_repetitions": 5, # supposedly less spiky loads
-    "tags": {
-        "operating_system": "routeros",
-    },
-}
-
-defaults = {
-    'apt': {
-        'packages': {
-            'snmp': {},
-            'snmp-mibs-downloader': {},
-        },
-    },
-    "telegraf": {
-        "processors": {
-            "enum": {
-                "mikrotik_host_mapping":{
-                    # - measurements get switch ip as agent_host tag
-                    # - wie define a value mapping ip -> node name
-                    # - agent_host gets translated and written into host tag
-                    "tagpass": {
-                        "operating_system": ["routeros"],
-                    },
-                    "mapping": [
-                        {
-                            "tags": ["source"],
-                            "dest": "host",
-                            "default": "unknown",
-                            "value_mappings": {
-                                routeros_node.hostname: routeros_node.name
-                                    for routeros_node in repo.nodes_in_group("routeros")
-                            },
-                        },
-                    ],
-                },
-            },
-        },
-        "inputs": {
-            "snmp": {
-                "mikrotik_switches_fast": {
-                    "interval": "2m",
-                    "collection_jitter": "20s",
-                    **input_defaults,
-                    "table": [
-                        # CPU load (HR-MIB)
-                        {
-                            "name": "mikrotik_cpu",
-                            "oid": "HOST-RESOURCES-MIB::hrProcessorTable",
-                            "field": [
-                                {
-                                    "name": "frw_id",
-                                    "oid": "HOST-RESOURCES-MIB::hrProcessorFrwID",
-                                    "is_tag": True,
-                                },
-                                {
-                                    "name": "load",
-                                    "oid": "HOST-RESOURCES-MIB::hrProcessorLoad",
-                                },
-                            ],
-                        },
-                        # Storage (HR-MIB)
-                        {
-                            "name": "mikrotik_storage",
-                            "oid": "HOST-RESOURCES-MIB::hrStorageTable",
-                            "field": [
-                                {
-                                    "name": "index",
-                                    "oid": "HOST-RESOURCES-MIB::hrStorageIndex",
-                                    "is_tag": True,
-                                },
-                                {
-                                    "name": "type",
-                                    "oid": "HOST-RESOURCES-MIB::hrStorageType",
-                                    "is_tag": True,
-                                },
-                                {
-                                    "name": "descr",
-                                    "oid": "HOST-RESOURCES-MIB::hrStorageDescr",
-                                    "is_tag": True,
-                                },
-                                {
-                                    "name": "alloc_unit",
-                                    "oid": "HOST-RESOURCES-MIB::hrStorageAllocationUnits",
-                                },
-                                {
-                                    "name": "size",
-                                    "oid": "HOST-RESOURCES-MIB::hrStorageSize",
-                                },
-                                {
-                                    "name": "used",
-                                    "oid": "HOST-RESOURCES-MIB::hrStorageUsed",
-                                },
-                                {
-                                    "name": "alloc_failures",
-                                    "oid": "HOST-RESOURCES-MIB::hrStorageAllocationFailures",
-                                },
-                            ],
-                        },
-                        # MikroTik Health (table)
-                        {
-                            "name": "mikrotik_health",
-                            "oid": "MIKROTIK-MIB::mtxrGaugeTable",
-                            "field": [
-                                {
-                                    "name": "sensor",
-                                    "oid": "MIKROTIK-MIB::mtxrGaugeName",
-                                    "is_tag": True,
-                                },
-                                {
-                                    "name": "value",
-                                    "oid": "MIKROTIK-MIB::mtxrGaugeValue",
-                                },
-                                {
-                                    "name": "unit",
-                                    "oid": "MIKROTIK-MIB::mtxrGaugeUnit",
-                                    "is_tag": True,
-                                },
-                            ],
-                        },
-                    ],
-                },
-                "mikrotik_switches_slow": {
-                    "interval": "7m",
-                    "collection_jitter": "2m",
-                    **input_defaults,
-                    "table": [
-                        # Interface statistics (standard IF-MIB)
-                        {
-                            "name": "mikrotik_interface_generic",
-                            "oid": "IF-MIB::ifTable",
-                            "field": [
-                                # 6: ethernetCsmacd (physischer Ethernet-Port)
-                                # 24: softwareLoopback
-                                # 53: propVirtual (oft VLANs bei MikroTik)
-                                # 131: tunnel
-                                # 135: l2vlan
-                                # 161: ieee8023adLag (Bonding/LACP)
-                                # 209: bridge
-                                {
-                                    "name": "ifType",
-                                    "oid": "IF-MIB::ifType",
-                                    "is_tag": True,
-                                },
-
-                                # Labels (optional but recommended)
-                                {
-                                    "name": "ifName",
-                                    "oid": "IF-MIB::ifName",
-                                    "is_tag": True,
-                                },
-                                {
-                                    "name": "ifAlias",
-                                    "oid": "IF-MIB::ifAlias",
-                                    "is_tag": True,
-                                },
-
-                                # Bytes (64-bit)
-                                {
-                                    "name": "in_octets",
-                                    "oid": "IF-MIB::ifHCInOctets",
-                                },
-                                {
-                                    "name": "out_octets",
-                                    "oid": "IF-MIB::ifHCOutOctets",
-                                },
-
-                                # Packets (64-bit unicast)
-                                {
-                                    "name": "in_ucast_pkts",
-                                    "oid": "IF-MIB::ifHCInUcastPkts",
-                                },
-                                {
-                                    "name": "out_ucast_pkts",
-                                    "oid": "IF-MIB::ifHCOutUcastPkts",
-                                },
-                                {
-                                    "name": "in_mcast_pkts",
-                                    "oid": "IF-MIB::ifHCInMulticastPkts",
-                                },
-                                {
-                                    "name": "in_bcast_pkts",
-                                    "oid": "IF-MIB::ifHCInBroadcastPkts",
-                                },
-                                {
-                                    "name": "out_mcast_pkts",
-                                    "oid": "IF-MIB::ifHCOutMulticastPkts",
-                                },
-                                {
-                                    "name": "out_bcast_pkts",
-                                    "oid": "IF-MIB::ifHCOutBroadcastPkts",
-                                },
-
-                                # Drops / Errors
-                                {
-                                    "name": "in_discards",
-                                    "oid": "IF-MIB::ifInDiscards",
-                                },
-                                {
-                                    "name": "out_discards",
-                                    "oid": "IF-MIB::ifOutDiscards",
-                                },
-                                {
-                                    "name": "in_errors",
-                                    "oid": "IF-MIB::ifInErrors",
-                                },
-                                {
-                                    "name": "out_errors",
-                                    "oid": "IF-MIB::ifOutErrors",
-                                },
-                            ],
-                        },
-                        # Interface PoE
-                        {
-                            "name": "mikrotik_poe",
-                            "oid": "MIKROTIK-MIB::mtxrPOETable",
-                            "field": [
-                                {
-                                    "name": "ifName",
-                                    "oid": "IF-MIB::ifName",
-                                    "is_tag": True,
-                                },
-                                {
-                                    "name": "ifAlias",
-                                    "oid": "IF-MIB::ifAlias",
-                                    "is_tag": True,
-                                },
-                                {
-                                    "name": "ifindex",
-                                    "oid": "MIKROTIK-MIB::mtxrPOEInterfaceIndex",
-                                    "is_tag": True,
-                                },
-                                {
-                                    "name": "status",
-                                    "oid": "MIKROTIK-MIB::mtxrPOEStatus",
-                                },
-                                {
-                                    "name": "voltage",
-                                    "oid": "MIKROTIK-MIB::mtxrPOEVoltage",
-                                },
-                                {
-                                    "name": "current",
-                                    "oid": "MIKROTIK-MIB::mtxrPOECurrent",
-                                },
-                                {
-                                    "name": "power",
-                                    "oid": "MIKROTIK-MIB::mtxrPOEPower",
-                                },
-                            ],
-                        },
-                    ],
-                },
-                "mikrotik_switches_very_slow": {
-                    "interval": "20m",
-                    "collection_jitter": "5m",
-                    **input_defaults,
-                    "table": [
-                        # Interface statistics (MikroTik-specific mib)
-                        {
-                            "name": "mikrotik_interface_detailed",
-                            "oid": "MIKROTIK-MIB::mtxrInterfaceStatsTable",
-                            "field": [
-                                # Join key / label (usually identical to IF-MIB ifName)
-                                {
-                                    "name": "ifName",
-                                    "oid": "MIKROTIK-MIB::mtxrInterfaceStatsName",
-                                    "is_tag": True,
-                                },
-
-                                # join IF-MIB for better labels
-                                {
-                                    "name": "ifAlias",
-                                    "oid": "IF-MIB::ifAlias",
-                                    "is_tag": True,
-                                },
-
-                                # =========================
-                                # Physical layer (L1/L2)
-                                # =========================
-                                # CRC/FCS errors → very often cabling, connectors, SFPs, signal quality (EMI)
-                                {
-                                    "name": "rx_fcs_errors",
-                                    "oid": "MIKROTIK-MIB::mtxrInterfaceStatsRxFCSError",
-                                },
-                                # Alignment errors → typically duplex mismatch or PHY problems
-                                {
-                                    "name": "rx_align_errors",
-                                    "oid": "MIKROTIK-MIB::mtxrInterfaceStatsRxAlignError",
-                                },
-                                # Code errors → PHY encoding errors (signal/SFP/PHY)
-                                {
-                                    "name": "rx_code_errors",
-                                    "oid": "MIKROTIK-MIB::mtxrInterfaceStatsRxCodeError",
-                                },
-                                # Carrier errors → carrier lost (copper issues, autoneg, PHY instability)
-                                {
-                                    "name": "rx_carrier_errors",
-                                    "oid": "MIKROTIK-MIB::mtxrInterfaceStatsRxCarrierError",
-                                },
-                                # Jabber → extremely long invalid frames (faulty NIC/PHY, very severe)
-                                {
-                                    "name": "rx_jabber",
-                                    "oid": "MIKROTIK-MIB::mtxrInterfaceStatsRxJabber",
-                                },
-
-                                # ==================================
-                                # Length / framing anomalies (diagnostic)
-                                # ==================================
-                                # Frames shorter than minimum (noise, collisions, broken sender)
-                                {
-                                    "name": "rx_too_short",
-                                    "oid": "MIKROTIK-MIB::mtxrInterfaceStatsRxTooShort",
-                                },
-                                # Frames longer than allowed (MTU mismatch, framing errors)
-                                {
-                                    "name": "rx_too_long",
-                                    "oid": "MIKROTIK-MIB::mtxrInterfaceStatsRxTooLong",
-                                },
-                                # Fragments (often collision-related or duplex mismatch)
-                                {
-                                    "name": "rx_fragment",
-                                    "oid": "MIKROTIK-MIB::mtxrInterfaceStatsRxFragment",
-                                },
-                                # Generic length errors
-                                {
-                                    "name": "rx_length_errors",
-                                    "oid": "MIKROTIK-MIB::mtxrInterfaceStatsRxLengthError",
-                                },
-
-                                # ==================
-                                # Drops (real packet loss)
-                                # ==================
-                                # RX drops (queue/ASIC/policy/overload) → highly alert-worthy
-                                {
-                                    "name": "rx_drop",
-                                    "oid": "MIKROTIK-MIB::mtxrInterfaceStatsRxDrop",
-                                },
-                                # TX drops (buffer/queue exhaustion, scheduling, ASIC limits)
-                                {
-                                    "name": "tx_drop",
-                                    "oid": "MIKROTIK-MIB::mtxrInterfaceStatsTxDrop",
-                                },
-
-                                # =========================================
-                                # Duplex / collision indicators
-                                # (should be zero on full-duplex links)
-                                # =========================================
-                                # Total collisions (relevant only for half-duplex or misconfigurations)
-                                {
-                                    "name": "tx_collisions",
-                                    "oid": "MIKROTIK-MIB::mtxrInterfaceStatsTxCollision",
-                                },
-                                # Late collisions → almost always duplex mismatch / bad autoneg
-                                {
-                                    "name": "tx_late_collisions",
-                                    "oid": "MIKROTIK-MIB::mtxrInterfaceStatsTxLateCollision",
-                                },
-                                # Aggregate collision counter (context)
-                                {
-                                    "name": "tx_total_collisions",
-                                    "oid": "MIKROTIK-MIB::mtxrInterfaceStatsTxTotalCollision",
-                                },
-                                # Excessive collisions → persistent duplex problems
-                                {
-                                    "name": "tx_excessive_collisions",
-                                    "oid": "MIKROTIK-MIB::mtxrInterfaceStatsTxExcessiveCollision",
-                                },
-
-                                # ==================
-                                # Flow control (diagnostic)
-                                # ==================
-                                # Pause frames received (peer throttling you)
-                                {
-                                    "name": "rx_pause",
-                                    "oid": "MIKROTIK-MIB::mtxrInterfaceStatsRxPause",
-                                },
-                                # Pause frames sent (you throttling the peer)
-                                {
-                                    "name": "tx_pause",
-                                    "oid": "MIKROTIK-MIB::mtxrInterfaceStatsTxPause",
-                                },
-                                # Pause frames actually honored
-                                {
-                                    "name": "tx_pause_honored",
-                                    "oid": "MIKROTIK-MIB::mtxrInterfaceStatsTxPauseHonored",
-                                },
-
-                                # ==========
-                                # Stability
-                                # ==========
-                                # Link-down events (loose cables, bad SFPs, PoE power drops, reboots)
-                                {
-                                    "name": "link_downs",
-                                    "oid": "MIKROTIK-MIB::mtxrInterfaceStatsLinkDowns",
-                                },
-                            ],
-                        },
-                    ],
-                },
-            },
-        },
-    },
-}
-
-
-# @metadata_reactor.provides(
-#     'telegraf/processors/enum',
-# )
-# def tag_important_ports(metadata):
-#     # We want a graph, only for important ports. We deem ports important, if they have untagged vlans configured.
-#     return {
-#         "telegraf": {
-#             "processors": {
-#                 "enum": {
-#                     f"mikrotik_port_mapping_{routeros_node.name}":{
-#                         "tagpass": {
-#                             "agent_host": [routeros_node.hostname],
-#                         },
-#                         "mapping": [
-#                             {
-#                                 "tag": "ifName",
-#                                 "dest": "is_infra",
-#                                 "default": "false",
-#                                 "value_mappings": {
-#                                     port_name: "true"
-#                                         for port_name, port_conf in repo.libs.mikrotik.get_netbox_config_for(routeros_node)['interfaces'].items()
-#                                         if port_conf['mode'] == "tagged-all" or port_conf['tagged_vlans']
-#                                         if port_conf['type'] != "lag"
-#                                 },
-#                             },
-#                         ],
-#                     }
-#                         for routeros_node in repo.nodes_in_group("switches-mikrotik")
-#                 },
-#             },
-#         },
-#     }

bundles/routeros/items.py

@@ -27,15 +27,15 @@ routeros['/system/identity'] = {
 # for topic in LOGGING_TOPICS:
 #     routeros[f'/system/logging?action=memory&topics={topic}'] = {}
 
-routeros['/snmp'] = {
-    'enabled': True,
-}
-routeros['/snmp/community?name=public'] = {
-    'addresses': '0.0.0.0/0',
-    'disabled': False,
-    'read-access': True,
-    'write-access': False,
-}
+# routeros['/snmp'] = {
+#     'enabled': True,
+# }
+# routeros['/snmp/community?name=public'] = {
+#     'addresses': '0.0.0.0/0',
+#     'disabled': False,
+#     'read-access': True,
+#     'write-access': False,
+# }
 
 routeros['/system/clock'] = {
     'time-zone-autodetect': False,
@@ -55,7 +55,7 @@ for vlan_name, vlan_id in node.metadata.get('routeros/vlans').items():
         'vlan-id': vlan_id,
         'interface': 'bridge',
         'tags': {
-            'routeros-vlans',
+            'routeros-vlan',
         },
     }
 
@@ -68,33 +68,10 @@ for vlan_name, vlan_id in node.metadata.get('routeros/vlans').items():
             'routeros-vlan-ports',
         },
         'needs': {
-            'tag:routeros-vlans',
+            'tag:routeros-vlan',
         },
     }
 
-for port_name, port_conf in node.metadata.get('routeros/ports').items():
-    untagged_vlan = node.metadata.get('routeros/vlan_groups')[port_conf.get('vlan_group')]['untagged']
-
-    routeros[f'/interface/bridge/port?interface={port_name}'] = {
-        'disabled': False,
-        'bridge': 'bridge',
-        'pvid': node.metadata.get('routeros/vlans')[untagged_vlan],
-        'tags': {
-            'routeros-ports'
-        },
-        'needs': {
-            'tag:routeros-vlan-ports',
-        },
-    }
-
-    routeros[f'/interface?name={port_name}'] = {
-        '_comment': port_conf.get('description', ''),
-    }
-
-    if comment := port_conf.get('comment', None):
-        routeros[f'/interface/bridge/port?interface={port_name}']['_comment'] = comment
-        routeros[f'/interface?name={port_name}']['_comment'] = comment
-
 # create IPs
 for ip, ip_conf in node.metadata.get('routeros/ips').items():
     routeros[f'/ip/address?address={ip}'] = {
@@ -103,8 +80,7 @@ for ip, ip_conf in node.metadata.get('routeros/ips').items():
             'routeros-ip',
         },
         'needs': {
-            'tag:routeros-vlans',
-            'tag:routeros-ports'
+            'tag:routeros-vlan',
         },
     }
 
@@ -114,8 +90,7 @@ routeros['/interface/bridge?name=bridge'] = {
     'priority': node.metadata.get('routeros/bridge_priority'),
     'protocol-mode': 'rstp',
     'needs': {
-        'tag:routeros-vlans',
-        'tag:routeros-ports',
+        'tag:routeros-vlan',
         'tag:routeros-vlan-ports',
         'tag:routeros-ip',
     },
@@ -127,7 +102,7 @@ routeros['/interface/vlan'] = {
         'id-by': 'name',
     },
     'needed_by': {
-        'tag:routeros-vlans',
+        'tag:routeros-vlan',
     }
 }
 
@@ -139,6 +114,6 @@ routeros['/interface/bridge/vlan'] = {
         },
     },
     'needed_by': {
-        'tag:routeros-vlans',
+        'tag:routeros-vlan',
     }
 }

bundles/smartctl/metadata.py

@@ -11,22 +11,24 @@ defaults = {
     },
     'smartctl': {},
     'telegraf': {
-        'inputs': {
-            'exec': {
-                'smartctl_power_mode': {
-                    'commands': [
-                        f'sudo /usr/local/share/telegraf/smartctl_power_mode',
-                    ],
-                    'data_format': 'influx',
-                    'interval': '20s',
+        'config': {
+            'inputs': {
+                'exec': {
+                    h({
+                        'commands': [
+                            f'sudo /usr/local/share/telegraf/smartctl_power_mode',
+                        ],
+                        'data_format': 'influx',
+                        'interval': '20s',
+                    }),
+                    h({
+                        'commands': [
+                            f'sudo /usr/local/share/telegraf/smartctl_errors',
+                        ],
+                        'data_format': 'influx',
+                        'interval': '6h',
+                    })
                 },
-                'smartctl_errors': {
-                    'commands': [
-                        f'sudo /usr/local/share/telegraf/smartctl_errors',
-                    ],
-                    'data_format': 'influx',
-                    'interval': '6h',
-                }
             },
         },
     },

bundles/ssh/items.py

@@ -10,11 +10,7 @@ directories = {
         'purge': True,
         'mode': '0755',
         'skip': dont_touch_sshd,
-    },
-    '/etc/ssh/ssh_config.d': {
-        'mode': '0755',
-        'skip': dont_touch_sshd,
-    },
+    }
 }
 
 files = {

bundles/ssh/metadata.py

@@ -19,7 +19,7 @@ def users(metadata):
             'allow_users': set(
                 name
                     for name, conf in metadata.get('users').items()
-                    if conf.get('authorized_keys', []) or conf.get('authorized_users', {})
+                    if conf.get('authorized_keys', []) or conf.get('authorized_users', [])
             ),
         },
     }

bundles/systemd-timers/metadata.py

@@ -44,8 +44,6 @@ def systemd(metadata):
             units[f'{name}.service']['Service']['KillMode'] = config['kill_mode']
         if config.get('RuntimeMaxSec'):
             units[f'{name}.service']['Service']['RuntimeMaxSec'] = config['RuntimeMaxSec']
-        if config.get('environment_files'):
-            units[f'{name}.service']['Service']['EnvironmentFile'] = config['environment_files']
 
         services[f'{name}.timer'] = {}
 

bundles/systemd/items.py

@@ -1,8 +1,5 @@
 from bundlewrap.utils.dicts import merge_dict
 
-files = {}
-svc_systemd = {}
-
 directories = {
     '/usr/local/lib/systemd/system': {
         'purge': True,
@@ -33,7 +30,7 @@ for name, unit in node.metadata.get('systemd/units').items():
                 'svc_systemd:systemd-networkd.service:restart',
             ],
         }
-    elif extension in ['timer', 'service', 'mount', 'swap', 'target', 'slice']:
+    elif extension in ['timer', 'service', 'mount', 'swap', 'target']:
         path = f'/usr/local/lib/systemd/system/{name}'
         dependencies = {
             'triggers': [
@@ -45,9 +42,6 @@ for name, unit in node.metadata.get('systemd/units').items():
     else:
         raise Exception(f'unknown type {extension}')
 
-    for attribute in ['needs', 'needed_by', 'triggers', 'triggered_by']:
-        if attribute in unit:
-            dependencies.setdefault(attribute, []).extend(unit.pop(attribute))
 
     files[path] = {
         'content': repo.libs.systemd.generate_unitfile(unit),

bundles/tasmota-charge/metadata.py

@@ -15,7 +15,7 @@ defaults = {
 
 
 @metadata_reactor.provides(
-    'telegraf/inputs/exec',
+    'telegraf/config/inputs/exec',
 )
 def telegraf(metadata):
     return {
@@ -23,11 +23,11 @@ def telegraf(metadata):
             'config': {
                 'inputs': {
                     'exec': {
-                        'tasmota_charge': {
+                        repo.libs.hashable.hashable({
                             'commands': ["/usr/local/share/telegraf/tasmota_charge"],
                             'name_override': "tasmota_charge",
                             'data_format': "influx",
-                        },
+                        }),
                     },
                 },
             },

bundles/telegraf/items.py

@@ -1,53 +1,19 @@
 import tomlkit
-
-
-def inner_dict_to_list(dict_of_dicts):
-    """
-    Example:
-    {
-        'cpu': {
-            'default': {'something': True},
-            'another': {'something': False},
-        },
-    }
-    becomes
-    {
-        'cpu': [
-            {'something': True},
-            {'something': False},
-        ],
-    }
-    """
-    return {
-        key: [value for _, value in sorted(dicts.items())]
-            for key, dicts in sorted(dict_of_dicts.items())
-    }
-
+import json
+from bundlewrap.metadata import MetadataJSONEncoder
 
 files = {
-    "/etc/telegraf/telegraf.conf": {
-        'owner': 'telegraf',
-        'group': 'telegraf',
-        'mode': '0440',
-        'needs': [
-            "pkg_apt:telegraf",
+    '/etc/telegraf/telegraf.conf': {
+        'content': tomlkit.dumps(
+            json.loads(json.dumps(
+                node.metadata.get('telegraf/config'),
+                cls=MetadataJSONEncoder,
+            )),
+            sort_keys=True,
+        ),
+        'triggers': [
+            'svc_systemd:telegraf:restart',
         ],
-        'content': tomlkit.dumps({
-            'agent': node.metadata.get('telegraf/agent'),
-            'inputs': inner_dict_to_list(node.metadata.get('telegraf/inputs')),
-            'processors': inner_dict_to_list(node.metadata.get('telegraf/processors')),
-            'outputs': inner_dict_to_list(node.metadata.get('telegraf/outputs')),
-        }),
-        'triggers': {
-            'svc_systemd:telegraf.service:restart',
-        },
-    },
-    '/etc/default/telegraf': {
-        'content': 'TELEGRAF_OPTS="--strict-env-handling"\n',
-        'mode': '0644',
-        'triggers': {
-            'svc_systemd:telegraf.service:restart',
-        },
     },
     '/usr/local/share/telegraf/procio': {
         'content_type': 'download',
@@ -61,26 +27,9 @@ files = {
     },
 }
 
-actions = {
-    'telegraf-test-config': {
-        'command': "sudo -u telegraf bash -c 'telegraf config check --config /etc/telegraf/telegraf.conf --strict-env-handling'",
-        'triggered': True,
-        'needs': [
-            'bundle:sudo',
-            'file:/etc/telegraf/telegraf.conf',
-            'pkg_apt:telegraf',
-        ],
-    },
-}
-
-svc_systemd = {
-    'telegraf.service': {
-        'needs': ['pkg_apt:telegraf'],
-        'preceded_by': {
-            'action:telegraf-test-config',
-        },
-        'needs': {
-            'action:telegraf-test-config',
-        },
-    },
+svc_systemd['telegraf'] = {
+    'needs': [
+        'file:/etc/telegraf/telegraf.conf',
+        'pkg_apt:telegraf',
+    ],
 }

bundles/telegraf/metadata.py

@@ -7,11 +7,6 @@ defaults = {
             # needed by crystal plugins:
             'libgc-dev': {},
             'libevent-dev': {},
-            (
-                'libpcre2-8-0'
-                if node.os == 'debian' and node.os_version >= (13,)
-                else 'libpcre3'
-            ): {},
         },
         'sources': {
             'influxdata': {
@@ -28,29 +23,26 @@ defaults = {
         },
     },
     'telegraf': {
-        'agent': {
-            'hostname': node.name,
-            'collection_jitter': '20s',
-            'flush_interval': '20s',
-            'flush_jitter': '5s',
-            'interval': '2m',
-            'metric_batch_size': 1000,
-            'metric_buffer_limit': 10000,
-            'omit_hostname': False,
-            'round_interval': True,
-            'skip_processors_after_aggregators': True,
-        },
-        'inputs': {
-            'cpu': {
-                'default': {
+        'config': {
+            'agent': {
+                'hostname': node.name,
+                'collection_jitter': '0s',
+                'flush_interval': '15s',
+                'flush_jitter': '0s',
+                'interval': '15s',
+                'metric_batch_size': 1000,
+                'metric_buffer_limit': 10000,
+                'omit_hostname': False,
+                'round_interval': True,
+            },
+            'inputs': {
+                'cpu': {h({
                     'collect_cpu_time': False,
                     'percpu': True,
                     'report_active': False,
                     'totalcpu': True,
-                },
-            },
-            'disk': {
-                'default': {
+                })},
+                'disk': {h({
                     'ignore_fs': [
                         'tmpfs',
                         'devtmpfs',
@@ -60,60 +52,42 @@ defaults = {
                         'aufs',
                         'squashfs',
                     ],
-                }
-            },
-            'procstat': {
-                'default': {
+                })},
+                'procstat': {h({
                     'interval': '60s',
                     'pattern': '.',
                     'fieldinclude': [
                         'cpu_usage',
                         'memory_rss',
                     ],
-                },
-            },
-            'diskio': {
-                'default': {
+                })},
+                'diskio': {h({
                     'device_tags': ["ID_PART_ENTRY_NUMBER"],
-                }
-            },
-            'kernel': {
-                'default': {},
-            },
-            'mem': {
-                'default': {},
-            },
-            'processes': {
-                'default': {},
-            },
-            'swap': {
-                'default': {},
-            },
-            'system': {
-                'default': {},
-            },
-            'net': {
-                'default': {},
-            },
-            'exec': {
-                # h({
-                #     'commands': [
-                #         f'sudo /usr/local/share/telegraf/procio',
-                #     ],
-                #     'data_format': 'influx',
-                #     'interval': '20s',
-                # }),
-                'pressure_stall': {
-                    'commands': [
-                        f'/usr/local/share/telegraf/pressure_stall',
-                    ],
-                    'data_format': 'influx',
-                    'interval': '10s',
+                })},
+                'kernel': {h({})},
+                'mem': {h({})},
+                'processes': {h({})},
+                'swap': {h({})},
+                'system': {h({})},
+                'net': {h({})},
+                'exec': {
+                    h({
+                        'commands': [
+                            f'sudo /usr/local/share/telegraf/procio',
+                        ],
+                        'data_format': 'influx',
+                        'interval': '20s',
+                    }),
+                    h({
+                        'commands': [
+                            f'/usr/local/share/telegraf/pressure_stall',
+                        ],
+                        'data_format': 'influx',
+                        'interval': '10s',
+                    }),
                 },
             },
         },
-        'processors': {},
-        'outputs': {},
     },
     'grafana_rows': {
         'cpu',
@@ -131,21 +105,21 @@ defaults = {
 
 
 @metadata_reactor.provides(
-    'telegraf/outputs/influxdb_v2/default',
+    'telegraf/config/outputs/influxdb_v2',
 )
 def influxdb(metadata):
     influxdb_metadata = repo.get_node(metadata.get('telegraf/influxdb_node')).metadata.get('influxdb')
 
     return {
         'telegraf': {
-            'outputs': {
-                'influxdb_v2': {
-                    'default': {
+            'config': {
+                'outputs': {
+                    'influxdb_v2': [{
                         'urls': [f"http://{influxdb_metadata['hostname']}:{influxdb_metadata['port']}"],
                         'token': str(influxdb_metadata['writeonly_token']),
                         'organization': influxdb_metadata['org'],
                         'bucket': influxdb_metadata['bucket'],
-                    },
+                    }]
                 },
             },
         },
@@ -153,3 +127,20 @@ def influxdb(metadata):
 
 
 # crystal based (procio, pressure_stall):
+@metadata_reactor.provides(
+    'apt/packages/libpcre2-8-0',
+    'apt/packages/libpcre3',
+)
+def libpcre(metadata):
+    if node.os == 'debian' and node.os_version >= (13,):
+        libpcre_package = 'libpcre2-8-0'
+    else:
+        libpcre_package = 'libpcre3'
+
+    return {
+        'apt': {
+            'packages': {
+                libpcre_package: {},
+            },
+        },
+    }

bundles/users/metadata.py

@@ -20,15 +20,11 @@ def authorized_users(metadata):
         users[name] = {
             'authorized_keys': set(),
         }
-        for authorized_user, options in config.get('authorized_users', {}).items():
+        for authorized_user in config.get('authorized_users', set()):
             authorized_user_name, authorized_user_node = authorized_user.split('@')
-            authorized_user_public_key = repo.get_node(authorized_user_node).metadata.get(f'users/{authorized_user_name}/pubkey')
-
-            for command in options.get('commands', []):
-                users[name]['authorized_keys'].add(f'command="{command}" ' + authorized_user_public_key)
-            else:
-                users[name]['authorized_keys'].add(authorized_user_public_key)
-
+            users[name]['authorized_keys'].add(
+                repo.get_node(authorized_user_node).metadata.get(f'users/{authorized_user_name}/pubkey')
+            )
     return {
         'users': users,
     }

bundles/wireguard/metadata.py

@@ -112,18 +112,20 @@ def systemd_networkd_netdevs(metadata):
         },
     }
 
-    for kind in ('s2s', 'clients'):
-        for peer in metadata.get(f'wireguard/{kind}'):
-            peer_id = metadata.get(f'wireguard/{kind}/{peer}/peer_id')
-            netdev[f'WireGuardPeer#{peer}'] = {
-                'PublicKey': repo.libs.wireguard.pubkey(peer_id),
-                'PresharedKey': repo.libs.wireguard.psk(peer_id, metadata.get('id')),
-                'AllowedIPs': ', '.join(metadata.get(f'wireguard/{kind}/{peer}/allowed_ips', [])),
+    for peer, config in {
+        **metadata.get('wireguard/s2s'),
+        **metadata.get('wireguard/clients'),
+    }.items():
+        netdev.update({
+            f'WireGuardPeer#{peer}': {
+                'PublicKey': repo.libs.wireguard.pubkey(config['peer_id']),
+                'PresharedKey': repo.libs.wireguard.psk(config['peer_id'], metadata.get('id')),
+                'AllowedIPs': ', '.join(config.get('allowed_ips', [])),
                 'PersistentKeepalive': 30,
             }
-            endpoint = metadata.get(f'wireguard/{kind}/{peer}/endpoint', None)
-            if endpoint:
-                netdev[f'WireGuardPeer#{peer}']['Endpoint'] = endpoint
+        })
+        if config.get('endpoint'):
+            netdev[f'WireGuardPeer#{peer}']['Endpoint'] = config['endpoint']
 
     return {
         'systemd': {

bundles/wol-sleeper/metadata.py

@@ -44,21 +44,16 @@ defaults = {
 
 
 @metadata_reactor.provides(
-    'wol-sleeper/mac',
-    'wol-sleeper/waker_command',
     'wol-sleeper/wake_command',
 )
 def wake_command(metadata):
     waker_hostname = repo.get_node(metadata.get('wol-sleeper/waker')).hostname
     mac = metadata.get(f"network/{metadata.get('wol-sleeper/network')}/mac")
-    network = ip_interface(metadata.get(f"network/{metadata.get('wol-sleeper/network')}/ipv4"))
-    waker_command = f"/usr/bin/wakeonlan -i {network.network.broadcast_address} {mac}"
+    ip = ip_interface(metadata.get(f"network/{metadata.get('wol-sleeper/network')}/ipv4")).ip
 
     return {
         'wol-sleeper': {
-            'mac': mac,
-            'waker_command': waker_command,
-            'wake_command': f"ssh -o StrictHostKeyChecking=no wol@{waker_hostname} '{waker_command}' && while ! ping {network.ip} -c1 -W3; do true; done",
+            'wake_command': f"ssh -o StrictHostKeyChecking=no wol@{waker_hostname} 'wakeonlan {mac} && while ! ping {ip} -c1 -W3; do true; done'",
         },
     }
 

bundles/wol-waker/metadata.py

@@ -6,25 +6,17 @@ defaults = {
     },
 }
 
-
 @metadata_reactor.provides(
-    'users/wol/authorized_users',
+    'users/wol',
 )
 def user(metadata):
     return {
         'users': {
             'wol': {
                 'authorized_users': {
-                    f'root@{ssh_client.name}': {
-                        'commands': {
-                            sleeper.metadata.get('wol-sleeper/waker_command')
-                                for sleeper in repo.nodes
-                                if sleeper.has_bundle('wol-sleeper')
-                                and sleeper.metadata.get('wol-sleeper/waker') == node.name
-                        }
-                    }
-                        for ssh_client in repo.nodes
-                        if ssh_client.dummy == False and ssh_client.has_bundle('ssh')
+                    f'root@{node.name}'
+                        for node in repo.nodes
+                        if node.dummy == False and node.has_bundle('ssh')
                 },
             },
         },

bundles/zfs/metadata.py

@@ -59,9 +59,9 @@ defaults = {
         },
     },
     'telegraf': {
-        'inputs': {
-            'zfs': {
-                'default': {},
+        'config': {
+            'inputs': {
+                'zfs': [{}],
             },
         },
     },