Setup & usage

Standing up the hookstrd daemon on always-online infra, and running the hookstr drain on the consumer. For the animated walk-through of a single delivery's lifetime, see the overview.

01Concepts & roles

Two binaries, one immutable event per delivery.

hookstr splits webhook delivery from webhook consumption so a flaky consumer never makes a provider give up. There are two halves:

Each delivery becomes one immutable, signed event: the raw body as content (base64 + an encoding tag when it isn't UTF-8), the ingest path in a path tag for replay routing, the provider name in an indexed t tag, and one header tag per preserved header. Nothing is ever deleted — the store is the source of truth, and negentropy (NIP-77) reconciles over the full set, which is what makes "catch up after being offline for a week" a non-event.

02Build the binaries

One Rust workspace; two binaries fall out of it.

$ git clone https://github.com/jb55/hookstr
$ cd hookstr
$ cargo build --release
# → target/release/hookstrd   (the daemon)
# → target/release/hookstr    (the drain CLI)

Copy hookstrd to the server and hookstr to the consumer. They share no state and never talk except over the relay's websocket, through the tunnel. cargo test --workspace runs the full end-to-end suite (real daemon + real drain, in-process) with no external services.

03Server: hookstrd

The always-online half. Do this on the VPS.

1 — Generate the server keypair

keygen is on the CLI binary, but the key it prints can be used for either side. Make one for the server and write the nsec to a file (config never holds the secret inline — only a path to it):

$ hookstr keygen
nsec:   nsec1qy2…
pubkey: 8f2a…

$ umask 077
$ hookstr keygen | awk '/nsec/{print $2}' > /var/lib/hookstr/server.nsec

2 — Write hookstrd.toml

Every key but relay_addr is required. See §09 for the full table.

db_path      = "/var/lib/hookstr/ndb"
mapsize      = 34359738368            # 32 GiB — store-everything, size it generously
ingest_addr  = "127.0.0.1:8080"       # axum ingest (the proxy fronts this)
relay_addr   = "127.0.0.1:8081"       # relay ws, localhost only (the default); NEVER proxy it
nsec_path    = "/var/lib/hookstr/server.nsec"

# Per-provider ingest secrets. Keep this table LAST: a TOML table swallows
# every key beneath it. Each value must be unique — the token names its provider.
[ingest_tokens]
acme   = "long-random-secret-1"
stripe = "long-random-secret-2"

3 — Run it

$ hookstrd --config /etc/hookstr/hookstrd.toml
hookstrd ingest on 127.0.0.1:8080, relay on ws://127.0.0.1:8081

--config defaults to hookstrd.toml in the working directory. Set RUST_LOG=info (or debug) for logs. A minimal systemd unit:

[Service]
ExecStart=/usr/local/bin/hookstrd --config /etc/hookstr/hookstrd.toml
Environment=RUST_LOG=info
Restart=always
User=hookstr

04Reverse proxy & TLS

Only ingest is public.

The proxy terminates TLS and forwards to the axum ingest. Don't route the relay through it. Caddy:

hooks.example.com {
        reverse_proxy localhost:8080   # axum ingest only
}

05Registering providers

One URL per provider; the token is the whole auth story.

For each provider, register a URL of the shape:

https://hooks.example.com/ingest/<token>/<any/path/you/like>

e.g. register …/ingest/long-random-secret-1/acme/events with Acme. On the wire:

• The token alone authenticates the delivery and names its provider (that's why token values must be unique). A wrong token is a 404, indistinguishable from a missing route — probing reveals neither which providers exist nor their paths.
• Everything after the token is free-form. It's echoed into the event's path tag and becomes the replay routing key (§07).
• A successful, durably-stored delivery gets a 204. A 5xx means the store failed — the provider should retry. Any other webhook auth (an x-signature HMAC) is preserved and re-validated by your consumer on replay; hookstrd doesn't check it.

06Client: the drain CLI

The consumer half. Do this on the machine that handles the webhooks.

1 — Open a tunnel to the relay

Any tunnel to the server's relay_addr works. With ssh:

$ ssh -NL 8081:127.0.0.1:8081 hooks.example.com

2 — hookstr init

First-run setup: writes a starter config. Pass the relay URL as the tunnel exposes it (default ws://127.0.0.1:8081):

$ hookstr init ws://127.0.0.1:8081
wrote config /home/you/.config/hookstr/config.toml
then: hookstr drain --target http://localhost:3000/webhooks <provider>

The config lands at $XDG_CONFIG_HOME/hookstr/config.toml (falling back to ~/.config/hookstr/); the local mirror db and replay state go under ~/.local/share/hookstr/. init refuses to overwrite an existing config.

3 — Review the config

The config holds the durable sync identity: relay and local db. Nothing target-related lives here: where the drain delivers is a per-run flag (next step), specific to the consumer it feeds rather than to the sync relationship the config describes.

relay_url = "ws://127.0.0.1:8081"
db_path   = "/home/you/.local/share/hookstr/ndb"
redb_path = "/home/you/.local/share/hookstr/replays.redb"

4 — hookstr drain

$ hookstr drain --target http://localhost:3000/webhooks acme
synced 42 missing webhook event(s)
drain complete: 42 replayed, 0 failed, 0 with no target (42 stored total)
following: handling new webhooks as they arrive

--target is where deliveries replay — a per-run flag, not config, since it's specific to the consumer this drain feeds. That negentropy-reconciles the backlog, replays it oldest-first, and then follows — each new webhook replays in under a second while connected, and a dropped connection reconnects and re-syncs on its own. Useful additions (each still takes --target):

To look without delivering, hookstr peek syncs the same way and prints each undelivered webhook (path, event id, headers, body) instead of POSTing it. It never writes replay state, so a later drain still delivers everything it showed. --follow keeps printing new arrivals and --all includes ones already replayed. A plain peek can't run beside a running drain, since redb lets one process hold the replay state; use peek --all there.

peek --json prints one JSON envelope per line instead: id, path, received_at_ms, provider, headers, body and body_encoding (json, utf8 or base64). These are the lines drain --exec feeds its command. To develop such a consumer, run it under peek --exec <cmd>: the same batches, phases and retries as drain --exec, with acks kept in memory and never written.

A consumer can be a script instead of a server. hookstr drain --exec <cmd>, in place of --target, runs <cmd> under sh -c once per batch. The batch arrives on stdin as those envelope lines, oldest first, with HOOKSTR_PHASE=backfill during catch-up and live while following. Exit 0 acks the whole batch. A non-zero exit or a timeout (--exec-timeout, default 60s) retries the whole batch after backoff. --batch-max (default 500) caps a batch, and --batch-window (default 2s) gathers live arrivals into one. Give an exec drain its own config with its own redb_path and db_path, since replay state is keyed by event id, not by consumer. The README has the full envelope contract.

Replay state (attempt counts, backoff) is client-local in redb, keyed by event id. A target that's down just means backoff and retry — the events are already durable on the server, so nothing is lost. Replay is at-least-once: your handler must dedupe on the provider's delivery-id header, which makes crash-mid-replay safe.

07Replay routing

From ingest path to local URL.

The event's path tag mirrors everything after the token in the ingest URL. By default a delivery replays to {--target}/{path}:

# provider POSTs to:
https://hooks.example.com/ingest/<token>/acme/events

# with --target http://localhost:3000/webhooks, replays to:
http://localhost:3000/webhooks/acme/events

When a consumer's route doesn't mirror the ingest path — a second app on another port, a different route shape — add a [targets] table to the config keyed by the exact path. An explicit override always wins over --target:

[targets]
"acme/events" = "http://localhost:4000/hooks"   # this path only

With some routing configured, a delivery whose path isn't covered — no --target base and no matching override — is left unreplayed (counted as "no target") so a later run or config change can pick it up; it's never dropped. But a drain started with neither a --target nor any [targets] entry has nowhere to deliver anything, so it errors out immediately rather than syncing a backlog it can't route.

08CLI reference

hookstr <command> — the drain binary.

Config defaults to $XDG_CONFIG_HOME/hookstr/config.toml. Both binaries log via tracing; the drain defaults to info, and RUST_LOG overrides either.

09Config reference

Every key in both TOML files. Paths are used verbatim — no ~ expansion.

hookstrd.toml (server)

config.toml (drain)

The replay base (--target) and provider scope (positional args) are not config keys — they're per-run flags on drain.

10Operations

What breaks, and what happens when it does.

Storage grows monotonically — nothing is deleted, by design. Webhook bodies are a few KB each, so even a million deliveries is single-digit GB; watch mapsize headroom the way you'd watch any append-only log. Full design rationale, including the reasoning behind each constraint, is in SPIKE.md.