docs(geode): correct OK ordering/durability assumptions in ingestion plan

OK frames carry the event id, so clients pair replies by id rather
than by arrival order. NIP-01 also treats OK true as "accepted," not
"fsynced." That removes two constraints the plan was carrying:

- no per-connection FIFO requirement on OKs
- no need to delay OKs until after batch fsync

Tier 1 can fan OKs out as soon as the per-row INSERT returns inside
the open transaction, hiding the group-commit fsync entirely from
publisher latency. Tier 2 drops the order-preserving commit log and
just sends OKs straight to outQueue. The pipelined benchmark now
checks one-OK-per-event-id rather than ordering.
This commit is contained in:
Claude
2026-05-07 21:35:49 +00:00
parent 1bee982e34
commit 39e81c1eb5
@@ -19,14 +19,20 @@ contention, not WS throughput).
## Constraints we must keep ## Constraints we must keep
- **OK ordering**: NIP-01 requires the OK reply to follow its EVENT. - **OK pairs by event id, not by order**: the OK frame carries the
We cannot reply OK before the insert decision (the OK carries event id, so clients pair replies to publishes by id. OKs can be
accepted/rejected + reason). emitted in any order — including reordered against the EVENT
- **Durability semantics**: clients reasonably assume `OK true` means stream, and against each other on the same connection. This frees
"stored." Batching must not make us reply OK before fsync. us to fan OKs out as soon as the writer has a per-row decision.
- **Per-connection FIFO**: a publisher that sends three EVENTs in a - **OK semantics = accepted, not fsynced**: NIP-01 treats `OK true`
row expects three OKs in that order. Reordering across connections as "accepted by the relay," not "durably on disk." We can reply as
is fine. soon as SQLite returns success for the row (inside the open
transaction, before commit/fsync). Group commit can batch the
fsync without holding OKs back.
- **Per-row decision still required**: the OK reason field is
per-event (duplicate, blocked, invalid sig, pow, etc.), so we
cannot fan out a single batch-level OK. Each row's OK must reflect
that row's outcome.
## Sketch ## Sketch
@@ -36,7 +42,12 @@ Confirm `PRAGMA journal_mode=WAL` + `PRAGMA synchronous=NORMAL` on the
event-store DB; group commits across the writer mutex's hold window. event-store DB; group commits across the writer mutex's hold window.
Today each insert is its own transaction. Wrap N inserts (or a 5 ms Today each insert is its own transaction. Wrap N inserts (or a 5 ms
budget, whichever first) in a single transaction managed by the writer budget, whichever first) in a single transaction managed by the writer
coroutine. On commit, fan back N OK replies. coroutine.
Because OK reflects acceptance not durability, each row can fan an OK
as soon as the per-row INSERT statement returns inside the
transaction — we do not need to wait for the batch's commit. The
fsync is hidden from the publisher latency budget entirely.
Implementation lives in quartz's `EventStore` / `SQLiteConnectionPool`, Implementation lives in quartz's `EventStore` / `SQLiteConnectionPool`,
not geode — but geode owns the benchmark and validates the gain. not geode — but geode owns the benchmark and validates the gain.
@@ -48,17 +59,18 @@ commit is well-trodden territory (nostr-rs-relay, strfry both do it).
`RelaySession.receive` is currently single-flight: one EVENT in, `RelaySession.receive` is currently single-flight: one EVENT in,
process, OK out, next EVENT. Allow a connection to push N EVENTs process, OK out, next EVENT. Allow a connection to push N EVENTs
concurrently, dispatch them to a per-connection ingest pipeline, and concurrently and dispatch them to a per-connection ingest pipeline.
serialise OKs back in arrival order via a small commit log.
A `Channel<EventCmd> with capacity = INGEST_PIPELINE_DEPTH` per A `Channel<EventCmd>` with capacity = `INGEST_PIPELINE_DEPTH` per
connection, drained by a coroutine that batches into the group-commit connection, drained by a coroutine that feeds the group-commit writer
above. OK responses are written to an `outQueue.send()` already — so above. OKs go straight to `outQueue.send()` the moment each row
the pipeline just needs to record arrival order and emit OKs in that returns from INSERT — no ordering bookkeeping needed, since the OK
order after each batch commits. frame already carries the event id and the spec doesn't require
order. A pipelined publisher keying on event id will pair replies
correctly.
Expected: hides the verify+insert latency behind another EVENT's Expected: hides verify+insert latency behind the next EVENT's parse,
parse, gets us closer to network-bound throughput. gets us closer to network-bound throughput.
### Tier 3 — eager Schnorr verify off the writer thread ### Tier 3 — eager Schnorr verify off the writer thread
@@ -74,7 +86,8 @@ Add to `geode.perf.LoadBenchmark`:
- `publishGroupCommitSingleClient` — same workload as the current - `publishGroupCommitSingleClient` — same workload as the current
single-client benchmark, asserts >5000 EPS. single-client benchmark, asserts >5000 EPS.
- `publishPipelinedSingleClient` — sends 100 EVENTs without awaiting - `publishPipelinedSingleClient` — sends 100 EVENTs without awaiting
intermediate OKs; measures end-to-end and OK-ordering correctness. intermediate OKs; measures end-to-end throughput and verifies that
every event id receives exactly one OK (in any order).
Existing benchmarks stay as the regression floor. Existing benchmarks stay as the regression floor.