docs(geode): correct OK ordering/durability assumptions in ingestion plan

OK frames carry the event id, so clients pair replies by id rather
than by arrival order. NIP-01 also treats OK true as "accepted," not
"fsynced." That removes two constraints the plan was carrying:

- no per-connection FIFO requirement on OKs
- no need to delay OKs until after batch fsync

Tier 1 can fan OKs out as soon as the per-row INSERT returns inside
the open transaction, hiding the group-commit fsync entirely from
publisher latency. Tier 2 drops the order-preserving commit log and
just sends OKs straight to outQueue. The pipelined benchmark now
checks one-OK-per-event-id rather than ordering.
This commit is contained in:
Claude
2026-05-07 21:35:49 +00:00
parent 1bee982e34
commit 39e81c1eb5
@@ -19,14 +19,20 @@ contention, not WS throughput).
## Constraints we must keep
- **OK ordering**: NIP-01 requires the OK reply to follow its EVENT.
We cannot reply OK before the insert decision (the OK carries
accepted/rejected + reason).
- **Durability semantics**: clients reasonably assume `OK true` means
"stored." Batching must not make us reply OK before fsync.
- **Per-connection FIFO**: a publisher that sends three EVENTs in a
row expects three OKs in that order. Reordering across connections
is fine.
- **OK pairs by event id, not by order**: the OK frame carries the
event id, so clients pair replies to publishes by id. OKs can be
emitted in any order — including reordered against the EVENT
stream, and against each other on the same connection. This frees
us to fan OKs out as soon as the writer has a per-row decision.
- **OK semantics = accepted, not fsynced**: NIP-01 treats `OK true`
as "accepted by the relay," not "durably on disk." We can reply as
soon as SQLite returns success for the row (inside the open
transaction, before commit/fsync). Group commit can batch the
fsync without holding OKs back.
- **Per-row decision still required**: the OK reason field is
per-event (duplicate, blocked, invalid sig, pow, etc.), so we
cannot fan out a single batch-level OK. Each row's OK must reflect
that row's outcome.
## Sketch
@@ -36,7 +42,12 @@ Confirm `PRAGMA journal_mode=WAL` + `PRAGMA synchronous=NORMAL` on the
event-store DB; group commits across the writer mutex's hold window.
Today each insert is its own transaction. Wrap N inserts (or a 5 ms
budget, whichever first) in a single transaction managed by the writer
coroutine. On commit, fan back N OK replies.
coroutine.
Because OK reflects acceptance not durability, each row can fan an OK
as soon as the per-row INSERT statement returns inside the
transaction — we do not need to wait for the batch's commit. The
fsync is hidden from the publisher latency budget entirely.
Implementation lives in quartz's `EventStore` / `SQLiteConnectionPool`,
not geode — but geode owns the benchmark and validates the gain.
@@ -48,17 +59,18 @@ commit is well-trodden territory (nostr-rs-relay, strfry both do it).
`RelaySession.receive` is currently single-flight: one EVENT in,
process, OK out, next EVENT. Allow a connection to push N EVENTs
concurrently, dispatch them to a per-connection ingest pipeline, and
serialise OKs back in arrival order via a small commit log.
concurrently and dispatch them to a per-connection ingest pipeline.
A `Channel<EventCmd> with capacity = INGEST_PIPELINE_DEPTH` per
connection, drained by a coroutine that batches into the group-commit
above. OK responses are written to an `outQueue.send()` already — so
the pipeline just needs to record arrival order and emit OKs in that
order after each batch commits.
A `Channel<EventCmd>` with capacity = `INGEST_PIPELINE_DEPTH` per
connection, drained by a coroutine that feeds the group-commit writer
above. OKs go straight to `outQueue.send()` the moment each row
returns from INSERT — no ordering bookkeeping needed, since the OK
frame already carries the event id and the spec doesn't require
order. A pipelined publisher keying on event id will pair replies
correctly.
Expected: hides the verify+insert latency behind another EVENT's
parse, gets us closer to network-bound throughput.
Expected: hides verify+insert latency behind the next EVENT's parse,
gets us closer to network-bound throughput.
### Tier 3 — eager Schnorr verify off the writer thread
@@ -74,7 +86,8 @@ Add to `geode.perf.LoadBenchmark`:
- `publishGroupCommitSingleClient` — same workload as the current
single-client benchmark, asserts >5000 EPS.
- `publishPipelinedSingleClient` — sends 100 EVENTs without awaiting
intermediate OKs; measures end-to-end and OK-ordering correctness.
intermediate OKs; measures end-to-end throughput and verifies that
every event id receives exactly one OK (in any order).
Existing benchmarks stay as the regression floor.