docs(geode): correct OK ordering/durability assumptions in ingestion plan
OK frames carry the event id, so clients pair replies by id rather than by arrival order. NIP-01 also treats OK true as "accepted," not "fsynced." That removes two constraints the plan was carrying: - no per-connection FIFO requirement on OKs - no need to delay OKs until after batch fsync Tier 1 can fan OKs out as soon as the per-row INSERT returns inside the open transaction, hiding the group-commit fsync entirely from publisher latency. Tier 2 drops the order-preserving commit log and just sends OKs straight to outQueue. The pipelined benchmark now checks one-OK-per-event-id rather than ordering.
This commit is contained in:
@@ -19,14 +19,20 @@ contention, not WS throughput).
|
|||||||
|
|
||||||
## Constraints we must keep
|
## Constraints we must keep
|
||||||
|
|
||||||
- **OK ordering**: NIP-01 requires the OK reply to follow its EVENT.
|
- **OK pairs by event id, not by order**: the OK frame carries the
|
||||||
We cannot reply OK before the insert decision (the OK carries
|
event id, so clients pair replies to publishes by id. OKs can be
|
||||||
accepted/rejected + reason).
|
emitted in any order — including reordered against the EVENT
|
||||||
- **Durability semantics**: clients reasonably assume `OK true` means
|
stream, and against each other on the same connection. This frees
|
||||||
"stored." Batching must not make us reply OK before fsync.
|
us to fan OKs out as soon as the writer has a per-row decision.
|
||||||
- **Per-connection FIFO**: a publisher that sends three EVENTs in a
|
- **OK semantics = accepted, not fsynced**: NIP-01 treats `OK true`
|
||||||
row expects three OKs in that order. Reordering across connections
|
as "accepted by the relay," not "durably on disk." We can reply as
|
||||||
is fine.
|
soon as SQLite returns success for the row (inside the open
|
||||||
|
transaction, before commit/fsync). Group commit can batch the
|
||||||
|
fsync without holding OKs back.
|
||||||
|
- **Per-row decision still required**: the OK reason field is
|
||||||
|
per-event (duplicate, blocked, invalid sig, pow, etc.), so we
|
||||||
|
cannot fan out a single batch-level OK. Each row's OK must reflect
|
||||||
|
that row's outcome.
|
||||||
|
|
||||||
## Sketch
|
## Sketch
|
||||||
|
|
||||||
@@ -36,7 +42,12 @@ Confirm `PRAGMA journal_mode=WAL` + `PRAGMA synchronous=NORMAL` on the
|
|||||||
event-store DB; group commits across the writer mutex's hold window.
|
event-store DB; group commits across the writer mutex's hold window.
|
||||||
Today each insert is its own transaction. Wrap N inserts (or a 5 ms
|
Today each insert is its own transaction. Wrap N inserts (or a 5 ms
|
||||||
budget, whichever first) in a single transaction managed by the writer
|
budget, whichever first) in a single transaction managed by the writer
|
||||||
coroutine. On commit, fan back N OK replies.
|
coroutine.
|
||||||
|
|
||||||
|
Because OK reflects acceptance not durability, each row can fan an OK
|
||||||
|
as soon as the per-row INSERT statement returns inside the
|
||||||
|
transaction — we do not need to wait for the batch's commit. The
|
||||||
|
fsync is hidden from the publisher latency budget entirely.
|
||||||
|
|
||||||
Implementation lives in quartz's `EventStore` / `SQLiteConnectionPool`,
|
Implementation lives in quartz's `EventStore` / `SQLiteConnectionPool`,
|
||||||
not geode — but geode owns the benchmark and validates the gain.
|
not geode — but geode owns the benchmark and validates the gain.
|
||||||
@@ -48,17 +59,18 @@ commit is well-trodden territory (nostr-rs-relay, strfry both do it).
|
|||||||
|
|
||||||
`RelaySession.receive` is currently single-flight: one EVENT in,
|
`RelaySession.receive` is currently single-flight: one EVENT in,
|
||||||
process, OK out, next EVENT. Allow a connection to push N EVENTs
|
process, OK out, next EVENT. Allow a connection to push N EVENTs
|
||||||
concurrently, dispatch them to a per-connection ingest pipeline, and
|
concurrently and dispatch them to a per-connection ingest pipeline.
|
||||||
serialise OKs back in arrival order via a small commit log.
|
|
||||||
|
|
||||||
A `Channel<EventCmd> with capacity = INGEST_PIPELINE_DEPTH` per
|
A `Channel<EventCmd>` with capacity = `INGEST_PIPELINE_DEPTH` per
|
||||||
connection, drained by a coroutine that batches into the group-commit
|
connection, drained by a coroutine that feeds the group-commit writer
|
||||||
above. OK responses are written to an `outQueue.send()` already — so
|
above. OKs go straight to `outQueue.send()` the moment each row
|
||||||
the pipeline just needs to record arrival order and emit OKs in that
|
returns from INSERT — no ordering bookkeeping needed, since the OK
|
||||||
order after each batch commits.
|
frame already carries the event id and the spec doesn't require
|
||||||
|
order. A pipelined publisher keying on event id will pair replies
|
||||||
|
correctly.
|
||||||
|
|
||||||
Expected: hides the verify+insert latency behind another EVENT's
|
Expected: hides verify+insert latency behind the next EVENT's parse,
|
||||||
parse, gets us closer to network-bound throughput.
|
gets us closer to network-bound throughput.
|
||||||
|
|
||||||
### Tier 3 — eager Schnorr verify off the writer thread
|
### Tier 3 — eager Schnorr verify off the writer thread
|
||||||
|
|
||||||
@@ -74,7 +86,8 @@ Add to `geode.perf.LoadBenchmark`:
|
|||||||
- `publishGroupCommitSingleClient` — same workload as the current
|
- `publishGroupCommitSingleClient` — same workload as the current
|
||||||
single-client benchmark, asserts >5000 EPS.
|
single-client benchmark, asserts >5000 EPS.
|
||||||
- `publishPipelinedSingleClient` — sends 100 EVENTs without awaiting
|
- `publishPipelinedSingleClient` — sends 100 EVENTs without awaiting
|
||||||
intermediate OKs; measures end-to-end and OK-ordering correctness.
|
intermediate OKs; measures end-to-end throughput and verifies that
|
||||||
|
every event id receives exactly one OK (in any order).
|
||||||
|
|
||||||
Existing benchmarks stay as the regression floor.
|
Existing benchmarks stay as the regression floor.
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user