Merge pull request #2769 from vitorpamplona/claude/fix-ok-message-ordering-fNXmT

Implement event ingestion batching with group commit and parallel verify
This commit is contained in:
Vitor Pamplona
2026-05-07 19:21:44 -04:00
committed by GitHub
15 changed files with 854 additions and 49 deletions
+6
View File
@@ -53,6 +53,12 @@ file = "/var/lib/geode/events.db"
# only for trusted-input scenarios (test fixtures, mirror replays). # only for trusted-input scenarios (test fixtures, mirror replays).
# verify_signatures = true # verify_signatures = true
# Run signature verification in parallel inside the IngestQueue
# (across all CPU cores) instead of serially on each connection's
# WebSocket pump. Default: true. Set false to fall back to the
# legacy in-policy verify path.
# parallel_verify = true
# Require clients to NIP-42 AUTH before REQ/EVENT/COUNT. # Require clients to NIP-42 AUTH before REQ/EVENT/COUNT.
require_auth = false require_auth = false
@@ -19,62 +19,119 @@ contention, not WS throughput).
## Constraints we must keep ## Constraints we must keep
- **OK ordering**: NIP-01 requires the OK reply to follow its EVENT. - **OK pairs by event id, not by order**: the OK frame carries the
We cannot reply OK before the insert decision (the OK carries event id, so clients pair replies to publishes by id. OKs can be
accepted/rejected + reason). emitted in any order — including reordered against the EVENT
- **Durability semantics**: clients reasonably assume `OK true` means stream, and against each other on the same connection. This frees
"stored." Batching must not make us reply OK before fsync. us to fan OKs out as soon as the writer has a per-row decision.
- **Per-connection FIFO**: a publisher that sends three EVENTs in a - **OK semantics = accepted, not fsynced**: NIP-01 treats `OK true`
row expects three OKs in that order. Reordering across connections as "accepted by the relay," not "durably on disk." We can reply as
is fine. soon as SQLite returns success for the row (inside the open
transaction, before commit/fsync). Group commit can batch the
fsync without holding OKs back.
- **Per-row decision still required**: the OK reason field is
per-event (duplicate, blocked, invalid sig, pow, etc.), so we
cannot fan out a single batch-level OK. Each row's OK must reflect
that row's outcome.
## Sketch ## Sketch
### Tier 1 — SQLite WAL + group commit (cheap win) ### Tier 1 — SQLite WAL + group commit (cheap win)
Confirm `PRAGMA journal_mode=WAL` + `PRAGMA synchronous=NORMAL` on the WAL is already on (`PRAGMA journal_mode=WAL`). The pool runs with
event-store DB; group commits across the writer mutex's hold window. `PRAGMA synchronous=OFF`, which is one notch more permissive than
Today each insert is its own transaction. Wrap N inserts (or a 5 ms the originally-sketched `synchronous=NORMAL` — we keep it as-is
budget, whichever first) in a single transaction managed by the writer because the project already accepted the OS-crash trade-off there.
coroutine. On commit, fan back N OK replies.
Implementation lives in quartz's `EventStore` / `SQLiteConnectionPool`, Group commit is implemented via a new `IEventStore.batchInsert`:
not geode — but geode owns the benchmark and validates the gain. the SQLite override holds the writer mutex once and wraps N events
in one `BEGIN IMMEDIATE … COMMIT`. Per-row error isolation uses
SAVEPOINTs so one bad event (expired, duplicate id) doesn't roll
back the good ones — just that row reports `Rejected`.
OKs fire as soon as each row's outcome is known inside the writer
batch, not waiting for fsync (per the OK-semantics constraint above).
Implementation lives in `quartz/nip01Core/store/sqlite/SQLiteEventStore.batchInsertEvents`,
exposed through `IEventStore.batchInsert` and consumed by the new
`IngestQueue` (Tier 2 below).
Expected: **~510× write throughput** on a fast SSD. SQLite group Expected: **~510× write throughput** on a fast SSD. SQLite group
commit is well-trodden territory (nostr-rs-relay, strfry both do it). commit is well-trodden territory (nostr-rs-relay, strfry both do it).
### Tier 2 — pipelined OK over multiple in-flight EVENTs ### Tier 2 — pipelined OK over multiple in-flight EVENTs
`RelaySession.receive` is currently single-flight: one EVENT in, `RelaySession.receive` was single-flight: one EVENT in, process, OK
process, OK out, next EVENT. Allow a connection to push N EVENTs out, next EVENT. With Tier 2 the connection's pump posts to the
concurrently, dispatch them to a per-connection ingest pipeline, and shared `IngestQueue` and returns immediately — the WS pump moves
serialise OKs back in arrival order via a small commit log. straight to the next frame.
A `Channel<EventCmd> with capacity = INGEST_PIPELINE_DEPTH` per `IngestQueue` (one per `NostrServer`) holds a bounded
connection, drained by a coroutine that batches into the group-commit `Channel<Submission>` (capacity = 1024 per the `DEFAULT_CAPACITY`
above. OK responses are written to an `outQueue.send()` already — so constant) drained by a single writer coroutine. The writer pulls
the pipeline just needs to record arrival order and emit OKs in that the first item to start a batch then `tryReceive`-drains everything
order after each batch commits. else queued (up to 64 — `DEFAULT_MAX_BATCH`), feeds the whole batch
to `IEventStore.batchInsert`, and dispatches each row's
`onComplete` callback as soon as the batch returns. The callback
turns into the `OK` frame at the WS layer.
Expected: hides the verify+insert latency behind another EVENT's OKs are not order-preserving (per the constraints above). The
parse, gets us closer to network-bound throughput. writer coroutine starts lazily on first `submit` so subscription-
only sessions don't pay for it and don't perturb `Dispatchers.Default`
scheduling.
Expected: hides verify+insert latency behind the next EVENT's parse,
gets us closer to network-bound throughput.
### Tier 3 — eager Schnorr verify off the writer thread ### Tier 3 — eager Schnorr verify off the writer thread
`VerifyPolicy` is in the policy stack and runs synchronously on `VerifyPolicy` ran synchronously on `receive`, serialising verify
`receive`. Move it into the ingest pipeline so verification of EVENT N+1 on each connection's pump coroutine. With Tier 3, `IngestQueue`
runs concurrently with the SQLite commit of EVENT N. secp256k1 verify takes a `verify: ((Event) -> Boolean)?` hook; when set, the writer
is parallelisable; the writer should never block on it. fan-outs a `coroutineScope { events.map { async(Default) { verify(it) } }.awaitAll() }`
on each batch before opening the SQLite transaction. Failed
verifies pre-mark `Rejected` and skip the insert.
Wired through `NostrServer(parallelVerify = ...)` and
`geode.Relay(parallelVerify = ...)`, controlled by
`[options].parallel_verify` in the relay config (default `true`)
and `--no-parallel-verify` on the CLI. Internal direct callers of
`NostrServer` (tests, library users) are opt-in: the flag defaults
to `false` to keep existing `VerifyPolicy`-in-chain semantics
unchanged.
`VerifyPolicy` was split into a parameterised
`VerifyEventsAndAuthPolicy(verifyEvents)` with two singletons:
- `VerifyPolicy` (default): verifies both `EVENT` and `AUTH`.
- `VerifyAuthOnlyPolicy`: verifies `AUTH` only, used when the
`IngestQueue` is doing the EVENT verify.
When `parallelVerify` is on, `composePolicy` swaps `VerifyPolicy`
for `VerifyAuthOnlyPolicy` so EVENTs aren't verified twice while
AUTH commands — which bypass the queue entirely — keep their
signature check. Without this split, removing `VerifyPolicy` from
the chain would let a forged AUTH event mark a pubkey as
authenticated.
Expected: ≈CPU_COUNT× verify-step speed-up on burst publishes
from a single connection, where verify was previously serial on
that pump.
## How to verify ## How to verify
Add to `geode.perf.LoadBenchmark`: `geode.perf.LoadBenchmark` carries the perf tests:
- `publishGroupCommitSingleClient` — same workload as the current - `publishGroupCommitSingleClient` — sequential publish-and-confirm
single-client benchmark, asserts >5000 EPS. on one connection (the same shape as the original
- `publishPipelinedSingleClient` — sends 100 EVENTs without awaiting `publishThroughputSingleClient`). Synchronous publishing means
intermediate OKs; measures end-to-end and OK-ordering correctness. batch size is always 1, so this case shows per-event SQLite tx
cost rather than the group-commit win — kept as a 500-EPS floor
to catch regressions from the rewrite.
- `publishPipelinedSingleClient` — bursts 10 000 EVENTs back-to-
back without awaiting intermediate OKs; verifies end-to-end
throughput and that every event id receives exactly one OK (in
any order). This is where Tier 1 + Tier 2 both light up.
Existing benchmarks stay as the regression floor. Existing benchmarks stay as the regression floor.
@@ -28,6 +28,7 @@ import com.vitorpamplona.quartz.nip01Core.relay.server.policies.FullAuthPolicy
import com.vitorpamplona.quartz.nip01Core.relay.server.policies.KindAllowDenyPolicy import com.vitorpamplona.quartz.nip01Core.relay.server.policies.KindAllowDenyPolicy
import com.vitorpamplona.quartz.nip01Core.relay.server.policies.PubkeyAllowDenyPolicy import com.vitorpamplona.quartz.nip01Core.relay.server.policies.PubkeyAllowDenyPolicy
import com.vitorpamplona.quartz.nip01Core.relay.server.policies.RejectFutureEventsPolicy import com.vitorpamplona.quartz.nip01Core.relay.server.policies.RejectFutureEventsPolicy
import com.vitorpamplona.quartz.nip01Core.relay.server.policies.VerifyAuthOnlyPolicy
import com.vitorpamplona.quartz.nip01Core.relay.server.policies.VerifyPolicy import com.vitorpamplona.quartz.nip01Core.relay.server.policies.VerifyPolicy
import com.vitorpamplona.quartz.nip01Core.store.IEventStore import com.vitorpamplona.quartz.nip01Core.store.IEventStore
import com.vitorpamplona.quartz.nip01Core.store.sqlite.EventStore import com.vitorpamplona.quartz.nip01Core.store.sqlite.EventStore
@@ -83,6 +84,12 @@ fun main(args: Array<String>) {
// opts out (CLI `--no-verify` or `[options].verify_signatures = false` // opts out (CLI `--no-verify` or `[options].verify_signatures = false`
// in the config). // in the config).
val verifySigs = !a.flag("--no-verify") && config.options.verify_signatures val verifySigs = !a.flag("--no-verify") && config.options.verify_signatures
// Parallel verify is on whenever signature checking is on; the
// IngestQueue handles it instead of VerifyPolicy. Operators can
// force the legacy in-policy path with `--no-parallel-verify` or
// `[options].parallel_verify = false`.
val parallelVerify =
verifySigs && !a.flag("--no-parallel-verify") && config.options.parallel_verify
// Advertised URL: explicit `info.relay_url` wins, then build from // Advertised URL: explicit `info.relay_url` wins, then build from
// host/port/path. 0.0.0.0 bind → 127.0.0.1 in the URL so NIP-42 // host/port/path. 0.0.0.0 bind → 127.0.0.1 in the URL so NIP-42
@@ -98,11 +105,19 @@ fun main(args: Array<String>) {
val store: IEventStore = EventStore(dbName = dbFile, relay = advertisedUrl) val store: IEventStore = EventStore(dbName = dbFile, relay = advertisedUrl)
val policyBuilder: () -> IRelayPolicy = { val policyBuilder: () -> IRelayPolicy = {
composePolicy(config, advertisedUrl, requireAuth, verifySigs) composePolicy(config, advertisedUrl, requireAuth, verifySigs, parallelVerify)
} }
val stateFile = config.admin.state_file?.let { File(it) } val stateFile = config.admin.state_file?.let { File(it) }
val relay = Relay(advertisedUrl, store, info, policyBuilder, stateFile = stateFile) val relay =
Relay(
advertisedUrl,
store,
info,
policyBuilder,
stateFile = stateFile,
parallelVerify = parallelVerify,
)
// Frame cap honors max_ws_frame_bytes when set; max_ws_message_bytes // Frame cap honors max_ws_frame_bytes when set; max_ws_message_bytes
// is treated as the same cap (Ktor's WebSockets plugin only exposes // is treated as the same cap (Ktor's WebSockets plugin only exposes
// a single per-frame limit; multi-frame messages remain unbounded). // a single per-frame limit; multi-frame messages remain unbounded).
@@ -151,6 +166,7 @@ private fun composePolicy(
advertisedUrl: com.vitorpamplona.quartz.nip01Core.relay.normalizer.NormalizedRelayUrl, advertisedUrl: com.vitorpamplona.quartz.nip01Core.relay.normalizer.NormalizedRelayUrl,
requireAuth: Boolean, requireAuth: Boolean,
verifySigs: Boolean, verifySigs: Boolean,
parallelVerify: Boolean,
): IRelayPolicy { ): IRelayPolicy {
val pieces = mutableListOf<IRelayPolicy>() val pieces = mutableListOf<IRelayPolicy>()
@@ -171,7 +187,11 @@ private fun composePolicy(
} }
if (verifySigs) { if (verifySigs) {
pieces += VerifyPolicy // When parallel verify is on, the IngestQueue handles EVENT
// verification on the writer's CPU fan-out — but AUTH events
// bypass the queue, so we still need the policy chain to
// verify those. `VerifyAuthOnlyPolicy` does exactly that.
pieces += if (parallelVerify) VerifyAuthOnlyPolicy else VerifyPolicy
} }
return pieces.fold<IRelayPolicy, IRelayPolicy>(EmptyPolicy) { acc, p -> return pieces.fold<IRelayPolicy, IRelayPolicy>(EmptyPolicy) { acc, p ->
@@ -73,6 +73,17 @@ class Relay(
* everything in memory only — fine for tests. * everything in memory only — fine for tests.
*/ */
stateFile: File? = null, stateFile: File? = null,
/**
* Run Schnorr signature verification in parallel inside the
* [com.vitorpamplona.quartz.nip01Core.relay.server.IngestQueue]
* instead of serially in the policy chain. Enables the Tier-3
* win in `geode/plans/2026-05-07-event-ingestion-batching.md`.
*
* When set, callers MUST omit `VerifyPolicy` from [policyBuilder]
* — having both verifies the same event twice for no benefit.
* `Main.kt` skips `VerifyPolicy` when this flag is on.
*/
parallelVerify: Boolean = false,
) : AutoCloseable { ) : AutoCloseable {
private val stateStore: RelayStateStore? = stateFile?.let { RelayStateStore(it) } private val stateStore: RelayStateStore? = stateFile?.let { RelayStateStore(it) }
@@ -158,6 +169,7 @@ class Relay(
if (user === EmptyPolicy) BanListPolicy(banStore) else user + BanListPolicy(banStore) if (user === EmptyPolicy) BanListPolicy(banStore) else user + BanListPolicy(banStore)
}, },
parentContext, parentContext,
parallelVerify = parallelVerify,
) )
/** /**
@@ -141,6 +141,18 @@ data class RelayConfig(
* for trusted-input scenarios (test fixtures, mirror replays). * for trusted-input scenarios (test fixtures, mirror replays).
*/ */
val verify_signatures: Boolean = true, val verify_signatures: Boolean = true,
/**
* Run signature verification in parallel inside the IngestQueue
* (CPU fan-out across `Dispatchers.Default`) instead of serially
* on each connection's WebSocket pump. Tier-3 of the
* `event-ingestion-batching` plan. Wins scale with how many
* EVENTs a single connection sends back-to-back: ~CPU_COUNT×
* verify-step speed-up on burst publishes. Set false to keep
* the legacy in-policy verify path.
*
* Only takes effect when [verify_signatures] is also true.
*/
val parallel_verify: Boolean = true,
) )
data class LimitsSection( data class LimitsSection(
@@ -363,6 +363,52 @@ class LoadBenchmark {
} }
} }
/**
* Same workload as [publishThroughputSingleClient] (sequential
* publish-and-confirm on one connection) — kept as a regression
* floor for the group-commit code path. Synchronous publishes
* never coalesce in the writer (batch size is always 1), so the
* EPS here measures per-event SQLite tx cost. The pipelined win
* shows up in [publishPipelinedSingleClient].
*/
@Test
fun publishGroupCommitSingleClient() =
benchmark("publish group-commit single client") {
runBenchmarkServer { server, http ->
val scope = CoroutineScope(Dispatchers.Default + SupervisorJob())
val client = NostrClient(BasicOkHttpWebSocket.Builder { _ -> http }, scope)
try {
val signer = NostrSignerSync(KeyPair())
val relayUrl = server.url.normalizeRelayUrl()
val n = 10_000
var ok = 0
val elapsed =
measureTime {
runBlocking {
repeat(n) { i ->
val event = signer.sign(TextNoteEvent.build("group-commit $i"))
if (client.publishAndConfirm(event, setOf(relayUrl))) ok++
}
}
}
val eps = (n * 1000.0) / elapsed.inWholeMilliseconds
println(
"events=$n ok=$ok elapsedMs=${elapsed.inWholeMilliseconds} eps=${"%.0f".format(eps)}",
)
check(ok == n) { "expected all $n events accepted, got $ok" }
// Floor: the pre-batching baseline was ~760 EPS
// single-client (see plan). Anything below 500
// means the group-commit / ingest-queue rewrite
// regressed the synchronous path.
check(eps > 500) { "synchronous EPS $eps fell below the 500 floor" }
} finally {
client.disconnect()
scope.cancel()
}
}
}
/** /**
* One publisher, N subscribers. Publishes one EVENT and measures * One publisher, N subscribers. Publishes one EVENT and measures
* fan-out latency: time from publish to last subscriber receiving. * fan-out latency: time from publish to last subscriber receiving.
@@ -450,6 +496,101 @@ class LoadBenchmark {
} }
} }
/**
* One publisher fires N EVENTs back-to-back without awaiting
* intermediate OKs, then collects all OKs by event id. This is
* the workload that exercises Tier 2 (per-connection ingest
* pipeline) + Tier 1 (group commit) together — multiple events
* are in flight on the same connection, so the writer can batch.
*
* Verifies the relaxed OK contract: every event id receives
* exactly one OK frame, in any order.
*/
@Test
fun publishPipelinedSingleClient() =
benchmark("publish pipelined single client") {
runBenchmarkServer { server, http ->
val n = 10_000
val signer = NostrSignerSync(KeyPair())
val events =
runBlocking {
(0 until n).map { i ->
signer.sign(TextNoteEvent.build("pipe $i"))
}
}
val ids = events.mapTo(HashSet()) { it.id }
val httpUrl =
okhttp3.Request
.Builder()
.url(server.url.replace("ws://", "http://"))
.build()
val okSeen = AtomicLong()
val okFailures = AtomicLong()
val unknownIds = AtomicLong()
val seenIds =
java.util.concurrent.ConcurrentHashMap
.newKeySet<String>()
val done = java.util.concurrent.CountDownLatch(1)
val ws =
http.newWebSocket(
httpUrl,
object : okhttp3.WebSocketListener() {
override fun onMessage(
webSocket: okhttp3.WebSocket,
text: String,
) {
if (!text.startsWith("[\"OK\"")) return
// ["OK","<id>",true|false,"<reason>"] —
// a tiny string scan is enough for a
// bench. Index 6 is past `["OK","`.
val idStart = 7
val idEnd = text.indexOf('"', idStart)
if (idEnd <= idStart) return
val id = text.substring(idStart, idEnd)
if (!ids.contains(id)) {
unknownIds.incrementAndGet()
return
}
if (!seenIds.add(id)) return
if (text.contains(",true,")) {
okSeen.incrementAndGet()
} else {
okFailures.incrementAndGet()
}
if (okSeen.get() + okFailures.get() == n.toLong()) done.countDown()
}
},
)
val elapsed =
measureTime {
// Burst-send: queue every EVENT to OkHttp's
// outbound buffer without any await, then
// wait for the corresponding OK frames.
for (event in events) {
ws.send("""["EVENT",${event.toJson()}]""")
}
check(done.await(60, java.util.concurrent.TimeUnit.SECONDS)) {
"timed out waiting for OKs: ok=${okSeen.get()} rej=${okFailures.get()} unknown=${unknownIds.get()}"
}
}
val eps = (n * 1000.0) / elapsed.inWholeMilliseconds
println(
"events=$n ok=${okSeen.get()} rejected=${okFailures.get()} " +
"unknownIds=${unknownIds.get()} elapsedMs=${elapsed.inWholeMilliseconds} eps=${"%.0f".format(eps)}",
)
check(okSeen.get() == n.toLong()) {
"expected $n accepted OKs, got ${okSeen.get()} (rejected ${okFailures.get()})"
}
check(seenIds.size == n) {
"expected $n unique OK ids, got ${seenIds.size} — duplicate or missing OKs"
}
ws.cancel()
}
}
/** /**
* Many concurrent publishers, each on their own WebSocket. Tells * Many concurrent publishers, each on their own WebSocket. Tells
* us whether the SQLite single-writer bottleneck is the floor or * us whether the SQLite single-writer bottleneck is the floor or
@@ -0,0 +1,305 @@
/*
* Copyright (c) 2025 Vitor Pamplona
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of
* this software and associated documentation files (the "Software"), to deal in
* the Software without restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
* Software, and to permit persons to whom the Software is furnished to do so,
* subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
package com.vitorpamplona.quartz.nip01Core.relay.server
import com.vitorpamplona.quartz.nip01Core.core.Event
import com.vitorpamplona.quartz.nip01Core.store.IEventStore
import com.vitorpamplona.quartz.utils.Log
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.SupervisorJob
import kotlinx.coroutines.async
import kotlinx.coroutines.awaitAll
import kotlinx.coroutines.cancel
import kotlinx.coroutines.channels.Channel
import kotlinx.coroutines.channels.ClosedReceiveChannelException
import kotlinx.coroutines.coroutineScope
import kotlinx.coroutines.launch
import kotlin.coroutines.CoroutineContext
/**
* Group-commit writer for incoming EVENT publishes.
*
* Submissions from any number of [RelaySession]s land in [incoming],
* a single drain coroutine pulls one item to start a batch then
* greedily drains everything else already queued (up to [maxBatch]),
* and forwards the whole batch through [IEventStore.batchInsert] —
* one writer-mutex acquisition + one BEGIN / COMMIT for the lot.
*
* Why this lives here: the SQLite event store enforces a single-writer
* mutex (matching SQLite's own file-level rule), so per-event
* `useWriter` calls serialise no matter how many publishers we have.
* Group commit collapses N mutex round-trips into one and lets
* BEGIN / WAL-append / COMMIT amortise across the batch.
*
* OK semantics (NIP-01):
* - The OK frame carries the event id, so clients pair replies by
* id, not by arrival order. We dispatch each [Submission.onComplete]
* as its row resolves; reordering across connections (and on the
* same connection) is allowed.
* - `OK true` means "accepted by this relay," not "fsynced." Since
* the underlying SQLite pool runs `synchronous = OFF` and WAL,
* a successful row inside the open transaction is the strongest
* guarantee we provide; replying after the batch's COMMIT (which
* is what happens in the current implementation) just adds the
* in-memory commit cost and keeps the write-failure path simple.
*
* Per-row error isolation lives in the store layer (SAVEPOINT in
* [com.vitorpamplona.quartz.nip01Core.store.sqlite.SQLiteEventStore]).
* A duplicate, expired event, etc. is a Rejected outcome for that
* row only.
*
* Backpressure: [incoming] is bounded at [capacity]. A flood of
* EVENTs that outpaces the writer suspends [submit] callers (i.e.
* the per-connection ingest path) until the writer drains. This
* propagates back through the WebSocket pump so a slow disk
* eventually slows the publisher rather than ballooning JVM memory.
*/
class IngestQueue(
private val store: IEventStore,
parentContext: CoroutineContext,
private val maxBatch: Int = DEFAULT_MAX_BATCH,
capacity: Int = DEFAULT_CAPACITY,
/**
* Optional pre-insert validator. When set, the writer runs each
* batch through this hook in parallel before opening the SQLite
* transaction. Events that return `false` skip the insert and are
* reported as Rejected with [verifyRejectionReason].
*
* The intended use is Schnorr signature verification: an event's
* `verify()` is CPU-bound and parallelisable, so spreading a
* batch's verifies across [Dispatchers.Default] threads is
* straight throughput. Hook fires off the WS pump so a single
* publisher streaming many EVENTs doesn't serialise verify on
* one connection's read coroutine.
*
* Default `null` skips this stage — callers that already verify
* inside their `IRelayPolicy` chain should leave it null to avoid
* double-verify.
*/
private val verify: ((Event) -> Boolean)? = null,
private val verifyRejectionReason: String = "invalid: bad signature or id",
) : AutoCloseable {
/**
* One outstanding ingest request: the event to insert plus the
* callback the writer fires once the row's outcome is known.
*/
class Submission(
val event: Event,
val onComplete: (IEventStore.InsertOutcome) -> Unit,
)
private val incoming = Channel<Submission>(capacity)
private val scope = CoroutineScope(parentContext + SupervisorJob())
/**
* Lazily-launched drain coroutine. We don't start it in `init`
* because eagerly launching from a server's lazy
* `LiveEventStore` allocates a Default-dispatcher slot at server
* construction time — visible to other tests sharing the same
* `Dispatchers.Default` pool, where it can perturb scheduling
* for unrelated REQ/EOSE timing. Starting on first `submit`
* keeps relays that never see an EVENT (read-only sessions,
* negentropy-only) from paying for the writer at all.
*/
@Volatile
private var writerStarted = false
private val startLock = Any()
/**
* Hand off [event] for insertion. [onComplete] is invoked once
* with the per-row outcome from the writer batch — exactly once,
* unless the queue closes mid-flight.
*
* Suspends only when [incoming] is full (writer fell behind by
* [DEFAULT_CAPACITY] events). Otherwise this returns as fast as a
* channel `send`, freeing the WebSocket pump to read the next
* frame — that's where the per-connection pipeline win comes
* from.
*/
suspend fun submit(
event: Event,
onComplete: (IEventStore.InsertOutcome) -> Unit,
) {
ensureWriterStarted()
incoming.send(Submission(event, onComplete))
}
private fun ensureWriterStarted() {
if (writerStarted) return
synchronized(startLock) {
if (writerStarted) return
scope.launch { drainLoop() }
writerStarted = true
}
}
private suspend fun drainLoop() {
val batch = ArrayList<Submission>(maxBatch)
try {
while (true) {
// Block for the first item — anything else would be a
// hot loop. Once we have one, drain greedily without
// blocking so back-to-back publishes coalesce into a
// single transaction.
batch.add(incoming.receive())
while (batch.size < maxBatch) {
val next = incoming.tryReceive().getOrNull() ?: break
batch.add(next)
}
processBatch(batch)
batch.clear()
}
} catch (_: ClosedReceiveChannelException) {
// Normal shutdown via close().
}
}
private suspend fun processBatch(batch: List<Submission>) {
val verifyResults = verifyBatch(batch)
val finalOutcomes = runInsertStage(batch, verifyResults)
dispatchOutcomes(batch, finalOutcomes)
}
/**
* Tier 3: per-row verify. Returns `null` when no [verify] hook is
* configured (skip the stage entirely). For multi-event batches
* each verify runs as its own `async(Default)` so they spread
* across CPU cores; single-event batches short-circuit to a
* direct call to avoid coroutine-scope overhead.
*/
private suspend fun verifyBatch(batch: List<Submission>): BooleanArray? {
val hook = verify ?: return null
if (batch.size == 1) return BooleanArray(1) { hook(batch[0].event) }
return coroutineScope {
batch
.map { sub -> async(Dispatchers.Default) { hook(sub.event) } }
.awaitAll()
.toBooleanArray()
}
}
/**
* Run the SQLite transaction for the verified subset of [batch]
* and stitch outcomes back to a per-batch-index array. Failed
* verifies pre-mark `Rejected` and skip the insert. A whole-batch
* commit failure converts every persisted entry to `Rejected`
* with the throw message.
*/
private suspend fun runInsertStage(
batch: List<Submission>,
verifyResults: BooleanArray?,
): Array<IEventStore.InsertOutcome?> {
val toInsert = ArrayList<Event>(batch.size)
val insertIndices = ArrayList<Int>(batch.size)
for (i in batch.indices) {
if (verifyResults == null || verifyResults[i]) {
toInsert.add(batch[i].event)
insertIndices.add(i)
}
}
val insertOutcomes: List<IEventStore.InsertOutcome> =
if (toInsert.isEmpty()) {
emptyList()
} else {
try {
store.batchInsert(toInsert)
} catch (e: Throwable) {
Log.w("IngestQueue") { "batchInsert failed for ${toInsert.size} events: ${e.message}" }
val reason = e.message ?: e::class.simpleName ?: "insert failed"
List(toInsert.size) { IEventStore.InsertOutcome.Rejected(reason) }
}
}
val finalOutcomes = arrayOfNulls<IEventStore.InsertOutcome>(batch.size)
for (j in insertIndices.indices) {
finalOutcomes[insertIndices[j]] =
insertOutcomes.getOrNull(j) ?: missingOutcome
}
if (verifyResults != null) {
for (i in batch.indices) {
if (!verifyResults[i]) {
finalOutcomes[i] = IEventStore.InsertOutcome.Rejected(verifyRejectionReason)
}
}
}
return finalOutcomes
}
private fun dispatchOutcomes(
batch: List<Submission>,
outcomes: Array<IEventStore.InsertOutcome?>,
) {
for (i in batch.indices) {
val sub = batch[i]
val outcome = outcomes[i] ?: missingOutcome
try {
sub.onComplete(outcome)
} catch (e: Throwable) {
// A misbehaving callback must not poison the writer
// loop; the outcome is delivered best-effort.
Log.w("IngestQueue") { "onComplete threw: ${e.message}" }
}
}
}
/**
* Stop accepting new submissions and cancel the writer.
* In-flight submissions whose batch hadn't started yet may never
* receive their callback — the WebSocket on the other side is
* also being torn down in that case, so the OK reply has nowhere
* to go anyway.
*/
override fun close() {
incoming.close()
scope.cancel()
}
companion object {
/**
* Default for a missing per-row outcome — only reachable on a
* contract violation (the store returned fewer outcomes than
* inserts), so the message is informational, not user-facing.
*/
private val missingOutcome =
IEventStore.InsertOutcome.Rejected("internal error: missing outcome")
/**
* Cap per batch. Sized to keep per-batch latency low (each
* transaction holds the SQLite writer mutex; over-large
* batches starve other writers and hurt p99 publish latency).
* 64 events at ~0.2 ms per insert ≈ 13 ms held — well under
* a perceptible UI tick.
*/
const val DEFAULT_MAX_BATCH: Int = 64
/**
* In-flight cap before [submit] suspends. With ~510× group
* commit speed-up over the single-event path, one batch
* cycle is on the order of milliseconds, so a 1024-deep
* queue tolerates short bursts (a publisher dumping a
* thousand notes) without blocking the WS pump.
*/
const val DEFAULT_CAPACITY: Int = 1024
}
}
@@ -23,6 +23,7 @@ package com.vitorpamplona.quartz.nip01Core.relay.server
import com.vitorpamplona.quartz.nip01Core.core.Event import com.vitorpamplona.quartz.nip01Core.core.Event
import com.vitorpamplona.quartz.nip01Core.relay.filters.Filter import com.vitorpamplona.quartz.nip01Core.relay.filters.Filter
import com.vitorpamplona.quartz.nip01Core.store.IEventStore import com.vitorpamplona.quartz.nip01Core.store.IEventStore
import kotlinx.coroutines.CompletableDeferred
import kotlinx.coroutines.channels.BufferOverflow import kotlinx.coroutines.channels.BufferOverflow
import kotlinx.coroutines.flow.MutableSharedFlow import kotlinx.coroutines.flow.MutableSharedFlow
import kotlinx.coroutines.flow.onSubscription import kotlinx.coroutines.flow.onSubscription
@@ -39,6 +40,7 @@ import kotlinx.coroutines.flow.onSubscription
*/ */
class LiveEventStore( class LiveEventStore(
private val store: IEventStore, private val store: IEventStore,
private val ingest: IngestQueue,
) { ) {
private val newEventStream = private val newEventStream =
MutableSharedFlow<Event>( MutableSharedFlow<Event>(
@@ -47,9 +49,47 @@ class LiveEventStore(
onBufferOverflow = BufferOverflow.DROP_LATEST, // Default behavior onBufferOverflow = BufferOverflow.DROP_LATEST, // Default behavior
) )
/**
* Fire-and-forget enqueue: hand [event] to the [IngestQueue] and
* fire [onComplete] once the writer's batch has a per-row
* decision. On `Accepted` the live stream is also emitted to so
* subscribers see the event. Suspends only when the ingest queue
* is full (backpressure).
*/
suspend fun submit(
event: Event,
onComplete: (IEventStore.InsertOutcome) -> Unit,
) {
ingest.submit(event) { outcome ->
if (outcome is IEventStore.InsertOutcome.Accepted) {
newEventStream.tryEmit(event)
}
onComplete(outcome)
}
}
/**
* Suspending insert kept for callers that don't care about
* pipelining (tests, scripted paths). Routes through the same
* [IngestQueue] as [submit] so the batch write path is exercised
* even by tests that prefer a sequential `insert` API.
* Throws on rejection so callers can `try` around it the way the
* old API did.
*/
suspend fun insert(event: Event) { suspend fun insert(event: Event) {
store.insert(event) val done = CompletableDeferred<Unit>()
newEventStream.tryEmit(event) submit(event) { outcome ->
when (outcome) {
IEventStore.InsertOutcome.Accepted -> {
done.complete(Unit)
}
is IEventStore.InsertOutcome.Rejected -> {
done.completeExceptionally(IllegalStateException(outcome.reason))
}
}
}
done.await()
} }
suspend fun query( suspend fun query(
@@ -20,6 +20,7 @@
*/ */
package com.vitorpamplona.quartz.nip01Core.relay.server package com.vitorpamplona.quartz.nip01Core.relay.server
import com.vitorpamplona.quartz.nip01Core.crypto.verify
import com.vitorpamplona.quartz.nip01Core.relay.server.policies.VerifyPolicy import com.vitorpamplona.quartz.nip01Core.relay.server.policies.VerifyPolicy
import com.vitorpamplona.quartz.nip01Core.store.IEventStore import com.vitorpamplona.quartz.nip01Core.store.IEventStore
import com.vitorpamplona.quartz.utils.cache.LargeCache import com.vitorpamplona.quartz.utils.cache.LargeCache
@@ -36,17 +37,38 @@ import kotlin.coroutines.CoroutineContext
* *
* @param store The [IEventStore] backing this relay. * @param store The [IEventStore] backing this relay.
* @param policyBuilder Controls requirements for relay commands. * @param policyBuilder Controls requirements for relay commands.
* @param parallelVerify When `true`, Schnorr verification runs in
* parallel inside the [IngestQueue] (one async per event, dispatched
* on `Dispatchers.Default`) rather than serially on the WS pump
* coroutine inside [VerifyPolicy]. Callers that flip this on should
* *omit* `VerifyPolicy` from their [policyBuilder] chain to avoid
* double-verifying.
*/ */
class NostrServer( class NostrServer(
private val store: IEventStore, private val store: IEventStore,
private val policyBuilder: () -> IRelayPolicy = { VerifyPolicy }, private val policyBuilder: () -> IRelayPolicy = { VerifyPolicy },
private val parentContext: CoroutineContext = SupervisorJob(), private val parentContext: CoroutineContext = SupervisorJob(),
parallelVerify: Boolean = false,
) : AutoCloseable { ) : AutoCloseable {
private val subStore = LiveEventStore(store)
/** Scope for all subscriptions. */ /** Scope for all subscriptions. */
private val scope = CoroutineScope(parentContext + SupervisorJob()) private val scope = CoroutineScope(parentContext + SupervisorJob())
/**
* Group-commit writer shared across every connected session.
* Sessions hand off EVENT publishes here instead of awaiting
* [IEventStore.insert] inline; the queue coalesces back-to-back
* publishes into a single SQLite transaction. See [IngestQueue]
* for the OK ordering and durability semantics.
*/
private val ingest =
IngestQueue(
store = store,
parentContext = parentContext,
verify = if (parallelVerify) ({ it.verify() }) else null,
)
private val subStore = LiveEventStore(store, ingest)
/** Active client sessions keyed by an opaque connection id. */ /** Active client sessions keyed by an opaque connection id. */
private val connections = LargeCache<Int, RelaySession>() private val connections = LargeCache<Int, RelaySession>()
@@ -95,6 +117,7 @@ class NostrServer(
override fun close() { override fun close() {
connections.forEach { _, session -> session.cancelAllSubscriptions() } connections.forEach { _, session -> session.cancelAllSubscriptions() }
connections.clear() connections.clear()
ingest.close()
scope.cancel() scope.cancel()
store.close() store.close()
} }
@@ -34,6 +34,7 @@ import com.vitorpamplona.quartz.nip01Core.relay.commands.toRelay.CloseCmd
import com.vitorpamplona.quartz.nip01Core.relay.commands.toRelay.CountCmd import com.vitorpamplona.quartz.nip01Core.relay.commands.toRelay.CountCmd
import com.vitorpamplona.quartz.nip01Core.relay.commands.toRelay.EventCmd import com.vitorpamplona.quartz.nip01Core.relay.commands.toRelay.EventCmd
import com.vitorpamplona.quartz.nip01Core.relay.commands.toRelay.ReqCmd import com.vitorpamplona.quartz.nip01Core.relay.commands.toRelay.ReqCmd
import com.vitorpamplona.quartz.nip01Core.store.IEventStore
import com.vitorpamplona.quartz.nip77Negentropy.NegCloseCmd import com.vitorpamplona.quartz.nip77Negentropy.NegCloseCmd
import com.vitorpamplona.quartz.nip77Negentropy.NegMsgCmd import com.vitorpamplona.quartz.nip77Negentropy.NegMsgCmd
import com.vitorpamplona.quartz.nip77Negentropy.NegOpenCmd import com.vitorpamplona.quartz.nip77Negentropy.NegOpenCmd
@@ -42,6 +43,7 @@ import com.vitorpamplona.quartz.utils.cache.LargeCache
import kotlinx.coroutines.CancellationException import kotlinx.coroutines.CancellationException
import kotlinx.coroutines.CoroutineScope import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Job import kotlinx.coroutines.Job
import kotlinx.coroutines.channels.ClosedSendChannelException
import kotlinx.coroutines.launch import kotlinx.coroutines.launch
/** /**
@@ -129,11 +131,29 @@ class RelaySession(
return return
} }
// Fire-and-forget: hand the event to the group-commit writer
// and continue reading from the WebSocket without waiting on
// SQLite. The OK frame is sent from the writer's callback,
// possibly out of arrival order — NIP-01 pairs OKs to events
// by id, so reordering is fine.
try { try {
store.insert(cmd.event) store.submit(cmd.event) { outcome ->
send(OkMessage(cmd.event.id, true, "")) when (outcome) {
} catch (e: Exception) { IEventStore.InsertOutcome.Accepted -> {
send(OkMessage(cmd.event.id, false, e.message ?: e::class.simpleName ?: "unkown error")) send(OkMessage(cmd.event.id, true, ""))
}
is IEventStore.InsertOutcome.Rejected -> {
send(OkMessage(cmd.event.id, false, outcome.reason))
}
}
}
} catch (_: ClosedSendChannelException) {
// Server is shutting down — the queue is closed. Reply
// with a transient failure so a client re-trying against
// the next instance gets a sane signal; the WS itself is
// about to be torn down by the server-stop path.
send(OkMessage(cmd.event.id, false, "error: relay shutting down"))
} }
} }
@@ -31,13 +31,24 @@ import com.vitorpamplona.quartz.nip01Core.relay.server.IRelayPolicy
import com.vitorpamplona.quartz.nip01Core.relay.server.PolicyResult import com.vitorpamplona.quartz.nip01Core.relay.server.PolicyResult
/** /**
* Allows all commands without authentication. This is the default policy. * Verifies the Schnorr signature + id hash of every incoming
* `EVENT` and `AUTH` event. Other commands pass through.
*
* The default `VerifyPolicy` singleton verifies both. The
* [VerifyAuthOnlyPolicy] singleton skips the EVENT path — use it
* when the [com.vitorpamplona.quartz.nip01Core.relay.server.IngestQueue]
* is doing parallel verify (Tier 3 of the event-ingestion plan)
* so EVENTs aren't verified twice. AUTH is still verified inline
* because the AUTH path bypasses the queue entirely; without it,
* a forged event could mark a pubkey as authenticated.
*/ */
object VerifyPolicy : IRelayPolicy { open class VerifyEventsAndAuthPolicy(
private val verifyEvents: Boolean,
) : IRelayPolicy {
override fun onConnect(send: (Message) -> Unit) { } override fun onConnect(send: (Message) -> Unit) { }
override fun accept(cmd: EventCmd) = override fun accept(cmd: EventCmd) =
if (cmd.event.verify()) { if (!verifyEvents || cmd.event.verify()) {
PolicyResult.Accepted(cmd) PolicyResult.Accepted(cmd)
} else { } else {
PolicyResult.Rejected("invalid: bad signature or id") PolicyResult.Rejected("invalid: bad signature or id")
@@ -56,3 +67,18 @@ object VerifyPolicy : IRelayPolicy {
override fun canSendToSession(event: Event) = true override fun canSendToSession(event: Event) = true
} }
/**
* Default verify policy — checks every EVENT and every AUTH. Use
* this when nothing else in the stack verifies signatures.
*/
object VerifyPolicy : VerifyEventsAndAuthPolicy(verifyEvents = true)
/**
* Verify policy that skips the EVENT path. Use when the
* `IngestQueue` is configured with `parallelVerify`, so EVENTs are
* verified once on the writer's CPU fan-out instead of inline on
* the WebSocket pump. AUTH is still verified inline because that
* command never reaches the queue.
*/
object VerifyAuthOnlyPolicy : VerifyEventsAndAuthPolicy(verifyEvents = false)
@@ -41,6 +41,42 @@ interface IEventStore : AutoCloseable {
suspend fun transaction(body: ITransaction.() -> Unit) suspend fun transaction(body: ITransaction.() -> Unit)
/**
* Per-row outcome from [batchInsert]. The OK frame on the wire is
* built from this — `Accepted` becomes `OK true`, `Rejected.reason`
* becomes the false reason. NIP-01 says OK pairs to its EVENT by
* id, not by order, so callers may dispatch outcomes in any order.
*/
sealed class InsertOutcome {
data object Accepted : InsertOutcome()
data class Rejected(
val reason: String,
) : InsertOutcome()
}
/**
* Bulk insert in a single transaction with per-row error isolation.
* Returns one outcome per input event in the same order.
*
* Implementations must isolate per-row failures so one bad event
* doesn't roll back the others (SQLite uses SAVEPOINTs). If the
* outer commit itself fails, every entry in the returned list is
* `Rejected` with the commit-failure reason.
*
* Default impl runs each insert in its own transaction — correct
* but loses the group-commit win. SQLite overrides this.
*/
suspend fun batchInsert(events: List<Event>): List<InsertOutcome> =
events.map { event ->
try {
insert(event)
InsertOutcome.Accepted
} catch (e: Throwable) {
InsertOutcome.Rejected(e.message ?: e::class.simpleName ?: "insert failed")
}
}
suspend fun <T : Event> query(filter: Filter): List<T> suspend fun <T : Event> query(filter: Filter): List<T>
suspend fun <T : Event> query(filters: List<Filter>): List<T> suspend fun <T : Event> query(filters: List<Filter>): List<T>
@@ -96,6 +96,54 @@ class ObservableEventStore(
_changes.emit(StoreChange.Insert(event)) _changes.emit(StoreChange.Insert(event))
} }
override suspend fun batchInsert(events: List<Event>): List<IEventStore.InsertOutcome> {
// Split into ephemeral (no-store) and persistable, delegate the
// persistable subset to the inner store's batched path so we
// keep the group-commit win, then merge outcomes back in input
// order. Already-expired ephemerals are dropped (matching
// [insert]). Accepted events are emitted on [_changes] only
// after the inner batch returns, so a commit failure that
// converts everything to Rejected suppresses the emits.
if (events.isEmpty()) return emptyList()
val outcomes = arrayOfNulls<IEventStore.InsertOutcome>(events.size)
val persistableIndices = ArrayList<Int>(events.size)
val persistable = ArrayList<Event>(events.size)
for (i in events.indices) {
val event = events[i]
if (event.kind.isEphemeral()) {
outcomes[i] =
if (event.isExpired()) {
IEventStore.InsertOutcome.Rejected("blocked: Cannot insert an expired event")
} else {
IEventStore.InsertOutcome.Accepted
}
} else {
persistableIndices.add(i)
persistable.add(event)
}
}
if (persistable.isNotEmpty()) {
val innerOutcomes = inner.batchInsert(persistable)
for (j in persistable.indices) {
outcomes[persistableIndices[j]] = innerOutcomes[j]
}
}
for (i in events.indices) {
if (outcomes[i] is IEventStore.InsertOutcome.Accepted) {
_changes.emit(StoreChange.Insert(events[i]))
}
}
// Every index in `events.indices` was populated above (either
// from the ephemeral pre-pass or the `inner.batchInsert`
// result), so no nulls remain. `requireNoNulls()` enforces
// that with a runtime check instead of an unchecked cast.
return outcomes.requireNoNulls().asList()
}
override suspend fun transaction(body: IEventStore.ITransaction.() -> Unit) { override suspend fun transaction(body: IEventStore.ITransaction.() -> Unit) {
val accepted = ArrayList<Event>() val accepted = ArrayList<Event>()
inner.transaction { inner.transaction {
@@ -43,6 +43,8 @@ class EventStore(
override suspend fun transaction(body: IEventStore.ITransaction.() -> Unit) = store.transaction(body) override suspend fun transaction(body: IEventStore.ITransaction.() -> Unit) = store.transaction(body)
override suspend fun batchInsert(events: List<Event>) = store.batchInsertEvents(events)
override suspend fun <T : Event> query(filter: Filter) = store.query<T>(filter) override suspend fun <T : Event> query(filter: Filter) = store.query<T>(filter)
override suspend fun <T : Event> query(filters: List<Filter>) = store.query<T>(filters) override suspend fun <T : Event> query(filters: List<Filter>) = store.query<T>(filters)
@@ -201,6 +201,63 @@ class SQLiteEventStore(
} }
} }
/**
* Group-commit batch insert with per-row error isolation via
* SAVEPOINTs. Acquires the writer mutex once and wraps every
* inserts in a single outer transaction so the WAL append + sync
* cost is paid once for the whole batch.
*
* Per-row contract:
* - Validation errors (expired) and per-row INSERT failures
* (UNIQUE constraint, etc.) ROLLBACK only that row's savepoint;
* other rows commit.
* - Ephemeral kinds are accepted without writing — the live
* stream still surfaces them; persistence is intentionally a
* no-op per NIP-01.
*
* Outer-commit failure throws; the caller treats every entry as
* `Rejected` (this is what the IEventStore contract documents).
*/
suspend fun batchInsertEvents(events: List<Event>): List<IEventStore.InsertOutcome> {
if (events.isEmpty()) return emptyList()
val outcomes = ArrayList<IEventStore.InsertOutcome>(events.size)
pool.useWriter { db ->
db.transaction {
events.forEachIndexed { i, event ->
outcomes.add(insertWithSavepoint(event, i, this))
}
}
}
return outcomes
}
private fun insertWithSavepoint(
event: Event,
index: Int,
db: SQLiteConnection,
): IEventStore.InsertOutcome {
if (event.isExpired()) {
return IEventStore.InsertOutcome.Rejected("blocked: Cannot insert an expired event")
}
if (event.kind.isEphemeral()) return IEventStore.InsertOutcome.Accepted
val sp = "ev$index"
db.execSQL("SAVEPOINT $sp")
return try {
innerInsertEvent(event, db)
db.execSQL("RELEASE SAVEPOINT $sp")
IEventStore.InsertOutcome.Accepted
} catch (e: Throwable) {
// Roll back just this row, then release the (now empty)
// savepoint frame so the next iteration's BEGIN works.
// Both calls are individually try/catch'd because a failed
// ROLLBACK shouldn't mask the original cause.
runCatching { db.execSQL("ROLLBACK TRANSACTION TO SAVEPOINT $sp") }
runCatching { db.execSQL("RELEASE SAVEPOINT $sp") }
IEventStore.InsertOutcome.Rejected(e.message ?: e::class.simpleName ?: "insert failed")
}
}
inner class Transaction( inner class Transaction(
val db: SQLiteConnection, val db: SQLiteConnection,
) : IEventStore.ITransaction { ) : IEventStore.ITransaction {