f7d4e3340911a7976a2ae806bb7bbf05a6a13de2
13 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Claude
|
f7d4e33409 |
refactor: promote relay-server toolkit from quartz-relay to quartz
Generalize the operator-agnostic pieces so any embed of the relay server can use them without depending on quartz-relay (Ktor, TOML, operator wrapper). quartz-relay shrinks to its actual job: TOML config, Ktor wiring, persistence sidecar, and the Relay composition. - Nip98AuthVerifier -> quartz nip98HttpAuth (verify() now suspend, uses kotlinx Mutex for KMP). Pairs with HTTPAuthorizationEvent. - PassThroughPolicy + KindAllowDenyPolicy + PubkeyAllowDenyPolicy + RejectFutureEventsPolicy -> quartz nip01Core/relay/server/policies alongside the existing EmptyPolicy / VerifyPolicy / FullAuthPolicy. - Collapse EmptyPolicy into 'object EmptyPolicy : PassThroughPolicy()' (PassThroughPolicy is now an open class instead of abstract). - BanStore -> quartz nip86RelayManagement/server. Reimplemented lock-free with kotlin.concurrent.atomics.AtomicReference + immutable state snapshots so it works in commonMain. - DynamicBanPolicy -> renamed to BanListPolicy; lives next to the BanStore it consults. The "Dynamic" qualifier was a contrast to static policies in quartz-relay; in its new home the name describes what it actually does. - Nip86Server -> quartz nip86RelayManagement/server next to Nip86Client. Drops the RelayInfo wrapper indirection: InfoHolder now operates on Nip11RelayInformation directly. - InProcessWebSocket -> quartz nip01Core/relay/server/inprocess. Constructor takes NostrServer (was: the operator-side Relay wrapper) so any embed can wire the same in-process bridge. - Move BanStoreTest, Nip86ServerTest, Nip98AuthVerifierTest into quartz/jvmAndroidTest. Adjust runBlocking-bodied tests so JUnit 4 sees Unit returns. PoliciesTest stays in quartz-relay tests because it uses module-local SyntheticEvents fixtures. |
||
|
Claude
|
e78561af67 |
refactor(relay): split overgrown files + reduce duplication
Audit follow-ups, no behavior change. - Centralize NIPs/name/version constants in RelayInfo so RelayConfig and the default doc share one source of truth. - Extract NegSessionRegistry (NIP-77 state + open/msg/close) out of RelaySession; the connection class now routes commands. - Move multi-filter snapshot union/dedupe onto LiveEventStore. - Pull Nip86HttpRoute and WebSocketSessionPump out of LocalRelayServer (was 485 lines, three responsibilities). - Collapse Nip86Server.dispatch repetition with withHex/withHexAndReason/ withInt/withString helpers; reuse Hex.isHex64 instead of a local regex. - Make Nip11RelayInformation (and nested types) data classes so Nip86Server uses the synthesized copy() directly — drops the hand-rolled field-by-field shim. |
||
|
Claude
|
c440186575 |
feat(relay): NIP-77 negentropy reconciliation
Server-side negentropy: NEG-OPEN snapshots the matching event set, NEG-MSG drives the round-trip via the kmp-negentropy library, and NEG-CLOSE frees per-subId state. Same access controls as REQ apply to NEG-OPEN. supported_nips bumped to advertise "77". |
||
|
Claude
|
d49d4a1025 |
perf(relay): load benchmark + bump per-session outbound buffer
Adds an opt-in load benchmark suite (`-DrunLoadBenchmark=true`)
covering: held-open WebSocket count, single + concurrent EVENT
publish throughput, and live-event fan-out latency to N
subscribers on one connection.
Numbers from a small VM (4 GiB RAM, 4 vCPU, ulimit -n 4096):
Connections (raw WS, one REQ each)
100 : 100 OK, 0.7 s
500 : 500 OK, 1.2 s
1000 : 1000 OK, 2.0 s
2000 : 1993 OK (hits the 4096-fd ceiling: 2 fds per WS)
Single-publisher serial publish + OK
10000 events, 13.2 s, 760 EPS (round-trip latency bound)
Concurrent publishers (each on its own WS)
parallel=2 : 807 EPS
parallel=4 : 1452 EPS
parallel=8 : 1989 EPS ← knee
parallel=16 : 1704 EPS ← SQLite single-writer ceiling
parallel=32 : 1778 EPS
Fanout (one publish, N active subs on a single WS)
100 subs : 67 ms last
500 subs : 52 ms last
1000 subs : 51 ms last
2000 subs : 111 ms last
Bumped SESSION_OUTGOING_BUFFER from 1024 → 8192. The 1024 cap was
hit by the 2000-sub fanout test (2000 outbound frames into one
session's queue), causing the relay to drop the connection as a
slow-consumer protection. 8192 fits the realistic upper bound for
a high-fan-out client (a few thousand subs on one WS) and caps
per-session memory at ~2 MiB before we drop.
Numbers also confirm:
- The SQLite single-writer plateau is around 2000 EPS on this
box. Production behind WAL + a faster disk should beat that.
- Each WebSocket consumes 2 file descriptors. Operators must
raise `ulimit -n` to ~3× their target connection count.
- Fan-out scales sub-linearly (2000 subs ≈ 2× the latency of
100 subs) — the bottleneck is shared (broadcast + SQLite
write), not per-sub.
Total :quartz-relay tests: 99 (4 new benchmarks, opt-in), 0 failures.
|
||
|
Claude
|
b7ba3f6d15 |
fix(relay): security + concurrency audit + on-disk persistence
Addresses every Critical/High/Medium finding from the code-quality audit, plus the operator request to persist NIP-86 admin state and NIP-11 doc mutations across restarts. ## Security (audit Critical) - C1 — NIP-98 verifier no longer trusts the client-supplied `Host` header. New `LocalRelayServer.publicUrl` (and `[admin].public_url`) must be set in any production deployment; the verifier compares against this fixed string. Falls back to Host for loopback unit tests with a docstring warning. - C2 — NIP-98 replay protection. Verifier now keeps a bounded LinkedHashMap of recently-accepted event ids (TTL = 2× tolerance, max 1024 entries, LRU evicted) and rejects duplicates. Replay check runs LAST so a one-shot id isn't burned on a request that fails signature/url/method validation. - C3 — NIP-86 admin POST body is now capped at 1 MiB (configurable via `LocalRelayServer.maxAdminBodyBytes`). The `Content-Length` header is honored as a fast pre-read reject; any body that exceeds the cap during streaming returns 413 PayloadTooLarge before being buffered. ## Concurrency / correctness (audit High) - H1 — Per-session writer coroutine + bounded outbound queue (SESSION_OUTGOING_BUFFER = 1024). When the queue fills, the connection is closed cleanly instead of silently `trySend`-ing into a void; subscribers will never silently miss EVENT/EOSE again. - H3 — `BanStore` kind ops serialize through a new `kindLock` so concurrent admin RPCs and policy reads see consistent state. `allowKind` and `disallowKind` are now symmetric: each removes the kind from the opposite set, so `allowKind(K)` after `disallowKind(K)` actually re-allows K. - H4 — `InProcessWebSocket` reconnect after disconnect is now supported. Each `connect()` allocates a fresh CoroutineScope + drainer channel, so a prior `disconnect()` (which cancelled the previous scope) doesn't leave the new connect with a dead drainer. - H5 — `Nip86Server.dispatch` re-throws `CancellationException` through the runCatching's getOrElse so structured concurrency works (cancellation no longer reported as an RPC error). - M1 — `LiveEventStore.query` dedupes events between the historical replay and the live SharedFlow during the in-flight overlap window. Events seen during `store.query` are tracked in a transient set and filtered out of the live stream until EOSE; the set is dropped after EOSE so live-only events don't accumulate memory. ## Operator-facing (audit Medium / Low) - L4 — Audit-trail log: every admin RPC writes a single structured line to stderr (pubkey + method + ok/error). - L3 — RelayConfig.resolveInfo() default supported_nips list now matches RelayInfo.default() (both advertise NIP-86). - M2 — Hex-id and pubkey params validated as 64-char hex on ban/allow/list methods; malformed input returns "invalid params" instead of being silently stored. - M4 — argparse supports `--key=value` form in addition to `--key value`. - M5 — `RelayHub.close()` is idempotent and rejects subsequent `getOrCreate` calls so a relay created mid-shutdown can't leak its store. - M7 — `LocalRelayServer.stop()` sets a `shuttingDown` flag *before* notifying clients; new WebSocket upgrades during the grace window are rejected so they don't miss the NOTICE. - L5 — Shutdown hook runCatching-wraps both `server.stop()` and `relay.close()` so a throw in one doesn't skip the other. - `--verify` was renamed to `--no-verify` semantically (verify is the default). The `--verify` flag is no longer documented. ## On-disk persistence (operator request) New `RelayStateStore` writes a single JSON sidecar file holding the live NIP-11 doc + all NIP-86 ban / allow / kind lists. Atomic write via temp + ATOMIC_MOVE rename so a crash mid-save can never leave the file half-written. - `BanStore` now takes an optional `onMutation: () -> Unit` callback; every mutation fires it. `Relay` wires it to `snapshot()` which captures BanStore + info into a `RelayPersistedState` and calls `RelayStateStore.save`. - `Relay` accepts a new `stateFile: File?` constructor arg. At construction it loads the snapshot via `RelayStateStore.load` and seeds the in-memory state — using a private `seedFromSnapshot` bulk-load that bypasses the mutation callback so we don't write back what we just read. - New config field `[admin].state_file = "/var/lib/quartz-relay/events.db.admin.json"`. Convention: place next to the SQLite event-store file. - Corrupt state files are tolerated: log to stderr and start fresh (refuse to silently overwrite operator data). ## Tests 8 new persistence tests: cold-boot writes nothing, ban/allow/info round-trip across restart, kind allow/deny round-trip, corrupt-file recovery, atomic write (no leftover .tmp), in-memory mode when no state file is configured, full RelayStateStore round-trip across all sections. Total :quartz-relay tests: 95, 0 failures. |
||
|
Claude
|
7c7908d373 |
feat(relay): NIP-86 relay management API
Implements the operator-facing JSON-RPC admin protocol on the relay
side, layered on top of the existing NIP-98 HTTP-Auth + NIP-86 wire
types in :quartz. Same path as the NIP-01 WebSocket and NIP-11 GET —
HTTP POST with Content-Type application/nostr+json+rpc, gated by a
NIP-98 Authorization header signed by an operator-listed pubkey.
Architecture:
- BanStore — concurrent in-memory state for runtime
pubkey/event/kind ban + allow lists.
- DynamicBanPolicy — IRelayPolicy that consults BanStore on every
EVENT; auto-prepended to every Relay's policy
stack so admin actions take effect without a
restart.
- Nip98AuthVerifier — parses `Authorization: Nostr <base64-event>`,
verifies kind 27235 + Schnorr sig + ±60 s
clock skew + method/url/payload-hash match.
- Nip86Server — JSON-RPC dispatcher. Mutates BanStore for
ban/allow/list methods, mutates the live
RelayInfo doc for changerelayname/desc/icon
(Relay.info is now @Volatile var, mutation
through Relay.updateInfo).
- LocalRelayServer — adds POST handler at the relay path:
403 if no admin pubkeys configured,
401 if NIP-98 missing/invalid,
403 if signer's pubkey isn't on the admin list,
200 with Nip86Response otherwise.
- RelayConfig.AdminSection — `[admin].pubkeys = [...]` config key.
- RelayInfo.default() now advertises NIP-86 alongside 1/9/11/40/42/45/50/62.
Methods implemented:
supportedmethods, banpubkey, unbanpubkey, listbannedpubkeys,
allowpubkey, unallowpubkey, listallowedpubkeys,
banevent (also deletes from store), allowevent, listbannedevents,
allowkind, disallowkind, listallowedkinds,
changerelayname, changerelaydescription, changerelayicon.
Tests (28 new):
- BanStoreTest (6) — ban/allow round-trips, case-insensitive
pubkey match, kind allow/deny precedence,
audit-trail listing.
- Nip86ServerTest (8) — every method's accept/reject path.
- Nip98AuthVerifierTest (8) — happy path, missing/wrong scheme,
url/method/payload-hash mismatch,
stale created_at, wrong event kind.
- Nip86EndToEndTest (6) — real HTTP POST through LocalRelayServer:
supportedmethods returns 200,
outsider returns 403, no auth header
returns 401 + WWW-Authenticate, banpubkey
blocks subsequent EVENT publish over WS,
changerelayname flows to NIP-11 GET,
admin endpoint is disabled when no
pubkeys configured.
Total :quartz-relay tests: 87, 0 failures.
|
||
|
Claude
|
9ebdfc0b81 |
feat(relay): drop max_event_bytes + rate-limit configs; verify on by default
Removes config keys + policies that don't earn their complexity:
- [limits].max_event_bytes / MaxEventBytesPolicy — duplicates
[limits].max_ws_frame_bytes which is the right layer (Ktor frame
cap at the wire), and the policy-level check ran AFTER the event
was already parsed and bound for the store.
- [limits].messages_per_sec, [limits].subscriptions_per_min /
RateLimitPolicy — per-session token buckets without the matching
per-IP / global-EPS infrastructure are mostly cosmetic; an
operator who needs real rate limits will use a reverse proxy.
Also flips [options].verify_signatures default from `false` to `true`.
A relay accepting traffic from real clients should verify Schnorr
signatures, and verify-by-default closes the footgun of forgetting
the flag. The CLI gains `--no-verify` for explicit opt-out
(test fixtures, mirror replays); the old `--verify` is dropped (a
no-op now anyway).
Removed:
- quartz-relay/.../policies/MaxEventBytesPolicy.kt
- quartz-relay/.../policies/RateLimitPolicy.kt
- 7 obsolete tests in PoliciesTest + 1 in PoliciesIntegrationTest
- The matching config fields in LimitsSection
- Sample config entries in config.example.toml
- Wiring in Main.composePolicy
Added:
- Test verifySignaturesCanBeExplicitlyDisabled covering the
new explicit-opt-out path.
Total :quartz-relay tests: 59, 0 failures.
|
||
|
Claude
|
65311590f2 |
feat(relay): graceful drain on LocalRelayServer.stop
Closes the "100ms grace can drop in-flight EVENTs" gap from the
audit.
Behavioural change:
- stop() default grace is now 5 s (was 100 ms) and total timeout 10 s
(was 1 s). A SQLite write + OK reply round-trip easily fits.
- stop() first sends a NOTICE("closing: relay is shutting down —
please reconnect later") to every connected client, so well-
behaved clients can reconnect rather than hammering a dead socket.
- Active client sessions are tracked in a ConcurrentHashMap-backed
set populated by the WebSocket handler's connect/finally pair.
Exposed read-only via [activeSessionCount] so operators (and
tests) can observe lifecycle.
- stop() is now idempotent.
Tests (4 new in GracefulShutdownTest):
- activeSessionCountTracksConnectAndDisconnect — counter goes
0 → 1 on subscribe, → 0 on disconnect.
- stopSendsShutdownNoticeToActiveClients — connected client
receives a NOTICE whose message starts with "closing:".
- stopIsIdempotent — second stop() is a safe no-op.
- rawWsClientObservesNoticeBeforeServerCloses — bare OkHttp ws
client sees the NOTICE frame before the server closes the socket
(proves the order: NOTICE first, then engine.stop).
Total :quartz-relay tests: 66, 0 failures.
|
||
|
Claude
|
633d0c3c74 |
feat(relay): enforce [limits] + [authorization] config sections
Wires the parsed-but-unenforced config sections into actual relay
behavior via five new IRelayPolicy implementations under
quartz-relay/.../policies/. They compose through the existing
PolicyStack so operators stack only what they need; cheap rejection
paths run before expensive ones (rate-limit → AUTH → future/size/lists
→ signature verification).
New policies:
- KindAllowDenyPolicy — kind_whitelist + kind_blacklist
- PubkeyAllowDenyPolicy — pubkey_whitelist + pubkey_blacklist
(case-insensitive)
- RejectFutureEventsPolicy — options.reject_future_seconds
- MaxEventBytesPolicy — limits.max_event_bytes
(size of canonical NIP-01 JSON form)
- RateLimitPolicy — token-bucket per session for
messages_per_sec + subscriptions_per_min;
monotonic clock so wall-time jumps
don't reset buckets.
Plus:
- LocalRelayServer now honors limits.max_ws_frame_bytes /
limits.max_ws_message_bytes via Ktor's WebSockets.maxFrameSize.
- Main.kt builds the policy stack from RelayConfig in composePolicy().
Warning surface is reduced to only the three sections still
pending (max_subscriptions_per_session, max_filters_per_req,
network.remote_ip_header).
- PassThroughPolicy base lets each policy declare only the hook(s)
it actually enforces, keeping call sites readable.
Tests (20 new):
- PoliciesTest (16) — per-policy unit coverage for each accept/reject
boundary, including allow + deny precedence, case-insensitive pubkey
matching, deterministic rate-limit refill via injected clock, and
composition via IRelayPolicy.plus.
- PoliciesIntegrationTest (4) — end-to-end through NostrClient →
RelayHub → Relay; proves OK false comes back over the wire when
policies reject (kind blacklist, pubkey allow-list, future
timestamps, oversize events).
Total :quartz-relay tests: 62, 0 failures.
Also: bump Nip40 deleteExpiredEvents wait from 1500ms to 2500ms — the
previous margin was thin enough to flake on busy CI when the SQLite
unixepoch() rounds down across the wait.
|
||
|
Claude
|
cbe5dbe07b |
test(relay): comprehensive NIP coverage — 09, 40, 42 (success), 62
Closes the gaps from the test audit. Adds 24 new tests covering
features we advertise in NIP-11 but previously had zero coverage for,
plus regression tests for the wire-format bugs fixed earlier on this
branch.
New files:
- Nip09DeletionTest (4 tests) — deletion removes targeted events,
deletion event itself is queryable, reinsert is blocked, cross-
author deletion is ignored.
- Nip40ExpirationTest (3 tests) — expired-on-arrival events
rejected, future-expiration events stored and retrievable,
deleteExpiredEvents() purges past-expiration entries.
- Nip62VanishTest (3 tests) — vanish cascades prior events,
blocks re-insertion of older events, doesn't affect other authors.
Extended LocalRelayServerTest (+4 tests, now 9):
- nip42_successfulAuthUnlocksPublishing — full AUTH dance over
real WebSocket: relay challenge → RelayAuthenticator signs →
OK true → publishAndConfirm now succeeds.
- nip01_okMessageRoundtripWithEmptyAndNonEmptyMessage —
regression test for the OkMessage serializer bug fixed earlier.
- nip11_servesConfigDrivenInfoDoc — custom RelayInfo flows
through to the NIP-11 GET response.
- nip01_closeStopsLiveSubscription — CLOSE over the wire
actually terminates a live subscription.
Extended Nip01ComplianceTest (+5 tests, now 18):
- reqFiltersByPTag, reqFiltersByGenericSingleLetterTag (#t),
reqTagFilterValuesAreOred, reqMultipleFiltersAreOred,
multipleSubscriptionsOnOneConnectionAreIndependent.
Total: 42 tests in :quartz-relay (was 23), 0 failures.
|
||
|
Claude
|
e214143def |
feat(relay): TOML config file (--config /path/to/relay.toml)
Adds operator-facing TOML configuration to :quartz-relay, with the
section layout deliberately mirroring nostr-rs-relay's config.toml so
existing operators can port across with little churn.
Sections parsed AND enforced today:
[info] — NIP-11 doc fields (name, description, contact, pubkey,
software, supported_nips, …); replaces the previous
hardcoded RelayInfo.default()
[network] — host, port, path
[database] — in_memory toggle + file path
[options] — verify_signatures, require_auth (compose to the right
IRelayPolicy stack)
Sections parsed today but NOT YET ENFORCED (forward-compat for the
upcoming rate-limit / authorization work — relay logs a warning when
they're set):
[limits] — max_event_bytes, messages_per_sec, …
[authorization] — pubkey_whitelist/blacklist, kind_whitelist/blacklist
[options].reject_future_seconds
[network].remote_ip_header
CLI flag precedence over the config file is preserved: --host, --port,
--path, --info, --db, --auth, --verify all override the matching field.
Adds:
- cc.ekblad:4koma 1.2.0 for TOML parsing
- quartz-relay/config.example.toml as the canonical operator reference
- 5 unit tests (defaults, full parse, NIP-11 mapping, bundled example
file, optional sections)
|
||
|
Claude
|
58f6a0af6b |
fix(relay): forward ephemeral events to live subscribers (NIP-01)
LiveEventStore.query had a race window between emitting EOSE and registering as a SharedFlow collector — any event emitted in that window was lost because newEventStream has replay=0. The race was usually masked by SQLite write latency for persisted kinds, but it fired reliably for ephemeral kinds (20000-29999) where store.insert is a no-op. Per NIP-01 ephemeral events are not persisted but MUST still reach matching active subscriptions. Fixed by registering the live collector before signalling EOSE via Flow.onSubscription. Adds two tests: one proves an ephemeral event reaches an active subscriber, the other proves it isn't persisted (a follow-up REQ returns zero events). |
||
|
Claude
|
eb80dbcd15 |
feat(quartz-relay): rename + promote to a real Nostr relay
Renames :quartz-test-relay to :quartz-relay and promotes it from a test-only fixture to a real, runnable Nostr relay that just happens to also be used in tests. Two transports now share the same `Relay` core: - `RelayHub` + `InProcessWebSocket` — no socket, fastest path; ideal for unit tests inside one JVM. - `LocalRelayServer` — Ktor `embeddedServer` (CIO engine) listening on a real `ws://` port. Use for `cli` interop tests, Android instrumented tests, or running the relay standalone. NIPs implemented (all driven through production `NostrClient` in the new `LocalRelayServerTest`): - NIP-01 wire protocol (REQ/EVENT/EOSE/CLOSE) over real WebSockets - NIP-09 deletion (via existing `DeletionRequestModule`) - NIP-11 relay info doc — `RelayInfo` wraps the existing `Nip11RelayInformation` model and is served on HTTP GET when `Accept: application/nostr+json` is requested. Loadable from a JSON config file via `RelayInfo.fromFile(...)`. - NIP-40 expiration (existing `ExpirationModule`) - NIP-42 AUTH (existing `FullAuthPolicy`, opted-in via the `--auth` flag in the standalone runner) - NIP-45 COUNT - NIP-50 search via the existing FTS index - NIP-62 right-to-vanish (existing module) Adds a standalone runner: `./gradlew :quartz-relay:run --args="--port 7447 --verify"` binds the relay to a real port. Flags: --host, --port, --path, --info <file>, --db <file>, --auth, --verify. Test-only event generators moved to a clearly-named `com.vitorpamplona.quartz.relay.fixtures` package so they're still shareable with consumer tests without polluting the production API. Adds `LocalRelayServerTest` covering NIP-01/11/42/45/50 over a real loopback WebSocket. Existing 11-test `Nip01ComplianceTest` (in-process transport) continues to pass. |