Claude
2c0ad4fbf5
Four sketches, queued by impact, each grounded in current code paths and observed benchmark numbers: - event-ingestion-batching: SQLite group commit + EVENT pipelining + off-thread Schnorr verify. Targets 5–10× EPS on a fast SSD. - live-broadcast-fanout-index: indexed filter matching to replace the O(N_subs × N_filters) per-event walk in LiveEventStore. Targets flat fanout p99 up to high subscriber counts. - connection-scaling: shrink the per-session outQueue footprint (currently the dominant per-conn cost), tune Ktor CIO group sizes, reduce JSON parse allocations. Targets 10 000+ concurrent conns. - negentropy-large-corpus: id-and-time-only snapshot path so NEG-OPEN on a 5M-event store doesn't materialise full Event objects, plus bounded-window defaults and concurrent-session caps. Each plan names the verification benchmark to add. Plans are queued, not committed work — README orders them by expected impact.
94 lines
4.2 KiB
Markdown
94 lines
4.2 KiB
Markdown
# Connection scaling: pushing past 2 000
|
||
|
||
## Problem
|
||
|
||
Current measurement (`LoadBenchmark.connectionsHeldOpen`): **~2 000
|
||
concurrent connections** before file-descriptor pressure / Ktor CIO
|
||
event-loop saturation. Real-world relays (e.g. nostr.wine, nos.lol)
|
||
sustain 10–30k. Geode shouldn't be the bottleneck for an Amethyst-
|
||
adjacent operator who scales beyond a thousand-user community.
|
||
|
||
## What's spending memory per connection today
|
||
|
||
| Cost | Per connection | At 5 000 conns |
|
||
| ----------------------- | -------------------------------------------------------- | -------------- |
|
||
| `outQueue` Channel | 8 192 string slots × ~8 b ref | ~320 MB pinned |
|
||
| `RelaySession` | `LargeCache<String, Job>` for subs (likely 1–10 entries) | ~negligible |
|
||
| `NegSessionRegistry` | `HashMap<String, NegentropyServerSession>` — usually 0 | ~negligible |
|
||
| Ktor CIO buffers | TCP read + write buffers | ~10 MB |
|
||
| Per-session writer Job | one coroutine | ~few KB |
|
||
|
||
The `outQueue` reservation is the dominant cost. The 8 192 was sized
|
||
for a worst case "thousands of subscriptions, one event matches all" —
|
||
but at 5 000 connections we've over-provisioned by ~300 MB just on
|
||
the channel array, even though most connections never fan out.
|
||
|
||
## Sketch
|
||
|
||
### A — adaptive outQueue capacity
|
||
|
||
Start every connection with `INITIAL_OUTGOING_BUFFER = 64`. When the
|
||
producer side trySends and we observe queue depth crossing a high-water
|
||
mark (e.g. 75% full), grow the channel up to `MAX_OUTGOING_BUFFER =
|
||
8192`. This is not how `kotlinx.coroutines.channels.Channel` is
|
||
structured (capacity is fixed at construction), so the implementation
|
||
is "swap in a wider channel under a per-session lock when watermark
|
||
trips" — drains the old, then routes new sends through the new.
|
||
|
||
Expected: 90% of connections never fan out, so they stay at 64 slots
|
||
× ~512 B per ref ≈ 32 KB. At 5 000 conns that's ~160 MB → ~5 MB.
|
||
Hot-fanout connections still get the 2 MB cap.
|
||
|
||
### B — per-relay event-loop pool sizing
|
||
|
||
Ktor CIO defaults to one event-loop thread per available CPU.
|
||
Beyond a few thousand connections, this becomes the bottleneck — and
|
||
none of geode's per-connection work is CPU-bound (it's mostly waiting
|
||
on incoming frames). Tune CIO via:
|
||
|
||
```kotlin
|
||
embeddedServer(CIO, ...) {
|
||
connectionGroupSize = max(2, Runtime.getRuntime().availableProcessors() / 2)
|
||
workerGroupSize = max(4, Runtime.getRuntime().availableProcessors())
|
||
callGroupSize = max(8, Runtime.getRuntime().availableProcessors() * 4)
|
||
}
|
||
```
|
||
|
||
Expose these through `RelayConfig.NetworkSection` so an operator on a
|
||
big VM can lift them.
|
||
|
||
### C — reduce per-message JSON allocations
|
||
|
||
`OptimizedJsonMapper.fromJsonToCommand` allocates a `JsonNode` tree per
|
||
incoming frame. At 10k connections with 1 msg/s each that's 10k tree
|
||
allocations/sec. Investigate streaming Jackson + reusing `ObjectMapper`
|
||
per session, or using kotlinx-serialization's lower-overhead path.
|
||
|
||
This is more of a quartz-level change than geode-specific, but
|
||
geode's load benchmark is the right place to measure it.
|
||
|
||
## How to verify
|
||
|
||
Add to `geode.perf.LoadBenchmark`:
|
||
|
||
- `connectionsHeldOpen10k` — opens 10 000 idle WebSocket connections;
|
||
asserts no FD exhaustion + RSS stays under 1 GB.
|
||
- `connectionsHeldOpenWithFanout` — 5 000 idle subscribers,
|
||
10 EPS published; measures p99 fanout latency at scale.
|
||
|
||
The current `connectionsHeldOpen` benchmark stays as the baseline
|
||
floor (~2 000 conns).
|
||
|
||
## Risks
|
||
|
||
- **Adaptive channel swap is fiddly**: drains under the producer's nose
|
||
must preserve OK ordering. A simpler alternative: keep capacity fixed,
|
||
but lazily allocate a small `ArrayDeque<String>` only when the first
|
||
message is sent. Channels in kotlinx.coroutines do allocate up-front.
|
||
- **Bumping CIO group sizes can hurt**: more threads can mean worse
|
||
L1/L2 locality. Always benchmark before/after, don't trust
|
||
intuitive sizing.
|
||
- **OS-level FD limit**: per-process FD limit on Linux defaults to
|
||
1024 in many environments. Document the `ulimit -n` requirement
|
||
for operators targeting >1k connections.
|