perf(geode): adaptive outQueue + CIO pool sizing for 10k+ connections
Implements geode/plans/2026-05-07-connection-scaling.md to push the
single-relay connection ceiling past the ~2k floor measured by
LoadBenchmark.connectionsHeldOpen.
- WebSocketSessionPump: switch outQueue to Channel.UNLIMITED bounded
by an AtomicInteger backlog cap. Idle connections no longer reserve
an 8 192-slot fixed buffer; lazy-allocated head segments cost only
a few hundred bytes per idle session. Slow-client policy preserved:
once outstanding > MAX_OUTGOING_BUFFER, the queue is closed and
the connection drops, so NIP-01 ordering is never silently
violated.
- LocalRelayServer / RelayConfig.NetworkSection: expose CIO
connectionGroupSize / workerGroupSize / callGroupSize so big-VM
operators can tune Ktor's event-loop pools without forking. Switch
to the serverConfig {} + configure {} embeddedServer overload so
CIO tunables can be set.
- LoadBenchmark: add connectionsHeldOpen10k (asserts 10 000 idle
WebSockets settle under a 1 GiB heap ceiling) and
connectionsHeldOpenWithFanout (5 000 subscribers, 10 EPS fan-out
for 10 s, reports p50/p99 last-fanout latency).
- config.example.toml: document the three CIO tunables and the
ulimit -n requirement for operators targeting >1k connections.
This commit is contained in:
@@ -25,6 +25,20 @@ contact = "admin@example.com"
|
||||
host = "0.0.0.0"
|
||||
port = 7447
|
||||
path = "/"
|
||||
# Ktor CIO event-loop pool sizing. Leave commented-out for sensible
|
||||
# per-CPU defaults (typical for <2k concurrent connections). Lift on
|
||||
# big-VM deployments targeting 10k+ connections — over-threading at
|
||||
# low connection counts hurts L1/L2 cache locality, so always
|
||||
# benchmark before/after when tuning these.
|
||||
#
|
||||
# Operators targeting >1k concurrent WebSockets should also raise the
|
||||
# OS file-descriptor limit: `ulimit -n 65536` (or higher) before
|
||||
# launching, plus a matching `LimitNOFILE=` in any systemd unit. The
|
||||
# default of 1024 on most distros caps the relay well below 1k FDs
|
||||
# (one per WS plus DB and listening sockets).
|
||||
# connection_group_size = 4
|
||||
# worker_group_size = 16
|
||||
# call_group_size = 64
|
||||
|
||||
[database]
|
||||
# True keeps an in-memory SQLite db (events vanish on restart). Useful
|
||||
|
||||
Reference in New Issue
Block a user