1. Split AudioPlayer.start() into allocate + beginPlayback to restore
the synchronous DeviceUnavailable error path that the previous
pre-roll fix collapsed.
- AudioPlayer gains `beginPlayback()` (default no-op) so test
fakes / desktop sinks aren't forced to grow a method they
don't need.
- AudioTrackPlayer.start() now allocates the AudioTrack + audio-
priority writer thread but does NOT call AudioTrack.play();
beginPlayback() flips the device into the playing state.
AudioTrack in MODE_STREAM explicitly supports write() before
play() per the platform docs, which is exactly the contract
pre-roll wants.
- NestPlayer.play() now calls player.start() synchronously
(caller catches DeviceUnavailable + rolls back the slot like
before) and defers beginPlayback() until pre-roll fills.
2. MediaCodecOpusDecoder: drain output + retry input dequeue before
dropping a frame. The prior 10 ms input-buffer dequeue followed
by an unconditional `return ShortArray(0)` turned every transient
stall (thermal throttling, GC pause, output not yet pulled) into
a 20 ms audio gap. Now we drain whatever output is ready —
freeing input slots — then retry input dequeue with a 5 ms
timeout. Only after both misses do we drop the packet, and even
then we return any output samples that the drain produced so
the player doesn't underrun on the back of one tight cycle. The
decoder's drain logic is extracted into a `drainAvailableOutput`
helper so both the pressure-relief path and the post-queue main
drain share it.
3. ReconnectingNestsListener: exponential backoff for the opener-
throws retry path. Replaces the flat 1 000 ms `SUBSCRIBE_RETRY_BACKOFF_MS`
with 250 → 500 → 1 000 ms, reset on first successful subscribe.
The 250 ms floor is well under moq-rs's typical announce-
propagation latency (< 200 ms), so a subscribe-before-announce
miss usually recovers fast enough that the wrapper SharedFlow's
~1.3 s buffer hides the gap entirely.
1. Listener-side pre-roll + bigger playback buffer + audio-priority thread.
- NestPlayer buffers 5 decoded frames (~100 ms) before starting
the AudioPlayer, masking the first-frame underrun that fires
whenever Compose / GC briefly stalls Main.
- AudioTrackPlayer sizes the AudioTrack at max(minBuffer*16, 250 ms)
instead of minBuffer*4 (~80 ms) and writes via a per-instance
audio-priority single-thread executor (Process.THREAD_PRIORITY_AUDIO)
instead of Dispatchers.IO, so WRITE_BLOCKING never contends with
unrelated IO work.
2. MoqLiteSession.subscribe: hoist response typeCode out of the
collect lambda. readVarint advances pos permanently while
readSizePrefixed only rolls back its own length-varint, so a
chunk-split between type and body would re-read the body's size
prefix as the type code on the next chunk and misframe the
response. Mirrors the same fix already in handleInboundBidi.
3. MoqLiteSession.subscribe: register the subscription in the map
BEFORE writing the SUBSCRIBE bytes on the wire. The relay can
open the first group's uni stream before our continuation
re-enters [state] to register; if so, drainOneGroup looked the
id up against an empty map and silently dropped the frame
(~1 frame / 20 ms gap on first attach). Wrap the post-register
writes so a transport-failure unwinds the orphan registration.
4. Hot-swap moq-lite publisher across JWT-refresh boundaries.
- NestMoqLiteBroadcaster: publisher is now @Volatile + supports
swapPublisher(); capture loop snapshots the reference per frame
and resets framesInCurrentGroup on swap.
- MoqLiteNestsSpeaker implements a new internal
HotSwappablePublisherSource interface that exposes
openPublisherForHotSwap(track) without spinning up a broadcaster.
- ReissuingBroadcastHandle keeps a single long-lived broadcaster
across session recycles when the speaker supports hot swap;
legacy IETF / fake speakers fall back to close-then-restart.
- connectReconnectingNestsSpeaker.orchestrator hoists the old-
session close onto a sibling launch so the wrapper can swap
the publisher into the broadcaster on the new session before
the old session's WebTransport drops. Eliminates the previously-
accepted 50–150 ms audible silence at every JWT refresh.
After drafting the congestion-control plan we concluded the audio-rooms
workload doesn't actually need CC — speakers push ~8 KB/sec, which
never fills any modern link's capacity. The one real concern that
surfaced — STREAM retransmit wasting bandwidth on stale Opus frames
on lossy uplinks — is much cheaper to fix directly than to bound via
a 14-test CC subsystem.
SendBuffer gains a `bestEffort: Boolean = false` constructor flag.
When true, markLost drops the lost ranges instead of moving them to
the retransmit queue and lets the underlying byte storage compact as
if the bytes had been ACK'd. The FIN flag (if covered) also stays
sent — best-effort skips FIN re-emission too. The peer may end up
with a truncated stream; moq-lite's per-stream timeouts handle that.
Plumbed through QuicStream → QuicConnection.openUniStream(bestEffort)
→ QuicWebTransportSessionState.openUniStream(bestEffort) →
WebTransportSession.openUniStream(bestEffort). Default is false
everywhere, so reliable streams (HTTP/3 control, moq-lite SUBSCRIBE
bidi, etc.) keep RFC 9000 §3.5 semantics.
MoqLiteSession.openGroupStream now passes `bestEffort = true` —
group streams carry a single Opus packet, are real-time, and don't
benefit from retransmit.
Internal cleanup: `removeOverlap`'s `ackedNotLost: Boolean` parameter
became `OverlapAction { ACK, RETRANSMIT, DROP }` so the third best-
effort disposition has a name. Same code paths, same tests, just
clearer at the call site.
CC plan (quic/plans/2026-05-05-congestion-control.md) is updated to
"parked indefinitely" with a note that this commit is the lighter-
weight alternative that addresses the only practical concern. The
plan is preserved as a reference if a future workload justifies CC.
New tests: SendBufferBestEffortTest (6 cases — reliable baseline,
best-effort drops, FIN drop in best-effort mode, partial overlap,
idempotent stale loss, ACK path still works).
https://claude.ai/code/session_01PYYez8a6sjiakyjAxsfCEQ
Two changes once the cliff is confirmed sidestepped:
1. Raise initialMaxStreamsUni from 10 000 → 1 000 000.
At framesPerGroup=5 with 20 ms Opus frames the relay opens ~10
uni streams/sec to a listener. The half-window threshold check
(`count + initialMaxStreamsUni/2 >= advertisedMaxStreamsUni`)
now trips at count=500 000 ≈ 13.9 hours of continuous audio.
For any realistic Nest the rolling MAX_STREAMS_UNI extension
path — which is what tripped the moq-rs cliff — is dormant.
Memory cost: QuicConnection.streams grows for the connection's
lifetime (no removal in current model), so 2 hours costs ~72k
stream entries. Per-stream overhead is small enough that this
is tolerable for an audio-room workload; bounded growth is a
known follow-up.
2. Strip the high-frequency diagnostic logs that were added during
investigation. Production keeps:
- SUBSCRIBE_DROP (rare error)
- MAX_STREAMS_UNI / MAX_STREAMS_BIDI emit (should never fire
in normal operation now; if it does, we want to know)
- pumpUniStreams / pumpInboundBidis ended (pump death)
- announce / subscribe bidi.incoming() exception path
- ReconnectingHandle.opener throw + retry
Stripped:
- per-stream "transport delivered uni stream #N"
- per-group "uni grpHdr id=N seq=M"
- per-group "openGroup seq=N keyedOnSubId=…"
- per-25-stream peerInitiatedUniCount milestone
- per-chunk "subscribe id=N: bidi chunk #M"
- per-update RoomAnnouncement
- 5-second QuicWebTransportSession flow-control snapshot ticker
- "VM.openSubscription ->/<- subscribeSpeaker"
- "VM.onSpeakerActivity FIRST frame"
- "broadcaster: send accepted (subscriber attached)"
- "broadcaster: publisher.send returned false (no inbound subscriber)"
- "first inbound subscriber attached"
- "ignoring inbound SUBSCRIBE id=N track=catalog.json"
- "publish suffix=…"
- "transport delivered inbound bidi #N"
- "inbound AnnouncePlease prefix=…"
- "inbound SUBSCRIBE id=… track=…"
- "inbound SUBSCRIBE FIN'd: removing id=…"
The diagnostic logs can be re-added behind a debug flag later if
we need to chase a different regression.
Confirmed durable for at least 15 s of continuous audio against
nostrnests.com production with the prior 10 000-cap fix; bumping to
1 000 000 expands the headroom to multi-hour broadcasts without any
new code path firing.
https://claude.ai/code/session_01PYYez8a6sjiakyjAxsfCEQ
Listener cliffs at uni stream #61 even though MAX_STREAMS_UNI(150)
was emitted at count=50 — and the cliff number is variable across
runs (124 in one trace, 61 in another), which strongly suggests the
limiter isn't the stream-id cap any more. Need visibility into what
QUIC flow-control state looks like at the moment streams stop.
QuicWebTransportSession now optionally takes a parentScope and, when
provided, launches a daemon coroutine that calls
QuicConnection.flowControlSnapshot() every 5 s. The snapshot dumps
the fields the cliff-investigation plan
(nestsClient/plans/2026-05-01-quic-stream-cliff-investigation.md)
identified as smoking-gun candidates:
- peerInitiatedUniCount + advertisedMaxStreamsUni (stream-id cap)
- peerInitMaxStreamsUni (peer's view of our cap, from handshake)
- sendCredit + consumed (connection-level data flow control)
- pendingBytes / pendingStreams (anything stuck in send buffers)
- udpRecvDatagrams + udpRecvBytes (raw socket-level reception)
The factory passes its parentScope into the session so the snapshot
job dies cleanly with the same supervisor that owns the driver.
QuicWebTransportFactory tests and SendTraceScenario don't pass a
scope and won't see the periodic logger.
Filter `adb logcat -s NestQuic:D` and look for "snapshot peerInitiatedUni=…".
At cliff time the snapshot will print whichever counter is wedged.
https://claude.ai/code/session_01PYYez8a6sjiakyjAxsfCEQ
When a listener joins a Nest before the speaker starts publishing,
moq-rs FINs the SUBSCRIBE bidi immediately without sending
SubscribeOk or SubscribeDrop. MoqLiteSession.subscribe surfaces that
as a MoqLiteSubscribeException("subscribe stream FIN before reply"),
which the listener wrapper translated to MoqProtocolException via
MoqLiteNestsListener.wrapSubscription.
The reconnecting wrapper's inner re-issue loop then did:
val handle = try { opener(listener) } catch (...) { null } ?: break
— breaking out of the loop forever. The outer collectLatest only
re-runs on listener swap (session reconnect), so once the first
SUBSCRIBE failed the audio path stayed dead until the user manually
disconnected and reconnected. With the typical join-order being
"listener taps Join before speaker taps Start", this hit nearly
every nest.
The kdoc on pumpAnnounceWatch claims "moq-lite supports subscribe-
before-announce, so a subscribe issued during the gap … attaches
cleanly when the new publisher comes up" — true for some publisher-
cycle gaps mid-broadcast, but verifiably false for the cold-start
case where the publisher hasn't existed yet on the relay's view of
the namespace. Production trace (commit 283e776):
14:23:29.556 subscribe id=0 track='audio/data': SUBSCRIBE bytes flushed
14:23:29.597 subscribe id=0: bidi closed BEFORE any response parsed
14:23:29.617 ReconnectingHandle.opener threw MoqProtocolException — pump breaks
14:23:33.604 announce update status=Active suffix=… (4 s later)
14:23:34.218 publish suffix=… (speaker starts publishing)
Fix: instead of `break`, treat opener-throw as a transient failure
and retry after SUBSCRIBE_RETRY_BACKOFF_MS (1 s). The outer
collectLatest still cancels the inner loop on listener swap, and
unsubscribeAction.cancel() still tears the pump down on consumer
release, so the retry loop is bounded by both the session and the
caller. 1 s is well over moq-rs's typical announce-propagation
window (< 200 ms in traces) and short enough that the listener
attaches within a second of the speaker actually publishing.
https://claude.ai/code/session_01PYYez8a6sjiakyjAxsfCEQ
Listeners stuck on a spinning speaker avatar with no audio (against
both Amethyst-hosted and nostrnests.com-hosted Nests). The trace
captured on a working speaker phone showed every group keyed to
subId=1 / track='catalog.json' even though the broadcaster was
pumping Opus frames:
13:21:27.570 inbound SUBSCRIBE id=1 track='catalog.json'
13:21:27.574 inbound SUBSCRIBE id=0 track='audio/data'
13:21:27.609 openGroup seq=0 keyedOnSubId=1 track='catalog.json'
13:21:27.610 broadcaster: send accepted (subscriber attached)
... every subsequent group keyed=1, every audio frame on the
catalog stream
Root cause: `PublisherStateImpl.openNextGroupLocked` keyed each uni
stream off `inboundSubs.first()` with no track filter. The kdoc
asserted "Inbound subscription set is expected to be small (1 in
nests's listener-per-room model)" — that was true when listeners
only opened the audio sub. Once the listener side started opening a
catalog sub alongside (commits ~Apr 2026), whichever SUBSCRIBE
arrived first (typically the catalog one, by ~4 ms in this trace)
became the routing target for all audio frames. The relay forwarded
those frames to the listener with subscribeId=catalog, the listener
routed them to its catalog handler, RoomSpeakerCatalog.parseOrNull
returned null (it's not JSON), and the audio subscription's frames
channel never received anything — so onSpeakerActivity never fired
and the spinner stayed forever.
Fix: per moq-lite Lite-03, a publisher is responsible for one
(broadcast, track) tuple. Thread `track` through `MoqLiteSession.publish`
and have `PublisherStateImpl.registerInboundSubscription` reject
inbound SUBSCRIBEs whose track doesn't match — those still receive
SUBSCRIBE_OK from the bidi handler (best-effort per Lite-03's
optional-content semantics for catalog) but don't influence the
audio routing. `MoqLiteNestsSpeaker.startBroadcasting` passes
`track = MoqLiteNestsListener.AUDIO_TRACK` (= "audio/data").
Test callsites (MoqLiteSessionTest, NostrnestsProdAudioTransmissionTest,
NostrNestsSustainedSendOutcomesInteropTest) updated to pass the new
required parameter; all use track="audio/data" matching what they
exercise.
The diagnostic logs from the prior commits stay in — they're how we
caught this and they'll catch the next one. They can be stripped in
a follow-up once the fix is confirmed in the field.
https://claude.ai/code/session_01PYYez8a6sjiakyjAxsfCEQ
Round-2 instrumentation after phone-B trace showed silence between
SUBSCRIBE_OK-expected and the first audio frame. Three additions:
- MoqLiteSession.subscribe: log the SUBSCRIBE-bytes-flushed transition,
every chunk arriving on the response bidi (chunks=N), and any
exception or natural close on bidi.incoming() that previously
swallowed the cause.
- MoqLiteSession.announce: same treatment for the AnnouncePlease bidi
— chunks=N, natural-close, and exception path now visible.
- ReconnectingNestsListener.reissuingSubscribe: previously had a
`catch (_: Throwable) { null }` black hole on the re-issue pump
(line 354); now logs the throwable's class + message before the
pump breaks. Also logs natural-close of handle.objects so a
publisher cycle vs a real failure are distinguishable in the trace.
These narrow the listener-stuck-on-spinner diagnosis from
"something downstream of subscribe never fires" to which exact step
inside `MoqLiteSession.subscribe` is silent: the write, the response
read, or the framing.
https://claude.ai/code/session_01PYYez8a6sjiakyjAxsfCEQ
Splits the diagnosis into "QUIC delivered streams" vs "moq routed
them". Without this seam log, a silent NestRx trace after SUBSCRIBE_OK
gives no signal whether the relay is forwarding (and :quic isn't
delivering streams to the moq pump) or moq is dropping frames.
Also surfaces the prior `_: Throwable` swallow in both pumps — those
hid transport-close exceptions which are exactly the kind of clue
we want when the listener silently stalls.
https://claude.ai/code/session_01PYYez8a6sjiakyjAxsfCEQ
Diagnostic instrumentation to localise the listener-stuck-on-spinner bug.
None of these are intended for merge — pair logs with a logcat capture,
diagnose, then revert.
Receiver (tag NestRx):
- MoqLiteNestsListener: log subscribeSpeaker / subscribeCatalog entry
and per-update RoomAnnouncement emission to the VM
- MoqLiteSession.subscribe: log SUBSCRIBE_OK / SUBSCRIBE_DROP outcomes
- MoqLiteSession.drainOneGroup: log every uni group header decode
and warn on subscription-lookup miss (frame dropped)
- MoqLiteSession.pumpAnnounceWatch: log every announce update and
flag the Ended branch that closes the subscription's frames
- NestViewModel: log VM.openSubscription wiring and the first-frame
trigger that clears the connectingSpeakers spinner
Speaker (tag NestTx):
- MoqLiteSession.publish: log entry suffix
- MoqLiteSession.handleInboundBidi: log inbound AnnouncePlease and
SUBSCRIBE dispatches plus FIN cleanup
- MoqLiteSession PublisherStateImpl: log first inbound subscriber
attach and every group-stream open
- NestMoqLiteBroadcaster: log when publisher.send returns false
(no inbound subscriber on relay) and the subscriber-attached
transition, plus surface throws that runCatching previously
swallowed only into onError
Capture with `adb logcat -s NestRx:D NestTx:D`.
https://claude.ai/code/session_01PYYez8a6sjiakyjAxsfCEQ
Wire the local user's avatar into the existing speaking-ring animation so
the host/speaker sees the same green ring + amplitude glow on their own
cell that remote speakers already get.
Ground truth is the broadcaster's `publisher.send(opus)` success: the new
`onLevel` callback fires only after a frame actually leaves on the wire,
computed from the raw PCM peak (shared `peakAmplitude` util used by both
the player decode loop and the broadcaster capture loop). This means the
animation reflects what the relay sees, not any UI-side mute / role
state that could be stale or buggy — if frames are flowing, the ring
plays.
Plumbing: `NestsSpeaker.startBroadcasting` gains an optional `onLevel`
that the moq-lite + IETF broadcasters both honour, the reconnecting
wrapper replays it on every session reissue (alongside the existing
mute-intent replay), and `NestViewModel.startBroadcast` hands it a
lambda that calls the existing `onSpeakerActivity` / `onAudioLevel`
plumbing keyed on the local pubkey. The 250 ms speaking-timeout fades
the ring naturally when the user mutes or stops broadcasting.
`AudioTrackPlayer` was constructing its `AudioTrack` with
`USAGE_VOICE_COMMUNICATION` + `CONTENT_TYPE_SPEECH` so the room would
behave like a phone call (volume rocker controls call volume, ducks
notifications). The `AudioTrack` then renders on `STREAM_VOICE_CALL`,
which Android only services audibly while the device is in
`MODE_IN_COMMUNICATION`. Nests never drives `AudioManager.mode` (only
NIP-100's `CallAudioManager` does), so on most devices the playback
either dropped to the earpiece at near-zero volume or produced no audio
at all — making rooms appear silent on both phones.
Fix: route through `USAGE_MEDIA` + `CONTENT_TYPE_SPEECH` (=
`STREAM_MUSIC`) so audio-room playback comes out of the loudspeaker by
default and the volume rocker controls it any time. Same approach
Twitter/X Spaces and Clubhouse take for hands-free audio rooms. Echo
cancellation still works on the capture side via
`MediaRecorder.AudioSource.VOICE_COMMUNICATION` regardless of the
playback usage.
Also update `NestForegroundService.requestAudioFocus` to request focus
under the matching `USAGE_MEDIA` attributes so the focus claim and the
playback attributes line up.
ScriptedSpeaker publishes startCount > 0 from inside startBroadcasting,
so polling startCount alone races the wrapper pump's subsequent
`if (desiredMuted) handle.setMuted(true)` step. Under load (full suite
run on CI) the assertion fired before the pump replayed mute intent,
producing setMutedCalls=0. Wait for the post-condition the assertion
checks instead.
- NestsTokenResponse.parse catch list now includes IllegalStateException
so a malformed escape from a misbehaving auth server surfaces as
NestsException instead of crashing the listener.
- OkHttpNestsClient gains a configurable callTimeoutMs (default 90 s)
enforcing an upper bound on the entire mintToken round-trip including
retries. Without it a stalled server suspends the reconnect
orchestrator indefinitely (the orchestrator's openOnce step parks on
mintToken). 90 s leaves headroom over the worst-case 63 s 429
retry chain documented in MAX_RATE_LIMIT_RETRIES.
- parseEndpoint tightens IPv6 bracket check from `closeBracket > 0` to
`> 1`, rejecting the empty `[]` literal.
- NestBroadcaster + NestMoqLiteBroadcaster gain an `onTerminalFailure`
callback that fires once when the consecutive-send-error guard bails.
MoqLiteNestsSpeaker wires this to flip the speaker state to Failed,
giving ReconnectingNestsSpeaker the signal it needs to recycle the
session — without this hook the broadcaster bailed silently and the
outward speaker state stayed on Broadcasting forever.
Adds regression test:
- onTerminalFailure_fires_once_after_consecutive_send_failures
225 tests pass, 0 failures. Android target compiles clean.
A fresh audit caught a real race in the round-1 subscribe() rewrite plus
several distinct bugs in the Android MediaCodec layer.
**MoqLiteSession.subscribe() race regression** — the launched collector
that reads the SubscribeResponse and watches for peer FIN ran cleanup
(remove from subscriptionsBySubscribeId, close frames) before subscribe()
itself reached the post-await registration. If the publisher FIN'd
immediately after sending Ok, the collector exited first against an
empty map; subscribe() then registered the subscription, leaving frames
in the map with no live collector to ever close it on transport drop.
Consumer hung forever — exactly the failure mode round-1 fix#3 was
supposed to prevent. Fix: pre-register the subscription BEFORE launching
the collector. Drop unused `ok` field from ListenerSubscription.
**MoqLiteNestsSpeaker.startBroadcasting publisher leak** — if
session.publish() succeeded but broadcaster.start() then threw (mic
permission denied, AudioRecord allocation failure), the publisher was
never closed and stayed registered as session.activePublisher,
permanently blocking subsequent startBroadcasting calls.
**runCatching{suspend close} swallowing CancellationException** —
MoqLiteNestsSpeaker.close, MoqLiteBroadcastHandle.close,
MoqLiteNestsListener.close all wrapped suspending closes in
runCatching, breaking structured cancellation when the parent scope
cancelled teardown. Replaced with explicit cancel-rethrowing try/catch.
**MediaCodecOpusDecoder ArrayList<Short> boxing on every audio frame**
— at 50 fps × 960 samples × N speakers ≈ 48 000 boxed Short
allocations/sec/speaker on the audio hot path. Rewritten to write
directly into a pre-sized ShortArray via ShortBuffer.get(dst, off, len).
**MediaCodecOpusEncoder bugs**
- KEY_AAC_PROFILE = AACObjectLC was being set on an audio/opus
encoder. Meaningless for Opus; stricter Codec2 stacks on Android
13+ reject the configure() call with IllegalArgumentException and
surface as DeviceUnavailable. Removed.
- The drain loop's INFO_OUTPUT_FORMAT_CHANGED branch had no progress
guard. A buggy encoder re-emitting FORMAT_CHANGED without producing
output would busy-spin against the 10 ms dequeue timeout. Now
absorbed at most once per encode call. Same guard added to the
decoder.
Adds regression test:
- frames_flow_completes_when_peer_FINs_immediately_after_Ok
224 tests pass, 0 failures. Android target compiles clean.
- NestsReconnectPolicy gains a `jitter` parameter (default 0.3, AWS-
style equal jitter). Without it, every client reconnecting after
a relay restart retries on the identical 1s/2s/4s/… schedule and
thunders the relay during recovery. delayForAttempt also accepts
an explicit Random source for deterministic tests.
- MoqLiteFrameBuffer decouples capacity from size so power-of-two
growth actually amortises (the old shape allocated a fresh
ByteArray then truncated capacity back to `needed` per chunk,
defeating doubling). Adds a guard against varint reads past the
live region into uninitialised slack capacity.
- prefixWithType inlines the size-prefix wrap so a publisher
SubscribeOk/Drop reply doesn't allocate three ByteArrays.
- MoqLiteSubscribeOk gains the same `init { require(priority in 0..255) }`
bounds check its sibling MoqLiteSubscribe has — a buggy caller
building a malformed Ok reply now fails loudly instead of writing
a truncated byte on the wire.
- NestBroadcaster + NestMoqLiteBroadcaster track consecutive
publisher.send exception count and bail after 250 (~5 s at 50 fps)
instead of holding the mic open forever on a permanently dead
transport. publisher.send returning `false` (no inbound
subscriber) is NOT counted — empty rooms are a normal state.
Adds 8 regression tests:
- 5 in MoqLiteFrameBufferTest (multi-chunk reads, back-to-back
payloads, growth amortisation, compact, varint past-live guard)
- 3 in NestsReconnectPolicyTest (jitter spread band, jitter=0
determinism, jitter=1 collapses to 0..base)
223 tests pass, 0 failures.
- Unknown IETF control message types now skip just the unknown frame
instead of dropping the entire merge buffer (a peer sending a
draft-17 message we don't enumerate — FETCH, GOAWAY, MAX_SUBSCRIBE_ID
— would otherwise wedge the pump). Adds MoqUnknownTypeException
carrying bytesConsumed so runControlPump can advance past it.
- pumpUniStreams + pumpInboundBidis wrap their inner collect in
coroutineScope so per-stream drains are children of the pump's job
(was: launched on the outer scope as siblings, leaking past
cancelAndJoin until the transport's own flow errored out).
- parseEndpoint handles IPv6 authorities ([::1], [2001:db8::1]:4443).
Naive lastIndexOf(':') used to find a colon inside the address.
- buildRelayConnectTarget percent-encodes the namespace path so a
malicious / careless `d` tag containing `?`, `#`, `&`, ` ` etc.
can't truncate the URL and shove the JWT into the wrong slot.
- Reconnect orchestrators (listener + speaker) replace
`runCatching { openOnce / close / startBroadcasting }` with explicit
try/catch that rethrows CancellationException, so cooperative
cancellation from the parent scope dies promptly instead of running
one more iteration after the cancel.
Adds three regression tests:
- parseEndpoint_handles_IPv6_authorities
- buildRelayConnectTarget_percent_encodes_path_unsafe_chars
- unknown_message_type_throws_typed_exception_with_full_frame_size
215 tests pass, 0 failures.
- send() now nulls currentGroup and returns false on uni-stream write
failure (was: silently returned true while reusing the dead stream)
- Subscribe bidi gets a single long-running collector that closes the
frames channel on peer FIN / transport tear-down (was: consumer
could hang forever on a black-holed UDP path with no Announce(Ended))
- AudioException.Kind gains EncoderError; mic-side encode failures no
longer mis-route through DecoderError handlers
- MoqCodec.decode bounds the payload-length varint before .toInt() so
a peer-driven absurd length can't wedge the parser forever
- Publish hot-path framing collapses to a single ByteArray allocation
(was: Varint.encode + plus operator, two allocs per Opus frame at
50 fps)
- Drops the now-unused readSubscribeResponseFromBidi /
readSizePrefixedFromBidiInto / EarlyExit helpers
Adds a regression test that asserts the frames flow completes when
the peer FINs the subscribe bidi.
213 tests pass, 0 failures.
The moq-lite Lite-03 spec puts subscriber-side group-staleness control in
max_latency on the SUBSCRIBE frame: 0 = unlimited (relay falls back to its
MAX_GROUP_AGE = 30 s default); a positive value tells the relay to evict
groups older than that in favour of newer ones during transient backpressure.
Plumb it through:
NestsListener.subscribeSpeaker(pubkey, maxLatencyMs = 0L)
└─ MoqLiteNestsListener → MoqLiteSession.subscribe(.., maxLatencyMillis)
└─ DefaultNestsListener (IETF) ignores — no equivalent on that wire
└─ ReconnectingHandle re-issues across reconnects
Default stays at 0L for back-compat with the JS reference watcher; callers
opt in for low-latency live audio.
Also rewrites nestsClient/plans/2026-05-01-quic-stream-cliff-investigation.md
with corrected framing: of the four "upstream issues" the prior draft listed,
only our own missing MAX_STREAMS_UNI emission was a real bug (RFC 9000 §4.6
violation, fixed in d391ae1d). The relay's per-subscriber queue policy,
unbounded FuturesUnordered, no-timeout open_uni().await, and lack of
lagging-consumer detection are spec-permitted architectural decisions in a
gap moq-lite explicitly leaves to implementations — the spec puts staleness
control on the subscriber via max_latency, which we were sending as 0 the
whole time. Reframed as "feature request worth filing upstream", not "bugs".
Previous status note speculated about a per-subscriber forwarding ceiling.
This pins it down with file:line citations from the production version of
kixelated/moq so the upstream issue we file (or the next person to debug
this) has the exact code paths in hand:
1. Unbounded per-track group queue (moq-lite/src/model/track.rs:69-90),
evicted only by 30 s age (track.rs:29).
2. serve_group's session.open_uni().await has no timeout
(moq-lite/src/lite/publisher.rs:317-379) — blocks forever when the
subscriber's CWND collapses.
3. Publisher's FuturesUnordered task pool is unbounded
(publisher.rs:325, 346) — keeps spawning blocked tasks with no
backpressure to upstream.
4. max_concurrent_uni_streams = 10000 (moq-native/src/quinn.rs:30-33)
so the symptom is NOT the QUIC stream-id cap; it's the cascade
above.
5. No lagging-consumer RESET/STOP_SENDING anywhere; the relay never
gives up on a stuck subscriber.
The framesPerGroup=5 mitigation we shipped sidesteps the cascade by
keeping upstream rate (10 streams/sec) below the threshold any of those
gaps would accumulate at. It doesn't fix the underlying issues — those
are upstream to address.
Round-2 production sweep confirmed the residual sustained-broadcast
loss is on the relay side: every uni stream the relay opened to the
listener delivered cleanly through QUIC to the application
(peerInitiatedUni == received + 1 across every cliffed scenario,
pendingBytes always 0, cap headroom unused). The relay's
per-subscriber forward pipeline runs at ≈ 40 streams/sec sustained;
at 1 frame per group = 50 streams/sec we exceed it by ~9 streams/sec
and the relay's per-subscriber buffer overflows after 6–14 seconds
of continuous push, after which it stops opening new streams to that
subscriber entirely. Reproduced on:
- sweep_30s 1500 frames -> 610-737 received (variance run-to-run)
- sweep_120s 6000 frames -> 61-419 received
- sweep_frames400 400 frames -> 36-400 received (huge variance)
- sweep_3subs sub[1] 100 frames -> 94 received (others 100/100)
Packing 5 frames per moq-lite group cuts stream creation to 10/sec,
well under the relay's ceiling. Sweep-evidence: framesPerGroup=5
delivered 100/100 across every scenario including the long ones.
Late-join initial gap goes from ≤ 20 ms to ≤ 100 ms — imperceptible
for live audio rooms.
Updates the plan doc to mark status PRODUCTION-FIXED with the
two-layer fix (MAX_STREAMS_UNI extensions + framesPerGroup=5) and
flags the upstream-issue follow-up at kixelated/moq.
All :quic + :nestsClient JVM tests pass.
The previous round of fc-* lines exclusively reported the speaker's QUIC
connection state. That hid the real story for the residual stream loss:
each Opus frame the relay forwards arrives as a fresh peer-initiated
uni stream on the LISTENER side, so the listener's peerInitiatedUni
counter and advertisedMaxStreamsUni evolution are what matter — the
speaker's view is stable at peerInitiatedUni=1 for the whole run
(only the WT control stream is relay-initiated on that side).
Changes:
- MoqLiteSession.transport changed from `private` to `internal val`
so test code can downcast.
- MoqLiteNestsListener exposes an `internal val transport` getter
that returns the underlying WebTransportSession via the session's
new internal accessor.
- SendTraceScenario.run takes a new optional
`listenerFlowControlSnapshots: List<suspend () -> QuicFlowControlSnapshot>`.
When supplied, it emits `fc-listener[idx]-pre / -post-pump /
-post-grace` lines in the same shape as the speaker side.
- Refactored the three checkpoint emit sites into a shared
`logFcSnapshot(scope, label, snap, includeRcvBuf)` helper so the
speaker and listener lines print identical column layouts.
- withProdSpeakerAndListeners + withHarnessSpeakerAndListeners cast
each listener to MoqLiteNestsListener (the only impl in the tree)
and downcast its WebTransportSession to QuicWebTransportSession,
yielding a per-listener snapshot supplier list to the scenario
block.
What the next sweep tells us: for each cliffed scenario, fc-listener[0]
should now show whether the listener is the bottleneck (e.g.
peerInitiatedUni stuck below the relay's intent, or udpDatagrams not
matching the relay's send count) or whether the relay just stopped
forwarding (peerInitiatedUni climbs but matches the partial received
count).
All :quic and :nestsClient JVM tests pass.
Three targeted changes against the residual sustained-load stream loss
that survived the MAX_STREAMS_UNI fix (long broadcasts and bursty
scenarios still cliffed mid-pump). Each addresses one of the round-1
hypotheses; the next prod sweep should tell us how much each
contributes.
quic/UdpSocket:
- Bump SO_RCVBUF to 4 MiB at bind time. The kernel default (~200 KB
on Linux/macOS, similar on Android) holds barely 130 MTU-sized
datagrams; a multi-second moq-lite broadcast that the relay fans
out to several subscribers transiently overflows this. Anything
queued past `rmem` is silently dropped by the kernel and never
reaches our QUIC stack.
- Add lifetime diagnostic counters (receivedDatagramCount,
receivedByteCount, receiveBufferSizeBytes) on the expect surface
so commonMain code can read them via the new udpStatsSupplier
hook on QuicConnection without each platform having to surface its
own native bookkeeping.
- QuicConnectionDriver wires the supplier on start; the connection's
flowControlSnapshot bundles the stats into a new
QuicFlowControlSnapshot.udp field.
QuicConnection.flowControlSnapshot:
- Now also surfaces advertisedMaxStreamsUni / advertisedMaxStreamsBidi
and peerInitiatedUniCount / peerInitiatedBidiCount so a sweep can
see the listener-side stream-cap evolution alongside speaker-side
counters that were already there.
SendTraceScenario:
- verbosePerFrame default flipped from true to false. The per-frame
`tx i=…` and `rx[idx] gid=…` logs go through InteropDebug ->
JUnit's stdout capture; at 50 frames/sec the capture thread
serialises the receive coroutine and starves the QUIC read loop,
biasing the recorded received count downward on long runs. Tests
that need the per-frame timeline opt in explicitly.
- Replace the receive-side CopyOnWriteArrayList sink (O(N) per add,
~1.1M element copies cumulative for a 1500-frame run) with
Collections.synchronizedList(ArrayList(capacityHint)) — O(1)
amortised. Snapshot under the same lock at end of run so the
JDK's iterator-locking contract is satisfied.
- fc-pre / fc-post-pump / fc-post-grace lines now include
`udpDatagrams=… udpBytes=… udpRcvBuf=…` plus
`advertisedMaxStreamsUni=… peerInitiatedUni=…` so the next sweep's
diagnostic dump shows directly whether the kernel actually
delivered datagrams the relay sent.
Plan doc: nestsClient/plans/2026-05-01-quic-stream-cliff-investigation.md
documents round-2 changes and lists the remaining open hypotheses
(relay-side per-subscriber queue policy, receive-side
MAX_STREAM_DATA threshold).
All :quic + :nestsClient JVM tests pass.
Production sweep on commit c0269859 confirms the MAX_STREAMS_UNI fix landed:
- sweep_frames_200: received=200/200 missing=[] ✅
- sweep_baseline 100×20ms: received=100/100 ✅
- sweep_2subs (both subscribers): 100/100 each ✅
- sweep_cad{10,40,80,200}: all 100/100 ✅
- sweep_payload_{1kb,4kb}: 100/100 ✅
- sweep_slowconsumer: 100/100 ✅
14 of 27 scenarios now deliver 100% of frames. The original "audio cuts
out at frame ~99 in two-user rooms" bug — the practical user-facing
issue — is fixed.
Residual loss in extreme scenarios (long broadcasts, very large payloads,
mid-stream pauses, fast bursts, multi-subscriber asymmetry) is documented
in the plan doc with three hypotheses to test next:
1. Listener QUIC RX coalescing / UDP socket recv overhead
2. Receive-side MAX_STREAM_DATA extension threshold
3. moq-rs relay's per-subscriber buffer policy
Next step is separate investigation; the speaker side is now
demonstrably pristine (pendingBytes=0, consumed grows monotonically,
nextLocalUniIdx matches the frame count) so further work is on the
listener-RX or relay-forward side.
Production sweep_frames_200 confirms received=200/200 missing=[] after the
:quic MAX_STREAMS_UNI extension landed in d391ae1d. The framesPerGroup=5
mitigation in NestMoqLiteBroadcaster (commit 10ad69f1) is no longer
needed and was a workaround for a real QUIC bug; reverting to 1 brings
the wire shape in line with the JS reference broadcaster (one Opus frame
per moq-lite group → late-join initial gap drops from ≤100ms back to
≤20ms).
Tests that still want one-frame-per-group semantics keep their explicit
framesPerGroup=1 overrides — they pin the assumption independent of any
future default change. The broadcaster's framesPerGroup field kdoc is
updated to explain the history (mitigation → root-caused fix → revert).
All :quic and :nestsClient JVM tests pass.
Plan doc: nestsClient/plans/2026-05-01-quic-stream-cliff-investigation.md
status flipped to CLOSED with the production-validation numbers.
flowControlSnapshot dump from sweep_frames_200 against nostrnests.com proved
the speaker side was perfectly healthy (~5 EB conn-credit unused, 10 000-stream
peer cap unused, 0 bytes stuck in send buffers, all 200 streams opened) yet
the listener cliffed at frame 99. The constraint was on the listener's
receive side: our :quic was advertising initial_max_streams_uni = 100 at
handshake and never extending it, so the relay could only open 100 uni
streams to us *for the lifetime of the connection*. Each Opus frame the
relay forwards is a fresh peer-initiated uni stream, so any broadcast
longer than ~100 frames silently truncated at the audience.
QuicConnection:
- peerInitiatedUniCount / peerInitiatedBidiCount counters (incremented
in getOrCreatePeerStreamLocked).
- advertisedMaxStreamsUni / advertisedMaxStreamsBidi tracking, starting
at config.initialMaxStreams* and raised by the writer.
QuicConnectionWriter.appendFlowControlUpdates:
- When peerInitiatedUniCount + cfg.initialMaxStreamsUni / 2 >=
advertisedMaxStreamsUni, emit MaxStreamsFrame(bidi=false, newCap)
where newCap = peerInitiatedUniCount + cfg.initialMaxStreamsUni.
Same pattern the existing MaxDataFrame / MaxStreamDataFrame
extension uses; same half-window threshold so we don't spam the
peer.
- Symmetric branch for bidi.
PeerStreamCreditExtensionTest:
- Writer DOES emit MAX_STREAMS_UNI when peer's lifetime uni-stream count
crosses the half-window threshold; advertisedMaxStreamsUni updates.
- Writer DOES NOT emit MAX_STREAMS_UNI below the threshold;
advertisedMaxStreamsUni stays at the initial cap.
Updated nestsClient/plans/2026-05-01-quic-stream-cliff-investigation.md
with the fc-snapshot dump that proved the cause and the fix that
addresses it. The framesPerGroup=5 mitigation in NestMoqLiteBroadcaster
can be reverted to 1 once the prod sweep confirms the cliff is gone.
All :quic and :nestsClient JVM tests pass.
Adds a read-only diagnostic surface that lets a test (or any caller)
read the peer's transport parameters, the live connection-level send
credit / consumed counters, the current peer-granted MAX_STREAMS_*
values, and the total bytes sitting in stream send buffers but not
yet handed to STREAM frames.
Goal: pin which budget runs out at the production "stream cliff"
described in nestsClient/plans/2026-05-01-quic-stream-cliff-investigation.md.
The plan flagged three candidates — connection-level MAX_DATA, per-
stream MAX_STREAM_DATA, or the relay's MAX_STREAMS_UNI extension
policy. The snapshot makes it possible to attribute the stall by
reading the diff between the pre-pump, post-pump, and post-grace
snapshots from the test's stdout.
QuicConnection:
- flowControlSnapshot(): suspend, lock-protected, returns
QuicFlowControlSnapshot (new data class) — peer TPs + live
accounting + sum of enqueued-not-sent bytes across streams.
QuicWebTransportSession (jvmAndroid adapter):
- quicFlowControlSnapshot() passthrough so the test can downcast
its WebTransportSession and read the underlying connection's
state without poking through the common transport interface.
SendTraceScenario:
- Optional flowControlSnapshot lambda parameter; when supplied,
logs three checkpoints — fc-pre, fc-post-pump, fc-post-grace —
each on a single line with the full snapshot.
NostrnestsProdAudioTransmissionTest + NostrNestsSustainedSendOutcomesInteropTest:
- withProdSpeakerAndListeners / withHarnessSpeakerAndListeners now
yield a snapshot lambda to the scenario block. Every sweep test
automatically dumps fc-* lines to the JUnit XML system-out.
FlowControlSnapshotTest:
- Pre-handshake: peer TP fields are null, counters zero.
- Post-handshake: every TP field reflects what the in-process TLS
server advertised; sendConnectionFlowCredit equals
initial_max_data; consumed = 0.
- Post-allocate-and-enqueue: nextLocalUni/BidiIndex advance,
totalEnqueuedNotSentBytes sums the buffered chunks, and
streamsWithPendingBytes counts only streams with > 0 pending.
Reading the production sweep output after this commit:
- fc-pre dumps what the relay grants on handshake (initial_max_data,
initial_max_stream_data_uni, initial_max_streams_uni).
- fc-post-pump shows whether sendConnectionFlowConsumed has
plateaued at the cap (peer didn't extend MAX_DATA) or whether
bytes are stuck in stream send buffers.
- The diff between fc-post-pump and fc-post-grace tells us
whether the relay's eventual MAX_DATA / MAX_STREAMS update did
or didn't arrive during the 30-60 s grace window.
Production sweep against nostrnests.com cliffed at exactly 99 frames received
across every multi-frame scenario (frames200/400, 30s, 120s, etc.) when the
broadcaster opened one QUIC uni stream per Opus frame. The investigation in
nestsClient/plans/2026-05-01-quic-stream-cliff-investigation.md walks through
why the obvious "raise the stream cap" hypothesis didn't pan out (publisher.send
was never returning false past the cap, the local QuicConnection was already
seeing the relay extend MAX_STREAMS_UNI dynamically).
The same sweep showed sweep_frames_per_group_5 / _20 / _all delivering 100/100
frames cleanly. The cliff scales with stream-creation rate, not byte volume or
total stream count — so packing N frames per group (one uni stream per N frames
instead of per frame) bypasses whatever the underlying limit is.
NestMoqLiteBroadcaster:
- Add framesPerGroup constructor param, default 5 (≈ 100 ms of audio per
moq-lite group). Stream-creation rate drops from 50/sec to 10/sec, well
below the production cliff threshold.
- Counter resets after each endGroup; trailing partial group flushes its
FIN on capture EOF so the listener doesn't sit on a half-open stream.
MoqLiteNestsSpeaker + connectNestsSpeaker:
- Plumb framesPerGroup through to the broadcaster. Default flows from
NestMoqLiteBroadcaster.DEFAULT_FRAMES_PER_GROUP so production callers
get the mitigation without code changes; tests can override to 1 to
keep their groupId-per-frame assertions.
Tests:
- All harness interop tests (RoundTrip, LateJoin, MultiPeer, Mute,
Unsubscribe, SpeakerClose, both Reconnecting variants) plus the prod
NostrnestsProdAudioTransmissionTest now pass framesPerGroup = 1
explicitly to preserve their per-frame group/object id assertions.
- All :quic and :nestsClient JVM tests pass.
Trade-off: a brand-new subscriber that joins mid-broadcast picks up at the
next group boundary. With 5 frames/group the late-join initial gap is up
to 100 ms (perceptually inaudible for live audio). The previous wire shape
gave at most 20 ms.
Investigation plan for the underlying QUIC cliff lives in
nestsClient/plans/2026-05-01-quic-stream-cliff-investigation.md. The
attempted-then-reverted suspend/STREAMS_BLOCKED commit is summarised
under "Attempt 1" in that doc.
The previous run pinned the production failure to "82/100 frames at 20 ms cadence
two-users, all 18 trailing groups lost; publisher.send returned true for every
frame". To narrow the cause the next run needs to vary one dimension at a time
and dump rich per-frame data, so we can attribute the loss to:
- QUIC stream-credit (RTT-bound) vs absolute count cap
- Per-stream-creation cost vs per-byte cost
- Listener-side buffer overflow vs relay-side drop
- Steady-state vs initial-burst behaviour
- Shared end-of-stream FIN flush race vs prod-only loss
SendTraceScenario.kt: shared per-frame instrumented pump driver. Records
publisher.send Boolean per frame, send latency, endGroup exceptions,
arrivals per subscriber with wall-clock timing, missing-objectId run
lengths, send-latency p50/p99/max + count of >1ms sends. Verbose logging
(InteropDebug.checkpoint) emits one line per tx and one per rx so the
JUnit XML system-out doubles as a timing trace.
Sweep methods (mirrored prod-mode and harness-mode):
- baseline 100×20ms (reproduce known cliff)
- cadence sweep: 5 / 10 / 40 / 80 / 200 ms (RTT-dependent?)
- burst (no cadence) 100 frames (max inflight)
- frame-count sweep: 50 / 200 / 400
- payload sweep: 1 KB / 4 KB / 16 KB
- frames-per-group: 5 / 20 / 100 (uni-stream vs bytes)
- late subscribe at frame 25, frame 50 (from-latest semantics)
- mid-pump 5 s pause after frame 50
- 2-, 3-subscriber fan-out
- 50 ms slow-consumer listener (per-subscription buffer overflow)
- long-run 30 s and 120 s (steady-state)
Each test method is a one-line Scenario literal; setup helpers
withProdSpeakerAndListeners / withHarnessSpeakerAndListeners build
publisher + N listeners, run the scenario, and tear down in order.
Smoke-tested against the local kixelated/moq reference relay: every
scenario except framesPerGroup=100 reproduces the same trailing-frame
artifact (received=99/100, missing=[99]) the original test surfaced.
framesPerGroup=100 receives every frame, confirming the local issue is
specifically per-uni-stream FIN flushing — independent of the production
82/100 cliff.
Mirror NostrnestsProdAudioTransmissionTest.sustained_per_frame_send_outcomes_two_users
but drive the local Docker-backed kixelated/moq reference relay via
NostrNestsHarness. Lets us reproduce the production "82/100" cliff
without nostrnests.com so we can attribute the loss to client code
(`:nestsClient` / `:quic`) vs. the production deployment.
Same per-frame instrumentation: bypass NestMoqLiteBroadcaster, call
publisher.send() / publisher.endGroup() directly, record the boolean
return per frame plus any endGroup exception, dump a per-frame
send/recv table on failure.
Test 4 showed a hard 82/100 cliff at 20 ms cadence two-users, all 18
trailing groups lost permanently. Root-cause-narrowing requires knowing
whether MoqLitePublisherHandle.send returned false (frame dropped at the
moq-lite layer due to inboundSubs.isEmpty() / publisherClosed) or true
(frame queued by moq-lite and lost downstream — uni-stream write error
swallowed by runCatching, QUIC flow control wedged, or relay drop).
The production NestMoqLiteBroadcaster wraps everything in
runCatching {…}.onFailure {…} and ignores the boolean, so frame loss is
structurally invisible to the app. This new test bypasses the broadcaster
and calls publisher.send / publisher.endGroup directly while mirroring
the auth + transport + session setup that connectNestsSpeaker performs.
Records sendOutcomes[i] and endGroupErrors[i] per frame; on failure
prints a per-frame send/recv table so we can see exactly when send
flips false (or whether all 100 sends report true and the loss is
strictly downstream of moq-lite).
Previously test 4 used .take(N).toList() inside withTimeoutOrNull, so a
timeout discarded every received frame and the only failure signal was
"only got partial audio". When the production run flagged frame loss,
we couldn't tell whether the stream stalled mid-flight, never started
fanning out, or dropped scattered groups.
Drain into an external CopyOnWriteArrayList via .onEach { } so the
partial contents survive the timeout, then on failure surface:
- received N/100 with first/last arrival timestamps
- missingGroups summary as run-length ranges (e.g. [3-7,42,90-99])
- first/last 20 arrivals so the front and tail of the sequence are
visible without flooding stdout
The gap pattern alone narrows the diagnosis: front-loaded losses point
to subscribe-settle being too short, tail-loaded losses point to a
broadcaster shutdown race, scattered drops point to per-subscription
buffer overflow / datagram loss.
Four progressively-narrower JVM tests that drive the production
connectNestsSpeaker / connectNestsListener / OkHttpNestsClient /
QuicWebTransportFactory / NestMoqLiteBroadcaster code paths against
the real moq.nostrnests.com + moq-auth.nostrnests.com infrastructure
(no Docker harness, no Nostr events) so we can isolate why audio is
not flowing between two real users:
1. auth_minting_works_for_publish_and_listen_paths
2. same_user_speaker_and_listener_round_trip
3. two_users_speaker_publishes_listener_subscribes
4. sustained_real_time_cadence_two_users (~2s @ 20ms cadence)
Skipped by default; opt in with -DnestsProd=true and (optionally)
override URLs via -DnestsProdEndpoint=... / -DnestsProdAuth=...
ScriptedSpeaker mutated `handles` AFTER incrementing `_startCount`,
so a reader that polled startCount on a different thread (the test
runs on the runBlocking thread while the broadcast pump runs on
Dispatchers.Default) could see startCount==1 via the volatile
AtomicInteger read before the subsequent list write became visible,
then fail `assertEquals(1, first.handles.size)` with a stale 0.
Also `mutableListOf` itself is not thread-safe — concurrent reads
during a write are undefined.
Reorder so the list append happens BEFORE the increment (the
volatile write now publishes the list mutation under the same
happens-before edge tests already rely on for startCount), and
swap the backing store for CopyOnWriteArrayList so concurrent
reads of `handles[0]` are well-defined.
Observed as a flake on Ubuntu CI; passes locally.
The publisher's inbound-bidi handler wrote SubscribeOk to the bidi
before calling registerInboundSubscription. The peer's first
publisher.send() after observing Ok could race the registration on
dispatchers that resume the peer's continuation before the handler's
(notably Windows under Dispatchers.Default), causing send to observe
an empty inboundSubs and return false. Reordering makes the peer's
view of Ok a happens-after of the registration.
Fixes the Windows-only failure in
MoqLiteSessionTest.publisher_acks_subscribe_and_pushes_group_data_on_uni_stream.
The deployed nostrnests reference (NestsUI v2 + moq-auth + moq-rs)
emits a different on-the-wire schema than the previous EGG-01 / EGG-09
drafts and Quartz writers. Verified by reading the production
NestsUI bundle. The deployed schema is now canonical:
kind:30312 (room event)
- relay URL → ["streaming", url] (was ["endpoint", url])
- auth URL → ["auth", url] (was ["service", url])
- live status → ["status", "live"] (was "open")
- ended status → ["status", "ended"] (was "closed")
- room name → ["title", name] (was ["room", name])
kind:10112 (user MoQ-server list)
- 3-element entries: ["server", relay, auth]
- first-element name "relay" accepted as a synonym (legacy)
- 2-element entries: derive auth host by replacing leading "moq."
with "moq-auth.", or prepending "moq-auth." otherwise
Implementation
- ServiceUrlTag.TAG_NAME = "auth", LEGACY_TAG_NAME = "service"
- EndpointUrlTag.TAG_NAME = "streaming", LEGACY_TAG_NAME = "endpoint"
- StatusTag enum: PLANNED/LIVE/PRIVATE/ENDED with code "live"/"ended";
legacy "open"/"closed" still parsed on read
- NestsServersEvent: emit/read 3-element [server, relay, auth] tags
with the legacy-shape tolerances above; expose a NestsServer pair
- Account.nestsServers.flow now produces List<NestsServer> pairs
- NestsServersScreen: rewritten edit-field asks for both URLs,
recommended row shows the nostrnests pair, list rows show both URLs
- NestsScreen first-time setup writes the nostrnests pair, not a
single URL; gate the create FAB on both URLs being parseable
- CreateNestViewModel: drop the resolveServerPair hardcoded mapping —
the saved pair is now authoritative; defaults stay correct for an
empty kind-10112 list
Specs
- EGG-01 rewritten to canonical streaming/auth/live/ended with a
legacy-spelling table; example uses the real nostrnests pair
- EGG-02 references "auth" tag throughout; error taxonomy says "ended"
- EGG-09 rewritten to 3-element server tag with derivation fallback
- New nestsClient/specs/nip-53-proposed.md — a single self-contained
proposed update to upstream NIP-53 covering kind 30312 + kind 10112
with the deployed schema and the JWT-mint flow
All existing unit tests adjusted; quartz:jvmTest, amethyst:testPlayDebugUnitTest,
and nestsClient:jvmTest pass.
`subscribeSpeaker_survives_session_swap` and
`unsubscribe_before_session_swap_releases_handle` had two race
conditions exposed only on macOS scheduling:
1. ScriptedListener.subscribeCount is incremented inside
opener(listener) — BEFORE the wrapper's pump reaches
`handle.objects.collect { frames.emit(it) }`. Waiting on
subscribeCount alone doesn't prove the pump is collecting
from first.frames. An emit upstream before the pump's collect
registers is silently dropped.
2. The wrapper's outer `frames` SharedFlow is replay=0. The
`scope.async { take(2).toList() }` consumer might not have
subscribed by the time the test thread races into the first
emit, dropping the value.
Both are fixed by waiting for actual subscription registration:
- first/second.frames.subscriptionCount.first { it > 0 } — flips
to 1 only after the pump's collect lands, which is strictly
after liveHandleRef.set(handle).
- onSubscription + CompletableDeferred — fires after the
consumer's collector is registered against the SharedFlow.
Lets a clustered burst of mint calls (typical interop test scenario,
also any moq-auth deployment with stricter rate limits) outlast the
60 s rate-limit window instead of cascading into a hard failure.
- Retry up to 7 times on HTTP 429. Worst-case total wait at 1 s
initial + 16 s cap is 1+2+4+8+16+16+16 = 63 s — just past the
reference moq-auth 60 s/IP window.
- Respect the `Retry-After` header (delta-seconds OR HTTP-date)
when present; fall back to capped exponential backoff otherwise.
- Re-sign the NIP-98 auth event on every attempt. The reference
validator accepts a 60 s validity window; without re-signing,
any retry that waits past that window fails 401 "Event too old".
- Suspending `delay` so coroutine cancellation tears the loop down
cleanly.
`./gradlew :nestsClient:jvmTest -DnestsInterop=true -DnestsInteropExternal=true`
now goes green in a single invocation: 157 tests across 33 classes,
0 failures. Previously the same command cascaded 11 failures once
the moq-auth 20/min/IP bucket filled.
Unit tests: 7 new cases covering Retry-After parsing (seconds,
HTTP-date, garbage, missing), exponential cap, the 429-then-200
retry loop, max-retries exhaustion, and that non-429 4xx is NOT
retried. Backed by a tiny ServerSocket-based HTTP fake so no new
test deps.
Captures the group-rotation, orchestrator-break-on-Closed removal,
ensureAnnounceWatchStarted, handleInboundBidi refactor, and
removeInboundSubscription changes that landed in d8ab4fd9. All
three reconnecting-listener interop scenarios now pass.
To make a SubscribeHandle survive a publisher session swap on the same
relay, three things had to change together:
1. Broadcaster emits one Opus frame per moq-lite group
(`publisher.send` + `publisher.endGroup`). moq-lite's "from-latest"
subscribe semantics deliver a new subscriber the NEXT group's
frames; without per-frame rotation a subscriber that attaches
mid-broadcast waits forever for the (single, never-ending) group
to end.
2. MoqLiteSession opens its announce-watch bidi synchronously before
the first subscribe, dispatches subscribe + announce frames over a
single long-running collector (varint type code hoisted outside
`collect`), and FINs the publisher's currentGroup when an inbound
subscribe bidi closes — so the next send opens a fresh group keyed
off the live subscriber instead of the recycled one.
3. ReconnectingNestsListener no longer breaks on terminal=Closed in
the orchestrator; the user-driven stop path goes through
`orchestrator.cancel()`, so any other Closed (peer-driven
transport close, half-broken session, publisher recycle) is a
reconnect trigger.
Round-trip interop test updated to assert `groupId == idx` (one group
per frame) rather than `groupId == 0`. All three reconnecting-listener
interop scenarios now pass against the real moq-rs relay:
happy-path, session-swap, and listener-survives-publisher-recycle.
The takeWhile/collect pattern only re-checks its predicate when the
NEXT upstream value emits, so once nextSubscribeBidi found its
target Subscribe bidi, the helper sat blocked waiting for a
follow-up bidi that may never come — turning groups_are_demuxed_by_
subscribeId into a 5-minute hang on tests that open exactly two
subscribes (the announce-watch bidi happened to land between, but
relying on it to nudge the flow forward is fragile).
Switched to transformWhile { … emit(…); false }.firstOrNull() so
the upstream collection terminates synchronously after the first
match. Warm-daemon test time drops from multi-minute timeout back
to the expected ~10 ms range.
https://claude.ai/code/session_01HXf3zG3F2ev2ASeQju7Y5S
The session-layer fix in 851045c6 lazy-launches a single shared
announce-watch bidi on first subscribe (publisher-disconnect
detection). The unit test groups_are_demuxed_by_subscribeId
assumed each session.subscribe() opened exactly one peer-side bidi
and grabbed them by raw `peerOpenedBidiStreams().first()`, which
races with the announce-watch bidi.
New helper nextSubscribeBidi(serverSide) iterates accepted bidis,
peeks the control-byte varint, and skips any that aren't Subscribe.
The race window (announce-watch bidi between subscribe #1 and
subscribe #2) is now handled gracefully — the test reliably
finds both subscribe bidis regardless of the announce-watch's
launch ordering.
Also extends the listener-survives-publisher-recycle plan doc
with the full diagnosis of why
reconnecting_wrapper_keeps_handle_alive_across_session_swap
remains a separate, narrower failure (publisher single-group
architecture: NestMoqLiteBroadcaster never rotates groups, so
mid-stream listener resubscribes have nothing to attach to).
https://claude.ai/code/session_01HXf3zG3F2ev2ASeQju7Y5S
Two layered fixes for the gap captured in
nestsClient/plans/2026-04-28-listener-survives-publisher-recycle.md.
Session layer (MoqLiteSession.kt):
- Lazy single shared announce-watch pump per session, opened on
first subscribe. moq-lite Lite-03 has no explicit "publisher
gone" message on the subscribe bidi (the relay keeps that
bidi open across publisher cycles in case a fresh publisher
takes over the suffix), so the announce stream's Ended event
is the only reliable signal.
- On Announce(Ended) for a broadcast suffix, close the
matching ListenerSubscription's frames Channel and remove it
from the map. The wrapper-level frames.consumeAsFlow() flow
ends naturally — same shape as a user-driven
handle.unsubscribe() — so the wrapper pump's collect-
completion path drives the re-issue.
Wrapper layer (ReconnectingNestsListener.kt):
- Inner re-subscribe `while (currentCoroutineContext().isActive)`
loop in reissuingSubscribe. When the underlying frames flow
completes (publisher cycled, signalled by the session layer
above), re-issue subscribe against the same listener with a
100 ms backoff. moq-lite supports subscribe-before-announce so
a re-subscribe issued during the gap attaches cleanly when
the next publisher comes up under the same suffix.
Verified against the real moq-rs relay (host build, external
mode): the new
NostrNestsReconnectingListenerInteropTest.subscribe_handle_survives_publisher_recycle
test passes — single SubscribeHandle keeps emitting frames across
multiple speaker JWT-refresh cycles. Speaker reconnect tests
still pass too.
In production, this closes the audio dropout that would have
fired every 9 minutes per speaker JWT refresh on long Nest calls
(see the plan doc for the original diagnosis).
https://claude.ai/code/session_01HXf3zG3F2ev2ASeQju7Y5S
Discovered while validating connectReconnectingNestsSpeaker against
the real moq-rs relay: when a publisher cycles its session (e.g. via
JWT refresh), an existing listener-side SubscribeHandle does not
auto-reattach to the new publisher under the same broadcast suffix.
Frames stop until the listener's own JWT refresh fires.
Root cause is multi-layer:
- moq-lite session: SubscribeHandle.frames is a
Channel.consumeAsFlow() that the session never closes on remote
disconnect.
- moq-lite Lite-03: graceful close emits Announce(Ended), which
the wrapper would need to observe to react.
- ReconnectingNestsListener: reissuingSubscribe only re-issues on
listener-session swaps, not on announce-Ended.
A wrapper-layer announce-driven re-subscribe was attempted (race
collect vs announce-Ended.first(), cancel loser, unsubscribe, retry).
It compiled and the unit tests passed, but interop against the real
relay still showed listener silence after the recycle — the listener's
QUIC session terminated 4 ms after the publisher's "subscribe
cancelled" log line, suggesting the announce-bidi cleanup or
sub-unsubscribe path is being interpreted as session close at the
moq-lite layer. Did not pin down the exact chain.
Reverted the wrapper change to keep the branch clean. Speaker-side
connectReconnectingNestsSpeaker is unaffected and continues to pass
its interop tests. Plan doc captures the diagnosis and three
candidate fix paths for follow-up — preferred direction is
session-layer (have MoqLiteSession close the frames Channel when the
relay forwards Announce(Ended) or FINs the subscribe bidi) so the
existing wrapper pump's natural collect-completion handles it.
https://claude.ai/code/session_01HXf3zG3F2ev2ASeQju7Y5S
The first cut of the JWT-refresh interop test held a single listener-
side SubscribeHandle open across the speaker's session recycle and
asserted both pre- and post-recycle frames arrived on it. That fails
because moq-lite's relay doesn't auto-route a vanilla SubscribeHandle
to a fresh publisher session under the same suffix — the
listener-survival-across-publisher-recycle is a separate concern,
NOT a speaker-reconnect bug. Verified against a real moq-rs relay
(host-built, external mode, --auth-key-dir + per-kid JWK file).
Reworked to validate just what the speaker reconnect should
guarantee:
- Phase 1: pre-refresh frames round-trip on the first session.
- Phase 2: orchestrator recycles (openCount → 2+, wrapper state
reaches Broadcasting on the new session).
- Phase 3: post-refresh frames round-trip on a FRESH listener
subscription against the new session.
- Wrapper invariant: outward state never surfaced
Reconnecting / Failed during the recycle.
Both speaker interop tests pass against the real relay.
The "listener subscription survives publisher recycle" gap is a
real production concern for >9-min stage time but lives outside
this PR — it would need either listener-side announce-watching or
a coupled refresh-on-publisher-cycle hook in the listener wrapper.
https://claude.ai/code/session_01HXf3zG3F2ev2ASeQju7Y5S
After verifying the nostrnests reference (NestsUI-v2 @ main):
- ProfileCard.tsx writes kicks as ['action','kick'] tags with empty content
- useAdminCommands.ts reads action via tags.find(t => t==='action') and
applies a 60-s relay since plus a processedRef Set to dedup re-deliveries
- p-tag role marker for moderators is 'admin', not 'moderator'
Amethyst was diverging on every one of those, which means our outbound
admin commands were invisible to nostrnests, theirs to us, and any
nostrnests admin (role='admin') failed our isModerator() / canSpeak()
gates entirely — kicks and force-mutes signed by them were silently
dropped.
Changes:
quartz/AdminCommandEvent.kt
- Emit ['action', '<verb>'] tag with empty content
- Reader prefers the tag, falls back to content for any in-flight
Amethyst-built kick from before this commit
- kick() and forceMute() share a common build() helper
quartz/ParticipantTag.kt
- ROLE.MODERATOR.code = 'admin' (matches nostrnests + EGG-07)
- Adds legacyCodes = ['moderator'] so older Amethyst-emitted
kind-30312 events still parse as MODERATOR
- effectiveRole() walks both code + legacyCodes
amethyst/AdminCommandsCollector
- Filter carries since = now - 60 (EGG-07 #7)
- Defensive per-event freshness re-check for cached events / clock skew
- mutableSetOf<String>() processed-id dedup for the lifetime of the
collector, mirroring useAdminCommands.ts's processedRef
EGG-07.md
- Documents the Amethyst-only ['action','mute'] extension under a new
'Implemented extensions' section. nostrnests doesn't emit or honour
it today; cross-client force-mutes only work between Amethyst peers
- 'warn' stays in the future-actions list — nostrnests has no plans
for it either
Tests:
- ParticipantTagTest: new asserts that 'admin' / 'Admin' / 'ADMIN'
parse as ROLE.MODERATOR; pins the wire string to 'admin' and the
legacy alias to 'moderator'
- AdminCommandEventTest: kick/forceMute templates carry ['action', _]
tag with empty content; legacy content-form still parses; tag wins
over content when both are present
Claude